High classification accuracy does not imply effective genetic search

Learning classifier systems, their parameterisation, and their rule discovery systems have often been evaluated by measuring classification accuracy on small Boolean functions. We demonstrate that by restricting the rule set to the initial random population (that is, a random discretisation of the input space), high classification accuracy can still be achieved, and that on relatively small functions this requires few rules. We argue that this demonstrates that high classification accuracy on small functions is not evidence of effective rule discovery. However, we argue that small functions can nonetheless be used to evaluate rule discovery when a certain more powerful type of metric is used.