Algorithm Evaluation by Probabilistic Fitness/Cost Analysis and Application to Image Segmentation

In many areas of computer vision the research producing new algorithms greatly exceeds work on their evaluation. Evaluation of vision algorithms is often difficult because of the multiple objectives that an algorithm should meet, for example accuracy and computational efficiency, and because algorithms typically have several parameters which must be specified by the user. In this paper we propose a framework for evaluation of algorithms with multiple objectives, which allows probabilistic analysis of the behavior of a set of algorithms in a joint fitness/cost space. We take the image segmentation problem as an example application domain and use our approach to compare seven state-of-the-art image segmentation algorithms.