Measuring performance of optimization algorithms in evolutionary computation

Scientific paper - Original scientific paper

• Author: Nikola Ivković
• Author: Domagoj Jakobović
• Author: Marin Golub
• Author's institution: University of Zagreb; Faculty of Organization and Informatics
• Scientific / art field, discipline and subdiscipline: SOCIAL SCIENCES; Information and Communication Sciences

Abstract (english)

Reporting the results of optimization algorithms in evolutionary computation is a challenging task with many potential pitfalls. The source of problems is their stochastic nature and inability to guarantee an optimal solution in polynomial time. One of the basic questions that is often not addressed concerns the method of summarizing the entire distribution of solutions into a single value. Although the mean value is used by default for that purpose, the best solution obtained is also occasionally used in addition to or instead of it. Based on our analysis of different possibilities for measuring the performance of stochastic optimization algorithms presented in this paper we propose quantiles as the standard measure of performance. Quantiles can be naturally interpreted for the designated purpose. Besides, they are defined even when the arithmetic mean is not, and are applicable in cases of multiple executions of an algorithm. Our study also showed that, on the contrary to many other fields, in the case of stochastic optimization algorithms the greater variability in measured data can be considered as an advantage.

• Keywords (english): algorithmic performance; experimental evaluation; metaheuristics; quantile
• Language: english
• Publication type: Scientific paper - Original scientific paper
• Publication status: Published
• Peer review: Peer review - international
• Publication version: Published version
• Journal title: International Journal of Machine Learning and Computing
• Numbering: vol. 6, no. 3, pp. 167-171
• p-ISSN: 2010-3700
• DOI: https://doi.org/10.18178/ijmlc.2016.6.3.593
• URN:NBN: urn:nbn:hr:211:418448
• Publication: 2016
• Document URL: http://www.ijmlc.org/vol6/593-A27.pdf
• Type of resource: Text
• Access conditions: Open access
• Created on: 2017-10-04 07:02:51