This site has the purpose of providing the people interested in Good Experimental Methodology and Benchmarking in robotics a web hub. They are researchers and practitioners both from academia and industry with an interest in experimental robotics.
The Euron GEM Sig is also aimed at benchmarking and objectively evaluating performance of robots. Accordingly, we believe it is interesting for anyone who has an interest in quantitative performance evaluation of robots and/or robot algorithms. Some controversial issues need to be discussed such as: measuring autonomy or information metrics of intelligent systems, or the concept itself of replicability or benchmarking of research results in robotics.
If you have any suggestion or wish to volunteer ...pls contact
This e-mail address is being protected from spambots. You need JavaScript enabled to view it. or This e-mail address is being protected from spambots. You need JavaScript enabled to view it. or This e-mail address is being protected from spambots. You need JavaScript enabled to view it.
As the complexity of current robotic and embodied intelligent systems grows, it is more and more necessary to define proper experimental approaches and benchmarking procedures. On the one hand, reliable benchmarks are called for in order to allow the comparison of the many research results in robotics research, so that their industrial application is eventually possible. On the other hand, if robotics aims to be regarded as serious science, replication of experiments deserves consciencious attention; it is necessary to be able to verify if and by which measure new procedures and algorithms proposed in research papers constitute a real advancement and can be used in new applications. New more successful implementations of concepts already presented in the literature, but not implemented with exhaustive experimental methodology, risk to be ignored, if appropriate benchmarking procedures are not in place, allowing to compare the actual practical results with reference to standard accepted procedures. Both replication and benchmarking are needed to foster a cumulative advancement of our knowledge of intelligent physical agents and even to correctly appreciate disruptive innovation in the science and technology of robots. Should we take inspiration from experimental practice in disciplines such as biology or medicine?
In order to address these needs the European Robotics Network of Excellence (EURON) has funded a Special Interest Group on Good Experimental Methodology and Benchmarking.
The European Commission sponsored the EURON network (EURON I and EURON II) till the end of April 2008. Since then, EURON 3 continues on the momentum that it gained during almost a decade of networking. All the activities of Euron 3, including this SIG, continue without public funding and on voluntary basis.
