High-level cognitive competencies encompassing knowledge representation, perception, control, and learning are considered essential elements that will allow robots to perform complex tasks within highly uncertain environments with an appropriate level of autonomy in various domains (service, manufacturing, and search and rescue). As the complexity and variety of required tasks and of the targeted environments increase exploring new application capabilities and domains, it becomes necessary to develop well principled procedures that allow quantitative comparison of the solutions provided by robotics research thereby facilitating exchanges of methods and solutions between different research groups and assessment of the state of the art.
New more successful implementations of concepts already presented in literature, but not implemented with exhaustive experimental methodology, run the risk of being ignored, if appropriate benchmarking procedures allowing to compare the actual practical results with reference to standard accepted procedures, are not in place.
The emphasis of the workshop will be on principles, methods, and applications expanding beyond the current limit of robotics applications in terms of cognitive capabilities and autonomy. Another key issue will be a capability-led understanding of cognitive robots: how to define shared ontologies or dictionaries to discuss robotic cognitive systems in terms of their performance, relationships between different cognitive robotics capabilities, requirements, theories, architectures, models and methods that can be applied across multiple engineering and application domains, detailing and understanding better the requirements for robots in terms of performance, the approaches to meeting these requirements and the trade-offs in terms of performance.
The proper definition of benchmarking is related to the problem of measuring capabilities of robots in a context in which, in many cases, the ‘robotics experiments’ themselves are difficult to ‘replicate’.