Workshop at RSS'08 (Zurich, Switzerland, June 28, 2008).

Organizers

This e-mail address is being protected from spambots. You need JavaScript enabled to view it.    This e-mail address is being protected from spambots. You need JavaScript enabled to view it.    This e-mail address is being protected from spambots. You need JavaScript enabled to view it.

Motivation

As the complexity of current robotic and embodied intelligent systems grows, it is more and more necessary to define proper experimental approaches and benchmarking procedures. On the one hand, reliable benchmarks are called for in order to allow the comparison of the many research results in robotics research, so that their industrial application is eventually possible. On the other hand, if robotics aims to be regarded as serious science, replication of experiments deserves consciencious attention; it is necessary to be able to verify if and by which measure new procedures and algorithms proposed in research papers constitute a real advancement and can be used in new applications.

 

This workshop aims to provide a discussion forum on these topics and to identify guidelines for the future. It will consist of invited and regular presentations with a significant amount of additional time for discussions. It is envisioned to be useful for anyone with an interest in quantitative performance evaluation of robots and/or robot algorithms. Some controversial issues will be discussed such as: measuring autonomy or information metrics of intelligent systems, or the concept itself of replicability or benchmarking of research results in robotics.

Program

13:30

Introduction

Angel P. del Pobil

 

14:00

The Jacobs Map Analysis Toolkit

Ioana Varsadan Andreas Birk Max Pingsthorn Soren Schwertfeger Kaustubh Pathak

Jacobs University

 

14:20

RobotStadium: Online Humanoid Robot Soccer Simulation Competition

Olivier Michel

Cyberbotics ltd

 

14:40

Can We Benchmark The Influence of Information-Processing Architectures on Intelligent Systems?

Nick Hawes Jeremy Wyatt

School of Computer Science

University of Birmingham, Birmingham, UK

 

15:00

Coffee Break

 

15:30

The RAWSEEDS Proposal for Representation-Independent Benchmarking of SLAM

Giulio Fontana Matteo Matteucci Domenico G. Sorrenti

Politecnico di Milano and University of Milano Bicocca

 

15:50

A unified benchmark framework for autonomous Mobile robots and Vehicles Motion Algorithms (MoVeMA benchmarks)

Daniele Calisi Luca Iocchi Daniele Nardi

Department of Computer and Systems Science

"Sapienza" University of Rome, Italy

 

16:10

Performance metric for vision based robot localization

Emanuele Frontoni Andrea Ascani Adriano Mancini Primo Zingaretti

Univerità Politecnica delle Marche

 

16:30

The Euron GEM Review guidelines

Fabio Bonsignorio John Hallam Angel P. del Pobil

 

16:50

Open discussion


Additional information