Workshop at RSS 2009 (Seattle, WA, USA, June 28, 2009)
Organizers
This e-mail address is being protected from spambots. You need JavaScript enabled to view it. This e-mail address is being protected from spambots. You need JavaScript enabled to view it. This e-mail address is being protected from spambots. You need JavaScript enabled to view it.
Content
As the complexity of current robotic and embodied intelligent systems grows, it is more and more necessary to define proper experimental approaches and benchmarking procedures. The Special interest group on Good Experimental Methodology was formed by EURON in response to a wide perception — evidenced by some 90 responses to an email call for interest — that roboticists could do much better at performing and, particularly, reporting experimental work. A major output of the Special Interest Group is a set of guidelines for good experimental (reporting) practice, which we believe supports the community’s wish to produce better experimental work. Equally significant is a series of workshops on Benchmarking and Good Experimental Methodology held at major Robotics conferences. Good experimental work is not novel in robotics: there are many good experimental scientists in our community. However, uniformly good experimental work and reporting has not yet been achieved. The evidence of community interest in the topic mentioned above suggests that the community is willing to take steps to improve in this area. Thus we propose to take the work of good experimental robotics groups and use it to illustrate high-quality experimental work and reporting, using the workshop format to draw out the strengths (and weaknesses) of presented work and encourage sharing and adoption of good practices.
Workshop Highlights:
List of topics
- Design of Experiments in Robotics
- Execution of Experiments in Robotics
- Reporting Experiments in Robotics
- Examples of Good Practice
- Evaluation of Experimental Robotics Work
- Proposals for Promotion of Good Experimental Work
Format:
The workshop will consist of presentations interleaved with a significant amount of additional time for discussions between the presentations and at the end of the full day single track sessions.
Proceedings:
The workshop has proceedings HERE BELOW!; selected contributions may be invited to submit to a journal special issue on good experimental robotics.
Previous workshops:
Previous workshops on the topic of Benchmarking have been organized successfully at IROS (3 in a row) and RSS conferences.
Unconference style interaction is encouraged:
This will be a 'working' :-) workshop interaction in person and by social networking media with people in the room and outside is welcome
Program & Pre-Proceedings
08:00
Coffee and registration
09:00 - 09:10
Introduction
F. Bonsignorio
Publishing Identifiable Experiment Code And Configuration Is Important, Good and Easy
J. Wawerla R. T. Vaughan
School of Computing Science, Simon Fraser University
Toward a Science of Robotics: Goals and Standards for Experimental Research
L. Takayama
Willow Garage
9:50 -10:10
Experiment Design for Large Multi-Robot Systems
J. McLurkin
Computer Science and Engineering
University of Washington
K. M. Tsui H. A. Yanco
University of Massachusetts Lowell
10:30
Coffee Break
11:00 - 11:20
Experimental Identification of Friction and Dynamic Coupling in a Dual Actuator Testbed
D. Rabindran D. Tesar
Robotics Research Group
Department of Mechanical Engineering
The University of Texas at Austin, USA
11:20 - 11:40
Good Experimental Methodologies for Mobile Robot Olfaction
L. Marques
Institute of Systems and Robotics
University of Coimbra
A Proposal of a Set of Metrics for Collective Movement of Robots
I. Navarro F. Matia
Intelligent Control Group
Universidad Politecnica de Madrid
A Traceable Inertial Calibration Parameter Estimation Procedure Suited for MEMS Sensing
S. P. N. Singh
Australian Centre for Field Robotics, at the School of Aerospace, Mechanical, and Mechatronic Engineering
University of Sydney
12:20 - 13:00
Open Discussion
13:00 - 14:00
Lunch
14:00 - 14:10
Morning recap
F. Bonsignorio
T. Peynot S. Scheding
ARC Centre of Excellence for Autonomous Systems
Australian Centre for Field Robotics
The University of Sydney
14:30 - 14:50
Acquisition of 2-D ground truth data in multirobot experiments
M. Delaine Anderson
The University of Alabama
Evaluating the Performance of Robot Mapping Systems
E. Olson, University of Michigan
M. Kaess, MIT
Evaluation of Adaptive Technology Algorithms in Small Robotic Platforms
M. Farmer J. Pippine C. Sullivan C. Valdez A. Watson
System Planning Corporation
15:30 - 16:00
Coffee Break
Defining the requisites of a replicable robotics experiment
F.P. Bonsignorio Heron Robots
A.P. Del Pobil University Jaume I Castellon
J. Hallam South Denmark University Odense
16:30 - 17:45
Open discussion
17:45 - 18:00
Wrap-up
