Humanoids 2011 11th IEEE-RAS International Conference on Humanoid Robots Bled, Slovenia October 26th−28th, 2011

Workshop Proposal: “The DEXMART project for advanced bimanual manipulation”, Oct 26, 2011


>>> Organisation

>>> Synopsis

>>> Program

>>> Abstracts



Prof. Bruno Siciliano

Dipartimento di Informatica e Sistemistica

Università di Napoli Federico II

Via Claudio 21, 80125 Napoli, Italy


“The DEXMART project for advanced bimanual manipulation”- workshop takes place:

Oct 26, 2011 09:00-12:30


For further information and workshop announcements see:





DEXMART is a Large scale integrating project funded by the European Community under FP7 which started in February 2008, and will end by January 2012. The ambition of the project is to fill the gap between the use of robots in industrial environments and the use of future robots in everyday human and unstructured environments, contributing to reinforce European competitiveness in all those domains of personal and service robotics where dexterous and autonomous dual-hand manipulation capabilities are required.


DEXMART is contributing to the development of robotic systems endowed with dexterous and human-aware dual-arm/hand manipulation skills for objects, operating with a high degree of autonomy in unstructured real-world environments. These are the main objectives of the project:

  • allow a dual-arm robot including two multi-fingered redundant hands to grasp and manipulate the same objects (different shape, dimension and weight) used by human beings;
  • manipulation will take place in unsupervised, robust and dependable manner so as to allow the robot to safely cooperate with humans for the execution of given tasks;
  • a robotic system able to autonomously decide between different manipulation options, and to learn new action sequences aimed at creating a consistent and comprehensive manipulation knowledge base;
  • possible exploitation of high power-to-weight ratio of smart materials and structures, aimed at design of new hand components (finger, thumb, wrist) and sensors for the next generation of dexterous robotic hands.

The goal of this workshop is to present the principal results of the project during the latest four years to the scientific community of people working in the field of grasping and dual arm/hand manipulation. The achievement of the research objectives of the project will have an important impact toward the realisation of a robust and versatile behaviour of artificial systems in open-ended environments providing intelligent response in unforeseen situations, and enhancing human-machine interaction.


The topics to be discussed at the workshop include:

  • development of original approaches to interpretation, learning, and modelling, from the observation of human manipulation at different levels of abstraction;
  • development of original approaches to task planning, coordination and execution so as to confer to the robotic system self-adapting capabilities and reactivity to changing environment and unexpected situations, also in the case of humans cooperating with it;
  • design of effective control strategies for a dual-hand/arm robot manipulator that can be easily parameterised so as to preserve smoothness during the transitions at the contact with objects;
  • design and development of new actuators, as well as smart force and tactile sensors, able to overcome the limitations of current manipulation devices;
  • development of meaningful benchmarks for dual-hand manipulation.

The presentations will be accompanied by videos illustrating the experimental results on the available set-ups, including the new DEXMART hand at University of Bologna, the humanoid Rolling Justin at DLR, the dual-arm system at FZI, and the mobile manipulator Jido with KUKA lightweight arm at LAAS.






Michael Beetz (Technische Universität München), “Cognition-enabled everyday



Bruno Siciliano (Università di Napoli Federico II), “DEXMART: Main achievements,

discussion of open problems and research trends in the field”


Rainer Jäkel (FZI Universität Karlsruhe), “Programming by demonstration — a planningbased



Christoph Borst (DLR), “Observation and execution”


Daniel Sidobre (LAAS), “Human−robot interaction”


Luigi Villani (Università di Napoli Federico II), “Grasping and control of multifingered hands”


Gianluca Palli (Università di Bologna), “Innovative technologies for the next generation of

robotic hands”


Oussama Khatib (Stanford University), “Robots and the human”





Michael Beetz (Technische Universität München), “Cognition-enabled everyday manipulation”

In recent years we have seen tremendous advances in the mechatronic, sensing and computational infrastructure of robots, enabling them to act faster, stronger and more accurately than humans do. Yet, when it comes to accomplishing manipulation tasks in everyday settings, robots often do not even reach the sophistication and performance of young children. Housework is an activity domain where the superiority of the human cognitive mechanisms and their role in competent activity control is particularly evident. In this talk I will present our ongoing research, in which we investigate cognitive mechanisms that are to enable autonomous robots to produce flexible, reliable and highperformance behavior for everyday manipulation activities. The talk will concentrate on our framework for cognition-enabled control and the naturalistic specification of manipulation

actions and what it takes to execute these specifications competently.


Bruno Siciliano (Università di Napoli Federico II), “DEXMART: Main achievements, discussion of open problems and research trends in the field”

DEXMART will end in a few months. This talk will give a brief introduction of the project, its goal and organization, and the main achievements. In particular, the interconnections among the research activities to achieve the common goal of “development of bimanual manipulation robotic system” will be highlighted. Such interconnections are formalized by three functional architectures called “bricks”; namely: A) Programming by demonstration/Observation and execution, B) Human−robot interaction, C) New technologies and feedback control. The conceptual approach is that to separate hardware development from abstract levels of software architecture. The “components” are identified as the basic portions of work into which each Brick is articulated, while their integration is entrusted to a number of interfaces where data, format and communication proposal are of concern.


Rainer Jäkel, Andreas Hermann, Martin Lösch, Steffen Rühl, Sven R. Schmidt-Rohr, Zhixing Xue (FZI Universität Karlsruhe), “Programming by demonstration — a planningbased approach”

In the human environment, autonomous service robots have to be highly flexible to manipulate different kinds of objects in a restricted workspace with multiple obstacles. Search-based AI, e.g. motion planning, has been applied successfully to solve common manipulation tasks, e.g. opening doors or loading a dish washer, but a sophisticated planning model is required, which is usually hard to define manually. In this talk, our research in Dexmart in the field of Robot Programming by Demonstration (PbD) will be summarized. Our goal is to learn planning models, which can be executed successfully in different environments and on multiple robot systems, based on the observation of a single or multiple human teachers. The focus will be on learning, generalization and execution of planning models. I will present different examples including videos of the execution on the robots Adero and Albert II.


Christoph Borst, Franziska Zacharias, Florian Schmidt, Daniel Leidner (DLR), Sven Schmidt-Rohr, Rainer Jäckel, Martin Loesch (FZI Universität Karlsruhe), Pietro Falco, Ciro Natale (Seconda Università di Napoli), “Observation and execution”

Learning motion generation and grasping from human observation has been a fruitful topic for robotics research in the last years. Methods for autonomous segmentation and classification of demonstrated trajectories as well as techniques to adapt on new scenarios have been presented. Recently also the successful extraction of task specific constraints from observed human manipulation with statistical learning methods have been shown. On the other hand geometric grasp and path planners are integrated on more and more humanoid robots and show how these models based solutions can increase the flexibility of the robot in one and two handed manipulation actions. Specifying all the necessary task constraints for a large number of tasks however is a laborious job, and knowledge from observing human task execution is beneficial. In DEXMART a main focus was put on data representations that can be obtained from human observation and representations that are needed or produced for implementing or parameterizing robot control programs for manipulation tasks. We identified different representations for different levels of abstraction. Representations from observation and execution were compared and similarities are exploited to support the autonomous generation of control programs by characteristics from human observation.


Daniel Sidobre, Xavier Broquère, Jim Mainprice, Wuwei He (LAAS), “Human−robot interaction”

In recent years robots interacting with humans became a reality in our laboratories. However, complete solution to exchange objects naturally is not yet available. The reactive grasping of object is still challenging and there is no solution to detect that the partner has grasped or released the object. From a control point of view, it is not easy to ensure the safety and comfort of the movements in the vicinity of the human. In this talk we will present our ongoing research on interactive manipulation to exchange object with humans. In the first part we will introduce the architecture of a robot capable of exchanging objects with a human. This architecture is built around the concept of soft trajectory to ensure human safety and comfort. Trajectories are used from the motion planning to the control levels. We will show that the choice of the exchange point is essential for the human comfort and influence also the choice of the grasp. In the second part, we will detail how it is possible to choose a grasp in real time. The proposed method is based on a series of adapted models for objects, hands and grasps and a maximum pre-computation approach.


Luigi Villani, Vincenzo Lippiello, Fabio Ruggiero, Fanny Ficuciello, Bruno Siciliano (Università di Napoli Federico II), Gianluca Palli (Università di Bologna), “Grasping and control of multifingered hands”

An important issue in controlling a multi-fingered robotic hand grasping an object is the synthesis of the optimal contact points and the evaluation of the minimal contact forces able to guarantee the stability of the grasp and its feasibility. Both these problems can be solved online if suitable sensing information is available. In detail, using vision, a surface reconstruction algorithm and a grasp planner evolving in a synchronized parallel way have been designed to grasp objects of unknown geometry. On the other hand, using finger tactile information and contact force measurements, an efficient algorithm was developed to compute the optimal contact forces, assuming that, during the execution of a manipulation task, both the position of the contact points on the object and the wrench to be balanced by the contact forces may change with time. Another goal pursued in DEXMART was the development of a human-like grasping approach inspired to neuroscience studies. In order to simplify the synthesis of a grasp, a configuration subspace based on few predominant postural synergies of the robotic hand has been computed. This approach was evaluated at kinematic level, showing that power and precise grasps can be performed using up to the third predominant synergy.


Gianluca Palli, Claudio Melchiorri, Gabriele Vassura (Università di Bologna), Salvatore Pirozzi, Ciro Natale, Giuseppe De Maria (Seconda Università di Napoli), Chris May (Universität des Saarlandes), “Innovative technologies for the next generation of robotic hands”

Aiming at reproducing in some way the grasping and manipulation capabilities of humans, many robotic devices have been developed all over the world in more than 50 years of research on this topic, starting from very simple grippers, often used also in industrial activities, to very complex anthropomorphic robotic hands, but the reduced functionality and/or reliability of the devices developed so far prevents, together with the cost, their usability in unstructured environments, and in particular in human everyday activities. The adoption of design solutions inherited from conventional mechanics and the lack of purposely developed sensors and actuators are the main causes of the partial fail in achieving the final goal of reproducing human manipulation capabilities. Our research activity aims to the development of innovative solutions from the point of view of both the mechanical design, and the sensory equipment and actuation. The final goal is the implementation of anthropomorphic robotic hands with improved reliability, functionality and reduced complexity and cost, considering also aspect related to safety during human robot interaction, paving the way toward the next generation of robotic hands.


Oussama Khatib (Stanford University), “Robots and the human”

In the field of robotics, the motivation to emulate human movement has been driven by the desire to endow robots, humanoids in particular, with human-like movement characteristics. Inspired by human behaviors, our extensive study of human musculoskeletal system has brought insights and results that proved extremely valuable in human biomechanics. Understanding human motion requires accurate reconstruction of movement sequences, modeling of musculoskeletal kinematics, dynamics, and actuation. These issues have much in common with the problems of articulated body systems studied in robotics research. Building on methodologies and techniques developed in robotics, a host of new effective models and tools have been established for the synthesis of human motion. These include efficient algorithms for the simulation of musculoskeletal systems, novel physio-mechanical criteria and performance measures, real-time tracking and reconstruction of human motion, and dynamic human performance characterization. These developments are providing new avenues for exploring human motion — with exciting prospects for novel clinical therapies, athletic training, character animation, and human performance improvement.


Double strike in one day 16.12.2013

What an incredible coincidence: two Ex-DEXMARTians were awarded on the very same day!

Best scientific computer science PhD thesis of 2012 01.08.2013

Prize for Sven Schmidt-Rohr of Karlsruhe University, Germany