Work performed during second year

With reference to Annex I of the Grant Agreement and the timing of the six research and development work packages into which the project is articulated, the work performed during the first year has dealt with the following challenges:

  • observation of dual-arm/hand manipulation activities performed by humans, where the observation is done at different levels of granularity to deduce goals and strategies at different levels of abstraction;
  • representation of objects, obstacles, grasps and human capabilities and behaviours, leading to models of scene dynamics at different levels of abstraction from basic actions to high level manipulation activities;
  • generalized manipulation strategy representation from observation of trajectory level human demonstrations including temporal aspects;
  • complete dynamic model and control structure of fingers and hand, including nonlinear dynamic model of tendon and joint friction;
  • development of twisted string actuation concept and its application to finger with integrated sensors and electronics, as well as preliminary prototype of whole robot hand;
  • specification of common matrix structure aimed at describing different benchmarks.

Results achieved so far

In the following the main results achieved so far are described, whereas a more detailed account with reference to the various tasks of the project is deferred to the reports of the individual work packages.


WP1 ― Observation and Learning from Human

  • Automatic exploration process to extract relevant features from observed bimanual manipulation activities
  • Integration of tactile sensors into sensorized data glove for fingertip contact force measurement
    during observation of manipulation actions
  • Bidule sensorized object to observe object hand-over tasks involving human-robot interaction

WP2 ― Scene, Objects and Dexterous Manipulation Representation

  • Representation of obstacles in the capability maps
  • Extended representation of manipulation strategies based on atomic constraints by introducing temporal constraints
  • Models of interaction between robot and human exchanging objects


WP3 ― Artificial Cognitive System for Dual-Arm/Hand Manipulation

  • Methodology to learn generalized manipulation strategies, applicable to motion planning from human trajectory level demonstrations
  • Enhanced human aware planning
  • Method for positioning mobile manipulators for constrained manipulation tasks

WP4 ― Dual-Arm/Hand Control

  • Complete model of tendon and finger for new robot hand, with testing of finger control laws
  • GFO algorithm for real-time optimization of contact forces, exploiting contact forcemeasurements
  • Supervisory attentional system suitable for monitoring and regulating safe and human aware manipulation

WP5 ― Towards the Next Generation of Robotic Hands

  • Design of tactile sensors based on LED-phototransistor couples and deformable elastic layer
  • Development of finger with integrated sensors and electronics actuated by means of twisted string actuation concept
  • Investigation on innovative solution for compliant joints and soft covers/li>
  • Preliminary prototype of robotic hand

WP6 ― Benchmarking and Experiments

  • Specification of common matrix structure in order to describe each benchmark
  • Decision on individual benchmarks to be implemented by beneficiaries using their test-beds
  • Preparation of test-beds for experimentation and benchmarking

Expected results and their potential impact

The achievement of the research objectives proposed within DEXMART will have an important impact toward the achievement of robust and versatile behaviour of artificial systems in open-ended environments providing intelligent response in unforeseen situations, and enhancing human-machine interaction.


The key innovations to bringing about this impact through the research carried out within the DEXMART project are:

  • development of original approaches to interpretation, learning, and modelling, from the observation of human manipulation at different levels of abstraction;
  • development of original approaches to task planning, coordination and execution so as to confer to the robotic system self-adapting capabilities and reactivity to changing environment and unexpected situations, also in the case of humans cooperating with it;
  • design of effective control strategies for a dual-hand/arm robot manipulator that can be easily parameterised so as to preserve smoothness during the transitions at the contact with objects;
  • design and development of new actuators, as well as new mechanical structures and materials, able to overcome the limitations of current manipulation devices;
  • development of meaningful benchmarks for dual-hand manipulation.

To sum up, the DEXMART project has the ambition to fill the gap between the use of robots in industrial environments and the use of future robots in everyday human and unstructured environments, contributing to reinforce European competitiveness in all those domains of personal and service robotics where dexterous and autonomous dual-hand manipulation capabilities are required.

Double strike in one day 16.12.2013

What an incredible coincidence: two Ex-DEXMARTians were awarded on the very same day!

Best scientific computer science PhD thesis of 2012 01.08.2013

Prize for Sven Schmidt-Rohr of Karlsruhe University, Germany