Work performed third year

With reference to Annex I of the Grant Agreement and the timing of the six research and development work packages into which the project is articulated, the work performed during the first year has dealt with the following challenges:

  • observation of dual-arm/hand manipulation activities performed by humans, where the observation is done at different levels of granularity to deduce goals and strategies at different levels of abstraction;
  • representation of objects, obstacles, grasps and human capabilities and behaviours, leading to models of scene dynamics at different levels of abstraction from basic actions to high level manipulation activities;
  • generalized manipulation strategy representation from observation of trajectory level human demonstrations including temporal aspects;
  • complete dynamic model and control structure of fingers and hand, including nonlinear dynamic model of tendon and joint friction;
  • development of twisted string actuation concept and its application to finger with integrated sensors and electronics, as well as preliminary prototype of whole robot hand;
  • specification of common matrix structure aimed at describing different benchmarks.

Work performed during third year and results achieved so far

With reference to Annex I of the Grant Agreement and the timing of the six research and development work packages into which the project is articulated, the work performed during the third year has dealt with the following challenges:


WP1 ― Observation and Learning from Human

  • Unsupervised procedure for feature extraction and segmentation of kinetostatic data
  • Recognition and modeling procedure of full body manipulation actions through coupled hidden Markov models
  • Learning of bimanual manipulation strategies through strategy graphs using constraints as atomic elements


WP2 ― Scene, Objects and Dexterous Manipulation Representation

  • Concept of grasping affordances of an object to help learning how to grasp unknown objects
  • Fuzzy clustering strategy to classify the primitive actions of a set of elementary grasping and manipulation tasks performed by a human demonstrator
  • Dexterous bimanual manipulation strategies at the level of learning and execution


WP3 ― Artificial Cognitive System for Dual-Arm/Hand Manipulation

  • Extension of scene analysis and task reasoning to consider naturalness of bi-manual motions and include online monitoring in the symbolic planning process
  • Development of new methods of grasp and motion planning during interaction with humans
  • Thoroughly improved learning planning models from human demonstration on both symbolic, abstract planning as well as motion planning level


WP4 ― Dual-Arm/Hand Control

  • Integration of a first prototype of the new mechanical hand with standard actuation on an industrial arm and first tests of motion coordination
  • Development of a control strategy for dual-arm-hand manipulation based on tactile force sensing
  • Integration of a supervisory attentional system for monitoring and regulating safe and human-aware manipulation into the planner


WP5 ― Towards the Next Generation of Robotic Hands

  • Final implementation of the tactile sensor
  • Integration of all the sensors and electronics within the finger and the hand structure
  • Development of finger actuated by means of twisted string actuation concept


WP6 ― Benchmarking and Experiments

  • SImplementation of the first individual benchmarks to test and validate components for the artificial cognitive system as well as for the new technologies integrated in the UB Hand IV
  • Definition of common application level benchmarks to test and validate integrated components from different project beneficiaries in the plan for integration of DEXMART
  • Validation in simulation of the actuation and control system of the new hand, and design verification in static grasping experiments

Expected results and their potential impact


The achievement of the research objectives will have an important impact toward the achievement of robust and versatile behaviour of artificial systems in open-ended environments providing intelligent response in unforeseen situations, and enhancing human-machine interaction.


The key innovations to bringing about this impact through the research carried out within DEXMART are:

  • development of original approaches to interpretation, learning, and modelling, from the observation of human manipulation at different levels of abstraction;
  • development of original approaches to task planning, coordination and execution so as to confer to the robotic system self-adapting capabilities and reactivity to changing environment and unexpected situations, also in the case of humans cooperating with it;
  • design of effective control strategies for a dual-hand/arm robot manipulator that can be easily parameterised so as to preserve smoothness during the transitions at the contact with objects;
  • design and development of new actuators, as well as new mechanical structures and materials, able to overcome the limitations of current manipulation devices;
  • development of meaningful benchmarks for dual-hand manipulation.


To sum up, the DEXMART project has the ambition to fill the gap between the use of robots in industrial environments and the use of future robots in everyday human and unstructured environments, contributing to reinforce European competitiveness in all those domains of personal and service robotics where dexterous and autonomous dual-hand manipulation capabilities are required.

Double strike in one day 16.12.2013

What an incredible coincidence: two Ex-DEXMARTians were awarded on the very same day!

Best scientific computer science PhD thesis of 2012 01.08.2013

Prize for Sven Schmidt-Rohr of Karlsruhe University, Germany