Robot Learning and Interaction

The Robot Learning & Interaction group focuses on human-centered robotic applications in which the robots can learn new skills by interacting with the end-users.

From a machine learning perspective, the challenge is to acquire skills from only few demonstrations and interactions, with strong generalization demands. It requires the development of intuitive active learning interfaces to acquire meaningful demonstrations, the development of models that can exploit the structure and geometry of the acquired data in an efficient way, and the development of adaptive control techniques that can exploit the learned task variations and coordination patterns.

The developed models must serve several purposes (recognition, prediction, online synthesis), and be compatible with different learning strategies (imitation, emulation, incremental refinement or exploration). For the reproduction of skills, these models need to be enriched with force and impedance information to enable human-robot collaboration and to generate safe and natural movements.

These models and algorithms can be applied to a wide range of robotic applications, with robots that are either close to us (assistive robots and human-robot collaboration in SMEs), part of us (prosthetic hands and exoskeletons), or far away from us (shared control of remote robots).

Video Presentation

Current Group Members

CALINON, Sylvain
(Senior Researcher)
- website

SILVERIO, João (Pedro Lourenço)
(Postdoctoral Researcher)
- website

EWERTON, Marco (Antonio)
(Postdoctoral Researcher)
- website

PIGNAT, Emmanuel
(Research Assistant)
- website

(Research Assistant)
- website

(Research Assistant)
- website

(Research Assistant)
- website

SHETTY, Suhan (N.)
(Research Assistant)
- website

LEMBONO, Teguh (Santoso)
(Research Assistant)
- website

KULAK, Thibaut
(Research Assistant)
- website

GAO, Xiao
(Research Intern)
- website

VAES, Clément
- website


  • ABBET, Christian
  • BERIO, Daniel
  • HAVOUTIS, Ioannis
  • KUPCSIK, Andras (Gabor)
  • TANWANI, Ajay (Kumar)
  • TAO, Dominique
  • WÜTSCHERT, Robin

Current Projects

Recent Projects

Group News

Robots and humans collaborate to play music together
research — Jan 22, 2020

Collaborative robots are the perfect tool to link art and science. Thanks to his visit at Idiap, a researcher from Goldsmiths University of London used this approach to demonstrate the concept with a piano.

Idiap has a new opening for a PhD position in Robot learning from demonstration
research — Dec 23, 2019

The Robot Learning & Interaction Group at Idiap has an open PhD position within the CODIMAN project (cobotics, digital skills and the re-humanization of the workplace), in collaboration with the Bern University of Applied Sciences (BFH). The research will be supervised by Dr Sylvain Calinon in the Robot Learning & Interaction group at Idiap, and will be achieved in tight collaboration with Prof. Sarah Dégallier Rochat at the Human-Centered Engineering Institute (HuCE) of BFH.

"Deep neural networks remain for the most part black boxes"
research — May 08, 2019

Artificial deep neural networks are a powerful tool, able to extract information from large datasets and, using this acquired knowledge, make accurate predictions on previously unseen data. Due to the very large number of parameters required, they are also particularly difficult to understand.

Like humans, robots can learn to walk
research — Feb 06, 2019

Following a comparable path, robots could learn to move and walk as human being do. The goal of the MEMMO project is to develop a unified approach to motion generation for complex robots with arms and legs.

Vision and machine learning for robot manipulation
education — Dec 19, 2018

The Perception and Activity Understanding group (Jean-Marc Odobez,, along with the Robot Learning and Interaction group (Sylvain Calinon, at Idiap (, seeks one motivated postdoctoral fellow to work on interaction strategies and deep learning for object manipulations in cluttered object scenes.

On February 2nd, 2018, Ajay Tanwani successfully defended his PhD thesis entitled 'Generative Models for Learning Robot Manipulation Skills from Humans'
education — Feb 07, 2018

The thesis presents generative mixture models for learning robot manipulation skills from a few human demonstrations, and providing assistance to a human operator in performing these skills at a distance under limited bandwidth and communication delays. The models are based on invariant task-parameterized representations of hidden semi-Markov models that allow segmentation and reproduction of manipulation skills, while generalizing effectively across different environmental situations such as size, position, and orientation of objects in the environment.

3 promotions to Senior Researcher at Idiap
institute — Jan 30, 2018

Following a very strict evaluation process – comprising the nomination by Idiap's management and a formal approval by the Scientific College - , Idiap is pleased to announce the promotion of three of its researchers to Senior Researchers:

Idiap has a new opening for a PhD positions in multimodal perception for robotics and gesture learning
education — Dec 13, 2017

The Perception and Activity Understanding group (Jean-Marc Odobez, seeks one PhD candidate for a Swiss NSF funded project aiming to study robot skills acquisition through active learning and social interaction strategies (ROSALIS, see below). In the project, the PhD candidate will work on the multimodal perception of persons and objects, and collaborate with two other PhD students working on skill learning and interaction modeling.

Idiap of Martigny develops a new imaging system for research
research — Nov 06, 2017

The Idiap Research Institute develops a new imaging system for research. The precision instrument has been built together with the air base of Sion and could be used by companies and the Universities of Applied Science of the Canton.

"Combining Electromyography and Tactile Myography to Improve Hand and Wrist Activity Detection in Prostheses" paper posted on Technologies/ journal social media platform
research — Oct 16, 2017

The paper of Noémie Jaquier, Mathilde Connan, Claudio Castellini and Sylvain Calinon from Idiap Research Institute and DLR-German Aerospace Center, "Combining Electromyography and Tactile Myography to Improve Hand and Wrist Activity Detection in Prostheses", has been posted on /Technologies/ journal social media platform.

New research groups at Idiap
research — Jul 09, 2014

The Idiap Research Institute is proud to announce that 2 new researchers have recently joined the research team. Sylvain Calinon and Michael Liebling will reinforce, complement and develop the Institute’s expertise and competencies in human-robot interaction and computational bioimaging.

Meet Baxter, Idiap’s new robot
research — Sep 04, 2014

Idiap pursues its development with new scientific research groups. Recently, the institute has acquired an American robot worth CHF 30’000.-. This investment opens new perspectives, in particular thanks to the interconnection between the different domains of competencies developed in Martigny.

Idiap Video amongst the 10 finalists of the Rethink Robotics Video Challenge
research — Dec 15, 2015

In the summer of 2015, Rethink Robotics, the worldwide leader in collaborative robotics research and education launched the Rethink Robotics Video Challenge. The Challenge was created to highlight the amazing work being done by the research and education community with the Baxter robot. The Challenge attracted more than 90 total entries from 19 countries around the globe. The video of Dr. Sylvain Calinon’s Robot Learning & Interaction group of the Idiap Research Institute in Martigny was selected amongst the 10 finalists.

Idiap's DexROV project featured in RTS's CQFD radio show today
research — Feb 24, 2016

DexROV is a robot of a special kind. This underwater robot is destined to take the place of human divers for tasks in very deep water (up to 1’300 meters under the sea). The Idiap Research Institute in Martigny is in charge of developing the algorithms that will help teleoperating the two arms of this robot.

Canal9 visits the Baxter robot at Idiap
research — Apr 14, 2016

Dr. Sylvain Calinon, head of Idiap’s Robot Learning & Interaction group, receives Romain Boisset, reporter for the Canap9 TV channel. They demonstrate how Idiap’s Baxter robot learns by demonstration.

DexROV holds first official review with European Commission
research — Nov 21, 2016

Nineteen months after the undersea-operations project was launched, DexROV has successfully completed its first funding review as part of the European Commission’s Horizon 2020 Research and Innovation programme.

3 promotions au poste de Senior Researcher à l’Idiap
institute — Jan 30, 2018

Suite à un processus d’évaluation très stricte – comprenant une nomination par la Direction et une validation par le Collège scientifique -, l’Idiap est heureux d’annoncer la promotion de trois de ses chercheurs au poste de Senior Researcher :

Une video d’Euronews consacrée aux tests du projet DexROV : « Comment nous bénéficierons de la nouvelle génération intelligente de ROVs (remotely operated vehicles) ? »
research — Jul 18, 2017

L’équipe de DexROV vient juste de compléter 2 semaines de campagne d’intégration et de tests dans le cadre du projet DexROV. Pendant ces travaux, une liaison satellite a été établie entre Bruxelles et Marseille pour permettre la surveillance à longue distance d’un robot sous-marin.

Best Paper Award for the Dr Sylvain Calinon
research — Jul 12, 2017

Dr Sylvain Calinon received the 2017 Best Paper Award of the Intelligent Service Robotics (ISR) journal for his article "A Tutorial on Task-Parameterized Movement Learning and Retrieval".

Idiap has a new opening for 3 PhD positions in perception, robot skill learning and HRI
education — Aug 02, 2017

The Perception and Activity Understanding group (Jean-Marc Odobez, and the Robot Learning and Interaction group (Sylvain Calinon, seek 3 PhD candidates for a Swiss NSF funded project aiming to study robot skills acquisition through active learning and social interaction strategies (ROSALIS, see below). The 3 PhD students will collaborate together, with 3 axes of research:

Idiap has three new openings: Senior Researcher, Postdoc and Phd position
education — Sep 06, 2017

To strengthen and diversify its growth, the Idiap Research Institute invites applications for a permanent Senior Researcher position in the areas of high-performance computing for AI, visual analytics, or cyber-security and data privacy. The Robot Learning & Interaction Group has also two new open positions, a Postdoc and a phd position within the new MEMMO (Memory of Motion) ICT-H2020 European project.