Human-centered Robotics and AI
The Human-centered Robotics and AI’s research lies at the intersection of robotics, human-robot interaction, machine learning, and participatory design, with the goal of creating robots that are physically useful and socially acceptable.
Introduction
Led by Dr. Emmanuel Senft, the group studies how robots and AI systems can be developed with users through participatory design approaches and deployed in real-world environments where they can interact safely and naturally with people.
We use a broad range of methodological approaches to meaningfully involve users in shaping robot behavior. At design time, we develop and apply participatory design methods to facilitate communication between users and researchers, and to empower end users to actively contribute to the creation of new technologies. At runtime, we investigate end-user programming, interactive learning, and shared autonomy, providing multiple ways for users to guide and adapt robot behavior so it aligns with their individual needs.
Our work spans several application domains, from serviced robotics in everyday environments to collaborative robots in industrial settings. A central focus, however, is on assistive robotics: supporting people with disabilities at home, enhancing care in settings for older adults, and advancing rehabilitation technologies that help therapists deliver more personalized and effective interventions. Across these domains, we aim to enable a wide range of users, especially non-experts in robotics, to confidently and effectively interact with and benefit from robotic systems.
Current group members
Alumni
Ongoing projects
Au cours des vingt dernières années, les robots ont progressivement quitté les environnements industriels classiques pour entrer dans les espaces de vie et de travail des humains. De nouveaux robots ont ainsi été conçus spécifiquement pour interagir avec des personnes, ce qui implique des capacités avancées de manipulation, de déplacement autonome, de programmation intuitive, et d’interaction sociale afin d’être réellement utiles et acceptés par leurs utilisateurs. Ces évolutions ont donné naissance à la robotique d’assistance, un domaine de recherche qui étudie comment les robots peuvent soutenir les humains dans des contextes variés, allant de l’aide aux personnes âgées ou en situation de handicap jusqu’au soutien des travailleurs en entreprise. Toutefois, cette recherche pose des défis importants, notamment l’implication de patients et d’utilisateurs réels dans la conception des technologies, ainsi que la création d’environnements de développement et d’évaluation qui reproduisent fidèlement les conditions réelles de vie et d’activité. Pour renforcer le réalisme et l’impact de ses futurs projets, l’Idiap souhaite développer un nouveau Centre de Robotique d’Assistance, destiné à centraliser les recherches en robotique de l’Idiap, à recréer des conditions d’usage proches du réel, à accueillir des participants locaux et à mieux communiquer les avancées scientifiques auprès du public.
We propose a novel Interactive ML framework for advanced manufacturing. We combine incremental learning (few-shot learning and active learning) and human know-how (human-in-the-loop) to face data scarcity in quality control applications, from predictive manufacturability to predictive maintenance.
Rationale:
In Switzerland, 1 769 000 people are living with disabilities, and might need frequent physical therapy to maintain functions. However, most patients can only access therapy at a low frequency (due to insurance or availability of therapists) which might be insufficient. We believe that robotic-assisted physical therapy could help address this gap by helping patients to exercise more frequently. Due to the variability between pathologies and individuals, these physical human-robot interactions can take different forms and should be set by a therapist. However, as pointed in Van der Loos et al. (2016), there are critical needs for interfaces to personalize these physical human-robot interaction that can be used by non-experts in robotics.
Objectives:
This project’s main objective is to push the science on physical human-robot interaction and develop user-centered tools that can be shared with the community to personalize physically assistive behaviors. More precisely, this project aims to answer the following question: “How can end users easily and intuitively specify complex physically interactive robot behaviors?”, using rehabilitation robotics as a use case.
The aims of the project are: (1) Develop new flexible behavior encodings for kinesthetic interaction between robots and people; (2) Develop multi-modal interfaces to specify rich kinesthetic behaviors; (3) Validate these systems in user studies with clinicians and patients.
Methods:
This project approaches the challenge of end-user design of kinesthetic robot behavior through participatory design (PD), a development method centered around the users of the technology. We will collaborate with an application partner (Centre Neu’Rhône) to develop interaction paradigms (Thread 1), behavior encodings (Thread 2), and interfaces (Thread 3) to support greater personalization and usability of physical human-robot interaction. Finally, we will evaluate our system in multiple user studies and conclude the project with a summative study where patients will be able to interact with the robot with little supervision (Thread 4). The project will leverage previous work from the PI (PD research, end-user programming, assistive robotics, and shared autonomy) and partners at the institution (learning from demonstration and behavior encodings) to develop parameterizable motions that can be easily and intuitively specified by end users, such as physical therapists.
Expected Results:
With this work, we aim to develop and make available new tools for end users to create rich kinesthetic behaviors. Through our user studies, we will demonstrate the applicability of our approach to rehabilitation therapy, showing that collaborative robots can be flexible tools to safely simplify access to therapy. Finally, we will open-source our findings to help the community build upon the research made in this project. These contributions are significant for the community, as there is today little research on the specification of rich kinesthetic behaviors directly by end users in allied health domains. Most research in end-user specification of physical behaviors remains centered on industrial application and misses challenges specific to this domain.
Impact:
This project will impact the robotic community by developing new control algorithms and interfaces for safe and personalizable physical human-robot interaction. On top of disseminating knowledge though publications in scientific venues (conferences and journals), this project will provide an open source flexible software implementation for the wider community. However, the impact will also reach beyond roboticists and researchers. By taking a user-centered approach, this project will help form the next generation of multidisciplinary researchers. Finally, this project also aims to have societal impacts by making personalized physical therapy more accessible and communicating with the public through workshops and other outreach activities.
Le projet propose de développer une installation événementielle incluant vision, IA et robotique pour la création automatique d’une mosaïque à partir de fragments aux formes et couleurs variées, issus du recyclage. L’installation est destinée à une exposition publique, évoquant à la fois l’origine romaine de Martigny (Martigny-la-Romaine, Octodure), tout en montrant les avancées de l’IA et de la robotique dans les domaines créatifs et artistiques, avec des IA à la fois physique (robots) et computationnelle (vision, image processing, analyse et classification automatique de formes, optimisation géométriques de placements).
Très utilisée pendant l'Antiquité romaine, la mosaïque est un art décoratif dans lequel on utilise des fragments de pierre, d'émail, de verre, ou de céramique pour former des motifs ou des figures en assemblant et orientant les fragments de manière judicieuse et artistique selon les éléments et contours essentiels de la figure à représenter. Notre projet propose de revisiter cet art du fragment provenant de la Rome antique, en l’adaptant aux nouvelles technologies de l’IA et de la robotique pour trier, analyser et classifier automatiquement les fragments mis à disposition, planifier le choix d’utilisation de ces fragments, leurs placements et leurs orientations sur la fresque à réaliser. Ces
fragments, qui peuvent être de différentes formes (en plus de la variété de couleurs et d’orientations possible), seront automatiquement assemblés pour représenter une image souhaitée, en discussion avec la ville de Martigny, définie selon leur souhait et l’événement ciblé (par exemple, hommage à Léonard Gianadda, fresque du Château de la Bâtiaz, Fondation Barry du Grand-Saint-Bernard, dessin correspondant au thème d’un événement organisé par la ville comme le carnaval ou la foire du Valais, etc.).
The project proposes to develop an event-based installation incorporating vision, AI, and robotics for the automatic creation of a mosaic
from recycled fragments of varying shapes and colors. The installation is intended for a public exhibition, evoking both the Roman origins of Martigny (Martigny-la-Romaine, Octodure) and showcasing the advancements of AI and robotics in creative and artistic domains, employing both physical (robots) and computational AI (vision, image processing, automatic shape analysis and classification, and geometric optimization of placements).
Widely used during Roman Antiquity, mosaic is a decorative art form in which fragments of stone, enamel, glass, or ceramic are used to
create patterns or figures by judiciously and artistically assembling and orienting the fragments according to the essential elements and contours of the figure to be represented. Our project proposes to revisit this art of the fragment from ancient Rome, adapting it to new AI and robotics technologies to automatically sort, analyze, and classify the available fragments, and plan their use, placement, and orientation within the artwork to be created. These fragments, which can be of various shapes (in addition to the variety of possible colors and orientations), will be automatically assembled to represent a desired image, in consultation with the city of Martigny, defined according to their wishes and to the targeted event (for example, a tribute to Léonard Gianadda, a canvas representing Le Château de la Bâtiaz, the Barry Foundation of the Grand-St-Bernard or a design corresponding to the theme of an event organised by the city such as the carnival or the Foire du Valais, etc).