Human-Robot Interfaces for Interactive Robot Programming

For robots to be widely adopted across industries and beyond structured manufacturing environments,it is critical for them to be programmable by a wide range of users. Growing research on End User Programming (EUP) for robotics aims to address this problem with novel user interfaces, programming languages, and techniques to aid or fully automate robot programming.

In this project, you will design a Human-Robot Interface for Robot Programming, integrating an existing programming framework based on iterative Linear Quadratic Regulator (iLQR) [1] on a robotic manipulator, the FRANKA EMIKA Panda. The interface will allow users to compose robot programs defined as sequences of components like goal poses or constraints (e.g., maintaining a gripper orientation)which, in turn, inform the generation of executable robot trajectories.

More specifically, in a first phase, you will build the basic interface allowing the the user to interact with the robot. A Graphic User Interface (GUI) will be developed to specify program components, taking into account the Panda’s capability to be effortlessly displaced (i.e., moved in gravity compensation),and allowing the user to define via-points in different frames of reference. The GUI will also guide the user in the addition of other robot actions that cannot be easily demonstrated, like actuating grippers, or detecting objects in the environment. Finally, a simple visualization interface will be integrated, allowing the user to inspect the current program, to visualize trajectories, and hence test them on new scenarios.

In a second phase, you will study and investigate further interaction schemes for the user to refine and modify the current program. For instance, new interfaces for the user to add constraints on the program could be devised (similar to [2]), such as maintaining a certain orientation or being more precise when reaching a pose. Other modalities could allow the user to stop a robot execution to add new via-points,or provide partial demonstrations (as done in [3]), with or without the use of kinesthetic teaching (for example, by letting users show a motion or define via-points with their body or gestures). This second phase is more exploratory in nature, and therefore leaves you with more freedom to experiment a specific programming interface or modality.

How to apply:

Interested candidates are invited to submit a cover letter, a detailed CV, and the names of three references through the Idiap online recruitment system:

Human-Robot Interfaces for Interactive Robot Programming

More information and details about the project can be found here