Robot Learning & Interaction

Au cours des vingt dernières années, les robots ont progressivement quitté les environnements industriels classiques pour entrer dans les espaces de vie et de travail des humains. De nouveaux robots ont ainsi été conçus spécifiquement pour interagir avec des personnes, ce qui implique des capacités avancées de manipulation, de déplacement autonome, de programmation intuitive, et d’interaction sociale afin d’être réellement utiles et acceptés par leurs utilisateurs. Ces évolutions ont donné naissance à la robotique d’assistance, un domaine de recherche qui étudie comment les robots peuvent soutenir les humains dans des contextes variés, allant de l’aide aux personnes âgées ou en situation de handicap jusqu’au soutien des travailleurs en entreprise. Toutefois, cette recherche pose des défis importants, notamment l’implication de patients et d’utilisateurs réels dans la conception des technologies, ainsi que la création d’environnements de développement et d’évaluation qui reproduisent fidèlement les conditions réelles de vie et d’activité. Pour renforcer le réalisme et l’impact de ses futurs projets, l’Idiap souhaite développer un nouveau Centre de Robotique d’Assistance, destiné à centraliser les recherches en robotique de l’Idiap, à recréer des conditions d’usage proches du réel, à accueillir des participants locaux et à mieux communiquer les avancées scientifiques auprès du public.

Recent advances in anthropomorphic and multi-limbs robots have led to an increased interest in exploiting these platforms for human-robot collaboration (HRC). HRC is challenging in multiple aspects, covering perception, learning, control and planning levels. The underlying problems share as common ground that they need some form of distances to be measured, which are used as some form of cost functions that need to be minimized. Distance measurement is thus an essential common ingredient at the core of robotics. For control, these distances are typically used in the form of residuals that are corrected by a feedback controller. For planning, trajectories minimizing the total distance between two points correspond to geodesic paths (for Euclidean distances, these paths correspond to straight segments between the two points). By extension, these distances are also the core component to model diffusion processes and kernel function to model (co)variations and uncertainty.

In HORACE, we will treat these distances from a geometric perspective, which boils down to the selection of efficient representations and manifolds to specify these distances. The goal is to measure distances by going beyond Euclidean distances between two points, but to instead consider distances between other geometric objects and to account for manifolds that are not Euclidean, which is often the case in robotics. We will then exploit these distance measures in the form of kernel functions that can be used in Gaussian processes to account for variations and uncertainty.

By leveraging our previous expertise, we will demonstrate the developed approaches in several human-robot collaboration scenarios involving joint physical collaboration, assembly, manipulation of deformable objects, all with high degree of freedom robots, in particular, the TALOS humanoid robot readily available at JSI. We will investigate how the control of a humanoid robot can be effectively achieved by using the proposed geometric descriptors and how the geodesics in such encoding can be used for effective planning.

HORACE will lead to novel methodological developments yielding more efficient human-robot collaboration and better understanding of its inherent geometrical structure. The three partners each bring knowledge and equipment that complement the others. Idiap brings knowledge on the geometric perspectives, UNIZG-FER on planning and HRC and JSI on humanoid robots and HRC. The complementarity of the partners will enable concrete advancements in the field of human-robot collaboration.

Recent advances in generative AI and robotics have revolutionized the connection that people have with creativity and art. We propose an installation to shed light on the roles of AI and robotics in creative endeavors, aiming to generate bilateral discussions between the public and the scientific researchers. On the one hand, with new AI tools such as Midjourney or Dall-E, anyone can quickly create visual artistic renderings with just a few words. On the other hand, robot manipulators are becoming increasingly accessible to non-specialized users, with user-friendly programming interfaces (no-code training of robots). These revolutions raise questions on the roles of AI and robotics in artistic and creative applications. Still, although AI and robotics are often presented jointly, the underlying challenges remain very different. To highlight this, our installation will combine both images generation and real robots, showing the additional challenges that it brings to control a robot to move in a physical world to reproduce a series of gestures to draw on a paper canvas. Two different facets of AI will thus be showcased: the digital/virtual world and the physical/real world. With this Agora project, we would like to initiate a discussion between scientists and the public about the role of science in the arts, and what modern AI and robotics technology can bring to different forms or arts, either digital or physical. This installation will consist of two parts, a first one where visitors can interact with a digital AI creation tool using their own body, and a second one where a robot will draw portraits of visitors. These two sides of the installation will be complemented with explanation describing how the digital art and the robot movements are generated, highlighting the technological progress behind recent widely available tools and what is the technology that can allow robots to perceive the world, interpret it and draw.
Idiap, a research center of national importance dedicated to AI research in Switzerland, will team up with two project partners: Phänomena and Skyentific. Phänomena is a unique event in the form of a spectacular and sensual world of experience on the topics of humans, nature and science. At the core of Phänomena are around 20 key experiences developed innovatively with partners, distinct for their impact on the expected 1 million visitors. Gennady Plyushchev (Skyentific) is a science communicator and robotics expert.
This project will be connecting the visitors’ representations and understanding of AI and robotics with research. First, our installation will enhance the understanding of science, by explaining and demonstrating the opportunities and limitations of current technologies, both in the digital realm and the physical one. Second, the installation will be highly interactive to serve as a dynamic and satisfying educational tool. Finally, through our multiple community engagement activities, we will foster stronger/in-depth discussions between the scientific community (both researchers and PhD students) and the public.

Oka theory is a very geometric part of research in the area of Several Complex Variables. It is both a very traditional and a very active area of research. In the new 2020 AMS classifiation the new classification number 32Q56 "Oka principle and Oka manifolds" was introduced, witnessing about the importance of this field in modern Mathematics and its active development.

In this project we will consider naturally given natural mathematical problems and work out whether for their solution a certain Oka principle can be applied, similar as we did in our solution to the Gromov-Vaserstein problem. On one hand these are applications to K-theory of rings, where we always work with concrete examples of rings, namely rings of holomorphic functions on Stein spaces. We also will expand our studies to rings of quaternionic holomorphic functions, an area which has been successfully developing in the last years and gives a wealth of wonderful examples of non-commutative rings of very natural origin to be studied in the realm of K-theory.

On the other hand we will try to apply our methods developed for holomorphic and continuous matrix factorization to problems in theoretical robotics. A natural question is the localization and description of singularities of a kinematic chain depending on the joint variables. Another crucial problem is the transformation of the movement of the final tip of a robot into a movement of the robot’s joints, together with continuous dependence on the movement. We try to investigate under which conditions a continuous/smooth path described inside the workspace by the end manipulator can be achieved by a continuous/smooth movement of the joints. We also plan to develop a complexified version of the kinematic chain and prove approximation by holomorphic or partially holomorphic (Cauchy Riemann-) movements. Similar PDE-techniques as used in the D-bar problem could lead to find energy minimizing movements of the robot’s joints.

CR geometry studies the geometry of real manifolds in connection with a complex structure. An important situation when CR geometry comes into play is when one considers smoothly bounded domains in complex spaces. For example, if a Stein space is realized as such, then the boundary of the domain must be pseudoconvex. In general, many CR geometric properties of a boundary imply interesting properties of the domain it bounds.

In this project, we consider problems in CR geometry of a boundary having implications to the analysis and geometry of the domain. We shall work with weakly pseudoconvex boundaries and seek mild conditions them which implies, e.g., the regularity of the d-bar problems on the domain. In the strictly pseudoconvex case, there are finer boundary invariants that are important for geometry of the domain. 

Obstruction tensor, for example, is a boundary invariant that control the local extendability to the boundary of Kahler-Einstein potential of the Cheng-Yau metric. It has been studied recently with several important results, yet there are still interesting open problems need to be resolved.

The D-bar problem is one of the most intensively studied PDEs in several complex variables.

Different from classical PDEs — the most tractable problems are elliptic — the D-bar problem is a non-elliptic problem. However, the pioneering work of J. Kohn, L.

Construire un démonstrateur du robot Baxter préparant un café, dans le but de solliciter des financements pour réaliser des projets plus poussés avec les services Marketing de Nespresso

This project proposes the development of a new automation solution for the electroplating jig loading. The strategy consists of using recent developments in robotics and industrial vision for jigs detection and adaptive planning and control for dexterous manipulation.

The Swiss economy is known for its productivity, precision and expertise. However the comparatively igh wages make it difficult for companies to stay worldwide competitive and avoid offshoring. The digitalization of work has the potential to help sustain and further improve the Swiss competitive advantages as well as strengthen their position in a global market. However, the implications of digitalization for work are not fully explored and give rise to contestation with respect to how labour in the future will look like and what competencies are needed in order to empower workers to take advantage of and participate in the digital transformation of work. This project will examine these questions by addressing factors that support the implementation and acceptance of new technologies with a particular focus on flexible automation and human-machine interactions. The project’s objectives are: a) to provide guidelines as to which digital skills are required by workers to make interactions with collaborative robotics empowering; and b) to provide insights into the necessary technical and educational tools for this empowerment to take place. Empirically, the project will employ a mixed-method approach consisting of interviews, non-participant observations and survey questionnaires as well as prototype testing. Besides guidelines for a successful implementation, the project will also deliver a platform for intuitive human-machine interaction that will provide a basis to acquire the necessary skills. To achieve these aims, the project is set up interdisciplinarily, combining expertise from computer sciences and engineering with social sciences. Building on research from those two strands, the project will make a significant and practical contribution to the advancement of a future of work that is not only more digital but also more humane.

Traditional manufacturing systems lack the necessary flexibility and reconfigurability that can allow short production cycles and fast deployment of the updated system. Although the use of automation technologies based on industrial robots can increase the adaptability of a production line, the desired flexibility cannot be achieved until abilities for genuine collaboration of the robots with the human workers are developed. CoLLaboratE will revolutionize the way industrial robots learn to cooperate with human workers for performing new manufacturing tasks, with a special focus on the challenging area of assembly operations. The envisioned system for collaborative assembly will be capable of allocating human and robotic resources for executing the production plan sharing the tasks according to the capabilities of the available actors. The CoLLaboratE project will build upon state-of-the-art methods for teaching the robot assembly tasks using human demonstration, extending them to facilitate genuine human-robot collaboration. To this end, a framework for equipping the robots and AGV mobile platforms with basic collaboration skills, such as load sharing, human touch recognition and human intention detection, will also be developed, coupled with deep reinforcement learning algorithms for increasing adaptability. Special attention will be paid to providing effective safety strategies allowing the use of a fenceless approach within the production cell. As a result, closer collaboration will be achievable and efficient production plans making optimal use of the available resources will be designed and executed. The proposed solution will be evaluated in four different pilot sites, which will be implemented as collaborative factory floors of the industrial partners in Italy, Slovenia, Turkey, and Poland.