Idiap online recruitment system

Open Position

Open position's detailed data


Type: PhD Student
Name (ID): Multimodal gesture, visual attention an interaction activity recognition for autism diagnosis (OP-20210811-170115)
Supervised by: Jean-Marc ODOBEZ


The Perception and Activity Understanding group (Jean-Marc Odobez, seeks one highly motivated PhD candidate to work within the AI4Autism project aiming at improving the digital phenotyping of children with Autistic Spectrum Disorders (ASD). The PhD candidate will work on the multimodal perception of small children involved in free play activities as well as their social interactions with adults. In particular, he will investigate deep learning methods and models for the recognition of gestures and visual attention events, including the modeling of their coordination, from visual data and IoT sensors. Experiments will be conducted on various project data (e.g. data coming from standard ADOS evaluation protocol of more than 300 toddlers with partial behavior annotations) as well as standard datasets from the computer vision and multimodal domains (for gesture recognition, attention).

The ideal PhD candidate should hold a MS degree in computer science, engineering, physics or applied mathematics. S/he should have a good background in statistics, linear algebra, signal processing and programming, machine learning. Experience in computer vision and deep learning are definitely a plus. The successful applicant will have good analytical skills, written and oral communication skills.


About the AI4Autism project and the PhD position.

AI4Autism is a sinergia project funded by the SNSF and involving the University of Geneva (Marie Schaer, as well as Thomas Maillart), the Human behavioral analysis research unit at the DTI department of the University of Applied Sciences and Arts of Southern Switzerland (SUPSI) (Michela Papandrea), and the Perception and Activity Understanding group (Jean-Marc Odobez, at the Idiap Research Institute (

Project description: Nowadays, 1 in 59 children diagnosed with autism spectrum disorders (ASD), which makes this condition one of the most prevalent neurodevelopmental disorders. The AI4Autism project is grounded on the recognition that, on the one hand, early diagnosis at scale of autism in young children requires the development of tools for digital phenotyping and automated screening, through digital computer vision and Internet of Things sensing. On the other hand, it aims to examine the potential of digital sensing to provide automated measures of the extended and more fine-grained autism phenotypes. To address these two questions, the project proposes an interdisciplinary project combining the skills of experts in clinical research, engineering and computational social sciences to develop precise and scaled approaches for autism screening and profiling, by investigating three following research directions which are critical to move beyond the state-of-the-art. (1) Clinical research in autism: we propose a comprehensive and reproducible research approach designed to propose groundbreaking digital tools for screening and automated profiling of autism phenotype. It relies on the investigation of both a structured and well established protocol and a less structured one (free play) which may scale better. (2) Internet of Things (IoT): exploring the hypothesis that some ASD phenotypes might be related to the motor skills of very young children, we will explore the use IoT sensors for ASD diagnosis. (3) Computational perception and machine learning. The project will be rooted in modern AI, investigating novel machine learning and computer vision techniques leveraging the availability of large behavioral and clinical annotation data to propose novel behavioral cues extraction models working in challenging sensing conditions.

PhD position: the Phd student will join a team of one PhD student and a postdoc at Idiap working on the project. He will work with them and study methods and models (domain adaptation, unsupervised or weakly supervised learning; temporal graph neural networks, attention-based neural networks and transformers) for the analysis of motor (recognizing gestures) and gaze coordination patterns which are at the core of ASD based on computer vision and IoT sensors, and investigate multimodal interaction deep-learning techniques for ASD diagnosis and profiling with a focus towards interpretable models

About Idiap:

Idiap is an independent, not-for-profit, research institute recognized and funded by the Swiss Federal Government, the State of Valais, and the City of Martigny, and is academically affiliated with EPFL and University of Geneva.

Idiap offers competitive salaries and conditions at all levels in a young, high-quality, dynamic, and multicultural environment. Idiap is an equal opportunity employer and is actively involved in the "Advancement of Women in Science" European initiative. The Institute seeks to maintain a principle of open competition (on the basis of merit) to appoint the best candidate, provides equal opportunity for all candidates, and equally encourage both genders to apply. Employment at Idiap is thus based solely on a person's merit and qualifications directly related to professional competence. Idiap does not discriminate against any employee or applicant because of race, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, marital status, pregnancy, or any other basis protected by law.

Idiap is located in the town of Martigny in Valais, a scenic region in the south of Switzerland, surrounded by the highest mountains of Europe, and offering exceptional quality of life, exciting recreational activities, including hiking, climbing and skiing, as well as varied cultural activities. It is within close proximity to Lausanne and Geneva. Although Idiap is located in the French part of Switzerland, English is the official working language. Free French lessons are also provided on a complimentary basis.

For frequently asked questions (FAQs) about living in Switzerland, please go to

By continuing your browsing on this site, you agree to the use of cookies to improve your user experience and to make statistics of visits. For any questions regarding data protection or the use of cookies, please refer to the following webpage:

 Show display preferences 


Fill-in and submit your application through Idiap online recruitment system
Request further information about this position