Perception & Activity Understanding

The research group on the understanding of perception and activity conducts research analyses of human activities from multimodal data.

The analysis of human activities from multimodal data is useful for surveillance, behavior analysis, human–robot interfaces, and multimedia content analysis. This includes investigating the fundamental tasks of scene analysis such as detection, segmentation and tracking of people, their representation, and the characterization of their condition, as well as the modeling of sequential data and their interpretation in the form of gestures, activities, behavior, or social relationships, through the design of sound algorithms which exploit and extend models and methods of computer vision, machine learning, and multimodal data-fusion. Surveillance, traffic analysis, analysis of behavior, human-robot interfaces, and multimedia content analysis are the main application domains.

Group News

Idiap and Octopeek sign a 4-year R&D partnership
tech — Jun 17, 2021

Artificial intelligence and big data pioneers Octopeek (France) and Idiap announce a partnership. A member of Octopeek’s scientific staff will spend four years at the very heart of Idiap, culminating in a PhD—a unique opportunity to use video data to develop research on multimodal learning, and to spur innovation.

Swiss algorithms to help customers in a Finnish shopping mall
research — Sep 08, 2020

Human-robot interactions are often lacking fluidity, especially outside of the lab. Today, researchers from Idiap are publishing in open access the algorithms which allowed a robot to be used in real conditions in a shopping mall in Finland in the framework of the European project Mummer.

Group Job Openings

Multimodal gesture, visual attention an interaction activity recognition for autism diagnosis by admin — last modified Aug 12, 2021
The Perception and Activity Understanding group seeks one highly motivated PhD candidate to work within the AI4Autism project aiming at improving the digital phenotyping of children with Autistic Spectrum Disorders (ASD).
Human-Robot Interfaces for Interactive Robot Programming by admin — last modified Jul 20, 2021
For robots to be widely adopted across industries and beyond structured manufacturing environments,it is critical for them to be programmable by a wide range of users. Growing research on End User Programming (EUP) for robotics aims to address this problem with novel user interfaces, programming languages, and techniques to aid or fully automate robot programming.
Several Openings for Cross-Disciplinary Senior Researcher positions by admin — last modified Mar 01, 2021
With the growth of the institute, in addition to increased Federal funding to further support its activities, Idiap is opening several additional permanent senior research scientist positions.

Current Group Members

The group is led Jean-Marc Odobez.

ODOBEZ, Jean-Marc
(Senior Research Scientist with Academic Title)
- website

VILLAMIZAR, Michael (Alejandro)
(Research Associate)
- website

RACCA, Mattia
(Postdoctoral Researcher)
- website

(Postdoctoral Researcher)
- website

SOUSA EWERTON, Marco (Antonio)
(Postdoctoral Researcher)
- website

GUPTA, Anshul
(Research Assistant)
- website

(Research Assistant)
- website

- website


  • AMINIAN, Bozorgmehr (Nima)
  • BA, Silèye
  • CAN, Gulcan
  • CAO, Yuanzhouhan
  • CHEN, Cheng
  • CHEN, Yiqiang
  • DESPRÈS, Nicolas
  • DUVAL, Matthieu
  • EMONET, Rémi
  • FUNES MORA, Kenneth Alberto
  • GAY, Paul
  • HE, Weipeng
  • HEILI, Alexandre
  • HU, Rui
  • KHALIDOV, Vasil
  • LE, Nam
  • LEFÈVRE, Stéphanie
  • LIU, Gang
  • LOPEZ-MENDEZ, Adolfo
  • MOSTAANI, Zohreh
  • RICCI, Elisa
  • ROMAN RANGEL, Edgar Francisco
  • SHEIKHI, Samira
  • TAVENARD, Romain
  • VARADARAJAN, Jagannadan
  • WU, Di
  • YAO, Jian
  • YU, Yu

Active Research Grants

Past Research Grants