odobez
 

Current projects

  • MuMMER - MultiModal Mall Entertainment Robot (website)
  • Role : Principal Investigator
    Funding agency : H2020, RIA
    Dates : 2016 - 2020
    Partners: University of Glasgow (UK), Idiap Research Institute (CH), Aldebaran Robotics (F), Heriot-Watt University (UK), LAAS-CNRS (F), VTT Technical Research Centre (Finland), Ideapark (Finland).
    Description : The overall goal is the development of a humanoid robot (based on Aldebaran's Pepper platform) that can interact autonomously and naturally in the dynamic environments of a public shopping mall, providing an engaging and entertaining experience to the general public. Using co-design methods, we will work together with stakeholders including customers, retailers, and business managers to develop truly engaging robot behaviours. Crucially, our robot will exhibit behaviour that is socially appropriate: combining speech-based interaction with non-verbal communication and human-aware navigation. To support this behaviour, we will develop and integrate new methods from audiovisual scene processing, social-signal processing, high-level action selection, and human-aware robot navigation.


  • EUMSSI - Event Understanding through Multimodal Social Stream. (website)
  • Role : Principal Investigator
    Funding agency : European community (FP 7-ICT, STREP)
    Dates : 2014-2016
    Partners: University Pompeu Fabra (SP), Laboratoire d'Informatique de l'University du Mans (LIUM,F), GFAI (D), Leibniz University Hannover (D), Deutsche Welle (D), Video Stream Network (SP).
    Description : Development of identification and aggregation technologies of unstructured information sources of very different nature (video, image, audio, speech, text and social context), including both online (e.g., YouTube) and traditional media (e.g. audiovisual repositories). The multimodal analytics will help organize, classify and cluster cross-media streams, by enriching its associated metadata in a cross-modal interoperable semantic representation framework. The resulting platform will be exploited for intelligent content management systems and more specifically for helping journalists writing articles.


  • UBImpressed - Ubiquitous First Impressions and Ubiquitous Awareness (website)
  • Role : Co-Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2014-2017
    Partners: University of Lausanne (CH), Cornell University (USA)


Past projects

  • G3E - Geometric Generative Gaze Estimation (website)
  • Role : Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2014-2015
    Partners: Idiap Research Institute


  • VANAHEIM - Video/Audio Networked Surveillance System Enhancement through Human-Centered Adaptive Monitoring (website)
  • Role : Principal Investigator
    Funding agency : European community (FP 7-ICT, IP)
    Dates : 2010 - 2013
    Partners: Multitel (B), Idiap Research Institute (CH), Vienna University (AUT), INRIA-Pulsar (F), Thales TCF (F), Thales Italia (I), GTT (I), RATP (France).
    Description : The goal is to develop autonomous computer vision and learning algorithms contributing to the automatic selection of camera views displayed in the surveillance control room, the extraction and use of non-verbal cues for social human behavior understanding in surveillance scenes, and the online building of collective behavior models from long term recordings.


  • HUMAVIPS - Humanoids with auditory and visual abilities in populated spaces. (website)
  • Role : Principal Investigator
    Funding agency : European community (FP 7-ICT, STREP)
    Dates : 2010-2013
    Partners: INRIA-Perception (F), Idiap Research Institute (CH), Czech Technical University (CZ), Bielefeld University (D), Aldebaran Robotics (F).
    Description : Endow humanoid robots with audio-visual perception abilities (exploration, recognition, interactions) allowing it to adopt an autonomous and welladapted social behavior in the presence of a group of people.


  • SONVB - Sensing and Analyzing Organizational Nonverbal Behavior (website)
  • Role : Co-Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2010-2013
    Partners: University of Neuch√Ętel (CH), Dartmouth College (USA)


  • SODA - perSon recOgnition in Debates and broAdcast news.
  • Role : Principal Investigator
    Funding agency : ANR (Agence Nationale pour la Recherche, France)
    Dates : 2011-2014
    Partners: Laboratoire d'Informatique de l'University du Mans (LIUM), Idiap Research Institute
    Description : automatic audio-visual person clustering and recognition


  • TRACOME - Robust face tracking, feature extraction and multimodal fusion for audiovisual speech recognition and visual attention modeling in complex environment (website)
  • Role : Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2010-2014
    Partners: EPFL-LTS5, Idiap Research Institute


  • ASLEEP -
  • Role : Principal Investigator
    Funding agency : Thales Italia
    Dates : 2012-2013
    Description : Developement and evaluation of static luggage detection module.


  • HAI - Human Activity and Interactivity modeling (website)
  • Role : Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2010-2014
    Description : investigate unsupervised data mining techniques relying on Bayesian topic models for activity and interactivity mining in three dierent application settings (video surveillance, cell-phones, social networks).


  • PROMOVAR - Probabilistic Motifs for Action Recognition
  • Role : Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2012-2013


  • CODICES - Automatic Analysis of Mexican Codex Collections (website)
  • Role : Co-Principal Investigator
    Funding agency : SNF (Swiss National Fund for scientific research)
    Dates : 2008-2012
    Partners: Mexico's National Institute of Anthropology and History (INAH)


  • CARETAKER - Content Analysis and Retrieval Technologies to Apply Knowledge Extraction to massive Recording (website)
  • Role : Principal Investigator
    Funding agency : European community (IST FP6-027231, STREP)
    Dates : 2006 - 2008
    Partners: Partners: Thales TCF (F), Idiap research institute (CH), INRIA-Orion (F), Multitel (B), Kingston University (GB), BUT (CZ), Solid/IBM (Fin), ATAC (I), GTT (I).
    Description : study, develop and evaluate content extraction algorithms applied to large collections of multimedia data captured from networks of camera and microphones in real sites (metro stations). Idiap was in charge of the development of robust and efficient algorithms for multi-camera multi-person tracking, the recognition of events (left-luggage, queue detection), and long term data mining (several camera recordings over 15 days).