Social / human behavior
This research area is concerned with the automatic analysis of a variety of real-life human behaviours. This activity emerged as a natural extension of our previous (and still ongoing) work in the processing, modeling and modeling of human-to-human interactions in meetings, where we realized the importance of non-verbal cues, impact of social sciences, and large potential in many other disciplines.
The resulting work in human and social behavior analysis, modeling, and understanding exploits expertise and synergies between key areas at Idiap, including multi-sensor human behaviour capture, machine learning, and perceptual processing.
Social media
The understanding of social media - a constellation of web socio-technical systems rooted on social interaction through media creation, exchange, critique, and repurposing - is a new goal in several branches of science and the humanities. We are developing computational models to analyze and enrich social media, which are able to integrate aspects related to human motivations, social relations, and content. Current research problems include multimedia data fusion (e.g. images and tags) for characterization of users and groups in Flickr, and community discovery in Flickr and YouTube.
Projects: MULTI
Contact:Daniel Gatica-Perez
Mobile media
The automatic analysis of real-life, long-term behavior and dynamics of individuals and groups from mobile sensor data constitutes an emerging area in computing. Mobile phones have rapidly become the ultimate sensor, providing access to human physical location and motion, content, and social context. We are researching this domain from sensing to applications, designing computational frameworks that, integrating multiple sources of physical and social evidence from large-scale mobile data, are able to recognize typical routines of individuals, discover social trends emerging from group interaction, and identify groups and communities and their defining attributes.
Projects: LS-CONTEXT
Contact:
Social signal processing
Social Signal Processing is the domain aimed at automatic understanding of social interactions through analysis of nonverbal behavior. The core idea of SSP is that nonverbal behavior is the machine detectable evidence of social signals, the relational attitudes exchanged between interacting individuals. Social signals include (dis-)agreement, empathy, hostility, and any other attitude towards others that cannot be expressed using just words. Current research directions include the analysis of dynamic behavioral cues in multiple modalities, the synthesis of social signals in artificial agents, and the modeling of social behavior in face-to-face interactions.
Projects: SSPNET, CCPP
Contact:
Verbal and nonverbal communication analysis
Verbal and Nonverbal Communication Analysis Human-Human interaction is the exchange of meaning through both verbal and non-verbal communication. The former is used consciously and aims at building a shared understanding between interacting individuals. The latter is used mostly unconsciously and helps to define the meaning of words while giving off information about state and attitudes of people. Current research directions include lexical analysis for role and personality analysis, synthesis of expressive speech, automatic assessment of communication effectiveness.
Projects: SSPNET, CCPP, AMIDA, IM2 and NOVICOM
Contact:
Alessandro Vinciarelli 