Sensing and Analyzing Organizational Nonverbal Behavior

The understanding of organizational behavior has been a fundamental goal in social sciences in recent years. Face-to-face communication remains the primary form of social interaction in the workplace, and thefore nonverbal communication plays a crucial role. While spoken language constitutes a strong communication channel, a wealth of information is conveyed nonverbally through prosody, proximity, body gestures and postures, gaze, and facial expressions. Nonverbal behavior (NVB) has different functions in social interactions. It not just serves to express emotions but also to signal attention, convey attitudes, and reveal personality characteristics. Furthermore, the establishment and evolution of multiple organizational constructs at the level of individuals and groups are the often measurable outcomes of face-to-face conversations with peers, superiors, and subordinates, where nonverbal communicative processes intervene through the display and interpretation of NVB.

This interdisciplinary project aims to advance the state-of-the-art in perceptual computing, ubiquitous computing, and social psychology for the analysis of NVB of individuals and groups in organizational scenarios. We aim to address three key interrelated aspects of behavior in organizations, namely leadership, personality, and performance. The project takes a broad view on the subject in the context of dyadic and group interaction both in the laboratory and in real life. In social science, we aim to advance theories and empirical research on the interplay between first impressions, NVB, performance, and personality in job interviews and job execution; and between personality traits and exhibited NVB of members of small groups. In computer science, we aim to develop new computational methods for automatic extraction of NVB from face-to-face interaction from audio, video, and mobile sensors, and to design new computational models for recognition and discovery of social constructs. Finally, the project provides a timely opportunity to train young scientists on the interplay between the human and computational facets of social interaction analysis.

• Dr. Daniel Gatica-Perez, Idiap (Principal Investigator)
• Dr. Jean-Marc Odobez, Idiap Prof. Marianne Schmid Mast, University of Neuchatel (UNINE)
• Prof. Tanzeem Choudhury, Dartmouth College