UBImpressed: Ubiquitous First Impressions and Ubiquitous Awareness

First impressions matter. Brief interpersonal interactions with people we do not know occur everywhere and often, and the evaluations we make of others (and that others make of us) can have profound effects. When we meet somebody for the first time, we form an impression very quickly based on their nonverbal behavior (tone of voice, speech modulation, speech duration, gaze, facial expressions, body movements and postures) and the spoken words, and this impression in turn guides our behavior.

Specifically, (first) impressions in the workplace can affect outcomes like being hired or promoted, and positive impressions are critical in entire sectors of the economy, including sales, service, and hospitality, where employees are frequently evaluated, rewarded, and advanced in their careers based on the impressions they convey. Customers judge quality of service based on their impressions about the service provider -- how friendly or competent or helpful. Research in organizational psychology and nonverbal communication has revealed some of the connections existing between nonverbal behavior and impressions in the workplace for some years, including links between immediacy behavior and hiring decisions or ratings of supervisors. However, and despite the fact that impressions at work are ubiquitous, much of the existing research in this domain has largely been done in the laboratory, for single organizational situations, and based on single interactions. Furthermore, one of the fundamental goals of organizational behavior research -- how to make these findings useful for training and improvement of skills by employees -- has often been disconnected from laboratory experiments given the lack of methods and tools to systematically study favorable first impressions and other related variables in the field, with multiple job-related situations, over time, and embedded with the training process.

This is what UBImpressed aims to achieve through a multidisciplinary research approach involving academics in Work and Organizational Psychology, Multimodal Signal Processing, Ubiquitous Computing, and Social Computing. The project innovatively integrates nonverbal communication research with mobile sensing, perceptual computing, machine learning, and mobile visualization applications to understand (1) what nonverbal behaviors are related to conveying favorable first impressions in different domains within the organizational context; (2) which of these impression-related behavioral cues can be robustly extracted and analyzed by automatic means over multiple physical settings; and (3) how favorable first impressions in the workplace can be trained by integrating expert knowledge with automatic sensing, analysis, and visualization technology. The project will advance the state of the art regarding nonverbal communication and organizational psychology and automated analysis of human behavior, and brings about new research directions that have the potential to impact jointly in psychology and computer science. These include: the partnership with a real organization in the hospitality industry for whose members favorable first impressions are critical for success; the ecologically valid analysis of first impressions in a population for an extended period of time and under psychological and computational lenses, both in the laboratory and the field; the investigation of online social video and video crowdsourcing techniques to generate large-scale first-impression data, whose analysis could be transferred to the physical world; the design of new machine perception and machine learning methodologies to extract behavioral features, discover behavioral patterns in individuals, and make automatic predictions about first impressions; the embedding of data-driven machine analysis and human expertise in training into real applications to be used by people to improve their skills; and the generation of research resources that have the potential of being widely reused by the research community.

Application Area - Human Machine Interaction, Application Area - Management of mobile systems
Cornell University
Swiss National Science Foundation
Jan 01, 2014
Nov 30, 2017