Understanding human behavior is one of the most intriguing and fascinating research domains, which encompasses several research fields, ranging from economics and sociology to more recently computer science. Immense progress in sensor and communication technologies has led to the development of devices and systems recording daily human activities in both real and virtual (web-based) settings. This has led to an increase of research on the design of algorithms capable of inferring meaningful behavioral patterns of human activities from the information contained in data logs or captured by sensors. Simultaneously, there are new application opportunities in many domains such as surveillance, health care monitoring, social networking, and recommendation systems.
The aim of the HAI project is to investigate the above domain by performing long-term research which addresses fundamental questions and common tasks of this domain: how to design robust features for accurate activity/interaction representation? How to learn or discover activity patterns, introduce hierarchies or temporal order at different scales, and deal efficiently with large amounts of data? How to infer contextual information that affects activity patterns or their occurrence, or facilitates their interpretation. To achieve these goals, we will investigate new approaches by anchoring the design of general activity models in the context of four different and relatively recent application domains with specific scenarios, types of activities, and data modalities.
HAI-1: Activity analysis from long term video recordings. The goal is to automatically discover the typical activities of moving entities (cars, people, groups), their characteristics, the relations between them within and across cameras, and detect abnormal activities. These goals will be reached by combining sequential state models at different levels with data mining tools relying on co-occurrence analysis.
HAI-2: Activity analysis from mobile phone data. The aim is the design of novel heterogenous data representations and probabilistic models for the modeling of varying time duration routines for location and proximity based activity discovery, for the identification of large scale human communication patterns based on phone calls or text messaging, and for the discovery of individual's life patterns from a rich set of phone data modalities.
HAI-3: Community activity analysis in social media. The goal is to investigate the structure, evolution, and practices of communities in social media with Flick as a target. In particular, using statistical models relying on textual and social metadata, the project will model the dynamical aspect of social media groups (including topic and memberships patterns), study and discover micro-activity patterns within sub-groups, and investigate the use of visual information extracted from photos and videos to refine user and group descriptions.
HAI-4: Context modeling for just-in-time multimedia information retrieval. The goal is to model the context of conversational activity for users of an Automatic Content Linking Device (ACLD), which is a just-in-time retrieval system that spontaneously displays documents and multimedia fragments based on the current topic of discussion. Context modeling will help to decide whether it is useful and appropriate to interrupt users with new results, and to estimate relevance feedback from users in order to guide upcoming searches.
While each of the sub-projects pursues its own goals, the grounding of the approaches on similar principled methodologies (e.g. bag-of-words and Bayesian topic models) will provide opportunities for research synergies and the strengthening of Idiap's activities on human behavior and interaction modeling.