MULTImodal Interaction and MULTImedia Data Mining

As a unified research theme, the goal of the MULTI project is to carry out fundamental research in the field of multimodal interaction, mainly covering a wide range of critical activities and applications, including recognition and interpretation of spoken, written and gestural language, recognition and interpretation of visual scenes, and processing and indexing of multi-channel (e.g., audio-visual) data streams. Other key sub-themes of MULTI include the control of information access, typically through biometric user authentication techniques (including speaker and face verification). Mainly exploiting related technologies of statistical modeling and data mining, MULTI also investigates advanced approaches towards the structuring, retrieval and presentation of multimedia information (e.g., resulting of multimodal meeting or lecture recordings), which also represents a wide-ranging and important research area that includes not only the multimodal interaction aspects, but also multimedia document analysis, indexing, and information retrieval, thus involving complex computer vision and data fusion algorithms. The research areas addressed in MULTI thus cover 6 complementary research themes, namely: (1) machine learning, (2) speech and audio processing, (3) computer vision, (4) information retrieval, (5) biometric authentication, and (6) multimodal interaction.

Among other applications, the research in MULTI often focuses on multimodal technologies to support human interaction, in the context of smart meeting rooms. In that specific context, MULTI aims at performing fundamental research towards the development of new audio, visual and multimodal tools for understanding, searching and browsing meetings data captured from a wide range of devices, as part of an integrated multimodal group communication.

See SubProject Website: MAM