Partenariat humain-IA

Ce programme de recherche capitalise sur l'expertise bien établie de l'Idiap en matière d'interaction multimodale. Il tire parti de la capacité unique de l'institut à entreprendre des recherches multidisciplinaires approfondies sur la communication verbale et non verbale, le traitement du langage, les systèmes perceptifs et cognitifs et l'interaction homme-robot.

L'objectif de ce programme est d'étendre les capacités humaines sur plusieurs aspects : créativité, limites cognitives, collaboration, connaissance. Ce programme vise à améliorer la détection et la compréhension des activités humaines par les machines, à améliorer l'accès à l'information, par exemple grâce à des chatbots servant d'experts de domaine à la demande, à utiliser le retour humain d'information pour améliorer les systèmes d'apprentissage et à utiliser des robots pour aider les humains dans leurs tâches quotidiennes au travail et à la maison.

Domaines d'expertise

#Bioinformatics&HealthInformatics
#DataScience&SocialComputing
#HumanComputerInteraction
#Imaging&ComputerVision
#MachineLearning
#NaturalLanguageProcessing
#Robotics&AutonomousSystems
#Security&Privacy
#SignalProcessing
#Speech&AudioProcessing

Plus  

Ce programme contribue aux ODD des Nations-Unies suivants


Plus  

Personnes

ABBET, Philip
(Senior Research and Development Engineer)
- website


BEHJATI, Melika
(PhD Student / Research Assistant)
- website


BEN MAHMOUD, Imen
(Research and Development Engineer)
- website


BHATTACHARJEE, Mrinmoy
(Postdoctoral Researcher)
- website


BILALOGLU, Cem
(PhD Student / Research Assistant)
- website


BITTAR, Alexandre
(PhD Student / Research Assistant)
- website


BORNET, Olivier
(Head of Research and Development Team)
- website


BURDISSO, Sergio (Gastón)
(R&D / Research Assistant)


CALINON, Sylvain
(Senior Research Scientist)
- website


CANÉVET, Olivier
(Senior Research and Development Engineer)
- website


CARRON, Daniel
(Research and Development Engineer)
- website


CHANG, Xiaoguang
(Research Intern)
- website


CHEN, Haolin
(PhD Student / Research Assistant)
- website


CHI, Xuemin
(Research Intern)
- website


CLIVAZ, Guillaume
(Senior Research and Development Engineer)
- website


COMAN, Andrei (Catalin)
(PhD Student / Research Assistant)
- website


COPPIETERS DE GIBSON, Louise
(PhD Student / Research Assistant)
- website


COURDIER, Evann
(PhD Student / Research Assistant)
- website


DASTENAVAR, Atharva
(Research Intern)
- website


DAYER, Yannick
(Research and Development Engineer)
- website


DELMAS, Maxime
(Postdoctoral Researcher)
- website


DROZ, William
(Senior Research and Development Engineer)
- website


DUFAU, Maximilien
(Junior R&D / Research Assistant)
- website


EL AYOUCH, Aliyasin
(Research Intern)
- website


EL HAJAL, Karl
(Research Assistant)
- website


EL ZEIN, Dina
(PhD Student / Research Assistant)
- website


FARKHONDEH, Arya
(PhD Student / Research Assistant)
- website


FEHR, Fabio (James)
(PhD Student / Research Assistant)
- website


FLEURET, François
(External Research Fellow)
- website


FREITAS, André
(Research Scientist)
- website


GAIST, Samuel
(Senior Research and Development Engineer)
- website


GARNER, Philip
(Senior Research Scientist)
- website


GENTILHOMME, Théophile
(Senior Research and Development Engineer)
- website


GUPTA, Anshul
(PhD Student / Research Assistant)
- website


HE, Mutian
(PhD Student / Research Assistant)
- website


HENDERSON, James (Brinton)
(Senior Research Scientist)
- website


HERMANN, Enno
(Postdoctoral Researcher)
- website


HOVSEPYAN, Sevada
(Research Associate)
- website


ISMAYILZADA, Mahammad
(PhD Student / Research Assistant)
- website


JANKOWSKI, Julius
(PhD Student / Research Assistant)
- website


JOHARI, Seyed Mohammad Mahdi
(PhD Student / Research Assistant)
- website


JUNG, Vincent
(Research Intern)
- website


KAYAL, Salim
(Senior Research and Development Engineer)
- website


KHALIL, Driss
(Junior R&D / Research Assistant)
- website


KUMAR, Shashi
(PhD Student / Research Assistant)
- website


LI, Yiming
(PhD Student / Research Assistant)
- website


LÖW, Tobias
(PhD Student / Research Assistant)
- website


MACEIRAS, Jérémy
(Research and Development Engineer)
- website


MADIKERI, Srikanth
(Research Associate)
- website


MAGIMAI DOSS, Mathew
(Senior Research Scientist)
- website


MARCEL, Christine
(Senior Research and Development Engineer)
- website


MARFURT, Andreas
(Postdoctoral Researcher)
- website


MARIĆ, Ante
(PhD Student / Research Assistant)
- website


MAYORAZ, André
(Research and Development Engineer)
- website


MICHEL, Samuel
(Research and Development Engineer)
- website


MITRO, Ioanni
(Postdoctoral Researcher)
- website


MOSTAANI, Zohreh
(PhD Student / Research Assistant)
- website


MOTLICEK, Petr
(Senior Research Scientist)
- website


MURALIDHAR, Skanda
(Research Associate)
- website


NADERI, Maryam
(AI Master Student)
- website


NANCHEN, Alexandre
(Senior Research and Development Engineer)
- website


ODOBEZ, Jean-Marc
(Senior Research Scientist with Academic Title)
- website


PANNATIER, Arnaud
(PhD Student / Research Assistant)
- website


PETERSEN, Molly (Rose)
(PhD Student / Research Assistant)
- website


PRASAD, Amrutha
(PhD Student / Research Assistant)
- website


PUROHIT, Tilak
(PhD Student / Research Assistant)
- website


QIU, Jiacheng
(Research Intern)
- website


RAJAPAKSHE, Shalutha
(PhD Student / Research Assistant)
- website


RANGAPPA, Pradeep
(Postdoctoral Researcher)
- website


RAZMJOO FARD, Amirreza
(PhD Student / Research Assistant)
- website


RUFAI, Amina
(Research Intern)
- website


SANCHEZ LARA, Alejandra
(Research Intern)
- website


SANCHEZ-CORTES, Dairazalia
(Postdoctoral Researcher)
- website


SARKAR, Eklavya
(PhD Student / Research Assistant)
- website


SENFT, Emmanuel
(Research Scientist (CRG Head))
- website


SHEIKH, Shakeel Ahmad
(Postdoctoral Researcher)
- website


TAFASCA, Samy
(PhD Student / Research Assistant)
- website


TARIGOPULA, Neha
(PhD Student / Research Assistant)
- website


TENEY, Damien
(Research Scientist)
- website


THORBECKE (NIGMATULINA), Iuliia
(PhD Student / Research Assistant)
- website


TORNAY, Sandrine
(Postdoctoral Researcher)
- website


VALENTINO, Marco
(Postdoctoral Researcher)
- website


VAN DER PLAS, Lonneke
(Research Scientist with Academic Title)
- website


VÁSQUEZ RODRÍGUEZ, Laura
(Postdoctoral Researcher)
- website


VERZAT, Colombine
(Research and Development Engineer)
- website


VIDAL, Maxime
(Research Assistant)
- website


VILLAMIZAR, Michael (Alejandro)
(Research Associate)
- website


VILLATORO TELLO, Esaú
(Research Associate)
- website


VLASENKO, Bogdan
(Research Associate)
- website


VUILLECARD, Pierre
(PhD Student / Research Assistant)
- website


WYSOCKI, Oskar
(Postdoctoral Researcher)
- website


XUE, Teng
(PhD Student / Research Assistant)
- website


ZANGGER, Alicia
(Research and Development Engineer)
- website


ZHANG, Xiaocheng
(Research Intern)


ZHANG, Yan
(PhD Student / Research Assistant)
- website


ZULUAGA GOMEZ, Juan Pablo
(PhD Student / Research Assistant)
- website


 

Publications choisies

Neural Network Adaptation and Data Augmentation for MultiSpeaker Direction-of-Arrival Estimation, W. He, P. Motlicek and J.-M. Odobez, IEEE/ACM Trans. on Audio, Speech and Language Processing, 29, pp. 1303-1317, 2021.

The first viable deep learning framework (task definition, network architecture, training paradigm) for solving fundamental auditory tasks such as sound source localization, speaker identification and speech/non-speech classification. The framework is suitable for highly noisy environments and overcomes limitations of previous methods, which heavily relied on idealized sound and environment models and are inadequate for everyday situations with multiple sound sources, background noise, short utterances, and lack of prior knowledge on the number of sound sources. The method learns sound source localization models with limited training resources leveraging simulated and weakly-labeled real audio data.

 

Active Learning by Feature Mixing, A. Parvaneh, E. Abbasnejad, D. Teney, G. R. Haffari, A. Van Den Hengel, & J. Q. Shi, In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 12227-12236, 2022.

A method to train deep learning models with humans in the loop. Current approaches to machine learning depend on large amounts of data that are costly or difficult to acquire. This paper presents an active learning approach where human experts interact with the learning algorithm to iteratively refine and resolve inconsistencies in a model by labeling a small set of training examples. This approach contributes to widening the accessibility of machine learning technologies to small organizations.

 

Learning Joint Space Reference Manifold for Reliable Physical Assistance, Razmjoo, A., Brecelj, T., Savevska, K., Ude, A., Petric, T. and Calinon, S., In Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), 2023.

Projets choisis

C-LING, 2022-2026, SNSF, Van der Plas: TOWARDS CREATIVE SYSTEMS WITH LINGUISTIC MODELLING

This project aims to investigate what aspects computational models need to perform creative cognitive tasks, from generating relatively simple novel concepts to more complex and structured ideas, across multiple domains and languages. More in particular, it aims to answer what types of structured and unstructured knowledge are needed and what models best integrate these types of knowledge.

 

NeuMath, 2022-2024, SNSF, Freitas: NEUMATH: NEURAL DISCOURSE INFERENCE OVER MATHEMATICAL TEXTS

NeuMath will develop models which can jointly represent and reason over two symbolic modalities (natural language and mathematical expressions) and will build the foundations to deliver embedding models which can interpret and support the generation of mathematical arguments (by leveraging available large-scale scientific corpora).

 

SMILE-II, 2021-2024, SNSF Sinergia, Magimai Doss: SMILE-II SCALABLE MULTIMODAL SIGN LANGUAGE TECHNOLOGY FOR SIGN LANGUAGE LEARNING AND ASSESSMENT PHASE-II

The proposed project SMILE-II aims to research and build advanced technology for sign language learning. More precisely, the proposed project builds on the groundwork laid down by the SNSF Sinergia project SMILE, which dealt with assessment of the manual activity of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) in isolated signs produced by early learners and L2 learners. SMILE-II will extend this technology to continuous sign language assessment including both manual and non-manual components of signs so that a DSGS learner’s sentence-level production can be assessed in an automatic manner.

Liste de tous les projets

C-LING, 2022-2026, SNSF, Van der Plas

Building computational models of human creative thinking to help with creative tasks

 

NeuMath, 2022-2024, SNSF, Freitas

Neuro-symbolic architectures for supporting mathematical discovery

 

NAST, 2020-2024, SNSF, Garner

Neural architectures for speech technology

 

SteADI, 2021-2025, SNSF, Garner

Storytelling algorithms for digital interviews

 

NKBP, 2020-2024, SNSF, Henderson

Deep learning models for continual extraction of knowledge from text

 

SINFONIA, 2023-2027, Innosuisse, Teney, Freitas

Generalization and domain adaptation of large language models

 

LUCIDELES, 2020-2023, SFOE, Kämpf

Research at the interface between humans and building control systems

 

CODIMAN, 2020-2024, National Research Programme "Digital Transformation", SNSF, Calinon

Cobotics, digital skills and the re-humanization of the workplace

 

SESTOSENSO, 2022-2025, Horizon Europe, Calinon

Physical cognition for intelligent control and safe human-robot interaction

 

SMILE-II, 2021-2024, SNSF Sinergia, Magimai Doss

Assistive technology for sign language learning and testing

 

Amazon research award, 2023, Teney

Addressing underspecification for improved fairness and robustness in conversational AI

 

MALORCA, 2016-2018, EU, Motlicek

Machine learning of speech recognition models for controller assistance

 

HAAWAII, 2020-2022, EU, Motlicek

Highly automated air-traffic controller workstations with artificial intelligence integration

 

ATCO2, 2019-2022, EU, Motlicek

Automatic collection and processing of voice data from air-traffic communications

 

EUROCONTROL, 2023-2024,  France, Motlicek

Automatic speech recognition in air-traffic control simulation