AIMANT

Assessing adult-child social interactions objectively remains a complex but essential task, especially for child psychiatrists evaluating neurodevelopmental progress. This is particularly critical for children with Autism Spectrum Disorder (ASD), where early, accurate diagnosis can significantly improve long-term outcomes. The earlier autism is identified, the sooner targeted interventions can support cognitive development and social interaction skills. However, current diagnostic methods are time-consuming, subjective, and difficult to scale, requiring lengthy observation by trained professionals. As a result, many children face delays in diagnosis and limited access to early therapeutic support and follow-up.

This project addresses these limitations by developing novel Computer Vision and Deep Learning algorithms to automate behavioral coding, making ASD diagnosis faster, more objective, and scalable. These algorithms will estimate severity scores and detect key behavioral markers, supporting clinicians in early screening and continuous neurodevelopmental monitoring. While progress has been made in activity and non-verbal behavior recognition, current methods struggle with adult-child interactions captured in naturalistic conditions—such as those in the Autism Diagnostic Observation Schedule (ADOS), the clinical gold standard. These difficulties arise from the complexity and subtlety of target behaviors: attention-related cues (e.g., gaze, pointing), infant micro-expressions, and motor stereotypes.

To support this, the project will leverage the ACTIVIS dataset, a large-scale, clinician-annotated audiovisual collection recorded during ADOS sessions. It includes over 180 sessions with 160 children performing various tasks assessing developmental domains. These recordings are precisely labeled using standardized protocols. A goal is to use this resource to train deep learning models for automatic behavioral scoring and ASD profiling.

In parallel, the project will collect a longitudinal dataset (BabySmile) capturing parent–child interactions between 3 and 18 months to investigate early neurodevelopmental markers in infants. Studies suggest that behaviors like early smiling and joint attention onset may indicate neurodevelopmental disorders, but the progression of these milestones remains poorly understood. Using computer vision and expert-labeled data, this dataset will help study early behavioral patterns and their predictive value for later diagnosis.

The project is organized around three main objectives:

- Methods. Develop advanced AI techniques to address key scientific challenges: (i) leveraging multimodal signals (e.g., video, audio, depth) to enrich behavior interpretation; (ii) detecting subtle, fine-grained motion cues; (iii) addressing data scarcity via weak supervision and efficient fine-tuning; (iv) fusing diverse cues across multiple processing levels; and (v) ensuring robustness and generalization to real-world clinical scenarios.

- Datasets. Extend the ACTIVIS dataset with additional behavioral annotations and ethical anonymization; collect the BabySmile dataset to enable early-stage developmental research with clinical relevance.

- Evaluation. Create interactive visual tools for clinicians to review videos, explore behavioral predictions, and validate ASD profiles. These interfaces will enable clinician-in-the-loop feedback to refine annotations and continuously improve model performance.

By addressing these scientific and clinical challenges, the project will close the gap between automated behavioral analysis and ASD assessment. The consortium brings together complementary expertise: UniCA for clinical leadership and ASD evaluation; Inria for activity and gesture analysis; Idiap for nonverbal behavior modeling; and LIRIS for emotion and action recognition. This multidisciplinary collaboration ensures innovation and strong clinical impact across all objectives.