A study about the generalization of mood inference models distinguished by an award
Mobile sensing and computing play an increasing role in our daily life. Researchers and businesses use these technologies to provide personalized insights and services. This is achieved through the analysis of mobile sensor data using machine learning models. These models enable context-aware and personalized user experiences in general mobile apps and valuable feedback in mobile health apps.
Despite model generalization issues highlighted by many previous studies, research has often focused on improving the accuracy of models built with datasets representing homogeneous populations. In contrast, less attention has been given to studying the performance of mood inference models from the perspective of generalization of these models to other populations, including other countries.
The paper published by Lakmal Meegahapola, a doctoral student, Prof. Daniel Gatica-Perez, head of the Social Computing group at Idiap, and an international team of colleagues, received an IMWUT Distinguished Paper Award (DPA) at the ACM Ubicomp/ISWC 2023 conference held in Cancun, Mexico. This recognition was for a study published in the Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies. The study was conducted in the context of the EU H2020 WeNet project.
A more diverse and inclusive approach
To work with diverse mobile sensing data, researchers engaged volunteers to collect hundreds of thousands of self-reports in eight countries—China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, and the UK. This approach allowed to assess the effects of geographical diversity on mood inference models.
“To evaluate their performance, we compared models trained and tested within a country, within a continent, tested on a country not seen on training data, and trained and tested with multiple countries. We did this for mood inference tasks with non-personalized and partially personalized models,” Meegahapola explains.
Results show that partially personalized country-specific models perform the best. Further, models not centered on a country do not perform well compared to country-specific settings, even when they are partially personalized. Interestingly, continent-specific models outperform multi-country models in the case of Europe, but not in the case of Asia.
Building AI systems that serve everybody well
This work helps uncover issues related to geographical biases in models using machine learning and mobile sensing. This situation can be is especially sensitive when exporting a model to new countries. “We hope that these findings will motivate other researchers to build mobile sensing applications that are aware of geographical diversity. Our research illustrates a current, pressing gap: that of designing AI systems that are built to take into account all world regions, and not only the economically wealthy ones.” Gatica-Perez concludes.
More information
- “Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries”, Lakmal Meegahapola, William Droz, Peter Kun, Amalia de Götzen, Chaitanya Nutakki, Shyam Diwakar, Salvador Ruiz Correa, Donglei Song, Hao Xu, Miriam Bidoglia, George Gaskell, Altangerel Chagnaa, Amarsanaa Ganbold, Tsolmon Zundui, Carlo Caprini, Daniele Miorandi, Alethia Hume, Jose Luis Zarza, Luca Cernuzzi, Ivano Bison, Marcelo Rodas Britez, Matteo Busso, Ronald Chenu-Abente, Can Günel, Fausto Giunchiglia, Laura Schelenz, Daniel Gatica-Perez, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (PACM IMWUT), January 2023
- Social computing research group
- EU H2020 WeNet project