Implementing ethics in medical AI is a complex issue that is context and worldview sensitive for which Critical Systems Thinking is a well-suited methodology. Such an approach involves a participatory process engaging diverse stakeholders. This study has set out to explore how to use a Systems Thinking approach involving stakeholders for a Covid-19 AI app. In this talk, study design, results and outcomes will be shared including guidelines about the process itself.

Idiap Create Challenge unique 9-day AI SUPER HACKATHON! The best way to transform your ideas into prototypes!

Discover participating teams final pitches before the results and awards ceremony.

More information:

www.createchallenge.org

Ever been frustrated by a partially achieved prototype created during a hackathon? As engineers, researchers, and creators, we experienced this limitation first-hand. Therefore we came up with our own unique 9-day AI SUPER HACKATHON! The best way to transform your ideas into prototypes!

Cheer for your team and discover a variety of AI related projects!

More information:

www.createchallenge.org

Idiap is launching a new public series of symposiums entitled: Perspectives on AI. These events aim to bring together academia, industry and policy makers to present their perspectives on artificial intelligence, in particular how we can leverage the power of AI to address key societal and industrial challenges.

In this Perspectives on AI edition, we will discuss emerging methods which are outside the traditional deep learning and statistics toolbox, aiming to address these challenges. Under the ethos of 'Doing more with less', invited speakers from academia and industry will discuss how recent methodological developments in AI and statistical methods can address existing challenges and deliver new opportunities for making sense of an increasingly complex data landscape.

The event is targeted towards academics, data scientists, practitioners and executives who need to make sense of complex, heterogeneous and small data and it aims to discuss the following questions:

What are the most promising methods for interpreting distributed and heterogeneous data/evidence?

How can these methods address grand societal challenges such as evidence-based policy making, personalised healthcare and predicting the impact of environmental changes?

What are the opportunities for creating value in strategic areas for Switzerland such as Healthcare and Finance?

More info: Perspectives on AI Symposiums

A Presentation Attack Detection (PAD) system is the crucial sub-component of the biometric systems when it comes to recognize or verify someone for further processing. In case of the lack of such PAD systems, one can penetrate into the protected areas in unauthorized way and causes the biometric validation to be broken. Therefore, a PAD system must be exist and robust against all kind of attacks including any kind of the paper, tablet screen, 2D or 3D mask etc. In this talk, I will present my past study about the generalization of PAD systems. I will propose an explainability based regularization method for the PAD systems and share the generalizability performance of the proposed method in public and cross-dataset experiments. I will also share a brief introduction about my other studies regarding of hard and soft biometrics.

The European Association for Biometrics (EAB) and the Center for Identification Technology Research (CITeR) are organising a 2-day biometrics workshop, hosted by the Idiap Research Institute in Martigny Switzerland on April 18-19 2023, on the topics of Bias mitigation, Template Security, Presentation Attack Detection and Deepfakes. This workshop is a second edition of the EAB workshop on Presentation Attack Detection previously hosted at the Idiap Research Institute in 2020.

The workshop will be co-located with the CITeR Spring 2023 Program Review and training events of the TReSPAsS-ETN and PriMa-ITN projects.

Entrance fees: normal: 360,00 EU / EAB members: 240,00 EU / CITeR member and/or CITeR speaker (in person): 90,00 EU / EAB Member virtual | CITeR member virtual: 100,00 EU / non-EAB/CITeR member virtual | (non)-EAB speakers: 200,00 EU

More:

Program here

Parkinson's disease (PD) is a (mainly) movement disorder and appears due to the progressive death of dopaminergic neurons in the substantia nigra of the midbrain (part of the basal ganglia). Diagnosis and monitoring of PD patients are still highly subjective, time-consuming, and expensive. Existing medical scales used to evaluate the neurological state of PD patients cover many different aspects, including activities of daily living, motor skills, speech, and depression. This makes the task of automatically reproducing experts' evaluations very difficult because several bio-signals and methods are required to produce clinically acceptable/practical results.

This talk tries to show how different bio-signals (e.g., speech, gait, handwriting, and facial expressions) can be used on the way to find suitable models for PD diagnosis and monitoring. Results with classical feature extraction and classification methods will be presented along with CNN and GRU -based architectures.

Understanding the technology, its limitations and opportunities

Recent Artificial Intelligence (AI) models such as ChatGPT have captured the imagination of the public on the recent advances of the field, sparking a debate on the potential applications and implications of this technology.

The event is dedicated to the discussion of Large Neural Language Models, the underlying framework behind systems such as ChatPGT. In the workshop we will aim to answer some critical questions about the principles behind these models and the associated challenges, emerging applications and opportunities that this technology brings.

The workshop is targeted towards academic and industrial participants willing to understand how these models work, their strengths, limitations and general societal impact.

The detailed program is available here:

Program

REGISTRATIONS ARE CLOSED!

Information geometry (IG) and Optimal transport (OT) have been attracting much research attention in various fields, in particular machine learning and statistics. In this talk, we present results on the generalization of IG and OT distances for finite-dimensional Gaussian measures to the setting of infinite-dimensional Gaussian measures and Gaussian processes. Our focus is on the Entropic Regularization of the 2-Wasserstein distance and the generalization of the Fisher-Rao distance and related quantities. In both settings, regularization leads to many desirable theoretical properties, including in particular dimension-independent convergence and sample complexity. The mathematical formulation involves the interplay of IG and OT with Gaussian processes and the methodology of reproducing kernel Hilbert spaces (RKHS). All of the presented formulations admit closed form expressions that can be efficiently computed and applied practically.

In this talk, we present preliminary results on experiments which were performed in order to understand, what information is represented in which layer of deep neural networks. We will motivate our experiments with an image processing problem (identification of orca individuals based on the dorsal fin), where we show that the result of unsupervised clustering of previously unseen individuals strongly depends on the underlying embedding and for what that embedding was trained in a supervised manner. We then present preliminary results on t-SNE projections of different pathologic an control corpora based on the different layers of a pre-trained wav2vec2 module and end with an outlook to current and future research.

In this talk, we will discuss how Video Analytics can be applied to human monitoring using as input a video stream. Existing work has either focused on simple activities in real-life scenarios, or on the recognition of more complex (in terms of visual variabilities) activities in hand-clipped videos with well-defined temporal boundaries. We still lack methods that can retrieve multiple instances of complex human activity in a continuous video (untrimmed) flow of data in real-world settings.

Therefore, we will first review few existing activity recognition/detection algorithms. Then, we will present several novel techniques for the recognition of ADLs (Activities of Daily Living) from 2D video cameras. We will illustrate the proposed activity monitoring approaches through several home care application datasets: Toyota SmartHome, NTU-RGB+D, Charades and Northwestern UCLA. We will end the talk by presenting some results on home care applications.

Keywords: people tracking, behavior understanding, activity monitoring.

Our drawing robot e-David is able to create paintings using visual feedback. So far, our paintings have been created using a stroke-based metaphor. In my talk I will speak about the development of a n umber of stroke-based styles. However, being in close contact with artists we realized at some point that painting can much better be modeled by interacting and contrasting areas instead of strokes - which are more the basis of drawings. This paradigm shift allows us to construct paintings from a different perspective; the interaction between areas enables us to model different forms of abstraction and reshape areas according to style settings. We will also be able to integrate machine-learning based tools for analyzing and deconstructing input images. This enhances our creative space and will allow us to find our own forms of machine abstraction and representation.

In this talk, we will discuss a panoramic view of digital forensics in the last 10 years and how it needed to evolve from basic computer vision and simple natural language processing techniques to powerful AI-driven methods to deal with the signs of the new age. We will discuss tampering detection, fact-checking, deepfakes, and authorship analysis as well as recent advances in self-supervised learning to deal with large-scale search in some forensics problems.

Thanks to recent advances in sensing, perception and actuation technologies, robots are no longer just mindless machines designed to perform repetitive tasks on factory floors. Nevertheless, the vision of intelligent robotic assistants capable of helping us with every-day tasks at work and at home remains elusive. This is to a large extent because, unlike humans, robots lack an innate understanding of the physical principles that govern the dynamics of the physical world. To overcome this technological barrier, our group develops theoretical and algorithmic foundations for computational models that enable machines to predict how physical objects move and deform. Our efforts in this area have led to an analytically differentiable formulation of dynamics for multi-body systems. Within a unified framework, our simulation model handles rigid bodies, deformable objects, as well as frictional contact. In this talk, I will present our simulation framework and show how it can be used for tasks such as trajectory optimization, policy learning and computational design. Through a set of applications that range from soft robot locomotion to dynamic manipulation of deformable objects, I will also highlight early successes in using our simulation model to bridge the reality gap.

In this talk, a complete automatic real-time system for sport analytics, including tracking 3D skeletal pose of multiple players and the ball from multiple video cameras will be presented. This system was developed by Second Spectrum and successfully deployed in top tier sport leagues including the NBA and the English Premier League.

About Second Spectrum

Second Spectrum creates products that fuse design with spatiotemporal pattern recognition, machine learning, and computer vision to create sports insight and experiences. In Sep 2015, Second Spectrum acquired PlayfulVision, a spin-off from EPFL, which provided video-based player and ball tracking technology. After integrating its core technology, Second Spectrum has signed multiple multi-years league-wide contracts with major sports leagues such as the NBA (basketball), the English Premier League and MLS (football) to be their official player tracking provider. In May of 2021, Second Spectrum was acquired by the Genius Sports group, which covers more than 400 leagues worldwide.

We illustrate through the example of a robot coach for physical rehabiliation the application of GMM in Riemanian manifolds, but also the need to represent complex movements and tasks, as well as the need to evaluate the motivation for patients to interact with their coach.

This motivation to interact is modeled through the theory of intrinsic motivation.

Multi-task learning by robots poses the challenge of the domain knowledge: complexity of tasks, complexity of the actions required, relationship between tasks for transfer learning. However, this domain knowledge can be learned to address the challenges of high-dimensionality and unboundedness in life-long learning. For instance, the hierarchy between tasks of various complexities can be learned to bootstrap transfer of knowledge from simple to composite tasks.

We focus in hierarchical reinforcement learning framework, on algorithms based on intrinsic motivation to explore the action space and task space. They can discover the relationship between tasks and learnable subtasks. Robots can efficiently associate sequences of actions to multiple control tasks: representations of task dependencies, emergence of affordances mechanism, curriculum learning and active imitation learning. These active learning algorithms choose the most appropriate exploration strategy based on empirical measures of competence and learning progress. It infers its curriculum by deciding which tasks to explore first, how to transfer knowledge, and when, how and whom to imitate.

To celebrate its 50th anniversary, the Dalle Molle Foundation is organizing a conference including two AI oriented speeches by renowned international speakers:

- Artificial intelligence for Thinking Humans by Prof. Melanie Mitchell from the Santa Fe Institute

- Human compatible: AI and problem of control by Prof. Stuart Russell from the University of California, Berkeley, and Honorary Fellow, Wadham College, Oxford

Free registrations and the full program are available here: https://idiap.ch/dallemolle

Schille Conference Room: ground floor, Idiap Research Institute

Sign language technology, unlike spoken language technology, is an emerging area of research. Sign language technologies can help in bridging the gap between the Deaf community and the hearing community. One such computer-aided technology is sign language learning technology. To build such a technology, there is a need for sign language technologies that can assess sign production of learners in a linguistically valid manner. Such a technology is yet to emerge. This thesis is a step towards that, where we aim to develop an "explainable" sign language assessment framework. Development of such a framework has some fundamental open research questions: (a) how to effectively model hand movement channel? (b) how to model the multiple channels inherent in sign language? and (c) how to assess sign language at different linguistic levels?

The present thesis addresses those open research questions by: (a) development of a hidden Markov model (HMM) based approach that, given only pairwise comparison between signs, derives hand movement subunits that are sharable across sign languages and domains; (b) development of phonology-based approaches, inspired from modeling of articulatory features in speech processing, to model the multichannel information inherent in sign languages in the framework of HMM, and validating it through monolingual, cross-lingual and multilingual sign language recognition studies; and (c) development of a phonology-based sign language assessment approach that can assess in an integrated manner a produced sign at two different levels, namely, lexeme level (i.e., whether the sign production is targeting the correct sign or not) and at form level (i.e. whether the handshape production and the hand movement production is correct or not), and validating it on the linguistically annotated Swiss German Sign Language database SMILE.

Defense de these publique (en francais)

Ce travail de these couvre plusieurs methodes numerique pour assembler des series d'images dynamiques du coeur a partir d'une large gamme de microscopes, dont notamment des microscopes confocaux a balayage laser, qui sont lents mais courants, et des microscopes dont la frequence d'acquisition d'images est lente.

This talk will present two recent work done at Naver Labs Europe at the intersection of robotic manipulation and deep learning.

In the first part of the talk, we will discuss about imitation learning. Recently, one-shot imitation learning has shown encouraging results for executing variations of initial conditions of a given task without requiring task-specific engineering. However, it remains inefficient for generalizing in variations of tasks involving different reward or transition functions. In this work, we aim at improving the generalization ability of demonstration based learning to unseen tasks that are significantly different from the training tasks. We introduce the use of transformer-based sequence-to-sequence policy networks trained from limited sets of demonstrations. Then, we propose to meta-train our model from a set of training demonstrations by leveraging optimization-based meta-learning. We evaluate our approach and report encouraging results using the recently proposed framework Meta-World which is composed of a large set of robotic manipulation tasks organized in various categories.

In the second part of the talk, we will present our recent work on validating kinematic feasibility of a planned robot motion and finding corresponding inverse solutions are time-consuming processes, especially for long-horizon manipulation tasks. As most existing approaches are based on solving iterative gradient based optimization, the processes are time consuming and have high risk of falling in local minima. So, we propose a unified framework to learn a kinematic feasibility model and one-shot inverse mapping model for a redundant robot manipulator. Once they are trained, the models can compute kinematic reachability of a target pose and its inverse solutions directly without iterative process. We validate our approach using a 7-DOF robot arm with an object grasping application.

Technoheritage - when AI meets art.

How does future heritage look like? What role can AI and deep learning play? How can we visually reassemble the patterns of specific eras or locations in heritage digitisation such as the richness of forms from Mesopotamia or todays' Iraq? How can we built new decolonial non-biased databases and thereby add to the visual contemporary imaginary?

Nora Al-Badri will talk about her artistic research and her new work - the "Neuronal Ancestral Sculptures Series" that examines the potential of GANs in this context as a new artistic tool and use heritage data from artefacts of today's Iraq, which is data based on ancestral knowledge (forms, patterns, artefacts) to create a form of generative aesthetics that go going beyond representation and mimesis.

She will also show a few other projects such as the Nefertiti Hack or a critical chatbot, that decontextualises colonial museum collections.

Iterative Reasoning Path Retrieval for Multi-Hop Question Answering

Multi-hop Machine Reading necessitates retrieving multiple pieces of evidence over a possibly large collection of documents. One of the challenges comes from the fact that only a few lexical or semantic relationships are overlapping with the question. For this reason, classic information retrieval methods, which are mainly based on word matching techniques, even when distributional, have failed to achieve usable results in such a task. This talk introduces a series of novel neural iterative retrieval approaches that learn to find the sequence of necessary pieces of evidence, also called reasoning paths, to answer open-domain multi-hop questions. The approach trains a neural network based on a multi-headed transformer architecture that learns to retrieve evidence paragraphs conditioned on the previously retrieved documents. Any sequential search algorithm, like beam-search or MCTS, can then be coupled to our model to find the most probable sequence regarding a question. Finally, we use a neural approach as a stopping criteria for sequential retrieval. Experimental results show encouraging results in an open-domain QA dataset, HotpotQA. We will conclude the talk with current limitations and perspectives associated to this task.

Cognitive and neuro-biological profiles of elderly individuals are very heterogeneous. Healthy aging, psychiatric disorders, genetic variability, and neuro-degenerative diseases all contribute to this heterogeneity. Classification systems with categories and sub-categories of neuro-degenerative disorders were traditionally used to identify differences across groups. However, a large portion of the observed variability in clinical manifestation on the individual level remains unexplained. Uncertain diagnostic labels, the presence of possibly multiple interacting conditions, and systematic biases due to technical variations in the acquisition hardware further call for innovative solutions. During his PhD, Ahmed Abdulkadir focused on improving MRI image classification and segmentation using deep (end-to-end) and shallow machine learning. The outcomes from the thesis and the literature corroborated that disruptive innovations in the field will be facilitated by the ability to make predictions with confidence bounds in heterogeneous populations with incomplete data. Methodological grounds and preliminary results of ongoing studies as well as a quick preview on the recently accepted SNF Postdoc.Mobility proposal titled MRI-based characterization of subgroups in aging and dementia will be presented.

Behaviomedics is the application of automatic analysis and synthesis of affective and social signals to aid objective diagnosis, monitoring, and treatment of medical conditions that alter one's affective and socially expressive behaviour. Or, put more succinctly, it is the objective assessment of clinically relevant expressive behaviour. Objective assessment of expressive behaviour has been around for a couple of decades at least, perhaps most notably in the form of facial muscle action detection (FACS AUs) or pose estimation. While often presented alongside work on emotion recognition, with many works presented as a solution to both emotion and objective behaviour asessment, the two problems are actually incredibly different in terms of machine learning problems. I would argue that a rethink of behaviour assessment is useful, with emotion recognition and other 'higher level behaviours' building on objective assessment methods. This is particularly pertinent in an era where the interpretability of machine learning systems is increasingly a basic requirement. In this talk I will firstly present our lab's efforts in the objective asssessment of expressive behaviour, followed by three areas where we have applied this to automatic assessment of behaviomedical conditions, to wit, depression analysis, distinguishing ADHD from ASD, and measuring the intensity of pain in infants and adults with shoulder-pain. Finally, I will discuss how we see Virtual Humans can be used to aid the process of screening, diagnosing, and monitoring of behaviomedical conditions.

Voice-based virtual assistants on smartphones or smart speakers are quite wide-spread, but they are often specialized in tasks related to their providers' business. Some chatterbots, on the contrary, support extended conversations but offer no access to explicit knowledge. We will present a conversational agent designed at HEIG-VD which combines task-oriented dialogue (to answer questions based on the knowledge enclosed in documents such as Wikipedia pages) with the capacity to chat, i.e. to reply to non-task-oriented utterances and manage the social aspects of a conversation. The combination is achieved using dialogue act recognition, which routes user utterances to one of the components. The system is demonstrated as an 'action' or 'skill' on a smart speaker.

Clinical research has never been more active and diverse than it is at this moment. Research efforts span national and cultural borders and broad online dissemination of results makes insights available at a global scale with ever decreasing latency. In the face of these developments, individual researchers and practitioners are confronted with a seemingly intractable amount of material (approximately 1 Million scholarly articles are newly published in the life sciences each year). While highly trained human experts excel at making precision diagnoses, coverage, especially for uncommon conditions can be greatly improved. In this talk, we will discuss a range of (deep) machine learning techniques that provide automatic clinical decision support on the basis of large-scale data collections. I will present early and ongoing work on a) Predictive assistants in post-operative care of cardiac surgery patients, that serve as early warning systems in case of undesirable and dangerous complications. b) Automatic summarization of individually long patient records to obtain concise and topically targeted summaries for physicians. c) Data-driven diagnosis of rare diseases that individually occur too infrequently to allow clinical specialists to establish the necessary routine and experience.

Communication skill is an important soft skill that candidates need and what employers look for, to succeed in teams. From candidate's point of view, automatic feedback to aid behavior change is helpful, and for interviewers interviewing 100 is easier than 1000. Automatic methods to assess and select candidates for an in depth interview or a training program is a relevant problem in social computing, involving analysis of nonverbal and verbal cues, using speech, spoken text and visual behavior analysis.

In our research, we have systematically investigated Asynchronous Video Interviews (AVIs) (starting to be widely deployed by companies like Hirevue) for interviewing, vis-a-vis face-to-face interviews, in terms of interviewer perception, performance of automatic assessment methods and interviewee setting preferences. Face-to-face interviews serve as benchmark setting, while AVIs are relatively new and less understood. Unlike Video resumes which don't allow prompting, AVIs are both scalable (i.e. can be done anywhere anytime) and allow prompting. Later we have also compared spoken vs written interviews - Written in text and hand written form. Apart from this, we will also discuss works on Automatic feedback prediction, and assessment in human agent interaction setting. This leads to automatic follow up question generation, an interesting text generation problem. Finally, we will conclude with some preliminary work in behavior generation in the context of Indian Sign Language Synthesis, and show some demos of the works discussed.

In this talk, I will introduce our recent research on weakly-supervised learning and reliable learning.

The motivation for weakly-supervised learning is to accurately perform machine learning only from "weak" data that can be collected more easily/cheaply than fully-labeled data. In the first half of this talk, I give an overview of our recently developed empirical risk minimization framework for weakly-supervised classification, covering binary classification only from PU data, PNU data, Pconf data, UU data, SU data, and Comp data (P:positive, N:negative, U:unlabeled, Conf:confidence, S:similar, and Comp:complementary).

For reliable deployment of machine learning systems in the real world, various types of robustness is needed. In the latter half of this talk, I will give an overview of our recent work on robust learning towards noisy training data, changing environments, and adversarial test input.

Finally, I will briefly introduce our RIKEN Center for Advanced Intelligence Project (AIP), which is a national AI project in Japan started in 2016. AIP covers a wide range of topics from generic AI research (machine learning, optimization, applied math., etc.), goal-oriented AI research (material, disaster, cancer, etc.), and AI-in-society research (ethics, data circulation, laws, etc.).

Deep learning-based analysis methods are gaining interest in cosmology due to their unique ability to create very rich and complex models.

These models are particularly well suited for analysis of large scale structure data, as the matter density fields are comprised of highly nonlinear, complicated features, such as halos, filaments, sheets and voids.

But can this information be utilised by the deep learning algorithm to gain a better understanding of the cosmological model?

In this talk I will present the application of Convolutional Neural Networks (CNNs) for constraining cosmological parameters.

I will compare the constraining power against the commonly used statistic, the power spectrum, and explore different regimes in quality of data and simulations.

Finally, I will introduce the Generative Adversarial Networks (GANs): a CNN-based technique, which can learn from a training set and then generate new, statistically similar data.

I will present a study of applying GANs to generating samples of the cosmic web and discuss the prospects of applying them to render 2D and 3D N-body simulation - like data.

We address the optimization of a computer model, where each simulation either fails or returns a valid output performance. We suggest a joint Gaussian process model for classification of the inputs (computation failure or success) and for regression of the performance function. We discuss the maximum likelihood estimation of the covariance parameters, with a stochastic approximation of the gradient. We then extend the celebrated expected improvement criterion to our setting of joint classification and regression, thus obtaining a global optimization algorithm. We prove the convergence of this algorithm. We also study its practical performances, on simulated data, and on a real computer model in the context of automotive fan design.

Topics covered:

- simple definition AI and Radiology

- radiology needs and problems

- why we've been talking so much about AI and radiology for the past few years

- Can radiologist be replaced by algorithms ?

- the radiologist's value chain and clinical workflow

- the AI and Radiology industrial ecosystem

- AI solutions approved by medical regulators in 2019

- clinical use cases to explore

- possible future scenarios.

Traditionally the behaviour of social robots has been programmed by engineers, but robots should be able to learn from their users to increase their range of application and improve their behaviour over time. This talk will start by presenting Supervised Progressively Autonomous Robot Competencies (SPARC), a machine learning framework enabling non-technical users to control and teach a robot to interact meaningfully with people in an efficient and safe way. The core premise is that the user initially remotely operates the robot, while an algorithm associates actions to states and gradually learns. Over time, the robot is able to take over from the user while still giving the user oversight of its behaviour by ensuring that every action the robot executes has been actively or passively approved by the user. The latter half of this talk will present results of a recent study evaluating SPARC in a real human-robots interaction where a robot was taught to support children in an educational activity.

Optimal transport (OT) has become a fundamental mathematical tool at the interface between calculus of variations, partial differential equations and probability. It took however much more time for this notion to become mainstream in numerical applications. This situation is in large part due to the high computational cost of the underlying optimization problems. There is a recent wave of activity on the use of OT-related methods in fields as diverse as image processing, computer vision, computer graphics, statistical inference, machine learning.

In this talk, I will review an emerging class of numerical approaches for the approximate resolution of OT-based optimization problems. This offers a new perspective for the application of OT in high dimension, to solve supervised (learning with transportation loss function) and unsupervised (generative network training) machine learning problems.

More information and references can be found on the website of our book "Computational Optimal Transport" https://optimaltransport.github.io/

The project Code is dedicated to research on digital issues within our societal structures. Through a program of artistic residencies, conferences, exhibitions and publications, axes of reflection aim to deploy new discursive questioning, by connecting art and science. The digital integration in our daily lives redefines data related to societal aspects, and it is exactly on this axis of reflection that CODE is situated. Too often seen as a threat to our own reality, or on the contrary embodying the utopian promise of a better world, the ubiquitous digital environment is approached as an element that creates a gap between "true and wrong." Whereas on the contrary, bonds must be created. By allowing art to be integrated as a scientist element of discussion related to our everyday environment and societal, the project Code positioning theoretical and empirical reflections side by side.

Full autonomy in aircraft requires systems that can solve problems that are currently the exclusive domain of human cognitive function. Deep Neural Networks are the only feasible option we have today, but we need new methods to satisfy the burden of proof of airworthyness for the safety authorities. We will lay out the problem and propose directions in which to find the answer.

Dr. Luuk van Dijk is founder and CEO of Daedalean, a startup dedicated to make flying cars have their own pilot's license, so you won't have to.

Video understanding is one of the hardest challenges in AI. If a machine can look at videos and "understand" the events that are being shown, then machines could learn by themselves, perhaps even without supervision, simply by "watching" broadcast TV, Facebook, Youtube, or similar sites. Making progress towards this goal requires contributions from experts in diverse fields, including computer vision, automatic speech recognition, machine translation, natural language processing, multimodal information processing, and multimedia. I will report the outcomes of the JSALT 2018 Workshop on this topic, including advances in multitask learning for joint audiovisual captioning, summarization, and translation, as well as auxiliary tasks such as text-only translation, language modeling, story segmentation, and classification. I will demonstrate a few results on the "How-to" dataset of instructional videos harvested from the web by my team at Carnegie Mellon University and discuss remaining challenges and possible other datasets for this research.

Entropy regularization, while a standard technique in the online learning toolbox, has only been recently discovered by the reinforcement learning community: In recent years, numerous new reinforcement learning algorithms have been derived using this principle, largely independently of each other. So far, a general framework for these algorithms has remained elusive. In this work, we propose such a general framework for entropy-regularized average-reward reinforcement learning in Markov decision processes (MDPs). Our approach is based on extending the linear-programming formulation of policy optimization in MDPs to accommodate convex regularization functions. Our key result is showing that using the conditional entropy of the joint state-action distributions as regularization yields a dual optimization problem closely resembling the Bellman optimality equations. This result enables us to formalize a number of state-of-the-art entropy-regularized reinforcement learning algorithms as approximate variants of Mirror Descent or Dual Averaging, and thus to argue about the convergence properties of these methods. In particular, we show that the exact version of the TRPO algorithm of Schulman et al. (2015) actually converges to the optimal policy, while the entropy-regularized policy gradient methods of Mnih et al. (2016) may fail to converge to a fixed point.

The first part of this talk will introduce data science research at ZHAW - from the broader picture of the ZHAW Datalab, an interdisciplinary and inter-departmental network of collaborations within our university comprising more than 70 researchers, to the specific work of my core group of PhD and master students. I will showcase specific results of industrial-academic collaborations to illustrate our setup for applied research. Examples are: automated newspaper segmentation for real-time print media monitoring; enabling database queries in natural language; or improving hiring processes with information retrieval.

The second part will focus on recent results of my core group in pattern recognition with deep learning. I will demonstrate recent results in speaker clustering using learned embeddings; learning novel clustering methods purely from scratch, thus extending current metric learning approaches to neural architectures that learn to output a clustering of previously unseen data / unseen classes end-to-end; and our novel dataset and object detection method to detect many tiny objects on large images of musical symbols.

Stochastic Approximation (SA) is useful when the aim is to find optimal points, or zeros of a function, given only its noisy estimates. In this talk, we will review our recent advances in techniques for SA analysis. This talk has four major parts. In the first part, we will see a motivating application of SA to network tomography. Here, we shall also discuss the convergence of a novel stochastic Kaczmarz method. In the second part, we shall discuss a novel tool based on the Alekseev's formula to obtain the rate of convergence of a nonlinear SA to a specific solution, given that there are multiple locally stable solutions. In the third part, we shall extend the previous tool to the two timescale but linear SA setting. We shall also discuss how this tool applies to gradient Temporal Difference (TD) methods such as GTD(0), GTD2, and TDC used in reinforcement learning. For the analyses in the second and third parts to hold, the initial step size must be chosen sufficiently small, depending on unknown problem-dependent parameters; or, alternatively, one must use projections. In the fourth part, we shall discuss a trick to obviate this in context of the one timescale, linear TD(0) method. We strongly believe that this trick is generalizable. We also provide here a novel expectation bound. We shall end with some future directions.

This is joint work with several people.

To ensure food and water security, it is crucial to understand the impact of climate on our capabilities to meet water demand for all sectors, and in particular for its most consumptive one, agriculture. A great challenge is the scale at which this needs to be tackled. Indeed, while all water demands need to be met locally, large river and groundwater systems, food distribution networks and political decisions span the scale of states or nations.

In this paper, we propose to quantify the risks associated with climate variability under our current water demands. This assessment is performed by exploring the response of an integrated water model developed for the continental United States at a county scale. The model comprises a surface water network with river nodes and reservoirs, a simple representation of groundwater, water demands at the county-scale and a statistical crop model. The integrated model then allocates water across the sectors by solving an optimization function, where the choice of water sources is driven both by relative cost and water availability as it varies over months and yearsi.

To understand the impact of climate variability, reconstruction of past precipitation, temperature and run-off for the last 60 years are used as input for the integrated model. We analyse the spatial and temporal variation in water stress in response to the climate forcing to highlight regions at risk. This analysis is extended by considering 500 years of streamflow reconstructed from paleoclimate data. We then quantify impacts the risks of crop failures, groundwater depletion, and losses of farmer income. The proposed approach results in a quantification of the risks associated with our present water consumption and might serve as a tool for the identification of integrated solution for water across large spatial scales.

xtensor is a C++ high-performance n-dimensional array computing library which sets out to be as fast as Eigen, while borrowing many API ideas from NumPy. The library was started in 2016 and uses many features of modern C++ and a good deal of template metaprogramming to be as efficient as possible. Besides the n-dimension arrays, xtensor also features the same broadcasting rules as NumPy. We hope that xtensor will be of great use in the Robotics community, bridging the gap between Python prototyping and implementing algorithms in C++.

Accompanying to the core package, we already have an ecosystem of several useful libraries:

- xtensor-python (bindings from Python to C++, supporting NumPy arrays)

- xtensor-ros (ROS bindings, making it easy to send & receive xtensor-arrays in C++)

- xtensor-blas (BLAS/LAPACK bindings mimicking the numpy.linalg interface)

- xtensor-fftw (Fast Fourier Transforms using NumPy's API, and the well-known fftw-library)

- xtensor-julia / xtensor-R (bindings for R and Julia)

- xtensor-io (easy reading and writing of audio, image, and NumPy npy/npz files)

Link to interactive presentation on Binder:

mybinder.org/v2/gh/wolfv/presentations/master

In the absence of power-sharing, minority groups in opposition have powerful incentives to substitute the ballot with the bullet. In contrast, when power is shared among all major groups in society, the relative gains of sticking to electoral politics are larger for minority groups. After making the theoretical argument, we provide in the current paper an empirical analysis of the impact of power-sharing at the local level, making use of fine-grained data from Northern Ireland's 26 local district councils over the 1973-2001 period. We find that power-sharing has a sizable and robust conflict-reducing impact.

Many data in scientific computing and machine learning is highly structured. When this structure is given as a mathematically smooth manifold, it is usually advisable to explcilty exploit this property in theoretical analyses and numerical algorithms. I will illustrate this using two examples. In the first, the manifold is classical: the set of symmetric and positive definite matrices. The problem we consider is the computation of the geometric mean, also called Karcher mean, which is a generalization of the arithmetic mean where we explicitly take into account that the data lives on a manifold. The application is denoising or interpolation of covariance matrices. The other example considers a non-standard manifold: the set of matrices of fixed rank. The application is now recommender systems (the Netflix problem) and the algorithm low-rank matrix completion. I will show that one of the benefits of the manifold approach is that the generalisation to low-rank tensor completion is conceptually straightforward but also computationally efficient.

During the past two decades, biological imaging has undergone a revolution in the development of new microscopy techniques that allow visualization of tissues, cells, proteins and macromolecular structures at all levels of resolution. Thanks to recent advances in optics, digital sensors and labeling probes, one can now visualize sub-cellular components and organelles at the scale of a few dozens nanometers to several hundreds of nanometers. As a result, fluorescent microscopy and multimodal imaging has become the workhorse of modern biology. All these technological advances in microscopy, created new challenges for researchers in quantitative image processing and analysis. Therefore, dedicated efforts are necessary to develop and integrate cutting-edge approaches in image processing and optical technologies to push the limits of the instrumentation and to analyze the large amount of data being produced.

In this talk, we present image processing methods, mathematical models, and algorithms to build an integrated imaging approach that bridges the resolution gaps between the molecule and the whole cell, in space and time. The presented methods are dedicated to the analysis of proteins in motion inside the cell, with a special focus on Rab protein trafficking observed in time-lapse confocal microscopy or total internal reflection fluorescence microscopy. Nevertheless, the proposed image processing methods and algorithms are flexible in most cases, with a minimal number of control parameters to be tuned. They can be applied to a large range of problems in cell imaging and can be integrated in generic image-based workflows, including for high content screening applications.

When speech processing systems are designed for use in multilingual environments, additional complexity is introduced. Identifying when language switching has occurred, predicting how cross-lingual terms will be pronounced, obtaining sufficient speech data from diverse language backgrounds: such factors all complicate the development of practical speech-oriented systems. In this talk, I will discuss our research group's experience in building speech recognition systems for the South African environment, one in which 11 official languages are recognised. I will also show how this relates to our participation in the BABEL project, a recent 5-year international collaborative project aimed at solving the spoken term detection task in under-resourced languages.

Charisma has been devilishly difficult to measure; there has also been a dearth of studies estimating the causal impact of charisma on outcomes. In this seminar I will use a new definition of charisma to demonstrate how it can be manipulated, and will also show the economic impact of charisma on worker productivity. Moreover, I will discuss how charisma can be coded from archival data, and demonstrate its utility for predicting a range of outcomes including winning the U.S. presidential election, the amount of views on TED talks, and retweets of tweets.

In this talk, I will present our work on Domain Adaptation, which tackles scenarios where the training (source) and test (target) data have been acquired in different conditions. To address this, we have introduced learning algorithms that attempt to make the distributions of the source and target data as similar as possible. In particular, I will present a (shallow) transformation learning method, and discuss different measures that can be used to compare the source and target distributions. I will then turn to a Deep Learning approach, in which I will show that allowing the weights of the network to differ between the source and target samples yields better accuracy. I will show results on standard image recognition benchmarks, as well as on the task of leveraging synthetic data to train a classifier for real images.

The Natural Speech Technology EPSRC Programme Grant was a 5 year collaboration between Edinburgh,Cambridge and Sheffield Universities, with the aim of improving core speech recognition and synthesis technology. During the lifetime of the project dramatic changes took place in the underlying technology for speech processing with the introduction of deep learning. This has yielded significant performance improvements, as well as offering a very rich space of model to investigate. This talk discusses the general area of deep learning for speech processing, with a particular emphasis on sequence-to-sequence models: in speech recognition, waveform to text; and in synthesis, text to waveform. Both generative and discriminative models for sequence-to-sequence models are described along with variants on the standard topologies and the implications for both training and inference. Rather than focusing on results for particular models, the talk aims to describe the connections and differences between sequence-to-sequence models and the underlying assumptions for these models.

Tutorial on Regression based on the article:

Freek Stulp and Olivier Sigaud (2015). Many Regression Algorithms, One Unified Model - A Review. Neural Networks, 69:60-79.

Link:

freekstulp.net/publications/pdfs/stulp15many.pdf

A popular approach to robot skill learning is to initialize a skill through imitation learning, and to then refine and improve the skill through reinforcement learning. In this presentation, I highlight three contributions to this approach:

1) Enabling skills to adapt to task variations by using multiple demonstrations for imitation learning,

2) Improving skills through reinforcement learning based on reward-weighted averaging and black-box optimization with evolution strategies.

3) Using covariance matrix adaptation to automatically tune exploration during reinforcement learning.

Throughout the presentation I show several applications to challenging manipulation tasks on several humanoid robots.

Complex emotions are any emotional state except for Ekman's 6 basic emotions: happy, sad, fear, anger, surprise and disgust. Complex emotions can include mixtures of the basic emotions (e.g. horror), emotions outside the basic emotions (e.g. musical "tension"), and emotions mixed with mental states that are not emotions (e.g. engagement and boredom). Eliciting and recognising complex emotions, and allowing systems to respond to them, will be useful for eLearning, human factors (including vigilance), and responsive systems including human-robot interaction.

In this talk we will present our work towards the elicitation and recognition of conscious or subconscious responses. Engineering and psychological solutions to non-invasively determine such mental states and complex emotions may use movement, posture, facial expression, physiology, and sound. Furthermore, our team has shown that what people suppress is as revealing as what they do. We consider aspects of music listening, movie watching, game playing, quiz-taking, reading, and walking to untangle the complex emotions that can arise. The mental states of engagement and boredom are considered in relation to fidgeting and to Non-Instrumental Movement Inhibition (NIMI), in order to clarify fundamental research problems and direct research design toward improved solutions.

Multiple-point geostatistics (MPS) has received a lot of attention in the last decade for modeling complex spatial patterns. The underlying principle consists in representing spatial variability using training images. A common conception is that a training image can be seen as a prior for the desired spatial variability. As a result, a variety of algorithmic tools have been developed to generate stochastic realizations of spatial processes based on what can be seen broadly as texture generation algorithms.

While the initial applications of MPS were dedicated to the characterization of 3D subsurface structures and the study of geological/hydrogeological reservoirs, a new trend is to use MPS for the modeling of earth surface processes. In this domain, the availability of remote sensing data as a basis to construct training images offers new possibilities for represent complexity with such non-parametric data-driven approaches. Repeated satellite observations or climate models outputs, available at a daily frequency for periods of several years, provide the required patterns repetition for having robust statistics on high-order patterns that vary in both space and time.

This presentation will delineate recent results in this direction, including MPS applications to the stochastic downscaling of climate models, the completion of partially informed satellite images, the removal of noise in remote sensing data, and modeling of complex spatio-temporal phenomena such as precipitation.

Neural networks can learn invariances through many layers of non-linear transformations. Explicit adaptation to speaker or acoustic characteristics can further improve accuracy. A good adaptation technique should: (1) have a compact representation to allow the speaker-dependent parameters to be estimated from small amounts of adaptation data, and minimises storage requirements; (2) operate in an unsupervised fashion without requiring labelled adaptation data; and (3) allow for both test-only adaptation and speaker-adaptive training.

In this talk I'll discuss some approaches to the adaptation of neural network acoustic models - for both speech recognition and speech synthesis - with a focus on some approaches that we have explored in the "Natural Speech Technology" programme: factorised i-vectors, LDA domain codes, learning hidden unit contributions (LHUC), and differentiable pooling.

Basic Multi-biometric Authentication System was thought to have sealed the vulnerabilities and escape route from cyber criminals, but emerging attack patterns have proved us wrong. In spite of their benefits, multi-biometric systems also have peculiar challenges especially circumvention of security strategy. Circumvention refers to how susceptible the system or the presented biometric trait is to spoof attacks and identity fraud. Liveness detection has long been applied as an anti-spoofing mechanism to checkmate spoofing, however its application approach has thrown up more vulnerabilities. We have adopted the Structured Systems Analysis and Design Methodology (SSADM) to assist us understand the weaknesses and propose a solution which integrates liveness detection to halt spoofing. In this seminar, we present a different approach to performing liveness detection in multi-biometric systems to significantly minimize the probability of circumvention and considerably strengthen the overall security strategy of the authentication process.

The Kinematic Theory of rapid human movements and its family of lognormal models provide analytical representations of pen tip strokes, often considered as the basic unit of handwriting. This paradigm has not only been experimentally confirmed in numerous predictive and physiologically significant tests but it has also been shown to be the ideal mathematical description of the impulse response of a neuromuscular system. This proof has led to postulate the LOGNORMALITY PRINCIPLE. In its simplest form, this fundamental premise states that the lognormality of the neuromuscular impulse responses is the result of an asymptotic convergence, a basic global feature reflecting the behaviour of individuals who are in perfect control of their movements. As a corollary, motor control learning in young children can be interpreted as a migration toward lognormality. For the larger part of their lives, healthy human adults take advantage of lognormality to control their movements. Finally, as aging and health issues intensify, a progressive departure from lognormality is occurring. To illustrate this principle, we present various software tools and psychophysical tests used to investigate the fine motor control of subjects, with respect to these ideal lognormal behaviors, from childhood to old age. In this latter case, we focus particularly on investigations dealing with brain strokes, Parkinson and Alzheimer diseases. We also show how lognormality can be exploited in many pattern recognition applications for automatic generation of gestures, signatures, words and script independent patterns as well as CAPTCHA production, graffiti generation, anthropomorphic robot control and even speech modelling. Among other things, this lecture aims at elaborating a theoretical background for many handwriting applications as well as providing some basic knowledge that could be integrated or taking care of in the development of new automatic pattern recognition systems to be used for e-Learning, e-Security and e-Health.

The barriers to production are being lowered so it's a good time to build platforms which make it as simple as possible for everyone to join in and help train and refine language technologies, share their stories and spread the word. Gareth draws on digital storytelling with the BBC, democratic activism via hyperlocal journalism and tools for citizenship to see if there's a new way to corral people's enthusiasm for languages to help build better, more relevant resources.

Music performance is an epitome of complex and creative motor skills. It is indeed striking that musicians can continuously show more physical virtuosity in playing their instrument and can show more creativity in varying their interpretation. Technology-mediated music performance has naturally explored the potential of interfaces and interactions for enhancing musical expression. It is however a difficult (and ill-posed) problem and musical interactive systems cannot yet challenge traditional instruments in terms of expressive control and skill learning.

I believe that an important aspect of the problem relies on the understanding of variability in the performer's movements. I will start my talk by presenting the computational approach based on probabilistic models, particularly suited to handle uncertainty in motion data that stem from noise or intentional variations of the performers. I will then illustrate the potential of the approach in the design of expressive music interactions through experiments with proofs of concept developed and evaluated in the lab; as well as real world applications in artistic projects and in industrial products for consumer devices. Finally, I will present my upcoming EU-funded research project that takes a more theoretical perspective by examining how this approach could potentially be used to infer an understanding of the cognitive processes underlying sensorimotor learning in music performance.

I will present on-going research in my group with a focus on shape understanding with applications to computer vision, robotics and the creative industries. I will principally discuss our recent work on building an algorithmic chain exploiting models of shape derived from the cognitive science literature but relating closely to well-known approaches in computer vision and computational geometry: that of medial descriptors of shape.

Recent relevant publications:

[1] Point-based medialness for 2D shape description and identification

P. Aparajeya and F. F. Leymarie

Multimedia Tools and Applications, May 2015

link.springer.com

[2] Portrait drawing by Paul the robot

P. Tresset and F. F. Leymarie

Computers & Graphics, April 2013

Special Section on Expressive Graphics

www.sciencedirect.com

This talk will present my research path under the overarching theme of enabling non-specialists to create behaviours for autonomous robot. I will start with a short description on my work on scaling up robot autonomy in the context of autonomous construction. I will then focus on modular 3-D mapping using the iterative closest point algorithm and programming by demonstration with a method requiring little user-defined parameters to be tuned. Finally, I will present my work on teaching the computer science concept of event handling using the Thymio mobile robot. I will present quantitative and qualitative results with students of different ages, and will show an experiment exploring the use of augmented reality to provide real-time program tracing. Finally, I will propose a roadmap for future work.

The most common approach to automatic segmentation of speech is, to perform forced alignment using monophone HMM models that have been obtained using embedded reestimation after flat start initialisation. Segmentation using this approach requires large amounts of data and does not work very well for low resource languages. To address the issue of paucity of data, signal processing cues are used to restrict embedded reestimation.

Voice activity detection is first performed to determine the voiced regions in an utterance. Short-term energy (STE) and spectral flux (SF) are computed on intra voiced segments. STE yields syllable boundaries, while locations of significant change in spectral flux are indicative of fricatives, nasals. STE and SF can not be used directly to segment an utterance. Minimum phase group delay based smoothing is performed to preserve these landmarks, while at the same time reducing the local fluctuations. Boundary corrections are performed at the syllable level, wherever it is known that the syllable boundaries are correct. Embedded reestimation of monophone HMM models is then restricted to the syllable boundaries. The boundaries obtained using group delay smoothing results in a number of false alarms. HMM boundaries are used to correct these boundaries. Similarly, spectral flux is used to correct fricative boundaries. Thus, using signal processing cues and HMM reestimation in tandem, robust monophone HMM models are built. These models are then used in an HTS framework to build speech synthesis systems for a number (9 at the time of this presentation) of Indian languages. Both quantitative and qualitative assessments indicate that there is a significant improvement in quality of synthesis.

In another experiment on key word spotting (KWS) in speech, the group delay based syllable boundaries are used to reduce the search space for keyword spotting on Indian English lectures. Appropriate score normalisation based on vowel normalisation in a neural network framework is used to learn the thresholds. An F-score of 72.32% was obtained on a subset of the NPTEL lectures (http://www.nptel.iitm.ac.in).

The main goal of this proposal is to discover a person's life patterns (e.g., where she goes, what she does, how she is and feels and whom she spends time with) namely those situations that repeat themselves, almost but not exactly identical, with regularity, and to exploit this knowledge for improving her quality of life.

The challenge is how to synchronize a sensor and data-driven representation of the world, which is noisy, imprecise and agnostic of the user needs with a knowledge level representation of the world which should be: (i) general, by allowing for the representation and integration of different combinations of sensors and interesting aspects of the user's life and, (ii) adaptive, by representing life happenings at the desired level of abstraction, capturing their progress, and adapting to changes in the life dynamics.

The solution exploits three main components: (i) a methodology and mechanisms for an incremental evolution of a knowledge level representation of the world (e.g., ontologies), (ii) an extension of deep learning to take into account and adapt to the constraints coming from the knowledge level and (iii) a Question Answering (Q/A) service which allows the user to interact with the computer according to her needs and terminology.

Human face-to-face communication is a little like a dance, in that participants continuously adjust their behaviors based on verbal and nonverbal cues from the social context. Today's computers and interactive devices are still lacking many of these human-like abilities to hold fluid and natural interactions. Leveraging recent advances in machine learning, audio-visual signal processing and computational linguistic, my research focuses on creating human-computer interaction (HCI) technologies able to analyze, recognize and predict human subtle communicative behaviors in social context. I formalize this new research endeavor with a Human Communication Dynamics framework, addressing four key computational challenges: behavioral dynamic, multimodal dynamic, interpersonal dynamic and societal dynamic. Central to this research effort is the introduction of new probabilistic models able to learn the temporal and fine-grained latent dependencies across behaviors, modalities and interlocutors. In this talk, I will present some of our recent achievements modeling multiple aspects of human communication dynamics, motivated by applications in healthcare (depression, PTSD, suicide, autism), education (learning analytics), business (negotiation, interpersonal skills) and social multimedia (opinion mining, social influence).

Dr Morrison is currently Scientific Counsel, Office of Legal Affairs, INTERPOL General Secretariat. He is contributing to the European Union funded Speaker Identification Integrated Project (SIIP), which aims to develop investigative and police intelligence solutions for law enforcement agencies, including sharing of data via INTERPOL. He is also an Adjunct Associate Professor, Department of Linguistics, University of Alberta. He has been Director of the Forensic Voice Comparison Laboratory, School of Electrical Engineering & Telecommunication, University of New South Wales; Chair of the Forensic Acoustics Subcommittee, Acoustical Society of America; and a Subject Editor for the journal Speech Communication. He has been involved in forensic casework in Australia and the United States.

Extracting image features such as feature points or edges is a critical step of many Computer Vision systems, however this is still performed with carefully handcrafted methods. In this talk, I will first present a new Machine Learning-based approach to detecting local image features, with application to contour detection in natural images, but also biomedical and aerial images, and to feature point extraction under drastic weather and lighting changes. I will then show that it is also possible to learn efficient object description based on low-level features for scalable 3D object detection.

Medical imaging has enormously increased in importance and volume in medical institutions, particularly 3D tomographic imaging. Via digital analysis the knowledge stored in medical cases can be used for more than a single patient to help decision-making.

This presentation will highlight several challenges in medical image data processing starting with the VISCERAL EU project that evaluates segmentation, lesion detection and similar case retrieval on large amounts of medical 3D data using a cloud-based infrastructure for participants. The description of the MANY project highlights techniques for 3D texture analysis that can be used in a variety of contexts. Finally an overview of the radiology search system of the Khresmoi project will show a combination of the 3D data and the 3D analyses in a multi-modal environment.

He shall present the demonstrator they are installing at the water treatment plant in Martigny. It is based on a redox flow battery able to produce hydrogen to maintain the battery at an optimum state of charge. He shall therefore explain how a redox flow battery works and discuss the advantages and disadvantages. Then, he shall present how our concept of service station for electric cars, with lithium batteries like the Tesla or with hydrogen fuel cells like the Hyundai ix35.

Maria will also present her group at the Hes-so Valais Wallis

While image inpainting is a relatively mature subject whose numerical results are often visually striking, the automatic filling-in of video is still prone to yield incoherent results in many situations. Moreover, the subject is impaired by strong computational bottlenecks. In this talk, we present a patch-based approach to inpaint videos, relying on a global, multi-scale optimization heuristic. Contrarily to previous approaches, the best patch candidates are selected using texture attributes, that are built within a multi-scale video representation. We show that this rationale prevents the usual wash-out of textured and cluttered parts of video. Combined with an appropriate nearest neighbor search and a simple stabilization-like procedure, the resulting approach is able to successfully and automatically inpaint complex situations, including high resolution sequences with dynamic textures and multiple moving objects.

Human communication is inherently multimodal: when we communicate with one another, we use a wide variety of channels, including speech, facial expressions, body postures, and gestures. An embodied conversational agent (ECA) is an interactive character -- virtual or physically embodied -- with a human-like appearance, which uses its face and body to communicate in a natural way. Giving such an agent the ability to understand and produce natural, multimodal communicative behaviour will allow humans to interact with such agents as naturally and freely as they interact with one another, enabling the agents to be used in applications as diverse as service robots, manufacturing, personal companions, automated customer support, and therapy.

To develop an agent capable of such natural, multimodal communication, we must first record and analyse how humans communicate with one another. Based on that analysis, we then develop models of human multimodal interaction and integrate those models into the reasoning process of an ECA. Finally, the models are tested and validated through human-agent interactions in a range of contexts.

In this talk, I will give three examples where the above steps have been followed to create interaction models for ECAs. First, I will describe how human-like referring expressions improve user satisfaction with a collaborative robot; then I show how data-driven generation of facial displays affects interactions with an animated virtual agent; finally, I describe how trained classifiers can be used to estimate engagement for customers of a robot bartender.

This talk presents an ongoing work in language identification for DARPA RATS programme. The talk will describe an application of Neural Network Bottleneck (BN) features in Language Identification (LID). BN features are generally used for Large Vocabulary Speech Recognition in conjunction with conventional acoustic features, such as MFCC or PLP. We compare the BN features to several common types of acoustic features used in the present-day state-of-the-art LID systems. The test set is from DARPA RATS (Robust Automatic Transcription of Speech) program, which seeks to advance state-of-the-art detection capabilities on audio from highly degraded radio communication channels. On this type of noisy data, we show that in average, the BN features provide a 45% relative improvement in the Cavg or Equal Error Rate (EER) metrics across several test duration conditions, with respect to our single best acoustic features.

Several speech technologies like speech transcription, keyword spotting, language identification, speaker identification will be discussed from the architecture point of view. Then cases how these speech technologies are used in call centers, banks, by governmental agencies, or by broad cast service providers for speech data mining, voice analytic or voice biometry will be presented. Each client and use case has some specific requirements on technology, data handling and services. The requirements and its implication on technology development and research will be mentioned.

In this talk I will address the problem of gesture recognition and pose estimation from videos, following two different strategies:

(i) estimation of articulated pose (full body or hand pose) alleviates subsequent recognition steps in many conditions and allows smooth interaction modes and tight coupling between object and manipulator;

(ii) in situations of low image quality (e.g. large distances between hand and camera), obtaining an articulated pose is hard. Training a deep model directly on video data can give excellent results in these situations.

We tackle both cases by training deep architectures capable of learning discriminative intermediate representations. The main goal is to integrate structural information into the model in order to decrease the dependency on large amounts of training data.To achieve this, we propose an approach for hand pose estimation that requires very little labelled data. It leverages both unlabeled data and synthetic data produced by a rendering pipeline. The key to making it work is to integrate structural information not into the model architecture, which would slow down inference, but into the training objective. We show that adding unlabeled real-world samples significantly improves results compared to a purely supervised setting.

In the context of multi-modal gesture detection and recognition, we propose a deep recurrent architecture that iteratively learns and integrates discriminative data representations from individual channels (pose, video, audio), modeling complex cross-modality correlations and temporal dependencies. It is based on multi-scale and multi-modal deep learning. Each visual modality captures spatial information at a particular spatial scale (such as motion of the upper body or a hand), and the whole system operates at two temporal scales. Key to our technique is a training strategy which exploits i) careful initialization of individual modalities; and ii) gradual fusion of modalities from strongest to weakest cross-modality structure.

We present experiments on the "ChaLearn 2014 Looking at People Challenge" gesture recognition track organized in conjunction with ECCV 2014, in which we placed 1st out of 17 teams. The objective of the challenge was to detect, localize and classify Italian conversational gestures from large database of 13858 gestures. The multimodal data included color video, range maps and a skeleton stream.

The talk will be preceded by a brief introduction to the work done in my LIRIS team.

liris.cnrs.fr/christian.wolf/research/gesturerec.html

The Maya hieroglyphic codices provide a rich dataset concerning astronomical beliefs, divinatory practices, and the ritual life of prehispanic Maya cultures inhabiting the Yucatan Peninsula in the years leading up to the Spanish conquest in the early sixteenth century. Structurally, the codices are organized in terms of almanacs and astronomical tables, both of which incorporate several types of data--calendrical, iconographic, and textual--that together allowed Maya scribes to encode complex relationships among deities, dates having ritual and/or celestial significance, and associated activities. In order to better understand these relationships, the Maya Hieroglyphic Codices Database project was initiated to develop sophisticated online research tools to aid in analysis of these manuscripts. Because the Maya scribes did not live in a culture that demanded strict adherence to paradigms that we take for granted when organizing information for electronic search and retrieval, this posed a significant challenge in efforts to discover how data contained in ancient manuscripts could be converted into data structures that facilitated computer searching and information retrieval. This presentation discusses the approaches taken by the author and the architect of the database project to find compromises that enable computer analysis of a set of texts created by scribes more than half a millennium ago, while avoiding the biases inherent in translating knowledge across spatial and cultural divides. The presentation will be made by Dr. Vail; the technical architect to the project, William Giltinan, will be available to answer questions at the conclusion of the lecture.

Learning to recognise the motion or actions of people in video has wide applications covering topic from sign or gesture recognition through to surveillance and HCI. This talk will discuss approaches to video mining, allowing the discovery of weakly supervised spatiotemporal signatures such as actions embedded in video or Signs/facial motion weakly supervised by language. Whether the task is recognising an atomic action of an individual or their implied activity, the continuous multichannel nature of sign language recognition or the appearance of words on the lips, all approaches can be categorised at the most basic level as the learning and recognition of spatio-temporal patterns. However, in all cases, inaccuracies in labelling and the curse of dimensionality lead us to explore new learning approaches that can operate in a weakly supervised setting. This talk will discuss the adaptation of mining to the video domain and new approaches to learning spatiotemporal signatures covering a broad range of application areas such as facial feature extraction and regression, lip reading, activity recognition and sign and gesture recognition in both 2D and 3D.

Multiparty conversations are characterized by various degrees of participants' engagement and group involvement. Humans are able to detect and interpret these degrees, basing their perception on multimodal cues. The automatic detection, in particular for bigger groups of people, poses however many challenges. In this talk, I will mainly focus on a study in which we analysed group-behaviour in an eight-party, multimodal corpus. We propose four features that summarize different aspects of eye-gaze patterns and allow us to describe individual engagement as well as group involvement in time. Our overall aim is to build a system which is able to foster group involvement.

In addition, I will briefly comment on 2 studies in which we use the robot head Furhat to advance in this direction. Furhat is a robotic head that combines state-of-the-art facial animation with physical embodiment in order to facilitate multi-party dialogues with robots.

The Web continues to grow and evolve very fast, changing our daily lives. This activity represents the collaborative work of the millions of institutions and people that contribute content to the Web as well as more than two billion people that use it. In this ocean of hyperlinked data there is explicit and implicit information and knowledge. But how is the Web? What are the activities of people? How content is generated? Web data mining is the main approach to answer these questions. Web data comes in three main flavors: content (text, images, etc.), structure (hyperlinks) and usage (navigation, queries, etc.), implying different techniques such as text, graph or log mining. Each case reflects the wisdom of some group of people that can be used to make the Web better. For example, user generated tags in Web 2.0 sites. In this presentation we explore the wisdom of crowds in relation to several dimensions such as bias, privacy, scalability, and spam. We also cover related concepts such as the long tail of the special interests of people, or the digital desert, content that nobody sees.

Social signal processing is an emerging research domain with rich and open fundamental and applied challenges. In this talk, I'll focus on the development of social signal processing techniques for real applications in the field of psycho-pathology. I'll overview recent research and investigation methods allowing neuroscience, psychology and developmental science to move from isolated individuals paradigms to interactive contexts by jointly analyzing behaviors and social signals of partners. From the concept of interpersonal synchrony, we'll show how to address the complex problem of evaluating children with pervasive developmental disorders. These techniques are also demonstrated in the context of human-robot interaction by a new way of using robots in autism (moving from assistive devices to clinical investigations tools). I will finish by closing the loop between behaviors and physiological states by presenting new results on oxytocin and proxemics during early parent-infant interactions.

In this talk, we would like to introduce our past trials on the human-robot / human-agent interactions especially focusing on the user's attention and the gaze communication.

At first, in "Communication on Anthropomorphic Media", Dr. Tomoko Yonezawa will make a presentation on the past researches on gaze-communication and the robot's behaviors. Additionally, she will talk about her current research on touch interaction between human and wearable robot.

Second, in "Presences with Avatars' Appearances Attached to Tex Communication in Twitter", Ms. Yukari Nakatani introduces her research theme on the representations of multiple virtual agents for sustainable communications in SNS.

Finally we introduce the students' researches in our laboratory with some presentation movies.

In this talk Martin Volk will present the Text+Berg project, an initiative to digitize and annotate all the yearbooks of the Swiss Alpine Club from its start in 1864 until today. The resulting corpus of 40 million words contains texts in the 4 official Swiss languages, with a large parallel part in German and French. Based on these translations Martin's group works on domain-specific machine translation systems, but also on search systems for word-aligned parallel corpora as a new resource for translators and linguists. Most of the yearbooks (more than 100'000 pages) were scanned and converted to text at the University of Zurich. Martin Volk will share his experiences on automatically correcting OCR errors as well as on dealing with tokenization, lemmatization and PoS-tagging issues in a corpus that spans 150 years and multiple languages. He will also report on the Text+Berg toponym detection and classification as well as person name recognition and tagging of temporal expressions. Recently the group has released Kokos, a system for collaborative correction of OCR errors in the yearbooks of the 19th century (http://kokos.cl.uzh.ch) and asked the SAC members to join in creating a clean corpus.

This talk will overview recent progress and open challenges in human action recognition. Specifically, I will focus on the three problems of (i) action representation in video, (ii) weakly-supervised action learning and (iii) ambiguity of action vocabulary. To the first problem, I will overview local feature methods providing state-of-the-art results on current action recognition benchmarks. Motivated by the difficulty of large-scale video annotation, I will next present our recent work on weakly-supervised action learning from video and corresponding video scripts. I will finish by highlighting limitations of the standard action classification paradigm and will show some of our work addressing this problem.

Future public displays have the potential to become much more than a simple digital signage -- they can form the basis for a novel communication medium. By interconnecting displays and opening them up to applications and content from a wide range of sources, they can not only support individuals and their communities, but also increase their relevance and ultimately their economic benefits. Ultimately, open display networks could have the same impact on society as radio, television and the Internet. In this talk, I will briefly summarize this vision and its related challenges, in particular with respect to privacy and trust, and present the work that we did in this area in the context of a recently finished FET-Open project titled "PD-Net".

Current large scale web applications pose enormous and dynamic processing and storage requirements. Failures of any type are common in current datacenters, partly due to the higher scales of the data stored. As data scales up, its availability becomes more complex, while different availability levels per application or per data item may be required. At the same time, cloud infrastructures should be able to effectively deal with the elastic nature of these applications in an autonomic manner. To make things worse, as clients get increasingly averse to vendor lock-in and data unavailability risks, client data has to be efficiently split across clouds. In this talk, we briefly discuss three very effective cloud resource management solutions that deal with the different aforementioned requirements: Skute, Scarce and Scalia. Skute is a self-managed key-value store that dynamically allocates the resources of a data cloud to several applications in a cost-efficient and fair way. Scarce is a decentralized economic approach for dynamically adapting the cloud resources of various applications, so as to statistically meet their SLA performance and availability goals in the presence of varying loads or failures. Scalia is a cloud storage brokerage solution that continuously adapts the placement of data based on its access pattern, subject to optimizations objectives and data placement constraints, such as storage costs and vendor lock-in avoidance.

The CSIRO Division of Computational Informatics (CCI) aims to transform information and decision making to enhance productivity, foster collaboration and deliver impact through services across a wide range of sectors. CCI researchers have in-depth expertise in applying statistical and mathematical methods in a variety of scien- tific fields including, among others, environmental and agricultural informatics, wireless sensor networks, in- formation and communication technologies for healthcare and clinical treatment, development of early screening tests for Alzheimer's disease (bioinformatics), computational and simulation sciences (high perform- ance computing), as well as statistical modelling for seasonal climate forecasting and complex biogeochemical systems (e.g. marine environments).

This presentation will focus on some aspects of the research being carried out at CCI on applications of statisti- cal and computational methods for environmental modelling and natural resource management. In particular, I will present an overview of my recent work on the following topics:

- multi-sensor integration of remote sensing data for large-scale vegetation mapping and monitoring,

- data fusion methods for water resources assessment using ground-based and remote sensing data, and

- spatial modelling of extreme weather events and associated risks in the context of a changing climate.

These projects involve several aspects of multivariate Bayesian modelling and analysis (spatial and temporal), computational simulation methods (Markov chain Monte Carlo), issues of data quality and continuity, as well as scientific dissemination and stakeholder engagement.

Robots in current industrial settings reproduce repetitive movements in a stiff and precise manner, with sensory information often limited to the role of stopping the motion if a human or object enters the robot's workspace. The new developments in robot sensors and compliant actuators bring a new human-centric perspective to robotics. An increase of robots in small and medium-sized enterprises (SMEs) is predicted for the next few years. Products in SMEs are characterized by small batch sizes, short life-cycles and end-user driven customization, requiring frequent re-programming of the robot. SMEs also often involve confined spaces, so that the robots must work in safe collaboration with the users by generating natural movements and anticipating co-workers' movements with active perception and human activity understanding.

Interestingly, these robots are much closer to human capabilities in terms of compliance, precision and repeatability. In contrast to previous technology, the planning, control, sensing and interfacing aspects must work hand-in-hand, where the robot is only one part of a broader robotics-based technology. The variety of signals to process and the richness of interaction with the users and the environment constitute a formidable area of research for machine learning.

Current programming solutions used by the leading commercial robotics companies do not satisfy the new requirements of re-using the same robot for different tasks and interacting with multiple users. The representation of manipulation movements must be augmented with forces (for task execution, but also as a communication channel for collaborative manipulation), compliance and reactive behaviors. An attractive approach to the problem of transferring skills to robots is to take inspiration from the way humans learn by imitation and self-refinement.

I will present a task-parametrized model based on dynamic movement primitives and Gaussian mixture regression to exploit the local correlations in the movement and the varying accuracy requirements of the task. The model is used to devise a controller for the robot that can adapt to new situations and that is safe for the surrounding users. Examples of applications with a compliant humanoid and with gravity-compensated manipulators will be showcased.

Because of limited resources (e.g. number and type of cameras, amount of time to focus on an individual, real-time processing power), using intelligence within standoff biometric capture systems can help in determining which individuals to focus on and for how long. Benchmark datasets available to the general research community are needed designing a stand-off multimodal biometric system. The overall goal of the research to investigate the fusion approaches to measure face, iris, and voice through experiments for identity at distances from 10 to 25 meters. This research includes a growing corpus of data, entitled Quality in Face and Iris Research Ensemble-Q-FIRE dataset which includes the following: (1) Q-FIRE Release 1 (made available in early 2010) is composed of 4T of face and iris video for 90 subjects out to 8.3meters (25 feet) with controlled quality degradation. (2) Release 2 is an additional 83 subjects with same collection specifications. Release 1 and 2 were used by NIST in IREX II: Iris Quality Calibration and Evaluation (IQCE). (3) Last, an extension of the dataset has been collected with unconstrained behavior of subjects on the same set of subjects, entitled Q-FIRE Phase II Unconstrained out to 8.3 meters. In this talk, the datasets will be described as well as results of experiments fusing face and iris scores with quality.

http://people.clarkson.edu/~sschucke/

In this talk I will discuss issues related to multimodal interaction with intelligent agents, and in particular, present the Nao Wikitalk, an application that enables the user to query Wikipedia via the Nao robot. The robot can talk about an unlimited range of topics, so it supports open-domain conversations using Wikipedia as a knowledge source. The robot suggests some topics to start with, and the user can shift to related topics by speaking the topic names after the robot mentions them. The user can also switch to a totally new topic by spelling the first few letters. The challenge in presenting Wikipedia information is how to convey its structure to the user so that she can understand what is new information, and how to navigate in the topic structure. In Wikipedia, new relevant information is marked with hyperlinks to other entries, and the robot's interaction capabilities have been extended so that it signals these links non-verbally while reading the text. As well as speaking, the robot uses gestures, nods and other multimodal signals to enable clear and rich interaction. Gesture and posture changes can also be used to manage turn-taking, and to add liveliness to the interaction in general. To manage the interaction in a smooth way, it is also important to capture the user's emotional and attentional state. For this, we have experimented with gazing and face tracking to infer the user's interest level. The Nao WikiTalk system was evaluated by comparing the users' expectations with their experience of the robot interaction. In many respects the users had high expectations regarding the robot's interaction capabilities, but they were impressed by the robot's lively appearance and natural gesturing.

Image processing in bio-microscopy is no longer confined to the post-processing stage, but has gained wide acceptance as an integral part of the image acquisition process itself, as it allows overcoming hard limits set by instrumentation and biology. In this talk, I will present my lab's efforts to image dim and highly dynamic biological samples by boosting the temporal and spatial resolution of optical microscopes via software solutions and modified imaging protocols. Focusing on spatio-temporal image registration strategies to build 3D+time models of samples with repetitive motions, a superresolution algorithm to reconstruct image sequences from multiple low temporal resolution acquisitions, and a fast multi-channel deconvolution algorithm for multi-view imaging, I will illustrate the central role signal processing can play to advance bio-imaging. I will share the approaches we implemented in my group to rapidly bring new ideas from theory to full deployment in remote biology labs , where our tools can be applied with a variety of microscopy types. Finally, I will speculate on the future of image processing in bio-microscopy and suggest areas where efforts may be most rewarding.

Critical cyber-physical environments such as the ones found in many healthcare settings or on the flight deck of modern airplanes are built on complex systems characterized by important properties spanning the physical and digital world, and centered on human activity. In order to properly understand this critical activity, researchers need to first understand the context and environment in which the activity is situated. Central in those environments is often interaction with the available technology and the communication between the individuals, both of which often involve multiple parallel modalities. Only an in-depth understanding of the properties of these multimodal distributed environments can inform the design and development of multimodal human-centered computing.

After presenting an overview of my current research in human-centered computing, this talk will present some of the challenges and proposed solutions in terms of technologies and theoretical frameworks for collecting and making sense of rich multimodal data in two critical cyber-physical environments: the cockpit of a Boeing 787 airplane, and the medical office. The talk will explain how the combination of a range of data collection devices such as depth cameras, eye tracking, digital-pens, and HD video cameras, combined with powerful data visualization and a flexible analysis suite, allows in-depth understanding of those complex environments. I will end with a discussion of cutting-edge multimodal technology and how devices such as depth cameras and wearable augmented reality glasses open up a range of opportunities to develop new technology for knowledge workers of critical cyber-physical environments.

BIO: Dr. Nadir Weibel is a Research Assistant Professor in the Department of Computer Science and Engineering at the University of California San Diego (UCSD), where he is teaching human-computer interaction and ubiquitous computing. His research is situated at the intersection of computer science, cognitive science, communication, health and social sciences. Dr. Weibel investigates tools, techniques and infrastructure supporting the deployment of innovative interactive multimodal and tangible devices in context, and studies the cognitive consequences of the introduction of this technology in the everyday life. Current work focuses on interactive physical-digital systems that exploit pen-based and touch-based devices, depth-cameras, wearable and mobile devices, in the setting of critical populations such as healthcare and education. Dr. Weibel is author of more than 45 publications on these topics. His work has been funded by the Swiss National Science Foundation, the European Union, Boeing, the US NSF, NIH and AHRQ.

Humans do not think like computers. Our minds are 'designed' for us to function as embodied beings in the world in ways that are: 1. Physical-Spatial; 2. Temporal-Dynamic; 3 Social-Cultural; and 4. Affective-Emotional. These aspects of embodiment give us four lenses to understand the embodied mind and how computation/technology may support its function. I adopt a two-pronged to human-computer interaction research by first harnessing technological means to contribute to the understanding of how embodiment ultimately ascends into mind, and second, to inform the design and engineering of technologies that support and augment human higher psychological functions of learning, sensemaking, creating, and experiencing.

In line with the first approach, I shall first show how language, as a core human capacity, is rooted in human embodied function. We will see that mental imagery shapes multimodal (gesture, gaze, and speech) human discourse. In line with the second approach, I shall then present an assemblage of interactive projects that illustrate how our concept of human embodiment can inform technology design through the light of our four lenses. Projects cluster around three application domains, namely 1. Technology for special populations (e.g. mathematics instruction and reading for the blind, games for older adults); 2. Learning and Education (e.g. learning and knowledge discovery through device/display ecologies, creativity support for children); and 3. Experience (e.g. socially-based information access, experience of images, affective communication).

This talk focuses on technology related innovation within companies like Nokia - and covers the flow from early phase ideas towards the technology transfer and productization. Further, the role of research partnerships as a part of the overall innovation process is discussed. More specifically, various modes of industry-academia collaboration and related drivers for each of them are briefly covered. Aspects like, technology licensing are touched briefly too.

More particularly this presentation focuses on collaboration between Idiap and Nokia as a case study and investigates the role of Idiap-Nokia interactions from the perspective of overall innovation chain. This part covers e.g. Idiap's contribution on Nokia's Call for Research Proposals in 2008, joint initiatives around mobile data (Lausanne Data Collection Campaign 2009-2012 and Mobile Data Challenge 2011-2012) as well as bi-lateral research projects.

There are almost as many mobile phones in the world as humans. The mobile phone is the piece of technology with the highest levels of adoption in human history. We carry them with us all through the day (and night, in many cases). Therefore, mobile phones have become sensors of human activity in the large scale and also the most personal devices.

In my talk, I will present some of the work that we are doing at Telefonica Research in the area of mobile computing, both in terms of analyzing and understanding large-scale human behavioral data from mobile traces and in designing novel mobile systems in the areas of healthcare, education and information access.

A central question in mobile computing is how do you test mobile applications, that depend on real context, in real environments with real users? User studies done in lab environments are frequently insufficient to understand the real-world interactions between user context, environmental factors, application behaviour, and performance results. In this talk, I will describe LiveLabs, a new 5 year project that started at the Singapore Management University in early 2012. The goal of LiveLabs is to convert four real environments, the entire Singapore Management University campus, a popular resort island, a large airport, and a popular shopping mall, into living testbeds where we instrument both the environment and the cell phones of opted-in participants (drawn from the student population and members of the public). We can then provide 3rd party companies, and researchers the opportunity to test their mobile applications and scenarios on the opted-in participants -- on their real phones in the four real environments described above. LiveLabs will provide the software necessary to collect network statistics and any necessary context information. In addition, LiveLabs will provide software and mechanisms to ensure that privacy, proper participant selection, resource management, and experimental results and data are maintained and provided on a need-to-know basis to the appropriate parties.

I will describe the broad LiveLabs vision and identify the key research challenges and opportunities. In particular, I will highlight our current insight into indoor location tracking, dynamic group and queue detection, and energy aware context sensing for mobile phones.

In our daily lives, we cannot help but communicate with people. Aside from organised and more structured communication like emails, meetings, or phone calls, we communicate instantaneously and often in adhoc, freely formed groups where it is not known beforehand how long the conversation will last for, who will be in the conversation, or what it will be about. In crowded settings like a

conference, for example, this type of conversing group exists and who gravitates towards whom tells us a lot about the relationship between the members of the group. In this talk, I will discuss the challenges of this problem, solutions, and open questions of this emerging topic.

http://biometrics.cse.msu.edu

http://scholar.google.com/citations?user=g-_ZXGsAAAAJ&hl=en

If you are like many people, navigating the complexities of everyday life depends on an array of cards and passwords that confirm your identity. But lose a card, and your ATM will refuse to give you money. Forget a password, and your own computer may balk at your command. Allow your card or passwords to fall into the wrong hands, and what were intended to be security measures can become the tools of fraud or identity theft. Biometrics - the automated recognition of people via distinctive anatomical and behavioral traits has the potential to overcome many of these problems.

Biometrics is not a new idea. Pioneering work by several British scholars, including Fauld, Galton and Henry in the late 19th century established that fingerprints exhibit a unique pattern that persists over time. This set the stage for the development of Automatic Fingerprint Identification Systems that are now used by law enforcement agencies worldwide. The success of fingerprints in law enforcement coupled with growing concerns related to homeland security, financial fraud and identity theft has generated renewed interest in research and development of biometric systems. It is, therefore, not surprising to see biometrics permeating our society (laptops and mobile phones, border crossing, civil registration, and access to secure facilities). Despite these successful deployments, biometrics is not a panacea for human recognition. There are challenges related to data acquisition, image quality, robust matching, multibiometrics, biometric system security and user privacy. This talk will introduce three challenging problems of particular interest to law enforcement and border crossing agencies: (i) face sketch to photo matching, (ii) scars, marks & tattoos (SMT) and (iii) fingerprint obfuscation.

Enabling computers to understand human behavior has the potential to revolutionize many areas that benefit society such as clinical diagnosis, human computer interaction, and social robotics. A critical element in the design of any behavioral sensing system is to find a good representation of the data for encoding, segmenting, classifying and predicting subtle human behavior. In this talk I will propose several extensions of Component Analysis (CA) techniques (e.g., kernel principal component analysis, support vector machines, spectral clustering) that are able to learn spatio-temporal representations or components useful in many human sensing tasks.

In the first part of the talk I will give an overview of several ongoing projects in the CMU Human Sensing Laboratory, including our current work on depression assessment from videos. In the second part, I will show how several extensions of CA methods outperform state-of-the-art algorithms in problems such as facial feature detection and tracking, temporal clustering of human behavior, early detection of activities, weakly-supervised visual labeling, and robust classification. The talk will be adaptive, and I will discuss the topics of major interest to the audience.

Conventional speech analysis techniques are based on estimating the spectral content of relatively short (about 10-20 ms) segments of the signal. However, an alternate way to describe a speech signal is a long-term summation of amplitude modulated frequency bands, where each frequency band consists of a smooth envelope (gross structure) modulating a carrier signal (fine structure). We develop an auto-regressive (AR) modeling approach for estimating the smooth envelope of the sub-band signal. This model, referred to as frequency domain linear prediction (FDLP), is based on the application of linear prediction on discrete cosine transform of the signal and it describes the perceptually dominant peaks in the signal while removing the finer details. This suppression of detail is useful for developing a parametric representation of speech/audio signals. In this talk, I will also show several applications of the FDLP model for speech and audio processing systems.

In the last leg of the talk, I will focus on our recent efforts at IBM for speech analysis in noisy radio communication channels. This will highlight the challenges involved along with a few solutions addressing parts of the problem.

User authentication to online services is at a cross-roads. Attacks are increasing, and current authentication schemes are no longer able to provide adequate protection. The time has come to include the third factor of authentication, and start using biometrics to authenticate people. However, despite signficant progress in biometrics, they still suffer from a major mode of attack: replay attacks, where biometric signals may be captured previously and reused. Replay attacks defeat all current liveness tests. Current literature recognises replay attacks as a significant issue, but there are no practical and tested solu- tions available today. The purpose of this research is to improve authentication to online services by including a face recognition biometric, as well as providing one solution to the replay attack problem for the proposed face recognition system. If this research is success- ful, it will enable the use of enhanced authentication mechanisms on mobile devices, and open new research into methods of addressing biometric replay attacks.

Human communication relies on signals like speech, mimics, or gestures and the interpretation of these signals seems to be innate to humans. In contrast, human interaction with machines and thus human communication mediated through machines is far from being natural. To date, it is restricted to few channels and the capabilities of machines to interpret human signals are still very limited.

At the Cognitive Systems Lab (CSL) we explore human-centered cognitive systems to improve human-machine interaction as well as machine-mediated human communication. We aim to benefit from the strength of machines by departing from just mimicking the human way of communication. Rather we focus on considering the full range of biosignals emitted from the human body, such as electrical biosignals like brain and muscle activity. These signals can be directly measured and interpreted by machines, leveraging emerging wearable, small and wireless sensor technologies. Using these biosignals offers an inside perspective on human mental activities, intentions, or needs and thus complement the traditional way of observing humans from the outside.

In my talk I will discuss ongoing research on "Biosignals and Interfaces" at CSL, such as speech recognition, silent speech interfaces that rely on articulatory muscle movement, and interfaces that use brain activity to determine users' mental states, such as task activity, cognitive workload, attention, emotion, and personality. We hope that our research will lead to a new generation of human centered systems, which are completely aware of the users' needs and provide an intuitive, efficient, robust, and adaptive input mechanism to interaction and communication.

Chris will be presenting his doctoral thesis as the result of a research effort performed in two fields of speech technology, i.e., speech recognition and mispronunciation detection. Although the two areas are clearly distinguishable, the proposed approaches share a common hypothesis based on psychoacoustic processing of speech signals. The conjecture implies that the human auditory periphery provides a relatively good separation of different sound classes. Hence, it is possible to use recent findings from psychoacoustic perception together with mathematical and computational tools to model the auditory sensitivities to small speech signal changes.

Blaise will be talking about his doctoral research on the acoustic-to-articulatory inversion problem. The main aim of his Ph. D. was to investigate the use of additional constraints (phonetical and visual) to improve the realism of the solutions found by an existing inversion framework. This research was conducted in LORIA, Nancy, France, under the supervision of Yves Laprie.