Visual Focus-of-attention

Poster Watchers

Idiap is working on innovative approach to solve the problem of tracking the wandering  visual  focus-of-attention (VFOA)  of  multiple people,  an important   problem  with   many  applications   in   human  behavior understanding in different areas such as human-computer interaction, robot-human interaction,  and surveillance. 

Idiap has developed a  method which  can be  used to  build a  Nielsen-like rating  system for outdoor  display advertisements.  It can  count the number of  people who have actually  viewed a given  ad, determine the percentage  of people  who have  actually looked  at the  ad  to total number of  people exposed to  it, estimate the  time-of-attention (how long people looked at the add), and the number of times they looked at the ad.

The method  works by simultaneously  inferring the number of  people in the  scene,  track their  body  and  head  locations, and  their  head pose/orientation, and is based on recent advances achieved at Idiap on multi-object tracking and head pose estimation.

In this video, the goal is to detect whether people are actually looking at the advertisement poster (the bottom of the advertisement can be seen at the top of the image)

• the long green box indicate the estimated body position;

• the green box on the top indicate the head location.

• The arrow indicates the head pointing direction, i.e. how is the head oriented and from which it is decided whether the person is looking at the add (in this case the arrow is in red) or not (green arrow).

 JointRec.is1001c

This video shows the automatic estimation of two aspects of social verticality (status and dominance) in small-group meetings using nonverbal cues.