A Hierarchical KeyFrame User Interface for Browsing Video over the Internet

A Hierarchical KeyFrame User Interface
for Browsing Video over the Internet

Purpose:

This page aims at presenting an interactive content-based video browser allowing non linear and hierarchical navigation of video over the Internet through multiple levels of key-frames that provide a visual summary of video content. Our method is based on an XML framework, dynamically generated parameterized XSL style sheets, and SMIL. The architecture is designed to incorporate additional recognized features (e.g. from audio) in future versions. A user study indicates that this browsing interface is more comfortable to use and approximately three times faster for locating remembered still images within videos compared to the simple VCR controls built into RealPlayer.

< 30 seconds RM Video that demonstrates the user interface (download RealPlayer) >

The mmm server on which the hierarchical video browser is on supports browsing, playing, retrieving, and adding of recorded multimodal data files.

Browsing We use the C++ Computer Vision OpenCV Intel (C) library, mplayer video codecs and Matlab for video analysis. The Graphical User Interface is based on XML/XSL standard, and Perl/CGI for dynamic web page generation and fast user interaction. By clicking on + and - buttons, the hierarchy expands or collapses, like a standard file manager.

hierarchical video browser
example of file manager

Playing:

SMIL is used to play low quality video streamed over the Internet using RTSP and Real player. Each SMIL presentation is generated dynamically based on the media clips selected by the user and the endpoint platform (Windows or Unix).

Client-Server Architecture

The XML file is transformed into an HTML user interface under the control of a dynamically generated XSL style sheet (XSL spec, 2001). The transformation into HTML can run on the server or in the web browser (e.g. Mozilla or Internet Explorer s XSLT processor). The XSL style sheet is dynamically generated by a Perl/CGI script (XSLmaker) based on user input parameters which specify the video name and level at which to expand or collapse the hierarchy. Dynamic generation of XSL style sheets under user control is our approach to flexible presentation of multiple video browsing interfaces based on the same underlying XML data.

Small user experiment:
- Task: Locate a target image in a video.
- Material: 2 home videos (from Kodak database).
- 7 subjects without particular knowledge on video analysis.
- RealPlayer versus full Video Browser.
- 2 target images: randomly selected frames not included in the keyframe set.
- Check last section of the paper for detailed results.

For more information, you may want to browse the PDF slides used at INTERACT 03 talk. Also, related work on video browsing can be found here.

People:

Maël Guillemot, IDIAP, Switzerland.
Pierre Wellner, IDIAP, Switzerland.
Daniel Gatica-Pérez, IDIAP, Switzerland.
Jean-Marc Odobez, IDIAP, Switzerland.

Publications:

Maël Guillemot, Pierre Wellner, Daniel Gatica Perez, Jean-Marc Odobez A Hierarchical Keyframe User Interface for Browsing Video over the Internet, in Proceedings of the ninth IFIP TC13 international conference on Human-Computer Interaction (INTERACT 2003), Zurich, Switerland, Sept. 2003.

Jean-Marc Odobez, Daniel Gatica Perez and Maël Guillemot Spectral Structuring Of Home Videos, in Proceedings of International Conference on Image and Video Retrieval (CIVR 03 Beckman Institute, University of Illinois at Urbana-Champaign, Urbana, IL, US, July 2003.

Jean-Marc Odobez, Daniel Gatica Perez and Maël Guillemot Video Shot Clustering using Spectral Methods, in Proceedings of the International Workshop on Content-Based Multimedia Indexing (CBMI 2003) (CBMI 03), IRISA, Rennes, France, Sept. 2003

Maël Guillemot Algorithms For Video Structuring, Master thesis, Université de Rennes1, IDIAP COM report , Sept. 2002.

Sponsors:

Interactive Multimodal Information Management -- (IM)2, part of the swiss National Centre of Competence in Research (NCCR).

EU project -- Multi-Modal Meeting Manager -- M4