[Show/Hide Left Column]

Development of an HTML5 / Javascript touch player to facilitate the annotation of audiovisual documents 

Project : TouchPlayer
Web page :
Source coding :
Axis : DataSense
Porteur(s) du projet : Hervé Bredin
Candidate : Benjamin MAURICE
Institution : LIMSI
Engagement : Janvier 2018 - Décembre 2018

Context :
Many of the DataSense research teams' work is based on "learning" approaches that require a prior phase of data collection and annotation (often manual). In particular, several teams are working on so-called "multimedia" data of the "video", "image" or "audio" type. Many tools exist to annotate this type of content (Anvil, ELAN, ...) but all rely on interaction via the mouse, which can quickly become tedious when it comes to hijacking or tracking objects. 

Objective :
We propose to develop touch interfaces to meet the concrete needs of the various project partners, while ensuring that the software base is generic enough to be extended to other types of annotations. This project targets mainly researchers (in multimedia, computer vision, etc.) whose approaches are based on a learning phase requiring annotated audiovisual data. Tactile annotation is obviously not limited to audiovisual content and it will be interesting to see how the new tactile annotation paradigms that will result from it could be adapted to text annotation or audio recording, for example. In the longer term, our ambition is to offer this tool to international evaluation campaigns (such as MediaEval for example) and/or collaborative crowdsourcing (since the same sample is presented to several users, it is then possible to keep only annotations that are sufficiently congruent). 

Expected results :
  • Player vidéo HTML5 générique
    • déclenchant des événements Javascript en réponse aux interactions tactiles (e.g. rotate, pinch, press, pan, tap, double-tap, swipe, etc.)
    • affichant en sur-impression dans une couche SVG des annotations spatio-temporelles existantes (e.g. détourage de visage)
  • Documentation (sur Github Pages) de son API 
* Interface web tactile basée sur ce player tactile permettant le découpage temporel de flux vidéo
  • Interface web tactile basée sur ce player tactile permettant le suivi d’objets (e.g. visage) dans une vidéo
  • Interface web tactile basée sur ce player tactile permettant le détourage d’objets dans une image 

Added value :
Through this doctoral mission, the student will be able to familiarize himself with many new technologies very popular in the world of web development (Javascript, WebComponent, etc..) - which will be a significant asset for the young researcher who aims an industrial career. This funding also makes it possible to perpetuate the work carried out in the Camomile project and to relaunch the annotation activities which were on forced pause since the end of the initial CHISTERA project.