title: Scalable Semi-Automatic Annotation for  Multi-Camera Person Tracking.
abstract: This paper proposes a generic methodology for the semi-automatic generation of reliable position annotations for evaluating multi-camera people-trackers on large video data sets. Most of the annotation data are automatically computed, by estimating a consensus tracking result from multiple existing trackers and people detectors and classifying it as either reliable or not. A small subset of the data, composed of tracks with insufficient reliability, is verified by a human using a simple binary decision task, a process faster than marking the correct person position. The proposed framework is generic and can handle additional trackers. We present results on a data set of $sim 6$ h captured by 4 cameras, featuring a person in a holiday flat, performing activities such as walking, cooking, eating, cleaning, and watching TV. When aiming for a tracking accuracy of 60 cm, 80% of all video frames are automatically annotated. The annotations for the remaining 20% of the frames were added after human verification of an automatically selected subset of data. This involved $sim 2.4$ h of manual labor. According to a subsequent comprehensive visual inspection to judge the annotation procedure, we found 99% of the automatically annotated frames to be correct. We provide guidelines on how to apply the proposed methodology to new data sets. We also provide an exploratory study for the multi-target case, applied on the existing and new benchmark video sequences.
authors.forename: Jorge
authors.initials: J
authors.lastname: Niño-Castañeda
authors.forename: Andrés
authors.initials: A
authors.lastname: Frías-Velázquez
authors.forename: Nyan Bo
authors.initials: NB
authors.lastname: Bo
authors.forename: Maarten
authors.initials: M
authors.lastname: Slembrouck
authors.forename: Junzhi
authors.initials: J
authors.lastname: Guan
authors.forename: Glen
authors.initials: G
authors.lastname: Debard
authors.forename: Bart
authors.initials: B
authors.lastname: Vanrumste
authors.forename: Tinne
authors.initials: T
authors.lastname: Tuytelaars
authors.forename: Wilfried
authors.initials: W
authors.lastname: Philips
mesh_terms: D000465:Algorithms; D066289:Data Curation / Q000379:methods*; D006802:Human Activities / Q000145:classification*; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D014741:Video Recording / Q000379:methods*