Semi-supervised video segmentation using tree structured graphical models.
We present a novel patch-based probabilistic graphical model for semi-supervised video segmentation. At the heart of our model is a temporal tree structure that links patches in adjacent frames through the video sequence. This permits exact inference of pixel labels without resorting to traditional short time window-based video processing or instantaneous decision making. The input to our algorithm is labeled key frame(s) of a video sequence and the output is pixel-wise labels along with their confidences. We propose an efficient inference scheme that performs exact inference over the temporal tree, and optionally a per frame label smoothing step using loopy BP, to estimate pixel-wise labels and their posteriors. These posteriors are used to learn pixel unaries by training a Random Decision Forest in a semi-supervised manner. These unaries are used in a second iteration of label inference to improve the segmentation quality. We demonstrate the efficacy of our proposed algorithm using several qualitative and quantitative tests on both foreground/background and multiclass video segmentation problems using publicly available and our own datasets.
University of Cambridge, Cambridge.
Vijay
V
Badrinarayanan
Ignas
I
Budvytis
Roberto
R
Cipolla
D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D008962:Models, Theoretical; D010363:Pattern Recognition, Automated / Q000379:methods*; D010781:Photography / Q000379:methods*; D012815:Signal Processing, Computer-Assisted; D013382:Subtraction Technique; D014741:Video Recording / Q000379:methods*