title: Weakly supervised training of a sign language recognition system using multiple instance learning density matrices.
abstract: A system for automatically training and spotting signs from continuous sign language sentences is presented. We propose a novel multiple instance learning density matrix algorithm which automatically extracts isolated signs from full sentences using the weak and noisy supervision of text translations. The automatically extracted isolated samples are then utilized to train our spatiotemporal gesture and hand posture classifiers. The experiments were carried out to evaluate the performance of the automatic sign extraction, hand posture classification, and spatiotemporal gesture spotting systems. We then carry out a full evaluation of our overall sign spotting system which was automatically trained on 30 different signs.
authors.affiliation: Computer Science Department, National University of Ireland Maynooth, Maynooth, Ireland. dankelly@cs.nuim.ie
authors.forename: Daniel
authors.initials: D
authors.lastname: Kelly
authors.forename: John
authors.initials: J
authors.lastname: Mc Donald
authors.forename: Charles
authors.initials: C
authors.lastname: Markham
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D006225:Hand / Q000033:anatomy & histology*; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D010363:Pattern Recognition, Automated / Q000379:methods*; D010781:Photography / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012813:Sign Language