Beyond novelty detection: incongruent events, when general and specific classifiers disagree.
Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of- Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.
School of Computer Science and Engineering, Hebrew University of Jerusalem, Jerusalem 91904, Israel. daphna@cs.huji.ac.il
Daphna
D
Weinshall
Alon
A
Zweig
Hynek
H
Hermansky
Stefan
S
Kombrink
Frank W
FW
Ohl
Jörn
J
Anemüller
Jörg-Hendrik
JH
Bach
Luc
L
Van Gool
Fabian
F
Nater
Tomas
T
Pajdla
Michal
M
Havlena
Misha
M
Pavel
D000465:Algorithms; D001185:Artificial Intelligence; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods; D015233:Models, Statistical; D009038:Motion; D010363:Pattern Recognition, Automated / Q000379:methods*; D012815:Signal Processing, Computer-Assisted; D049250:Speech Recognition Software; D014741:Video Recording