title: Performance evaluation and benchmarking of six-page segmentation algorithms.
abstract: Informative benchmarks are crucial for optimizing the page segmentation step of an OCR system, frequently the performance limiting step for overall OCR system performance. We show that current evaluation scores are insufficient for diagnosing specific errors in page segmentation and fail to identify some classes of serious segmentation errors altogether. This paper introduces a vectorial score that is sensitive to, and identifies, the most important classes of segmentation errors (over-, under-, and mis-segmentation) and what page components (lines, blocks, etc.) are affected. Unlike previous schemes, our evaluation method has a canonical representation of ground truth data and guarantees pixel-accurate evaluation results for arbitrary region shapes. We present the results of evaluating widely used segmentation algorithms (x-y cut, smearing, whitespace analysis, constrained text-line finding, docstrum, and Voronoi) on the UW-III database and demonstrate that the new evaluation scheme permits the identification of several specific flaws in individual segmentation methods.
authors.affiliation: Image understanding and Pattern Recongnition Research Group, German Research Center for Artificial Intelligence, Kaiserslautern, Germany. faisal.shafait@dfki.de
authors.forename: Faisal
authors.initials: F
authors.lastname: Shafait
authors.forename: Daniel
authors.initials: D
authors.lastname: Keysers
authors.forename: Thomas
authors.initials: T
authors.lastname: Breuel
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D019985:Benchmarking; D003196:Computer Graphics; D004282:Documentation / Q000379:methods*; D001330:Electronic Data Processing / Q000379:methods*; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016247:Information Storage and Retrieval / Q000379:methods*; D015233:Models, Statistical; D009716:Numerical Analysis, Computer-Assisted; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012815:Signal Processing, Computer-Assisted; D013382:Subtraction Technique; D014584:User-Computer Interface