This tool is part of a framework for evaluating the performance of layout analysis methods. It combines efficiency and accuracy by using a special interval based geometric representation of regions. A wide range of sophisticated evaluation measures provides the means for a deep insight into the analysed systems, which goes far beyond simple benchmarking. The support of user-defined profiles allows the tuning for practically any kind of evaluation scenario related to real world applications.
The framework has been successfully delivered as part of a major EU-funded project (IMPACT) to evaluate large-scale digitisation projects and has been validated applied within the past three ICDAR Page Segmentation Competitions.
The Text Evaluation tool implements the word and character accuracy measures developed by the University of Nevada Las Vegas (UNLV dissertation by S. V. Rice). It has been complemented by a bag-of-words method which is independent from the reading order.
The tool is platform independent and comes with a basic graphical user interface as well as a command line interface.
A survey of OCR evaluation tools and metrics
In The 6th International Workshop on Historical Document Imaging and Processing (HIP '21). Association for Computing Machinery, New York, NY, USA, 13–18.
Flexible character accuracy measure for reading-order-independent evaluation
Pattern Recognition Letters, Volume 131, March 2020, Pages 390-397
Quality Prediction System for Large-Scale Digitisation Workflows
Proceedings of the 12th IAPR International Workshop on Document Analysis Systems (DAS2016), Santorini, Greece, April 11-14, 2016