Supervised machine learning and feature selection for a document analysis application

James Pope, Daniel Powers, J. A.Jim Connell, Milad Jasemi, David Taylor, Xenofon Fafoutis

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

1 Citation (Scopus)

Abstract

Over the past three decades large amounts of information have been converted to image formats from paper documents. Though in digital form, extracting the information, usually textual, from these documents requires complex image processing and optical character recognition techniques. The processing pipeline from the image to information typically includes an orientation correction task, document identification task, and text analysis task. When there are many document variants the tasks become difficult requiring complex sub-analysis for each variant and quickly exceeds human capability. In this work, we demonstrate a document analysis application with the orientation correction and document identification task carried out by supervised machine learning techniques for a large, international airline. The documents have been amassed over forty years with numerous variants and are mostly black and white, typically consist of text and lines, and some have extensive noise. Low level symbols are extracted from the raw images and separated into partitions. The partitions are used to generate statistical features which are then used to train the classifiers. We compare the classifiers for each task (e.g. decision tree, support vector machine, and random forest) to choose the most appropriate. We also perform feature selection to reduce the complexity of the document type classifiers. These parsimonious models result in comparable accuracy with 80% or fewer features.

Original languageEnglish
Title of host publicationICPRAM 2020 - Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods
EditorsMaria De Marsico, Gabriella Sanniti di Baja, Ana Fred
PublisherSciTePress
Pages415-424
Number of pages10
ISBN (Electronic)9789897583971
Publication statusPublished - 2020
Event9th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2020 - Valletta, Malta
Duration: 22 Feb 202024 Feb 2020

Publication series

NameICPRAM 2020 - Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods

Conference

Conference9th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2020
CountryMalta
CityValletta
Period22/02/2024/02/20

Bibliographical note

Funding Information:
This work was supported in part by the University of Montevallo Contract #19-0501-001. The authors greatly appreciate thesupport of the airline company employees involved in the project. Without their efforts this research could not have been conducted.

Publisher Copyright:
Copyright © 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved.

Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.

Keywords

  • Document Analysis
  • Feature Selection
  • Optical Character Recognition
  • Supervised Machine Learning

Fingerprint

Dive into the research topics of 'Supervised machine learning and feature selection for a document analysis application'. Together they form a unique fingerprint.

Cite this