Road Defect Detection Based on Semantic Transformed Disparity Image Segmentation

Road defects can severely affect the safety of road users and vehicle conditions. Over the past decade, due to the limited amount of labeled training data, machine vision-based road defect detection approaches have been mainly used, while machine/deep learning-based methods were merely discussed. With the recent development of artificial intelligence, convolutional neural network (CNN)-based road defect detection systems for automated road condition assessment have become an active sphere of study. In this regard, this paper presents a comprehensive road defect detection system based on computer stereo vision, non-linear regression, and CNN. A dense disparity image is first estimated from a pair of stereo road images using an efficient stereo matching algorithm. The estimated disparity image is then transformed to better identify road defects by minimizing a global energy function w.r.t. road disparity projection model coefficients and stereo rig roll angle, using the non-linear regression approach. Finally, three popular semantic segmentation CNNs are trained using the transformed disparity images. Extensive experiments are conducted to demonstrate the performance of our proposed road defect detection approach. The achieved pixel-level accuracy and intersection over union (IoU) are 98.37% and 67.65%, respectively.