Recently, Texas State University was awarded with the research project “Artificial Intelligence for Pavement Condition Assessment from 2D/3D Surface Images” by Texas Department of Transportation (TxDOT). The proposal of this project was initiated by Dr. Feng Wang and his research team, along with Dr. Jelena Tešić from Department of Computer Science.

The two doctoral students, Jueqiang Tao (left) and Haitao Gong (right), are testing the distress detection code at the Transportation lab.

Pavement condition assessment is a very important integral part in highway pavement management. The accurate assessment results are essential for determining the current pavement condition, predicting future performances, and making comprehensive Maintenance & Rehabilitation plans. Currently, every state Department of Transportation (DOT) performs pavement condition survey on the state level annually. Although the technologies of in-field pavement condition data collection have marched into fully automation, the engineers and researchers are still struggling in finding an efficient way for data processing.

The mostly adopted pavement surface data are high-resolution 2D/3D images collected using professional survey vehicles. The characteristics of these images vary across the state-wide dataset, due to different pavement types, surface types, distress morphologies, and so on.

The high-resolution 3D pavement surface images are widely used by state DOTs for pavement surface condition assessment.

The mainstream technologies for processing this type of data are based on traditional image processing methods, which are inadequate to handle the complexities across the state-wide dataset. A typical phenomenon is that these methods tend to generate false positive distress detections over high-texture pavements.

Dr. Wang has been focusing on pavement management system for many years. In 2020, he and his research team started working on developing cost-effective pavement condition assessment system, aiming for both lower costs and higher accuracies. Based on abundant literature review and preliminary experiment results, the team decided to focus on the Machine Learning (ML) based image processing methods.

To acquire available training datasets, the team has spent large amounts of time and resources doing manual data annotation, with many other attempts, such as building synthetic datasets. A survey vehicle equipped with high resolution 3D image system is being ensembled and will be put in use in this summer. “One of the hurdles for the application of ML methods in pavement distress detection is the lack of high-quality training datasets.” Haitao Gong, a doctoral student in the team, stated, “Developing a pavement distress image dataset is challenging, as it requires professionally trained engineers for annotation. And even professionals could be subjective when dealing with ambiguous distress instances.” In the TxDOT project, the team is expected to develop a pavement image library, which will help facilitate the research of machine learning applications.

The team members are currently exploring suitable methods for pavement distress detection and segmentation. Their recently accepted publication shows that the widely used object detection methods may not be adequate for crack-like object detection. “Yes, there are already many sources we could use when it comes to machine learning methods,” Dr. Wang said, “but we don’t know how these methods would perform in the aspect of distress detection. Human faces have explicit identical features, such as eyes and nose, but pavement distresses are amorphous. There is a knowledge gap between (the Computer Science and Civil Engineering domain) that we need to close.”

Dr. Wang’s research team comprises one postdoc Dr. Xiaohua Luo, two doctoral students from Material Science, Engineering, and Commercialization (MSEC), Haitao Gong and Jueqiang Tao, and one master student. Currently, the team is working on a TxDOT project relating to pavement condition data qualities, which received a grant of $449,720.  For the upcoming project, the team will receive a total of $451,875 from TxDOT in the coming three years for the study of AI-based pavement condition assessment.

“This is just the beginning.” Dr. Wang said, “And we are just excited to be part of the transition to a more intelligent infrastructure management system.”