R and DE Robotic Sensor Intern Team: Sensor Suite Software and User Interface

The sensor tree developed for the H-Canyon tunnel inspection project produces a large quantity of camera and Lidar data. A postprocessing pipeline was created to facilitate the analysis of these data. The pipeline follows the following routine: 1) Registration - Salient local features in subsequent scans are identified and used to align the scans with one another. This allows production of a single cohesive map of the tunnel. 2) Sensor Fusion - RGB camera data is 'painted' onto the depth data created by the Lidar. 3) Segmentation - Individual surfaces from within the scene are segmented out separately for individual analysis. 4) Degradation Analysis - Automatic mapping of local deviation from the plane in position or surface normal is performed, as is detection of exposed rebar using Lidar return intensity. 5) Obstacle Detection - Lidar data is also used in real time to detect obstacles in the environment. This could give feedback to the operator to successfully drive in the tunnel. 6) Graphic User Interface - a custom GUI was created to facilitate customer use of the system. The system provides real-time feedback from both the cameras and Lidar. During alignment, local features are used to recognize points within the walls that are especially unique and noticeable (salient) based on their deviation from the expected surface of the wall. These are used as landmarks to register the clouds, which also informs precise localization of the base at each scan point. After registration, clouds are 'painted' with RGB data from the spherical cameras, as shown below. The painted clouds provide a 3D, colorized model of the environment which improves the realism of the experience for the user. Painting also labels high-resolution color imagery from the 3D cameras with depth values. Following sensor fusion, individual walls and cylindrical duct surfaces are segmented from the scene for individual analysis using a Random Sample Consensus algorithm. This algorithm allows segmentation of planar walls and cylindrical ducts. Once individual walls are identified, each point within the wall can be compared to its neighbors and the wall overall in order to detect local areas with damaged concrete. Automatic rebar detection is also performed using the contrast in Lidar return intensity between iron and concrete. The Lidar point-cloud data is also used for obstacle detection. As the Lidar spins, various algorithms are run to analyze the current 'slice' and obstacles in the environment are determined and registered on a cost-map. A Graphical User Interface (GUI) was developed to allow the user to easily command the crawler to set the rotational speed of the Lidar, perform a scan, and register and save the 3D generated cloud after a scan is done. For the good of the crawler, a critical aspect of the GUI design is to allow the user to cancel a scan any time after it is commanded and before it is saved