Classification of ALS Point Clouds Using End-to-End Deep Learning

Deep learning, referring to artificial neural networks with multiple layers, is widely used for classification tasks in many disciplines including computer vision. The most popular type is the Convolutional Neural Network (CNN), commonly applied to 2D image data. However, CNNs are difficult to adapt to irregular data like point clouds. PointNet, on the other hand, has enabled the derivation of features based on the geometric distribution of a set of points in nD-space utilising a neural network. We use PointNet on multiple scales to automatically learn a representation of local neighbourhoods in an end-to-end fashion, which is optimised for semantic labelling on 3D point clouds acquired by Airborne Laser Scanning (ALS). The results are comparable to those using manually crafted features, suggesting a successful representation of these neighbourhoods. On the ISPRS 3D Semantic Labelling benchmark, we achieve 80.6% overall accuracy, a mid-field result. Investigation on a bigger dataset, namely the 2011 ALS point cloud of the federal state of Vorarlberg, shows overall accuracies of up to 95.8% over large-scale built-up areas. Lower accuracy is achieved for the separation of low vegetation and ground points, presumably because of invalid assumptions about the distribution of classes in space, especially in high alpine regions. We conclude that the method of the end-to-end system, allowing training on a big variety of classification problems without the need for expert knowledge about neighbourhood features can also successfully be applied to single-point-based classification of ALS point clouds.