Get Started with PointNet++

PointNet++ is a popular neural network used for semantic segmentation of unorganized lidar point clouds. Semantic segmentation associates each point in a 3-D point cloud with a class label, such as car, truck, ground, or vegetation. To learn more about semantic segmentation with point clouds, see Semantic Segmentation in Point Clouds Using Deep Learning.

PointNet++ network hierarchically processes individual points in a point cloud. The network uses PointNet to extract local features from point clusters and groups those features to produce higher level features from the entire point cloud.

Applications of PointNet++ include:

• Tree segmentation for digital forestry applications.

• Extracting a digital terrain model from aerial lidar data.

• Perception for indoor navigation in robotics.

• 3-D city modelling from aerial lidar data.

PointNet++ Network

The PointNet++ network contains an encoder with set abstraction modules and a decoder with feature propagation modules.

The set abstraction module processes and extracts a set of points to produce a new set with fewer elements. Each set abstraction module contains a sampling and grouping layer followed by a PointNet layer.

• The sampling and grouping layer performs sampling by identifying the centroids of local regions. It then performs grouping by constructing local region sets of the neighboring points around the centroids.

• The PointNet layer contains a mini-PointNet network with a series of convolution, normalization, relu layers followed by a max pooling layer. This layer encodes the local region patterns into feature vectors.

The feature propagation module interpolates the subsampled points and then concatenates them with the point features from the set abstraction modules. The network then passes these features through a unit PointNet network where they are further processed to produce higher level features.

In other words, PointNet++ applies PointNet recursively on the nested, partitioned inputs to extract multi-scale features for accurate semantic segmentation.

Create PointNet++ Network

Use the pointnetplusNetwork function to create a PointNet++ network for segmenting point cloud data.

Train PointNet++ Network

To learn how to train a PointNet++ network for segmenting point cloud data, see Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning.

Code Generation

To learn how to generate CUDA® code for a PointNet++ network, see Code Generation for Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning.

References

[1] Qi, Charles R., Li Yi, Hao Su, and Leonidas J. Guibas. ‘PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space’. ArXiv:1706.02413 [Cs], 7 June 2017. https://arxiv.org/abs/1706.02413.

[2] Varney, Nina, Vijayan K. Asari, and Quinn Graehling. ‘DALES: A Large-Scale Aerial LiDAR Data Set for Semantic Segmentation’. ArXiv:2004.11985 [Cs, Stat], 14 April 2020. https://arxiv.org/abs/2004.11985.