PointCNN: Convolution On X-Transformed Points
We present a simple and general framework for feature learning from point cloud. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point cloud are irregular and unordered, thus a direct convolving of kernels against the features associated with the points will result in deserting the shape information while being variant to the orders. To address these problems, we propose to learn a X-transformation from the input points, which is used for simultaneously weighting the input features associated with the points and permuting them into latent potentially canonical order. Then element-wise product and sum operations of typical convolution operator are applied on the X-transformed features. The proposed method is a generalization of typical CNNs into learning features from point cloud, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.
PDF AbstractCode
MNIST
ModelNet
ScanNet
S3DIS
Sketch
IntrA
DALES
Results from the Paper
Ranked #2 on
Semantic Segmentation
on S3DIS Area5
(Number of params metric)
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| 3D Semantic Segmentation | DALES | PointCNN | mIoU | 58.4 | # 7 | |
| Overall Accuracy | 97.2 | # 3 | ||||
| Model size | N/A | # 1 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 10-way (10-shot) | PointCNN | Overall Accuracy | 46.60 | # 24 | |
| Standard Deviation | 4.8 | # 18 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 10-way (20-shot) | PointCNN | Overall Accuracy | 49.95 | # 24 | |
| Standard Deviation | 7.2 | # 24 | ||||
| Few-Shot 3D Point Cloud Classification | ModelNet40 5-way (20-shot) | PointCNN | Overall Accuracy | 68.64 | # 22 | |
| Standard Deviation | 7.0 | # 20 | ||||
| Semantic Segmentation | S3DIS Area5 | PointCNN | oAcc | 85.9 | # 31 | |
| Number of params | N/A | # 2 | ||||
| Semantic Segmentation | ScanNet | PointCNN | test mIoU | 45.8 | # 26 |
