Joint 3D Proposal Generation and Object Detection from View Aggregation

6 Dec 2017  ·  Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, Steven Waslander ·

We present AVOD, an Aggregate View Object Detection network for autonomous driving scenarios. The proposed neural network architecture uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network. The proposed RPN uses a novel architecture capable of performing multimodal feature fusion on high resolution feature maps to generate reliable 3D object proposals for multiple object classes in road scenes. Using these proposals, the second stage detection network performs accurate oriented 3D bounding box regression and category classification to predict the extents, orientation, and classification of objects in 3D space. Our proposed architecture is shown to produce state of the art results on the KITTI 3D object detection benchmark while running in real time with a low memory footprint, making it a suitable candidate for deployment on autonomous vehicles. Code is at: https://github.com/kujason/avod

PDF Abstract

Datasets


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
3D Object Detection KITTI Cars Easy AVOD + Feature Pyramid AP 81.94% # 20
Birds Eye View Object Detection KITTI Cars Easy AVOD-FPN AP 88.53 # 7
3D Object Detection KITTI Cars Hard AVOD + Feature Pyramid AP 66.38% # 18
Birds Eye View Object Detection KITTI Cars Moderate AVOD-FPN AP 83.79% # 7
3D Object Detection KITTI Cars Moderate AVOD + Feature Pyramid AP 71.88% # 27
3D Object Detection KITTI Cyclists Easy AVOD + Feature Pyramid AP 64.0% # 11
3D Object Detection KITTI Cyclists Hard AVOD + Feature Pyramid AP 46.61% # 11
3D Object Detection KITTI Cyclists Moderate AVOD + Feature Pyramid AP 52.18% # 11
Birds Eye View Object Detection KITTI Cyclists Moderate AVOD-FPN AP 57.48% # 5
3D Object Detection KITTI Pedestrians Easy AVOD + Feature Pyramid AP 50.8% # 7
3D Object Detection KITTI Pedestrians Hard AVOD + Feature Pyramid AP 40.88% # 5
3D Object Detection KITTI Pedestrians Moderate AVOD + Feature Pyramid AP 42.81% # 8
Birds Eye View Object Detection KITTI Pedestrians Moderate AVOD-FPN AP 51.05% # 3

Methods