Search Results for author:
Found 14 papers, 9 papers with code
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction
This paper presents ShapeLLM, the first 3D Multimodal Large Language Model (LLM) designed for embodied interaction, exploring a universal 3D object understanding with 3D point clouds and languages.
Ranked #1 on
3D Question Answering (3D-QA)
on 3D MM-Vet
3D Object Captioning
3D Point Cloud Linear Classification
+10
DreamLLM: Synergistic Multimodal Comprehension and Creation
This paper presents DreamLLM, a learning framework that first achieves versatile Multimodal Large Language Models (MLLMs) empowered with frequently overlooked synergy between multimodal comprehension and creation.
Ranked #1 on
Visual Question Answering
on MMBench
(GPT-3.5 score metric)
VPP: Efficient Conditional 3D Generation via Voxel-Point Progressive Representation
VPP leverages structured voxel representation in the proposed Voxel Semantic Generator and the sparsity of unstructured point representation in the Point Upsampler, enabling efficient generation of multi-category objects.
ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuning
Based on precise referring instruction, we propose ChatSpot, a unified end-to-end multimodal large language model that supports diverse forms of interactivity including mouse clicks, drag-and-drop, and drawing boxes, which provides a more flexible and seamless interactive experience.
CORSD: Class-Oriented Relational Self Distillation
Knowledge distillation conducts an effective model compression method while holding some limitations:(1) the feature based distillation methods only focus on distilling the feature map but are lack of transferring the relation of data examples; (2) the relational distillation methods are either limited to the handcrafted functions for relation extraction, such as L2 norm, or weak in inter- and intra- class relation modeling.
Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perception
In this paper, we explore an embarrassingly simple long-term recurrent fusion strategy built upon the LSS-based methods and find it already able to enjoy the merits from both sides, i. e., rich long-term information and efficient fusion pipeline.
CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIP
Training a 3D scene understanding model requires complicated human annotations, which are laborious to collect and result in a model only encoding close-set object semantics.
Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining
This motivates us to learn 3D representations by sharing the merits of both paradigms, which is non-trivial due to the pattern difference between the two paradigms.
Ranked #1 on
Zero-Shot Transfer 3D Point Cloud Classification
on ModelNet10
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
+2
Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?
The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natural languages.
Ranked #5 on
Few-Shot 3D Point Cloud Classification
on ModelNet40 10-way (10-shot)
(using extra training data)
Few-Shot 3D Point Cloud Classification
Knowledge Distillation
+1
Contrastive Deep Supervision
The success of deep learning is usually accompanied by the growth in neural network depth.
Region-aware Knowledge Distillation for Efficient Image-to-Image Translation
In this paper, we propose Region-aware Knowledge Distillation ReKo to compress image-to-image translation models.
PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection
The remarkable breakthroughs in point cloud representation learning have boosted their usage in real-world applications such as self-driving cars and virtual reality.
Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks
Besides, an efficient deployment flow for the mobile CPU is developed, achieving up to 7. 46$\times$ inference acceleration on an octa-core ARM CPU.
Multi-Glimpse Network: A Robust and Efficient Classification Architecture based on Recurrent Downsampled Attention
Inspired by this observation, we propose an end-to-end trainable Multi-Glimpse Network (MGNet) which aims to tackle the challenges of high computation and the lack of robustness based on recurrent downsampled attention mechanism.
