ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More

no code implementations • 19 Mar 2024 • Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, Lin Wang

Then, we propose a conceptual reasoning-based uncertainty estimation module, which simulates the recognition process to enrich the semantic representation.

Action Recognition

UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All

no code implementations • 19 Mar 2024 • Yuanhuiyi Lyu, Xu Zheng, Jiazhou Zhou, Lin Wang

To make this possible, we 1) construct a knowledge base of text embeddings with the help of LLMs and multi-modal LLMs; 2) adaptively build LLM-augmented class-wise embedding center on top of the knowledge base and encoded visual embeddings; 3) align all the embeddings to the LLM-augmented embedding center via contrastive learning to achieve a unified and balanced representation space.

Contrastive Learning Zero-Shot Learning

EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding

no code implementations • 6 Aug 2023 • Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, Lin Wang

Accordingly, we first introduce a novel event encoder that subtly models the temporal information from events and meanwhile, generates event prompts for modality bridging.

Transfer Learning