Human-computer Interaction for Brain-inspired Computing Based on Machine Learning And Deep Learning: A Review

1 code implementation • 12 Dec 2023 • Bihui Yu, Sibo Zhang, Lili Zhou, Jingxuan Wei, Linzhuang Sun, Liping Bu

Focusing on the application scenarios of decoding text and speech from brain signals in human-computer interaction, this paper presents a comprehensive review of the brain-inspired computing models based on machine learning (ML) and deep learning (DL), tracking their evolution, application value, challenges and potential research trends.

A Survey on Image-text Multimodal Models

1 code implementation • 23 Sep 2023 • Ruifeng Guo, Jingxuan Wei, Linzhuang Sun, Bihui Yu, Guiyong Chang, Dawei Liu, Sibo Zhang, Zhengbing Yao, Mingjun Xu, Liping Bu

Amidst the evolving landscape of artificial intelligence, the convergence of visual and textual information has surfaced as a crucial frontier, leading to the advent of image-text multimodal models.

Construction Site Safety Monitoring and Excavator Activity Analysis System

no code implementations • 6 Oct 2021 • Sibo Zhang, Liangjun Zhang

Our perception system could detect multi-class construction machines and humans in real-time while estimating the poses and actions of the excavator.

Action Recognition Object Detection +1

Text2Video: Text-driven Talking-head Video Synthesis with Personalized Phoneme-Pose Dictionary

1 code implementation • 29 Apr 2021 • Sibo Zhang, Jiahong Yuan, Miao Liao, Liangjun Zhang

With the advance of deep learning technology, automatic video generation from audio or text has become an emerging and promising research topic.

Generative Adversarial Network Talking Face Generation +1

Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses

1 code implementation • 17 Jul 2020 • Miao Liao, Sibo Zhang, Peng Wang, Hao Zhu, Xinxin Zuo, Ruigang Yang

In this paper, we propose a novel approach to convert given speech audio to a photo-realistic speaking video of a specific person, where the output video has synchronized, realistic, and expressive rich body dynamics.

Generative Adversarial Network

DVI: Depth Guided Video Inpainting for Autonomous Driving

2 code implementations • ECCV 2020 • Miao Liao, Feixiang Lu, Dingfu Zhou, Sibo Zhang, Wei Li, Ruigang Yang

To get clear street-view and photo-realistic simulation in autonomous driving, we present an automatic video inpainting algorithm that can remove traffic agents from videos and synthesize missing regions with the guidance of depth/point cloud.

Autonomous Driving Image Inpainting +2

CVPR 2019 WAD Challenge on Trajectory Prediction and 3D Perception

1 code implementation • 6 Apr 2020 • Sibo Zhang, Yuexin Ma, Ruigang Yang

This paper reviews the CVPR 2019 challenge on Autonomous Driving.

Autonomous Driving object-detection +2

TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents

1 code implementation • 6 Nov 2018 • Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, Dinesh Manocha

To safely and efficiently navigate in complex urban traffic, autonomous vehicles must make responsible predictions in relation to surrounding traffic-agents (vehicles, bicycles, pedestrians, etc.).

Autonomous Vehicles Navigate +2

Event-Radar: Real-time Local Event Detection System for Geo-Tagged Tweet Streams

1 code implementation • 19 Aug 2017 • Sibo Zhang, Yuan Cheng, Deyuan Ke

The local event detection is to use posting messages with geotags on social networks to reveal the related ongoing events and their locations.

Event Detection