Search Results for author:
Found 17 papers, 9 papers with code
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
To help the open-source community have a better understanding of Mixture-of-Experts (MoE) based large language models (LLMs), we train and release OpenMoE, a series of fully open-sourced and reproducible decoder-only MoE LLMs, ranging from 650M to 34B parameters and trained on up to over 1T tokens.
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
By leveraging this information, we introduce an efficient sequence scheduling technique that groups queries with similar response lengths into micro-batches.
Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention
We evaluate our model on various dialogue understanding tasks including dialogue relation extraction, dialogue emotion recognition, and dialogue act classification.
Ranked #1 on
Dialog Relation Extraction
on DialogRE
Adaptive Computation with Elastic Input Sequence
However, most standard neural networks have a fixed function type and computation budget regardless of the sample's nature or difficulty.
A Study on Transformer Configuration and Training Objective
In this paper, we revisit these conventional configurations.
Ranked #103 on
Image Classification
on ImageNet
CowClip: Reducing CTR Prediction Model Training Time from 12 hours to 10 minutes on 1 GPU
Our experiments show that previous scaling rules fail in the training of CTR prediction neural networks.
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation
Text-based video segmentation aims to segment the target object in a video based on a describing sentence.
Ranked #10 on
Referring Expression Segmentation
on A2D Sentences
Optical Flow Estimation
Referring Expression Segmentation
+4
One Student Knows All Experts Know: From Sparse to Dense
Mixture-of-experts (MoE) is a powerful sparse architecture including multiple experts.
Large-Scale Deep Learning Optimizations: A Comprehensive Survey
Deep learning have achieved promising results on a wide spectrum of AI applications.
Cross-token Modeling with Conditional Computation
Mixture-of-Experts (MoE), a conditional computation architecture, achieved promising performance by scaling local module (i. e. feed-forward network) of transformer.
Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization
In this paper, we examine the use of Transfer Learning using Pretrained Audio Neural Networks (PANNs), and propose an architecture that is able to better leverage the acoustic features provided by PANNs for the Automated Audio Captioning Task.
Go Wider Instead of Deeper
To achieve better performance with fewer trainable parameters, recent methods are proposed to go shallower by parameter sharing or model compressing along with the depth.
Ranked #663 on
Image Classification
on ImageNet
Sequence Parallelism: Long Sequence Training from System Perspective
That is, with sparse attention, our sequence parallelism enables us to train transformer with infinite long sequence.
Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey
To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques.
An Embarrassingly Simple Model for Dialogue Relation Extraction
Dialogue relation extraction (RE) is to predict the relation type of two entities mentioned in a dialogue.
Ranked #9 on
Dialog Relation Extraction
on DialogRE
GDPNet: Refining Latent Multi-View Graph for Relation Extraction
Recent advances on RE task are from BERT-based sequence modeling and graph-based modeling of relationships among the tokens in the sequence.
Ranked #4 on
Dialog Relation Extraction
on DialogRE
(F1c (v1) metric)
Deep Graph Random Process for Relational-Thinking-Based Speech Recognition
Lying at the core of human intelligence, relational thinking is characterized by initially relying on innumerable unconscious percepts pertaining to relations between new sensory signals and prior knowledge, consequently becoming a recognizable concept or object through coupling and transformation of these percepts.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
