Search Results for author:
Found 13 papers, 9 papers with code
HGRN2: Gated Linear RNNs with State Expansion
Hierarchically gated linear RNN (HGRN, Qin et al. 2023) has demonstrated competitive training speed and performance in language modeling, while offering efficient inference.
Linear Attention Sequence Parallelism
In this paper, we introduce Linear Attention Sequence Parallel (LASP), an efficient SP method tailored to linear attention-based language models.
CO2: Efficient Distributed Training with Full Communication-Computation Overlap
CO2 is able to attain a high scalability even on extensive multi-node clusters constrained by very limited communication bandwidth.
Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
With its ability to process tokens in linear computational complexities, linear attention, in theory, can handle sequences of unlimited length without sacrificing speed, i. e., maintaining a constant training speed for various sequence lengths with a fixed memory consumption.
TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
TransNormerLLM evolves from the previous linear attention architecture TransNormer by making advanced modifications that include positional embedding, linear attention acceleration, gating mechanisms, tensor normalization, and inference acceleration and stabilization.
Fine-grained Audible Video Description
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD).
Audio-Visual Segmentation with Semantics
To deal with these problems, we propose a new baseline method that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process.
Linear Video Transformer with Feature Fixation
Therefore, we propose a feature fixation module to reweight the feature importance of the query and key before computing linear attention.
AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification
Fine-tuning is widely applied in image classification tasks as a transfer learning approach.
Feature Selection on Thermal-stress Dataset
These findings are likely to improve the accuracy of current stress recognition systems.
Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
Varying the proportions of male and female faces in the training data can have a substantial effect on behavior on the test data: we found that the seemingly obvious choice of 50:50 proportions was not the best for this dataset to reduce biased behavior on female faces, which was 71% unbiased as compared to our top unbiased rate of 84%.
Non-binary deep transfer learning for image classification
A model is either pre-trained or not pre-trained.
Ranked #2 on
Image Classification
on Caltech-256
(using extra training data)
Pairwise-GAN: Pose-based View Synthesis through Pair-Wise Training
This inspired our research which explores the performance of two models from pixel transformation in frontal facial synthesis, Pix2Pix and CycleGAN.
