Search Results for author:
Found 8 papers, 5 papers with code
Autonomous Tree-search Ability of Large Language Models
This approach yield a greater improvement compared to the ones fine-tuned on CoT data.
Baby's CoThought: Leveraging Large Language Models for Enhanced Reasoning in Compact Models
Large Language Models (LLMs) demonstrate remarkable performance on a variety of natural language understanding (NLU) tasks, primarily due to their in-context learning ability.
ModuleFormer: Modularity Emerges from Mixture-of-Experts
In our experiment, we found that the modular architecture enables three important abilities for large pre-trained language models: 1) Efficiency, since ModuleFormer only activates a subset of its modules for each input token, thus it could achieve the same performance as dense LLMs with more than two times throughput; 2) Extendability, ModuleFormer is more immune to catastrophic forgetting than dense LLMs and can be easily extended with new modules to learn new knowledge that is not included in the training data; 3) Specialisation, finetuning ModuleFormer could specialize a subset of modules to the finetuning task and the task-unrelated modules could be easily pruned for a lightweight deployment.
mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models
Recent multilingual pretrained language models (mPLMs) have been shown to encode strong language-specific signals, which are not explicitly provided during pretraining.
Open-Ended Question Answering
Zero-Shot Cross-Lingual Transfer
Unbiased Gradient Boosting Decision Tree with Unbiased Feature Importance
To this end, we provide a fine-grained analysis of bias in GBDT and demonstrate that the bias originates from 1) the systematic bias in the gain estimation of each split and 2) the bias in the split finding algorithm resulting from the use of the same data to evaluate the split improvement and determine the best split.
Discovering Customer-Service Dialog System with Semi-Supervised Learning and Coarse-to-Fine Intent Detection
Task-oriented dialog(TOD) aims to assist users in achieving specific goals through multi-turn conversation.
Indoor room Occupancy Counting based on LSTM and Environmental Sensor
This paper realizes the estimation of classroom occupancy by using the CO2 sensor and deep learning technique named Long-Short-Term Memory.
OpenFE: Automated Feature Generation with Expert-level Performance
In the two competitions, features generated by OpenFE with a simple baseline model can beat 99. 3% and 99. 6% data science teams respectively.
