Search Results for author:
Found 10 papers, 3 papers with code
LHMKE: A Large-scale Holistic Multi-subject Knowledge Evaluation Benchmark for Chinese Large Language Models
Current datasets collect questions from Chinese examinations across different subjects and educational levels to address this issue.
OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety
In addition to these benchmarks, we have implemented a phased public evaluation and benchmark update strategy to ensure that OpenEval is in line with the development of Chinese LLMs or even able to provide cutting-edge benchmark datasets to guide the development of Chinese LLMs.
FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
We conduct extensive experiments on a wide range of LLMs on FineMath and find that there is still considerable room for improvements in terms of mathematical reasoning capability of Chinese LLMs.
Do Large Language Models Mirror Cognitive Language Processing?
We employ Representational Similarity Analysis (RSA) to mearsure the alignment between 16 mainstream LLMs and fMRI signals of the brain.
A Comprehensive Evaluation of Quantization Strategies for Large Language Models
Our experimental results indicate that LLMs with 4-bit quantization can retain performance comparable to their non-quantized counterparts, and perplexity can serve as a proxy metric for quantized LLMs on most benchmarks.
FollowEval: A Multi-Dimensional Benchmark for Assessing the Instruction-Following Capability of Large Language Models
In pursuit of this goal, various benchmarks have been constructed to evaluate the instruction-following capacity of these models.
Evaluating Large Language Models: A Comprehensive Survey
We hope that this comprehensive overview will stimulate further research interests in the evaluation of LLMs, with the ultimate goal of making evaluation serve as a cornerstone in guiding the responsible development of LLMs.
Large Language Model Alignment: A Survey
We also envision bridging the gap between the AI alignment research community and the researchers engrossed in the capability exploration of LLMs for both capable and safe LLMs.
M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models
Comprehensively evaluating the capability of large language models in multiple tasks is of great importance.
Informative Language Representation Learning for Massively Multilingual Neural Machine Translation
Experiment results on two datasets for massively multilingual neural machine translation demonstrate that language-aware multi-head attention benefits both supervised and zero-shot translation and significantly alleviates the off-target translation issue.
