Search Results for author:
Found 6 papers, 2 papers with code
A Survey on Efficient Inference for Large Language Models
This paper presents a comprehensive survey of the existing literature on efficient LLM inference.
Evaluating Quantized Large Language Models
Post-training quantization (PTQ) has emerged as a promising technique to reduce the cost of large language models (LLMs).
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K
In contrast, the average context lengths of mainstream benchmarks are insufficient (5k-21k), and they suffer from potential knowledge leakage and inaccurate metrics, resulting in biased evaluation.
FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs
However, existing GPU and transformer-based accelerators cannot efficiently process compressed LLMs, due to the following unresolved challenges: low computational efficiency, underutilized memory bandwidth, and large compilation overheads.
Enabling Fast 2-bit LLM on GPUs: Memory Alignment and Asynchronous Dequantization
Weights are quantized by groups, while the ranges of weights are large in some groups, resulting in large quantization errors and nonnegligible accuracy loss (e. g. >3% for Llama2-7b with 2-bit quantization in GPTQ and Greenbit).
Characterizing Datasets for Social Visual Question Answering, and the New TinySocial Dataset
Modern social intelligence includes the ability to watch videos and answer questions about social and theory-of-mind-related content, e. g., for a scene in Harry Potter, "Is the father really upset about the boys flying the car?"
