Search Results for author:
Found 7 papers, 2 papers with code
Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall
By further decoupling model known and unknown knowledge, we find the degradation is attributed to exemplars that contradict a model's known knowledge, as well as the number of such exemplars.
UNITE: A Unified Benchmark for Text-to-SQL Evaluation
A practical text-to-SQL system should generalize well on a wide variety of natural language questions, unseen database schemas, and novel SQL query structures.
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness
Neural text-to-SQL models have achieved remarkable performance in translating natural language questions into SQL queries.
Importance of Synthesizing High-quality Data for Text-to-SQL Parsing
In this paper, we first examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data.
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding
When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.
Learned Prioritization for Trading Off Accuracy and Speed
Users want natural language processing (NLP) systems to be both fast and accurate, but quality often comes at the cost of speed.
Message-Passing for Approximate MAP Inference with Latent Variables
We consider a general inference setting for discrete probabilistic graphical models where we seek maximum a posteriori (MAP) estimates for a subset of the random variables (max nodes), marginalizing over the rest (sum nodes).
