BEAVER: Enterprise benchmark for LLM Text-to-SQL from private data warehouses
BEAVER is a large-scale enterprise text-to-SQL dataset containing 9128 queries across 19 diverse domains. The dataset is composed of queries and databases collected from private organizations, with 7978 queries publicly released and the remaining portion held out as a private test set. The dataset provides annotations for five subtasks to facilitate fine-grained evaluation and analysis.
- ▪BEAVER contains 9128 queries spanning 812 tables across 19 diverse domains.
- ▪The dataset includes 7978 publicly released queries and a private test set.
- ▪The dataset provides annotations for five subtasks: multi-table retrieval, join key detection, column mapping, domain knowledge extraction, and query decomposition.
Opening excerpt (first ~120 words) tap to expand
BEAVER is a large-scale enterprise text-to-SQL dataset containing 9128 queries spanning 812 tables across 19 diverse domains. Of these, 7978 queries are publicly released, while the remaining portion is held out as a private test set. Queries and databases were collected from private organizations. To facilitate fine-grained evaluation and analysis, we provide annotations for five subtasks: multi-table retrieval, join key detection, column mapping, domain knowledge extraction, and query decomposition three categories of queries: complex queries without domain knowledge, domain-specific queries with minimal complexity, and domain-specific complex queries
Excerpt limited to ~120 words for fair-use compliance. The full article is at Github.
Discussion
0 commentsMore from Github
