MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation

Computer Science > Artificial Intelligence arXiv:2604.23539 (cs) [Submitted on 26 Apr 2026] Title:MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation Authors:Haoxuan Zhang, Ruochi Li, Yang Zhang, Zhenni Liang, Junhua Ding, Ting Xiao, Haihua Chen View a PDF of the paper titled MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation, by Haoxuan Zhang and 6 other authors View PDF HTML (experimental) Abstract:The rapid proliferation of Generative AI necessitates rigorous documentation standards for transparency and governance. However, manual creation of Model and Data Cards is not scalable, while automated approaches lack large-scale, high-fidelity benchmarks for systematic evaluation. We introduce MetaGAI, a comprehensive benchmark comprising 2,541 verified document triplets constructed through semantic triangulation of academic papers, GitHub repositories, and Hugging Face artifacts. Unlike prior single-source datasets, MetaGAI employs a multi-agent framework with specialized Retriever, Generator, and Editor agents, validated through four-dimensional human-in-the-loop assessment, including human evaluation of editor-refined ground truth. We establish a robust evaluation protocol combining automated metrics with validated LLM-as-a-Judge frameworks. Extensive analysis reveals that sparse Mixture-of-Experts architectures achieve superior cost-quality efficiency, while a fundamental trade-off exists between faithfulness and completeness. MetaGAI provides a foundational testbed for benchmarking, training, and analyzing automated Model and Data Card generation methods at scale. Our data and code are available at: this https URL. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2604.23539 [cs.AI] (or arXiv:2604.23539v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2604.23539 Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: Yang Zhang [view email] [v1] Sun, 26 Apr 2026 05:24:03 UTC (7,317 KB) Full-text links: Access Paper: View a PDF of the paper titled MetaGAI: A Large-Scale and High-Quality Benchmark for Generative AI Model and Data Card Generation, by Haoxuan Zhang and 6 other authorsView PDFHTML (experimental)TeX Source view license Current browse context: cs.AI < prev | next > new | recent | 2026-04 Change to browse by: cs References & Citations NASA ADSGoogle Scholar Semantic Scholar export BibTeX citation Loading... BibTeX formatted citation × loading... Data provided by: Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) Demos Demos Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related Papers Recommenders and Search Tools Link to Influence Flower…

This excerpt is published under fair use for community discussion. Read the full article at arXiv.org.