Publication Radar · Source metadata

BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming

Clemens Kujat, Bharathi Kannan Nithyanantham, Stefan Telgmann, Tobias Sesterhenn, Joern Ploennigs, Stefan L\"udtke, Christian Bartelt

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

BIBIMBAP: A Benchmark for Instructional BIM-Based Automated Programming

Clemens Kujat, Bharathi Kannan Nithyanantham, Stefan Telgmann, Tobias Sesterhenn, Joern Ploennigs, Stefan L\"udtke, Christian Bartelt

First announced by arXiv: 2026-11-10 · Original record ↗

arxiv
2611.09933
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.09933v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Large Language Models (LLMs) are emerging as natural-language interfaces for Building Information Modeling (BIM), successfully achieving information retrieval, but no benchmark exists to evaluate their ability to edit Industry Foundation Classes (IFC). We introduce BIBIMBAP, a Text-to-BIM benchmark with 100 curated tasks covering Create-Read-Update-Delete (CRUD) operations across spatial, geometric, topological, numeric, and conceptual categories. Each task includes a natural-language prompt, an IFC model, expected structured outputs, and test scripts for automated evaluation. Baseline results show limited performance (best: 50.2%) and frequent failures in constraint preservation and spatial reasoning, with some correct outputs achieved despite flawed reasoning.