LED-BENCH: A Benchmark for Large-format Engineering Drawing Understanding

Ke Li; Tianyi Wang; Enning You; Chaoqun Wan; Zhen Huang; Shuxian Liang; Xian-Sheng Hua

LED-BENCH

A Benchmark for Large-format
Engineering Drawing Understanding

Ke Li, Tianyi Wang, Enning You, Chaoqun Wan, Zhen Huang, Shuxian Liang^†, Xian-Sheng Hua

Institute of AI for Engineering, Tongji University

^† Corresponding author

Paper Code Dataset Soon Leaderboard Explore drawings

107Complete large-format drawings

377Questions

3Engineering domains

11Evaluated models

EVOLVING BENCHMARKMore drawings, questions, domains and model evaluations are planned for future releases.

LED-BENCH overview: identify two shaft keyways, associate sections and views, and integrate dimensions into a complete engineering answer. — **From complete large-format drawings to complete engineering answers.** Models must identify the right objects, establish their relations, and integrate evidence across regions and views.

News

14 Oct 2026Our paper is now available on arXiv .

The challenge

Engineering drawings communicate design intent through dimensions, tolerances, symbols, geometric views, and technical notes. The evidence needed for a single answer can be scattered across an entire sheet.

LED-BENCH evaluates fine-grained understanding and complex reasoning over complete, large-format engineering drawings. It brings together mechanical, architectural, and transportation engineering, covering questions from bounded evidence understanding to cross-view association, topology reconstruction, and integrated engineering reasoning.

Across 11 evaluated multimodal models, the highest domain-average accuracy is 52.65%. Performance varies substantially across engineering domains, and every model scores lower on the Hard subset.

Identify objects

Locate the relevant part, space, or infrastructure element.

Establish relations

Associate annotations, views, and connected entities.

Integrate evidence

Combine the required constraints into a complete answer.

The benchmark

Three domains. Complete large-format drawings.

Different drawing conventions, a shared need to connect the right evidence.

Domain 01 / 03

Mechanical engineering

Dense orthographic projections, sections, dimensions, and tolerances require precise association between annotations and the correct part or view.

45drawings

174questions

Question difficultyMedium / Hard

Medium 119Hard 55

Evidence focus

Cross-view correspondence, dimension binding, and geometric or assembly relations.

Two levels of reasoning demand

Medium questions use a bounded evidence set and relatively direct combinations or calculations. Hard questions require broader or more interdependent evidence, spatial or topological reconstruction, or multi-step inference.

Read the dataset specification

Dataset composition by engineering domain and difficulty level
Domain	Drawings	Questions	Medium	Hard
Mechanical	45	174	119	55
Architecture	21	75	37	38
Transportation	41	128	55	73
Total	107	377	211	166

Dataset composition & capability hierarchy

Domain and difficulty distribution, and a conceptual hierarchy from perception and recognition to integrated engineering reasoning.

Experiment results

Leaderboard

Performance by difficulty

Beyond the score

Where do answers break down?

01 · Failure analysis

Relations are the largest failure category

Reconstructing connectivity, direction, scope, and links between views accounts for 47.3% of the coded error sample.

02 · Diagnostic interventions

Additional visual access can help

Four response models are evaluated on the same 377 questions, over three runs, with different visual-access procedures.

The best intervention depends on the domain

Fixed crops improve all three domains. Locate then reread is less reliable on Mechanical, while Search then reread gives the largest architectural gain.

Four-model mean accuracy by engineering domain. Smaller labels show change from the whole-sheet baseline.
Engineering domain	Whole-sheet1 call · baseline	Fixed crops1 call	Locate & reread3 calls	Search & reread3 calls
Mechanical	44.12Baseline	54.71+10.59 pp	36.47−7.65 pp	51.76+7.64 pp
Architecture	31.03Baseline	44.83+13.80 pp	44.83+13.80 pp	53.45+22.42 pp
Transportation	34.00Baseline	48.48+14.48 pp	43.00+9.00 pp	44.00+10.00 pp

Resources & citation

Build on LED-BENCH

If you use LED-BENCH in your research, please cite our paper.

Read the paperCode

DatasetComing soon

@misc{li2026ledbench,
  title={LED-BENCH: A Benchmark for Large-format
         Engineering Drawing Understanding},
  author={Ke Li and Tianyi Wang and Enning You and Chaoqun Wan
          and Zhen Huang and Shuxian Liang and Xian-Sheng Hua},
  year={2026},
  eprint={2610.19571},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.19571}
}