Benchmarking Robotic Manipulation under Industrial Process Constraints

Ming Shin Cindy Sung¹, Duan Leijiahao¹, Haoxiang Wang¹, Boning Tan¹, Liangna Fu¹, Chao Tan¹, Yafu Li², Yulong Chen^3,4, Chaoqun Wan¹, Zhen Huang¹, Shuxian Liang^1*, Xian-Sheng Hua¹

¹Institute of AI for Engineering, Tongji University ²The Chinese University of Hong Kong
³University of Aberdeen ⁴University of Cambridge

*Corresponding author

Industrial constraints as controlled task conditions

216 tasks · 180 training · 36 held-out evaluation

Task suites organized by source–destination dynamics

S = Static · D = Dynamic

S2D

Static → Dynamic loading Place the pump bottle flat into the moving groove. Level 2 · Scripted expert

D2S

Dynamic → Static unloading Sort the marked toys into stationary grooves. Level 2 · Scripted expert

D2D

Dynamic → Dynamic transfer Transfer the mouse from the conveyor into the moving groove. Level 2 · Scripted expert

Industrial process conditions are instantiated as controlled task variations across the three material-flow suites. Videos show accepted scripted-expert demonstrations from Level 2. View the paper figure →

Abstract

Industrial manipulation in flexible manufacturing, e.g., high-mix, low-volume (HMLV) electronics assembly, requires robot policies to adapt to changing products and production conditions. Such settings impose industrial process constraints, including product variants, moving material flows, finite execution windows, and tight geometric tolerances. However, existing evaluation frameworks do not systematically capture these constraints when assessing generalist robot policies. To address this gap, we introduce LIBERO-Industry, a novel benchmark built on the widely used robot learning benchmark LIBERO, that translates the aforementioned industrial constraints into controllable task conditions and measurable success criteria. Specifically, LIBERO-Industry comprises three classic task scenarios: Static-to-Dynamic (S2D) loading, Dynamic-to-Static (D2S) unloading, and Dynamic-to-Dynamic (D2D) transfer. The suites contain 216 tasks in total that systematically vary object geometry, conveyor speed, interaction-window duration, and geometric acceptance region size. Extensive experiments across five generalist robot policies (including vision-language-action models and world-action models) show that performance progressively degrades as industrial constraints are introduced and combined. Under a shared fine-tuning and evaluation protocol, the LIBERO-Industry co-training condition achieved higher observed target-transfer completion than the original-LIBERO condition.

Task generation and validation

One pipeline converts BDDL task specifications into validated task instances and admitted demonstrations.

LIBERO-Industry task-generation pipeline from BDDL definition to admitted demonstrations

Datasets

We use the Aubo i5h equipped with the CTAG2F90D gripper for real-world validation.

HDF5

LIBERO-Industry demonstrations

Native trajectory archives for benchmark training and evaluation.

Real-world HDF5

Real-World Validation Demonstrations

95 accepted Aubo i5h and CTAG2F90D demonstrations with aligned external and wrist-camera observations.

180 / 36

Train and held-out split

Use the training split for policy learning and the held-out split for official reporting.

S2D · D2S · D2D

Suite coverage

Resources follow the benchmark’s three source–destination dynamics.

SR · PCR-T · PCR-S

Evaluation metrics

SR: Complete-task success.

PCR-T: Target-transfer completion.

PCR-S: Final placement completion, including pre-existing destination workpieces.

Research

Cross-benchmark transfer evaluation (Table 3)

Five original-LIBERO-fine-tuned checkpoints, without further LIBERO-Industry fine-tuning. Evaluated on 36 held-out tasks, with 10 initial states per task: 360 rollouts per policy.

Evaluation-set results from Table 3 of the paper. Scores in %.
Policy	PCR-T	PCR-S
π_0.5	0.4	12.5
OpenVLA-OFT	0.0	11.2
UniVLA	0.0	12.2
X-VLA	0.0	11.9
Cosmos Policy	0.0	11.8

All five policies recorded 0/360 complete-task successes. PCR-S includes pre-existing destination workpieces that remain valid; their mean initial-state baseline is 12.3%. PCR-T isolates newly completed target transfers.

Diagnostic stress progression

The cumulative D0–D6 diagnostic stress progression shows how generalist robot policies respond as industrial constraints accumulate.

Cumulative D0 to D6 diagnostic progression for success rate and task progression — Cumulative D0–D6 diagnostic progression. Each setting retains the preceding changes; this is not a causal one-factor ablation.

Representative failure cases

Playback speed

Failure classification: Pre-reach policy freeze → Reach failure → Grasp acquisition failure → Lift failure (early slip) → Dropped midway → Post-release goal failure

Reach Failure D2S · D3 · Task 6 · Episode 10

Lift Failure (early slip) S2D · D3 · Task 9 · Episode 15

Dropped Midway D2S · D5 · Task 4 · Episode 1

Post-release Goal Failure S2D · D5 · Task 9 · Episode 15

Real-world Aubo workcell evaluation

An Aubo i5h workcell evaluation compares policies fine-tuned on LIBERO and LIBERO-Industry across consecutive pick-and-place trials and controlled perturbations.

Real-world AUBO-i5H evaluation setup and results — Real-world AUBO-i5H evaluation summary reported in the manuscript.

Paper

Manuscript link and citation

Read the paper on arXiv →

@misc{sung2026liberoindustry,
  title={{LIBERO-Industry}: Benchmarking Robotic Manipulation under Industrial Process Constraints},
  author={Sung, Ming Shin Cindy and Duan, Leijiahao and Wang, Haoxiang and
          Tan, Boning and Fu, Liangna and Tan, Chao and Li, Yafu and
          Chen, Yulong and Wan, Chaoqun and Huang, Zhen and Liang, Shuxian and
          Hua, Xian-Sheng},
  year={2026},
  eprint={2610.16782},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  doi={10.48550/arXiv.2610.16782},
  url={https://arxiv.org/abs/2610.16782}
}

LIBERO-Industry builds on LIBERO, the lifelong robot-learning benchmark.