WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation
Published in arXiv preprint, 2026
Shuangrui Ding*, Xuanlang Dai*, Long Xing*, Shengyuan Ding, Ziyu Liu, Jingyi Yang, Penghui Yang, Zhixiong Zhang, Xilin Wei, Xinyu Fang, Yubo Ma, Haodong Duan, Jing Shao, Jiaqi Wang, Dahua Lin, Kai Chen, and Yuhang Zang
Paper | Leaderboard | Code | Dataset
WildClawBench evaluates whether AI agents can complete realistic long-horizon work in the runtimes where they are deployed. Each of its 60 tasks runs in a reproducible Docker container hosting an actual CLI agent harness (OpenClaw, Claude Code, Codex, or Hermes Agent) with access to real tools, averaging roughly 8 minutes and over 20 tool calls.
Across 19 frontier models in the paper, the best reaches only 62.2% overall under OpenClaw, and switching harness alone shifts a single model by up to 18 points.
Recommended citation: Shuangrui Ding*, Xuanlang Dai*, Long Xing*, Shengyuan Ding, Ziyu Liu, Jingyi Yang, Penghui Yang, Zhixiong Zhang, Xilin Wei, Xinyu Fang, Yubo Ma, Haodong Duan, Jing Shao, Jiaqi Wang, Dahua Lin, Kai Chen, and Yuhang Zang. (2026). "WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation." arXiv preprint arXiv:2605.10912.
Download Paper