Visual-ERM: Reward Modeling for Visual Equivalence
Published in NeurIPS 2026, 2026
Ziyu Liu*, Shengyuan Ding*, Xinyu Fang, Xuanlang Dai, Penghui Yang, Jianze Liang, Jiaqi Wang, Kai Chen, Dahua Lin, and Yuhang Zang
Paper | Code | Dataset | Model
Visual-ERM is a multimodal generative reward model for vision-to-code tasks such as chart-to-code, table-to-markdown, and SVG-to-code. Instead of textual rules or coarse embedding similarity, it compares the ground-truth image with the rendered prediction and reports fine-grained, interpretable discrepancies.
Used as a reward in RL, it improves Qwen3-VL-8B-Instruct by +8.4 on chart-to-code, with gains of +2.7 and +4.1 on table and SVG parsing. The paper also introduces VC-RewardBench for judging fine-grained image-to-image discrepancies on structured visual data.
Recommended citation: Ziyu Liu*, Shengyuan Ding*, Xinyu Fang, Xuanlang Dai, Penghui Yang, Jianze Liang, Jiaqi Wang, Kai Chen, Dahua Lin, and Yuhang Zang. (2026). "Visual-ERM: Reward Modeling for Visual Equivalence." NeurIPS 2026. arXiv:2603.13224.
Download Paper