.jsonl --judge-model gpt-5.1 --reasoning-effort high
```
Evaluation is binary: a task is solved only if the model's response passes **all** associated rubrics. For reasoning models, only the final solution is evaluated; thinking traces are excluded.
### Data Structure
Both datasets use the same JSONL format:
```json
{
"messages": [
{"role": "system", "content": "..."},
{"role": "user", "content": "..."},
{"role": "assistant", "content": "..."}
],
"rubrics": ["Rubric 1", "Rubric 2", "..."],
"metadata": {
"task_id": "unique-task-identifier",
"context_category": "..."
}
}
```
## 📮 Submit Results
Want to add your model to the leaderboard? Run inference and evaluation using the scripts above, then submit a PR to this repo with your graded result file (`*_graded.jsonl`). We will verify the results and update the leaderboard.
## 📝 Citation
```bibtex
@misc{dou2026clbenchbenchmarkcontextlearning,
title={CL-bench: A Benchmark for Context Learning},
author={Shihan Dou and Ming Zhang and Zhangyue Yin and Chenhao Huang and Yujiong Shen and Junzhe Wang and Jiayi Chen and Yuchen Ni and Junjie Ye and Cheng Zhang and Huaibing Xie and Jianglu Hu and Shaolei Wang and Weichao Wang and Yanling Xiao and Yiting Liu and Zenan Xu and Zhen Guo and Pluto Zhou and Tao Gui and Zuxuan Wu and Xipeng Qiu and Qi Zhang and Xuanjing Huang and Yu-Gang Jiang and Di Wang and Shunyu Yao},
year={2026},
eprint={2602.03587},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2602.03587},
}
@misc{dou2026clbenchlifelanguagemodels,
title={CL-bench Life: Can Language Models Learn from Real-Life Context?},
author={Shihan Dou and Yujiong Shen and Chenhao Huang and Junjie Ye and Jiayi Chen and Junzhe Wang and Qianyu He and Shichun Liu and Changze Lv and Jiahang Lin and Jiazheng Zhang and Ming Zhang and Shaofan Liu and Tao Ji and Zhangyue Yin and Cheng Zhang and Huaibing Xie and Jianglu Hu and Jingcheng Deng and Lincheng Li and Minda Hu and Shaolei Wang and Syrus Zhao and Weichao Wang and Yan Lei and Yang Liu and Yanling Xiao and Yiting Liu and Zenan Xu and Zhen Guo and Ziliang Zhao and Pluto Zhou and Tao Gui and Qi Zhang and Xuanjing Huang and Yu-Gang Jiang and Di Wang and Shunyu Yao},
year={2026},
eprint={2604.27043},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2604.27043},
}
```
Copyright © 2025-2026 Tencent Hunyuan Team & Fudan NLP Group. All rights reserved.
