Arxiv 2509 26628 Attention As A Compass Efficient Exploration For Process Sup
ASecurity
We introduce AttnRL, a novel PSRL framework which enables efficient exploration for reasoning models. Motivated by observations that steps exhibiting high attention scores correlate with reasoning behaviors, we propose to branch from positions with high values. We develop an adaptive sampling strategy and a one-step off-policy training pipeline. Extensive experiments on mathematical reasoning benchmarks demonstrate consistent improvements.
Installs into .claude/skills of the current project.
Are you the author of Arxiv 2509 26628 Attention As A Compass Efficient Exploration For Process Sup?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/hiyenwong-arxiv-2509-26628-attention-as-a-compass-efficient)
---
title: "Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models"
authors: "Runze Liu, Jiakang Wang, Yuling Shi, Zhihui Xie, Chenxin An, Kaiyan Zhang, Jian Zhao, Xiaodong Gu, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, Kun Gai"
arxiv_id: "2509.26628"
categories: "cs.LG; cs.CL"
utility: 0.9
date_added: "2026-09-29"
category: "multi-agent-rl"
---
# Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
## Abstract
We introduce AttnRL, a novel PSRL framework which enables efficient exploration for reasoning models. Motivated by observations that steps exhibiting high attention scores correlate with reasoning behaviors, we propose to branch from positions with high values. We develop an adaptive sampling strategy and a one-step off-policy training pipeline. Extensive experiments on mathematical reasoning benchmarks demonstrate consistent improvements.
## Key Contributions
- Novel approach in multi agent rl domain
- Utility score: 0.9
- Published on arXiv: 2509.26628
## Potential Applications
- Research reference for multi agent rl
- Building block for related systems
