Online Skill Learning for Web Agents via State-Grounded Dynamic Retrieval
Abstract
State-Grounded Dynamic Retrieval enables web agents to dynamically reuse skills based on current webpage state rather than fixed task-level strategies, improving automation performance across multiple domains.
Language agents increasingly rely on reusable skills to improve multi-step web automation across related tasks. A growing line of work studies online skill learning, where agents continually induce skills from previous task trajectories and reuse them in future tasks on the fly. However, existing methods mainly reuse skills at the task-level: a fixed set of skills is retrieved based on the initial task instruction and then held fixed throughout execution. This static strategy is misaligned with web execution, where the appropriate next action depends not only on the task goal but also on the current webpage state, which often transitions into situations that the initial skills fail to cover. To address this gap, we propose State-Grounded Dynamic Retrieval (SGDR), an online skill learning method that enables stepwise skill reuse for web agents. SGDR consists of three components: a sliding-window extraction process that turns completed trajectories into reusable sub-procedures invokable at intermediate execution states, a dual text-code representation that connects skill retrieval with executable action, and a state-grounded dynamic retrieval mechanism that matches skills to both the task goal and the current webpage state. Experiments on WebArena across five domains show that SGDR consistently outperforms strong baselines, achieving average success rates of 37.5% with GPT-4.1 and 24.3% with Qwen3-4B, corresponding to relative gains of 10.6% and 10.0% over the strongest baseline, respectively. The code is available at https://github.com/plusnli/skill-dynamic-retrieval.
Community
This paper studies online skill learning for web agents, where agents continually induce reusable skills from previous task trajectories and reuse them for future web tasks.
A key motivation is that most prior skill-based web agent methods retrieve or inject skills mainly based on the initial task instruction. However, web tasks are interactive: during execution, the webpage state keeps changing, and the skill that is useful at one step may be different from the skill needed later. This creates a mismatch between task-level skill retrieval and state-dependent decision making.
To address this, the paper proposes State-Grounded Dynamic Retrieval (SGDR). Instead of retrieving a fixed set of skills only once from the task instruction, SGDR retrieves skills dynamically at each decision step based on both the task goal and the current webpage state. This allows the agent to reuse skills that are grounded in the actual execution context, such as the current page, form, button, or interaction state.
SGDR further extracts intermediate-granularity skills from completed trajectories using sliding windows, and represents each skill as a text-code pair: natural language descriptions for retrieval and executable code for action support.
On WebArena, SGDR improves over strong online skill-learning baselines across five domains, achieving 37.5% success rate with GPT-4.1 and 24.3% with Qwen3-4B, while also reducing the average number of steps. I think this is an interesting direction for making web agents improve continuously from experience, especially by moving skill reuse from static instruction-level retrieval to dynamic state-grounded retrieval during execution.
Get this paper in your agent:
hf papers read 2606.04391 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper