LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent.

One sentence.

LiteResearcher-4B is a 4B deep research agent trained with zero marginal RL API cost, outperforming 30B open-source deep research agents and matching frontier systems such as Claude-4.5-Sonnet and GPT-5-high. Its RL stage runs entirely in a local search/browse environment, enabling 73.2M tool calls without live search or browse API consumption.

The shape of the result.

The full story.

Reinforcement Learning (RL) has emerged as a powerful training paradigm for LLM-based agents. However, scaling agentic RL for deep research remains constrained by two coupled challenges: hand-crafted synthetic data fail to elicit genuine real-world search capabilities, and real-world search dependency during RL training introduces instability and expensive cost, which limit the scalability of Agentic RL.

LiteResearcher is a training framework to make Agentic RL scalable and low-cost: by constructing a lite virtual world that mirrors the real-world search dynamics, we enabled a continuously improving training recipe that empowers a tiny search agent to outperform large-scale open-source and commercial models (e.g. Tongyi DeepResearch and Claude-4.5 Sonnet). The RL stage runs entirely in a local search/browse environment, removing external API consumption during RL while preserving realistic tool-use dynamics. Specifically, on most common benchmarks like GAIA and Xbench, our LiteResearcher-4B achieves the open-source state-of-the-art results of 71.3% and 78.0% respectively, proving that scalable RL training is essential for Deep Research Agents.

Three pillars.

LiteResearcher constructs a virtual world with identical architecture to the real web but isolated in execution. The framework consists of three key components:

(1) Co-constructed Training Data & Corpus: We scale up information sources (32M+ webpages, 1M+ domains) and identify five atomic search capabilities — direct retrieval, aggregation, enumeration, cross-verification, and statistics — to generate diverse, realistic training tasks.

(2) Stable Local Tool Environment: A local search engine (BGE-M3 + Milvus, ~0.15s/query) and local browse tool (PostgreSQL, ~0.17s/page) that enable 73.2M tool calls during training fully locally, with no external API consumption during RL and zero marginal tool cost.

(3) Difficulty-Aware Curriculum RL: Multi-stage training that progressively increases task difficulty and context length, retaining only partially-solvable instances to maintain consistent training signal.

Build the local environment.

The local search/browse environment (BGE-M3 + Milvus hybrid search) is open-sourced under Environment/, and the ~32M-record corpus is on HuggingFace. Stand up your own local search tool in four steps:

# 0. Install (also needs local BGE-M3 weights + a Redis instance)
pip install -r requirements.txt

# 1. Start Milvus (edit milvus_config/.env first)
cd milvus_config && docker-compose up -d

# 2. Download the released 32M corpus and decompress
huggingface-cli download simplex-ai-inc/LiteResearcher-Corpus \
  serper_test_text.jsonl.zst --repo-type dataset --local-dir ./corpus
zstd -d ./corpus/serper_test_text.jsonl.zst

# 3. Build the index (configure paths/collection in config.py)
python data.py

# 4. Serve hybrid search at :8018
cd server && REDIS_HOST=127.0.0.1 EMBED_WORKERS=1 bash start.sh

Full setup, API reference, and optional PostgreSQL full-text fetch are documented in the Environment README.

Main Results.

LiteResearcher-4B consistently outperforms open-source models up to 8× larger and matches or exceeds proprietary systems across eight benchmarks, while remaining a low-cost 4B agent trained with fully local RL tool calls.

Best open-source results in bold. Results with * use a 64k context window with a memory mechanism.

Curriculum keeps signal alive.

Our difficulty-aware curriculum learning prevents training saturation. Stage 2 with adjusted difficulty yields +3.6% GAIA accuracy after Stage 1 plateaus, demonstrating the importance of progressive curriculum design.

Watch it think.

A few of our trajectories — pick one and watch LiteResearcher-4B think, search, and answer.

Open full trajectory viewer →

BibTeX.

@article{li2026literesearcher,
  title={LiteResearcher: A Scalable Agentic RL Training Framework for Deep Research Agent},
  author={Li, Wanli and Qu, Bince and Pan, Bo and Zhang, Jianyu and Liu, Zheng and Zhang, Pan and Chen, Wei and Zhang, Bo},
  journal={arXiv preprint arXiv:2604.17931},
  year={2026}
}