PokeeResearch-7B: An Open 7B Deep-Analysis Agent Skilled with Reinforcement Studying from AI Suggestions (RLAIF) and a Strong Reasoning Scaffold

Pokee AI has open sourced PokeeResearch-7B, a 7B parameter deep analysis agent that executes full analysis loops, decomposes a question, points search and browse calls, verifies candidate solutions, then synthesizes a number of analysis threads right into a last response.

The agent runs a analysis and verification loop. In analysis, it calls exterior instruments for net search and web page studying or proposes an interim reply. In verification, it checks the reply in opposition to retrieved proof, and both accepts or restarts analysis. This construction reduces brittle trajectories and catches apparent errors earlier than finalization. The analysis crew formalizes this loop and provides a test-time synthesis stage that merges a number of unbiased analysis threads.

Coaching recipe, RLAIF with RLOO

PokeeResearch-7B is finetuned from Qwen2.5-7B-Instruct utilizing an annotation-free Reinforcement Studying from AI Suggestions, known as RLAIF, with the REINFORCE Depart-One-Out algorithm, known as RLOO. The reward targets semantic correctness, quotation faithfulness, and instruction adherence, not token overlap. The Mannequin’s Hugging Face card lists batch measurement 64, 8 analysis threads per immediate throughout RL, studying price 3e-6, 140 steps, context 32,768 tokens, bf16 precision, and a checkpoint close to 13 GB. The analysis crew emphasizes that RLOO supplies an unbiased on coverage gradient and contrasts it with the PPO household that’s roughly on coverage and biased.

Reasoning scaffold and Analysis Threads Synthesis

The scaffold contains three mechanisms. Self correction, the agent detects malformed device calls and retries. Self verification, the agent inspects its personal reply in opposition to proof. Analysis Threads Synthesis, the agent runs a number of unbiased threads per query, summarizes them, then synthesizes a last reply. The analysis crew reviews that synthesis improves accuracy on troublesome benchmarks.

Analysis protocol

The analysis crew evaluates textual content solely questions from 10 benchmarks, NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, Musique, Bamboogle, GAIA, BrowseComp, and Humanity’s Final Examination. They pattern 125 questions per dataset, besides GAIA with 103, for a complete of 1,228 questions. For every query, they run 4 analysis threads, then compute imply accuracy, imply at 4, utilizing Gemini-2.5-Flash-lite to guage correctness. The utmost interplay turns are set to 100.

Outcomes at 7B scale

PokeeResearch-7B reviews one of the best imply at 4 accuracy amongst 7B deep analysis brokers throughout the ten datasets. On HLE the mannequin reviews 15.2 with out RTS and 17.6 with RTS. On GAIA the mannequin reviews 36.9 with out RTS and 41.3 with RTS. On BrowseComp the mannequin reviews 5.4 with out RTS and eight.4 with RTS. On the seven QA benchmarks, Bamboogle, 2WikiMultiHopQA, TriviaQA, NQ, PopQA, Musique, HotpotQA, the mannequin improves over latest 7B baselines. Beneficial properties from RTS are largest on HLE, GAIA, and BrowseComp, and smaller on the QA units.

Key Takeaways

1. Coaching: PokeeResearch-7B advantageous tunes Qwen2.5-7B-Instruct with RLAIF utilizing the RLOO estimator, optimizing rewards for factual accuracy, quotation faithfulness, and instruction adherence, not token overlap.
2. Scaffold: The agent runs a analysis and verification loop with Analysis Threads Synthesis, executing a number of unbiased threads, then synthesizing proof to a last reply.
3. Analysis protocol: Benchmarks span 10 datasets with 125 questions every, besides GAIA with 103, 4 threads per query, imply@4 accuracy judged by Gemini-2.5-Flash-lite, with a 100 flip cap.
4. Outcomes and launch: PokeeResearch-7B reviews cutting-edge amongst 7B deep analysis brokers, for instance HLE 17.6 with RTS, GAIA 41.3 with RTS, BrowseComp 8.4 with RTS, and is launched underneath Apache-2.0 with code and weights public.

PokeeResearch-7B is a helpful step for sensible deep analysis brokers. It aligns coaching with RLAIF utilizing RLOO, so the target targets semantic correctness, quotation faithfulness, and instruction adherence. The reasoning scaffold contains self verification and Analysis Threads Synthesis, which improves troublesome benchmarks. The analysis makes use of imply at 4 with Gemini 2.5 Flash lite because the choose, throughout 10 datasets. The discharge ships Apache 2.0 code and weights with a transparent device stack utilizing Serper and Jina. The setup runs on a single A100 80 GB and scales.

Take a look at the Paper, Model on HF and GitHub Repo. Be happy to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Additionally, be happy to comply with us on Twitter and don’t overlook to affix our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.