tokin: Make Any Agent Harness Token‑Native

Harnesses speak text

A common framing today is Agent = Model + Harness. Harnesses such as Claude Code, Codex, pi, opencode and Terminus, simple or elaborate, define the interaction loop: when to send the conversation so far to the model, and how to act on its reply.

A harness, though, speaks text, and RL trains on tokens.¹Note 1We consider text here. With images in the conversation, a harness still speaks messages rather than the model’s inputs, and what the model samples is still text tokens, so the same argument holds there. tokin v0.1.0 handles text only. Connect a harness straight to the inference engine and every turn makes a round trip: the model samples token ids $y$, the engine decodes and parses them into a text message, the harness acts on it, and its next request sends back the whole conversation, which the engine renders through the chat template and encodes again. Nothing makes this round trip return the ids the model sampled (Figure 2), and the chat template may further drop past reasoning or re-serialize tool calls. Wherever an id changes, the trainer updates a token the policy never sampled, a harm discussed in [1, 2, 3, 4].

A gateway with two properties

tokin solves this for any harness.²Note 2Appendix A compares tokin with related projects. It is a gateway between harnesses and the inference engine (Figure 1), and it guarantees two properties. For the request at turn $t$ of a trajectory (defined below), write $h_t$ for the history the harness sends, a list of messages; $x_t$ for the prompt ids tokin sends the engine; $y_t$ for the token ids the model samples back; $a_t$ for the assistant message tokin parses from them; and $o_t$ for the observation the request adds, such as tool results, or the task at $t = 0$. Juxtaposition concatenates sequences, ids and messages alike, e.g., $x_t\, y_t$ is $x_t$ followed by $y_t$.

A harness thus needs only a new base URL, and tokin recovers all token state from the history. A request continues an earlier one when its history is that request’s history and reply, matched by text, reasoning and call ids, plus new messages, such as tool results, under the same tools; a reply cut at the length limit is never continued. tokin makes each request a child of the deepest request it continues, the latest on a tie, or a root if it continues none. Usually the parent is the latest request, but a retry or an edit to history attaches to an earlier one, so the requests of a session form a tree, or several after a compaction (Figure 3). Each path from a root to a leaf is one trajectory, and $t$ counts turns within it:

$$h_t = h_{t-1}\, a_{t-1}\, o_t, \qquad h_0 = o_0.$$

The property holds by construction: tokin stores each turn’s $\Delta_t$ and $y_t$ and sends $x_t$ as the stored ids of turns $0$ to $t-1$ followed by $\Delta_t$, so the only ids it ever encodes are each turn’s $\Delta_t$. A trajectory ending at turn $T$ is the rollout $x_T\, y_T$: every $y$ in it exactly as sampled and every $\Delta$ masked from the loss.

Inside the gateway

Figure 1 follows one request through tokin. The trainer opens a session per sample with POST /sessions and hands the harness its base URL, /sessions/{id}/v1; from there the harness is an ordinary OpenAI client. For every request, tokin:

1. attaches it to its parent in the session, turn $t-1$ of its trajectory;
2. renders only the observation $o_t$, encodes it as $\Delta_t$, and sends $x_t = x_{t-1}\, y_{t-1}\, \Delta_t$;
3. generates on the engine from token ids to token ids, keeping each position’s logprob and the weight version the engine reports for it and, when the trainer asks, the sampler’s top-p support and the experts each token was routed to;
4. parses $y_t$ into the assistant message $a_t$, with its reasoning and tool calls, for the harness.

Take the session of Figure 3. The harness first sends the task, $o_0$. Nothing precedes it, so request A is a root: tokin renders $o_0$ with the tools into $x_0 = \Delta_0$, the engine samples $y_0$, say a tool call, and the harness gets it back as $a_0$. The harness runs the tool and sends $h_1 = o_0\, a_0\, o_1$. Request B continues A, so tokin encodes only the tool result $o_1$ and sends $x_1 = x_0\, y_0\, \Delta_1$: the ids of $a_0$ are $y_0$ as sampled, not $a_0$ encoded again. Request C continues B the same way. Request D resends the conversation with $o_1$ rewritten as $o_1'$; it continues A but not B, so it branches at A and sends $x_1' = x_0\, y_0\, \Delta_1'$. A compaction, which many harnesses do (Claude Code, Codex, opencode, pi), replaces the history with a summary and starts a new root.

When the harness finishes, the trainer collects the session with GET /sessions/{id}, one safetensors payload that stores each turn once, however many rollouts share it. Here it holds two rollouts, C and D, sharing $x_0\, y_0$. Session.collect reads it back as a Session, whose rollouts each give token_ids ($x_T\, y_T$), a loss_mask that is 1 on every $y$ and 0 on every $\Delta$, and aligned logprobs; each turn keeps the rest of what step 3 returned.

Training the pi coding agent with tokin

We train the pi coding agent on the code tasks of MiMo-V2.6-RL-oss, holding out 256 for validation,³Note 3Each task is a Harbor task, from MiMo-V2.6-RL-harbor-code, run in an E2B sandbox; the held-out tasks are listed in heldout.txt. with slime [5] in fully asynchronous mode as the trainer. examples/pi_mimo_rl has the recipe to reproduce it. Each trial runs pi on its own tokin session and is rewarded by the task’s verifier. A session can yield several rollouts, one per leaf, and each becomes its own training sample. They share the session’s one reward and count once in the group-mean advantage [6]. Where they share a prefix, its sampled tokens are trained only once: the recipe sets their loss mask to 1 in the first rollout and 0 in the others.⁴Note 4In 400 sessions sampled from the 27B run, 85% have one rollout; 14.5% split at compactions and 0.5% at a turn cut on length; none at a retry. The algorithm is REINFORCE [7] with truncated importance sampling [8], score centering [9] and dynamic sampling [10]. We train a dense model, Qwen3.8-27B, and an MoE model, Qwen3.5-35B-A3B, which also replays the sampler’s expert routing in training (R3) [11]; Table 1 gives their settings.

The train reward rises for both models (Figure 4, left): from about 0.5 to 0.8 over Qwen3.8-27B’s 300 steps, and from about 0.27 to 0.40 over Qwen3.5-35B-A3B’s 140. Held-out MiMo pass@1, evaluated every 60 steps and every 10 near the end, rises with it, from 0.50 to 0.80 for the 27B model and from 0.24 to 0.37 for the 35B.

We report each run’s last checkpoint (Table 2). Held-out MiMo measures generalization to unseen tasks from the training distribution; Terminal-Bench 2.1 (89 tasks) measures transfer beyond it. Both evaluations run pi with 4 samples per task and a per-turn limit of 32,768 tokens.⁵Note 5Training caps a turn at 8,192 tokens to keep rollouts fast; evaluation allows 32,768 so the model’s reasoning is not cut short.

tokin’s release

We release tokin v0.1.0 as a beta: a harness keeps the API it already speaks, and every prefix of its rollouts stays exactly as sampled. The pi recipe above shows it at work, improving on held-out MiMo and on Terminal-Bench 2.1 for a dense and an MoE model (Table 2).

tokin has run more than pi. examples/harbor_eval evaluates mini-swe-agent, opencode and pi through it on Terminal-Bench 2.1 and SWE-bench Verified. tokin currently covers the chat templates of Qwen2.5–Qwen3.8 and GLM-4.5–GLM-5.3, serves OpenAI’s chat completions, and generates on SGLang. Next are the APIs other harnesses speak, such as OpenAI’s Responses API for Codex and Anthropic’s Messages API for Claude Code; each only translates requests and replies, and the rollout management behind them stays the same.

Community feedback is highly appreciated, and contributions are welcome, from a model family or a tool-call grammar to a harness API or a practical example. If you are interested, open an issue or a pull request on open-rollout/tokin.

References

• [1] The Agent Lightning (AGL) Team. No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL. vLLM blog, 2025.
• [2] Yuan He and Chengxing Xie. Strands-SGLang: Bridging Agent Scaffolding and RL Training. yuanhe.wiki, 2026.
• [3] Quentin Gallouédec and Kashif Rasul. Agentic RL: Token-In, Token-Out Done Right. Hugging Face blog, 2026.
• [4] Miles Team (Jiajun Li et al.). No Token Left Behind: Demystifying Token-In-Token-Out in Miles. LMSYS blog, 2026.
• [5] Zilin Zhu, Chengxing Xie, Xin Lv and slime Contributors. slime: An LLM post-training framework for RL Scaling. github.com/THUDM/slime, 2025.
• [6] Zhihong Shao et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300, 2024.
• [7] Ronald J. Williams. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning 8(3–4):229–256, 1992.
• [8] Feng Yao et al. On the Rollout-Training Mismatch in Modern RL Systems. NeurIPS 2025 Workshop on Efficient Reasoning (OpenReview); blog version: Your Efficient RL Framework Secretly Brings You Off-Policy RL Training (blog), 2025.
• [9] Martin Marek and Max Ryabinin. Score Centering Stabilizes Off-policy Reinforcement Learning. arXiv:2609.20807, 2026.
• [10] Qiying Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476, 2025.
• [11] Wenhan Ma et al. Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers. arXiv:2510.11370, 2025.
• [12] Zhiyuan He et al. Agent Lightning v1.0: Towards Harnessed Agentic RL. arXiv:2608.17528, 2026.

Appendix

A. tokin compared with related projects

Several open projects also record the token ids of an agent’s turns. Table 3 compares tokin with three of them: Agent Lightning [1], strands-sglang [2] and Miles [4].⁶Note 6Read at Agent Lightning bc3a9e8, strands-sglang dda03af and Miles 2f627e0.

• Any harness. strands-sglang serves only agents built on Strands Agents; the other three serve harnesses over HTTP.
• TITO at every prefix. Agent Lightning has vLLM re-render each chat request and joins calls into one sample only where the ids happen to match; only an agent that tokenizes for itself and sends ids to /v1/completions keeps the sampled ids.⁷Note 7By design [12], Agent Lightning matches a plain deployment, so it starts a new sample wherever TITO breaks. tokin keeps TITO throughout, and Appendix B finds that doing so has no measurable effect on eval. strands-sglang holds TITO only while the history grows, and Miles at any earlier prefix only in its experimental v2.
• Tree of turns. When the harness retries a turn, Agent Lightning keeps only the last attempt; when the harness edits or compacts its history, Agent Lightning starts a new sample. strands-sglang raises on a shorter history and, after an edit, still sends the old ids. Miles keeps a tree in its experimental v2.
• Any trainer. Agent Lightning ships only verl, though it exports each call’s ids over HTTP. Miles launches its session server from its own training arguments and trains on its samples, though the server exports them over HTTP.

B. tokin has no measurable effect on evaluation

With tokin, later turns see the sampled ids rather than their re-encoding. To check whether that changes evaluation with the policy frozen, we ran Qwen3.6-35B-A3B with mini-swe-agent, opencode and pi on Terminal-Bench 2.1 (89 tasks × 4 samples) and SWE-bench Verified (500 tasks × 1 sample), once through tokin and once through the same gateway re-rendering every turn from text. Over the 2,568 paired samples, pass@1 through tokin minus re-rendering is +0.1 points (95% CI [−1.5, +1.7]).