# TOON (Token-Oriented Object Notation): When “Long Context” Becomes Affordable > TOON cuts JSON tokens ~40%, fitting 1.7× more long context By Alexander Sugakov · published 2026-02-26 · updated 2026-04-09 Source: https://donkit.ai/blog/toon-token-oriented-object-notation If you’re feeding LLMs lots of **structured** context (JSON logs, tool outputs, tables, CRM exports, config blobs), TOON is a practical lever: it keeps the *same information* but consumes *fewer tokens*, which directly improves long-context efficiency. ## What TOON is (in one sentence) **TOON is a text serialization format for JSON-like data that’s optimized to minimize LLM token usage** by removing repetitive structural tokens and making schema/fields explicit once per block. ([github.com](https://github.com/toon-format/toon)) --- ## The numbers that matter ### 1) Token reduction vs JSON Benchmarks and writeups consistently report **~30–40% fewer tokens**, with **best cases up to ~60%**, depending on how repetitive/field-heavy your JSON is. **Rule of thumb** - **Flat objects with repeated keys across many rows** → TOON shines (bigger savings). - **Deeply nested, irregular JSON** → savings shrink. ### 2) Accuracy / task performance (parsing & reasoning on structured blobs) The TOON repository reports a mixed-structure benchmark where TOON achieved **~74% accuracy vs JSON’s ~70%** while using **~40% fewer tokens**, across multiple models. ([github.com](https://github.com/toon-format/toon)) Interpretation: less token noise + clearer repeated structure can *slightly* improve model reliability on structured-input tasks, not just cost. --- ## Why TOON is *more* efficient in long context Long-context costs (and latency) are dominated by **how many tokens you send**. If you cut tokens by **40%**, you get **three compounding wins**: ### Win A — Lower prompt cost (roughly proportional) If your prompt payload is 100k tokens/day and TOON saves 40%: - JSON: 100k tokens - TOON: 60k tokens That’s **40k fewer tokens/day** (and typically ~40% lower prompt spend), independent of provider pricing. ### Win B — More information fits into the same context window If your model has a **128k** context window and your structured context is the bottleneck: - With JSON you can fit **128k tokens** of JSON. - With TOON at **40% reduction**, the *same* raw data would take ~**0.6×** the tokens. So the **effective capacity** for that same data type becomes: **128k / 0.6 = ~213k tokens worth of “JSON-equivalent content”**. That’s **~1.67× more structured context** inside the same window. ### Win C — Faster prefill (often noticeable at big contexts) At long contexts, the “prefill” phase (processing input tokens before generating) tends to dominate latency. Reducing input tokens by 30–60% usually translates to a meaningful latency reduction for long prompts, because there’s simply less to read. > Exact millisecond savings depend on model + infra, but the direction is deterministic: fewer input tokens → less prefill work. --- ## Advantages ### 1) Best-in-class for repeated-key structured dumps Think: `users[]`, `tickets[]`, `events[]`, `observability logs`, `SQL results`, `CRM rows`. TOON encodes the schema once, then streams rows compactly. ### 2) Better “schema clarity” for models Explicit headers like `{fields}` and `[N]` lengths give the model strong hints about structure (and give you deterministic parsing guardrails). ### 3) Directly improves long-context viability If you’re near context limits, TOON often prevents you from having to do lossy truncation/summarization just to fit the prompt. --- ## Disadvantages (the ones you’ll feel in production) ### 1) Not a standard (yet) JSON has universal tooling. TOON does not. You may need: - encoding/decoding libraries - linters/formatters - schema/version handling - internal guidelines for contributors ### 2) Interop friction with existing pipelines If your ecosystem expects JSON at every boundary (Kafka topics, REST payloads, logs, ETL jobs), TOON adds conversion steps. That can mean: - extra CPU for encode/decode - more moving parts to monitor - more “what format is this?” debugging ### 3) Less friendly for ad-hoc human editing It’s readable, but not as universally familiar as JSON. On-call engineers who can eyeball JSON instantly may slow down with TOON at first. ### 4) Savings aren’t guaranteed If your data is highly irregular, deeply nested, or mostly free-form text (not repeated structure), token savings can be modest. --- ## When TOON is the right call (decision checklist) Use TOON when **all** are true: - You pass **structured** context frequently (tool results, tables, logs, knowledge graphs). - Prompt size is a meaningful pain (cost, latency, or context limits). - You can control the serialization at least at the LLM boundary. Skip TOON when: - Your payload is mostly **natural language** (docs, emails, chats) → compressing structure won’t help much. - You depend on many third-party systems that require JSON end-to-end. --- ## What this means for RAG + agents Agentic systems often shove **tool outputs + retrieved metadata** back into context repeatedly. That’s exactly where token bloat accumulates. In practice, TOON is most valuable in RAG/agent flows for: - large tool results (SQL/CRM/export tables) - long audit trails / run logs - multi-step planners that carry state forward Reducing that overhead is one of the simplest ways to make “long context” actually usable without paying luxury-car money per prompt.