You Don't Need Long Context: Compaction, Selective Reading, and Semantic State Space Models

Key Takeaway

NVIDIA researchers evaluate whether frontier language models require 200K–1M token contexts (arXiv:2611.05306). Serving at 16K instead of 256K boosts H100 decode throughput by 3.6x–7x, End-of-Conversation Compaction and Selective Reading match or beat full-context baselines across 13 benchmarks, and Single-Action Compaction solves 70.2%–82.4% of SWE-bench Verified at a 7K–8K median peak context.

Abstract and Core Findings

Later tokens in a language model's context cost substantially more to generate than earlier tokens because attention compute and KV-cache memory grow linearly with context depth. In You don't need long context (arXiv:2611.05306), Peter Belcak, Saurav Muralidharan, Jindong Jiang, Yonggan Fu, Wonmin Byeon, Hanrong Ye, Ryan Angilly, Ali Taghibakhshi, Shih-Yang Liu, Siru Ouyang, Greg Heinrich, and Pavlo Molchanov at NVIDIA test whether long-horizon agent workflows and long-input reasoning tasks actually require large active contexts.

The study evaluates two closed-weight frontier models (Claude Opus 4.5 and GPT 5.6 Sol) and six open-weight coding and reasoning models (GLM 5.2, Kimi K2.6, Qwen 3 Coder Next, Qwen 3.6 27B, Gemma 4 31B, Step 3.7 Flash, and MiniMax M2.7) across six long-horizon agent benchmarks and seven long-input document benchmarks. Four empirical results anchor the paper:

  1. Serving at 16K instead of 256K yields a 3.6x to 7.0x decode speedup and cost reduction. Under saturated vLLM serving on NVIDIA H100 nodes ($3.85 per GPU-hour), shrinking the configured context window size from 256K to 16K tokens increases sustained decode throughput by 3.6x for GLM 5.2, 5.6x for Kimi K2.6, and 7.0x for Qwen 3 Coder Next. On a local NVIDIA DGX Spark (128 GB unified memory), inter-token latency approximately doubles across the 256K context range for multiple models, and MiniMax M2.7 runs out of memory around 60K tokens.
  2. Long-horizon agent tasks maintain accuracy under End-of-Conversation Compaction (EoCC). Across SWE-bench Verified, SWE-bench Multilingual, Terminal-Bench 2.1, FRAMES, BFCLv3, and Tau2Bench, capping total context between 8K and 48K tokens keeps Claude Opus 4.5 within -3.7 to +5.6 percentage points (pp) of its full-history baseline and GPT 5.6 Sol within -3.3 to +4.4 pp. For either model, at most 5 of 22 compaction configurations lose more than 2 pp, while estimated optimized retail costs drop by 2.2x to 8.8x for Opus 4.5 and 2.4x to 5.0x for GPT 5.6 Sol.
  3. Long-input benchmarks improve when reduced to selective file reading plus fallback compaction. Exposing long documents as files inspected via grep and bounded doc-read calls ( characters per read) shifts Claude Opus 4.5 accuracy by -0.4 to +40.0 pp and GPT 5.6 Sol accuracy by +1.0 to +30.9 pp across LongBench v2, LOONG, AA-LCR, NIAH, RULER, Counting-Stars, and BABILong. Estimated optimized retail costs fall by 6.5x to 54.6x for Opus 4.5 and 4.4x to 233.3x for GPT 5.6 Sol, with a median of zero compactions on most tasks.
  4. Single-Action Compaction (SAC) works with 7K–9K median peak contexts, motivating Semantic State Space Models (SSSMs). Restricting mini-swe-agent to at most one environment action per conversation before resetting to a natural-language state description reaches 70.2% on SWE-bench Verified (+0.2 pp over full history) and 51.8% on Terminal-Bench 2.1 (+4.6 pp) with Claude Opus 4.5, while cutting peak context from a maximum of 200K down to 16K tokens (median 8K). GPT 5.6 Sol reaches 82.4% (-2.0 pp) and 84.3% (-3.3 pp) at a 7K–8K median peak context.

1. The Economics and Systems Burden of Long Context

Retail API pricing quotes a flat fee per million input or output tokens across most of a model's supported context window. That flat retail rate masks the underlying hardware cost of generating tokens at deep context positions. In standard attention architectures, every decoded token reads the key-value (KV) cache of all preceding tokens in the input context, so memory bandwidth consumption and KV-cache capacity scale linearly with context depth.

Figure 1: Dollar cost of generating 1M output tokens on an existing input KV cache as a function of context window size on H100 GPUs.

Belcak et al. measure sustained aggregate decode throughput under vLLM on the smallest H100 deployment capable of holding each model checkpoint and its 256K KV cache (eight H100 GPUs for Qwen 3 Coder Next in BF16; sixteen H100 GPUs for GLM 5.2 and Kimi K2.6 with INT4 weights and BF16 activations, plus FP8 KV cache on Kimi K2.6). Converting peak decode throughput into dollar cost per 1M output tokens at Nebius on-demand H100 rates ($3.85 per GPU-hour, September 2026) reveals steep cost escalation between 16K and 256K context window sizes:

Figure 4: Peak sustained decode throughput (tokens/s) and speedup vs 256K for open-weight coding models served under vLLM on NVIDIA H100 GPUs.

Open-Weight Model Serving Precision & Hardware 16K-to-256K Cost Ratio () Decode Speedup at 16K vs. 256K
GLM 5.2 INT4 experts, BF16 activations (16x H100) 3.6x
Kimi K2.6 INT4 weights, BF16 activations, FP8 KV (16x H100) 5.6x
Qwen 3 Coder Next BF16 weights and activations (8x H100) 7.0x

Across the three deployments, the mean 16K-to-256K output cost ratio is (a 5.0x average cost reduction at 16K). This gain matches or exceeds the throughput improvements typically obtained from 4-bit weight quantization or structured sparsity.

Figure 2: Inter-token latency, batch-1 decode throughput, and memory footprint of five open-weight models on an NVIDIA DGX Spark across context lengths up to 256K.

Local edge deployments exhibit the same degradation under single-request (batch size 1) execution. On an NVIDIA DGX Spark with 128 GB of unified memory (~109 GB usable for inference), the authors benchmark five quantized coding models across context lengths up to 256K (Qwen 3.6 27B and Qwen 3 Coder Next in NVFP4; Gemma 4 31B, Step 3.7 Flash 198B A11B, and MiniMax M2.7 230B A10B in 3- to 4-bit GGUF via llama.cpp). Inter-token latency roughly doubles between short prefixes and 256K tokens for Gemma 4 31B, Step 3.7 Flash, and Qwen 3.6 27B, while MiniMax M2.7 exhausts the 109 GB device memory limit near 60K tokens.

To place modern 200K–1.05M context windows in scale, Table 1 of the paper compares model context capacities against standard English literary works (assuming ~1.33 tokens per word):

Literary Work Approx. Words Approx. Tokens Model Approx. Words Context Window Size
The Old Man and the Sea ~27K ~37K MiniMax M2.7 ~150K 200K
The Great Gatsby ~48K ~64K Claude Opus 4.5 ~150K 200K
Qur'an (Saheeh Int'l) ~156K ~207K Gemma 4 31B ~192K 256K
King James New Testament ~181K ~241K Kimi K2.6 ~192K 256K
Great Expectations ~184K ~245K Qwen 3 Coder Next ~197K 262K
The Lord of the Rings ~481K ~640K Gemini 3.8 Flash ~750K 1M
War and Peace ~566K ~753K GLM 5.2 ~750K 1M
King James Old Testament ~603K ~802K GPT 5.6 Sol ~790K 1.05M

A "short" working context of 16K to 50K tokens already holds an entire novella such as The Old Man and the Sea (37K tokens) or most of The Great Gatsby (64K tokens).


2. Two Sources of Long Context and Three Management Regimes

Long contexts in language model workloads arise from two distinct operational patterns:

  1. Long-horizon tasks (LCW1): Sequential workflows where an agent executes many environment actions and appends every command, tool output, retrieved passage, and intermediate decision to a growing conversation.
  2. Long-input tasks (LCW2): Workloads accompanied by large source documents (financial filings, code repositories, multi-document corpora) that form a long context if pasted directly into the initial prompt.

Every LCW2 task can be reduced to an LCW1 task by storing the source documents as files in the execution environment and giving the agent selective reading tools (grep, sed, awk, head, doc-read). Under that reduction, the root question Are long contexts necessary? (RQ0) decomposes into three testable questions:

  • RQ1: Can long-horizon tasks be completed without long contexts?
  • RQ2: Can long-input tasks be completed without long contexts?
  • RQ3: Can long-horizon tasks be completed taking no more than a single action per context?

Figure 3: The three context-management regimes evaluated in the paper: Full-History Accumulation (FHA), End-of-Conversation Compaction (EoCC), and Single-Action Compaction (SAC).

To test RQ1–RQ3, the authors pair two minimal ReAct Bash harnesses (mini-swe-agent for repository/container tasks and mini-agent for general tool/QA benchmarks) with three context-management regimes:

Context-Management Regime Operational Rule Active Context Profile
Full-History Accumulation (FHA) Appends every assistant message, tool call, and tool response to a single unbroken conversation for the entire episode. Grows monotonically with step count; peak context equals final conversation length.
End-of-Conversation Compaction (EoCC) Runs standard ReAct until the conversation reaches a pre-set compaction threshold, prompts the model to write a structured handoff recap within a total context limit, and starts a fresh conversation seeded only with the system prompt, task, and compacted state. Bounded by the total context limit (evaluated at 48K/36K, 32K/24K, 24K/18K, 16K/12K, 12K/8K, and 8K/6K).
Single-Action Compaction (SAC) Starts each conversation with the task and current semantic state, executes exactly one environment action, observes the output, updates the semantic state, and resets the conversation. Shortest possible active window; median peak context is 7K–9K tokens.

What Makes a Compaction Preserve Accuracy?

In Appendix A and Appendix G, the authors compare generic conversation summarization against structured execution briefs. Generic summarization weights repeated historical events equally and often drops user constraints or file paths needed for upcoming edits. Accuracy-preserving compactions act as forward-looking handoff briefs built around six explicit fields (mini-swe-agent compaction prompt):

Stop work. Write only a concise handoff recap; do not run commands or call tools.
The conversation will reset to the system prompt, task, and this recap.
Record:
1. Objective - one line describing the required result.
2. Established facts - exact paths, relevant symbols or commands, observed values,
   and important errors/output. Separate observations from unverified ideas.
3. Applied changes - each changed/created file, what changed, and whether verified.
4. Verification - exact checks run, outcomes, failures, obstacles, and workarounds.
5. Dead ends - failed approaches that should not be repeated.
6. Remaining plan - outstanding subgoals, in execution order; the immediate next
   command or edit; and the condition for completion.
If needed output was truncated, give a narrow command to retrieve it.

Modeling Optimized Retail Cost (EORC)

Because cloud GPU throughput increases when the maximum context window shrinks, Appendix B defines the Estimated Optimized Retail Cost (EORC) for a model operating under total context limit (where for Opus 4.5 at output tokens, and for GPT 5.6 Sol at output tokens). Anchoring at the 16K-to-256K mean cost ratio :

Preserving the 5:1 output-to-uncached-input price ratio and 10:1 uncached-to-cached-input ratio ( and ), the total cost for million uncached input tokens, million cached input tokens, and million output tokens is:


3. RQ1: Long-Horizon Tasks Under Frequent Compaction

The authors evaluate EoCC across six long-horizon benchmarks: SWE-bench Verified (500 GitHub issues), SWE-bench Multilingual (300 issues across 9 languages), Terminal-Bench 2.1 (89 container CLI tasks), FRAMES (824 multi-hop Wikipedia questions requiring 2–15 articles), BFCLv3 (200 multi-turn tool-use episodes), and Tau2Bench (184 dual-control retail, airline, and telecom customer-service tasks).

Figure 5: Long-horizon accuracy changes (percentage points) and estimated optimized retail cost savings across six benchmarks as total context limit decreases from Baseline down to 16K/12K/8K.

Across all six benchmarks, shrinking the context window from full history down to 16K–48K preserves or improves accuracy on the vast majority of settings while cutting EORC by 2.2x to 8.8x:

Benchmark & Configuration (Limit / Threshold) Claude Opus 4.5 Accuracy (%) GPT 5.6 Sol Accuracy (%) Opus 4.5 Nominal / Optimized Cost ($) GPT 5.6 Sol Nominal / Optimized Cost ($) Opus 4.5 Compactions (Mean / Med / Max) GPT 5.6 Sol Compactions (Mean / Med / Max)
SWE-bench Verified (500 tasks)
Baseline (no compaction) 70.0 84.4 $420.00 / $420.00 $167.68 / $167.68 0 / 0 / 0 0 / 0 / 0
Compaction, 48K / 36K 73.6 88.8 $379.85 / $130.20 $186.08 / $55.82 0.3 / 0 / 6 0.4 / 0 / 5
Compaction, 32K / 24K 72.0 85.0 $360.45 / $97.82 $184.16 / $46.04 0.8 / 1 / 13 0.9 / 1 / 9
Compaction, 24K / 18K 74.8 85.0 $295.75 / $69.71 $188.96 / $42.52 1.0 / 1 / 18 1.5 / 1 / 18
Compaction, 16K / 12K 68.8 84.6 $312.65 / $62.53 $203.12 / $40.62 2.1 / 1 / 32 3.0 / 2 / 61
Terminal-Bench 2.1 (89 tasks)
Baseline (no compaction) 47.2 87.6 $87.80 / $87.80 $48.56 / $48.56 0 / 0 / 0 0 / 0 / 0
Compaction, 48K / 36K 50.6 88.8 $114.75 / $39.33 $47.48 / $14.24 0.9 / 0 / 7 0.6 / 0 / 8
Compaction, 32K / 24K 52.8 86.5 $99.55 / $27.02 $53.64 / $13.41 1.7 / 1 / 13 1.4 / 0 / 18
Compaction, 16K / 12K 48.3 85.4 $89.50 / $17.90 $60.92 / $12.18 3.5 / 1 / 25 3.8 / 2 / 29
SWE-bench Multilingual (300 tasks)
Baseline (no compaction) 66.3 79.3 $326.55 / $326.55 $88.08 / $88.08 0 / 0 / 0 0 / 0 / 0
Compaction, 48K / 36K 68.0 80.0 $259.25 / $87.56 $99.44 / $29.48 0.4 / 0 / 3 0.3 / 0 / 5
Compaction, 16K / 12K 64.3 77.7 $185.20 / $37.04 $124.20 / $24.84 2.7 / 2 / 26 3.4 / 2 / 44
FRAMES (824 tasks)
Baseline (no compaction) 81.0 90.2 $507.75 / $507.75 $206.64 / $206.64 0 / 0 / 0 0 / 0 / 0
Compaction, 16K / 12K 79.6 87.1 $327.60 / $65.52 $271.68 / $54.34 1.6 / 1 / 9 2.6 / 1 / 13
BFCLv3 (200 episodes)
Baseline (no compaction) 62.0 39.5 $34.95 / $34.95 $15.20 / $15.20 0.0 / 0 / 0 0.0 / 0 / 0
Compaction, 24K / 18K 64.0 39.0 $36.80 / $8.64 $16.60 / $3.74 0.3 / 0 / 3 0.2 / 0 / 2
Compaction, 12K / 8K 62.5 39.5 $38.25 / $7.65 $17.56 / $3.51 1.1 / 1 / 5 0.6 / 1 / 4
Tau2Bench (184 tasks)
Baseline (no compaction) 79.3 73.9 $80.00 / $80.00 $49.60 / $49.60 — —
Compaction, 12K / 8K 78.2 77.2 $79.00 / $15.80 $66.80 / $13.36 — —
Compaction, 8K / 6K 78.8 73.9 $73.50 / $14.70 $63.60 / $12.72 — —

On SWE-bench Verified, EoCC boosts Claude Opus 4.5 by +4.8 pp at 24K/18K (, reducing optimized cost from to , a 6.0x saving) and boosts GPT 5.6 Sol by +4.4 pp at 48K/36K (, reducing optimized cost from to , a 3.0x saving). Even nominal API cost for Opus 4.5 drops from to at 24K/18K because compacting eliminates hundreds of millions of repeated cached-prefix tokens ( cached tokens).

The only benchmark with a consistent small accuracy dip under EoCC is FRAMES (-1.3 to -3.7 pp for Opus 4.5; -1.5 to -3.3 pp for GPT 5.6 Sol), where multi-hop entity synthesis across up to 15 Wikipedia articles occasionally loses an intermediate fact during lossy compaction. Section 3.1 of the paper notes that even when a task requires exact historical lookup (such as querying the -th outcome of a sequence of coin tosses), the harness can log raw conversation turns to an external file and let the agent query that log on a need-to-know basis via selective reading.

Open-Weight Models Under Tight Compaction

Appendix D tests the five locally deployed quantized open-weight models on SWE-bench Verified and Terminal-Bench 2.1 under EoCC (48K/40K, 32K/24K, and 16K/12K):

Open-Weight Model SWE-bench Verified Base / 48K / 32K / 16K (%) Terminal-Bench 2.1 Base / 48K / 32K / 16K (%) Compaction Behavior
Qwen 3.6 27B (NVFP4) 68.6 / 69.4 / 68.2 / 65.0 43.8 / 48.3 / 49.4 / 43.8 Highly robust; improves on Terminal-Bench at 48K and 32K and retains baseline at 16K.
MiniMax M2.7 (IQ3_XXS) 72.6 / 73.0 / 73.2 / 67.8 40.4 / 37.1 / 38.2 / 40.4 Gains +0.6 pp on SWE-bench at 32K and matches Terminal-Bench baseline at 16K.
Gemma 4 31B (Q4_K_M) 65.2 / 65.0 / 65.8 / 58.0 40.4 / 34.8 / 29.2 / 33.7 Holds steady through 32K on SWE-bench (-7.2 pp at 16K).
Step 3.7 Flash (Q3_K_M) 70.0 / 66.8 / 64.6 / 56.6 41.6 / 32.6 / 34.8 / 28.1 Gradual accuracy decay as compaction frequency increases.
Qwen 3 Coder Next (NVFP4) 53.0 / 49.2 / 40.4 / 3.6 29.2 / 34.8 / 34.8 / 30.3 Collapses to 3.6% on SWE-bench at 16K/12K because verbose single-turn trajectories consume the 12K threshold before making progress.

The Qwen 3 Coder Next collapse at 16K/12K highlights an important boundary condition: when a model generates overly verbose tool calls or reasoning traces without post-training for compact state handoffs, a 12K threshold triggers compaction before the agent can complete a meaningful edit cycle.


4. RQ2: Long-Input Tasks via Selective Reading

To evaluate RQ2, the authors compare full-input ingestion (pasting the entire document corpus into the initial conversation) against selective input reading (mini-agent with documents stored as files in the Bash environment, plus EoCC as a fallback). The system prompt provides grep -n -i -E, doc-stats, docs-list, doc-syn, and a bounded reader doc-read [--start CHAR] [--chars N] [--lines A:B] capped at 6,000–8,000 characters per call.

Figure 6: Long-input accuracy changes (percentage points) and estimated optimized retail cost savings across seven long-document and needle benchmarks.

Across all seven long-input benchmarks, selective reading with short contexts (16K–48K) matches or substantially outperforms pasting the full documents into the context window:

Long-Input Benchmark & Setting Claude Opus 4.5 Base vs. Short Context (%) GPT 5.6 Sol Base vs. Short Context (%) Opus 4.5 EORC Savings ($ / Multiple) GPT 5.6 Sol EORC Savings ($ / Multiple) Median Compactions (Opus / GPT)
LongBench v2 (503 QA, 8K–2M words, 16K/12K) 68.0 vs. 67.6 (-0.4 pp) 70.8 vs. 72.6 (+1.8 pp) $274.70 → $37.04 (7.4x) $79.20 → $18.04 (4.4x) 1 / 0
LOONG (1,600 multi-doc QA, avg score, 32K/24K) 70.1 vs. 80.2 (+10.1 pp) 79.3 vs. 84.0 (+4.7 pp) $1367.50 → $134.18 (10.2x) $1090.00 → $62.47 (17.4x) 0 / 0
LOONG (1,600 multi-doc QA, avg score, 16K/12K) 70.1 vs. 79.3 (+9.2 pp) 79.3 vs. 82.7 (+3.4 pp) $1367.50 → $100.42 (13.6x) $1090.00 → $84.56 (12.9x) 1 / 0
AA-LCR (100 real-world doc sets ~100K tok, 16K/12K) 65.0 vs. 84.0 (+19.0 pp) 82.0 vs. 83.0 (+1.0 pp) $52.00 → $5.30 (9.8x) $37.60 → $2.32 (16.2x) 0 / 0
NIAH (125 grid instances, 16K–500K tok, 48K/36K) 60.0 vs. 100.0 (+40.0 pp) 80.0 vs. 100.0 (+20.0 pp) $74.00 → $1.36 (54.6x) $56.00 → $0.24 (233.3x) 0 / 0
RULER (13 synthetic tasks at 256K & 500K, recall, 16K/12K) 62.5 vs. 95.7 (+33.2 pp) 65.5 vs. 96.4 (+30.9 pp) $880.00 → $20.57 (42.8x) $725.60 → $9.02 (80.5x) 0 / 0
Counting-Stars (192 tasks, exact match, 16K/12K) 65.1 vs. 81.2 (+16.1 pp) 76.6 vs. 81.2 (+4.6 pp) $55.00 → $2.12 (25.9x) $44.00 → $1.08 (40.7x) 0 / 0
BABILong (550 tasks, 64K–1M tok, 16K/12K) 67.1 vs. 92.0 (+24.9 pp) 70.7 vs. 94.0 (+23.3 pp) $360.50 → $11.44 (31.5x) $283.60 → $3.22 (88.0x) 0 / 0

Three patterns emerge from the long-input experiments:

  1. Multi-document reasoning improves when distracting haystack text stays out of attention. On LOONG (1,600 financial, legal, and academic multi-document questions), Claude Opus 4.5 jumps from a 70.1% average score (38.9% perfect-answer rate) under full-document ingestion to 80.2% (53.7% perfect) at 32K/24K and 79.3% (53.4% perfect) at 16K/12K, while nominal API spend drops from to and EORC drops to (13.6x lower). On AA-LCR, Opus 4.5 rises by +19.0 pp () at 16K/12K.
  2. Conversation length decouples from document size. Figures 7 and 8 of the paper plot total conversation tokens against document lengths spanning to tokens on LongBench v2 and LOONG. While the equal-length curve grows exponentially on a log-linear plot, selective-reading conversations stay within a flat band (mostly under 40K–100K total accumulated tokens across the entire episode), and the median number of compactions is zero on NIAH, RULER, Counting-Stars, BABILong, AA-LCR, and LOONG (at 32K/48K).
  3. Surgical grep + bounded doc-read replaces full-corpus prefill. In Appendix E.2, when translating a Kalamang sentence using a 1M-character (18,830-line) grammar book in LongBench v2, the agent runs grep -n -i -E '\bkademor\b' "$LONGBENCH_DOC" to find line 3840, then calls doc-read --lines 3835:3870 to inspect the 36-line grammatical rule and example sentence.

5. RQ3: Single-Action Compaction and Semantic State Space Models

Pushing compaction frequency to its logical limit, Section 5 asks whether an agent can solve repository-scale software engineering tasks when restricted to at most one environment action per context window (Single-Action Compaction, SAC). Under SAC, every turn receives only the system prompt, task description, and current natural-language semantic state, executes a single Bash command, observes the output, and emits an updated semantic state before the conversation is wiped.

Model & Benchmark mini Baseline (FHA) Accuracy (%) Best EoCC Accuracy (%) (Limit / Threshold) SAC Accuracy (%) mini Baseline Peak Context (Med / Max in K) Best EoCC Peak Context (Med / Max in K) SAC Peak Context (Med / Max in K)
Claude Opus 4.5
• SWE-bench Verified 70.0 74.8 (24K / 18K) 70.2 32 / 200 19 / 22 8 / 16
• Terminal-Bench 2.1 47.2 52.8 (32K / 24K) 51.8 27 / 130 24 / 31 9 / 19
GPT 5.6 Sol
• SWE-bench Verified 84.4 88.8 (48K / 36K) 82.4 18 / 173 22 / 35 7 / 15
• Terminal-Bench 2.1 87.6 88.8 (48K / 36K) 84.3 17 / 133 23 / 40 8 / 37

Under SAC, Claude Opus 4.5 matches its full-history SWE-bench Verified baseline ( vs. ) and beats its Terminal-Bench 2.1 baseline by +4.6 pp ( vs. ), while its median peak live context drops from 27K–32K down to 8K–9K tokens (and maximum context drops from 130K–200K down to 16K–19K tokens). GPT 5.6 Sol solves 82.4% of SWE-bench Verified (-2.0 pp) and 84.3% of Terminal-Bench 2.1 (75 of 89 tasks, -3.3 pp) with a median peak live context of 7K–8K tokens.

From Single-Action Compaction to Semantic State Space Models (SSSMs)

Because frontier models already sustain high accuracy under single-action compaction without dedicated post-training, Belcak et al. propose Semantic State Space Models (SSSMs). Where classical neural State-Space Models (such as S4 and Mamba) carry a continuous, black-box vector state between tokens, an SSSM maintains an explainable natural-language semantic state between agent turns:

  1. Comprehend the input natural-language state description and task specification.
  2. Reason about the immediate next action required to advance the task.
  3. Execute a single environment action and register its outcome .
  4. Update the natural-language state description so that an independent inference call can continue execution without raw conversational history.

Post-training language models directly as SSSMs removes the need to train attention heads over hundred-thousand-token sequences, keeps inference inside the 8K–16K peak throughput regime on both cloud H100 clusters and local unified-memory workstations, and grounds long-horizon computation in Turing's classic model of a finite state of mind reading a bounded workspace at each step.