Search Agents Need Better Decision Boundaries, Not Just Bigger Models

A search agent can spend substantial computation deciding whether to keep searching. Passage relevance, evidence sufficiency, and answer support are small outputs with large consequences: a premature stop can miss the decisive source, while an unnecessary search can add cost without improving the answer.

SearchJev asks whether these recurring decisions need the same generative model used for planning and writing. Its answer is a specialized, schema-conditioned decision model, with uncertain cases delegated to a larger model. The interesting question is how that delegation behaves when confidence stops being reliable.

1. Score the decision instead of generating its serialization

SearchJev receives a search state, a decision question, and a finite set of legal options. Each option is associated with a distinct single-token label. It reads the causal LM’s vocabulary logits at the first output position and normalizes only the logits of those labels. [1]

For legal options V_f, the readout is:

<code><code>p_f(v)=\frac{\exp(\ell_f(v))}{\sum_{u\in V_f}\exp(\ell_f(u))}.</code></code>

An option description can be long; the label assigned to it must be one token. The controller maps the selected label back to a structured decision. This avoids generating a JSON object token by token and then parsing it.

That does not make the input free to process. The backbone still processes the supplied search state. The saving targets sequential output generation and decision serialization, not all the computation in a search pipeline. Open-ended query writing remains a different operation.

In the dual-system agent, System 2 retains planning, query generation, and final-answer composition. SearchJev handles bounded judgments, and a search controller converts resolved decisions into actions. [1]

This is a useful design boundary: use a constrained predictor when the output space is enumerable, and keep generation where the task genuinely requires it. If the legal schema excludes the action the agent needs, confident selection among the remaining options will not repair the schema.

2. Why an option probability is not automatically confidence

Soft-Label Learning for Calibrated Decisions, or SLCD, learns from target distributions rather than forcing every example into a certain one-hot label. Its weighted objective combines cross-entropy with a Brier term, followed by held-out temperature fitting. [1]

Writing the target distribution as t_f, the objective is:

<code><code>L_f=w_f\left[-\sum_v t_f(v)\log p_f(v)
+\lambda_B\sum_v\big(p_f(v)-t_f(v)\big)^2\right].</code></code>

The implementation permutes unordered options to reduce position bias, downweights selected weaker supervision sources, and fits temperatures by output type. Positive temperature scaling preserves option order while changing probability sharpness. [1]

The connection to Guo et al.’s calibration study is direct: accurate classifiers can be poorly calibrated, and temperature scaling can improve probability estimates without changing their predicted class. [2] SearchJev adds decision-specific training and typed calibration; its contribution should not be reduced to applying one temperature to an ordinary classifier.

There is a further limitation in the loss itself. A proper scoring rule encourages agreement with the target distribution. If those targets encode an imperfect teacher’s judgment, learning them faithfully does not establish that the resulting probabilities describe real-world correctness. Calibration against labels and calibration against the user’s eventual task outcome are separate questions.

3. Delegation is a risk–coverage problem

SearchJev accepts a local decision when its maximum calibrated option probability exceeds a threshold \delta; otherwise, System 2 resolves the same bounded question. [1]

This connects to selective classification: a predictor can trade coverage for lower risk by rejecting cases it should not handle. [3] In an agent, rejection becomes escalation rather than simply withholding a classification.

A practical analysis would plot:

• Coverage: the fraction of decisions accepted from the small model.

• Selective risk: error among those accepted decisions.

• Escalation cost: extra calls, latency, and tokens paid for rejected decisions.

• Task quality: whether the final answer is correct and supported.

These are suggested deployment measurements, not guarantees supplied by SearchJev. Selective-classification results do not transfer automatically to a changing agent environment.

The threshold also need not be the same for every decision. A false relevance judgment might add a distracting passage; a false sufficiency judgment might end the search before essential evidence is found. Equal confidence can have unequal downstream costs. Comparing per-type thresholds with a single global threshold would test whether that asymmetry matters.

4. What the experiments establish

The paper evaluates 0.8B and 4B models against same-size Qwen3.5 autoregressive JSON baselines. It reports roughly 5.2–5.3 times faster local decisions and mean in-distribution ECE reductions of 74.4% and 40.7%, respectively. [1]

For the agent experiment, the authors sample 100 BrowseComp-Plus questions, run each agent once per question, use Qwen3.5-27B as System 2, and retrieve with BM25. Table 4 reports: [1]

Agent · Answer accuracy · Median active time, seconds · Output tokens, thousands

System 2 only · 45% · 4,080 · 73.7

SearchJev-0.8B + System 2 · 46% · 868 · 15.5

SearchJev-4B + System 2 · 54% · 1,102 · 22.1

The 4B configuration offers a different quality–cost point from the 0.8B configuration. It is more accurate in this sample but slower and more token-intensive. These are median active-search measurements under the paper’s setup, not a universal latency reduction or a p99 serving guarantee. One run on 100 questions also does not establish stability across repeated trajectories.

The harder result is the transfer test. On held-out datasets, decision performance transfers more consistently than calibration. Verification accuracy declines for the 4B model relative to its same-size autoregressive baseline, and calibration gains vary by decision family. [1]

A system can retain useful rankings of options while becoming unreliable about how certain it is. That distinction matters precisely because the controller uses confidence to decide when to escalate.

5. How this differs from training a search policy

ReAct interleaves reasoning with actions that obtain information or interact with an environment. [4] SearchJev changes the division of labor inside such a loop rather than eliminating the loop.

Search-R1 uses reinforcement learning to optimize search-using reasoning trajectories, including query generation and the use of retrieved information. [5] SearchJev’s local probability-learning objective is not that trajectory-level RL objective. A trained search policy and a calibrated decision module could be complementary, but their integration would require its own evaluation.

The cost motivation is real, but comparisons must stay scoped. A separate agentic retrieval study reports 107.4 seconds per query versus 0.67 seconds for standard retrieval, with substantial token consumption. [6] That is a different pipeline and benchmark. It motivates studying repeated decision overhead; it does not establish that SearchJev would remove those costs in that system.

6. A deployment test that would change my confidence

I would freeze the retriever, corpus snapshot, System-2 model, query prompts, and evidence budget, then compare a full-generative controller with SearchJev controllers across several thresholds. The experiment should charge input processing, escalation, retrieval, retries, and final synthesis.

The diagnostic interventions would include incomplete evidence, contradictory passages, option-order permutations, and changes in language or source domain. These are proposed tests. Their purpose is to distinguish useful uncertainty from confidence that merely tracks familiar formatting.

The key outcome would be a quality–cost frontier with uncertainty intervals, broken down by decision type. If the local model remains fast but accepts premature-stop decisions with high confidence, its calibration average can hide the most expensive failure. If it instead escalates those cases while handling routine judgments cheaply, the architectural separation has earned its place.