Lineage Is Memory: Why Parent Inheritance Drives LLM-Guided Prompt and Code Optimization

Key Takeaway

Across 1,750 pre-registered runs on nine prompt and code optimization benchmarks (arXiv:2610.41157), greedy parent inheritance adds +0.238 held-out accuracy, closes the gap with full-set error clustering by round 4, and reaches 0.856 vs. GEPA's 0.664 on Banking77 at 43% of the token cost when instructed to copy revealed facts into the prompt.

Abstract and Core Thesis

Iterative large language model (LLM) optimizers refine prompts and programs by repeating a three-step loop: evaluate a candidate on a search set, show diagnostic failures to a proposer model, and retain candidates that improve validation performance. Recent work on reflective prompt optimization has focused almost entirely on feedback disclosure — arguing that small random minibatches provide too narrow a view of the error distribution, and proposing full-set error clustering, contrastive exemplars, or adaptive sample allocation to widen what the proposer sees in a single round.

In Lineage Is Memory: Parent Inheritance and Feedback Disclosure in Short-Budget LLM-Guided Optimization (arXiv:2610.41157), Shanglin Yang and Zhan Shi show that this diagnosis confounds single-round feedback disclosure () with multi-round parent inheritance (). Every recent reflection-scope method evaluates its wider feedback window exclusively inside a lineage-preserving search loop (), never testing whether wide disclosure can substitute for building on the best prior candidate, or whether parent inheritance renders single-round clustering redundant once search runs for a few rounds.

To separate these mechanisms, the study pre-registers and executes 1,750 optimization runs across nine prompt and Python code synthesis benchmarks, two model families (gpt-4.1-mini and claude-haiku-5-5), a larger proposer ablation (gpt-4.1), and five sequential pre-registered batches (A.1–A.5). Five findings stand out:

  1. Parent inheritance is the primary driver of short-budget optimization. Switching from memoryless seed resampling () to greedy parent inheritance () raises held-out accuracy by ( bootstrap CI ) across all seven primary tasks on gpt-4.1-mini ( on the five non-ceiling tasks) and () on claude-haiku-5-5.
  2. Lineage outweighs both full-set error clustering and a larger proposer. Showing just raw failures to a lineage-preserving optimizer (parent-first, ) beats giving a memoryless optimizer a target-preserving cluster summary of all search-set failures (seed-clustergold, ) by (). Upgrading the proposer and clusterer from gpt-4.1-mini to gpt-4.1 yields on parent-first, whereas the lineage gap under gpt-4.1 reaches .
  3. Lineage works through three coupled mechanisms that prevent "crowding out." Under seed-first, showing four narrow failures anchors rewrites on displayed formats and crowds out zero-feedback priors on unshown categories ( drops by on duration and on size). Under parent-first, child candidates retain 70%–95% of parent-solved cases while solved errors drop out of the failure set, shifting the feedback window across 46%–86% of the search set over rounds (versus 11%–25% under seed-first).
  4. Full-set clustering accelerates rounds 1–2, but parent-first catches up by round 4. When target rubrics are preserved (clustergold), clustering all failures helps seed () and gives parent-clustergold an early lead at round ( vs. , ). By round , parent-first reaches vs. and finishes tied at half the token cost, yielding an attenuation interaction of .
  5. Lineage acts as externalized memory when paired with explicit fact retention (CARRY_FACTS) — with a sharp boundary condition. On 12-intent banking77, where valid snake_case intent labels appear only inside error feedback, baseline parent-first stalls at (retaining intents) behind GEPA 0.0.7 (, intents). Appending one sentence instructing the proposer to copy revealed facts into the prompt (CARRY_FACTS) lifts parent-first + carry to ( intents), beating GEPA by at 43% of GEPA's median tokens ( vs. ). Meanwhile, seed-first + carry sees the same four initial errors every round, retains intents, and stalls at . Conversely, on bbh_ste — where all six option labels are already present in the seed prompt — CARRY_FACTS reduces accuracy by () by overfitting to training-example details.

1. Background and the Confound in Reflective Optimization

How Iterative LLM Optimizers Work

Discrete prompt and program optimization treats an instruction or source file as a mutable artifact improved through black-box evaluations on a search split . Across the literature, four families of algorithms have emerged:

Optimization Family Representative Methods How Candidates Are Updated
Trajectory / Meta-Prompt Resampling OPRO (Yang et al., 2023), Promptbreeder (Fernando et al., 2023) Prompt an LLM with a scored history of prior candidates to sample higher-scoring instructions.
Evolutionary Operators EvoPrompt (Guo et al., 2023), FunSearch (Romera-Paredes et al., 2023), AlphaEvolve (Novikov et al., 2025), ImprovEvolve (Kravatskiy et al., 2026) Maintain an island or population of candidates; apply LLM-driven mutation, crossover, and basin-hopping.
Textual Gradients & Auto-Differentiation ProTeGi (Pryzant et al., 2023), TextGrad (Yuksekgonul et al., 2024), LLM-AutoDiff (Yin & Wang, 2025), Momentum TextGrad (Ding et al., 2026) Compute natural-language critiques on failing examples ("textual gradients") and apply an editing step to the current prompt.
Reflective Evolution & Agent Harnesses GEPA (Agrawal et al., 2025), ESPO (Liu et al., 2026), RPT (Bayat et al., 2026), AdaGEPA (Chen et al., 2026), CASD (Singh et al., 2026) Reflect on minibatch or clustered execution traces, propose targeted edits, and gate acceptance via validation checks.

Why Recent Critique Focused on Single-Round Disclosure ()

Standard reflective optimizers such as GEPA (Agrawal et al., 2025) sample a tiny random minibatch of examples per round, reflect on failures, and accept a child candidate only if it improves on that minibatch (followed by periodic full-set pareto tracking). A wave of 2026 papers argued that narrow minibatches starve the proposer of global error structure:

  • ESPO (Liu et al., 2026) invoked a coupon-collector argument: if a prompt fails on distinct error categories, random minibatches of size require draws just to observe every category once, and local fixes for one minibatch can regress previously solved categories. ESPO therefore evaluates the full training split and uses an LLM to cluster all errors into 3–7 diagnostic failure patterns before proposing an edit.
  • Reflective Prompt Tuning (RPT) (Bayat et al., 2026) exposes the training error table to the proposer via tool calls (inspect_errors, cluster_failures).
  • AdaGEPA (Chen et al., 2026), Contrastive Reflection (Koh et al., 2026), and Coding Agents as Prompt Optimizers (CASD) (Singh et al., 2026) similarly widen or structure the diagnostic context fed to the proposer.

The Hidden Confound: Lineage () vs. Disclosure ()

Notice what all of these studies hold fixed: every single one runs inside a lineage-preserving loop (). When an optimizer edits the best candidate found so far () rather than re-editing the initial seed (), the candidate text itself carries forward rules accumulated in earlier rounds. Furthermore, when evaluation filters for residual failures , every example solved in round disappears from the failure list in round .

Consequently, a small fixed window of failures under is not static across rounds — it automatically sweeps across the residual error distribution as earlier failure modes are eliminated. Testing wide disclosure () only under makes it impossible to tell:

  1. How much of the optimization gain comes from parent inheritance () versus feedback disclosure ()?
  2. Does wide single-round disclosure () allow memoryless resampling () to match lineage-preserving search?
  3. Does multi-round lineage () attenuate the advantage of full-set error clustering once search runs beyond the first two rounds?

2. The Factorial Framework and Estimands

To disentangle these factors, Yang and Shi factorize an iterative LLM optimizer into three orthogonal design choices at each round :

1. Lineage Policy ()

Given a seed candidate and a search set with mean per-example score , the best-so-far candidate updates via strict inequality:

Strict inequality () is essential: if ties () were accepted, neutral edits could drift away from verified instructions. The parent candidate handed to the proposer at round is then determined by :

2. Feedback Disclosure Policy ()

Let denote the ordered list of failing search examples on parent . The factorial crosses with six disclosure rules:

Disclosure Policy What the Proposer Sees at Round Extra LLM Calls
none Zero-feedback baseline: only the task objective and parent candidate (score=None, feedback="(no feedback)"). 0
first The first failing examples from in fixed dataset index order: input -> got, expected. 0
diverse failing examples selected via greedy farthest-point sampling under normalized character Levenshtein distance on input strings (Pu et al., 2017). 0
minibatch examples drawn uniformly at random from with round seed , showing both PASS and FAIL items (matching GEPA's unfiltered minibatch view). 0
cluster All failing examples grouped by one clusterer LLM call into 3–7 failure patterns with count and one representative wrong answer (Batch A.1). 1 clusterer call / round
clustergold Target-preserving clustering: all failures grouped into 3–7 failure patterns, explicitly preserving the input, wrong output, and full expected answer/rubric (Batches A.2–A.4). 1 clusterer call / round

3. Evaluation Scheduling () and External Baseline (GEPA 0.0.7)

All factorial cells evaluate once on and run up to proposal rounds on prompt tasks (, maximum search evaluations) and rounds on Python code synthesis tasks (, maximum evaluations), stopping early as soon as . On the seven prompt tasks, the factorial is compared at matched max_metric_calls = 180 against external GEPA 0.0.7 (reflection_minibatch_size = 3) both without early stopping (GEPA (mb3)) and with an early-stop wrapper (GEPA (mb3 + stop)).

Pre-Registered Estimands

Every cell is run for independent seeds (), and all confidence intervals use task-stratified bootstrap resamples. Three formal estimands govern the analysis:

  1. Lineage Main Effect (): The average gain of parent inheritance over seed resampling across the four non-none disclosure rules :
  2. Lineage Attenuation of Disclosure Breadth (): For a wider disclosure policy , how much larger its benefit over first is under memoryless seed than under parent:
    A positive proves that multi-round lineage substitutes for single-round feedback breadth.
  3. Unshown-Category Coverage (): For tasks with partitioned format categories , let be the set of categories that ever appeared inside the feedback window during run . Coverage on held-out examples belonging to unshown categories tests whether narrow feedback crowds out zero-feedback model priors:

3. Finding 1: Parent Inheritance Outweighs Disclosure Breadth and Proposer Size

Across all five pre-registered batches (1,750 runs total), parent inheritance () is the dominant factor, dwarfing both feedback disclosure strategy () and proposer model scale.

Figure 1: Held-out accuracy across the seven primary prompt and code tasks for all 10 cells of the 2x5 lineage-by-disclosure factorial (gpt-4.1-mini, n=10 seeds per cell, 95% bootstrap CIs). Blue curves (parent) sit far above orange curves (seed) on all five feedback-dependent tasks.

Cross-Task Factorial Results (gpt-4.1-mini and claude-haiku-5-5)

Of the seven primary tasks, two (dates and scone) are zero-feedback ceiling tasks: even with F = none (zero examples shown), rewriting the seed prompt raises held-out accuracy to on dates and on scone because the task model already knows ISO-8601 date rules and negation entailment once prompted clearly. On the remaining five feedback-dependent tasks (labels, routing, replies, duration, size), feedback and lineage are essential:

Cell ( – ) labels dates routing replies scone duration size 7-Task Mean (4.1-mini) 5-Task Non-Ceiling (4.1-mini) 5-Task Non-Ceiling (Haiku 5.5)
seed – none 0.277 1.000 0.000 0.057 0.900 0.633 0.738 0.515 0.341 0.335
seed – first 0.410 1.000 0.500 0.290 0.900 0.562 0.767 0.633 0.506 0.504
seed – diverse 0.633 0.993 0.440 0.420 0.903 0.729 0.738 0.694 0.592 0.626
seed – minibatch 0.553 0.990 0.500 0.433 0.890 0.625 0.804 0.685 0.583 0.559
seed – clustergold 0.637 0.990 0.520 0.867 0.900 0.675 0.838 0.775 0.707 0.653
parent – none 0.280 1.000 0.000 0.053 0.897 0.633 0.738 0.514 0.341 0.332
parent – first 0.923 1.000 0.933 0.980 0.910 0.962 0.925 0.948 0.945 0.906
parent – diverse 0.947 1.000 0.857 0.907 0.913 0.971 0.988 0.940 0.934 0.907
parent – minibatch 0.883 0.993 0.750 0.917 0.893 0.862 0.900 0.886 0.862 0.839
parent – clustergold 0.893 0.990 0.793 0.893 0.913 0.979 0.938 0.914 0.899 0.888

Look closely at the contrast between parent-first and seed-clustergold:

  • Even when seed-clustergold inspects every failure in and summarizes them with full expected answers via an extra clusterer call per round, it reaches on the 5 non-ceiling tasks ( in the dedicated Batch A.2 replication).
  • Meanwhile, parent-first — which shows only the first failing examples with zero clustering calls — achieves ( in Batch A.2), outperforming seed-clustergold by () on gpt-4.1-mini and () on claude-haiku-5-5.

Why a Bigger Proposer (gpt-4.1) Does Not Replace Lineage

A natural objection is that gpt-4.1-mini and claude-haiku-5-5 might be too weak as proposers to synthesize a global prompt from clustered failures in a single shot. Batch A.4 tests this directly by upgrading both the proposer and clusterer to gpt-4.1 while holding the task model fixed at gpt-4.1-mini:

Policy ( – ) gpt-4.1-mini Proposer gpt-4.1 Proposer Proposer Size Effect
seed – first 0.506 0.525
seed – clustergold 0.789 0.720
parent – first 0.946 0.927
parent – clustergold 0.928 0.922

Upgrading the proposer to gpt-4.1 yields a slightly negative shift on both parent-first () and seed-clustergold (), while the lineage advantage under gpt-4.1 widens to (). Why? Without lineage, a stronger proposer still only sees a compressed summary or 4 failures per round and over-tailors its single-step rewrite to those examples. Bottlenecks in iterative prompt optimization are structural (lack of state accumulation across steps), not proposer reasoning capacity.


4. Finding 2: How Lineage Works as Memory (and Prevents Crowding Out)

Why does editing with failures work so well, while editing with failures stalls — or even degrades below the zero-feedback baseline? The paper traces this to three coupled mechanisms.

Mechanism A: Narrow Feedback Without Lineage Causes "Crowding Out" ()

Look at the Python code synthesis task duration in Figure 1:

  • Under seed-none (zero feedback shown), asking gpt-4.1-mini to improve parse_duration(text) from its task description alone raises held-out accuracy from (seed ) to , because the LLM draws on its pretrained knowledge of common duration formats (HH:MM:SS, 1h30m, ISO-8601 PT..., word units).
  • Under seed-first (showing the first failing examples), held-out accuracy drops to ( below seed-none!).

Measuring unshown-category coverage across the 8 format categories of duration and size explains why:

Code Synthesis Task seed-none seed-first Crowding-Out Drop () parent-first
duration (8 format categories) 0.483 0.429 0.875
size (8 format categories) 0.676 0.563 0.750

When seed-first displays four specific failures (e.g., whitespace-padded " 45 s " and "2.5h"), the proposer writes a narrow patch tailored to those four strings and omits general branches it would have written under seed-none. And because seed-first resets to every round, never changes — so the exact same 4 failures are shown in all 12 rounds, locking the optimizer into a narrow local patch.

Mechanism B: High Parent-to-Child Retention + Residual-Error Exposure Shift

Under , two dynamics break this lock:

  1. Text Inheritance Preserves Solved Cases: On every accepted step (), the child candidate inherits the parent's prompt/code text and retains 70% to 95% of examples already solved by the parent ( on duration, on size, on labels, on routing).
  2. Residual-Error Exposure Shift: Because solved examples disappear from , the deterministic first window () automatically advances to the next unsolved failure modes in . Over a run, parent-first exposes 46% to 86% of the entire search set (86.5% on routing, 77.5% on labels, 67.1% on size), compared to just 11% to 25% under seed-first.
  3. Multi-Step Edit Decomposition: Instead of forcing the proposer to synthesize a 12-rule system prompt or an 8-branch parser in one shot, parent-first lets the proposer add 1–2 rules per round while strict selection () rejects any edit that breaks previously solved categories.

5. Finding 3: Error Clustering Speeds Up Early Rounds, But Lineage Catches Up by Round 4

The Batch A.1 A.2 Lesson: Why Naive Error Clustering Fails (cluster vs. clustergold)

In Batch A.1, the authors noticed an unexpected failure in parent-cluster on the customer-support policy task replies: while parent-first scored , parent-cluster collapsed to (seed-cluster scored ). Auditing the 156 clusterer outputs revealed a critical failure mode of naive LLM error summarization:

  • In replies, each failing example has a multi-clause policy rubric as its expected target want (e.g., "Must apologize for damaged item, offer replacement or refund without requiring return shipping on orders under $50, and not promise a specific delivery date").
  • The Batch A.1 CLUSTER_PROMPT asked the clusterer for "what goes wrong, how many of the wrong answers it covers, and one representative wrong answer copied verbatim."
  • In 125 of 156 calls (80.1%), the clusterer summarized the candidate's wrong reply but dropped the expected policy rubric! Without the exact target rubric, the proposer guessed at Acme's support policies and regressed.

In Batch A.2, clustergold modified seven words of the clusterer prompt to require "one representative failure copied verbatim with its input, the wrong answer and the expected answer in full." That single change raised rubric retention on replies from 19.9% to 100.0%, lifting seed-clustergold from and parent-clustergold from .

Anytime Trajectory () and the Attenuation Interaction

With target-preserving clustergold in place, how do the six policies compare across rounds? Tracking held-out accuracy of the search-best candidate after proposal rounds across the five feedback-dependent tasks reveals a clean crossover:

Policy ( – ) Round Round Round Final () Median Total Tokens / Run
seed – first 0.471 0.491 0.500 0.506 35.1k
seed – diverse 0.531 0.562 0.580 0.592 35.4k
seed – clustergold 0.674 0.728 0.764 0.789 61.8k
parent – minibatch 0.543 0.662 0.785 0.862 38.9k
parent – clustergold 0.712 0.842 0.929 0.928 49.4k
parent – first 0.547 0.669 0.946 0.946 26.7k

Three takeaways emerge from this trajectory:

  1. At ultra-tiny budgets ( or rounds), parent-clustergold leads parent-first by to ( vs. at ) because clustering exposes all failure families immediately in round 1.
  2. By round , parent-first completely closes the gap and overtakes parent-clustergold ( vs. ), finishing at vs. while using roughly half the total tokens ( vs. ) because it pays zero clusterer LLM calls and triggers early stopping more often.
  3. Lineage strongly attenuates the value of wide single-round disclosure: The pre-registered interaction is () on gpt-4.1-mini and () on claude-haiku-5-5 ( and ). Furthermore, parent-minibatch trails parent-first by () because random minibatches waste feedback slots on passing examples (PASS), starving the proposer of failure signal as accuracy climbs above 70%.

6. Finding 4: Fact Retention in the Prompt (CARRY_FACTS) and Its Boundary Condition

Why Baseline parent-first Trailed GEPA on banking77

To test out-of-distribution generalization on standard public benchmarks, Batch A.5 evaluated 12-intent banking77 (Casanueva et al., 2020) and BIG-Bench Hard Salient Translation Error Detection (bbh_ste) (Suzgun et al., 2022). On banking77, the seed prompt is simply "Classify the banking customer query." with output format "Reply with the intent label only." Crucially, none of the 12 exact snake_case target labels (such as beneficiary_not_allowed, declined_cash_withdrawal, pending_top_up, or visa_or_mastercard) appear in the seed prompt — they are only revealed one by one when an example fails (got 'wire_transfer_error', expected 'beneficiary_not_allowed').

On this task, baseline parent-first scored only , whereas external GEPA 0.0.7 (mb3) scored . Why did GEPA beat baseline parent-first here despite losing or tying on the primary tasks?

Inspecting the prompts evolved by both methods revealed the exact mechanism:

  • GEPA's built-in reflection prompt explicitly instructs the proposer: "Include all task-specific factual information revealed in the execution trace (such as domain labels, formatting rules, or edge cases) directly in the new instruction." Across 10 runs, GEPA's final prompts explicitly listed 9.0 of the 12 target intent names.
  • Baseline parent-first used a minimal 4-line meta-prompt ("Improve the prompt so it fixes the failures without breaking what already works."). Without an instruction to copy target label names into the prompt, the proposer often wrote generic semantic advice ("Use concise snake_case banking terms") and listed only 5.6 of the 12 intents on average.

The One-Sentence CARRY_FACTS Intervention ()

To test whether explicit instruction-level fact retention accounts for the entire gap, Batch A.5 appended a single pre-registered sentence (CARRY_FACTS) to the proposer objective:

"The task model sees only this prompt, never the feedback: copy every task-specific fact the feedback reveals (for example the exact expected labels or formats) into the prompt."

Figure 2: Banking77 held-out accuracy (left) and number of ground-truth snake_case intent labels retained in the final prompt out of 12 (right) across 5 conditions (n=10 seeds, 95% bootstrap CIs). Adding CARRY_FACTS lifts parent-first from 0.372 to 0.856 (11.7/12 intents), whereas seed-first + carry stalls at 0.317 (4.0/12 intents).

The results on banking77 and bbh_ste (Table 5 of the paper) demonstrate both the power of pairing lineage with fact retention and its exact boundary condition:

Optimizer Configuration banking77 Held-Out Acc banking77 Intents Retained (of 12) banking77 Median Tokens bbh_ste Held-Out Acc bbh_ste Median Tokens
seed-first (baseline) 0.225 3.2 / 12 25.0k 0.620 64.9k
parent-first (baseline) 0.372 5.6 / 12 31.4k 0.653 71.1k
GEPA 0.0.7 (mb3) 0.664 9.0 / 12 106.9k 0.633 127.8k
seed-first + carry 0.317 4.0 / 12 30.8k 0.587 72.4k
parent-first + carry 0.856 11.7 / 12 45.8k 0.592 82.5k

Look at why seed-first + carry fails while parent-first + carry dominates on banking77:

  • In round 1, the seed prompt fails on all 20 search examples (0.000). The first failing examples happen to belong to 4 distinct intents.
  • Under seed-first + carry, the proposer dutifully copies those 4 intent names into — scoring intents (). Next round, because resets the parent to , fails on the exact same examples, first displays the exact same 4 failures, and the optimizer is trapped at 4.0/12 intents in every single run ( CI )!
  • Under parent-first + carry, round 1's candidate solves the first 4 intents and becomes . In round 2, examples from those 4 intents pass and drop out of , so first reveals 3–4 new intents. By round 4–5, the prompt has accumulated 11.7 of the 12 intents and reaches held-out accuracy — beating parent-first by (), beating seed-first + carry by (), and beating GEPA 0.0.7 by () at 43% of GEPA's token budget ( vs. ).

This is the literal meaning of "Lineage Is Memory": the evolving prompt text serves as an externalized, append-and-refine memory buffer across rounds, while residual-error filtering acts as a read-pointer that advances to unmemorized facts each step.

The Boundary Condition (): When CARRY_FACTS Hurts (bbh_ste)

Why did CARRY_FACTS lower parent-first accuracy on bbh_ste from to (, CI )?

  • In bbh_ste, the seed prompt already lists all six multiple-choice options (A)–(F) and their definitions. There are no hidden label names to discover from feedback.
  • When instructed by CARRY_FACTS to "copy every task-specific fact the feedback reveals into the prompt," the proposer copies instance-specific German-to-English phrases and entities from the 20 search examples into the system instructions, which overfits the search split and hurts generalization on the 30 held-out translations.
  • Design rule: Enable CARRY_FACTS when target labels, schemas, or policy rules are absent or underspecified in and must be discovered from feedback; disable it when already enumerates the complete label space and task rules.

7. Finding 5: Where GEPA's Tokens Actually Go ()

Across the five primary prompt tasks, parent-first matches or beats GEPA 0.0.7 (mb3) on held-out accuracy ( vs. mean across the 5 prompt tasks) while consuming to fewer tokens. Where do GEPA's tokens go? Decomposing every GEPA API call across the 7 prompt tasks into three buckets — (1) Proposer Reflection, (2) Full Search-Set Validation, and (3) Minibatch Gating Evaluations — reveals a striking imbalance:

Prompt Task parent-first Median Tokens GEPA (mb3) Median Tokens GEPA Reflection Share GEPA Full-Set Validation Share GEPA Minibatch Gating Share
labels 24.7k 54.4k () 9.8% 60.5% 29.7%
dates 5.2k 17.8k () 2.1% 21.3% 76.6%
routing 29.4k 68.2k () 11.4% 56.8% 31.8%
replies 64.1k 189.3k () 4.6% 73.9% 21.5%
scone 14.8k 41.2k () 10.9% 51.2% 37.9%
banking77 45.8k (+carry) 106.9k () 12.6% 58.1% 29.3%
bbh_ste 71.1k 127.8k () 14.8% 66.8% 18.4%

Proposer reflection accounts for only 2% to 15% of GEPA's total token spend. The remaining 85% to 98% is spent on task-model evaluations (minibatch gating checks that reject proposals, plus full-set validations whenever a noisy 3-example minibatch passes). Because a 3-example random minibatch has high variance, many proposals that tie or pass on 3 examples fail to improve the full 20-example search set, triggering expensive full-set validations without advancing .

In contrast, parent-first evaluates the 20-example search set directly, conditions only on confirmed failures so every proposal targets real errors, and stops immediately once . This aligns closely with concurrent findings by Zhou & Cheng (2026) (Explore More, Validate Less) and Oved et al. (2026) (Evolution or Illusion?).


Understanding Lineage Is Memory also clarifies how recent threads in prompt optimization, program synthesis, and agent harness engineering connect:

  1. Coupon-Collector Bounds vs. State-Conditional Exposure Shift: ESPO (Liu et al., 2026) modeled error discovery as sampling with replacement from a static failure distribution, where seeing all error types in one round requires large or clustered feedback. Lineage Is Memory shows that under and strict improvement , failure sampling is state-conditional and without replacement on solved categories: once category is fixed in , its examples drop out of , shifting the deterministic first window to unsolved categories in round .
  2. Representative Example Selection in Program Synthesis: In classic programming-by-example (Pu et al., 2017), selecting diverse input shapes helps synthesize a program in a single call. The factorial confirms that input-diverse selection (diverse) helps one-step/memoryless seed ( on gpt-4.1-mini, on Haiku), just as clustergold helps seed ( to ). Yet once multi-step lineage () is enabled, residual-error filtering provides diversity across time rather than within a single prompt.
  3. Prompt Overfitting and Goodhart's Law on Small Search Splits: Lumer et al. (2026) (Reflective Prompt Optimization of a Deep-Research Agent) and Yang (2026) (Ideation Goodhart) document how aggressive iterative optimization can overfit small search sets when the optimizer memorizes training examples rather than transferable rules. Batch A.5's bbh_ste result ( under CARRY_FACTS) provides a controlled demonstration of this boundary: copying revealed facts is beneficial when the task specification is missing label definitions (banking77, ), and harmful when the specification is already complete (bbh_ste, ).
  4. Connection to Agent Harness Engineering (Harness²): Just as Harness² (arXiv:2610.14224) improves an agent by proposing, probing, and composing verified changes to its surrounding system harness, parent-first + carry treats the system prompt as a verified, monotonically improving state artifact () that accumulates domain rules across rounds.

9. Practical Engineering Guidelines and AdalFlow Implementation

The parent-first + carry algorithm from this paper ships as the default prompt optimizer in AdalFlow (adalflow.optim.evolution.optimize_prompt). Based on the 1,750-run empirical audit, four engineering rules apply when building or tuning an LLM-guided optimizer:

Rule 1: Default to Greedy Lineage + Deterministic First- Failures (parent-first)

Do not spend extra LLM calls clustering errors or sampling random pass/fail minibatches on budgets of rounds. Evaluate on a fixed 20–25 example search split, keep only strictly improving candidates (), and show the first failing examples (input -> got, expected).

Rule 2: Enable carry_facts=True When Labels or Formats Live in the Data

Whenever your seed prompt is a brief objective and the exact label vocabulary, output schema, or policy edge cases are only visible inside ground-truth answers or evaluator rubrics, enable carry_facts=True (the default in AdalFlow). Turn carry_facts=False if your seed prompt already contains the full closed-book specification and option definitions.

Rule 3: Use parent-clustergold Only for Ultra-Short Budgets () — and Never Drop Target Rubrics

If latency or evaluation cost restricts you to or proposal rounds, clustering all failures speeds up early convergence ( vs. at ). If you use an LLM clusterer, always copy the full expected answer or grading rubric verbatim (CLUSTER_GOLD_PROMPT); omitting expected rubrics caused an 80% rubric loss and a accuracy collapse on replies.

Rule 4: Minimal Usage in Python (adalflow & lineage-is-memory)

from adalflow.optim.evolution import optimize_prompt

# Default configuration runs parent-first with carry_facts=True (k=4 failures, strict > acceptance)
result = optimize_prompt(
    objective=(
        "A system prompt that makes the model classify an online-banking "
        "customer query into its intent. Intent labels are lower-case snake_case names."
    ),
    seed="Classify the banking customer query.",
    output_format="Reply with the intent label only.",
    search_data=banking77_search_split,   # 20 (input, expected_label) pairs
    rounds=8,
    k_failures=4,
    carry_facts=True,                     # Appends CARRY_FACTS; set False if seed already lists all labels
)

print("Best search score:", result.best_score)
print("Optimized system prompt:\n", result.best_prompt)

To reproduce all five pre-registered batches (A.1–A.5), bootstrap confidence intervals, and figures directly from the open-source repository:

git clone https://github.com/shangliy/lineage-is-memory.git
cd lineage-is-memory

# Run the 2x5 factorial on the 7 primary tasks (Batch A.1 / A.2)
uv run python -m benchmarks.evolution.factorial --rounds 8 --code-rounds 12 --seeds 10

# Run the Banking77 & BBH-STE CARRY_FACTS experiments (Batch A.5)
uv run python -m benchmarks.evolution.carry_facts --tasks banking77 bbh_ste --seeds 10

References

  1. Yang, S., & Shi, Z. (2026). Lineage Is Memory: Parent Inheritance and Feedback Disclosure in Short-Budget LLM-Guided Optimization. arXiv:2610.41157. Code & pre-registered raw data: github.com/shangliy/lineage-is-memory.
  2. Agrawal, L. A., Tan, S., Soylu, D., et al. (2025). GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457.
  3. Liu, L.-H., Tang, P., Singh, K. Y., & Ghadar, S. (2026). ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and Stabilize. arXiv preprint, 2026.
  4. Bayat, F. F., Aminnaseri, M., Pezeshkpour, P., & Hruschka, E. (2026). Reflective Prompt Tuning through Language Model Function-Calling. arXiv:2605.21781.
  5. Chen, J., Wang, Z., & Chen, J. (2026). AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization. arXiv:2609.39927.
  6. Koh, D., Mo, J., Le, B. H., et al. (2026). Contrastive Reflection for Iterative Prompt Optimization. arXiv:2606.30840.
  7. Singh, A., Gautam, S., Gupta, P., et al. (2026). Coding Agents Are Strong Prompt Optimizers. arXiv:2609.26261.
  8. Zhou, X., & Cheng, L. (2026). Explore More, Validate Less: Efficient Reflective Prompt Optimization. arXiv:2610.27367.
  9. Oved, T., Pony, R., Naparstek, O., & Barzelay, U. (2026). Evolution or Illusion? Rethinking Evaluation in LLM Evolutionary Search. arXiv:2609.19799.
  10. Yuksekgonul, M., Bianchi, F., Boen, J., et al. (2024). TextGrad: Automatic "Differentiation" via Text. arXiv:2406.07496.
  11. Yang, C., Wang, X., Lu, Y., et al. (2023). Large Language Models as Optimizers (OPRO). arXiv:2309.03409.
  12. Guo, Q., Wang, R., Guo, J., et al. (2023). EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers. ICLR 2024.
  13. Romera-Paredes, B., Barekatain, M., Novikov, A., et al. (2023). Mathematical Discoveries from Program Search with Large Language Models (FunSearch). Nature, 625, 468–475.
  14. Novikov, A., Vũ, N., Eisenberger, M., et al. (2025). AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. arXiv:2506.13131.
  15. Lumer, E., Sen, S., Subbiah, V. K., & Paul, K. (2026). Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting. arXiv:2610.12601.
  16. Yang, S. (2026). What Should You Optimize When You Optimize an LLM Research-Ideation Harness? (Ideation Goodhart). github.com/shangliy/ideation-goodhart.