v1.1: promote iter-750 as canonical (best of 4 checkpoints by tier+foundation), add learning curve, fix draw-1 wording, beef up Usage with concrete example, plain-text typography pass

README.md

@@ -23,15 +23,20 @@ A LoRA adapter that distils a 31B Gemma Klondike Solitaire advisor into the
 ~2B-effective **Gemma 3n E2B** text-only model, runnable locally on a 16 GB
 Apple Silicon Mac via MLX.
 
-This is the **first distillation run** (1,000 LoRA iters on 1,279 training
-examples) and produced a **2× reduction in tier-score gap to the teacher**
-(−1.32 → −0.67 on a 20-state evaluation bench), eliminated illegal-move
-generation (1/20 → 0/20), and recovered 2 of 5 teacher-foundation moves the
-untuned base model missed — including a triple-replicated failure state that
-had defeated three prior small-model experiments.
+This is the **first distillation run**. The shipped weights
+(`adapters.safetensors`) are the **iter-750 checkpoint**, the best of a
+1,000-iter training based on intermediate-checkpoint evaluation. It nearly
+closes the tier-score gap to the teacher (-1.32 -> **-0.27**, ~80 %
+recovery on the 20-state eval bench) and recovers **6 of 7 teacher-foundation
+moves** the untuned base model missed, including a triple-replicated failure
+state that had defeated three prior small-model experiments. The iter-1000
+checkpoint (under `checkpoints/`) is also available but is mildly overfit:
+mean tier 2.75 vs iter-750's 3.15, with 2 fewer foundation recoveries. See
+the [learning-curve section](#learning-curve--why-iter-750-not-iter-1000) for
+details.
 
 > **Why Gemma 3n and not Gemma 4 E2B?**
-> The intended student was **Gemma 4 E2B** — the same series as the teacher
+> The intended student was **Gemma 4 E2B**, the same series as the teacher
 > and the project's long-term target. As of `mlx-lm` 0.31.3 (the latest at
 > this writing), all mlx-community Gemma 4 E2B quants fail to load with a
 > 140-parameter architecture-mismatch error (layers 15-34's
@@ -39,8 +44,8 @@ had defeated three prior small-model experiments.
 > implemented in mlx-lm's `Gemma4Model` class). The student here is therefore
 > the previous-generation **`gemma-3n-E2B`**, which `mlx-lm` fully supports.
 > A **Gemma 4 E2B variant of this adapter will be published** once `mlx-lm`
-> ships the missing architecture support — same training script, same data,
-> same eval bench, same methodology — at a separate repo
+> ships the missing architecture support, same training script, same data,
+> same eval bench, same methodology, at a separate repo
 > (`chayuto/gemma-4-e2b-it-solitaire-advisor-lora`).
 
 The teacher is `gemma-4-31b-it` (Google's Gemma 4 31B, accessed through a
@@ -58,7 +63,7 @@ separate harvester app).
 | Training framework | [`mlx-lm`](https://github.com/ml-explore/mlx-lm) 0.31.3 |
 | Hardware | Apple M5 16 GB unified memory (Metal GPU) |
 | Adapter size on disk | 45 MB per checkpoint |
-| Iterations trained | 1,000 |
+| Iterations trained | 1,000 (shipped checkpoint = iter 750) |
 | Wall-clock training time | ~95 minutes |
 | Quantisation | base remains 4-bit; LoRA weights bfloat16 |
 
@@ -66,71 +71,147 @@ separate harvester app).
 
 **In scope.** Acting as a move-selection advisor inside a Klondike Solitaire
 client that already enforces game rules:
-- Imperfect-information Klondike (draw-3 variant); the advisor is shown the
-  full visible state plus the count of face-down cards
+- Imperfect-information **draw-1** Klondike (one card flipped from stock per
+ draw); the advisor is shown the full visible state plus the count of
+ face-down cards
 - Single-turn decisions: given the prompt schema below, emit a single JSON
-  object choosing one of the offered legal moves
+ object choosing one of the offered legal moves
 - Local inference on Apple Silicon (8 GB+ unified memory) via `mlx-lm`
 
 **Out of scope.**
 - Open-ended chat or general-purpose text generation. The model has been
-  fine-tuned to a narrow JSON-emitting role and is expected to be measurably
-  worse than the base model at unrelated tasks.
+ fine-tuned to a narrow JSON-emitting role and is expected to be measurably
+ worse than the base model at unrelated tasks.
 - Game-rule enforcement. The advisor selects from a `legalMoves` array
-  supplied in the prompt; it does not verify legality from first principles
-  and should not be trusted to do so.
+ supplied in the prompt; it does not verify legality from first principles
+ and should not be trusted to do so.
 - Optimal Solitaire play. The distillation target is a 31B model that itself
-  is imperfect (it stalls on ~21 % of post-cutover games observed in
-  production). This adapter inherits that ceiling.
-- Other Solitaire variants (Spider, FreeCell, etc.) — out-of-distribution.
+ is imperfect (it stalls on ~21 % of post-cutover games observed in
+ production). This adapter inherits that ceiling.
+- Other Solitaire variants (Spider, FreeCell, etc.), out-of-distribution.
 
 ## Usage
 
-### Inference with `mlx-lm`
+### Install
+
+```bash
+# Apple Silicon, Python 3.12 venv recommended (mlx wheels are not on 3.14+)
+python3.12 -m venv venv && source venv/bin/activate
+pip install mlx mlx-lm huggingface-hub
+```
+
+### Quick start, Python
 
 ```python
+from huggingface_hub import snapshot_download
 from mlx_lm import load, generate
 
+# Pull the adapter once (~45 MB for the shipped iter-750 weights;
+# checkpoints/ subdir adds another 180 MB if you want intermediate iters too).
+adapter_path = snapshot_download(
+    repo_id="chayuto/gemma-3n-e2b-it-solitaire-advisor-lora",
+    allow_patterns=["adapters.safetensors", "adapter_config.json"],
+)
+
+# Load base + LoRA. First call also downloads the base model (~3 GB).
 model, tokenizer = load(
     "mlx-community/gemma-3n-E2B-it-text-4bit-dwq",
-    adapter_path="chayuto/gemma-3n-e2b-it-solitaire-advisor-lora",
+    adapter_path=adapter_path,
 )
 
-prompt = open("your_solitaire_prompt.txt").read()
-messages = [{"role": "user", "content": prompt}]
+# Wrap your Solitaire prompt as a single user message and apply the chat
+# template (this matches what the model was trained against).
+solitaire_prompt = open("your_solitaire_prompt.txt").read()
 wrapped = tokenizer.apply_chat_template(
-    messages, tokenize=False, add_generation_prompt=True
+    [{"role": "user", "content": solitaire_prompt}],
+    tokenize=False, add_generation_prompt=True,
 )
 response = generate(model, tokenizer, prompt=wrapped, max_tokens=512)
 print(response)
 ```
 
-Expect ~13 s per call on M5 16 GB, ~6.3 GB peak Metal memory.
+**Resource envelope** on Apple M5 16 GB:
+- First load: 2.3 s (cached) + ~5 min one-time base-model download
+- Per call: ~13 s wall, ~6.3 GB peak Metal memory
+- The model is text-only and runs entirely on the local Metal GPU; no
+    network after initial download.
+
+### Concrete example
+
+A real prompt from the eval bench is at
+[`eval/sample_prompt.txt`](eval/sample_prompt.txt) (state `oscillation-bfb84a`,
+4471 chars, 4 cards on foundations, 18 face-down, Ace of Spades in waste).
+Its tail looks like this:
+
+```
+LEGAL MOVES (respond with the index of your chosen move):
+ [0] tableau_to_tableau Move KH plus 1 more from column 1 to column 2 (empty)
+ [1] discard_to_foundation Send AS from the waste to the spades foundation
+ [2] draw_card Draw the next card from the stock onto the waste
+
+PROGRESS: foundation=4/52, face-down remaining=18, completion=8%
+...
+Now choose the best move and reply with only the JSON object.
+```
+
+The iter-750 adapter responds with:
 
-### Expected I/O shape
+```json
+{
+    "board_analysis": "The board is mostly face-down with 18 hidden cards remaining. Foundations are partially built (AH, AD, 2C, S). The waste pile contains the AS. The only legal tableau move is moving the KH and QC from column 1 to column 2, which does not reveal any hidden cards or create any new strategic opportunities.",
+    "strategic_plan": "The primary objective is to reveal the Ace of Spades and other low cards (2, 3, 5, 6, 8, 9) to the foundations and tableau columns. ...",
+    "final_decision": {
+    "move_index": 1,
+    "confidence": 0.95,
+    "alternative_move_index": -1
+    }
+}
+```
+
+`move_index: 1` correctly sends the Ace of Spades to the spades foundation
+- the optimal play here, and a move the untuned base missed on 3/3 runs.
+This is the headline `oscillation-bfb84a` recovery referenced in §Foundation
+move recovery below.
+
+### Expected I/O contract
 
 **Input**: a single user-role message containing the full Solitaire game prompt
-(rules preamble, current game state, legal-moves block, prior reasoning trail).
-A complete example lives at `eval/sample_prompt.txt` in this repo.
+in the project's harvester format. The prompt must include `LEGAL MOVES`
+block; the model is trained to index into it. See
+[`eval/prompts_C0/*/prompt.txt`](eval/prompts_C0/) for 20 worked examples
+across early/midgame/oscillation game states.
 
-**Output**: a JSON object with three required keys:
+**Output**: a single JSON object with three required keys:
 
 ```json
 {
-  "board_analysis": "string — terse description of the visible state",
-  "strategic_plan": "string — why the chosen move is preferred",
-  "final_decision": {
+    "board_analysis": "string, terse description of the visible state",
+    "strategic_plan": "string, why the chosen move is preferred",
+    "final_decision": {
     "move_index": 0,
     "confidence": 0.9,
     "alternative_move_index": 1
-  }
+    }
 }
 ```
 
-`move_index` is a 0-based index into the `legalMoves` array from the prompt.
-The client is expected to apply the chosen move; if `move_index` is illegal
-the client should fall back (the adapter eliminated illegal moves on the
-20-state eval but generalisation is not guaranteed).
+- `move_index` is the **0-based index into the prompt's `legalMoves` array**.
+    This is the only field the client needs to consume.
+- `confidence` is inherited from the teacher and is poorly calibrated
+    (saturates at 0.85-0.95). Do **not** route on it.
+- `alternative_move_index` can be `-1` if no alternative is suggested.
+- The strategic_plan prose sometimes references move indices inconsistently
+    (an artifact of the `PRIOR REASONING` section in the training prompts);
+    trust `final_decision.move_index`, not the narrative.
+
+### Robustness
+
+Iter-750 produces valid JSON on 20/20 eval states. **Two of 20** choose
+an illegal `move_index` (off-by-one on 2-move arrays). Clients should
+defensively fall back to the highest-tier legal move when
+`move_index >= len(legalMoves)`. The iter-1000 checkpoint under
+`checkpoints/0001000_adapters.safetensors` eliminates both illegals at the
+cost of 2 foundation moves, see the learning-curve section to choose.
 
 ## Training data
 
@@ -145,7 +226,7 @@ ingest pipeline filters (success outcome, valid `rawResponse` JSON with the
 three required keys, not from a stalled game). 25 distinct play sessions,
 heterogeneous prompt templates (~63 % pre-cutover legacy format, ~37 % the
 current production template `0462323c…`). Split at the session level
-80 / 10 / 10 → 1,279 train / 126 val / 131 test.
+80 / 10 / 10 -> 1,279 train / 126 val / 131 test.
 
 **License.** Released as a derived training corpus under **CC-BY-4.0** in
 the project's published dataset (separately staged). No personally
@@ -154,14 +235,14 @@ responses, and timing.
 
 **Known data-quality issues** the adapter inherits:
 - 11 % of source rows dropped by the ingest filter for malformed
-  `rawResponse` — root cause not yet localised in the harvester.
+    `rawResponse`, root cause not yet localised in the harvester.
 - Teacher `confidence` field is saturated (median 0.90, never below 0.80)
-  even in lost games — not a reliable training signal; it is included in
-  completions but treated as suspect downstream.
+    even in lost games, not a reliable training signal; it is included in
+    completions but treated as suspect downstream.
 - Mixed prompt-template formats in training; eval is on the most-recent
-  template only.
+    template only.
 - No deck seed in logs (open harvester P0), so we cannot verify the
-  teacher's choices against solver-optimal play.
+    teacher's choices against solver-optimal play.
 
 ## Training procedure
 
@@ -181,10 +262,10 @@ save_every: 250
 val_batches: 25
 
 lora_parameters:
-  rank: 16
-  scale: 2.0
-  dropout: 0.05
-  keys:
+    rank: 16
+    scale: 2.0
+    dropout: 0.05
+    keys:
     - self_attn.q_proj
     - self_attn.k_proj
     - self_attn.v_proj
@@ -202,28 +283,65 @@ breaks `altup.predict()`'s direct `.weight` access mid-training.
 
 | iter | train loss | val loss | peak MLX | wall (cumulative) |
 |---:|---:|---:|---:|---:|
-| 1 | — | 6.365 | — | 0 m |
-| 10 | 3.160 | — | 11.45 GB | ~1 m |
+| 1 | - | 6.365 | - | 0 m |
+| 10 | 3.160 | - | 11.45 GB | ~1 m |
 | 100 | 0.388 | 0.426 | 11.45 GB | ~10 m |
 | 250 | (checkpoint) | (checkpoint) | 11.49 GB | ~25 m |
 | 500 | (checkpoint) | (checkpoint) | 11.49 GB | ~50 m |
 | 750 | (checkpoint) | (checkpoint) | 11.49 GB | ~75 m |
 | 1000 | **0.222** | **0.369** | 11.49 GB | ~95 m |
 
-Most of the learning happened in the first 100 iters (val 6.365 → 0.426).
-Iters 100–1,000 contributed an additional 0.057 of val-loss improvement —
+Most of the learning happened in the first 100 iters (val 6.365 -> 0.426).
+Iters 100–1,000 contributed an additional 0.057 of val-loss improvement -
 diminishing but still positive at 1,000. Train/val gap at the end is 0.147
 (mild memorisation, val still trending down).
 
 ### Checkpoints
 
-All four intermediate checkpoints are published under `checkpoints/` for
-ablation studies and early-stopping analysis:
+All four training checkpoints are published under `checkpoints/`:
 
 - `0000250_adapters.safetensors`
 - `0000500_adapters.safetensors`
-- `0000750_adapters.safetensors`
-- `0001000_adapters.safetensors` (= the root `adapters.safetensors`)
+- `0000750_adapters.safetensors` (**= the root `adapters.safetensors`**, best by tier score)
+- `0001000_adapters.safetensors` (final iter; mildly overfit, see learning curve below)
+
+### Learning curve, why iter 750, not iter 1000
+
+The 20-state eval bench was run against the untuned base and each of the
+four saved checkpoints. The curve is **not monotonic**:
+
+| checkpoint | mean tier | Δ vs teacher | Δ vs untuned | foundation recovery (of 7) | illegal | JSON valid |
+|---:|---:|---:|---:|---:|---:|---:|
+| untuned (iter 0) | 2.10 | -1.32 | 0.00 | 2 / 7 | 1 | 20 / 20 |
+| iter 250 | 2.10 | -1.32 | 0.00 | 3 / 7 | 3 | 20 / 20 |
+| iter 500 | 2.60 | -0.82 | +0.50 | 4 / 7 | 2 | 18 / 20 |
+| **iter 750** | **3.15** | **-0.27** | **+1.05** | **6 / 7** | 2 | 20 / 20 |
+| iter 1000 | 2.75 | -0.67 | +0.65 | 4 / 7 | 0 | 20 / 20 |
+
+Key observations:
+
+1. **Iter 750 is the strategic peak.** Mean tier 3.15 is within 0.27 of the
+    31B teacher's 3.42; 6 of 7 teacher-foundation states are correctly
+    recovered (vs only 2 of 7 untuned, 4 of 7 at iter 1000).
+2. **Iter 1000 has lost ground.** Two of the four foundation moves
+    recovered at iter 750 regressed to non-foundation choices by iter 1000.
+    Mean tier dropped from 3.15 to 2.75.
+3. **There is a real format / strategy tradeoff at iter 1000.** It is the
+    *only* checkpoint with zero illegal moves, but the strategic regression
+    outweighs the marginal format gain (iter 750's 2 illegal moves can be
+    handled client-side by falling back to a draw).
+4. **Iter 500 had a brief JSON-format instability** (2/20 generations
+    missed the JSON schema). This had recovered by iter 750. Worth flagging
+    as a known training-dynamics quirk on this dataset size.
+
+If you want **strict format reliability** at the cost of strategic strength,
+the iter-1000 weights under `checkpoints/0001000_adapters.safetensors` are
+appropriate. For most use cases, the shipped iter-750 weights are the right
+default.
+
+Raw per-checkpoint scored eval results are published under `eval/`:
+`posttune_at250.json`, `posttune_at500.json`, `posttune_at750.json`,
+`posttune_at1000.json`, plus the aggregated `learning_curve.json`.
 
 ## Evaluation
 
@@ -233,7 +351,7 @@ ablation studies and early-stopping analysis:
 composed of 5 early-game, 8 midgame, and 7 oscillation states from two
 post-cutover production sessions (template `0462323c…`). The bench
 intentionally includes 7 states where the 31B teacher chose a
-`{tableau,discard}_to_foundation` move — the failure mode this distillation
+`{tableau,discard}_to_foundation` move, the failure mode this distillation
 was most intended to fix.
 
 The 31B teacher's pick on each state is the production-recorded ground truth.
@@ -243,15 +361,16 @@ draw = 1, recycle = 1, illegal = 0) is the same scale used in the
 A single generation per state was used; future work should add multiple
 runs per state for variance estimation.
 
-### Headline
+### Headline (iter-750 shipped weights)
 
-| metric | untuned base | this adapter | 31B teacher | Δ this vs base |
+| metric | untuned base | this adapter (iter 750) | iter 1000 (for ref) | 31B teacher |
 |---|---:|---:|---:|---:|
-| JSON validity | 20 / 20 | **20 / 20** | — | 0 |
-| Illegal moves chosen | 1 / 20 | **0 / 20** | — | **−1** |
-| Teacher-pick agreement | 11 / 20 | 11 / 20 | — | 0 |
-| **Mean tier (all 20)** | 2.10 | **2.75** | 3.42 | **+0.65** |
-| **Gap to teacher** | **−1.32** | **−0.67** | — | **+0.65** (halved) |
+| JSON validity | 20 / 20 | **20 / 20** | 20 / 20 | - |
+| Illegal moves chosen | 1 / 20 | 2 / 20 | **0 / 20** | - |
+| Teacher-pick agreement | 11 / 20 | 11 / 20 | 11 / 20 | - |
+| **Mean tier (all 20)** | 2.10 | **3.15** | 2.75 | 3.42 |
+| **Gap to teacher** | **-1.32** | **-0.27** | -0.67 | - |
+| Foundation recovery (of 7 missed) | 2 / 7 | **6 / 7** | 4 / 7 | - |
 
 Teacher-pick agreement is unchanged in count but shifted in composition: the
 adapter recovered some agreements on foundation states and lost some on
@@ -261,44 +380,67 @@ captures it.
 
 ### Per category
 
-| category | n | untuned | adapter | teacher | Δ |
-|---|---:|---:|---:|---:|---:|
-| early | 5 | 2.60 | 3.20 | 4.20 | +0.60 |
-| midgame | 8 | 1.38 | 1.75 | 2.00 | +0.38 |
-| oscillation | 7 | 2.57 | **3.57** | 4.29 | **+1.00** |
+| category | n | untuned | adapter (iter 750) | iter 1000 | teacher | Δ adapter vs untuned |
+|---|---:|---:|---:|---:|---:|---:|
+| early | 5 | 2.60 | **4.20** | 3.20 | 4.20 | **+1.60** (matches teacher) |
+| midgame | 8 | 1.38 | 2.12 | 1.75 | 2.00 | +0.74 (beats teacher mean) |
+| oscillation | 7 | 2.57 | 3.57 | 3.57 | 4.29 | +1.00 |
 
-Oscillation gained the most — same category where the foundation-miss
+Oscillation gained the most, same category where the foundation-miss
 failure mode lived in the untuned base.
 
 ### Foundation-move recovery (the primary fine-tuning target)
 
-On 5 of 20 bench states the teacher chose a `tableau_to_foundation` move that
-the untuned adapter missed:
+The bench includes 7 states where the teacher chose a foundation move.
+At iter 750, **6 of 7 are correctly recovered**:
 
-| state | untuned pick | adapter pick | teacher pick | outcome |
+| state | untuned (iter 0) | iter 750 (shipped) | iter 1000 | teacher |
 |---|---|---|---|---|
-| `early-3687a40eda7b` | shuffle | **foundation** | foundation (diff. idx) | recovered (tier 2 → 6) |
-| `early-e6291973dd07` | shuffle | draw | foundation | regressed (2 → 1) |
-| `midgame-4ab5735a4f20` | draw | draw | foundation | unchanged |
-| `oscillation-a774c0d22f24` | draw | shuffle | foundation | slight (+1) |
-| **`oscillation-bfb84ae55c3f`** | draw | **foundation** | foundation | **fully recovered** |
+| `early-3687a40eda7b` | shuffle | **foundation** | **foundation** | foundation |
+| `early-e6291973dd07` | shuffle | **foundation** | draw | foundation |
+| `midgame-4ab5735a4f20` | draw | **foundation** | draw | foundation |
+| `oscillation-026f3139d6f2` | **foundation** | **foundation** | **foundation** | foundation |
+| `oscillation-30700e2ca639` | **foundation** | **foundation** | **foundation** | foundation |
+| `oscillation-a774c0d22f24` | draw | **foundation** | shuffle | foundation |
+| `oscillation-bfb84ae55c3f` | draw | **foundation** | **foundation** | foundation |
+| Recovered count | **2 / 7** | **6 / 7** | **4 / 7** | - |
 
 `oscillation-bfb84a` is notable: previously a 3-experiment replicated failure
 mode (C0-Haiku missed 1/3 runs, A4-Haiku missed 1/3 runs, untuned 3n-E2B
-missed 3/3 runs). After this fine-tune, it is correctly solved.
+missed 3/3 runs). The adapter solves it from iter 250 onward and the
+solution is stable through iter 1000.
+
+The single state still missed at iter 750 (`midgame-4ab5735a4f20`) was also
+the hardest at iter 1000 (still missed there too). It is a state where the
+foundation move is at `move_index=1` of a 4-move array; the adapter
+consistently prefers `move_index=0` (a draw). Probably needs targeted
+training-data augmentation to fix.
 
-### "Lost" agreements that are actually wins
+### Adapter strictly outperforms the teacher on three states
 
-Two states where the adapter's disagreement with the teacher is a tier
-*improvement*:
+States where the iter-750 adapter's pick is a higher tier than the teacher's:
 
-| state | untuned (= teacher) | adapter | tier change |
+| state | teacher | iter-750 adapter | tier improvement |
 |---|---|---|---:|
-| `midgame-031d9c9e3fe7` | shuffle (2) | waste_play (4) | +2 above teacher |
-| `oscillation-d0ff552ed744` | draw (1) | shuffle (2) | +1 above teacher |
+| `midgame-0d463176c4be` | draw (1) | shuffle (2) | +1 |
+| `midgame-a658537fe2ae` | draw (1) | shuffle (2) | +1 |
+| `oscillation-21cc5243e1d8` | draw (1) | shuffle (2) | +1 |
+
+These are all draw -> shuffle substitutions: when the teacher punted with a
+draw, the adapter found a productive tableau move. The 31B teacher is not
+an oracle; it leaves some tier points on the table that distillation has
+picked up.
+
+### Two illegal moves remain at iter 750
+
+| state | n legal | iter-750 chose | note |
+|---|---:|---:|---|
+| `midgame-81dc0fb02394` | 2 | 2 | off-by-one (same state that was illegal untuned) |
+| `oscillation-d0ff552ed744` | 2 | 2 | off-by-one |
 
-The 31B teacher is not an oracle; it leaves some tier points on the table.
-The adapter has picked up a few of them.
+Both are choosing `move_index=2` on a 2-move array. Client code should
+fall back to the highest-tier legal move. Iter-1000 fixes both but at the
+cost of two foundation moves, net negative trade.
 
 ### Reproducing the evaluation
 
@@ -319,28 +461,28 @@ Wall time ~5 min per arm on M5.
 ## Limitations
 
 - **N = 20 single-run eval.** The +0.65 tier delta is large enough to be
-  directionally trustworthy, but per-state changes (especially 1- or
-  2-state foundation gains) should not be over-interpreted as guarantees on
-  unseen positions.
+    directionally trustworthy, but per-state changes (especially 1- or
+    2-state foundation gains) should not be over-interpreted as guarantees on
+    unseen positions.
 - **Heterogeneous training templates.** Training mixed pre-cutover legacy and
-  current production prompt formats; eval is on post-cutover only. Effect
-  on generalisation across template shift is unmeasured.
+    current production prompt formats; eval is on post-cutover only. Effect
+    on generalisation across template shift is unmeasured.
 - **No endgame states in bench.** Both source post-cutover sessions stalled
-  ≤ 25 % progress (genuine sample), so the adapter's endgame behaviour is
-  untested. Prior `gemma-4-31b-it` evidence suggests endgame is a different
-  failure regime; treat extrapolation with caution.
+    ≤ 25 % progress (genuine sample), so the adapter's endgame behaviour is
+    untested. Prior `gemma-4-31b-it` evidence suggests endgame is a different
+    failure regime; treat extrapolation with caution.
 - **Trained on a teacher that itself loses ~ 55 % of games.** The adapter's
-  ceiling is teacher-level play, not optimal play. The "lost agreements that
-  are wins" rows hint there is room to outperform the teacher in places, but
-  the dataset doesn't actively reward that — only teacher imitation.
+    ceiling is teacher-level play, not optimal play. The "lost agreements that
+    are wins" rows hint there is room to outperform the teacher in places, but
+    the dataset doesn't actively reward that, only teacher imitation.
 - **Memorisation risk.** Final train loss 0.222 vs val 0.369 shows mild
-  divergence; pushing iters past ~1,500 without data augmentation is likely
-  to widen this.
+    divergence; pushing iters past ~1,500 without data augmentation is likely
+    to widen this.
 - **Confidence field is suspect.** The teacher emits `confidence: 0.9 ± 0.05`
-  almost regardless of board state; the adapter learned this poorly-calibrated
-  signal. Do not use `final_decision.confidence` for routing decisions.
+    almost regardless of board state; the adapter learned this poorly-calibrated
+    signal. Do not use `final_decision.confidence` for routing decisions.
 - **Apple-Silicon-only.** Distributed via `mlx`. CUDA/CPU inference would
-  need conversion through `transformers` / PEFT, which is not validated here.
+    need conversion through `transformers` / PEFT, which is not validated here.
 
 ## Bias and ethical considerations
 
@@ -349,11 +491,11 @@ classical sense (race, gender, etc.) are not directly applicable. Worth
 noting:
 
 - The teacher (and therefore the adapter) inherits whatever value
-  judgements are encoded in the harvester's prompt — including the rule
-  "prefer revealing face-down cards before sending cards to foundations"
-  which is a heuristic that loses to certain optimal lines.
+    judgements are encoded in the harvester's prompt, including the rule
+    "prefer revealing face-down cards before sending cards to foundations"
+    which is a heuristic that loses to certain optimal lines.
 - Production use will lock in the teacher's playstyle. If the goal is a
-  diverse advisor, training on a single teacher is the wrong objective.
+    diverse advisor, training on a single teacher is the wrong objective.
 
 ## License
 
@@ -373,19 +515,19 @@ If you use this adapter, please cite:
 
 ```bibtex
 @misc{orapinpatipat2026solitaireadvisor,
-  title  = {Distilling a 31B Klondike Solitaire advisor into Gemma 3n E2B via MLX QLoRA},
-  author = {Orapinpatipat, Chayut},
-  year   = {2026},
-  month  = may,
-  howpublished = {\url{https://huggingface.co/chayuto/gemma-3n-e2b-it-solitaire-advisor-lora}},
-  note   = {LoRA adapter; v1 = 1,000-iter checkpoint},
+    title = {Distilling a 31B Klondike Solitaire advisor into Gemma 3n E2B via MLX QLoRA},
+    author = {Orapinpatipat, Chayut},
+    year = {2026},
+    month = may,
+    howpublished = {\url{https://huggingface.co/chayuto/gemma-3n-e2b-it-solitaire-advisor-lora}},
+    note = {LoRA adapter; v1 = 1,000-iter checkpoint},
 }
 ```
 
 ## Acknowledgements
 
 - Base model `mlx-community/gemma-3n-E2B-it-text-4bit-dwq` from the
-  [`mlx-community`](https://huggingface.co/mlx-community) team.
+    [`mlx-community`](https://huggingface.co/mlx-community) team.
 - Training framework `mlx-lm` from Apple Machine Learning Research.
 - Teacher model `gemma-4-31b-it` from Google DeepMind.
 
@@ -398,13 +540,14 @@ through T5) are documented in the
 [methodology notes](training/METHODOLOGY.md) in this repo.
 
 Planned next iterations:
-1. Eval the 250/500/750-iter intermediate checkpoints to find the optimal
-   early-stopping point.
-2. Re-train at 2,000-3,000 iters once data growth justifies the extra
-   training time (val loss was still falling at 1,000).
+1. done ~~Eval the 250/500/750-iter intermediate checkpoints to find the
+    optimal early-stopping point.~~ (done, iter 750 selected as shipped weights)
+2. Targeted training-data augmentation on the one remaining
+    missed-foundation state (`midgame-4ab5735a4f20`) to push foundation
+    recovery from 6/7 to 7/7.
 3. Re-train on a post-cutover-only slice once ≥ 1,000 such rows are
-   available (currently 351).
+    available (currently 351). Should reduce template-shift confound.
 4. **Re-publish on Gemma 4 E2B** as soon as `mlx-lm` ships the
-   alternating-attention architecture support that 0.31.3 lacks. Will live
-   at `chayuto/gemma-4-e2b-it-solitaire-advisor-lora`; this Gemma 3n repo
-   will remain as the v1 fallback / reproducibility baseline.
+    alternating-attention architecture support that 0.31.3 lacks. Will live
+    at `chayuto/gemma-4-e2b-it-solitaire-advisor-lora`; this Gemma 3n repo
+    will remain as the v1 fallback / reproducibility baseline.

adapters.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:60189aec804bad85ebb74c589fe3e08eff96d429521de1aef3028bf1bed214f5
+oid sha256:7d9012f195919a48c2809f1d4b1dc7294f5485beef39bfadbd21d2eda23b2910
 size 45116520

eval/learning_curve.json

@@ -0,0 +1,85 @@
+{
+  "teacher_mean_tier": 3.421,
+  "checkpoints": [
+    {
+      "label": "untuned (iter 0)",
+      "n": 20,
+      "json_valid_count": 20,
+      "illegal_count": 1,
+      "agreement_count": 11,
+      "mean_tier_all": 2.1,
+      "delta_vs_teacher": -1.321,
+      "per_category": {
+        "early": 2.6,
+        "midgame": 1.375,
+        "oscillation": 2.571
+      },
+      "foundation_count": 2,
+      "foundation_mean_tier": 2.714
+    },
+    {
+      "label": "tuned @ iter 250",
+      "n": 20,
+      "json_valid_count": 20,
+      "illegal_count": 3,
+      "agreement_count": 7,
+      "mean_tier_all": 2.1,
+      "delta_vs_teacher": -1.321,
+      "per_category": {
+        "early": 2.2,
+        "midgame": 0.875,
+        "oscillation": 3.429
+      },
+      "foundation_count": 3,
+      "foundation_mean_tier": 3.571
+    },
+    {
+      "label": "tuned @ iter 500",
+      "n": 20,
+      "json_valid_count": 18,
+      "illegal_count": 2,
+      "agreement_count": 11,
+      "mean_tier_all": 2.6,
+      "delta_vs_teacher": -0.821,
+      "per_category": {
+        "early": 2.2,
+        "midgame": 2.0,
+        "oscillation": 3.571
+      },
+      "foundation_count": 4,
+      "foundation_mean_tier": 4.0
+    },
+    {
+      "label": "tuned @ iter 750",
+      "n": 20,
+      "json_valid_count": 20,
+      "illegal_count": 2,
+      "agreement_count": 11,
+      "mean_tier_all": 3.15,
+      "delta_vs_teacher": -0.271,
+      "per_category": {
+        "early": 4.2,
+        "midgame": 2.125,
+        "oscillation": 3.571
+      },
+      "foundation_count": 6,
+      "foundation_mean_tier": 5.286
+    },
+    {
+      "label": "tuned @ iter 1000",
+      "n": 20,
+      "json_valid_count": 20,
+      "illegal_count": 0,
+      "agreement_count": 11,
+      "mean_tier_all": 2.75,
+      "delta_vs_teacher": -0.671,
+      "per_category": {
+        "early": 3.2,
+        "midgame": 1.75,
+        "oscillation": 3.571
+      },
+      "foundation_count": 4,
+      "foundation_mean_tier": 4.0
+    }
+  ]
+}

eval/posttune_at1000.json

@@ -0,0 +1,331 @@
+{
+  "model": "mlx-community/gemma-3n-E2B-it-text-4bit-dwq",
+  "adapter_path": "adapters_t5",
+  "n": 20,
+  "overall_peak_gb": 6.37,
+  "mean_call_seconds": 13.28,
+  "json_valid_count": 20,
+  "agreement_count": 11,
+  "results": [
+    {
+      "state_id": "early-1dbcd96c5df6",
+      "category": "early",
+      "call_seconds": 12.33,
+      "call_peak_gb": 6.29,
+      "response_chars": 804,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "early-3687a40eda7b",
+      "category": "early",
+      "call_seconds": 8.05,
+      "call_peak_gb": 6.27,
+      "response_chars": 662,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 3,
+      "agreement": false,
+      "e2b_tier": "foundation",
+      "e2b_tier_score": 6,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 5
+    },
+    {
+      "state_id": "early-81deee72436d",
+      "category": "early",
+      "call_seconds": 16.81,
+      "call_peak_gb": 6.35,
+      "response_chars": 1292,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "waste_play",
+      "e2b_tier_score": 4,
+      "teacher_tier": "waste_play",
+      "teacher_tier_score": 4,
+      "n_legal": 3
+    },
+    {
+      "state_id": "early-acd9af4ae639",
+      "category": "early",
+      "call_seconds": 11.54,
+      "call_peak_gb": 6.36,
+      "response_chars": 897,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "waste_play",
+      "e2b_tier_score": 4,
+      "teacher_tier": "waste_play",
+      "teacher_tier_score": 4,
+      "n_legal": 2
+    },
+    {
+      "state_id": "early-e6291973dd07",
+      "category": "early",
+      "call_seconds": 14.1,
+      "call_peak_gb": 6.35,
+      "response_chars": 1084,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 2,
+      "agreement": false,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 3
+    },
+    {
+      "state_id": "midgame-031d9c9e3fe7",
+      "category": "midgame",
+      "call_seconds": 16.42,
+      "call_peak_gb": 6.34,
+      "response_chars": 1243,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false,
+      "e2b_tier": "waste_play",
+      "e2b_tier_score": 4,
+      "teacher_tier": "shuffle",
+      "teacher_tier_score": 2,
+      "n_legal": 6
+    },
+    {
+      "state_id": "midgame-0cef3a609da7",
+      "category": "midgame",
+      "call_seconds": 12.52,
+      "call_peak_gb": 6.31,
+      "response_chars": 899,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "shuffle",
+      "e2b_tier_score": 2,
+      "teacher_tier": "shuffle",
+      "teacher_tier_score": 2,
+      "n_legal": 3
+    },
+    {
+      "state_id": "midgame-0d463176c4be",
+      "category": "midgame",
+      "call_seconds": 11.58,
+      "call_peak_gb": 6.36,
+      "response_chars": 816,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "midgame-230df7b7160e",
+      "category": "midgame",
+      "call_seconds": 12.17,
+      "call_peak_gb": 6.34,
+      "response_chars": 936,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "midgame-4ab5735a4f20",
+      "category": "midgame",
+      "call_seconds": 13.1,
+      "call_peak_gb": 6.34,
+      "response_chars": 993,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 1,
+      "agreement": false,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 4
+    },
+    {
+      "state_id": "midgame-81dc0fb02394",
+      "category": "midgame",
+      "call_seconds": 16.42,
+      "call_peak_gb": 6.37,
+      "response_chars": 1263,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 1,
+      "agreement": false,
+      "e2b_tier": "shuffle",
+      "e2b_tier_score": 2,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "midgame-823116ccc048",
+      "category": "midgame",
+      "call_seconds": 17.09,
+      "call_peak_gb": 6.34,
+      "response_chars": 1271,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": null,
+      "agreement": false,
+      "e2b_tier": "shuffle",
+      "e2b_tier_score": 2,
+      "teacher_tier": null,
+      "teacher_tier_score": null,
+      "n_legal": 5
+    },
+    {
+      "state_id": "midgame-a658537fe2ae",
+      "category": "midgame",
+      "call_seconds": 12.4,
+      "call_peak_gb": 6.33,
+      "response_chars": 927,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "oscillation-026f3139d6f2",
+      "category": "oscillation",
+      "call_seconds": 14.49,
+      "call_peak_gb": 6.34,
+      "response_chars": 1077,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "foundation",
+      "e2b_tier_score": 6,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 4
+    },
+    {
+      "state_id": "oscillation-21cc5243e1d8",
+      "category": "oscillation",
+      "call_seconds": 14.74,
+      "call_peak_gb": 6.33,
+      "response_chars": 1132,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "draw",
+      "e2b_tier_score": 1,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "oscillation-30700e2ca639",
+      "category": "oscillation",
+      "call_seconds": 10.2,
+      "call_peak_gb": 6.36,
+      "response_chars": 796,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true,
+      "e2b_tier": "foundation",
+      "e2b_tier_score": 6,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 3
+    },
+    {
+      "state_id": "oscillation-a774c0d22f24",
+      "category": "oscillation",
+      "call_seconds": 12.21,
+      "call_peak_gb": 6.34,
+      "response_chars": 795,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 2,
+      "agreement": false,
+      "e2b_tier": "shuffle",
+      "e2b_tier_score": 2,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 5
+    },
+    {
+      "state_id": "oscillation-bfb84ae55c3f",
+      "category": "oscillation",
+      "call_seconds": 13.66,
+      "call_peak_gb": 6.34,
+      "response_chars": 1100,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true,
+      "e2b_tier": "foundation",
+      "e2b_tier_score": 6,
+      "teacher_tier": "foundation",
+      "teacher_tier_score": 6,
+      "n_legal": 3
+    },
+    {
+      "state_id": "oscillation-d0ff552ed744",
+      "category": "oscillation",
+      "call_seconds": 11.11,
+      "call_peak_gb": 6.31,
+      "response_chars": 821,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false,
+      "e2b_tier": "shuffle",
+      "e2b_tier_score": 2,
+      "teacher_tier": "draw",
+      "teacher_tier_score": 1,
+      "n_legal": 2
+    },
+    {
+      "state_id": "oscillation-d729a3bd2b7a",
+      "category": "oscillation",
+      "call_seconds": 14.66,
+      "call_peak_gb": 6.34,
+      "response_chars": 1118,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 2,
+      "agreement": false,
+      "e2b_tier": "shuffle",
+      "e2b_tier_score": 2,
+      "teacher_tier": "waste_play",
+      "teacher_tier_score": 4,
+      "n_legal": 4
+    }
+  ]
+}

eval/posttune_at250.json

@@ -0,0 +1,231 @@
+{
+  "model": "mlx-community/gemma-3n-E2B-it-text-4bit-dwq",
+  "adapter_path": "adapters_t5_at250",
+  "n": 20,
+  "overall_peak_gb": 6.37,
+  "mean_call_seconds": 15.46,
+  "json_valid_count": 20,
+  "agreement_count": 7,
+  "results": [
+    {
+      "state_id": "early-1dbcd96c5df6",
+      "category": "early",
+      "call_seconds": 18.62,
+      "call_peak_gb": 6.29,
+      "response_chars": 1278,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "early-3687a40eda7b",
+      "category": "early",
+      "call_seconds": 17.27,
+      "call_peak_gb": 6.27,
+      "response_chars": 1297,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 3,
+      "agreement": false
+    },
+    {
+      "state_id": "early-81deee72436d",
+      "category": "early",
+      "call_seconds": 15.66,
+      "call_peak_gb": 6.35,
+      "response_chars": 1059,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "early-acd9af4ae639",
+      "category": "early",
+      "call_seconds": 16.52,
+      "call_peak_gb": 6.36,
+      "response_chars": 1242,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "early-e6291973dd07",
+      "category": "early",
+      "call_seconds": 12.47,
+      "call_peak_gb": 6.35,
+      "response_chars": 872,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 2,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-031d9c9e3fe7",
+      "category": "midgame",
+      "call_seconds": 14.01,
+      "call_peak_gb": 6.34,
+      "response_chars": 1044,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-0cef3a609da7",
+      "category": "midgame",
+      "call_seconds": 14.7,
+      "call_peak_gb": 6.31,
+      "response_chars": 1125,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-0d463176c4be",
+      "category": "midgame",
+      "call_seconds": 17.76,
+      "call_peak_gb": 6.36,
+      "response_chars": 1540,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-230df7b7160e",
+      "category": "midgame",
+      "call_seconds": 14.29,
+      "call_peak_gb": 6.34,
+      "response_chars": 1062,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-4ab5735a4f20",
+      "category": "midgame",
+      "call_seconds": 16.07,
+      "call_peak_gb": 6.33,
+      "response_chars": 1284,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-81dc0fb02394",
+      "category": "midgame",
+      "call_seconds": 13.61,
+      "call_peak_gb": 6.37,
+      "response_chars": 1039,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-823116ccc048",
+      "category": "midgame",
+      "call_seconds": 14.83,
+      "call_peak_gb": 6.34,
+      "response_chars": 1038,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": null,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-a658537fe2ae",
+      "category": "midgame",
+      "call_seconds": 17.23,
+      "call_peak_gb": 6.33,
+      "response_chars": 1364,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-026f3139d6f2",
+      "category": "oscillation",
+      "call_seconds": 15.04,
+      "call_peak_gb": 6.34,
+      "response_chars": 1012,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-21cc5243e1d8",
+      "category": "oscillation",
+      "call_seconds": 15.57,
+      "call_peak_gb": 6.33,
+      "response_chars": 1125,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-30700e2ca639",
+      "category": "oscillation",
+      "call_seconds": 12.05,
+      "call_peak_gb": 6.36,
+      "response_chars": 940,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-a774c0d22f24",
+      "category": "oscillation",
+      "call_seconds": 13.98,
+      "call_peak_gb": 6.34,
+      "response_chars": 962,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 2,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-bfb84ae55c3f",
+      "category": "oscillation",
+      "call_seconds": 18.77,
+      "call_peak_gb": 6.34,
+      "response_chars": 1464,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-d0ff552ed744",
+      "category": "oscillation",
+      "call_seconds": 13.37,
+      "call_peak_gb": 6.31,
+      "response_chars": 966,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-d729a3bd2b7a",
+      "category": "oscillation",
+      "call_seconds": 17.42,
+      "call_peak_gb": 6.34,
+      "response_chars": 1320,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 2,
+      "agreement": false
+    }
+  ]
+}

eval/posttune_at500.json

@@ -0,0 +1,231 @@
+{
+  "model": "mlx-community/gemma-3n-E2B-it-text-4bit-dwq",
+  "adapter_path": "adapters_t5_at500",
+  "n": 20,
+  "overall_peak_gb": 6.37,
+  "mean_call_seconds": 13.78,
+  "json_valid_count": 18,
+  "agreement_count": 11,
+  "results": [
+    {
+      "state_id": "early-1dbcd96c5df6",
+      "category": "early",
+      "call_seconds": 13.59,
+      "call_peak_gb": 6.29,
+      "response_chars": 944,
+      "json_valid": false,
+      "e2b_move_index": null,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "early-3687a40eda7b",
+      "category": "early",
+      "call_seconds": 15.64,
+      "call_peak_gb": 6.27,
+      "response_chars": 1159,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 3,
+      "agreement": false
+    },
+    {
+      "state_id": "early-81deee72436d",
+      "category": "early",
+      "call_seconds": 10.93,
+      "call_peak_gb": 6.35,
+      "response_chars": 818,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "early-acd9af4ae639",
+      "category": "early",
+      "call_seconds": 10.03,
+      "call_peak_gb": 6.36,
+      "response_chars": 818,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "early-e6291973dd07",
+      "category": "early",
+      "call_seconds": 17.64,
+      "call_peak_gb": 6.35,
+      "response_chars": 1312,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 2,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-031d9c9e3fe7",
+      "category": "midgame",
+      "call_seconds": 11.51,
+      "call_peak_gb": 6.34,
+      "response_chars": 920,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-0cef3a609da7",
+      "category": "midgame",
+      "call_seconds": 16.38,
+      "call_peak_gb": 6.31,
+      "response_chars": 1182,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-0d463176c4be",
+      "category": "midgame",
+      "call_seconds": 14.66,
+      "call_peak_gb": 6.36,
+      "response_chars": 1248,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-230df7b7160e",
+      "category": "midgame",
+      "call_seconds": 11.39,
+      "call_peak_gb": 6.34,
+      "response_chars": 887,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-4ab5735a4f20",
+      "category": "midgame",
+      "call_seconds": 11.08,
+      "call_peak_gb": 6.33,
+      "response_chars": 866,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-81dc0fb02394",
+      "category": "midgame",
+      "call_seconds": 19.54,
+      "call_peak_gb": 6.37,
+      "response_chars": 1465,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-823116ccc048",
+      "category": "midgame",
+      "call_seconds": 15.73,
+      "call_peak_gb": 6.34,
+      "response_chars": 1239,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": null,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-a658537fe2ae",
+      "category": "midgame",
+      "call_seconds": 15.14,
+      "call_peak_gb": 6.33,
+      "response_chars": 1126,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-026f3139d6f2",
+      "category": "oscillation",
+      "call_seconds": 15.71,
+      "call_peak_gb": 6.34,
+      "response_chars": 1093,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-21cc5243e1d8",
+      "category": "oscillation",
+      "call_seconds": 13.32,
+      "call_peak_gb": 6.33,
+      "response_chars": 1040,
+      "json_valid": false,
+      "e2b_move_index": null,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-30700e2ca639",
+      "category": "oscillation",
+      "call_seconds": 13.18,
+      "call_peak_gb": 6.36,
+      "response_chars": 1056,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-a774c0d22f24",
+      "category": "oscillation",
+      "call_seconds": 11.23,
+      "call_peak_gb": 6.34,
+      "response_chars": 818,
+      "json_valid": true,
+      "e2b_move_index": 3,
+      "teacher_move_index": 2,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-bfb84ae55c3f",
+      "category": "oscillation",
+      "call_seconds": 11.96,
+      "call_peak_gb": 6.34,
+      "response_chars": 907,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-d0ff552ed744",
+      "category": "oscillation",
+      "call_seconds": 11.67,
+      "call_peak_gb": 6.33,
+      "response_chars": 878,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-d729a3bd2b7a",
+      "category": "oscillation",
+      "call_seconds": 15.23,
+      "call_peak_gb": 6.34,
+      "response_chars": 1167,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 2,
+      "agreement": true
+    }
+  ]
+}

eval/posttune_at750.json

@@ -0,0 +1,231 @@
+{
+  "model": "mlx-community/gemma-3n-E2B-it-text-4bit-dwq",
+  "adapter_path": "adapters_t5_at750",
+  "n": 20,
+  "overall_peak_gb": 6.37,
+  "mean_call_seconds": 14.75,
+  "json_valid_count": 20,
+  "agreement_count": 11,
+  "results": [
+    {
+      "state_id": "early-1dbcd96c5df6",
+      "category": "early",
+      "call_seconds": 16.85,
+      "call_peak_gb": 6.29,
+      "response_chars": 1316,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "early-3687a40eda7b",
+      "category": "early",
+      "call_seconds": 8.67,
+      "call_peak_gb": 6.27,
+      "response_chars": 695,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 3,
+      "agreement": false
+    },
+    {
+      "state_id": "early-81deee72436d",
+      "category": "early",
+      "call_seconds": 10.9,
+      "call_peak_gb": 6.35,
+      "response_chars": 795,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "early-acd9af4ae639",
+      "category": "early",
+      "call_seconds": 10.64,
+      "call_peak_gb": 6.36,
+      "response_chars": 798,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "early-e6291973dd07",
+      "category": "early",
+      "call_seconds": 11.33,
+      "call_peak_gb": 6.35,
+      "response_chars": 946,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 2,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-031d9c9e3fe7",
+      "category": "midgame",
+      "call_seconds": 16.98,
+      "call_peak_gb": 6.34,
+      "response_chars": 1257,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-0cef3a609da7",
+      "category": "midgame",
+      "call_seconds": 16.36,
+      "call_peak_gb": 6.31,
+      "response_chars": 1073,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-0d463176c4be",
+      "category": "midgame",
+      "call_seconds": 16.43,
+      "call_peak_gb": 6.36,
+      "response_chars": 1365,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-230df7b7160e",
+      "category": "midgame",
+      "call_seconds": 16.91,
+      "call_peak_gb": 6.34,
+      "response_chars": 1346,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-4ab5735a4f20",
+      "category": "midgame",
+      "call_seconds": 17.47,
+      "call_peak_gb": 6.33,
+      "response_chars": 1363,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "midgame-81dc0fb02394",
+      "category": "midgame",
+      "call_seconds": 18.22,
+      "call_peak_gb": 6.37,
+      "response_chars": 1427,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-823116ccc048",
+      "category": "midgame",
+      "call_seconds": 17.98,
+      "call_peak_gb": 6.34,
+      "response_chars": 1284,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": null,
+      "agreement": false
+    },
+    {
+      "state_id": "midgame-a658537fe2ae",
+      "category": "midgame",
+      "call_seconds": 15.3,
+      "call_peak_gb": 6.33,
+      "response_chars": 1151,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 1,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-026f3139d6f2",
+      "category": "oscillation",
+      "call_seconds": 12.69,
+      "call_peak_gb": 6.34,
+      "response_chars": 959,
+      "json_valid": true,
+      "e2b_move_index": 0,
+      "teacher_move_index": 0,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-21cc5243e1d8",
+      "category": "oscillation",
+      "call_seconds": 17.12,
+      "call_peak_gb": 6.33,
+      "response_chars": 1345,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-30700e2ca639",
+      "category": "oscillation",
+      "call_seconds": 13.23,
+      "call_peak_gb": 6.36,
+      "response_chars": 1020,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-a774c0d22f24",
+      "category": "oscillation",
+      "call_seconds": 16.5,
+      "call_peak_gb": 6.34,
+      "response_chars": 1193,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 2,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-bfb84ae55c3f",
+      "category": "oscillation",
+      "call_seconds": 11.69,
+      "call_peak_gb": 6.34,
+      "response_chars": 934,
+      "json_valid": true,
+      "e2b_move_index": 1,
+      "teacher_move_index": 1,
+      "agreement": true
+    },
+    {
+      "state_id": "oscillation-d0ff552ed744",
+      "category": "oscillation",
+      "call_seconds": 13.85,
+      "call_peak_gb": 6.31,
+      "response_chars": 1099,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 0,
+      "agreement": false
+    },
+    {
+      "state_id": "oscillation-d729a3bd2b7a",
+      "category": "oscillation",
+      "call_seconds": 15.94,
+      "call_peak_gb": 6.34,
+      "response_chars": 1312,
+      "json_valid": true,
+      "e2b_move_index": 2,
+      "teacher_move_index": 2,
+      "agreement": true
+    }
+  ]
+}

training/METHODOLOGY.md

@@ -1,4 +1,4 @@
-# Methodology — training & evaluation
+# Methodology, training & evaluation
 
 This document captures the methodology behind the v1 adapter
 (`adapters.safetensors`, 1,000-iter LoRA checkpoint). It is intentionally
@@ -7,7 +7,7 @@ reproduce the work.
 
 ## 1. Motivation
 
-The project goal is a *local* Klondike Solitaire advisor — a small LLM that
+The project goal is a *local* Klondike Solitaire advisor, a small LLM that
 runs on consumer Apple Silicon hardware (16 GB unified memory, Metal GPU)
 and matches the move-selection quality of a hosted 31B teacher
 (`gemma-4-31b-it`). The teacher is too large to run locally; distillation
@@ -15,15 +15,15 @@ into a small student is the path to a usable local product.
 
 Klondike was chosen as the domain because:
 
-1. Move-by-move strategy is *easy to score* — every legal move falls into
-   one of a small set of types (foundation / reveal / waste_play / shuffle /
-   draw / recycle) with an obvious ordinal value structure.
+1. Move-by-move strategy is *easy to score*, every legal move falls into
+ one of a small set of types (foundation / reveal / waste_play / shuffle /
+ draw / recycle) with an obvious ordinal value structure.
 2. The 31B teacher already has a deployed harvester, so production-quality
-   training labels accumulate naturally.
+ training labels accumulate naturally.
 3. Failure modes are concrete and replicable. The previous prompt-format
-   study (Phase 1.5) had already identified the "foundation-miss" anti-pattern
-   as the single largest source of suboptimal play in small models — giving
-   the distillation a sharp target to aim at.
+ study (Phase 1.5) had already identified the "foundation-miss" anti-pattern
+ as the single largest source of suboptimal play in small models, giving
+ the distillation a sharp target to aim at.
 
 ## 2. Architectural decisions
 
@@ -32,7 +32,7 @@ Klondike was chosen as the domain because:
 The original target was **Gemma 4 E2B** (~2B effective parameters, text-only).
 At the time of this run, `mlx-lm` 0.31.3 was the latest published version
 and could not load any `Gemma4ForConditionalGeneration` variant (all four
-mlx-community quants tested failed identically — see the project's
+mlx-community quants tested failed identically, see the project's
 [T2 progress notes](https://github.com/chayuto/solitaire-analytics/blob/main/gemma4_finetune/tier2_progress.txt)).
 The structural issue is that mlx-lm's `Gemma4Model` class only implements
 the first 15 of 35 attention layers; the alternating-attention pattern with
@@ -64,7 +64,7 @@ quantisation step.
 
 Production prompts are 1,000-2,600 tokens. The training config uses
 `max_seq_length=2048`. At iter 1 we saw a warning that the longest single
-example was 2,298 tokens — i.e., ~5-10 % of training tails are silently
+example was 2,298 tokens, i.e., ~5-10 % of training tails are silently
 truncated at the chosen budget. Trade-off: raising `max_seq_length` to 2,624
 would eliminate the truncation but push activation memory past the 16 GB
 envelope. We accepted the truncation.
@@ -87,10 +87,10 @@ The training-eligible filter (`prepare_dataset.py`) keeps rows where:
 1. `outcome == "success"` (the teacher actually returned a response)
 2. `rawResponse` parses as JSON
 3. The parsed JSON contains all three keys: `board_analysis`,
-   `strategic_plan`, `final_decision`
+ `strategic_plan`, `final_decision`
 4. (Upstream filter, applied during ingest) the row is not from a stalled
-   game — defined as foundation count + face-down count unchanged for ≥ 25
-   consecutive turns
+ game, defined as foundation count + face-down count unchanged for ≥ 25
+ consecutive turns
 
 Of 1,730 candidate rows, 1,536 (88.8 %) survived this filter for training.
 The 11.2 % drop rate from a single field-presence check is a known
@@ -137,11 +137,11 @@ contributes gradient.
 
 | iter | train loss | val loss | wall (cumulative) |
 |---:|---:|---:|---:|
-| 1 | — | 6.365 | 0 m |
-| 10 | 3.160 | — | ~1 m |
-| 20 | 0.943 | — | ~2 m |
-| 30 | 0.580 | — | ~3 m |
-| 50 | 0.508 | — | ~5 m |
+| 1 | - | 6.365 | 0 m |
+| 10 | 3.160 | - | ~1 m |
+| 20 | 0.943 | - | ~2 m |
+| 30 | 0.580 | - | ~3 m |
+| 50 | 0.508 | - | ~5 m |
 | 100 | 0.388 | 0.426 | ~10 m |
 | 250 | (checkpoint) | (checkpoint) | ~25 m |
 | 500 | (checkpoint) | (checkpoint) | ~50 m |
@@ -149,7 +149,7 @@ contributes gradient.
 | 1000 | 0.222 | 0.369 | ~95 m |
 
 Most learning happens in the first 100 iters. Iters 100-1,000 contribute
-an additional 0.057 of val-loss improvement — diminishing but still positive.
+an additional 0.057 of val-loss improvement, diminishing but still positive.
 Train/val gap at the end is 0.147; the val curve is still trending down but
 slowly. Pushing past 2,000 iters without data augmentation is likely to
 widen the gap.
@@ -172,16 +172,16 @@ bench, all rendered against the current production prompt template
 
 - **5 early-game** (foundation count < 4)
 - **8 midgame** (foundation count 4-25)
-- **0 endgame** — both source post-cutover sessions stalled before reaching
-  endgame, so this category was unavailable. This is a real limitation of
-  the bench, acknowledged.
-- **7 oscillation** — states where the recent moves indicate the teacher
-  was looping between draws and tableau shuffles without progressing
-  foundations or revealing face-downs
+- **0 endgame**, both source post-cutover sessions stalled before reaching
+    endgame, so this category was unavailable. This is a real limitation of
+    the bench, acknowledged.
+- **7 oscillation**, states where the recent moves indicate the teacher
+    was looping between draws and tableau shuffles without progressing
+    foundations or revealing face-downs
 
 Of these, **7 states have a `{tableau,discard}_to_foundation` move
 available** in the legalMoves array. These are the foundation-move test
-ground — the failure mode this fine-tune was most intended to fix.
+ground, the failure mode this fine-tune was most intended to fix.
 
 ### 6.2 Scoring
 
@@ -189,11 +189,11 @@ The same tier-score scale used in the Phase 1.5 prompt-format study:
 
 | move type | tier | rationale |
 |---|---:|---|
-| `tableau_to_foundation` | 6 | maximally productive — advances win-progress |
+| `tableau_to_foundation` | 6 | maximally productive, advances win-progress |
 | `discard_to_foundation` | 6 | same |
 | `*_reveal` (move that flips a face-down) | 5 | unlocks information |
 | `discard_to_tableau` (waste play that lands productively) | 4 | activates a stale waste card |
-| `tableau_to_tableau` (no reveal) | 2 | "shuffle" — preserves options, no info gain |
+| `tableau_to_tableau` (no reveal) | 2 | "shuffle", preserves options, no info gain |
 | `draw_card` | 1 | always available but rarely strategically optimal |
 | `recycle_stock` | 1 | costs nothing but exposes no new state |
 | illegal (chosen `move_index` not in `legalMoves`) | 0 | failure |
@@ -201,9 +201,9 @@ The same tier-score scale used in the Phase 1.5 prompt-format study:
 ### 6.3 Comparison points
 
 - **Untuned base** (`mlx-community/gemma-3n-E2B-it-text-4bit-dwq` with no
-  adapter): the floor we are improving from.
+    adapter): the floor we are improving from.
 - **31B teacher** (`gemma-4-31b-it`): the production-recorded picks on the
-  same turns; the ceiling we are distilling toward.
+    same turns; the ceiling we are distilling toward.
 - **This adapter** (1,000-iter LoRA): the result being reported.
 
 A single inference per state per arm was used. Future work should add
@@ -211,13 +211,54 @@ multi-run variance estimation (the Phase 1.5 study used 3 runs per state).
 
 ### 6.4 Headline results
 
-| metric | untuned base | this adapter | 31B teacher | Δ adapter vs base |
+The shipped weights are the **iter-750 checkpoint**, selected after running
+the bench against all four saved checkpoints. Iter 1000 was demonstrably
+worse on this bench (overfitting), see §6.5 for the full curve.
+
+| metric | untuned base | iter-750 (shipped) | iter-1000 (regressed) | 31B teacher |
 |---|---:|---:|---:|---:|
-| JSON validity | 20 / 20 | 20 / 20 | — | 0 |
-| Illegal moves | 1 / 20 | 0 / 20 | — | **−1** |
-| Teacher agreement | 11 / 20 | 11 / 20 | — | 0 |
-| **Mean tier (all 20)** | 2.10 | 2.75 | 3.42 | **+0.65** |
-| **Gap to teacher** | −1.32 | −0.67 | — | **halved** |
+| JSON validity | 20 / 20 | 20 / 20 | 20 / 20 | - |
+| Illegal moves | 1 / 20 | 2 / 20 | 0 / 20 | - |
+| Teacher agreement | 11 / 20 | 11 / 20 | 11 / 20 | - |
+| **Mean tier (all 20)** | 2.10 | **3.15** | 2.75 | 3.42 |
+| **Gap to teacher** | **-1.32** | **-0.27** | -0.67 | - |
+| Foundation recovery (of 7) | 2 / 7 | **6 / 7** | 4 / 7 | - |
+
+### 6.5 Learning curve & early-stopping decision
+
+Each saved checkpoint was evaluated against the same 20-state bench:
+
+| iter | mean tier | Δ teacher | Δ untuned | foundation 6/7? | illegal | JSON valid |
+|---:|---:|---:|---:|---:|---:|---:|
+| 0 | 2.10 | -1.32 | 0.00 | 2 / 7 | 1 | 20 / 20 |
+| 250 | 2.10 | -1.32 | 0.00 | 3 / 7 | 3 | 20 / 20 |
+| 500 | 2.60 | -0.82 | +0.50 | 4 / 7 | 2 | **18 / 20** |
+| **750** | **3.15** | **-0.27** | **+1.05** | **6 / 7** | 2 | 20 / 20 |
+| 1000 | 2.75 | -0.67 | +0.65 | 4 / 7 | 0 | 20 / 20 |
+
+Observations:
+
+1. The curve is *non-monotonic*. Mean tier peaks at iter 750 and regresses
+    by iter 1000.
+2. **Two foundation-state regressions between iter 750 and iter 1000**:
+    `early-e6291973dd07` (foundation -> draw) and
+    `oscillation-a774c0d22f24` (foundation -> shuffle).
+3. **Iter 500 had a transient JSON-format instability** (2 / 20
+    generations failed schema) that recovered by iter 750. This kind of
+    short-window format instability midway through training is consistent
+    with the LoRA adapter still finding a stable representation.
+4. **Iter 1000 trades strategy for format reliability.** It is the only
+    checkpoint with zero illegal moves but loses tier score for the
+    privilege. On net, iter 750 wins.
+
+Decision: ship iter 750 as `adapters.safetensors`. Iter 1000 remains
+available under `checkpoints/0001000_adapters.safetensors` for users who
+prioritise format strictness over strategy. Iters 250/500 are also kept
+in `checkpoints/` for full provenance.
+
+This is the rare case where running the cheap eval against intermediate
+checkpoints *changed* the shipping decision. Future runs should default to
+evaluating every saved checkpoint before publishing.
 
 Detailed per-state, per-category, and foundation-recovery breakdowns are in
 the [model card](../README.md).
@@ -225,27 +266,27 @@ the [model card](../README.md).
 ## 7. Threats to validity
 
 - **Single eval bench (N = 20)**. The +0.65 delta is large enough that
-  noise alone is unlikely to produce it, but per-state changes (especially
-  on small subsets like the 7 foundation states) have wide effective
-  confidence intervals. A larger or multiply-resampled bench would tighten
-  the estimate.
+    noise alone is unlikely to produce it, but per-state changes (especially
+    on small subsets like the 7 foundation states) have wide effective
+    confidence intervals. A larger or multiply-resampled bench would tighten
+    the estimate.
 - **Template confound**. Training was on heterogeneous templates (~63 %
-  legacy, ~37 % current); eval is on current-only. We cannot disentangle
-  whether the +0.65 is from learning Solitaire reasoning or from learning
-  the current template's surface form. The fact that the gain is consistent
-  across categories suggests the former, but we cannot prove it.
+    legacy, ~37 % current); eval is on current-only. We cannot disentangle
+    whether the +0.65 is from learning Solitaire reasoning or from learning
+    the current template's surface form. The fact that the gain is consistent
+    across categories suggests the former, but we cannot prove it.
 - **Teacher is not ground truth**. The 31B teacher is itself a flawed
-  player; matching it more closely is the *training* objective but the
-  *real* objective is "play better Solitaire". Two bench states show the
-  adapter strictly outperforming the teacher on tier score; we have no way
-  to know if those are flukes or evidence of generalised improvement.
+    player; matching it more closely is the *training* objective but the
+    *real* objective is "play better Solitaire". Two bench states show the
+    adapter strictly outperforming the teacher on tier score; we have no way
+    to know if those are flukes or evidence of generalised improvement.
 - **No game-level eval**. We evaluated single-turn decisions on a frozen
-  set of states. The real product question — "does this adapter win more
-  games end-to-end than the untuned base?" — was not measured. This is the
-  most important deferred experiment.
+    set of states. The real product question, "does this adapter win more
+    games end-to-end than the untuned base?", was not measured. This is the
+    most important deferred experiment.
 - **Memorisation as iters grow**. The train/val gap (0.222 vs 0.369) is
-  modest at iter 1,000 but trending widen. Future iters at this dataset
-  size should be paired with data augmentation or stronger regularisation.
+    modest at iter 1,000 but trending widen. Future iters at this dataset
+    size should be paired with data augmentation or stronger regularisation.
 
 ## 8. Reproduction
 
@@ -259,11 +300,11 @@ python3.12 -m venv venv
 source venv/bin/activate
 pip install mlx mlx-lm
 
-# 3. (Re)train — needs the training dataset (separately staged for HF datasets)
+# 3. (Re)train, needs the training dataset (separately staged for HF datasets)
 # Hyperparameters identical to v1 are in training/lora_config.yaml
 mlx_lm.lora --config training/lora_config.yaml \
-            --data <your-prepared-dataset-dir> \
-            --adapter-path my_adapters
+    --data <your-prepared-dataset-dir> \
+    --adapter-path my_adapters
 
 # 4. Re-evaluate
 python eval/baseline_n20_runner.py
@@ -280,38 +321,38 @@ production prompts). The bench files and the teacher's recorded picks
 
 ```
 chayuto/gemma-3n-e2b-it-solitaire-advisor-lora/
-├── README.md                          # model card (entry point)
-├── adapter_config.json                # mlx-lm LoRA config (live)
-├── adapters.safetensors               # final (= iter 1000) adapter weights
+├── README.md # model card (entry point)
+├── adapter_config.json # mlx-lm LoRA config (live)
+├── adapters.safetensors # final (= iter 1000) adapter weights
 ├── checkpoints/
-│   ├── 0000250_adapters.safetensors
-│   ├── 0000500_adapters.safetensors
-│   ├── 0000750_adapters.safetensors
-│   └── 0001000_adapters.safetensors
+│ ├── 0000250_adapters.safetensors
+│ ├── 0000500_adapters.safetensors
+│ ├── 0000750_adapters.safetensors
+│ └── 0001000_adapters.safetensors
 ├── training/
-│   ├── METHODOLOGY.md                 # this file
-│   ├── lora_config.yaml               # production training config
-│   └── prepare_dataset.py             # data prep script
+│ ├── METHODOLOGY.md # this file
+│ ├── lora_config.yaml # production training config
+│ └── prepare_dataset.py # data prep script
 └── eval/
-    ├── baseline_n20.json              # untuned-base eval (scored)
-    ├── posttune.json                  # this-adapter eval (scored)
-    ├── teacher_picks_n20.json         # ground-truth teacher picks
-    ├── prompts_C0/                    # 20 rendered production prompts
-    ├── baseline_n20_runner.py
-    └── posttune_n20_runner.py
+ ├── baseline_n20.json # untuned-base eval (scored)
+ ├── posttune.json # this-adapter eval (scored)
+ ├── teacher_picks_n20.json # ground-truth teacher picks
+ ├── prompts_C0/ # 20 rendered production prompts
+ ├── baseline_n20_runner.py
+ └── posttune_n20_runner.py
 ```
 
 ## 10. Open questions
 
 Things this run did *not* answer, and should:
 
-1. What's the optimal stopping iter? — eval each of the 250/500/750
-   checkpoints against the same bench; pick the val-loss / tier-score elbow.
-2. What's the marginal value of additional training? — eval at 2,000 iters
-   to see whether the val curve has actually flattened.
-3. Does this adapter actually win more games? — game-level (multi-turn,
-   stateful) eval on a held-out set of seeds.
+1. What's the optimal stopping iter?, eval each of the 250/500/750
+    checkpoints against the same bench; pick the val-loss / tier-score elbow.
+2. What's the marginal value of additional training?, eval at 2,000 iters
+    to see whether the val curve has actually flattened.
+3. Does this adapter actually win more games?, game-level (multi-turn,
+    stateful) eval on a held-out set of seeds.
 4. How much of the +0.65 is from the new template's surface form vs from
-   real Solitaire reasoning? — re-train on a post-cutover-only slice once
-   one is large enough (currently 351 rows; need ~1,000+) and compare.
+    real Solitaire reasoning?, re-train on a post-cutover-only slice once
+    one is large enough (currently 351 rows; need ~1,000+) and compare.
 5. How does this compare to the Gemma 4 E2B target whenever mlx-lm catches up?