code/NOTEBOOK.md

Lab notebook: reasoning decision model (Qwen3.5-4B)

Times are UTC (IST = UTC + 5:30). Observations as they happened; numbers are from the files named.

2026-10-31

00:49 Run started (run_all.sh, tmux run). Prep reused (identical sets under the Qwen3.5 tokenizer).

00:49-01:03 Probe (results/probe.json): 128 pool items x 4 chains. Procedural: some chain correct on 100%, majority 98.4%; tasksource: 75% / 65.6%. Chains median 1,878 tokens, p90 6,414; 4 of 512 looped to 32k. Throughput is tail-bound (1,957 tok/s overall; 4.5-5.5k tok/s while the queue was full).

• Tasksource "errors" read by hand are mostly labelling conventions, not bad reasoning: SemEval aspect sentiment ("an absolute must" = neutral), AXG pronoun entailment ("said only he could" -> entailment), a packed ToS clause whose gold looks wrong. The alpha = 0.5 target is meant to make the readout unsure on exactly these.
• Qwen3.5-4B writes .c + w ("A.cw") just before </think> in about a third of chains: a model quirk, harmless for parsing.

01:03-02:53 Baseline (untrained model, results/predictions/*_base.jsonl).

• Readout, temperatures fitted on calibration (choice 1.71, yes/no 2.41, score 1.28): procedural 46.8%, tasksource 60.4%, OpenDecisions 66.5%, LocalLLaMA 63.0%. OpenDecisions Capability 61.9 (paper: 63.6 for zero-shot Qwen3.5-4B).
• Reasoning, one sample, 5,957 items, 16.9M tokens in 85 min (3.3k tok/s, 256 running sequences):

Set	Readout	Reasoning	Post-chain readout	Readout = reasoning	Letter = post-chain	Median tokens	Loops	Disagreement AUROC
procedural	46.8	95.1	96.0	46.4	98.1	2,078	5	0.739
tasksource	60.4	73.3	73.6	67.1	98.5	1,637	7	0.769
OpenDecisions	66.5	94.9	95.0	67.3	99.2	1,043	16	0.805
LocalLLaMA	63.0	71.0	71.0	69.8	98.8	2,660	24	0.746
all	62.7	83.9	84.1	66.2	98.9	1,626	52	0.777

• Chains lengthen with difficulty: procedural level 0 to 4 median 1,283 to 4,446 tokens (accuracy 100% to 89.8%); OpenDecisions easy 635 vs hard 2,117 tokens, gap readout vs reasoning 3 points (easy) vs 39 points (hard).
• Every loop is a degenerate repeat on a hard procedural item (e.g. "Wait, check R7 at ts: 1. - R7 enabled." x370).
• Where reasoning loses to the readout: taste and convention tasks (humour, sentiment of neutral facts, Italian word sense).
• LocalLLaMA "misses" read by hand sit on split soft gold (0.38 vs 0.33; 0.50 vs 0.45); the reasoner's picks are defensible.
• System view (metrics on base only, results/metrics_baseline_only.json): OpenDecisions Capability 61.8 readout only, 67.2 / 72.3 / 76.5 / 82.8 with 10 / 20 / 30 / 50% routed by margin, 84.4 thinking always. Post-chain answers are overconfident (temperatures 4.77 / 3.49 / 4.02). Gap recovered at 30% routed: 54% (margin), 55% (learned router); fixable AUROC 0.58.
• The post-chain pass re-prefilled ~15M tokens (13 min) because the prefix cache had evicted the chains. From the joint eval on, the readout after the chain is taken right after each chain from its exact token ids.

02:51-03:08 Generation, first segment (512 running sequences): about 19 items/min, 3.0-3.3k finished tok/s against 3.4k at 256 sequences in the baseline: the GPU is compute-bound (100%, 382 W), so more concurrency does not help. At that pace the original deadline (05:01) yields about 2,500 items, and joint training on them takes about 40 min of its 75. Stopped at 03:08 (380 items kept, no partial lines), reverted to vLLM's A100 batching defaults and restarted with the deadline at 05:41 (window 8.67 h) so the freed training time goes to data. A first relaunch failed in 1 s: bash integer arithmetic cannot take TOTAL_H=8.67; the end time is now computed in Python.

03:09-03:16 Engine stats (now on) showed Running: 128 reqs, Waiting: ~850, KV cache 40%: the in-process async engine falls back to vLLM's global 128-sequence default. Restarted at 03:16 with max_num_seqs=256 (412 items kept): 4.7k tok/s steady at 70% KV, against 4.1-4.6k at 128 (+12%). The GPU is close to compute-bound from about 128 sequences. Correction: the earlier "512 is no faster" comparison set a ramp-up window and a lagging completion counter against a steady-state window, and is withdrawn. Expected generation by 05:41: about 3,500 items.

05:41-06:03 Generation end (results/generation.json): 3,169 items, 12,676 chains, 41.0M tokens; chains median 1,945, p90 6,661; parse 98.8%; 154 chains (1.2%) looped to 32k, mostly on natural-language judgement items (safety and helpfulness ratings, AI-text detection, fact checks) rather than procedural ones as in eval. 2,818 items (89%) have a correct chain; majority vote correct on 85.2%. Votes unanimous on 96% of procedural and 86% of tasksource items; on 15.5% of tasksource items the reasoner is unanimous against gold (labelling conventions). Shortest correct chain median 920 tokens vs 1,502 for an average sample.

06:03-06:54 Joint training (results/adapters/joint): 5,669 examples (2,673 reasoner, 2,996 readout; 318 dropped by the timing probe, which measured during kernel compilation), 207/207 steps, 8.74M tokens, 46 min at 3.14k tok/s, 19.8 GB peak. Chain loss flat at about 0.19 throughout (the model's own chains); readout loss about 1.0 at the start and 0.6-0.9 later, but per-window values are confounded by length-grouped batching (windows hold 108 to 700 readout examples of different kinds).

06:56 Gold-only ablation: 3,169 readout examples, 1.71M tokens, 37 steps, no drop.

07:07 Token-id post-chain readout checked on Qwen3.5-0.8B: on 9 closed chains it equals the text-based readout (max difference in probability 0.0001, argmax 9/9) and the generated letter (9/9). The joint eval uses it.

07:23-08:09 Joint eval, first segment (256 sequences). Readout (calibration temperatures 1.28 / 1.18 / 1.05, against 1.71 / 2.41 / 1.28 for base): procedural 46.8 -> 71.9, tasksource 60.4 -> 66.4, OpenDecisions 66.5 -> 72.2, LocalLLaMA 63.0 -> 66.5 (all 62.7 -> 69.5); OpenDecisions Capability 61.8 -> 66.7 (Intelligence 41.2 -> 52.8, Calibration 82.5 -> 80.7). Reasoning on the first 2,564 items, same items as base: accuracy equal or higher on every set, chains 20-30% shorter, readout = own reasoning 72.9 vs 47.2 (procedural), 76.2 vs 67.7 (tasksource), 83.7 vs 78.8 (OpenDecisions). Zero loops on items where base looped 13 times (e.g. a level-4 needle lookup: base ran to 32k tokens, joint answered correctly in 1,139). From about 2,400 items on the KV cache reached 99.5% with preemption (140 running, 116 waiting, 1.9k tok/s): stopped at 08:09 and resumed at 128 sequences (MAX_NUM_SEQS=128), all 2,564 records kept.

08:14 At 128 sequences the eval decodes at 4.2-4.6k tok/s (KV 30-40%). The token-id post-chain readouts do not hit the prefix cache (hit rate 0-4%; prompt throughput 2-5k tok/s): for this hybrid model vLLM 0.32 does not keep a finished request's generated blocks reusable. The readouts stay exact (checked at 07:07) but cost a full prefill, now interleaved with decoding (decode drops to ~2.7k tok/s while they run), so the saving expected at 02:53 did not materialise.

08:48 Correction. "Zero loops" for the joint reasoner held only for the first ~2,600 items. On 4,237 items both models have finished, base looped 37 times and joint 25 (OpenDecisions 16 vs 17, LocalLLaMA 9 vs 0 so far). Same items: reasoning 86.9 -> 87.4%, readout = own reasoning 65.3 -> 74.7%, median chain 1,402 -> 1,184 tokens.

08:40 Dashboard published (Mercity design language), rebuilt from the run files and republished each update cycle.

V2

10:09 V2 generation started in runs/v2 (256 sequences, 0.95 GPU memory, no speculative decoding by decision); KV cache 1.89M tokens; ~4.8-5.5k tok/s early. V2 pool: 41,994 readout questions (18,000 procedural, 23,994 tasksource from 717 sources, 3,292 of them from 25 soft-label sources capped at 150), dev slice 999.

10:20 Repetition filter retuned: a "line repeated 3+ times" rule would drop 2,388 healthy chains (rule quotes, per-record enumeration). No closed chain contains a true loop; about a dozen near-loops repeat a line 20+ times ("If A is self-contradictory, B follows." x240). The joint model's extra eval loops therefore come from training, not from the data; RL's loop penalty is the fix.

10:45 Chain audit (50 shortest-correct training chains, 25 procedural + 25 tasksource, checked against the evidence; runs/v2/data/chain_audit_50.jsonl): 39 sound, 9 minor slips off the decisive step, 2 flawed (n1 misreads the prior rule of policy_under_uncertainty; n28 guesses a MuTual pronoun and claims "I found a similar example online"), 0 uncheckable. Both flawed chains had 4/4 correct samples. Recurring habits: invented look-ups ("Wait, I found similar examples online"), dataset-convention guessing, repeated "final check" passes, '.cw' before </think> (9/50). Decisions: no judge model (4% flawed); drop chains with invented look-ups (precision checked on 10 random matches, all genuine; ~6% of candidate chains, almost all tasksource); strip '.cw'. Chains that merely name a dataset are kept.

11:20 OpenRouter teacher (user's key, $7): Qwen3.5-35B-A3B (:nitro; fp4 providers excluded, require_parameters, price cap $2/M out). Pilot 6 x 2: 12/12 closed, 9/12 correct, chains ~same length as the 4B's, raw reasoning returned in message.reasoning. Full run takes tasksource items from the back of the pool (local generation works from the front).

11:33 Teacher done: 662 items x 4 chains, $6.60, 11.5 min (Alibaba 1,981 / Parasail 667 requests), 2,645/2,648 closed, 72% correct. Per item correct-of-4: 0:21% 1:4% 2:4% 3:6% 4:64%; the 4B on its own tasksource items: 0:21% 1:5% 2:5% 3:8% 4:61%. A 9x larger sibling misses the same share, so 0/4 items are mostly label noise or ill-posed: of 5 read, 3 ambiguous gold (xglue relevance Bad vs Good, dynasent positive vs mixed, pragmeval premise type), 1 near impossible (TuringBench: which of 20 generators wrote this), 1 ordinal scale shown shuffled (relevance levels are not marked score, a prep issue to fix later). Teacher chains are shorter (median 1,083 vs 1,567 tokens) and never repetitive (0.0% vs 1.4%), but invent look-ups more (17% of samples). 835 training chains from 454 items survive the filters.

11:40 Speed benchmark written (bench_speed.py, runs first in run_v2.sh). Smoke on 0.8B in the GPU's 4.5 GB slack: no-checkpoint losses bit-identical to checkpointed; Liger off by 0.6% loss / 6.5% gradient vs HF bf16 -> equivalence is now judged against an fp32 reference (a setup may not be more than 1.5x further from fp32 than the current one). Sweep shortened to 1,500 questions / 500 dev, ties to the lower LR. Invented-recall filter widened (+1.8% of chains).

11:48 Pipeline re-read: the speed benchmark is now optional (a crash leaves the current setup and the pipeline goes on; tested in isolation). Disk: /workspace had 29 GB free and V2 writes 27 GB (three merged 4B checkpoints plus adapters and predictions). Moved the three v1 merged checkpoints (24 GB, rebuildable from their adapters in ~2 min each) to the container disk under /root/offload_checkpoints, symlinked in place: 52 GB free. Benchmark scratch (1.5 GB of gradients) goes to /root/bench_tmp, not results/ (which syncs to the public bucket).

11:52 RL plan checked against data and TRL 1.17.1 (every config field exists). The 658 RL-eligible items (1-3 of the 4B's 4 chains correct) have long chains: median 5,280 tokens, mean 6,985, 21% past 8,192, p95 30K (loops). At 32 prompts x 8 per step that is ~1.3M generated tokens a step, 100 steps ~15 h, not 4. Now: rounds of 128 chains (16 x 8) feeding 4 optimizer steps of 32 (TRL steps_per_generation; its importance-sampling correction covers the lag), one chain per forward (an 8K chain's full-vocabulary logits are ~4 GB in bf16), vLLM at 0.5 of the GPU: ~1.6 min/step, 120 steps ~3.2 h, each eligible prompt used at most once. --model must be a text-only checkpoint (TRL pushes weights into vLLM by name; the hub repo is the multimodal layout) and rl_grpo.py now refuses anything else; a text-only Qwen3.5-0.8B copy for the feasibility test is in checkpoints/qwen3.5-0.8b-text. Teacher records are left out of RL prompt selection (their 1-3 of 4 says nothing about the 4B's uncertainty).

12:00 Dashboard: V2 section (stages, chain sources local vs teacher, benchmark, sweep, V2 joint curves, results). Plan doc: Phase A/B steps updated with the teacher chains, benchmark, short sweep and RL round structure.

12:25 Chains with answers, all sources (no outside reasoning datasets used): 6,377 questions, 25,508 chains, 25,232 with an answer, 20,660 correct; 9,617 chains on 5,064 questions survive the training filters (4B v1: 4,973 / 2,611; 4B V2 so far: 3,809 / 1,999; teacher: 835 / 454). RL selection (rl_select.py, rules only): 771 mixed, 645 kept (233 exact, 359 hard-label with >= 2 of 4, 53 soft-label). Reward unit check: +1, -1, +0.2 (60/40 split), -1.5 (open).

13:31 V2 generation ended (deadline 13:09 + drain); gen.jsonl 7,333 records after the teacher merge. 13:45 Speed benchmark (results/speed_bench.json): A (current) 3,023 tok/s; B Liger 3,368 (+11%) but loss error vs fp32 1.49% against A's 0.64% (limit 0.97%) -> not equivalent (its gradient error is lower than A's, 0.287 vs 0.331; the rule is the rule); C no-checkpointing <= 8K 4,474 (+48%), equivalent, but 74.9 GB peak (limit 72 GB) -> out; D 5,205 (+72%) fails on B's loss error. Chosen: A, no flags. V2 joint estimate ~51.7M tokens vs ~53M that fit in 300 min at 3,000 tok/s: fits, just. Sweep 1e-4 (1,500 questions, 17 steps, 4.4 min): dev acc 0.614, NLL 0.880, ECE 0.091.

14:00 Sweep: 1e-4 acc 0.614 / NLL 0.880 / ECE 0.091; 2e-4 0.650 / 0.828 / 0.055; 3e-4 0.638 / 0.870 / 0.056 -> lr 2e-4. User asked for config C (+48%) despite the 72 GB line (deviation #26): pipeline stopped between stages, train.py gained an out-of-memory redo (checkpointed, gradients restored; forced-OOM test on the 0.8B passed), restarted 14:02:54 with --micro-tokens 8192 --ckpt-above 8192 and expandable segments. (The stop helper's pkill -f matched its own command line and exited 144 after the stop had already worked; nothing else was hit.)

16:49 V2 joint done: 1,309/1,309 steps, 47.8M tokens, 2 h 46 min at ~4,950 tok/s, peak 71.5 GiB, 0 out-of-memory redos. Held-out dev (999 questions: 504 procedural, 495 tasksource): acc 0.813, NLL 0.504, ECE 0.023 (sweep 2e-4 on 500 of them: 0.650 / 0.828 / 0.055). Training readout loss settled ~0.54 (v1 final 0.87); chain loss ended 0.22 (start 0.235). Overlap check: dev ids are disjoint from the pool, but 61/999 dev items (56 procedural) are exact duplicates of a pool item (same evidence, criterion, options; procedural generators repeat small instances), so dev acc without them is >= 0.801 even if all 61 were right. Eval: eval_proc has 50/526 (9.5%) exact duplicates of a V2 pool item (v1's trained items: 36, 6.8%); eval_ts, OpenDecisions and LocalLLaMA have none. Procedural eval will be reported with and without those items for every model.

18:30 Thinking-mode eval (V2 joint) at 1,901 / 2,013 (1,498 eval subset + 515 calibration), parse 97.3%, 52 looped so far (v1's eval run ended at 96.9% / 104 of 3,393). On the first 1,058 finished (short-biased): base 89.3, v1 89.9, V2 90.3. Chain openings ("Thinking Process:" vs "The user ...") split ~50/50 in V2, v1 and the 4B's own generation alike, so no teacher style shift. RL data (deviation #27): rl_fresh.py built 6,000 never-trained tasksource candidates in 21 s (3,000 from 53 reasoning sources, 3,000 from 651 others; 389 skipped as content seen in pool/held); rl_select.py now screens them with the V2 readout (gold 0.15-0.85) and mixes 288 fresh (192 reasoning) + 192 in-domain; rl_grpo.py takes them via --fresh-pool. Offline check: 480 prompts built, 288 fresh.

18:43 V2 thinking mode, full eval subset (1,498, same items as v1): 81.2% vs v1 84.4%, base 84.0 (proc 94.7 / 97.3, ts 70.3 / 73.7, LocalLLaMA 66.4 / 70.9, OD 94.2 / 96.4). The regression is the loops: V2 looped (hit the 32K cap, no answer) on 5.1% vs v1 1.4%, base 0.9%; v1 answered 49 of V2's 76 looped items correctly (3.3 points, the whole gap); on items neither model looped, v1 86.5 vs V2 85.7 (within noise). Loops are re-verification cycles: the chain states its answer ("The answer is B. Okay, I'm done.") then "Wait, I should check ..." repeats; 67 of 76 end in an exact repeat. Median chain tokens 1,340 vs 1,323. Correction: the 18:03 read ("thinking mode held up", 90.3 vs 89.9) used the first-finished items, which excludes loops by construction; and flat training chain loss could not show this (it scores imitation of closed chains, not whether sampling stops). RL's -1.5 for an open chain at 8,192 tokens and the length penalty above 4,096 target exactly this; post-RL loop rate is now the first gate.

19:00 Phase A finished (pipeline exit 0). metrics_v2: v2_joint readout all 0.727 (v1 0.695), OD Capability 67.3 (66.7), ECE 0.033 (v1_gold 0.019 is still the best calibrated); agreement 78.3% (75.7), disagreement AUROC 0.839 (0.780); routing on the 1,498 reasoning items: readout 76.0 -> 81.8 at f = 0.3 (v1 70.3 -> 78.0), 1,749 mean tokens (v1 1,166; loops cost 32K each), gap recovered 0.70 (v1 0.52). Ablations on eval: abl_gold 71.7 / ECE 0.031, abl_mix 70.5 / ECE 0.047 (mix +3.1 procedural, -1.6 OD, -2.3 LocalLLaMA): votes hurt out of distribution, the opposite of the in-distribution dev reading. Ablation checkpoints uploaded to run2_v2/checkpoints/ (sizes verified) and deleted locally. 19:05 RL smoke test on Qwen3.5-0.8B text-only (4 steps, 16 chains/step, 1,024-token cap): vLLM colocate, reward, backward, LoRA save all fine; 30 s/step; exit 0. 19:10 rl_select.py with V2 readout: in-domain 592 kept (167 already sure, 5 sure-wrong labels dropped); fresh 3,129 of 6,000 in the 0.15-0.85 band (2,322 sure right, 549 sure wrong or noisy). Final 480 = 192 fresh reasoning + 96 fresh general + 192 in-domain. 19:12 RL on the 4B started: rl_grpo.py --model checkpoints/v2_joint_merged, 120 steps, 128 chains per generation round (16 prompts x 8) feeding 4 optimizer steps of 32, 8,192-token cap, lr 1e-5, vLLM at 0.5.

19:26 RL round 1 (sequence_mask IS): reward -0.42, 26.6% of chains unfinished at 8,192, mean 4,458 tokens, IS ratio mean 0.55 / min 0, round time 537 s. Stopped and restarted with token_truncate IS (deviation #28). Round time means 120 steps would take ~5-6 h rather than 3.2 h; the LR is constant after 5 warm-up steps and checkpoints land every 20 steps, so the run can stop at any checkpoint without a schedule mismatch.

20:43 RL step 22 (6 rounds, 96 questions; checkpoint-20 saved). Per round (reward / unfinished at 8,192 / mean tokens): -0.42/27%/4,458, -0.11/23%/4,188, +0.64/5%/2,702, +0.31/11%/2,869, +0.31/12%/3,745, +0.11/20%/4,538. Rounds draw different questions, so per question: share right ~ readout p_gold + round gives +0.044 per round (se 0.022), unfinished -0.016 per round (se 0.015), reward +0.096 per round (se 0.051). Questions in the mix as planned (40 fresh / 24 in-domain in rounds 1-4). Loops in RL chains are indecision cycles between two options ("Wait, I'll consider ... B. Okay, I'll choose C."); hardest families: StepGame/SpaRTQA spatial puzzles and procedural event_triggered_execution (the family V2 looped on in the eval). Entropy 0.14-0.17, no collapse.

21:03 RL step 28 (7 rounds, 111 questions). Difficulty-adjusted per round: share right +0.028 (se 0.018), unfinished -0.011 (se 0.012). Finished chains: length flat (log chars +0.009 per round, se 0.015, adjusted for readout p_gold and prompt length), but "Wait" per finished chain falls 10.2 -> 6.1 (-0.41 per round, se 0.16). Within a question, right finished chains are shorter than wrong ones (median 12.5K vs 14.9K chars). Signal: 40-50% of questions split right/wrong (in-domain 60%, fresh reasoning 31%). By family (first vs second half of rounds): logic/rules 46% -> 77% right, spatial 67% -> 61%, multilingual 50% -> 44% (small n). Dashboard: RL section gains "Wait" per finished chain and share right among finished chains.

21:23 RL step 35 (9 rounds, 142 questions). Difficulty-adjusted per round: share right +0.014 (se 0.012; was +0.044 at 6 rounds), unfinished -0.012 (se 0.008), finished length +1.5% (se 1.0%), "Wait" per finished chain -0.23 (se 0.11). The early estimates shrank as rounds accumulated; the decisiveness effect is the only one near 2 se.

21:29 Post-RL eval armed: tmux stop60 waits for checkpoint-60 (trainer_state + adapter), stops the RL session by name, then runs runs/v2/run_rl_eval.sh (merge checkpoint-60 into V2 joint -> checkpoints/rl60_merged; evaluate.py predict exactly as V2: readout on all eval items, thinking mode on the 1,498 subset + calibration, latency; metrics_v2.py with rl60 beside base / v1 / V2 -> results/metrics_rl.json). Expected: stop ~22:40, results ~23:50 UTC.

21:35 RL rate change (deviation #29, user's call): tmux switch40 waits for checkpoint-40, stops the 1e-5 run by session name, continues 20 steps at 2.5e-5 from the step-40 adapter (fresh AdamW, the 320 questions not yet used, --exclude-seen tested offline: 480 -> 320), then runs run_rl_eval.sh on checkpoints/rl_grpo_b/final as rl60. Round 10 (steps 37-40): reward -0.35, 23% unfinished, 4,950 tokens (a hard batch).

21:44 Phase b first launch (21:40) used --seed 19, which reshuffles option orders, so --exclude-seen matched only 54 of the 160 used prompts by text; stopped after 3.5 min (still loading; log kept as rl_grpo_b_seed19_aborted.log) and relaunched with --seed 17: 160 excluded, 320 left. tmux phaseb runs it and then the post-RL eval.

22:20 RL step 50 (13 rounds, 205 questions; phase-b checkpoint-10 = step 50 saved). Difficulty-adjusted per round: share right +0.001 (se 0.007), unfinished -0.004 (se 0.004), finished length flat, "Wait" per finished chain -0.22 (se 0.06; 10.2 -> 5.2). Unfinished by family, first vs second half of rounds: procedural 46% -> 35%, logic 23% -> 13%, spatial 17% -> 9%, other tasksource 8% -> 12%. Correction: the 21:00 "logic 46% -> 77%" did not hold (68% -> 64% with 13 rounds and babi/lsat/entailment counted as logic). Late third of rounds 4 points below average on share right after adjustment (1 se). Entropy 0.13 at 2.5e-5 (0.15-0.17 at 1e-5); no answer collapse: distinct answers per question 1.31 (phase a) vs 1.50 (phase b), letter shares A/B/C 35/40/9% vs 23/40/21%. Entropy is not used as a stop rule.

22:28 Checkpoint-40 eval armed (tmux evalrl40: waits for the step-60 eval driver, then run_rl_eval.sh with CKPT=checkpoint-40 NAME=rl40); metrics_rl.json now takes every evaluated RL model. Dashboard pipeline section: view switch (V2 + RL / v1 / both) for the Gantt and stage table, and an "Up next" table from runs/v2/results/upcoming.json.

22:40 User going to sleep (back ~05:30-06:30 UTC); asked to start the 2B regardless of RL (deviation #30). runs/v2_2b/run_2b.sh: probe Qwen3.5-2B + MiniCPM5-2B -> pick (rule in the script) -> 2-step smoke (falls back to Qwen) -> sweep -> V2 joint on V2 data -> merge -> base + trained eval -> metrics_2b.json; tmux night chains it after the rl60 eval, then the rl40 eval. MiniCPM5-2B checked on CPU: LlamaForCausalLM, 42 layers, 131K context, Qwen-style template, A-Z single tokens; its template writes "", so vllm_utils now strips a leading BOS before vLLM adds its own (no change for Qwen, whose tokenizer has no BOS). stage() in run_2b.sh returns the command's status so a failure is never marked done.

23:03 RL step 60 done 22:47 (RL_EXIT 0), merged into checkpoints/rl60_merged (Qwen3_5ForCausalLM). rl60 fast readout on eval: all 72.5% (V2 72.7), proc 89.5 (89.4), ts 72.3 (71.8), OD 73.9 (74.2), LocalLLaMA 66.3 (66.7); ECE 0.030 (0.033), NLL 0.672 (0.669): RL left fast mode where it was, so no readout re-alignment is needed. Thinking eval running: 934 of 2,013 finished at 8 min (V2 took ~20 min for 1,055), median chain 800 tokens so far; early finishers exclude loops by construction, so no accuracy or loop read until it completes.

23:20 rl60 thinking mode, the same 1,498 items: 83.1% (V2 81.2, v1 84.4, base 84.0); looped 0.2% (3 items; V2 5.1%, v1 1.4%, base 0.9%); median chain 1,096 tokens (V2 1,340), mean 1,802 (V2 3,748: loops gone, half the compute); eval finished in 23 min (V2 57). By set: proc 97.0 (V2 94.7), ts 72.0 (70.3), LocalLLaMA 68.7 (66.4), OD 95.8 (94.2). Paired: RL right / V2 wrong 91, the reverse 62. On V2's 76 looped items RL answers 47 right and loops on 1. On items neither looped, V2 85.5 vs RL 84.4: RL fixed the loops and gave up ~1 point where V2 already finished. RL fast readout 72.5 (V2 72.7): no re-alignment needed. Decision on more RL: not tonight (the loop fix is in; in-training accuracy was flat; the small loss on finished items argues for a better setup rather than more steps of this one); the GPU goes to Jevals and the 2B, then the rl40 eval separates the two learning rates (after_2b.sh). Jevals (jevals.com, suite 0.1.0): 3 boards x 300 items rebuilt from the source datasets and matched to every published state hash (banking77 state key "message", pubmedqa "context"); jevals_eval.py scores with their rules (Brier / RPS Decision Score vs label prior, accuracy, 10-bin ECE, 2,000-sample item bootstrap). Our pool holds 40 Banking77 and 40 HelpSteer2-helpfulness train-split items (exact text overlap with the Jevals items: 0 and 2 of 300); PubMedQA is unseen.

23:29 Jevals thinking mode added (jevals_eval.py think): one chain per item, then the readout after the chain, post-chain temperatures from the model's own calibration chains; Banking77 reasons over the fast readout's 24-option shortlist (gold outside it = wrong); an unfinished chain falls back to the fast readout. Overnight driver reordered (tmux night3): Jevals fast for the 4B models -> Jevals thinking for rl60 -> 2B pipeline -> after_2b.sh (Jevals thinking for V2 / base / v1, Jevals for the 2B, rl40 eval). Base 4B fast, raw: Banking77 DS 56.2, PubMedQA 48.1, HelpSteer2 2.0; with our calibration temperatures 59.3 / 36.4 / 4.1 (the base's noul temperature, 2.41, does not transfer to PubMedQA).

23:34 User: use the idle GPU memory (vLLM reserved 0.85 = 68 GB). Scheduling knobs only, for runs not yet started: Jevals runs --gpu-mem 0.92, MAX_BATCHED_TOKENS 32768, thinking runs 384 concurrent chains (256 before); 2B probe GPU_MEM 0.92 + 32K batched tokens; 2B eval MAX_NUM_SEQS 256 + GPU_MEM 0.92 (the 2B's KV per token is smaller); rl40 eval MAX_NUM_SEQS 160 + GPU_MEM 0.92 (the 4B n=1 eval thrashed at 256 seqs / 0.90 with 99.5% KV, so a smaller step). (0.93 lowered to 0.92 after the first 0.92 run showed 79.4 of 81.9 GB in use: the tested setting, ~2.4 GB spare.)

23:45 Jevals fast mode (calibrated with our own calibration temperatures; raw in results/jevals.json). PubMedQA DS: rl60 58.6, V2 57.8, v1 50.3, base 36.4 (raw 48.1); board: Gemini 3.8 Flash 73.0, Jev 69.0, Qwen3.8 Flash 62.4, GLM-5.3 60.6, Mistral Medium 3.5 58.0, Mercury 55.7, DeepSeek V4.1 Flash 47.5 -> ours 5th of 8. Banking77: v1 61.5, rl60 61.3, V2 61.3, base 59.3; board 74.1 / 67.8 / 66.8 / 63.9 / 62.4 / 59.5 / 54.0 -> 6th of 8; ECE 2.1-2.8 points (V2, rl60), best or near best on the board (Jev 9.8). Gold in the 24-option shortlist: 98.7% (V2, rl60). HelpSteer2: all systems near zero, every interval includes 0 (v1 7.5, base 4.1, rl60 2.4 cal / 6.0 raw; Jev 9.2, GLM 7.8). Training effect on an unseen set: PubMedQA 48.1 (base, raw) -> 58.6 (rl60). Calibration temperatures hurt HelpSteer2 for V2/rl60 (6.0 raw -> 2.4) and the base on PubMedQA; raw vs calibrated reported side by side.

00:00 Jevals thinking mode, rl60 (one chain per item, readout after the chain, post-chain temperatures from our calibration chains: choice 2.15, noul 1.94, score 1.81): PubMedQA DS 63.5 [51.2, 74.8], accuracy 90.0% (fast 86.3%), -> would sit 3rd (Gemini 73.0, Jev 69.0, Qwen3.8 Flash 62.4); Banking77 DS 59.3, accuracy 73.3% (fast 73.1%), ECE 13.4 (fast 2.8); HelpSteer2 DS -20.1, accuracy 41.3%, ECE 37.8. Chains closed 896/900, medians 728 / 802 / 1,271 tokens. Thinking lifts accuracy where the task is reading-and-reasoning (PubMedQA) and not where it is recognition (Banking77); the post-chain probabilities are overconfident, which the Brier-based Decision Score punishes (raw: Banking77 49.5, HelpSteer2 -43.0). This is the known near-certain post-chain readout (TODO "Later / ideas"). 2B probe started 23:59:53.

00:07 Full bucket sync (user request; keys excluded): code/ (incl. ops/ driver scripts), run1/ (results with raw chains, data, data_qwen3, setup logs, logs), run2_v2/ (results with raw chains and adapters, data, logs, RL checkpoints rl_grpo and rl_grpo_b with completions and optimizer state, merged v2_joint and rl60), jevals/ (suite, items, runs, score table), dashboard/ (page + data), README.md rewritten for the layout. Sizes verified: both merged models 8,411,558,400 bytes, RL adapters 243,848,336 bytes, bucket = local. Key scan (both key strings and sk-or- / hf_ token patterns) over the project, scratchpad and the code copy: no hits; the key files stay in /workspace/.cache.

00:39 2B pick (rule in run_2b.sh): MiniCPM5-2B. Probes (v1 eval readout mean / first-chain accuracy on 128 pool items / parse / looped of 512 / median tokens): MiniCPM5-2B 48.3% / 80.5% / 99.6% / 2 / 532; Qwen3.5-2B 49.7% / 66.4% / 80.7% / 98 / 3,949 (fails parse >= 0.9); Qwen3.5-4B for reference 59.4% / 82.0% / 99.4% / 3 / 1,801. Smoke test passed (2 steps + dev). Sweep on 1,500 questions (dev 500): 1e-4 53.6% / NLL 1.076; 2e-4 57.0% / 1.039; 3e-4 58.6% / 1.058 -> 2e-4 by the NLL rule. 2B joint started 00:34: 52,968 examples, 46.0M MiniCPM tokens, 1,265 steps, ~7,900 tok/s, ~100 min, 49.6 GB.

01:10 after_2b.sh extended (runs/v2_2b/twob_extra.sh): the untrained MiniCPM5-2B's thinking mode on the same 1,498 items (+ calibration) as reason_base2b; then, only if V2-joint training lowered the 2B's thinking accuracy by more than 1 point on those items, a readout-only 2B on the same 42K questions (lr 2e-4, 90-min box), merged and evaluated (readout + thinking), plus its Jevals run. Jevals thinking for the 2B base added. Reason: the 2B's chain loss stays at ~0.44 (4B 0.25) while MiniCPM's native thinking is already strong (80.5% first chain, 532 tokens median), so copying the 4B's and the teacher's longer chains could cost it.

02:14 2B joint (MiniCPM5-2B, V2 recipe) done: 1,265 steps, 46.0M tokens, ~95 min at ~8,045 tok/s, peak 49 GB, 0 OOM redos. Held-out dev (999): acc 0.742, NLL 0.617, ECE 0.039 (4B V2 0.813 / 0.504 / 0.023; 2B sweep at 1,500 questions 0.570). My 02:03 guess (76-78%) was high: the last-stretch training match (80%) was a lucky slice.

02:23 2B fast readout on the eval set (5,442): base MiniCPM5-2B 50.2% (proc 39.7, ts 50.8, OD 55.4, LocalLLaMA 46.6; ECE 0.049) -> 2B V2 recipe 65.1% (85.7 / 63.1 / 65.8 / 59.3; ECE 0.089, NLL 0.857). 4B for reference: V2 72.7%, v1 69.5%. +15 points from training, most on procedural (+46); calibration after per-type temperatures is worse than the 4B's (0.089 vs 0.033). Thinking eval running (median 740 tokens, 0 loops in the first 323).

03:03 2B (V2 recipe) thinking eval at 1,512 / 2,013 after 42 min: 58 looped so far (3.8% of finished), parse 96.0%, median 954 tokens, max 32,658 (the untrained MiniCPM5-2B probe: 2 of 512 looped, median 532). The V2 chain training (shortest correct 4B/teacher chains) brings back the loops in the 2B as it did in the 4B (V2 5.1%). Tail of ~256 long chains: done ~03:50.

03:24 2B eval restarted at 96 concurrent chains (was 256): my 23:34 change assumed the 2B's KV per token was smaller than the 4B's; it is larger (MiniCPM5-2B has full attention in all 42 layers, Qwen3.5 in 8 of 32), so at 256 the cache sat at 99.9% with ~90 running and ~160 waiting, and throughput fell to ~1,480 tok/s while long looping chains were preempted and recomputed. evaluate.py resumes from the raw file (1,622 of 2,013 chains kept); the base-2B and readout-only 2B evals and the 2B Jevals thinking runs now also use 96.

04:03 2B (V2 recipe) thinking mode, the same 1,498 items: 71.7% (proc 86.7, ts 57.7, OD 81.0, LocalLLaMA 61.8), looped 7.4%, median 1,056 / mean 4,008 tokens; 77.4% on its non-looped items. 4B for reference: V2 + RL 83.1% (0.2% loops), V2 81.2% (5.1%), v1 84.4%, base 84.0%. The untrained 2B's thinking on these items runs next (twob_extra.sh); the probe had it at 80.5% on 128 pool items with 0.4% loops, so a drop above 1 point (which triggers the readout-only 2B) is likely.

04:18 Untrained MiniCPM5-2B thinking on the same 1,498 items: 79.4% (proc 92.7, ts 64.3, OD 91.5, LocalLLaMA 68.4), looped 0.1%, median 441 / mean 763 tokens. The V2 recipe cost it 7.7 points (71.7%), raised loops to 7.4% and mean tokens 5x (4,008). For scale, 4B V2 + RL: 83.1% at 1,802 mean tokens. The readout-only 2B (twob_extra.sh) is triggered (drop 7.68 > 1) and training since 04:18. Lesson for V3: train chains only for models whose own thinking is weaker than the chain source, and filter re-verification loops out of the training chains in any case. Jevals fast mode, 2B: PubMedQA DS 41.6 (base 7.9), Banking77 54.2 (base 31.9), HelpSteer2 -4.0 (base -6.3).

04:48 Extra evals (user: "find some good evals, run them properly, then report"). Candidates looked at:

• Jev Decision Index (HF Space multimodalart/jev-decision-index, v0.4.1): 37+ benchmarks, 110K requests incl. games and tool environments, own harness (github apolinario/decision-index). Too large for this window and needs its harness run as third-party code; not run.
• S1MB, System One Mosaic Benchmark (hotchpotch; blog "system-one-mosaic-benchmark"): English v1 = 137 typed benchmarks (Noul 59 / Choice 57 / Score 21), 14,009 cases, 26,269 judgments, public data (hotchpotch/S1MB-english-v1 @ 10bcb8c6) and 218 published model folders (hotchpotch/s1mb-result) incl. many 0.8B-9B Jev reproductions. Chosen. Running the S1MB evaluator (uv sync + its CLI) was refused by the permission check (third-party code); instead s1mb_eval.py reads its data and published results as data and reimplements SCORING.md, and checks itself by re-scoring published models from their saved predictions against their saved metrics. Items built: 137 benchmarks, 26,269 judgments, counts equal to the benchmark definitions; 904 judgments have 36-151 options (tournament readout). Queue (tmux s1q): after the overnight queue, S1MB for rl60, base 4B, v1, V2, 2B readout-only, 2B joint, 2B base.

05:00 2B readout-only (no chain training, same 42K questions, lr 2e-4): 549 steps, 20.8M tokens, 41 min. Held-out dev (999): acc 0.760, NLL 0.600, ECE 0.041, better than the 2B joint run (0.742 / 0.617 / 0.039): for the 2B the chain items did not even help the fast readout. Merge + eval (readout + thinking) next.

05:12 S1MB scorer check: re-scoring 3 published models from their saved predictions (16 benchmarks each, every task) matches their saved baseline-adjusted scores to <= 6e-16. Published data: 213 model folders, 211 complete measured models form the cohort (all 137 benchmarks eligible for Borda). Current top: Darwin-27B-ZTC-v9 Borda 89.4 / Task Avg 72.2; best 4B KnowLine-4B-Gen5 80.4 / 67.0 (20th); 2B: KnowLine-2B-Gen1 57.0 / 44.9, unee-2b 53.1 / 44.4, Open-Jev-2B 39.3 / 34.6.

05:30 2B readout-only eval. Fast readout (5,442): 66.0% (2B joint 65.1, untrained 50.2), ECE 0.074. Thinking (1,498): 70.8%, looped 2.1%, median 127 / mean 1,343 tokens: readout-only training also costs the 2B its thinking (untrained 79.4%), by teaching it to answer at once; the shared LoRA carries that into thinking mode. Hybrid, computed from saved predictions (no GPU): the readout-only 2B's fast readout routed to the untrained 2B's thinking (adapter off for thinking): 30% routed 75.6% at 253 thinking tokens per question, 50% 78.0% at 411, always 79.4% at 763; same routing for 4B V2 + RL: 81.9% at 742 / 82.3% at 1,152 / 83.1% at 1,802. Routing here = lowest readout margin to thinking, the thinking answer is the parsed letter (metrics_v2 uses the post-chain readout; same ranking).

06:04 V2's Jevals thinking run is slow (HelpSteer2: 238 of 300 chains closed, median 3,181 tokens; V2 loops). To start S1MB sooner, three low-value Jevals thinking runs are skipped (base 4B, v1, 2B V2-recipe; tmux skiplv kills them on start, logs/skipped_runs.log): their story is known (thinking-mode probabilities are overconfident under Brier). Kept: V2 (thinking vs RL) and the untrained 2B (the hybrid 2B's thinking path).

07:03 rl40 (checkpoint-40, 1e-5 only) thinking eval: 2,009 of 2,013 done, 45 looped (2.2%), max 32,636 tokens. Fast readout 72.7% (V2 72.7, rl60 72.5). Correction of my 06:43 read ("0 loops so far, most of the fix came from the first 40 steps"): the loops simply had not finished yet, the same first-finished bias as on 10-31 18:03. Loops: V2 5.1% -> rl40 ~2.2% -> rl60 0.2%, so the 2.5e-5 phase did a large part of the fix.

07:16 S1MB runs at ~20-25 min per model (prompt-bound readout, ~20K tok/s). Skipping two redundant runs (tmux skips1): V2 (fast readout = V2 + RL within 0.2 points) and the V2-recipe 2B (the readout-only 2B is better on every eval). S1MB models: rl60, base 4B, v1, 2B readout-only, 2B base.

08:06 S1MB done (5 models; 211 complete published models + ours; Borda over all 137 benchmarks). Task Avg / yes-no / choice / score / published models above in Borda order: V2 + RL 47.9 / 50.7 / 57.2 / 35.8 / 74; v1 43.5 / 50.1 / 52.3 / 28.0 / 100; 2B readout-only 37.3 / 42.3 / 45.7 / 24.0 / 123; Qwen3.5-4B 32.8 / 33.8 / 45.7 / 18.9 / 134; MiniCPM5-2B 18.1 / 22.9 / 29.5 / 2.0 / 162. Best Task Avg superion_s1 74.4, best 4B KnowLine-4B-Gen5 67.0, median 41.2.

• Base -> V2 + RL: 105 of 137 benchmarks up, 16 down. Largest gains on Open-Jev control tasks (drone +1.0, entity alignment +0.83, invoice processing +0.65); drops mostly yes/no (customer-control -0.28, SciTail -0.18, CanTTalk -0.14, Qasper -0.12).
• Cause of the yes/no drops: over all 12,357 yes/no judgments (target yes mass 27.5%) the base model predicts yes on 18.4% (true recall 0.44, false recall 0.91), V2 + RL on 31.6% (0.70 / 0.83). Training removed a "no" bias and overshoots on balanced sets.
• v1 carries most of the gain (+10.7 of +15.1); V2 + RL adds choice and score, not yes/no.
• Temperatures (from our calibration items) only touch Score; uncalibrated Task Avg V2 + RL 46.9, base 33.8.
• Correction to my 08:03 chat table: published models above the base 4B is 134 and above the 2B readout-only 123 (I wrote 135 / 124, an off-by-one when subtracting our own rows). Report: results/FINAL_REPORT.md and the Claude Docs report https://claude.ai/artifact/RUdY6E2PZgStqNURLGT8g8

08:30-11:20 Extra studies (user: "do whatever needs to be done", subagents allowed):

• Bootstrap CIs and H1-H4 for the final models (subagent; results/final_stats.md): all 65 point estimates reproduced; two report claims corrected (vote-mixed targets lowered readout accuracy -1.2 [-2.1, -0.3]; post-chain ECE rise after RL within noise). H2 confirmed, H3 met by rl60 only, H1 not confirmed, H4 79.5% [64, 97] undecided.
• S1MB training-overlap check: 104 of ~18,700 cases share most text with training (88 corr2cause templates); unseen-source benchmarks gained more (+17.4 vs +8.4, diff +9.0 [3.4, 14.5]).
• Yes/no offset fitted on 109 calibration items: our eval BA 0.784 -> 0.836 (rl60), S1MB yes/no 50.7 -> 49.9 (24 up / 27 down).
• 114 chains read (subagent; results/examples_analysis.md): fixes are computation; margin routing spends its budget on LocalLLaMA; dataset-guessing chains right ~55% vs ~90%; 55 of LocalLLaMA's 141 thinking errors are near-ties.
• Report figures (two subagents): only 49% of RL questions gave GRPO a right/wrong signal; shortest-correct selection cut 4B tasksource chains 1,662 -> 734 median tokens.
• Routed S1MB (s1mb_think.py; rl60 fast + rl60 thinking on the lowest-margin 30% = 7,881 judgments, 24.8M tokens, 2 h 46 min; 158 unfinished kept the fast answer): Task Avg 47.9 -> 55.2 (+7.3 [4.9, 9.9]), yes/no +7.6, choice +10.4, score +4.0 (n.s.); Borda: published above 74 -> 50. Gains: Snake, logical entailment, temporal NLI, tic-tac-toe, AQuA-RAT; losses: ETHICS, search-relevance score tasks. The 2B hybrid's routed run was skipped (time).
• PDF report: results/report/decision_model_report.pdf (23 pages, 23 figures; source REPORT.md, analysis/build_report_pdf.py).