Commit History
-
Phase 8: B4-final (last 2 living-doc stale counts) + final verification disposition
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)- B4-final: USER_GUIDE.md:678 + INTEGRATION_RECIPES.md:926 '115-test suite' → 266/62 (the last current-framed stale counts the independent verifier found outside the earlier grep scope; dated-historical + _archive mentions correctly left). - Recorded the Phase-8 disposition: full suite 381 passed/65 skipped/0 failed; independent verifier confirms B1-E3 resolved; submodule-export design note; final backlog disposition (zero actionable-on-host items open; remainder is user-gated GPU/token or tracked pre-existing lint debt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>c59d939 -
Wave 1: fix 8 failing tests + unblock Docker E2E + dep/doc debt
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Backlog-resolution Wave 1 (branch backlog/goal-resolution-2026-06). Bugs fixed: - B1 (P0): generate spikes/007 synthetic_session_with_error.jsonl fixture (never committed — .gitignore *.jsonl whitelisted the sibling but not this one) → the 8 failing test_trace_examples_adapter tests now pass. Added the missing .gitignore whitelist line. - B2 (P2): [dev] extra was un-installable on Apple Silicon (pulled Linux-only torchft-nightly). Dropped diloco from base [dev]; added [dev-full] for Linux. - B3 (P2): [serverless] extra now includes s3fs/boto3/kubernetes (needed for real S3 rendezvous + the EKSExecutor/SageMakerExecutor adapters). Newly unblocked: - D1 (…-245d): Docker substrate E2E — Docker now available on host; both gates (4-gate substrate inversion + cache-scrub-in-container) PASS. Long-standing "hardware-blocked" item closed. Doc/API debt: - B4: reconciled divergent test counts (115/176/210/232) to one canonical figure measured this session: 266 passed / 62 skipp…c11cf49 -
docs(reconcile): main-lags-master foot-gun is RESOLVED, not live
The refine brief predated the 2026-06-09 branch fix. main==master now and main is canonical, so "git checkout master after cloning" is stale/backwards. Reframe README + USER_GUIDE + OVERVIEW to "main is canonical & synced; fresh clone works as-is; ImportError = stale clone, git checkout main".d7e4b4e -
docs(wave1): correctness pass — channel-3 provenance, gap honesty, dead links
Propagate ADR-014 ground truth into the living docs and fix every dead relative link. Docs-only; no source files touched. Provenance (Fact A) — stop blurring the additive trace-replay-DPO channel into Cursor's recipe: - README v0.1 roadmap cell, HF_REPO_LAYOUT v0/v1 variant rows: reframe "Composer recipe" = channels 1 (Dr.GRPO) + 2 (SDPO) only; trace-replay is the framework's own addition (link ADR-014). Gap honesty (Fact D) — do not imply the full A1-A4 LMA ladder is runnable: - VISION_VALIDATION status banner + ALTERED_MINDS_TIE_IN Phase-3: only A1 (GRPO-only) has a real Modal runner; A2/A3/A4 are scaffold + plan-builder only, blocked on dataset construction. Real 8B run is additionally budget-gated. PO menu / strip_thinking / main-lag (Facts B, E, F): - VISION_VALIDATION: base RL objective is a selectable menu (default Dr.GRPO) per ADR-014, not hardcoded. - ALTERED_MINDS_TIE_IN: note SDPO requires strip_thinking=False on real traces (~67% of error-recovery turns are…20e3bd9
-
Wave 15: 4-angle multi-model self-critique caught 2 math BLOCKERs in primary loss kernels; fixed against upstream byte-for-byte + GSM8K example + ergonomics
PHASE 15a: 4 parallel adversarial reviewers, each with different framing to maximize independent-angle coverage: - MATH (Opus 4.7): cloned upstream OPSD/TAID/PRIME-RL/SimPO, did line-by-line diff. Found 2 BLOCKERs all 8+ prior reviewers missed. - TESTS (Opus 4.7): scrutinized 3 high-stakes test files for weak assertions. Found PRIME-RL parity test silently never runs; bit-exact test uses allclose; entropy-OPD test is pure smoke. - DOCS (Opus 4.7): audited 6 major docs + ADRs. Found test-count drift (77/107/124 vs 145), compose_loss kwarg drift, stale "Deferred to Wave 14" claim, PRIME-RL test count 10 vs 16. - USER-JOURNEY (Opus 4.7): walked "RL-finetune Qwen-7B on GSM8K" end-to-end. Scored framework 2.4/5 with Real-Task-Path = 1/5. #1 friction: NO GSM8K example anywhere. All 4 reports saved to /tmp/wave15_*.md. Headline: math reviewer's upstream-clone-and-diff approach caught what paper-level review missed. PHASE 15b: synthesized findings into 5-task fix scatter. PHA…e5add15 -
Wave 14: close every Wave 13 review finding + 4 documentation files; Wave 14b: real PRIME-RL parity + multi-process DiLoCo convergence
PHASE A: 4 parallel impl subagents closed every Wave 13 cross-model review item. T1 \u2014 compose_loss integration (closes W13 BLOCKER 2): - Added `dpo_variant: 'dpo'|'simpo'`, `sdpo_wrapper: 'none'|'taid'|'entropy_opd'`, `taid_schedule_step`, `taid_total_steps` plus 5 tuning kwargs - 11 new integration tests in composer_replication/tests/test_compose_loss_integration.py - Bit-exact reproduction of legacy compose_loss output when defaults preserved - All 38 spike-005 tests still pass T2 \u2014 replaysim DJ adapter reshape (closes W13 Suggestion 3): - _dpo_pair_to_dj_record now emits BOTH flat strings AND chat-messages for dual-shape compatibility with data-juicer's text ops - _dj_record_to_normalized round-trips both shapes - default.yaml fixed: text_keys plural \u2192 text_key singular (caught a related bug in the same op set) - Real data-juicer e2e test runs DefaultExecutor on a 3-record fixture - 15 tests (was 9) T3 \u2014 MockManager DiLoCo integration (closes W13 Suggest…d9dd3a5