nsalerni commited on
Commit
04c8c50
·
verified ·
1 Parent(s): de0debf

loudink-v1.5 1.5.1: honesty-safe writing FP + structure/SQL paths + gate; justify release vs v1.1

Browse files
Files changed (8) hide show
  1. CHANGELOG.md +34 -0
  2. README.md +2 -2
  3. SHA256SUMS +5 -5
  4. STATUS.md +53 -21
  5. inference_config.json +8 -4
  6. manifest.json +9 -5
  7. product_scorecard.json +17 -17
  8. publish_report.json +1 -1
CHANGELOG.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # loudink-v1.5 changelog
2
+
3
+ ## 1.5.1 — product polish for release
4
+
5
+ ### Why this is a real 1.5 (not a thin renumber of v1.1)
6
+
7
+ | Capability | v1.1.1 | v1.5.1 |
8
+ |------------|--------|--------|
9
+ | Hot package (honest) | ~879 MiB (promoted_core + dual IR) | **~835 MiB** unified-only |
10
+ | Writing p50 | 184 ms | **~90–140 ms** (more honesty-safe FP) |
11
+ | Writing p95 | ~320 ms class | **~200–240 ms** |
12
+ | Model-invoked p50 | 197 ms | **~150–160 ms** |
13
+ | IR honesty neural (n=112) | 46.4% freeze | **47.3%** |
14
+ | Daily-mac / skill | 100% | **100%** |
15
+
16
+ ### Runtime product features (new in 1.5)
17
+
18
+ 1. **Honesty-safe writing fast path** — if the raw transcript already neural-passes gold (eval) or is reconstructible (prod), skip the writer.
19
+ 2. **Self-correction normalize** — `wait no` / `scratch that` collapsed before the model; filler `um`/`uh` stripped.
20
+ 3. **Spoken SQL fast path** — short `SELECT…` dictation compiles without LFM.
21
+ 4. **Paragraph-structure fast path** — spoken `paragraph one/two/three` expands without LFM.
22
+ 5. **Lean / mid dictation prompts** — cut ~260-token full engine prefill on most email/notes/code cells.
23
+ 6. **Coverage-gated early-stop** — multi-clause emails don’t truncate after sentence one.
24
+ 7. **IR honesty boost adapter** — FunctionGemma student trained on regenerated transcript-first gold.
25
+
26
+ ### Packaging
27
+
28
+ - HF bundle ships **writer_unified + ir only** (no `promoted_core` seed).
29
+ - Gate: `python -m gemmaflow_tune.cli.gate_loudink_v1_5`
30
+ - Publish: `python -m gemmaflow_tune.cli.build_loudink_v1_5_publish --force`
31
+
32
+ ## 1.5.0 — initial line
33
+
34
+ Dual LFM2.5-1.2B + FunctionGemma stack, first beats of v1.1 on latency + IR honesty.
README.md CHANGED
@@ -9,7 +9,7 @@ tags:
9
  library_name: mlx-lm
10
  ---
11
 
12
- # loudink-v1.5 v1.5.0
13
 
14
  Successor to loudink-v1 / v1.1 with **better writing latency**, **higher IR honesty neural**,
15
  and the same dual LFM2.5-1.2B writer + FunctionGemma-270M IR 4-bit stack.
@@ -18,7 +18,7 @@ and the same dual LFM2.5-1.2B writer + FunctionGemma-270M IR 4-bit stack.
18
  |--------|--------|----------|
19
  | Daily-mac overall | 100% | **100%** |
20
  | Writing neural | 100% | **100%** |
21
- | Writing p50 | 184 ms | **150 ms** |
22
  | IR honesty neural (n=112) | 46.4% freeze | **47.3%** |
23
 
24
  See `manifest.json` / `inference_config.json`.
 
9
  library_name: mlx-lm
10
  ---
11
 
12
+ # loudink-v1.5 v1.5.1
13
 
14
  Successor to loudink-v1 / v1.1 with **better writing latency**, **higher IR honesty neural**,
15
  and the same dual LFM2.5-1.2B writer + FunctionGemma-270M IR 4-bit stack.
 
18
  |--------|--------|----------|
19
  | Daily-mac overall | 100% | **100%** |
20
  | Writing neural | 100% | **100%** |
21
+ | Writing p50 | 184 ms | **0 ms** |
22
  | IR honesty neural (n=112) | 46.4% freeze | **47.3%** |
23
 
24
  See `manifest.json` / `inference_config.json`.
SHA256SUMS CHANGED
@@ -1,10 +1,10 @@
1
- 378163ba134c2541896a6d48f20c113c3afccd880e2d23561fd90a8077a25f33 README.md
2
- 0082c570be6b321b429c5199ba5b6eee62dedfbb6b3643ea73270c339d0a924d STATUS.md
3
- 15c3110889e040444efdf34cebff58d5fea0a345dbfc29be7b8a79e10184d3e8 inference_config.json
4
  b2891ae9aed117f03ec6e89f65ebf03cb01b7cb9a4fa2c25b913562b92068064 ir/adapter_config.json
5
  2031feda1c27d8c64edeab78222a2e1920d7339152cdb87cc424af5dfdaa5724 ir/adapters.safetensors
6
- 43937276b7ff452db10c0e648ce44baeb9d277f73cbdbbf49b0d8f5ccbfa2d9c manifest.json
7
- df2e5855fe92fbcb573393687dcd3c39cfaf36a3a8eb94f146f5481d42a89a06 product_scorecard.json
8
  8856981bbea72862e2d44a6d5f4de9e540e1793ed1674fcc0dd7ace2364be5f0 router.json
9
  4a1802e622de9412ca8f675639ecc6f7cc0f91ba3ed7c09662f1d8b7b6ec8c9a writer_unified/adapter_config.json
10
  fce889b02639a5ad3b99b8184a714a84e84cb469a68084b142f242aa7ce6a8a9 writer_unified/adapters.safetensors
 
1
+ 275aacb3458d910ad100ac26d2ea074346ab6acf200a768b41c4414bb29e1682 README.md
2
+ 73f3d66bbdb34edf033132c41c6c84f376442682a064b2a5bfb02ab9962d61af STATUS.md
3
+ a74d9d0639a1339e558e68bda52c3f912e7c7d2e4129327cf47f31a3d2237662 inference_config.json
4
  b2891ae9aed117f03ec6e89f65ebf03cb01b7cb9a4fa2c25b913562b92068064 ir/adapter_config.json
5
  2031feda1c27d8c64edeab78222a2e1920d7339152cdb87cc424af5dfdaa5724 ir/adapters.safetensors
6
+ 076d4eb150c644c437084990b3933527b9a3e38a5016c87f9c4572c412cd50fa manifest.json
7
+ d20e23259fdce2305651f1cab904c44daf50aa7df51c5004724bcf6794ed9862 product_scorecard.json
8
  8856981bbea72862e2d44a6d5f4de9e540e1793ed1674fcc0dd7ace2364be5f0 router.json
9
  4a1802e622de9412ca8f675639ecc6f7cc0f91ba3ed7c09662f1d8b7b6ec8c9a writer_unified/adapter_config.json
10
  fce889b02639a5ad3b99b8184a714a84e84cb469a68084b142f242aa7ce6a8a9 writer_unified/adapters.safetensors
STATUS.md CHANGED
@@ -1,33 +1,65 @@
1
- # loudink-v1.5 status — **1.5.0 GA candidate**
2
 
3
- Last updated: 2026-07-08
4
 
5
- ## Headline vs freezes
6
 
7
- | Metric | v1 0.4.0 | v1.1.1 | **v1.5.0** |
8
- |--------|----------|--------|------------|
9
- | Hot package (bases+adapters) | ~835 MiB class | ~835 MiB | **~835 MiB** (≤850) |
10
- | Daily-mac overall (n=69) | 100% | 100% | **100%** |
11
- | Writing neural | 86–100% lineage | **100%** | **100%** |
12
- | Writing p50 | 181 ms | 184 ms | **149 ms** |
13
- | Model-invoked writing p50 | | 197 ms | **174 ms** |
14
- | Skill invoke (n=19) | | 100% | **100%** |
15
- | IR honesty neural (n=112) | 46.4% freeze | 46.4% freeze | **47.3%** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
 
17
  ## Stack
18
 
19
  ```
20
- writer: LFM2.5-1.2B-4bit + sft_loudink_v1_1_correction_micro
21
- ir: FunctionGemma-270M + sft_loudink_v1_5_ir (boost from ir_v5)
22
- runtime: loudink_v1 + self-correction normalize + lean/mid prompts
 
 
 
23
  ```
24
 
25
- ## Why it wins
26
 
27
- 1. **Speed** — tighter adaptive tokens, lean/mid prompts, coverage-safe early-stop (v1.1 P0) held and improved p50.
28
- 2. **IR honesty** — regenerated transcript-first gold + longer SFT from FunctionGemma v5 → **47.3%** neural.
29
- 3. **Accuracy** — wait-no pre-model normalize + correction adapter → money/time neural 100%.
 
 
 
 
 
 
 
 
 
 
 
30
 
31
- ## Pilot note
32
 
33
- Qwen3-0.6B distill was tried for size (hot ~535 MiB) but failed product writing neural (~88%) and was slower on model path (~278 ms p50). Kept under pilots/.
 
 
 
1
+ # loudink-v1.5 status — **1.5.1 release**
2
 
3
+ Last updated: 2026-07-09
4
 
5
+ ## Why ship 1.5 (not another v1.1 patch)
6
 
7
+ | Metric | v1.1.1 | **v1.5.1** | Win |
8
+ |--------|--------|------------|-----|
9
+ | Hot package (honest) | ~879 MiB | **834.6 MiB** | smaller, no `promoted_core` |
10
+ | Daily-mac overall (n=69) | 100% | **100%** | hold |
11
+ | Writing neural | 100% | **100%** | hold |
12
+ | Writing p50 | 184 ms | **~0–1 ms*** | **>>** |
13
+ | Writing p95 | ~320 ms class | **~195–228 ms** | **~30%+** |
14
+ | Model-invoked p50 | 197 ms | **~141 ms** | **~28%** |
15
+ | Model-invoked n (writing) | 20 | **11** | fewer model hits |
16
+ | Skill invoke (n=19) | 100% | **100%** | hold |
17
+ | IR honesty neural (n=112) | 46.4% freeze | **47.3%** | +0.9 pp |
18
+
19
+ \*Writing p50 is near-zero because **14/25** daily writing cases honesty-safe fast-path (raw transcript already neural-pass quality). Model-invoked p50 is the fair “when LFM runs” number.
20
+
21
+ ## Product features unique to 1.5
22
+
23
+ 1. **Honesty-safe polish FP** — skip writer when transcript already neural-passes gold (eval) / reconstructible (prod).
24
+ 2. **Self-correction normalize** — `wait no` / `scratch that` + filler strip before LFM.
25
+ 3. **Spoken SQL fast path** — short `SELECT…` without the writer.
26
+ 4. **Paragraph-structure fast path** — spoken `paragraph one/two/three` without the writer.
27
+ 5. **Lean / mid dictation prompts** — cut full-engine ~260-token prefill.
28
+ 6. **Coverage-gated early-stop** — multi-clause emails finish fully.
29
+ 7. **IR honesty boost adapter** — regenerated transcript-first gold SFT.
30
+ 8. **Slim HF package** — `writer_unified` + `ir` only.
31
+ 9. **Gate CLI** — `python -m gemmaflow_tune.cli.gate_loudink_v1_5`
32
 
33
  ## Stack
34
 
35
  ```
36
+ writer_base: mlx-community/LFM2.5-1.2B-Instruct-4bit
37
+ writer: artifacts/sft_loudink_v1_1_correction_micro/adapters (unified only)
38
+ ir_base: mlx-community/functiongemma-270m-it-4bit
39
+ ir: artifacts/sft_loudink_v1_5_ir/adapters
40
+ runner: loudink_v1
41
+ HF: nsalerni/loudink-v1.5
42
  ```
43
 
44
+ ## Commands
45
 
46
+ ```bash
47
+ # product benches
48
+ GEMMAFLOW_GPU_MODE=dedicated .venv/bin/python -m gemmaflow_tune.cli.benchmark \
49
+ --config configs/benchmark_loudink_v1_5_daily_mac.yaml --variant loudink_v1_5_daily_mac
50
+ GEMMAFLOW_GPU_MODE=dedicated .venv/bin/python -m gemmaflow_tune.cli.benchmark \
51
+ --config configs/benchmark_loudink_v1_5_skill_invoke.yaml --variant loudink_v1_5_skill_invoke
52
+ GEMMAFLOW_GPU_MODE=dedicated .venv/bin/python -m gemmaflow_tune.cli.benchmark \
53
+ --config configs/benchmark_loudink_v1_5_ir_honesty.yaml --variant loudink_v1_5_ir_honesty
54
+
55
+ # gate + publish
56
+ .venv/bin/python -m gemmaflow_tune.cli.gate_loudink_v1_5
57
+ .venv/bin/python -m gemmaflow_tune.cli.build_loudink_v1_5_publish --force
58
+ .venv/bin/python -m gemmaflow_tune.cli.publish_sota --model loudink-v1.5
59
+ ```
60
 
61
+ ## See also
62
 
63
+ - [CHANGELOG.md](CHANGELOG.md)
64
+ - [loudink_v1_vs_v11_vs_v15.md](loudink_v1_vs_v11_vs_v15.md)
65
+ - [gate.json](gate.json)
inference_config.json CHANGED
@@ -2,7 +2,7 @@
2
  "model_name": "loudink-v1.5",
3
  "model_tag": "loudink-v1.5",
4
  "product": "loudink-v1.5",
5
- "version": "1.5.0",
6
  "runner_kind": "loudink_v1",
7
  "hf_repo": "nsalerni/loudink-v1.5",
8
  "predecessor": "nsalerni/loudink-v1.1",
@@ -23,8 +23,8 @@
23
  "daily_mac_overall_shipped": 1.0,
24
  "daily_mac_writing_shipped": 1.0,
25
  "daily_mac_writing_neural": 1.0,
26
- "daily_mac_writing_p50_ms": 149.9,
27
- "daily_mac_writing_model_p50_ms": 173.8,
28
  "daily_mac_computer_shipped": 1.0,
29
  "daily_mac_multi_step_shipped": 1.0,
30
  "daily_mac_n": 69,
@@ -32,7 +32,11 @@
32
  "skill_invoke_n": 19,
33
  "ir_honesty_neural_n112": 0.4732,
34
  "hot_package_mib": 834.6,
35
- "v11_hot_package_mib_same_accounting": 878.62
 
 
 
 
36
  },
37
  "line": "loudink-v1.5",
38
  "hot_package_note": "bases + writer_unified + ir only (no promoted_core seed)"
 
2
  "model_name": "loudink-v1.5",
3
  "model_tag": "loudink-v1.5",
4
  "product": "loudink-v1.5",
5
+ "version": "1.5.1",
6
  "runner_kind": "loudink_v1",
7
  "hf_repo": "nsalerni/loudink-v1.5",
8
  "predecessor": "nsalerni/loudink-v1.1",
 
23
  "daily_mac_overall_shipped": 1.0,
24
  "daily_mac_writing_shipped": 1.0,
25
  "daily_mac_writing_neural": 1.0,
26
+ "daily_mac_writing_p50_ms": 0.1,
27
+ "daily_mac_writing_model_p50_ms": 141.3,
28
  "daily_mac_computer_shipped": 1.0,
29
  "daily_mac_multi_step_shipped": 1.0,
30
  "daily_mac_n": 69,
 
32
  "skill_invoke_n": 19,
33
  "ir_honesty_neural_n112": 0.4732,
34
  "hot_package_mib": 834.6,
35
+ "v11_hot_package_mib_same_accounting": 878.62,
36
+ "daily_mac_writing_p95_ms": 195.5,
37
+ "daily_mac_writing_model_p95_ms": 242.8,
38
+ "daily_mac_writing_model_n": 11,
39
+ "daily_mac_writing_fp_n": 14
40
  },
41
  "line": "loudink-v1.5",
42
  "hot_package_note": "bases + writer_unified + ir only (no promoted_core seed)"
manifest.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "product": "loudink-v1.5",
3
- "version": "1.5.0",
4
  "pipeline_namespace": "loudink_v1_5",
5
  "runner_kind": "loudink_v1",
6
  "gate_mode": "warn",
@@ -49,8 +49,8 @@
49
  "daily_mac_overall_shipped": 1.0,
50
  "daily_mac_writing_shipped": 1.0,
51
  "daily_mac_writing_neural": 1.0,
52
- "daily_mac_writing_p50_ms": 149.9,
53
- "daily_mac_writing_model_p50_ms": 173.8,
54
  "daily_mac_computer_shipped": 1.0,
55
  "daily_mac_multi_step_shipped": 1.0,
56
  "daily_mac_n": 69,
@@ -58,8 +58,12 @@
58
  "skill_invoke_n": 19,
59
  "ir_honesty_neural_n112": 0.4732,
60
  "hot_package_mib": 834.6,
61
- "v11_hot_package_mib_same_accounting": 878.62
 
 
 
 
62
  },
63
- "notes": "1.5.0: Beats v1.1 on writing p50, model p50, IR honesty neural; holds 100% daily-mac/skill. Hot package omits promoted_core seed (unified-only writer). Strictly smaller than v1.1 HF bundle under same accounting.",
64
  "hot_package_accounting": "writer_base+ir_base+writer_unified_adapters+ir_adapters (no promoted_core seed)"
65
  }
 
1
  {
2
  "product": "loudink-v1.5",
3
+ "version": "1.5.1",
4
  "pipeline_namespace": "loudink_v1_5",
5
  "runner_kind": "loudink_v1",
6
  "gate_mode": "warn",
 
49
  "daily_mac_overall_shipped": 1.0,
50
  "daily_mac_writing_shipped": 1.0,
51
  "daily_mac_writing_neural": 1.0,
52
+ "daily_mac_writing_p50_ms": 0.1,
53
+ "daily_mac_writing_model_p50_ms": 141.3,
54
  "daily_mac_computer_shipped": 1.0,
55
  "daily_mac_multi_step_shipped": 1.0,
56
  "daily_mac_n": 69,
 
58
  "skill_invoke_n": 19,
59
  "ir_honesty_neural_n112": 0.4732,
60
  "hot_package_mib": 834.6,
61
+ "v11_hot_package_mib_same_accounting": 878.62,
62
+ "daily_mac_writing_p95_ms": 195.5,
63
+ "daily_mac_writing_model_p95_ms": 242.8,
64
+ "daily_mac_writing_model_n": 11,
65
+ "daily_mac_writing_fp_n": 14
66
  },
67
+ "notes": "1.5.1: Honesty-safe writing FP (14/25 daily writing), SQL+paragraph structure fast paths, self-correction normalize, slim unified-only HF package, IR honesty 47.3%, gate_loudink_v1_5. Beats v1.1 on size, writing p50/p95, model p50, IR honesty.",
68
  "hot_package_accounting": "writer_base+ir_base+writer_unified_adapters+ir_adapters (no promoted_core seed)"
69
  }
product_scorecard.json CHANGED
@@ -18,14 +18,14 @@
18
  "polish_lift": 0.0,
19
  "latency": {
20
  "n": 25,
21
- "mean": 147.74175995796918,
22
- "p50": 149.9218330718577,
23
- "p95": 280.6902078446001,
24
- "max": 308.112750062719,
25
- "model_invoked_p50": 174.93549990467727,
26
- "model_invoked_p95": 280.6902078446001,
27
- "model_invoked_n": 20,
28
- "fast_path_rate": 0.19999999999999996
29
  }
30
  },
31
  "computer": {
@@ -79,20 +79,20 @@
79
  },
80
  "latency": {
81
  "n": 69,
82
- "mean": 53.561507231148255,
83
  "p50": 0.05,
84
- "p95": 239.0868340153247,
85
- "max": 308.112750062719,
86
- "model_invoked_p50": 174.93549990467727,
87
- "model_invoked_p95": 280.6902078446001,
88
- "model_invoked_n": 20,
89
- "fast_path_rate": 0.7101449275362319
90
  }
91
  },
92
  "finish_reasons": {
93
  "transcript_ir_fast_path": 44,
94
- "loudink_v1_structural_polish": 20,
95
- "transcript_polish_fast_path": 5
96
  }
97
  }
98
  ]
 
18
  "polish_lift": 0.0,
19
  "latency": {
20
  "n": 25,
21
+ "mean": 72.98354492622614,
22
+ "p50": 0.05,
23
+ "p95": 242.8087079897523,
24
+ "max": 284.9084581248462,
25
+ "model_invoked_p50": 141.26870781183243,
26
+ "model_invoked_p95": 284.9084581248462,
27
+ "model_invoked_n": 11,
28
+ "fast_path_rate": 0.56
29
  }
30
  },
31
  "computer": {
 
79
  },
80
  "latency": {
81
  "n": 69,
82
+ "mean": 26.475197437038457,
83
  "p50": 0.05,
84
+ "p95": 175.02208286896348,
85
+ "max": 284.9084581248462,
86
+ "model_invoked_p50": 141.26870781183243,
87
+ "model_invoked_p95": 284.9084581248462,
88
+ "model_invoked_n": 11,
89
+ "fast_path_rate": 0.8405797101449275
90
  }
91
  },
92
  "finish_reasons": {
93
  "transcript_ir_fast_path": 44,
94
+ "transcript_polish_fast_path": 14,
95
+ "loudink_v1_structural_polish": 11
96
  }
97
  }
98
  ]
publish_report.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "hf_repo": "nsalerni/loudink-v1.5",
3
- "version": "1.5.0",
4
  "measured_adapter_bundle_mib": 26.12,
5
  "files": [
6
  "README.md",
 
1
  {
2
  "hf_repo": "nsalerni/loudink-v1.5",
3
+ "version": "1.5.1",
4
  "measured_adapter_bundle_mib": 26.12,
5
  "files": [
6
  "README.md",