Buckets:
85.9 GB
853 files
Updated about 7 hours ago
Ctrl+K
| Name | Size | Uploaded | Xet hash |
|---|---|---|---|
| analysis | 5 items | ||
| code | 93 items | ||
| dashboard | 2 items | ||
| jevals | 21 items | ||
| report | 30 items | ||
| run1 | 170 items | ||
| run2_v2 | 279 items | ||
| run3_2b | 97 items | ||
| s1mb | 154 items | ||
| FINAL_REPORT.md | 14.1 kB xet | 2df5f3eb | |
| README.md | 3.07 kB xet | 4c6d2c0a |
Mercity / Desicion-Model-Custom
A Jev-style typed decision model on Qwen3.5-4B: a fast readout (one forward pass, a probability per option) plus a thinking mode whose answer agrees with the readout, a router that sends low-margin questions to thinking, and RL on the thinking mode. Single A100 80GB, October-November 2026. No API keys or tokens are stored here.
| Path | What |
|---|---|
code/ |
all code (training, generation, eval, metrics, RL, Jevals, dashboard), notes (NOTEBOOK.md, DEVIATIONS.md, TODO.md, report.md), ops/ = the overnight driver and watcher scripts |
run1/ |
v1: data, results (predictions incl. raw chains, metrics, report), logs, merged checkpoints (joint, gold_only, votes_only) |
run2_v2/ |
V2: 42K readout questions, 11K chains (4B + 35B teacher), speed benchmark, LR sweep, joint run, two ablations, evals; RL (checkpoints/rl_grpo steps 1-40 at 1e-5, rl_grpo_b steps 41-60 at 2.5e-5, with every sampled chain in completions/); merged v2_joint_merged, rl60_merged (final), rl40_merged (RL step 40), abl_*_merged |
run3_2b/ |
the ~2B model: probe of Qwen3.5-2B vs MiniCPM5-2B (MiniCPM5-2B picked), sweep, V2-recipe and readout-only training, evals, merged v2b_ro_merged and v2b_joint_merged (deploy: its LoRA for the readout, adapter off for thinking) |
jevals/ |
jevals.com suite 0.1.0 rebuilt and hash-checked, our runs (fast and thinking), jevals.json score table |
s1mb/ |
S1MB English v1: our 26,269 built items, raw readouts for 5 models (runs/), s1mb.json score table vs 211 published models |
report/ |
the full technical report: decision_model_report.pdf (23 pages, 23 figures), its markdown source and figs/ |
analysis/ |
bootstrap CIs and hypothesis verdicts (final_stats), fast-vs-thinking error analysis (examples_analysis), yes/no offset check |
FINAL_REPORT.md |
the final report (also as a doc: https://claude.ai/artifact/RUdY6E2PZgStqNURLGT8g8) |
dashboard/ |
the run dashboard page (index.html) and its data |
Headline numbers (eval set of 5,442 questions; thinking mode on a 1,498-question subset):
- fast readout: base 62.7% -> v1 69.5% -> V2 72.7% (V2 + RL 72.5%)
- thinking mode: base 84.0%, v1 84.4%, V2 81.2% (5.1% loops), V2 + RL 83.1% (0.2% loops, half the tokens)
- routed at 30%: v1 78.0%, V2 81.8%, V2 + RL 81.8% at 729 thinking tokens per question (V2 1,749)
- Jevals fast mode: PubMedQA DS 58.6, Banking77 61.3, HelpSteer2 2.4 (thinking: 63.5 / 59.3 / -20.1)
- 2B (MiniCPM5-2B): readout-only fast 66.0%; fast readout + untrained thinking routed 30%: 75.6% at 253 thinking tokens
- S1MB Task Avg (211 published models, median 41.2, best 74.4, best 4B 67.0): V2 + RL 47.9 (74 published models above in Borda order), v1 43.5, untrained Qwen3.5-4B 32.8; 2B readout-only 37.3, untrained MiniCPM5-2B 18.1
- S1MB with the routed system (V2 + RL, 30% to thinking): Task Avg 55.2 (+7.3 [4.9, 9.9]; 50 published models above)
Full write-up: report/decision_model_report.pdf (short version: FINAL_REPORT.md).
- Total size
- 85.9 GB
- Files
- 853
- Last updated
- Nov 1
- Pre-warmed CDN
- US EU US EU
