warrenwong641 commited on
Commit
c19eb61
·
verified ·
1 Parent(s): 7fcfe77

Add reproducible MCE-220 Model A runner

Browse files

Bundle the frozen MCE-220 dataset and audit trail, corrected Model A evaluation runner, pinned asset downloader, exact verifier, and hard-timeout supervisor.

Files changed (30) hide show
  1. README.md +20 -0
  2. mce220_runner/README.md +54 -0
  3. mce220_runner/SOURCE_MANIFEST.json +252 -0
  4. mce220_runner/beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py +2232 -0
  5. mce220_runner/beam_search_model2/__init__.py +0 -0
  6. mce220_runner/beam_search_model2/llm_api_compat.py +26 -0
  7. mce220_runner/beam_search_model2/llmstr_to_vector.py +104 -0
  8. mce220_runner/beam_search_model2/run_on_window_shear_jsonl_v12.py +821 -0
  9. mce220_runner/beam_search_model2/verify_proof_certificate.py +557 -0
  10. mce220_runner/data/__init__.py +0 -0
  11. mce220_runner/data/entropy_vector_converter.py +250 -0
  12. mce220_runner/dataset/README.md +89 -0
  13. mce220_runner/dataset/audit/diversity-audit.json +422 -0
  14. mce220_runner/dataset/audit/generation-report.jsonl +220 -0
  15. mce220_runner/dataset/audit/provenance.jsonl +0 -0
  16. mce220_runner/dataset/audit/run-manifest.json +1 -0
  17. mce220_runner/dataset/audit/validation-report.json +1 -0
  18. mce220_runner/dataset/data/questions.jsonl +0 -0
  19. mce220_runner/dataset/generation/e2n_v2.toml +63 -0
  20. mce220_runner/download_assets.py +249 -0
  21. mce220_runner/experiments/__init__.py +0 -0
  22. mce220_runner/experiments/reviewer_response/__init__.py +0 -0
  23. mce220_runner/experiments/reviewer_response/configs/inference_policy.toml +16 -0
  24. mce220_runner/experiments/reviewer_response/run_search_benchmark.py +1108 -0
  25. mce220_runner/experiments/reviewer_response/supervise_search_benchmark.py +780 -0
  26. mce220_runner/grpo_pipeline/__init__.py +0 -0
  27. mce220_runner/grpo_pipeline/modules/__init__.py +0 -0
  28. mce220_runner/grpo_pipeline/modules/llm_api_interface.py +427 -0
  29. mce220_runner/requirements.txt +10 -0
  30. mce220_runner/run_mce220.sh +182 -0
README.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - peft
4
+ - qwen3
5
+ - entropy
6
+ - theorem-proving
7
+ - benchmark
8
+ ---
9
+
10
+ # Entropy Inequality Prover models and scripts
11
+
12
+ This repository contains the project checkpoints and inference resources. The new [MCE-220 Model A runner](./mce220_runner/) packages the corrected beam-search implementation, exact proof verifier, hard-timeout supervisor, frozen inference policy, and a complete local copy of the 220-question benchmark with its audit trail.
13
+
14
+ ## MCE-220 release
15
+
16
+ - Runner and instructions: [`mce220_runner/`](./mce220_runner/)
17
+ - Model A adapter: [`qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534/`](./qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534/)
18
+ - Bundled benchmark: [`mce220_runner/dataset/`](./mce220_runner/dataset/)
19
+
20
+ The downloader pins this model repository at `7fcfe778e1c36cd67e88ea8aa2ee761a50a39127` and the base model at `c1899de289a04d12100db370d81485cdf75e47ca`. Existing checkpoint directories are unchanged. Model and benchmark reuse terms have not yet been assigned; this card does not add a license.
mce220_runner/README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MCE-220 Model A runner
2
+
3
+ This directory is a self-contained evaluation bundle for Model A checkpoint-9534 on the frozen 220-question MCE-220 benchmark. The benchmark, provenance, generation reports, validation evidence, diversity audit, generation configuration, runner, exact verifier, and supervision code are included here. Large model weights are downloaded on demand from immutable Hugging Face revisions.
4
+
5
+ ## Pinned inputs
6
+
7
+ - Dataset source commit: `5650d7c8198a68116e8d12934aed41a37866ba92` (a byte-identical copy is under `dataset/`).
8
+ - Model repository commit: `7fcfe778e1c36cd67e88ea8aa2ee761a50a39127`.
9
+ - Model A adapter: `qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534`.
10
+ - Base model: `Qwen/Qwen3-0.6B` at `c1899de289a04d12100db370d81485cdf75e47ca`.
11
+
12
+ `download_assets.py` downloads only the nine inference-required checkpoint files. It rejects optimizer, scheduler, RNG, trainer-state, and training-argument artifacts, verifies the adapter and tokenizer SHA-256 values, downloads the seven required base-model files, and compares the bundled dataset with the pinned dataset source.
13
+
14
+ ## Setup
15
+
16
+ Linux with Bash, Python 3.10 or newer, CUDA, and NVIDIA GPUs is expected. Create an environment, install the CUDA-enabled PyTorch build suitable for the host, then install the remaining pinned requirements:
17
+
18
+ ```bash
19
+ python3 -m venv .venv
20
+ source .venv/bin/activate
21
+ pip install -r mce220_runner/requirements.txt
22
+ python mce220_runner/download_assets.py
23
+ ```
24
+
25
+ If your platform needs a different PyTorch CUDA wheel, install that wheel first and then install the other pinned packages without replacing it. The base model is about 1.5 GB and the adapter about 162 MB. A worker holds an independent model instance; allow roughly 3–5 GiB of free VRAM per worker plus search overhead. The paper configuration uses eight workers. GPU IDs may be repeated only when a device has enough memory for multiple workers.
26
+
27
+ ## Commands
28
+
29
+ ```bash
30
+ # Validate every asset and print the frozen 220-row command without loading a model.
31
+ bash mce220_runner/run_mce220.sh dry-run --gpu-ids 0,1,2,3,4,5,6,7 --workers 8
32
+
33
+ # Run one question with beam 4, one rollout, depth 4, and one GPU.
34
+ bash mce220_runner/run_mce220.sh smoke --gpu-ids 0 --workers 1
35
+
36
+ # Run all questions with the published search and hard-timeout policy.
37
+ bash mce220_runner/run_mce220.sh full --gpu-ids 0,1,2,3,4,5,6,7 --workers 8 --output-dir ./mce220-results
38
+ ```
39
+
40
+ Set `PYTHON` to choose the interpreter and `MCE220_ASSET_DIR` to place downloaded weights outside this directory.
41
+
42
+ ## Frozen full-run policy
43
+
44
+ The full mode fixes beam size 32, six rollouts, maximum depth 24, seed 42, temperature 1.2, top-p 0.9, 2,048 generated tokens, 6,000 explored nodes, and a 1,000-second external hard timeout for every question. It uses the canonical raw prompt, candidate normalization, exact Shannon LP checks, and the pinned local base-model snapshot. The supervisor has a 42-hour total stop so interrupted jobs can be resumed deliberately.
45
+
46
+ ## Outputs and resume behavior
47
+
48
+ `full` writes `results.jsonl`, `supervisor-state.json`, and immutable per-attempt manifests/stdout/stderr under `supervisor-attempts/`. Each successful or timed-out question produces one row, and proof-bearing rows are checked by the exact rational verifier. Re-running `full` with the same output directory validates the existing prefix and continues at the next question. It never silently replaces a nonempty result file. `smoke` similarly resumes a valid one-row prefix. Use a new output directory to start a clean run.
49
+
50
+ ## Limitations
51
+
52
+ The benchmark contains constructed Shannon-type problems with supplied equality conditions; it does not cover arbitrary dense or non-Shannon inequalities. The base model and adapter are trusted local Hugging Face artifacts and the loader uses `trust_remote_code=True`, although the pinned Qwen revision contains no custom Python files. GPU sampling remains stochastic across kernels, devices, drivers, and library builds. A clean rerun follows the published protocol but is not guaranteed to reproduce exactly 155 proofs out of 220.
53
+
54
+ No AITIP/PSITIP baseline, training code, or historical segmented runner is included. No license is asserted by this bundle.
mce220_runner/SOURCE_MANIFEST.json ADDED
@@ -0,0 +1,252 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "created_utc": "2026-09-01T11:12:07.595478Z",
3
+ "files": {
4
+ "README.md": {
5
+ "bytes": 4249,
6
+ "sha256": "d7fb5484e254526cfca3476ef341fabcf14f34479f375ee8cb3416dc565aeaac",
7
+ "source": {
8
+ "kind": "generated-release-file"
9
+ }
10
+ },
11
+ "beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py": {
12
+ "bytes": 97133,
13
+ "sha256": "403370fcb05db7fb515109c76a87eb67ccd7033fdb06544fe6b239138a21b393",
14
+ "source": {
15
+ "kind": "local-source",
16
+ "path": "beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py"
17
+ }
18
+ },
19
+ "beam_search_model2/__init__.py": {
20
+ "bytes": 0,
21
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
22
+ "source": {
23
+ "kind": "generated-release-file"
24
+ }
25
+ },
26
+ "beam_search_model2/llm_api_compat.py": {
27
+ "bytes": 872,
28
+ "sha256": "8ed35fee37688c020051f13cf9945a402c8b1b969ca06acd204e629be9ed2cc0",
29
+ "source": {
30
+ "kind": "local-source",
31
+ "path": "beam_search_model2/llm_api_compat.py"
32
+ }
33
+ },
34
+ "beam_search_model2/llmstr_to_vector.py": {
35
+ "bytes": 3242,
36
+ "sha256": "0611db5d168460783755395b5a7aa4d95d78a8397139b53584eb5a17dc27c8ef",
37
+ "source": {
38
+ "kind": "local-source",
39
+ "path": "beam_search_model2/llmstr_to_vector.py"
40
+ }
41
+ },
42
+ "beam_search_model2/run_on_window_shear_jsonl_v12.py": {
43
+ "bytes": 30325,
44
+ "sha256": "197fd80b4dc39486e987b10547ed49a84fabf953e2ff6a2393b5c47cdadc95bf",
45
+ "source": {
46
+ "kind": "local-source",
47
+ "path": "beam_search_model2/run_on_window_shear_jsonl_v12.py"
48
+ }
49
+ },
50
+ "beam_search_model2/verify_proof_certificate.py": {
51
+ "bytes": 20711,
52
+ "sha256": "ba00337dac6e8715a8f4dd32166bb1393ec02e37ad931e9affdae7741807349a",
53
+ "source": {
54
+ "kind": "local-source",
55
+ "path": "beam_search_model2/verify_proof_certificate.py"
56
+ }
57
+ },
58
+ "data/__init__.py": {
59
+ "bytes": 0,
60
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
61
+ "source": {
62
+ "kind": "generated-release-file"
63
+ }
64
+ },
65
+ "data/entropy_vector_converter.py": {
66
+ "bytes": 9470,
67
+ "sha256": "3a1398661b146e661c91b2fa4d815a5e87b2a209751dba630ec6e88feb34d109",
68
+ "source": {
69
+ "kind": "local-source",
70
+ "path": "data/entropy_vector_converter.py"
71
+ }
72
+ },
73
+ "dataset/README.md": {
74
+ "bytes": 4651,
75
+ "sha256": "909828a43ace9c3d4e9aa2e2fe96812dadd1c08ece52c87cf4dc896ad4ce0452",
76
+ "source": {
77
+ "kind": "dataset-snapshot",
78
+ "repo": "entropyinequalityprover/MCE-220",
79
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
80
+ }
81
+ },
82
+ "dataset/audit/diversity-audit.json": {
83
+ "bytes": 8239,
84
+ "sha256": "56721eb34f35fa9acc2d81bbb13b0ea263a7f29bc8838372339dbc38ecda44a4",
85
+ "source": {
86
+ "kind": "dataset-snapshot",
87
+ "repo": "entropyinequalityprover/MCE-220",
88
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
89
+ }
90
+ },
91
+ "dataset/audit/generation-report.jsonl": {
92
+ "bytes": 69724,
93
+ "sha256": "50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c",
94
+ "source": {
95
+ "kind": "dataset-snapshot",
96
+ "repo": "entropyinequalityprover/MCE-220",
97
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
98
+ }
99
+ },
100
+ "dataset/audit/provenance.jsonl": {
101
+ "bytes": 4453377,
102
+ "sha256": "cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189",
103
+ "source": {
104
+ "kind": "dataset-snapshot",
105
+ "repo": "entropyinequalityprover/MCE-220",
106
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
107
+ }
108
+ },
109
+ "dataset/audit/run-manifest.json": {
110
+ "bytes": 2750,
111
+ "sha256": "ac32fdaecf845714f9802c584c91b8743398898ff7e7c1f36c43014dea36d9a6",
112
+ "source": {
113
+ "kind": "dataset-snapshot",
114
+ "repo": "entropyinequalityprover/MCE-220",
115
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
116
+ }
117
+ },
118
+ "dataset/audit/validation-report.json": {
119
+ "bytes": 381,
120
+ "sha256": "3f86d1391c15a343cb938092a456ce2b5b88bb7f1ebcbf4089d0c8174c7e4d3c",
121
+ "source": {
122
+ "kind": "dataset-snapshot",
123
+ "repo": "entropyinequalityprover/MCE-220",
124
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
125
+ }
126
+ },
127
+ "dataset/data/questions.jsonl": {
128
+ "bytes": 499119,
129
+ "sha256": "1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a",
130
+ "source": {
131
+ "kind": "dataset-snapshot",
132
+ "repo": "entropyinequalityprover/MCE-220",
133
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
134
+ }
135
+ },
136
+ "dataset/generation/e2n_v2.toml": {
137
+ "bytes": 1560,
138
+ "sha256": "3346eb6fc6aa0bb70ca15f9a840d7b0f366418072c58cb76d32e0f8ac9faae4a",
139
+ "source": {
140
+ "kind": "dataset-snapshot",
141
+ "repo": "entropyinequalityprover/MCE-220",
142
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
143
+ }
144
+ },
145
+ "download_assets.py": {
146
+ "bytes": 8434,
147
+ "sha256": "f308777bcf46500d4ce8fe2636e0b957ec0c361dd74fc06416204ccb17538610",
148
+ "source": {
149
+ "kind": "generated-release-file"
150
+ }
151
+ },
152
+ "experiments/__init__.py": {
153
+ "bytes": 0,
154
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
155
+ "source": {
156
+ "kind": "generated-release-file"
157
+ }
158
+ },
159
+ "experiments/reviewer_response/__init__.py": {
160
+ "bytes": 0,
161
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
162
+ "source": {
163
+ "kind": "generated-release-file"
164
+ }
165
+ },
166
+ "experiments/reviewer_response/configs/inference_policy.toml": {
167
+ "bytes": 671,
168
+ "sha256": "ba4e6497f6e688a0740cfbcc945488c20a9f447084b3e05bde4bb08ef51e07fe",
169
+ "source": {
170
+ "kind": "local-source",
171
+ "path": "experiments/reviewer_response/configs/inference_policy.toml"
172
+ }
173
+ },
174
+ "experiments/reviewer_response/run_search_benchmark.py": {
175
+ "bytes": 40875,
176
+ "sha256": "7c3a442548089419d0c1f1af24d764474aee7f540a818bd3bb27db74c997c0bd",
177
+ "source": {
178
+ "change": "portable --base-model argument and effective-path recording",
179
+ "kind": "derived-local-source",
180
+ "path": "experiments/reviewer_response/run_search_benchmark.py"
181
+ }
182
+ },
183
+ "experiments/reviewer_response/supervise_search_benchmark.py": {
184
+ "bytes": 30678,
185
+ "sha256": "3a748d5651de33aedb93d1ae9bf396ba9cbf3e87a69dce45fea1edcb52137d96",
186
+ "source": {
187
+ "kind": "local-source",
188
+ "path": "experiments/reviewer_response/supervise_search_benchmark.py"
189
+ }
190
+ },
191
+ "grpo_pipeline/__init__.py": {
192
+ "bytes": 0,
193
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
194
+ "source": {
195
+ "kind": "generated-release-file"
196
+ }
197
+ },
198
+ "grpo_pipeline/modules/__init__.py": {
199
+ "bytes": 0,
200
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
201
+ "source": {
202
+ "kind": "generated-release-file"
203
+ }
204
+ },
205
+ "grpo_pipeline/modules/llm_api_interface.py": {
206
+ "bytes": 16629,
207
+ "sha256": "4e7bd0f8c13be24d4e09ea90270799cbc440015beb0575e314b5d659b2a7544e",
208
+ "source": {
209
+ "kind": "local-source",
210
+ "path": "grpo_pipeline/modules/llm_api_interface.py"
211
+ }
212
+ },
213
+ "requirements.txt": {
214
+ "bytes": 240,
215
+ "sha256": "cbfac9b9d0f3a38e9c1a4c361856fe9678d95c31cf0b895ecab06de544a85a25",
216
+ "source": {
217
+ "kind": "generated-release-file"
218
+ }
219
+ },
220
+ "run_mce220.sh": {
221
+ "bytes": 5042,
222
+ "sha256": "fb3198e886fe53b09011344620da50b5734acd49e24ef147f8fe3e58f44f8959",
223
+ "source": {
224
+ "kind": "generated-release-file"
225
+ }
226
+ }
227
+ },
228
+ "pinned_revisions": {
229
+ "base_model": {
230
+ "repo": "Qwen/Qwen3-0.6B",
231
+ "revision": "c1899de289a04d12100db370d81485cdf75e47ca"
232
+ },
233
+ "dataset": {
234
+ "repo": "entropyinequalityprover/MCE-220",
235
+ "revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
236
+ },
237
+ "model": {
238
+ "repo": "entropyinequalityprover/models_and_scripts",
239
+ "revision": "7fcfe778e1c36cd67e88ea8aa2ee761a50a39127"
240
+ }
241
+ },
242
+ "schema_version": "mce220-runner-source-manifest-v1",
243
+ "self_reference": {
244
+ "path": "SOURCE_MANIFEST.json",
245
+ "reason": "A file cannot contain its own cryptographic digest; verify this file from the release commit.",
246
+ "sha256": null
247
+ },
248
+ "source_workspace": {
249
+ "git_head": "985afadcface3883a66a1c0bcd39d141737abc1c",
250
+ "note": "Selected files were copied from the validated current worktree without modifying it."
251
+ }
252
+ }
mce220_runner/beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py ADDED
@@ -0,0 +1,2232 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # This is for llm inference on 4type data trained llm model2.
2
+ import random
3
+ import os
4
+ import sys
5
+ from pathlib import Path
6
+ from concurrent.futures import ProcessPoolExecutor, as_completed
7
+ from concurrent.futures.process import BrokenProcessPool
8
+ import multiprocessing as mp
9
+
10
+ import numpy as np
11
+ import re
12
+ from scipy.sparse import csr_matrix, vstack, hstack, coo_matrix
13
+ from scipy.optimize import linprog
14
+ from typing import List, Dict, Tuple, Optional
15
+ import json
16
+ import time
17
+ import uuid
18
+ import pickle
19
+ # Make imports work when running from repo root (or anywhere)
20
+ _REPO_ROOT = Path(__file__).resolve().parents[1]
21
+ _DATA_DIR = _REPO_ROOT / "data"
22
+ _MODEL2_DIR = _REPO_ROOT / "beam_search_model2"
23
+ for _p in (str(_DATA_DIR), str(_MODEL2_DIR)):
24
+ if _p not in sys.path:
25
+ sys.path.append(_p)
26
+
27
+ try:
28
+ # Running as a script (local dir on sys.path)
29
+ from llmstr_to_vector import polym_ineq_to_vector
30
+ from entropy_vector_converter import string_to_vector, vector_to_string
31
+ from llm_api_compat import LLM as LLM_propose_inequalities
32
+ from llm_api_compat import get_last_llm_stats
33
+ except ModuleNotFoundError:
34
+ # Fallback to absolute/package-style imports if needed
35
+ from beam_search_model2.llmstr_to_vector import polym_ineq_to_vector
36
+ from entropy_vector_converter import string_to_vector, vector_to_string
37
+ from beam_search_model2.llm_api_compat import LLM as LLM_propose_inequalities
38
+ from beam_search_model2.llm_api_compat import get_last_llm_stats
39
+
40
+ # ---- Multi-GPU LLM worker pool support (one process per GPU; one model per process) ----
41
+ _GPU_ADAPTER_PATH = None
42
+ _GPU_USE_DUMMY = False
43
+ _GPU_READY = False
44
+
45
+ # Global cache for persistent GPU executors
46
+ _CACHED_GPU_EXECUTORS = None
47
+ _CACHED_GPU_PARAMS = None
48
+ _CACHED_GAMMA = {}
49
+ _COND_CACHE = {}
50
+ _LAST_RUN_ID = None
51
+
52
+
53
+ def _is_fatal_gpu_worker_exception(exc: BaseException) -> bool:
54
+ """Identify pool crashes and accelerator OOMs that must abort the run."""
55
+
56
+ current: BaseException | None = exc
57
+ seen: set[int] = set()
58
+ while current is not None and id(current) not in seen:
59
+ seen.add(id(current))
60
+ if isinstance(current, BrokenProcessPool):
61
+ return True
62
+ class_name = type(current).__name__.lower()
63
+ message = str(current).lower()
64
+ if "outofmemory" in class_name or any(
65
+ fragment in message
66
+ for fragment in (
67
+ "cuda out of memory",
68
+ "cuda error: out of memory",
69
+ "hip out of memory",
70
+ "cublas_status_alloc_failed",
71
+ "cudnn_status_alloc_failed",
72
+ "process in the process pool was terminated abruptly",
73
+ )
74
+ ):
75
+ return True
76
+ current = current.__cause__ or current.__context__
77
+ return False
78
+
79
+ ALPHABET = list("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
80
+ _X_TOKEN_RE = re.compile(r"X_\{([^}]+)\}")
81
+ _MULTIPLIER_PAREN_RE = re.compile(r"(\b\d+)\*\(")
82
+ _HI_TERM_RE = re.compile(r"(?<![A-Za-z])[HI]\([^)]*\)")
83
+ _SINGLE_POSITIVE_STEP_RE = re.compile(
84
+ r"^\+?(?:(\d*\.?\d+)\*?)?([HI]\([^()]*\))$"
85
+ )
86
+
87
+ def _use_alpha_prompt() -> bool:
88
+ return os.environ.get("BEAM_PROMPT_FORMAT", "alphabet").strip().lower() != "canonical"
89
+
90
+
91
+ def _sort_alpha_groups_enabled() -> bool:
92
+ return os.environ.get("BEAM_SORT_ALPHA_GROUPS", "0").strip() == "1"
93
+
94
+
95
+ def _normalize_candidates_enabled() -> bool:
96
+ return os.environ.get("BEAM_NORMALIZE_CANDIDATES", "0").strip() == "1"
97
+
98
+
99
+ def _format_scalar_text(value: float) -> str:
100
+ rounded = round(value)
101
+ if abs(value - rounded) <= 1e-12:
102
+ return str(int(rounded))
103
+ return f"{value:.12g}"
104
+
105
+
106
+ def _strip_wrapping_parens(expr: str) -> str:
107
+ s = (expr or "").strip()
108
+ while s.startswith("(") and s.endswith(")"):
109
+ depth = 0
110
+ wraps_all = True
111
+ for idx, ch in enumerate(s):
112
+ if ch == "(":
113
+ depth += 1
114
+ elif ch == ")":
115
+ depth -= 1
116
+ if depth < 0:
117
+ return s
118
+ if depth == 0 and idx != len(s) - 1:
119
+ wraps_all = False
120
+ break
121
+ if depth != 0 or not wraps_all:
122
+ break
123
+ s = s[1:-1].strip()
124
+ return s
125
+
126
+
127
+ def _split_top_level_linear_terms(expr: str) -> List[str]:
128
+ parts: List[str] = []
129
+ start = 0
130
+ depth = 0
131
+ for idx, ch in enumerate(expr):
132
+ if ch == "(":
133
+ depth += 1
134
+ elif ch == ")":
135
+ depth -= 1
136
+ elif depth == 0 and idx > start and ch in "+-":
137
+ parts.append(expr[start:idx])
138
+ start = idx
139
+ parts.append(expr[start:])
140
+ return [p for p in parts if p]
141
+
142
+
143
+ def _parse_linear_hi_term(term: str) -> Optional[Tuple[float, str]]:
144
+ s = _strip_wrapping_parens(term)
145
+ m = re.fullmatch(r"([+-]?)(?:(\d*\.?\d+)\*?)?([HI]\([^()]*\))", s)
146
+ if not m:
147
+ return None
148
+ sign_str, coeff_str, hi_term = m.groups()
149
+ coeff = 1.0
150
+ if coeff_str:
151
+ try:
152
+ coeff = float(coeff_str)
153
+ except ValueError:
154
+ return None
155
+ if sign_str == "-":
156
+ coeff *= -1.0
157
+ return coeff, hi_term
158
+
159
+
160
+ def _distribute_scalar_over_hi_expr(expr: str, scalar: float) -> Optional[str]:
161
+ inner = _strip_wrapping_parens(expr)
162
+ if not inner:
163
+ return None
164
+ pieces = _split_top_level_linear_terms(inner)
165
+ if not pieces:
166
+ return None
167
+ scaled_terms: List[str] = []
168
+ for piece in pieces:
169
+ parsed = _parse_linear_hi_term(piece)
170
+ if parsed is None:
171
+ return None
172
+ coeff, hi_term = parsed
173
+ scaled_terms.append(f"{_format_scalar_text(scalar * coeff)}*{hi_term}")
174
+ return "".join(
175
+ term if idx == 0 or term.startswith("-") else f"+{term}"
176
+ for idx, term in enumerate(scaled_terms)
177
+ )
178
+
179
+
180
+ def _collapse_leading_scalar_wrappers(expr: str) -> str:
181
+ s = _strip_wrapping_parens(expr)
182
+ while True:
183
+ m = re.fullmatch(r"([+-]?\d*\.?\d+)\*\((.*)\)", s)
184
+ if not m:
185
+ return s
186
+ try:
187
+ outer_coeff = float(m.group(1))
188
+ except ValueError:
189
+ return s
190
+ inner = _collapse_leading_scalar_wrappers(m.group(2))
191
+ distributed = _distribute_scalar_over_hi_expr(inner, outer_coeff)
192
+ if distributed is None:
193
+ return s
194
+ s = distributed
195
+
196
+
197
+ def _build_alpha_mapping(exprs: List[str]) -> Tuple[Optional[Dict[int, str]], Optional[Dict[str, int]]]:
198
+ """
199
+ Build a deterministic mapping between indices and letters for alphabet syntax.
200
+ Returns (idx_to_letter, letter_to_idx) or (None, None) if not applicable.
201
+ """
202
+ indices: List[int] = []
203
+ for expr in exprs:
204
+ indices.extend(_extract_var_indices(expr))
205
+ unique = sorted(set(indices))
206
+ if not unique:
207
+ return None, None
208
+ if len(unique) > len(ALPHABET):
209
+ sys.stderr.write(
210
+ f"[beam_debug] Too many unique indices ({len(unique)}) for alphabet mapping; "
211
+ "falling back to numeric syntax.\n"
212
+ )
213
+ sys.stderr.flush()
214
+ return None, None
215
+ idx_to_letter = {idx: ALPHABET[i] for i, idx in enumerate(unique)}
216
+ letter_to_idx = {v: k for k, v in idx_to_letter.items()}
217
+ return idx_to_letter, letter_to_idx
218
+
219
+
220
+ def _convert_old_to_alpha(expr: str, idx_to_letter: Optional[Dict[int, str]]) -> str:
221
+ if not expr or not idx_to_letter:
222
+ return expr
223
+ s = _normalize_to_braces(expr)
224
+ sort_groups = _sort_alpha_groups_enabled()
225
+
226
+ def repl(match: re.Match) -> str:
227
+ parts = [p.strip() for p in match.group(1).split(",")]
228
+ letters = []
229
+ for part in parts:
230
+ if not part.isdigit():
231
+ letters.append(part)
232
+ continue
233
+ letters.append(idx_to_letter.get(int(part), part))
234
+ if sort_groups:
235
+ letters = sorted(letters)
236
+ return "".join(letters)
237
+
238
+ s = _X_TOKEN_RE.sub(repl, s)
239
+ s = _MULTIPLIER_PAREN_RE.sub(r"\1(", s)
240
+ return s
241
+
242
+
243
+ def _convert_alpha_group(text: str, letter_to_idx: Dict[str, int]) -> str:
244
+ if "X_" in text:
245
+ return text
246
+ letters = re.findall(r"[A-Z]", text)
247
+ if not letters:
248
+ return text
249
+ if _sort_alpha_groups_enabled():
250
+ letters = sorted(letters)
251
+ indices = []
252
+ for ch in letters:
253
+ if ch not in letter_to_idx:
254
+ return text
255
+ indices.append(str(letter_to_idx[ch]))
256
+ return "X_{" + ",".join(indices) + "}"
257
+
258
+
259
+ def _convert_alpha_term(term: str, letter_to_idx: Dict[str, int]) -> str:
260
+ if term.startswith("H(") and term.endswith(")"):
261
+ inner = term[2:-1]
262
+ if "|" in inner:
263
+ left, right = inner.split("|", 1)
264
+ return f"H({_convert_alpha_group(left, letter_to_idx)}|{_convert_alpha_group(right, letter_to_idx)})"
265
+ return f"H({_convert_alpha_group(inner, letter_to_idx)})"
266
+ if term.startswith("I(") and term.endswith(")"):
267
+ inner = term[2:-1]
268
+ if "|" in inner:
269
+ main, cond = inner.split("|", 1)
270
+ else:
271
+ main, cond = inner, None
272
+ if ";" in main:
273
+ left, right = main.split(";", 1)
274
+ else:
275
+ left, right = main, ""
276
+ left_conv = _convert_alpha_group(left, letter_to_idx)
277
+ right_conv = _convert_alpha_group(right, letter_to_idx)
278
+ if cond is not None:
279
+ cond_conv = _convert_alpha_group(cond, letter_to_idx)
280
+ return f"I({left_conv};{right_conv}|{cond_conv})"
281
+ return f"I({left_conv};{right_conv})"
282
+ return term
283
+
284
+
285
+ def _convert_alpha_to_old(expr: str, letter_to_idx: Optional[Dict[str, int]]) -> str:
286
+ if not expr or not letter_to_idx:
287
+ return expr
288
+ s = _HI_TERM_RE.sub(lambda m: _convert_alpha_term(m.group(0), letter_to_idx), expr)
289
+ s = re.sub(r"(\b\d+)\(", r"\1*(", s)
290
+ return s
291
+
292
+
293
+ def _sort_alpha_group_text(text: str) -> str:
294
+ raw = text.strip()
295
+ if not raw or "X_" in raw:
296
+ return raw
297
+ letters = re.findall(r"[A-Z]", raw)
298
+ if not letters:
299
+ return raw
300
+ leftover = re.sub(r"[A-Z\s]", "", raw)
301
+ if leftover:
302
+ return raw
303
+ return "".join(sorted(letters))
304
+
305
+
306
+ def _canonicalize_alpha_term_text(term: str) -> str:
307
+ if term.startswith("H(") and term.endswith(")"):
308
+ inner = term[2:-1]
309
+ if "|" in inner:
310
+ left, right = inner.split("|", 1)
311
+ return f"H({_sort_alpha_group_text(left)}|{_sort_alpha_group_text(right)})"
312
+ return f"H({_sort_alpha_group_text(inner)})"
313
+ if term.startswith("I(") and term.endswith(")"):
314
+ inner = term[2:-1]
315
+ if "|" in inner:
316
+ main, cond = inner.split("|", 1)
317
+ else:
318
+ main, cond = inner, None
319
+ if ";" in main:
320
+ left, right = main.split(";", 1)
321
+ else:
322
+ left, right = main, ""
323
+ left = _sort_alpha_group_text(left)
324
+ right = _sort_alpha_group_text(right)
325
+ if cond is not None:
326
+ return f"I({left};{right}|{_sort_alpha_group_text(cond)})"
327
+ return f"I({left};{right})"
328
+ return term
329
+
330
+
331
+ def _normalize_candidate_text(cand_raw: str) -> str:
332
+ s = cand_raw.strip().strip("`").strip()
333
+ if not s:
334
+ return s
335
+ s = re.sub(r"^```[a-zA-Z0-9_-]*\s*", "", s)
336
+ s = re.sub(r"\s*```$", "", s)
337
+ s = re.sub(r"^\s*(?:try|candidate|answer|output|step)\s*:\s*", "", s, flags=re.IGNORECASE)
338
+ s = re.sub(r"^\s*(?:[*]|(\d+)[\.\)]|-(?:\s+|$))\s*", "", s).strip()
339
+ if s.lower() == "wrong":
340
+ return "wrong"
341
+ first_math = re.search(r"[HI]\(", s)
342
+ if first_math and first_math.start() > 0:
343
+ prefix = s[: first_math.start()]
344
+ if ":" in prefix or re.search(r"[A-Za-z]", prefix):
345
+ s = s[first_math.start() :].strip()
346
+ s = re.split(r"\bbecause\b", s, maxsplit=1, flags=re.IGNORECASE)[0].strip()
347
+ s = re.sub(r"\s+", "", s)
348
+ s = re.sub(r">\s*=\s*0\b", ">=0", s)
349
+ s = re.sub(r"<\s*=\s*0\b", "<=0", s)
350
+ s = re.sub(r"(?<![<>])=\s*0\b", "=0", s)
351
+ s = re.sub(r">\s*0\b", ">=0", s)
352
+ s = re.sub(r"\(\s*", "(", s)
353
+ s = re.sub(r"\s*\)", ")", s)
354
+ s = re.sub(r"\s*;\s*", ";", s)
355
+ s = re.sub(r"\s*\|\s*", "|", s)
356
+ s = re.sub(r"\s*,\s*", ",", s)
357
+ s = re.sub(r"\s*\+\s*", "+", s)
358
+ s = re.sub(r"\s*-\s*", "-", s)
359
+ s = re.sub(r"\s*\*\s*", "*", s)
360
+ # 若缺少比較運算子,自動補齊 >=0
361
+ if not any(op in s for op in [">=0", ">0", "<=0", "=0"]):
362
+ s += ">=0"
363
+
364
+ # 補全未閉合的括號 (Auto-balance parentheses before the comparator)
365
+ for op in [">=0", ">0", "<=0", "=0"]:
366
+ if op in s:
367
+ lhs, rhs = s.split(op, 1)
368
+ open_p = lhs.count("(")
369
+ close_p = lhs.count(")")
370
+ if open_p > close_p:
371
+ lhs += ")" * (open_p - close_p)
372
+ lhs = _collapse_leading_scalar_wrappers(lhs)
373
+ s = lhs + op + rhs
374
+ break
375
+
376
+ if _sort_alpha_groups_enabled():
377
+ s = _HI_TERM_RE.sub(lambda m: _canonicalize_alpha_term_text(m.group(0)), s)
378
+ return s
379
+
380
+
381
+ def _empty_llm_usage() -> Dict[str, int]:
382
+ return {
383
+ "llm_calls": 0,
384
+ "prompt_tokens_total": 0,
385
+ "generated_tokens_total": 0,
386
+ "generated_sequences_total": 0,
387
+ "extracted_candidates_total": 0,
388
+ "format_fail_total": 0,
389
+ }
390
+
391
+
392
+ def _merge_llm_usage(dst: Dict[str, int], src: Optional[Dict]) -> None:
393
+ if not src:
394
+ return
395
+ dst["llm_calls"] += 1
396
+ dst["prompt_tokens_total"] += int(src.get("prompt_tokens", 0) or 0)
397
+ dst["generated_tokens_total"] += int(src.get("generated_tokens_total", 0) or 0)
398
+ dst["generated_sequences_total"] += int(src.get("num_sequences", 0) or 0)
399
+ dst["extracted_candidates_total"] += int(src.get("extracted_candidates", 0) or 0)
400
+
401
+
402
+ def _split_candidate_comparator(cand: str) -> Tuple[str, Optional[str]]:
403
+ s = (cand or "").strip()
404
+ match = re.search(r"(>=|<=|>|<|=)\s*0\s*$", s)
405
+ if match:
406
+ comparator = {
407
+ ">=": ">=0",
408
+ "<=": "<=0",
409
+ ">": ">0",
410
+ "<": "<0",
411
+ "=": "=0",
412
+ }[match.group(1)]
413
+ return s[: match.start()].strip(), comparator
414
+ return s, None
415
+
416
+
417
+ def _normalize_constraint_comparator(expr: str, *, add_default: bool = False) -> str:
418
+ """Canonicalize a comparator, weakening strict inequalities safely."""
419
+ raw = (expr or "").strip()
420
+ lhs, comparator = _split_candidate_comparator(raw)
421
+ if comparator == ">0":
422
+ comparator = ">=0"
423
+ elif comparator == "<0":
424
+ comparator = "<=0"
425
+ if comparator is None:
426
+ return f"{lhs}>=0" if add_default else lhs
427
+ return f"{lhs}{comparator}"
428
+
429
+
430
+ def _condition_comparator_allows_candidate(
431
+ cond_comparator: Optional[str],
432
+ cand_comparator: Optional[str],
433
+ ) -> bool:
434
+ if cond_comparator == "=0":
435
+ return cand_comparator in {"=0", ">=0"}
436
+ return cond_comparator == ">=0" and cand_comparator == ">=0"
437
+
438
+
439
+ def _is_positive_scalar_multiple(vec_a: csr_matrix, vec_b: csr_matrix, tol: float = 1e-9) -> bool:
440
+ a = vec_a.tocsr(copy=True)
441
+ b = vec_b.tocsr(copy=True)
442
+ a.sum_duplicates()
443
+ b.sum_duplicates()
444
+ a.sort_indices()
445
+ b.sort_indices()
446
+
447
+ if a.nnz == 0 or b.nnz == 0 or a.nnz != b.nnz:
448
+ return False
449
+ if not np.array_equal(a.indices, b.indices):
450
+ return False
451
+
452
+ a_data = np.asarray(a.data, dtype=float)
453
+ b_data = np.asarray(b.data, dtype=float)
454
+ if np.any(np.abs(b_data) <= tol):
455
+ return False
456
+
457
+ ratio = a_data[0] / b_data[0]
458
+ if ratio <= tol:
459
+ return False
460
+
461
+ return np.allclose(a_data, ratio * b_data, rtol=1e-9, atol=tol)
462
+
463
+
464
+ def _is_scalar_multiple(vec_a: csr_matrix, vec_b: csr_matrix) -> bool:
465
+ return _is_positive_scalar_multiple(vec_a, vec_b) or _is_positive_scalar_multiple(vec_a, -vec_b)
466
+
467
+
468
+ def _vectors_are_opposites(vec_a: csr_matrix, vec_b: csr_matrix, tol: float = 1e-9) -> bool:
469
+ return _is_positive_scalar_multiple(vec_a, -vec_b, tol=tol)
470
+
471
+
472
+ def _parse_condition_constraints(
473
+ cond_lines: Optional[List[str]], n: int
474
+ ) -> List[Tuple[str, csr_matrix]]:
475
+ """Parse conditions once, promoting literal and two-sided equalities."""
476
+ equalities: List[csr_matrix] = []
477
+ inequalities: List[csr_matrix] = []
478
+ for raw in cond_lines or []:
479
+ condition = _normalize_constraint_comparator((raw or "").strip(), add_default=True)
480
+ if not condition or condition in {"(none)>=0", "NONE>=0", "NONE=0"}:
481
+ continue
482
+ _, comparator = _split_candidate_comparator(condition)
483
+ try:
484
+ vec = parse_cand_to_vector(condition, n)
485
+ except Exception:
486
+ continue
487
+ if vec is None or vec.nnz == 0:
488
+ continue
489
+ if comparator == "<=0":
490
+ vec = -vec
491
+ comparator = ">=0"
492
+ if comparator == "=0":
493
+ equalities.append(vec)
494
+ elif comparator == ">=0":
495
+ inequalities.append(vec)
496
+
497
+ paired = set()
498
+ for left in range(len(inequalities)):
499
+ if left in paired:
500
+ continue
501
+ for right in range(left + 1, len(inequalities)):
502
+ if right not in paired and _vectors_are_opposites(inequalities[left], inequalities[right]):
503
+ equalities.append(inequalities[left])
504
+ paired.update({left, right})
505
+ break
506
+
507
+ return [("eq", vec) for vec in equalities] + [
508
+ ("ge", vec) for idx, vec in enumerate(inequalities) if idx not in paired
509
+ ]
510
+
511
+
512
+ def _parse_condition_matrices(
513
+ cond_lines: Optional[List[str]], n: int
514
+ ) -> Tuple[Optional[csr_matrix], Optional[csr_matrix]]:
515
+ constraints = _parse_condition_constraints(cond_lines, n)
516
+ eq_vecs = [vec for kind, vec in constraints if kind == "eq"]
517
+ ge_vecs = [vec for kind, vec in constraints if kind == "ge"]
518
+ return (
519
+ vstack(eq_vecs).tocsr() if eq_vecs else None,
520
+ vstack(ge_vecs).tocsr() if ge_vecs else None,
521
+ )
522
+
523
+
524
+ def _is_condition_positive_multiple_step(cand: str, n: Optional[int], cond_lines: Optional[List[str]]) -> bool:
525
+ if n is None or not cond_lines:
526
+ return False
527
+
528
+ cand = _normalize_constraint_comparator(cand)
529
+ _, cand_comparator = _split_candidate_comparator(cand)
530
+ if cand_comparator is None:
531
+ return False
532
+
533
+ try:
534
+ cand_vec = parse_cand_to_vector(cand, n)
535
+ except Exception:
536
+ return False
537
+ if cand_vec is None or cand_vec.nnz == 0:
538
+ return False
539
+
540
+ for kind, cond_vec in _parse_condition_constraints(cond_lines, n):
541
+ cond_comparator = "=0" if kind == "eq" else ">=0"
542
+ if not _condition_comparator_allows_candidate(cond_comparator, cand_comparator):
543
+ continue
544
+ if kind == "eq" and _is_scalar_multiple(cand_vec, cond_vec):
545
+ return True
546
+ if kind == "ge" and _is_positive_scalar_multiple(cand_vec, cond_vec):
547
+ return True
548
+ return False
549
+
550
+
551
+ def _strict_step_format_reason(
552
+ cand: str,
553
+ *,
554
+ n: Optional[int] = None,
555
+ cond_lines: Optional[List[str]] = None,
556
+ ) -> Optional[str]:
557
+ """
558
+ Accept only one positive H(...) or I(...) term with a trailing comparator.
559
+ Also accept multi-term outputs that are a positive scalar multiple of one
560
+ provided condition, e.g. 2*(cond)>=0 or cond>=0.
561
+ """
562
+ s = _normalize_constraint_comparator((cand or "").strip())
563
+ if not s:
564
+ return "empty"
565
+
566
+ lhs, comparator = _split_candidate_comparator(s)
567
+ if comparator is None:
568
+ return "missing_comparator"
569
+ if comparator == "<=0":
570
+ return "nonpositive_comparator"
571
+ lhs = lhs.strip()
572
+ if not lhs:
573
+ return "empty_lhs"
574
+
575
+ hi_term_count = len(_HI_TERM_RE.findall(lhs))
576
+ if hi_term_count == 1:
577
+ m = _SINGLE_POSITIVE_STEP_RE.fullmatch(lhs)
578
+ if m:
579
+ coeff_str = m.group(1)
580
+ if coeff_str is not None:
581
+ try:
582
+ if float(coeff_str) <= 0:
583
+ return "nonpositive_coeff"
584
+ except ValueError:
585
+ return "bad_coeff"
586
+ if comparator == ">=0":
587
+ return None
588
+ # A Shannon atom is universally nonnegative, not universally zero.
589
+ # Equality output is allowed only when it matches a semantic
590
+ # equality condition in the condition-specific check below.
591
+ base_reason = "unjustified_equality"
592
+ else:
593
+ base_reason = "not_single_hi_term"
594
+ else:
595
+ base_reason = "multiple_terms"
596
+
597
+ if _is_condition_positive_multiple_step(s, n, cond_lines):
598
+ return None
599
+ return base_reason
600
+
601
+
602
+ def _get_cond_matrix_for_run(run_id, n, cond_lines):
603
+ """
604
+ Worker-local cache.
605
+ Cache is valid only within the same run_id (i.e., one tree_search call).
606
+ """
607
+ global _COND_CACHE, _LAST_RUN_ID
608
+
609
+ if run_id != _LAST_RUN_ID:
610
+ _COND_CACHE.clear()
611
+ _LAST_RUN_ID = run_id
612
+
613
+ if not cond_lines:
614
+ return None, None
615
+
616
+ key = (n, tuple(cond_lines))
617
+ cached = _COND_CACHE.get(key)
618
+ if cached is not None:
619
+ return cached
620
+
621
+ res = _parse_condition_matrices(cond_lines, n)
622
+ _COND_CACHE[key] = res
623
+ return res
624
+
625
+ def _generate_elemental_generators(n: int, dtype=np.int8) -> List[csr_matrix]:
626
+ """
627
+ Generate elemental Shannon-type inequalities for n variables:
628
+ 1) H(X | others) >= 0
629
+ 2) I(X;Y | Z) >= 0 for all pairs (X,Y) and all Z subset of remaining vars.
630
+ Return format matches existing code: [A], where A is csr_matrix (#ineq, 2^n-1).
631
+
632
+ Number of inequalities = n + C(n,2)*2^(n-2)
633
+ Complexity ~ O(n^2 * 2^n), much smaller than O(5^n).
634
+ """
635
+ if n in _CACHED_GAMMA:
636
+ return _CACHED_GAMMA[n]
637
+
638
+ # Use a different filename to avoid clashing with exhaustive ones
639
+ pkl_path = _MODEL2_DIR / f"elemental_generators_n{n}.pkl"
640
+ if pkl_path.exists():
641
+ try:
642
+ with open(pkl_path, "rb") as f:
643
+ gamma = pickle.load(f) # expects [A]
644
+ _CACHED_GAMMA[n] = gamma
645
+ sys.stderr.write(f"[beam_debug] Loaded precomputed elemental generators for n={n} from {pkl_path}\n")
646
+ sys.stderr.flush()
647
+ return gamma
648
+ except Exception as e:
649
+ sys.stderr.write(f"[beam_debug] Failed to load {pkl_path}: {e}\n")
650
+ sys.stderr.flush()
651
+
652
+ sys.stderr.write(f"[beam_debug] Generating elemental generators for n={n}...\n")
653
+ sys.stderr.flush()
654
+
655
+ A, _b = elemental_inequalities_sparse(n, dtype=dtype) # A is csr
656
+ gamma = [A]
657
+ _CACHED_GAMMA[n] = gamma
658
+
659
+ # Save to disk
660
+ try:
661
+ with open(pkl_path, "wb") as f:
662
+ pickle.dump(gamma, f, protocol=pickle.HIGHEST_PROTOCOL)
663
+ sys.stderr.write(f"[beam_debug] Saved elemental generators for n={n} to {pkl_path}\n")
664
+ sys.stderr.flush()
665
+ except Exception as e:
666
+ sys.stderr.write(f"[beam_debug] Failed to save {pkl_path}: {e}\n")
667
+ sys.stderr.flush()
668
+
669
+ return gamma
670
+
671
+
672
+ def elemental_inequalities_sparse(n: int, dtype=np.int8):
673
+ """
674
+ Generate basic Shannon-type information measure inequality matrix.
675
+ """
676
+ m = (1 << n) - 1 # Number of columns (entropy variables)
677
+ rows = []
678
+ cols = []
679
+ data = []
680
+
681
+ def add_coeff(row_id: int, mask: int, val: int):
682
+ cols.append(mask - 1) # 0-based
683
+ rows.append(row_id)
684
+ data.append(val)
685
+
686
+ row_id = 0
687
+ # H(X|Z) >= 0
688
+ for x in range(n):
689
+ # X mask
690
+ Xmask = 1 << x
691
+ # Z mask (everything else)
692
+ Zmask = ((1 << n) - 1) & (~Xmask)
693
+
694
+ # Add H(X|Z) >= 0 -> H(X U Z) - H(Z) >= 0
695
+ add_coeff(row_id, Xmask | Zmask, +1)
696
+ if Zmask != 0:
697
+ add_coeff(row_id, Zmask, -1)
698
+ row_id += 1
699
+
700
+ # I(X;Y|Z) >= 0
701
+ for x in range(n):
702
+ for y in range(x + 1, n):
703
+ zmask = 0
704
+ while zmask < (1 << n):
705
+ if (zmask & (1 << x)) == 0 and (zmask & (1 << y)) == 0:
706
+ XZ = zmask | (1 << x)
707
+ YZ = zmask | (1 << y)
708
+ XYZ = zmask | (1 << x) | (1 << y)
709
+ add_coeff(row_id, XZ, +1)
710
+ add_coeff(row_id, YZ, +1)
711
+ add_coeff(row_id, XYZ, -1)
712
+ if zmask != 0:
713
+ add_coeff(row_id, zmask, -1)
714
+ row_id += 1
715
+ zmask += 1
716
+ if (zmask & (1 << x)) != 0:
717
+ zmask += (1 << x)
718
+ if (zmask & (1 << y)) != 0:
719
+ zmask += (1 << y)
720
+
721
+ # Build sparse matrix A and b
722
+ A = coo_matrix((np.array(data, dtype=dtype), (np.array(rows), np.array(cols))),
723
+ shape=(max(rows) + 1, m)).tocsr()
724
+ b = np.zeros(A.shape[0], dtype=float)
725
+ return A, b
726
+
727
+
728
+ def _beam_debug_print_M(*, depth: int, rollout: int, path_exprs: List[str], residual_vec, action: str) -> None:
729
+ """
730
+ Debug-print the current residual and path that will be used in the LP solve/enqueue.
731
+ """
732
+ try:
733
+ m_sz = len(path_exprs) if path_exprs else 0
734
+ residual_str = vector_to_string(residual_vec)
735
+
736
+ sys.stderr.write(
737
+ f"[beam_debug] {action}: depth={depth} rollout={rollout} "
738
+ f"Path_size={m_sz} Residual={residual_str}\n"
739
+ )
740
+ sys.stderr.flush()
741
+ except Exception:
742
+ pass
743
+
744
+
745
+ # def _get_residual_score(vec: csr_matrix) -> float:
746
+ # """
747
+ # Score a residual vector: smaller is better (shorter/simpler expression).
748
+ # Primary: number of terms (nnz).
749
+ # Secondary: sum of absolute coefficients.
750
+ # """
751
+ # if vec is None:
752
+ # return 999999.0
753
+ # # nnz is the number of non-zero joint entropy terms
754
+ # num_terms = vec.nnz
755
+ # # sum of absolute coefficients
756
+ # coeff_sum = np.sum(np.abs(vec.data)) if vec.nnz > 0 else 0
757
+ # return float(num_terms) * 1000.0 + float(coeff_sum)
758
+
759
+ def _get_residual_score(vec: csr_matrix) -> float:
760
+ """
761
+ Score a residual vector for beam pruning: smaller is better.
762
+ Priority:
763
+ 1) fewer terms (sparser expression),
764
+ 2) smaller absolute magnitude on negative coefficients,
765
+ 3) smaller overall absolute magnitude.
766
+ """
767
+ if os.environ.get("BEAM_RANDOM_SCORE", "0") == "1":
768
+ return random.random()
769
+
770
+ if vec is None or vec.nnz == 0:
771
+ return 0.0
772
+
773
+ coeffs = np.asarray(vec.data, dtype=np.float64)
774
+ neg_abs_sum = float(np.abs(coeffs[coeffs < 0]).sum()) if np.any(coeffs < 0) else 0.0
775
+ return neg_abs_sum * 1e3
776
+
777
+
778
+ def _select_beam_nodes(nodes, beam_size: int, *, random_prune: bool, rng=None):
779
+ """Return uniformly sampled or score-ranked kept nodes and their complement."""
780
+ if beam_size <= 0 or len(nodes) <= beam_size:
781
+ return list(nodes), []
782
+ if random_prune:
783
+ rng = rng or random.Random()
784
+ kept_indices = set(rng.sample(range(len(nodes)), beam_size))
785
+ kept = [node for idx, node in enumerate(nodes) if idx in kept_indices]
786
+ pruned = [node for idx, node in enumerate(nodes) if idx not in kept_indices]
787
+ return kept, pruned
788
+
789
+ ranked = sorted(nodes, key=lambda node: _get_residual_score(node.residual_vec))
790
+ return ranked[:beam_size], ranked[beam_size:]
791
+
792
+
793
+ def _is_obviously_negative(vec: csr_matrix) -> bool:
794
+ """
795
+ Check if all coefficients are non-positive and at least one is strictly negative.
796
+ Returns True if the residual can never reach 0 by subtracting non-negative inequalities.
797
+ """
798
+ if vec is None or vec.nnz == 0:
799
+ return False
800
+ # Use a small epsilon for float precision
801
+ eps = 1e-9
802
+ # If any coefficient is significantly positive, it's not 'obviously negative'
803
+ if np.any(vec.data > eps):
804
+ return False
805
+ # If we are here, all coefficients are <= eps.
806
+ # Check if at least one is significantly negative.
807
+ return np.any(vec.data < -eps)
808
+
809
+
810
+ def _is_invalid_negative_ineq(
811
+ cand: str,
812
+ *,
813
+ n: Optional[int] = None,
814
+ cond_lines: Optional[List[str]] = None,
815
+ ) -> bool:
816
+ """
817
+ Check if the candidate is a negative term asserted as >= 0.
818
+ In Information Theory, terms like -I(X;Y) >= 0 are only valid if they are actually 0.
819
+ The training data uses -I(...) = 0.
820
+ """
821
+ s = cand.strip()
822
+ if ">=" in s and s.lstrip().startswith("-"):
823
+ if n is not None and cond_lines and _is_condition_positive_multiple_step(s, n, cond_lines):
824
+ return False
825
+ return True
826
+ return False
827
+
828
+
829
+ def _is_in_row_space(vec: csr_matrix, matrix: Optional[csr_matrix]) -> bool:
830
+ """
831
+ Check if vec is a linear combination of the rows of matrix.
832
+ Uses linprog to solve matrix.T @ y = vec.T
833
+ """
834
+ if vec.nnz == 0:
835
+ return True
836
+ if matrix is None or matrix.shape[0] == 0:
837
+ return False
838
+
839
+ try:
840
+ # Solve A_eq @ y = b_eq
841
+ # where A_eq is matrix.T and b_eq is vec.T
842
+ A_eq = matrix.toarray().T
843
+ b_eq = vec.toarray().flatten()
844
+ c = np.zeros(A_eq.shape[1])
845
+ # Bayesian coefficients can be negative, so bounds=(None, None)
846
+ res = linprog(c, A_eq=A_eq, b_eq=b_eq, bounds=(None, None), method='highs')
847
+ return res.success
848
+ except Exception:
849
+ return False
850
+
851
+
852
+ def _get_involved_vars(vec: csr_matrix) -> set:
853
+ involved_vars = set()
854
+ if vec is None:
855
+ return involved_vars
856
+ for idx in vec.indices:
857
+ subset_idx = idx + 1
858
+ v = 1
859
+ temp = subset_idx
860
+ while temp > 0:
861
+ if temp & 1:
862
+ involved_vars.add(v)
863
+ temp >>= 1
864
+ v += 1
865
+ return involved_vars
866
+
867
+
868
+ def _reduce_vector(vec: csr_matrix, mapping: Dict[int, int], num_vars: int) -> csr_matrix:
869
+ new_indices = []
870
+ for idx in vec.indices:
871
+ subset_idx = idx + 1
872
+ new_subset_idx = 0
873
+ for old_v, new_v in mapping.items():
874
+ if subset_idx & (1 << (old_v - 1)):
875
+ new_subset_idx |= (1 << (new_v - 1))
876
+ new_indices.append(new_subset_idx - 1)
877
+ new_dim = (1 << num_vars) - 1
878
+ return csr_matrix((vec.data, new_indices, [0, len(vec.data)]), shape=(1, new_dim))
879
+
880
+
881
+ def _check_shannon_status(vec: csr_matrix, n_sample: int, Gamma: Optional[List[csr_matrix]] = None,
882
+ cond_eq_matrix: Optional[csr_matrix] = None,
883
+ cond_ge_matrix: Optional[csr_matrix] = None,
884
+ max_vars_for_lp: int = 0,
885
+ debug: bool = False) -> int:
886
+ """
887
+ Check whether the residual has a direct certificate or is flagged by the
888
+ numerical LP false-pruning heuristic.
889
+
890
+ In exact arithmetic, an unbounded minimization supplies a counterexample
891
+ ray. This implementation trusts floating-point HiGHS status and does not
892
+ independently verify that ray, so the result can affect search completeness
893
+ but is never used to accept a proof. LP optimality likewise does not
894
+ terminate search because this routine does not reconstruct an explicit
895
+ conic certificate.
896
+ Returns:
897
+ 1 if directly proved (nearly zero or all coefficients nonnegative)
898
+ 0 if unknown, including an LP-true result without an explicit certificate
899
+ -1 if impossible (False/Prune) - only if n_sample <= max_vars_for_lp
900
+ """
901
+ if vec is None:
902
+ return 0
903
+
904
+ eps = 1e-9
905
+
906
+ # 1. Quick checks (Proved)
907
+ if vec.nnz > 0 and np.all(vec.data >= -eps):
908
+ return 1
909
+ if vec.nnz == 0 or np.allclose(vec.toarray(), 0, atol=eps):
910
+ return 1
911
+
912
+ if max_vars_for_lp > 0:
913
+ involved_res = _get_involved_vars(vec)
914
+
915
+ # Determine if we can do a full check (with conditions) or only a partial check (proved-only)
916
+ can_prune = (n_sample <= max_vars_for_lp)
917
+
918
+ # involved_vars for the LP context
919
+ if can_prune:
920
+ involved = set(range(1, n_sample + 1))
921
+ else:
922
+ # For n_sample > max_vars_for_lp (e.g. 6), we can only prove it if the residual variables fit in max_vars_for_lp
923
+ involved = involved_res
924
+ if len(involved) > max_vars_for_lp:
925
+ return 0
926
+ # If we are here, we are doing a "no-conditions" check to see if it's Shannon-true
927
+ cond_eq_matrix = None
928
+ cond_ge_matrix = None
929
+
930
+ num_vars = len(involved)
931
+ if num_vars == 0: return 0
932
+
933
+ # Select or generate Gamma
934
+ global _CACHED_GAMMA
935
+ if num_vars not in _CACHED_GAMMA:
936
+ _generate_elemental_generators(num_vars)
937
+ gamma_data = _CACHED_GAMMA[num_vars]
938
+ # Handle cached tuple format: ([A], names)
939
+ current_gamma = gamma_data[0] if isinstance(gamma_data, tuple) else gamma_data
940
+
941
+ if debug:
942
+ try:
943
+ type_str = "full" if can_prune else "residual-only"
944
+ sys.stderr.write(
945
+ f"[beam_debug] LP_check_start: n={num_vars} ({type_str}) vec={vector_to_string(vec)}\n")
946
+ sys.stderr.flush()
947
+ except:
948
+ pass
949
+
950
+ sorted_vars = sorted(list(involved))
951
+ mapping = {old: new for new, old in enumerate(sorted_vars, 1)}
952
+ vec_small = _reduce_vector(vec, mapping, num_vars)
953
+
954
+
955
+ try:
956
+ G = vstack(current_gamma) # shape: (num_gen, m)
957
+ c = vec_small.toarray().ravel() # shape: (m,)
958
+
959
+ # A_ub @ y <= b_ub
960
+ # Shannon inequalities: G @ y >= 0 => -G @ y <= 0
961
+ A_ub_list = [-G]
962
+
963
+ if cond_ge_matrix is not None and cond_ge_matrix.shape[0] > 0:
964
+ ge_small_list = []
965
+ for i in range(cond_ge_matrix.shape[0]):
966
+ ge_reduced = _reduce_vector(cond_ge_matrix[i], mapping, num_vars)
967
+ ge_small_list.append(-ge_reduced) # v @ y >= 0 => -v @ y <= 0
968
+ A_ub_list.append(vstack(ge_small_list))
969
+
970
+ A_ub = vstack(A_ub_list)
971
+ b_ub = np.zeros(A_ub.shape[0], dtype=float)
972
+
973
+ A_eq = None
974
+ b_eq = None
975
+ if cond_eq_matrix is not None and cond_eq_matrix.shape[0] > 0:
976
+ cond_small_list = []
977
+ for i in range(cond_eq_matrix.shape[0]):
978
+ c_reduced = _reduce_vector(cond_eq_matrix[i], mapping, num_vars) # (1, m)
979
+ cond_small_list.append(c_reduced)
980
+ A_eq = vstack(cond_small_list) # shape: (num_cond, m)
981
+ b_eq = np.zeros(A_eq.shape[0], dtype=float)
982
+
983
+ # y is free (IMPORTANT)
984
+ bounds = [(None, None)] * G.shape[1]
985
+
986
+ res = linprog(c, A_ub=A_ub, b_ub=b_ub, A_eq=A_eq, b_eq=b_eq,
987
+ bounds=bounds, method="highs")
988
+
989
+ if res.status == 3:
990
+ # Numerically unbounded. In exact arithmetic this means the
991
+ # residual is false over the constrained Shannon outer cone.
992
+ if debug:
993
+ sys.stderr.write("[beam_debug] lp_dual: numerically UNBOUNDED => false-prune\n")
994
+ sys.stderr.flush()
995
+ return -1 if can_prune else 0
996
+
997
+ if res.status == 0 and res.success:
998
+ # optimal => should be ~0 => TRUE
999
+ if debug:
1000
+ sys.stderr.write(f"[beam_debug] lp_dual: OPTIMAL fun={res.fun}\n")
1001
+ sys.stderr.flush()
1002
+
1003
+ # res.fun might be -1e-12 numerically, treat as 0
1004
+ if res.fun < -eps:
1005
+ # Theoretically this means it should be unbounded but solver didn't report it
1006
+ return -1 if can_prune else 0
1007
+
1008
+ return 0
1009
+
1010
+ # Other cases: iteration limit/numerical issues (unsafe to prune)
1011
+ if debug:
1012
+ sys.stderr.write(f"[beam_debug] lp_dual: status={res.status} msg={res.message}\n")
1013
+ sys.stderr.flush()
1014
+ return 0
1015
+
1016
+ except Exception as e:
1017
+ if debug:
1018
+ sys.stderr.write(f"[beam_debug] LP error: {e}\n")
1019
+ return 0
1020
+
1021
+ return 0
1022
+
1023
+
1024
+ def _is_positive_polymatroid(vec: csr_matrix, n_sample: int, Gamma: Optional[List[csr_matrix]] = None,
1025
+ cond_eq_matrix: Optional[csr_matrix] = None,
1026
+ cond_ge_matrix: Optional[csr_matrix] = None,
1027
+ max_vars_for_lp: int = 0, debug: bool = False) -> bool:
1028
+ """
1029
+ Check if the residual is a positive polymatroid (Shannon inequality).
1030
+ Returns True if vec >= 0 is already proved or follows from Gamma.
1031
+ """
1032
+ return _check_shannon_status(vec, n_sample, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug) == 1
1033
+
1034
+
1035
+ def _normalize_to_braces(s: str) -> str:
1036
+ """
1037
+ Ensure all H(...) and I(...) terms use X_{...} notation.
1038
+ Example: H(X_1,2) -> H(X_{1,2}), I(X_1;X_2) -> I(X_{1};X_{2}).
1039
+ """
1040
+ def repl_X(m):
1041
+ inside = m.group(1)
1042
+ if inside.startswith('{') and inside.endswith('}'):
1043
+ return m.group(0)
1044
+ # Split by comma to handle X_1,2
1045
+ parts = [p.strip() for p in inside.split(',') if p.strip()]
1046
+ return "X_{" + ",".join(parts) + "}"
1047
+
1048
+ # Replace all occurrences of X_ followed by digits/commas OR X_{...}
1049
+ return re.sub(r'X_(\{?[0-9,]+\}?)', repl_X, s)
1050
+
1051
+
1052
+ def parse_cand_to_vector(cand: str, n: int) -> csr_matrix:
1053
+ """
1054
+ Robustly parse an inequality candidate string from the LLM into a vector.
1055
+ Handles coefficients (including 8*H(...) or 8H(...)), signs, and conditional terms.
1056
+ """
1057
+ # 1. Strip comparator and all whitespace/asterisks
1058
+ s = _strip_comparator(cand).replace(" ", "").replace("*", "")
1059
+
1060
+ # 2. Normalize braces to ensure consistency (X_1 -> X_{1})
1061
+ s = _normalize_to_braces(s)
1062
+
1063
+ def _parse_linear_expr(expr: str) -> csr_matrix:
1064
+ # Try string_to_vector first (handles linear combinations of joint entropies)
1065
+ try:
1066
+ return string_to_vector(expr, n)
1067
+ except Exception:
1068
+ pass
1069
+
1070
+ # Handle single term with potential coefficient for polym_ineq_to_vector
1071
+ # Pattern: optional sign and coefficient, followed by a term starting with H( or I(
1072
+ m = re.match(r'^([+-]?\d*\.?\d*)([HI]\(.*\))$', expr)
1073
+ if m:
1074
+ coeff_str, term_expr = m.groups()
1075
+ coeff = 1.0
1076
+ if coeff_str == "-":
1077
+ coeff = -1.0
1078
+ elif coeff_str and coeff_str != "+":
1079
+ try:
1080
+ coeff = float(coeff_str)
1081
+ except ValueError:
1082
+ coeff = 1.0
1083
+ try:
1084
+ # polym_ineq_to_vector handles H(A|B), I(A;B), I(A;B|C)
1085
+ return coeff * polym_ineq_to_vector(term_expr, n)
1086
+ except Exception:
1087
+ pass
1088
+
1089
+ # Final fallback: try polym_ineq_to_vector on the whole thing
1090
+ return polym_ineq_to_vector(expr, n)
1091
+
1092
+ # 3. Handle leading scalar times parenthesized expression, e.g. 2*(...), -3(...)
1093
+ m = re.match(r'^([+-]?\d*\.?\d*)\((.*)\)$', s)
1094
+ if m:
1095
+ coeff_str, inner_expr = m.groups()
1096
+ coeff = 1.0
1097
+ if coeff_str == "-":
1098
+ coeff = -1.0
1099
+ elif coeff_str and coeff_str != "+":
1100
+ try:
1101
+ coeff = float(coeff_str)
1102
+ except ValueError:
1103
+ coeff = 1.0
1104
+ inner_vec = _parse_linear_expr(inner_expr)
1105
+ return coeff * inner_vec
1106
+
1107
+ # 4. Normal linear expression path
1108
+ return _parse_linear_expr(s)
1109
+
1110
+
1111
+ def _extract_var_indices(expr: str) -> List[int]:
1112
+ """
1113
+ Extract all variable indices appearing as X_{...} or X_... in an expression.
1114
+ """
1115
+ idxs: List[int] = []
1116
+ if not expr:
1117
+ return idxs
1118
+ # X_{1,2,3}
1119
+ for inside in re.findall(r"X_\{([^}]+)\}", expr):
1120
+ parts = [p.strip() for p in inside.split(",") if p.strip()]
1121
+ for p in parts:
1122
+ if p.isdigit():
1123
+ idxs.append(int(p))
1124
+ # X_1 (avoid double counting those already in braces; ok if duplicates)
1125
+ for m in re.findall(r"X_(\d+)\b", expr):
1126
+ try:
1127
+ idxs.append(int(m))
1128
+ except Exception:
1129
+ pass
1130
+ return idxs
1131
+
1132
+
1133
+ def _strip_comparator(s: str) -> str:
1134
+ # Keep only LHS if input contains "...>=0", "...>0", "...<=0", or "...=0"
1135
+ s = s.strip()
1136
+ for token in (">=0", ">0", "<=0", "=0"):
1137
+ if token in s:
1138
+ s = s.split(token, 1)[0]
1139
+ # Also handle spaced variants like ">= 0"
1140
+ s = re.sub(r'(>=|>|<=|=)\s*0\s*$', '', s).strip()
1141
+ return s
1142
+
1143
+
1144
+ def _build_llm_prompt(
1145
+ *,
1146
+ n: int,
1147
+ min_idx: int,
1148
+ max_idx: int,
1149
+ target_inequality_normalized: str,
1150
+ cond_lines: List[str],
1151
+ m_exprs: List[str],
1152
+ idx_to_letter: Optional[Dict[int, str]] = None,
1153
+ ) -> str:
1154
+ """
1155
+ Build prompt matching the prepared DPO training data format.
1156
+ """
1157
+ # 1. Format Target (ensure comparator)
1158
+ target = target_inequality_normalized.strip()
1159
+ if not any(op in target for op in [">=0", ">0", "<=0", "=0"]):
1160
+ target += ">=0"
1161
+ if _use_alpha_prompt():
1162
+ target = _convert_old_to_alpha(target, idx_to_letter)
1163
+
1164
+ # 2. Format Conditions (use NONE when empty)
1165
+ valid_conds = []
1166
+ for c in (cond_lines or []):
1167
+ c = c.strip()
1168
+ if not c:
1169
+ continue
1170
+ if c in {"(none)", "NONE", "NONE=0"}:
1171
+ continue
1172
+ valid_conds.append(_convert_old_to_alpha(c, idx_to_letter) if _use_alpha_prompt() else c)
1173
+
1174
+ cond_str = "NONE" if not valid_conds else "\n".join(valid_conds)
1175
+
1176
+ # 3. Assemble prompt in DPO 'input' format
1177
+ prompt = f"TARGET:\n{target}\n\n"
1178
+ prompt += f"CONDITIONS:\n{cond_str}\n\n"
1179
+
1180
+ return prompt
1181
+
1182
+
1183
+ def _gpu_worker_init(adapter_path: Optional[str], use_dummy: bool, gpu_id: int, max_vars_for_lp: int = 0, Gamma: Optional[List[csr_matrix]] = None):
1184
+ """
1185
+ One worker process binds to one GPU and loads the model once.
1186
+ """
1187
+ # Must be set before importing torch/transformers for correct device visibility.
1188
+ os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu_id)
1189
+ os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
1190
+
1191
+ global _GPU_ADAPTER_PATH, _GPU_USE_DUMMY, _GPU_READY, _CACHED_GAMMA
1192
+ _GPU_ADAPTER_PATH = os.path.abspath(adapter_path) if adapter_path else None
1193
+ _GPU_USE_DUMMY = bool(use_dummy)
1194
+
1195
+ if not _GPU_USE_DUMMY:
1196
+ # Warm model load once per worker process.
1197
+ try:
1198
+ from llm_api_compat import _load_local_model
1199
+ except ModuleNotFoundError:
1200
+ from beam_search_model2.llm_api_compat import _load_local_model
1201
+ # adapter_path may be None to use the baseline model (no PEFT adapter).
1202
+ _load_local_model(adapter_path=_GPU_ADAPTER_PATH, device_map="auto")
1203
+
1204
+ if max_vars_for_lp > 0:
1205
+ if Gamma is not None:
1206
+ # Use the Gamma matrix passed from the main process
1207
+ _CACHED_GAMMA[max_vars_for_lp] = Gamma
1208
+ sys.stderr.write(f"[beam_debug] GPU worker {gpu_id} using Gamma matrix passed from main process (max_vars_for_lp={max_vars_for_lp}).\n")
1209
+ sys.stderr.flush()
1210
+ else:
1211
+ # Pre-generate Shannon inequalities once per worker
1212
+ sys.stderr.write(f"[beam_debug] GPU worker {gpu_id} pre-generating Shannon generators for max_vars_for_lp={max_vars_for_lp}...\n")
1213
+ sys.stderr.flush()
1214
+ gamma_list = _generate_elemental_generators(max_vars_for_lp)
1215
+ shape_str = f" Shape: {gamma_list[0].shape}" if gamma_list else ""
1216
+ sys.stderr.write(f"[beam_debug] GPU worker {gpu_id} done pre-generating Shannon generators.{shape_str}\n")
1217
+ sys.stderr.flush()
1218
+
1219
+ _GPU_READY = True
1220
+
1221
+
1222
+ def _expand_node_payload(payload: Dict) -> Dict:
1223
+ """
1224
+ Worker-side expansion: starting from residual_vec, propose k inequalities and subtract them.
1225
+ Returns a dict with a list of potential child states.
1226
+ Note: All heavy modules (numpy, scipy, llm_api_interface) are imported at the top level
1227
+ of this file and will be already available in the worker process's memory.
1228
+ """
1229
+ n = int(payload["n"])
1230
+ min_idx = int(payload["min_idx"])
1231
+ max_idx = int(payload["max_idx"])
1232
+ run_id = payload.get("run_id")
1233
+ cond_lines = payload["cond_lines"]
1234
+ base_path_exprs = list(payload.get("path_exprs") or [])
1235
+ base_residual_vec = payload.get("residual_vec", None)
1236
+ k_steps = max(0, int(payload.get("k_per_call", 1)))
1237
+ llm_mode = (payload.get("llm_mode") or "sequential").strip().lower()
1238
+ max_tries_per_step = int(payload.get("max_tries_per_step", 3))
1239
+
1240
+ adapter_path = payload.get("adapter_path", None)
1241
+ use_dummy = bool(payload.get("use_dummy", False))
1242
+ Gamma = payload.get("Gamma", None)
1243
+ max_vars_for_lp = int(payload.get("max_vars_for_lp", 0))
1244
+ idx_to_letter = payload.get("idx_to_letter", None)
1245
+ letter_to_idx = payload.get("letter_to_idx", None)
1246
+
1247
+ if Gamma is None and max_vars_for_lp > 0:
1248
+ # Use worker-local cached Gamma (already initialized in _gpu_worker_init)
1249
+ Gamma = _CACHED_GAMMA.get(max_vars_for_lp, None)
1250
+
1251
+ debug = os.environ.get("BEAM_DEBUG", "0") == "1"
1252
+
1253
+ log_wrong_pruned = bool(payload.get("log_wrong_pruned", False))
1254
+ wrong_data = [] # List of [input_str, output_str]
1255
+
1256
+ # Parse conditions into matrices for row-space and LP checks
1257
+ # Worker-local cache (_COND_CACHE) ensures we don't re-parse for every node.
1258
+ cond_eq_matrix, cond_ge_matrix = _get_cond_matrix_for_run(run_id, n, cond_lines)
1259
+ # if cond_lines and cond_lines != ["NONE"]:
1260
+ # try:
1261
+ # eq_vecs = []
1262
+ # ge_vecs = []
1263
+ # for cl in cond_lines:
1264
+ # try:
1265
+ # # Note: cand might already have >=0 or =0 from tree_search's cond_lines
1266
+ # # but parse_cand_to_vector calls _strip_comparator internally.
1267
+ # v = parse_cand_to_vector(cl, n)
1268
+ # if v.nnz == 0: continue
1269
+ # if ">=0" in cl:
1270
+ # ge_vecs.append(v)
1271
+ # elif "=0" in cl:
1272
+ # eq_vecs.append(v)
1273
+ # else:
1274
+ # # Fallback if no comparator found in string (shouldn't happen with latest logic)
1275
+ # eq_vecs.append(v)
1276
+ # except: continue
1277
+ # if eq_vecs:
1278
+ # cond_eq_matrix = vstack(eq_vecs)
1279
+ # if ge_vecs:
1280
+ # cond_ge_matrix = vstack(ge_vecs)
1281
+ # except:
1282
+ # pass
1283
+
1284
+ # Check status: 1=proved, -1=impossible, 0=unknown
1285
+ status = _check_shannon_status(base_residual_vec, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug=debug)
1286
+ if debug:
1287
+ sys.stderr.write(f"[beam_debug] worker_root_status: {status}\n")
1288
+ sys.stderr.flush()
1289
+
1290
+ if status == 1:
1291
+ return {
1292
+ "ok": True,
1293
+ "results": [{
1294
+ "proved": True,
1295
+ "path_exprs": base_path_exprs,
1296
+ "residual_vec": base_residual_vec,
1297
+ "new_ineq": None
1298
+ }],
1299
+ "parse_fail": 0,
1300
+ "out_of_range": 0,
1301
+ }
1302
+ # User requested: No pruning at root node. Let search proceed even if status is -1.
1303
+
1304
+ results = []
1305
+ pruned_results = []
1306
+ parse_fail = 0
1307
+ out_of_range = 0
1308
+ llm_usage = _empty_llm_usage()
1309
+
1310
+ if llm_mode == "sequential":
1311
+ # Sequential: Subtract k things one after another. Return ONE branch.
1312
+ current_residual_vec = base_residual_vec
1313
+ current_path_exprs = list(base_path_exprs)
1314
+ subtracted_any = False
1315
+
1316
+ for _step in range(k_steps):
1317
+ got_one = False
1318
+ current_target_norm = vector_to_string(current_residual_vec)
1319
+
1320
+ for _try in range(max_tries_per_step):
1321
+ prompt = _build_llm_prompt(
1322
+ n=n, min_idx=min_idx, max_idx=max_idx,
1323
+ target_inequality_normalized=current_target_norm,
1324
+ cond_lines=cond_lines, m_exprs=current_path_exprs,
1325
+ idx_to_letter=idx_to_letter,
1326
+ )
1327
+ cand_list = LLM_propose_inequalities(
1328
+ prompt=prompt, k=1, adapter_path=adapter_path, use_dummy=use_dummy,
1329
+ )
1330
+ _merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
1331
+ cand_raw = cand_list[0] if cand_list else ""
1332
+ cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
1333
+ cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
1334
+ format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
1335
+ if not cand or cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
1336
+ if log_wrong_pruned and cand_raw:
1337
+ wrong_data.append([prompt, cand_raw])
1338
+ if debug and cand_raw:
1339
+ sys.stderr.write(
1340
+ f"[beam_debug] invalid_or_wrong_cand: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
1341
+ )
1342
+ sys.stderr.flush()
1343
+ if format_reason is not None:
1344
+ llm_usage["format_fail_total"] += 1
1345
+ pruned_results.append({"is_pruned": True, "reason": "invalid_or_wrong_cand", "path_exprs": current_path_exprs + [cand]})
1346
+ continue
1347
+
1348
+ cand_indices = _extract_var_indices(cand)
1349
+ if cand_indices and (min(cand_indices) < min_idx or max(cand_indices) > max_idx):
1350
+ if log_wrong_pruned:
1351
+ wrong_data.append([prompt, cand_raw])
1352
+ out_of_range += 1
1353
+ pruned_results.append({"is_pruned": True, "reason": "out_of_range", "path_exprs": current_path_exprs + [cand]})
1354
+ continue
1355
+ try:
1356
+ vec = parse_cand_to_vector(cand, n)
1357
+ except (ValueError, Exception):
1358
+ if log_wrong_pruned:
1359
+ wrong_data.append([prompt, cand_raw])
1360
+ parse_fail += 1
1361
+ pruned_results.append({"is_pruned": True, "reason": "parse_fail", "path_exprs": current_path_exprs + [cand]})
1362
+ continue
1363
+
1364
+ # Validation for Bayesian equalities
1365
+ if re.search(r'(?<![<>])=\s*0', cand):
1366
+ if not _is_in_row_space(vec, cond_eq_matrix):
1367
+ if log_wrong_pruned:
1368
+ wrong_data.append([prompt, cand_raw])
1369
+ if debug:
1370
+ sys.stderr.write(f"[beam_debug] hallucinated_equality: step={cand_raw} not in row-space of conditions\n")
1371
+ sys.stderr.flush()
1372
+ pruned_results.append({"is_pruned": True, "reason": "hallucinated_equality", "path_exprs": current_path_exprs + [cand]})
1373
+ continue # Hallucinated equality
1374
+ else:
1375
+ if debug:
1376
+ sys.stderr.write(f"[beam_debug] equality_verified: step={cand_raw} is in row-space\n")
1377
+ sys.stderr.flush()
1378
+
1379
+ new_residual = current_residual_vec - vec
1380
+ # Check status: 1=proved, -1=impossible (false)
1381
+ status = _check_shannon_status(new_residual, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug=debug)
1382
+
1383
+ # Check if obviously negative (simple check)
1384
+ if _is_obviously_negative(new_residual):
1385
+ if log_wrong_pruned:
1386
+ wrong_data.append([prompt, cand_raw])
1387
+ if debug:
1388
+ res_str = vector_to_string(new_residual)
1389
+ sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
1390
+ sys.stderr.flush()
1391
+ pruned_results.append({"is_pruned": True, "reason": "obviously_negative", "residual_vec": new_residual, "path_exprs": current_path_exprs + [cand]})
1392
+ got_one = False # Don't continue this branch
1393
+ break
1394
+
1395
+ # Prune if impossible via LP
1396
+ if status == -1:
1397
+ if log_wrong_pruned:
1398
+ wrong_data.append([prompt, cand_raw])
1399
+ if debug:
1400
+ res_str = vector_to_string(new_residual)
1401
+ sys.stderr.write(f"[beam_debug] lp_false_prune: residual={res_str}\n")
1402
+ sys.stderr.flush()
1403
+ pruned_results.append({"is_pruned": True, "reason": "lp_false", "residual_vec": new_residual, "path_exprs": current_path_exprs + [cand]})
1404
+ got_one = False
1405
+ break
1406
+
1407
+ current_path_exprs.append(cand)
1408
+ current_residual_vec = new_residual
1409
+ subtracted_any = True
1410
+ got_one = True
1411
+
1412
+ if status == 1:
1413
+ break
1414
+ break
1415
+ if not got_one: break
1416
+ if _is_positive_polymatroid(current_residual_vec, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp): break
1417
+
1418
+ if subtracted_any:
1419
+ results.append({
1420
+ "proved": _is_positive_polymatroid(current_residual_vec, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp),
1421
+ "path_exprs": current_path_exprs,
1422
+ "residual_vec": current_residual_vec,
1423
+ "new_ineq": current_path_exprs[len(base_path_exprs):]
1424
+ })
1425
+ else:
1426
+ # Batch: Ask for k things. Each becomes a SEPARATE branch (subtracting only one).
1427
+ current_target_norm = vector_to_string(base_residual_vec)
1428
+
1429
+ tries = 0
1430
+ while len(results) < k_steps and tries < max_tries_per_step:
1431
+ tries += 1
1432
+ prompt = _build_llm_prompt(
1433
+ n=n, min_idx=min_idx, max_idx=max_idx,
1434
+ target_inequality_normalized=current_target_norm,
1435
+ cond_lines=cond_lines, m_exprs=base_path_exprs,
1436
+ idx_to_letter=idx_to_letter,
1437
+ )
1438
+ cand_list = LLM_propose_inequalities(
1439
+ prompt=prompt, k=k_steps, adapter_path=adapter_path, use_dummy=use_dummy,
1440
+ num_return_sequences=k_steps,
1441
+ )
1442
+ _merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
1443
+ for cand in cand_list:
1444
+ if len(results) >= k_steps: break
1445
+ cand_raw = cand
1446
+ cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
1447
+ cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
1448
+ format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
1449
+ if not cand or cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
1450
+ if log_wrong_pruned and cand_raw:
1451
+ wrong_data.append([prompt, cand_raw])
1452
+ if debug and cand_raw:
1453
+ sys.stderr.write(
1454
+ f"[beam_debug] invalid_or_wrong_cand: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
1455
+ )
1456
+ sys.stderr.flush()
1457
+ if format_reason is not None:
1458
+ llm_usage["format_fail_total"] += 1
1459
+ pruned_results.append({"is_pruned": True, "reason": "invalid_or_wrong_cand", "path_exprs": base_path_exprs + [cand]})
1460
+ continue
1461
+
1462
+ cand_indices = _extract_var_indices(cand)
1463
+ if cand_indices and (min(cand_indices) < min_idx or max(cand_indices) > max_idx):
1464
+ if log_wrong_pruned:
1465
+ wrong_data.append([prompt, cand_raw])
1466
+ out_of_range += 1
1467
+ pruned_results.append({"is_pruned": True, "reason": "out_of_range", "path_exprs": base_path_exprs + [cand]})
1468
+ continue
1469
+ try:
1470
+ vec = parse_cand_to_vector(cand, n)
1471
+ except (ValueError, Exception):
1472
+ if log_wrong_pruned:
1473
+ wrong_data.append([prompt, cand_raw])
1474
+ parse_fail += 1
1475
+ pruned_results.append({"is_pruned": True, "reason": "parse_fail", "path_exprs": base_path_exprs + [cand]})
1476
+ continue
1477
+
1478
+ # Validation for Bayesian equalities
1479
+ if re.search(r'(?<![<>])=\s*0', cand):
1480
+ if not _is_in_row_space(vec, cond_eq_matrix):
1481
+ if log_wrong_pruned:
1482
+ wrong_data.append([prompt, cand_raw])
1483
+ if debug:
1484
+ sys.stderr.write(f"[beam_debug] hallucinated_equality: step={cand_raw} not in row-space of conditions\n")
1485
+ sys.stderr.flush()
1486
+ pruned_results.append({"is_pruned": True, "reason": "hallucinated_equality", "path_exprs": base_path_exprs + [cand]})
1487
+ continue # Hallucinated equality
1488
+ else:
1489
+ if debug:
1490
+ sys.stderr.write(f"[beam_debug] equality_verified: step={cand_raw} is in row-space\n")
1491
+ sys.stderr.flush()
1492
+
1493
+ new_residual = base_residual_vec - vec
1494
+ # Check status: 1=proved, -1=impossible (false)
1495
+ status = _check_shannon_status(new_residual, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug=debug)
1496
+ is_proved = (status == 1)
1497
+
1498
+ if not is_proved:
1499
+ # 1. Simple obviously negative check
1500
+ if _is_obviously_negative(new_residual):
1501
+ if log_wrong_pruned:
1502
+ wrong_data.append([prompt, cand_raw])
1503
+ if debug:
1504
+ res_str = vector_to_string(new_residual)
1505
+ sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
1506
+ sys.stderr.flush()
1507
+ pruned_results.append({"is_pruned": True, "reason": "obviously_negative", "residual_vec": new_residual, "path_exprs": base_path_exprs + [cand]})
1508
+ continue # Prune this candidate
1509
+
1510
+ # 2. LP impossible check
1511
+ if status == -1:
1512
+ if log_wrong_pruned:
1513
+ wrong_data.append([prompt, cand_raw])
1514
+ if debug:
1515
+ res_str = vector_to_string(new_residual)
1516
+ sys.stderr.write(f"[beam_debug] lp_false_prune: residual={res_str}\n")
1517
+ sys.stderr.flush()
1518
+ pruned_results.append({"is_pruned": True, "reason": "lp_false", "residual_vec": new_residual, "path_exprs": base_path_exprs + [cand]})
1519
+ continue # Prune this candidate
1520
+
1521
+ results.append({
1522
+ "proved": is_proved,
1523
+ "path_exprs": base_path_exprs + [cand],
1524
+ "residual_vec": new_residual,
1525
+ "new_ineq": [cand]
1526
+ })
1527
+
1528
+ return {
1529
+ "ok": len(results) > 0,
1530
+ "results": results,
1531
+ "pruned_results": pruned_results,
1532
+ "parse_fail": parse_fail,
1533
+ "out_of_range": out_of_range,
1534
+ "wrong_data": wrong_data,
1535
+ "llm_usage": llm_usage,
1536
+ }
1537
+
1538
+ # Define Node class
1539
+
1540
+ class Node:
1541
+ def __init__(
1542
+ self,
1543
+ residual_vec: Optional[csr_matrix] = None,
1544
+ path_exprs: Optional[List[str]] = None,
1545
+ proved: bool = False,
1546
+ children=None,
1547
+ parent=None,
1548
+ score: float = 0.0,
1549
+ is_pruned: bool = False,
1550
+ ):
1551
+ if children is None:
1552
+ children = []
1553
+ if path_exprs is None:
1554
+ path_exprs = []
1555
+
1556
+ self.residual_vec = residual_vec
1557
+ self.path_exprs = list(path_exprs)
1558
+ self.proved = proved
1559
+ self.children = children
1560
+ self.parent = parent
1561
+ self.score = score
1562
+ self.is_pruned = is_pruned
1563
+
1564
+ def relabel_f_and_conditions(
1565
+ f_str: str,
1566
+ cond_list: List[str],
1567
+ ) -> Tuple[str, List[str], Dict[int, int], int]:
1568
+ """
1569
+ 对 f_target 和多条 conditions 一起做变量重编号。
1570
+ 要求输入里变量形式都是 X_{...},例如:
1571
+ H(X_{1,3}) + H(X_{1,23}) >= 0
1572
+ H(X_{1,2}) >= 0
1573
+ H(X_{3}) = 0 (X_{3} 也可以)
1574
+
1575
+ 步骤:
1576
+ 1. 从 f_str + cond_list 里找出所有出现的 index(1,3,23,...)
1577
+ 2. 按升序排序,映射到 1..n
1578
+ 3. 把所有 X_{...} 中的旧 index 换成新 index
1579
+
1580
+ 返回:
1581
+ f_new: 重编号后的 f 字符串
1582
+ conds_new: 重编号后的 condition 字符串列表
1583
+ mapping: {旧 index : 新 index}
1584
+ n: 变量个数(新编号的最大值)
1585
+ """
1586
+
1587
+ # 1) 收集所有出现的旧变量下标
1588
+ all_exprs = [f_str] + list(cond_list)
1589
+ old_indices = set()
1590
+
1591
+ # Normalize expressions to brace format first (H(X_1,X_2) -> H(X_{1,2}))
1592
+ def normalize_to_braces(s):
1593
+ """Convert H(X_1,X_2) to H(X_{1,2}) format"""
1594
+ def repl_no_braces(match):
1595
+ prefix = match.group(1) or '' # Coefficient like "-35*"
1596
+ content = match.group(2) # X_1,X_2 or X_1
1597
+ if content.startswith('{') and content.endswith('}'):
1598
+ return match.group(0) # Already normalized
1599
+ if ',' in content:
1600
+ vars_list = [v.strip().replace('X_', '') for v in content.split(',')]
1601
+ normalized = ','.join(vars_list)
1602
+ return f"{prefix}H(X_{{{normalized}}})"
1603
+ else:
1604
+ var = content.replace('X_', '')
1605
+ return f"{prefix}H(X_{{{var}}})"
1606
+ return re.sub(r'([+-]?\d*\.?\d*\*?)H\(X_([^)]+)\)', repl_no_braces, s)
1607
+
1608
+ # Normalize all expressions first
1609
+ all_exprs_normalized = [normalize_to_braces(s) for s in all_exprs]
1610
+
1611
+ for s in all_exprs_normalized:
1612
+ # 匹配 X_{...},拿里面的 "1,3" 之类 (after normalization, all are in brace format)
1613
+ for inside in re.findall(r'X_\{([^}]+)\}', s):
1614
+ parts = [p.strip() for p in inside.split(',') if p.strip()]
1615
+ for p in parts:
1616
+ old_indices.add(int(p))
1617
+
1618
+ if not old_indices:
1619
+ # 没有变量,直接返回原表达式
1620
+ mapping: Dict[int, int] = {}
1621
+ return f_str, cond_list, mapping, 0
1622
+
1623
+ # 2) 排序后映射到 1..n
1624
+ sorted_old = sorted(old_indices) # 例如 [1,3,23]
1625
+ mapping: Dict[int, int] = {
1626
+ old: new for new, old in enumerate(sorted_old, start=1)
1627
+ }
1628
+ n = len(sorted_old)
1629
+
1630
+ # 3) 用 mapping 替换表达式中的 index
1631
+ def repl(match: re.Match) -> str:
1632
+ inside = match.group(1) # 原来的 "1,3" 或 "23"
1633
+ parts = [p.strip() for p in inside.split(',') if p.strip()]
1634
+ new_parts = [str(mapping[int(p)]) for p in parts]
1635
+ return f"X_{{{','.join(new_parts)}}}"
1636
+
1637
+ all_exprs_relabel = [
1638
+ re.sub(r'X_\{([^}]+)\}', repl, s)
1639
+ for s in all_exprs_normalized
1640
+ ]
1641
+
1642
+ f_new = all_exprs_relabel[0]
1643
+ conds_new = all_exprs_relabel[1:]
1644
+
1645
+ return f_new, conds_new, mapping, n
1646
+
1647
+ # Define the algorithm
1648
+ def tree_search(root_node, max_depth, rollout_count, target_inequality, conditions, n,
1649
+ Gamma: Optional[List[csr_matrix]] = None,
1650
+ adapter_path: Optional[str] = None,
1651
+ use_dummy: bool = False,
1652
+ k_per_call: int = 1,
1653
+ llm_mode: str = "sequential",
1654
+ lp_workers: int = 0,
1655
+ gpu_devices: Optional[List[int]] = None,
1656
+ gpu_executors: Optional[List[ProcessPoolExecutor]] = None,
1657
+ beam_size: int = 0,
1658
+ random_prune: bool = False,
1659
+ prune_seed: Optional[int] = None,
1660
+ max_vars_for_lp: int = 0,
1661
+ log_wrong_pruned: bool = False,
1662
+ wrong_pruned_path: Optional[str] = None,
1663
+ sample_time_budget_sec: float = 0.0,
1664
+ sample_tree_budget: int = 0,
1665
+ sample_llm_calls_budget: int = 0,
1666
+ sample_llm_fail_budget: int = 0,
1667
+ depth1_rollouts: Optional[int] = None):
1668
+ """
1669
+ Algorithm for Data Generation via LLM-Guided Tree Search (Model 2: Subtraction based).
1670
+ """
1671
+ _seed_indices = _extract_var_indices(target_inequality)
1672
+ for _c in (conditions or []):
1673
+ _seed_indices.extend(_extract_var_indices(_c))
1674
+ if _seed_indices:
1675
+ min_idx = min(_seed_indices)
1676
+ max_idx = max(_seed_indices)
1677
+ else:
1678
+ min_idx, max_idx = 1, int(n)
1679
+
1680
+ min_idx = max(1, int(min_idx))
1681
+ max_idx = min(int(n), int(max_idx))
1682
+ if min_idx > max_idx:
1683
+ min_idx, max_idx = 1, int(n)
1684
+
1685
+ debug = os.environ.get("BEAM_DEBUG", "0") == "1"
1686
+ prune_rng = None
1687
+ if beam_size and beam_size > 0 and random_prune:
1688
+ prune_rng = random.Random(prune_seed)
1689
+
1690
+ target_inequality = _normalize_to_braces(target_inequality)
1691
+ conditions = [
1692
+ _normalize_constraint_comparator(_normalize_to_braces(c), add_default=True)
1693
+ for c in (conditions or [])
1694
+ ]
1695
+ idx_to_letter, letter_to_idx = _build_alpha_mapping([target_inequality] + list(conditions or []))
1696
+
1697
+ if Gamma is None and max_vars_for_lp > 0:
1698
+ Gamma = _generate_elemental_generators(max_vars_for_lp)
1699
+
1700
+ f_csr = parse_cand_to_vector(target_inequality, n)
1701
+ cond_eq_csr, cond_ge_csr = _parse_condition_matrices(conditions, n)
1702
+
1703
+ root_node.residual_vec = f_csr
1704
+ root_node.path_exprs = []
1705
+
1706
+ seen_residuals = set()
1707
+ root_key = vector_to_string(root_node.residual_vec)
1708
+ seen_residuals.add(root_key)
1709
+
1710
+ nodes = [root_node]
1711
+ tree = [root_node]
1712
+ lp_calls = 0
1713
+ parse_fail = 0
1714
+ out_of_range = 0
1715
+ all_wrong_data = []
1716
+ llm_usage = _empty_llm_usage()
1717
+ sample_time_budget_sec = float(sample_time_budget_sec or 0.0)
1718
+ sample_tree_budget = int(sample_tree_budget or 0)
1719
+ sample_llm_calls_budget = int(sample_llm_calls_budget or 0)
1720
+ sample_llm_fail_budget = int(sample_llm_fail_budget or 0)
1721
+ depth1_rollouts = int(depth1_rollouts) if depth1_rollouts and int(depth1_rollouts) > 0 else None
1722
+ stop_reason = None
1723
+ t_start = time.monotonic()
1724
+
1725
+ llm_mode = (llm_mode or "sequential").strip().lower()
1726
+ run_id = uuid.uuid4().hex
1727
+ global _CACHED_GPU_EXECUTORS, _CACHED_GPU_PARAMS
1728
+ gpu_devices = list(gpu_devices) if gpu_devices else []
1729
+
1730
+ def _flush_wrong_data() -> None:
1731
+ if log_wrong_pruned and wrong_pruned_path and all_wrong_data:
1732
+ try:
1733
+ with open(wrong_pruned_path, "a", encoding="utf-8") as f:
1734
+ for inp_str, out_str in all_wrong_data:
1735
+ f.write(json.dumps({"input": inp_str, "output": out_str}, ensure_ascii=False) + "\n")
1736
+ all_wrong_data.clear()
1737
+ except Exception as e:
1738
+ if debug:
1739
+ sys.stderr.write(f"[beam_debug] error_writing_wrong_data: {e}\n")
1740
+ sys.stderr.flush()
1741
+
1742
+ def _budget_exceeded() -> Optional[str]:
1743
+ if sample_time_budget_sec > 0 and (time.monotonic() - t_start) > sample_time_budget_sec:
1744
+ return "time_budget"
1745
+ if sample_tree_budget > 0 and len(tree) >= sample_tree_budget:
1746
+ return "tree_budget"
1747
+ if sample_llm_calls_budget > 0 and llm_usage.get("llm_calls", 0) >= sample_llm_calls_budget:
1748
+ return "llm_calls_budget"
1749
+ if sample_llm_fail_budget > 0 and llm_usage.get("format_fail_total", 0) >= sample_llm_fail_budget:
1750
+ return "llm_fail_budget"
1751
+ return None
1752
+
1753
+ def _refresh_stop_reason() -> Optional[str]:
1754
+ nonlocal stop_reason
1755
+ if stop_reason is None:
1756
+ stop_reason = _budget_exceeded()
1757
+ return stop_reason
1758
+
1759
+ def _finalize(proved: bool, x, expr_str):
1760
+ _flush_wrong_data()
1761
+ return proved, tree, x, expr_str, llm_usage, stop_reason
1762
+
1763
+ current_params = (adapter_path, use_dummy, tuple(gpu_devices), max_vars_for_lp)
1764
+
1765
+ if gpu_executors is not None:
1766
+ pass
1767
+ elif _CACHED_GPU_EXECUTORS is not None and _CACHED_GPU_PARAMS == current_params:
1768
+ gpu_executors = _CACHED_GPU_EXECUTORS
1769
+ elif gpu_devices:
1770
+ if _CACHED_GPU_EXECUTORS is not None:
1771
+ for ex in _CACHED_GPU_EXECUTORS:
1772
+ try:
1773
+ ex.shutdown(wait=True, cancel_futures=True)
1774
+ except Exception:
1775
+ pass
1776
+
1777
+ gpu_executors = []
1778
+ try:
1779
+ gpu_ctx = mp.get_context("spawn")
1780
+ except Exception:
1781
+ gpu_ctx = None
1782
+ for gid in gpu_devices:
1783
+ gpu_executors.append(
1784
+ ProcessPoolExecutor(
1785
+ max_workers=1,
1786
+ mp_context=gpu_ctx,
1787
+ initializer=_gpu_worker_init,
1788
+ initargs=(adapter_path, use_dummy, int(gid), int(max_vars_for_lp), Gamma),
1789
+ )
1790
+ )
1791
+ _CACHED_GPU_EXECUTORS = gpu_executors
1792
+ _CACHED_GPU_PARAMS = current_params
1793
+
1794
+ if _is_positive_polymatroid(root_node.residual_vec, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug):
1795
+ if debug:
1796
+ sys.stderr.write(f"[beam_debug] proved_at_root: Target={target_inequality}\n")
1797
+ sys.stderr.flush()
1798
+ return _finalize(True, None, root_node.path_exprs)
1799
+
1800
+ try:
1801
+ cond_raw = conditions if conditions else []
1802
+ cond_lines = []
1803
+ for c in cond_raw:
1804
+ c_norm = _normalize_constraint_comparator(_normalize_to_braces(c.strip()), add_default=True)
1805
+ cond_lines.append(c_norm)
1806
+
1807
+ if not cond_lines:
1808
+ cond_lines = ["NONE=0"]
1809
+
1810
+ for depth in range(1, max_depth + 1):
1811
+ if _refresh_stop_reason():
1812
+ break
1813
+ next_level_nodes = []
1814
+ depth_rollouts = depth1_rollouts if depth == 1 and depth1_rollouts is not None else (32 if depth == 1 else rollout_count)
1815
+
1816
+ if gpu_executors:
1817
+ futures = []
1818
+ fut_meta = {}
1819
+ rr = 0
1820
+ max_tries_per_step = int(os.environ.get("BEAM_MAX_TRIES_PER_STEP", "3"))
1821
+
1822
+ for node in nodes:
1823
+ if _refresh_stop_reason():
1824
+ break
1825
+ for i in range(depth_rollouts):
1826
+ if _refresh_stop_reason():
1827
+ break
1828
+ payload = {
1829
+ "n": int(n),
1830
+ "min_idx": int(min_idx),
1831
+ "max_idx": int(max_idx),
1832
+ "run_id": run_id,
1833
+ "cond_lines": cond_lines,
1834
+ "path_exprs": list(node.path_exprs),
1835
+ "residual_vec": node.residual_vec,
1836
+ "k_per_call": int(k_per_call),
1837
+ "llm_mode": llm_mode,
1838
+ "max_tries_per_step": max_tries_per_step,
1839
+ "adapter_path": adapter_path,
1840
+ "use_dummy": bool(use_dummy),
1841
+ "Gamma": None,
1842
+ "max_vars_for_lp": int(max_vars_for_lp),
1843
+ "log_wrong_pruned": log_wrong_pruned,
1844
+ "idx_to_letter": idx_to_letter,
1845
+ "letter_to_idx": letter_to_idx,
1846
+ }
1847
+ ex = gpu_executors[rr % len(gpu_executors)]
1848
+ rr += 1
1849
+ fut = ex.submit(_expand_node_payload, payload)
1850
+ futures.append(fut)
1851
+ fut_meta[fut] = (node, i)
1852
+ if stop_reason:
1853
+ break
1854
+
1855
+ if not stop_reason:
1856
+ for fut in as_completed(futures):
1857
+ if stop_reason:
1858
+ break
1859
+ node, rollout_idx = fut_meta[fut]
1860
+ try:
1861
+ r = fut.result()
1862
+ except Exception as e:
1863
+ if _is_fatal_gpu_worker_exception(e):
1864
+ raise
1865
+ if debug:
1866
+ sys.stderr.write(f"[beam_debug] gpu_worker_error: {type(e).__name__}: {e}\n")
1867
+ sys.stderr.flush()
1868
+ continue
1869
+ parse_fail += int(r.get("parse_fail", 0))
1870
+ out_of_range += int(r.get("out_of_range", 0))
1871
+ worker_llm_usage = r.get("llm_usage")
1872
+ if worker_llm_usage:
1873
+ for key in llm_usage:
1874
+ llm_usage[key] += int(worker_llm_usage.get(key, 0))
1875
+ if log_wrong_pruned and "wrong_data" in r:
1876
+ all_wrong_data.extend(r["wrong_data"])
1877
+ if _refresh_stop_reason():
1878
+ break
1879
+
1880
+ for pr_data in r.get("pruned_results", []):
1881
+ pr_node = Node(
1882
+ proved=False,
1883
+ parent=node,
1884
+ path_exprs=list(pr_data.get("path_exprs") or []),
1885
+ residual_vec=pr_data.get("residual_vec"),
1886
+ is_pruned=True,
1887
+ )
1888
+ tree.append(pr_node)
1889
+ if _refresh_stop_reason():
1890
+ break
1891
+ if stop_reason:
1892
+ break
1893
+
1894
+ if not r.get("ok", False):
1895
+ continue
1896
+
1897
+ for res_data in r.get("results", []):
1898
+ if _refresh_stop_reason():
1899
+ break
1900
+ res_vec = res_data.get("residual_vec", None)
1901
+ if res_vec is None:
1902
+ continue
1903
+
1904
+ res_key = vector_to_string(res_vec)
1905
+ if res_key in seen_residuals:
1906
+ if debug:
1907
+ sys.stderr.write(f"[beam_debug] duplicate_prune: depth={depth} rollout={rollout_idx} Residual={res_key[:50]}...\n")
1908
+ sys.stderr.flush()
1909
+ pr_node = Node(
1910
+ proved=False,
1911
+ parent=node,
1912
+ path_exprs=list(res_data.get("path_exprs") or []),
1913
+ residual_vec=res_vec,
1914
+ is_pruned=True,
1915
+ )
1916
+ tree.append(pr_node)
1917
+ if _refresh_stop_reason():
1918
+ break
1919
+ continue
1920
+ seen_residuals.add(res_key)
1921
+
1922
+ child = Node(
1923
+ proved=res_data.get("proved", False),
1924
+ parent=node,
1925
+ path_exprs=list(res_data.get("path_exprs") or []),
1926
+ residual_vec=res_vec,
1927
+ )
1928
+
1929
+ if debug:
1930
+ _beam_debug_print_M(
1931
+ depth=depth,
1932
+ rollout=rollout_idx,
1933
+ path_exprs=child.path_exprs,
1934
+ residual_vec=child.residual_vec,
1935
+ action="expand_done",
1936
+ )
1937
+
1938
+ if child.proved:
1939
+ if debug:
1940
+ sys.stderr.write(f"[beam_debug] proved: path={child.path_exprs}\n")
1941
+ sys.stderr.flush()
1942
+ tree.append(child)
1943
+ return _finalize(True, None, child.path_exprs)
1944
+
1945
+ next_level_nodes.append(child)
1946
+ if stop_reason:
1947
+ break
1948
+
1949
+ if stop_reason:
1950
+ for fut in futures:
1951
+ try:
1952
+ fut.cancel()
1953
+ except Exception:
1954
+ pass
1955
+ else:
1956
+ for node in nodes:
1957
+ if _refresh_stop_reason():
1958
+ break
1959
+ for i in range(depth_rollouts):
1960
+ if _refresh_stop_reason():
1961
+ break
1962
+ current_residual_vec = node.residual_vec
1963
+ current_path_exprs = list(node.path_exprs)
1964
+
1965
+ if llm_mode == "sequential":
1966
+ subtracted_any = False
1967
+ for _step in range(k_per_call):
1968
+ if _refresh_stop_reason():
1969
+ break
1970
+ current_target_norm = vector_to_string(current_residual_vec)
1971
+
1972
+ prompt = _build_llm_prompt(
1973
+ n=n, min_idx=min_idx, max_idx=max_idx,
1974
+ target_inequality_normalized=current_target_norm,
1975
+ cond_lines=cond_lines, m_exprs=current_path_exprs,
1976
+ idx_to_letter=idx_to_letter,
1977
+ )
1978
+ cand_list = LLM_propose_inequalities(prompt, k=1, adapter_path=adapter_path, use_dummy=use_dummy)
1979
+ _merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
1980
+ if _refresh_stop_reason():
1981
+ break
1982
+ if not cand_list:
1983
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs, is_pruned=True))
1984
+ _refresh_stop_reason()
1985
+ break
1986
+ cand_raw = cand_list[0]
1987
+ cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
1988
+ cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
1989
+ format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
1990
+ if cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
1991
+ if log_wrong_pruned:
1992
+ all_wrong_data.append([prompt, cand_raw])
1993
+ if debug:
1994
+ sys.stderr.write(
1995
+ f"[beam_debug] invalid_or_wrong_cand_manual: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
1996
+ )
1997
+ sys.stderr.flush()
1998
+ if format_reason is not None:
1999
+ llm_usage["format_fail_total"] += 1
2000
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
2001
+ _refresh_stop_reason()
2002
+ break
2003
+ try:
2004
+ vec = parse_cand_to_vector(cand, n)
2005
+ if re.search(r'(?<![<>])=\s*0', cand):
2006
+ if not _is_in_row_space(vec, cond_eq_csr):
2007
+ if log_wrong_pruned:
2008
+ all_wrong_data.append([prompt, cand_raw])
2009
+ if debug:
2010
+ sys.stderr.write(f"[beam_debug] hallucinated_equality_manual: step={cand_raw} not in row-space of conditions\n")
2011
+ sys.stderr.flush()
2012
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
2013
+ _refresh_stop_reason()
2014
+ break
2015
+ else:
2016
+ if debug:
2017
+ sys.stderr.write(f"[beam_debug] equality_verified_manual: step={cand_raw} is in row-space\n")
2018
+ sys.stderr.flush()
2019
+
2020
+ new_residual = current_residual_vec - vec
2021
+ status = _check_shannon_status(new_residual, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug)
2022
+
2023
+ if _is_obviously_negative(new_residual):
2024
+ if log_wrong_pruned:
2025
+ all_wrong_data.append([prompt, cand_raw])
2026
+ if debug:
2027
+ res_str = vector_to_string(new_residual)
2028
+ sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
2029
+ sys.stderr.flush()
2030
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
2031
+ subtracted_any = False
2032
+ _refresh_stop_reason()
2033
+ break
2034
+
2035
+ if status == -1:
2036
+ if log_wrong_pruned:
2037
+ all_wrong_data.append([prompt, cand_raw])
2038
+ if debug:
2039
+ res_str = vector_to_string(new_residual)
2040
+ sys.stderr.write(f"[beam_debug] lp_false_prune_manual: residual={res_str}\n")
2041
+ sys.stderr.flush()
2042
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
2043
+ subtracted_any = False
2044
+ _refresh_stop_reason()
2045
+ break
2046
+
2047
+ if status == 1:
2048
+ current_residual_vec = new_residual
2049
+ current_path_exprs.append(cand)
2050
+ subtracted_any = True
2051
+ break
2052
+
2053
+ res_key = vector_to_string(new_residual)
2054
+ if res_key in seen_residuals:
2055
+ if debug:
2056
+ sys.stderr.write(f"[beam_debug] duplicate_prune: depth={depth} rollout={i} Residual={res_key[:50]}...\n")
2057
+ sys.stderr.flush()
2058
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
2059
+ _refresh_stop_reason()
2060
+ break
2061
+
2062
+ seen_residuals.add(res_key)
2063
+ current_residual_vec = new_residual
2064
+ current_path_exprs.append(cand)
2065
+ subtracted_any = True
2066
+ except Exception:
2067
+ if log_wrong_pruned:
2068
+ all_wrong_data.append([prompt, cand_raw])
2069
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
2070
+ _refresh_stop_reason()
2071
+ continue
2072
+
2073
+ if stop_reason:
2074
+ break
2075
+ if subtracted_any:
2076
+ is_proved = _is_positive_polymatroid(current_residual_vec, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug)
2077
+ child = Node(proved=is_proved, parent=node, path_exprs=current_path_exprs, residual_vec=current_residual_vec)
2078
+ if debug:
2079
+ _beam_debug_print_M(depth=depth, rollout=i, path_exprs=child.path_exprs, residual_vec=child.residual_vec, action="manual_expand")
2080
+ if is_proved:
2081
+ if debug:
2082
+ sys.stderr.write(f"[beam_debug] proved: path={current_path_exprs}\n")
2083
+ sys.stderr.flush()
2084
+ tree.append(child)
2085
+ return _finalize(True, None, child.path_exprs)
2086
+ next_level_nodes.append(child)
2087
+ else:
2088
+ current_target_norm = vector_to_string(current_residual_vec)
2089
+
2090
+ prompt = _build_llm_prompt(
2091
+ n=n, min_idx=min_idx, max_idx=max_idx,
2092
+ target_inequality_normalized=current_target_norm,
2093
+ cond_lines=cond_lines, m_exprs=current_path_exprs,
2094
+ idx_to_letter=idx_to_letter,
2095
+ )
2096
+ cand_list = LLM_propose_inequalities(prompt, k=k_per_call, adapter_path=adapter_path, use_dummy=use_dummy)
2097
+ _merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
2098
+ if _refresh_stop_reason():
2099
+ break
2100
+ for cand in cand_list:
2101
+ if _refresh_stop_reason():
2102
+ break
2103
+ cand_raw = cand
2104
+ cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
2105
+ cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
2106
+ format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
2107
+ if not cand or cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
2108
+ if log_wrong_pruned:
2109
+ all_wrong_data.append([prompt, cand_raw])
2110
+ if debug and cand_raw:
2111
+ sys.stderr.write(
2112
+ f"[beam_debug] invalid_or_wrong_cand_manual_batch: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
2113
+ )
2114
+ sys.stderr.flush()
2115
+ if format_reason is not None:
2116
+ llm_usage["format_fail_total"] += 1
2117
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
2118
+ _refresh_stop_reason()
2119
+ continue
2120
+ try:
2121
+ vec = parse_cand_to_vector(cand, n)
2122
+ if re.search(r'(?<![<>])=\s*0', cand):
2123
+ if not _is_in_row_space(vec, cond_eq_csr):
2124
+ if log_wrong_pruned:
2125
+ all_wrong_data.append([prompt, cand_raw])
2126
+ if debug:
2127
+ sys.stderr.write(f"[beam_debug] hallucinated_equality_manual: step={cand_raw} not in row-space of conditions\n")
2128
+ sys.stderr.flush()
2129
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
2130
+ _refresh_stop_reason()
2131
+ continue
2132
+ else:
2133
+ if debug:
2134
+ sys.stderr.write(f"[beam_debug] equality_verified_manual: step={cand_raw} is in row-space\n")
2135
+ sys.stderr.flush()
2136
+
2137
+ new_residual = current_residual_vec - vec
2138
+ status = _check_shannon_status(new_residual, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug)
2139
+ is_proved = (status == 1)
2140
+
2141
+ if not is_proved and _is_obviously_negative(new_residual):
2142
+ if log_wrong_pruned:
2143
+ all_wrong_data.append([prompt, cand_raw])
2144
+ if debug:
2145
+ res_str = vector_to_string(new_residual)
2146
+ sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
2147
+ sys.stderr.flush()
2148
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
2149
+ _refresh_stop_reason()
2150
+ continue
2151
+
2152
+ if not is_proved and status == -1:
2153
+ if log_wrong_pruned:
2154
+ all_wrong_data.append([prompt, cand_raw])
2155
+ if debug:
2156
+ res_str = vector_to_string(new_residual)
2157
+ sys.stderr.write(f"[beam_debug] lp_false_prune_manual_batch: residual={res_str}\n")
2158
+ sys.stderr.flush()
2159
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
2160
+ _refresh_stop_reason()
2161
+ continue
2162
+
2163
+ res_key = vector_to_string(new_residual)
2164
+ if res_key in seen_residuals:
2165
+ if debug:
2166
+ sys.stderr.write(f"[beam_debug] duplicate_prune: depth={depth} rollout={i} Residual={res_key[:50]}...\n")
2167
+ sys.stderr.flush()
2168
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
2169
+ _refresh_stop_reason()
2170
+ continue
2171
+
2172
+ seen_residuals.add(res_key)
2173
+ child = Node(proved=is_proved, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual)
2174
+ if debug:
2175
+ _beam_debug_print_M(depth=depth, rollout=i, path_exprs=child.path_exprs, residual_vec=child.residual_vec, action="manual_expand_batch")
2176
+ if is_proved:
2177
+ if debug:
2178
+ sys.stderr.write(f"[beam_debug] proved: path={current_path_exprs + [cand]}\n")
2179
+ sys.stderr.flush()
2180
+ tree.append(child)
2181
+ return _finalize(True, None, child.path_exprs)
2182
+ next_level_nodes.append(child)
2183
+ except Exception:
2184
+ tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
2185
+ _refresh_stop_reason()
2186
+ continue
2187
+ if stop_reason:
2188
+ break
2189
+ if stop_reason:
2190
+ break
2191
+
2192
+ if stop_reason:
2193
+ break
2194
+ if not next_level_nodes:
2195
+ if debug:
2196
+ sys.stderr.write(f"[beam_debug] search_stuck: depth={depth} next_level_nodes is empty\n")
2197
+ sys.stderr.flush()
2198
+ break
2199
+
2200
+ nodes = next_level_nodes
2201
+
2202
+ if beam_size and len(nodes) > beam_size:
2203
+ before = len(nodes)
2204
+ nodes, pruned_by_beam = _select_beam_nodes(
2205
+ nodes,
2206
+ beam_size,
2207
+ random_prune=random_prune,
2208
+ rng=prune_rng,
2209
+ )
2210
+ for node in nodes:
2211
+ node.score = _get_residual_score(node.residual_vec)
2212
+ for pn in pruned_by_beam:
2213
+ pn.is_pruned = True
2214
+ tree.append(pn)
2215
+ if _refresh_stop_reason():
2216
+ break
2217
+ if stop_reason:
2218
+ break
2219
+ if debug:
2220
+ strategy = "seeded_uniform_random" if random_prune else "short_residual_greedy"
2221
+ sys.stderr.write(
2222
+ f"[beam_debug] prune: depth={depth} strategy={strategy} beam_size={beam_size} "
2223
+ f"before={before} after={len(nodes)}\n"
2224
+ )
2225
+ sys.stderr.flush()
2226
+ tree.extend(nodes)
2227
+ if _refresh_stop_reason():
2228
+ break
2229
+
2230
+ return _finalize(False, None, None)
2231
+ finally:
2232
+ pass
mce220_runner/beam_search_model2/__init__.py ADDED
File without changes
mce220_runner/beam_search_model2/llm_api_compat.py ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Portable compatibility import for the shared local LLM interface.
2
+
3
+ The implementation lives in :mod:`grpo_pipeline.modules.llm_api_interface`.
4
+ This module replaces the repository's former absolute Linux symlink so the
5
+ beam-search entry points work from a fresh checkout on any platform.
6
+ """
7
+
8
+ from __future__ import annotations
9
+
10
+ import importlib
11
+ import sys
12
+ from pathlib import Path
13
+
14
+
15
+ _REPO_ROOT = Path(__file__).resolve().parents[1]
16
+ if str(_REPO_ROOT) not in sys.path:
17
+ sys.path.insert(0, str(_REPO_ROOT))
18
+
19
+ _shared = importlib.import_module("grpo_pipeline.modules.llm_api_interface")
20
+ __all__ = [name for name in vars(_shared) if not name.startswith("_")]
21
+ globals().update({name: getattr(_shared, name) for name in __all__})
22
+
23
+
24
+ def __getattr__(name: str):
25
+ """Delegate private compatibility imports to the shared implementation."""
26
+ return getattr(_shared, name)
mce220_runner/beam_search_model2/llmstr_to_vector.py ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Parse common information-theoretic inequality shorthands into entropy vectors.
3
+
4
+ Supported:
5
+ - I(X_A;X_B|X_C)
6
+ - I(X_A;X_B)
7
+ - H(X_A|X_B)
8
+ - H(X_A) / H(X_{...}) linear combinations are handled by data/entropy_vector_converter.string_to_vector
9
+
10
+ This module provides polym_ineq_to_vector(expr, n) to match existing imports.
11
+ """
12
+
13
+ from __future__ import annotations
14
+
15
+ import re
16
+ import numpy as np
17
+ from scipy.sparse import csr_matrix
18
+ from typing import List, Set
19
+
20
+
21
+ def _subset_to_index(subset: Set[int], n: int) -> int:
22
+ # Convert subset {i} into 0-based index in length (2^n - 1) vector.
23
+ # index = sum(2^(i-1)) - 1
24
+ s = 0
25
+ for i in subset:
26
+ if 1 <= i <= n:
27
+ s += 2 ** (i - 1)
28
+ else:
29
+ raise ValueError(f"variable index out of range: {i} (n={n})")
30
+ if s <= 0:
31
+ raise ValueError("empty subset has no entropy coordinate")
32
+ return s - 1
33
+
34
+
35
+ def _parse_X_term(s: str) -> Set[int]:
36
+ """
37
+ Parse X_{1,2} or X_1 or {1,2} into a set of ints.
38
+ """
39
+ s = s.strip()
40
+ # Allow raw "{1,2}" or "1,2"
41
+ s = s.strip("{}")
42
+ s = s.replace("X_", "").strip()
43
+ if not s:
44
+ return set()
45
+ parts = [p.strip() for p in s.split(",") if p.strip()]
46
+ out: Set[int] = set()
47
+ for p in parts:
48
+ out.add(int(p))
49
+ return out
50
+
51
+
52
+ def _vec_for_H(subset: Set[int], n: int, coeff: float) -> csr_matrix:
53
+ M = 2 ** n - 1
54
+ arr = np.zeros(M, dtype=np.float64)
55
+ arr[_subset_to_index(subset, n)] = coeff
56
+ return csr_matrix(arr.reshape(1, -1))
57
+
58
+
59
+ def polym_ineq_to_vector(expr: str, n: int) -> csr_matrix:
60
+ """
61
+ Convert shorthand info-theoretic expressions into an entropy vector.
62
+
63
+ Note: caller is expected to strip any trailing ">=0" etc.
64
+ """
65
+ s = expr.replace(" ", "").strip()
66
+
67
+ # H(X_A|X_B) = H(A∪B) - H(B)
68
+ m = re.fullmatch(r"H\(X_(\{[^}]+\}|\d+)\|X_(\{[^}]+\}|\d+)\)", s)
69
+ if m:
70
+ A = _parse_X_term(m.group(1))
71
+ B = _parse_X_term(m.group(2))
72
+ if not A:
73
+ raise ValueError(f"invalid conditional entropy: {expr}")
74
+ return _vec_for_H(A | B, n, 1.0) + _vec_for_H(B, n, -1.0)
75
+
76
+ # I(X_A;X_B|X_C) = H(A∪C)+H(B∪C)-H(C)-H(A∪B∪C)
77
+ m = re.fullmatch(
78
+ r"I\(X_(\{[^}]+\}|\d+);X_(\{[^}]+\}|\d+)\|X_(\{[^}]+\}|\d+)\)", s
79
+ )
80
+ if m:
81
+ A = _parse_X_term(m.group(1))
82
+ B = _parse_X_term(m.group(2))
83
+ C = _parse_X_term(m.group(3))
84
+ if not A or not B:
85
+ raise ValueError(f"invalid conditional mutual information: {expr}")
86
+ return (
87
+ _vec_for_H(A | C, n, 1.0)
88
+ + _vec_for_H(B | C, n, 1.0)
89
+ + _vec_for_H(C, n, -1.0)
90
+ + _vec_for_H(A | B | C, n, -1.0)
91
+ )
92
+
93
+ # I(X_A;X_B) = H(A)+H(B)-H(A∪B)
94
+ m = re.fullmatch(r"I\(X_(\{[^}]+\}|\d+);X_(\{[^}]+\}|\d+)\)", s)
95
+ if m:
96
+ A = _parse_X_term(m.group(1))
97
+ B = _parse_X_term(m.group(2))
98
+ if not A or not B:
99
+ raise ValueError(f"invalid mutual information: {expr}")
100
+ return _vec_for_H(A, n, 1.0) + _vec_for_H(B, n, 1.0) + _vec_for_H(A | B, n, -1.0)
101
+
102
+ raise ValueError(f"Unsupported polymatroid shorthand: {expr}")
103
+
104
+
mce220_runner/beam_search_model2/run_on_window_shear_jsonl_v12.py ADDED
@@ -0,0 +1,821 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ Batch-run beam_search tree_search on a JSONL dataset using V12 logic.
4
+ """
5
+
6
+ from __future__ import annotations
7
+
8
+ import argparse
9
+ import hashlib
10
+ import json
11
+ import os
12
+ import sys
13
+ import time
14
+ import uuid
15
+ from datetime import datetime, timezone
16
+ from pathlib import Path
17
+ from typing import Any, Dict, Optional, Tuple
18
+
19
+
20
+ EXECUTION_CONFIG_SCHEMA_VERSION = "e2n-execution-config-v1"
21
+
22
+
23
+ def _is_fatal_gpu_failure(exc: BaseException) -> bool:
24
+ """Return true for accelerator OOMs or abruptly terminated model workers."""
25
+
26
+ current: BaseException | None = exc
27
+ seen: set[int] = set()
28
+ while current is not None and id(current) not in seen:
29
+ seen.add(id(current))
30
+ class_name = type(current).__name__.lower()
31
+ message = str(current).lower()
32
+ if "brokenprocesspool" in class_name or "outofmemory" in class_name:
33
+ return True
34
+ if any(
35
+ fragment in message
36
+ for fragment in (
37
+ "cuda out of memory",
38
+ "cuda error: out of memory",
39
+ "hip out of memory",
40
+ "cublas_status_alloc_failed",
41
+ "cudnn_status_alloc_failed",
42
+ "process in the process pool was terminated abruptly",
43
+ )
44
+ ):
45
+ return True
46
+ current = current.__cause__ or current.__context__
47
+ return False
48
+
49
+ try:
50
+ from verify_proof_certificate import verify_certificate
51
+ except ModuleNotFoundError:
52
+ from beam_search_model2.verify_proof_certificate import verify_certificate
53
+
54
+
55
+ def _proof_trace_metrics(vec, score_fn) -> Dict[str, Any]:
56
+ import numpy as np
57
+
58
+ data = np.asarray(vec.data, dtype=float) if vec is not None else np.asarray([], dtype=float)
59
+ return {
60
+ "negative_l1": float(np.abs(data[data < 0]).sum()) if data.size else 0.0,
61
+ "nnz": int(vec.nnz) if vec is not None else 0,
62
+ "score": float(score_fn(vec)),
63
+ }
64
+
65
+
66
+ def build_proof_trace(
67
+ target_inequality: str,
68
+ n: int,
69
+ selected_steps,
70
+ *,
71
+ search_reported_proved: bool,
72
+ ) -> Dict[str, Any]:
73
+ """Reconstruct a compact, auditable residual trajectory from a selected path."""
74
+ try:
75
+ from Data_Generation_via_LLM_Guided_Search_v12 import (
76
+ _get_residual_score,
77
+ parse_cand_to_vector,
78
+ vector_to_string,
79
+ )
80
+ except ModuleNotFoundError:
81
+ from beam_search_model2.Data_Generation_via_LLM_Guided_Search_v12 import (
82
+ _get_residual_score,
83
+ parse_cand_to_vector,
84
+ vector_to_string,
85
+ )
86
+
87
+ residual = parse_cand_to_vector(target_inequality, n)
88
+ trace_steps = []
89
+ for index, inequality in enumerate(selected_steps or [], start=1):
90
+ before = residual
91
+ after = before - parse_cand_to_vector(str(inequality), n)
92
+ before_metrics = _proof_trace_metrics(before, _get_residual_score)
93
+ after_metrics = _proof_trace_metrics(after, _get_residual_score)
94
+ trace_steps.append(
95
+ {
96
+ "index": index,
97
+ "inequality": str(inequality),
98
+ "residual_before": vector_to_string(before),
99
+ "residual_after": vector_to_string(after),
100
+ "before": before_metrics,
101
+ "after": after_metrics,
102
+ "score_improvement": before_metrics["score"] - after_metrics["score"],
103
+ }
104
+ )
105
+ residual = after
106
+
107
+ terminal_metrics = _proof_trace_metrics(residual, _get_residual_score)
108
+ if terminal_metrics["nnz"] == 0:
109
+ certificate_kind = "zero_residual"
110
+ elif residual is not None and (residual.data >= -1e-9).all():
111
+ certificate_kind = "coefficientwise_nonnegative"
112
+ elif search_reported_proved:
113
+ certificate_kind = "reported_proved_without_explicit_certificate"
114
+ else:
115
+ certificate_kind = "not_certified"
116
+ return {
117
+ "steps": trace_steps,
118
+ "terminal": {
119
+ "residual": vector_to_string(residual),
120
+ **terminal_metrics,
121
+ "certificate_kind": certificate_kind,
122
+ },
123
+ }
124
+
125
+
126
+ def attach_proof_trace(
127
+ run_output: Dict[str, Any],
128
+ target_inequality: str,
129
+ n: int,
130
+ selected_steps,
131
+ *,
132
+ search_reported_proved: bool,
133
+ ) -> None:
134
+ """Attach a trace without allowing diagnostic failure to abort a batch."""
135
+ try:
136
+ run_output["proof_trace"] = build_proof_trace(
137
+ target_inequality,
138
+ n,
139
+ selected_steps,
140
+ search_reported_proved=search_reported_proved,
141
+ )
142
+ except Exception as exc:
143
+ run_output["proof_trace_error"] = f"{type(exc).__name__}: {exc}"
144
+
145
+
146
+ def _exact_verification_summary(result) -> Dict[str, Any]:
147
+ """Convert an exact verifier result into stable JSON-compatible diagnostics."""
148
+ return {
149
+ "status": "valid" if result.valid else "invalid",
150
+ "valid": bool(result.valid),
151
+ "reason": result.reason,
152
+ "terminal_kind": result.terminal_kind,
153
+ "step_count": len(result.steps),
154
+ "steps": [
155
+ {
156
+ "kind": step.kind,
157
+ "multiplier": str(step.multiplier),
158
+ "condition_index": step.condition_index,
159
+ }
160
+ for step in result.steps
161
+ ],
162
+ "residual": [
163
+ {"subset_mask": mask, "coefficient": str(coefficient)}
164
+ for mask, coefficient in sorted(result.residual.items())
165
+ ],
166
+ }
167
+
168
+
169
+ def apply_exact_verification_gate(
170
+ run_output: Dict[str, Any],
171
+ target_inequality: str,
172
+ conditions,
173
+ n: int,
174
+ selected_steps,
175
+ *,
176
+ search_reported_proved: bool,
177
+ ) -> None:
178
+ """Require an exact symbolic certificate for every search-reported success."""
179
+ if not search_reported_proved:
180
+ return
181
+ verification_start = time.perf_counter()
182
+ try:
183
+ result = verify_certificate(
184
+ n,
185
+ target_inequality,
186
+ list(conditions or []),
187
+ list(selected_steps) if isinstance(selected_steps, list) else selected_steps,
188
+ )
189
+ summary = _exact_verification_summary(result)
190
+ except Exception as exc:
191
+ summary = {
192
+ "status": "error",
193
+ "valid": False,
194
+ "reason": f"{type(exc).__name__}: {exc}",
195
+ "terminal_kind": None,
196
+ "step_count": len(selected_steps) if isinstance(selected_steps, list) else 0,
197
+ "steps": [],
198
+ "residual": [],
199
+ }
200
+ summary["time_sec"] = max(time.perf_counter() - verification_start, 0.0)
201
+ run_output["exact_verification"] = summary
202
+ if summary["valid"]:
203
+ run_output["proved"] = True
204
+ return
205
+
206
+ run_output["proved"] = False
207
+ failure = f"Exact verification {summary['status']}: {summary['reason']}"
208
+ existing_error = run_output.get("error")
209
+ run_output["error"] = f"{existing_error}; {failure}" if existing_error else failure
210
+
211
+
212
+ def _parse_per_n_budget(spec: str) -> Dict[int, Tuple[int, float]]:
213
+ budgets: Dict[int, Tuple[int, float]] = {}
214
+ raw_spec = (spec or "").strip()
215
+ if not raw_spec:
216
+ return budgets
217
+
218
+ for chunk in raw_spec.split(";"):
219
+ chunk = chunk.strip()
220
+ if not chunk:
221
+ continue
222
+ try:
223
+ n_str, caps_str = chunk.split(":", 1)
224
+ tree_str, time_str = [part.strip() for part in caps_str.split(",", 1)]
225
+ budgets[int(n_str.strip())] = (int(tree_str), float(time_str))
226
+ except Exception as exc:
227
+ raise SystemExit(
228
+ "Invalid --per-n-budget format. Expected '8:3134,1130;9:3192,1499'. "
229
+ f"Failed at chunk {chunk!r}: {exc}"
230
+ )
231
+ return budgets
232
+
233
+
234
+ def _resolve_sample_budget(
235
+ n: int,
236
+ per_n_budget: Dict[int, Tuple[int, float]],
237
+ fallback_tree_size: int,
238
+ fallback_time_sec: float,
239
+ ) -> Tuple[int, float]:
240
+ if n in per_n_budget:
241
+ return per_n_budget[n]
242
+ return fallback_tree_size, fallback_time_sec
243
+
244
+
245
+ def _direct_execution_config(args: argparse.Namespace) -> Dict[str, Any]:
246
+ """Self-describe direct runner use when no reviewer wrapper is present."""
247
+
248
+ gpu_ids = []
249
+ raw_gpu_ids = str(
250
+ args.gpu_devices or os.environ.get("CUDA_VISIBLE_DEVICES") or ""
251
+ ).strip()
252
+ if raw_gpu_ids:
253
+ gpu_ids = [
254
+ int(part.strip())
255
+ for part in raw_gpu_ids.replace(" ", ",").split(",")
256
+ if part.strip()
257
+ ][: int(args.gpu_workers or 0)]
258
+ elif args.gpu_workers:
259
+ gpu_ids = list(range(int(args.gpu_workers)))
260
+ normalized = list(sys.argv)
261
+ if "--execution-config-manifest" in normalized:
262
+ index = normalized.index("--execution-config-manifest")
263
+ if index + 1 < len(normalized):
264
+ normalized[index + 1] = "<live-wrapper-manifest>"
265
+ return {
266
+ "schema_version": EXECUTION_CONFIG_SCHEMA_VERSION,
267
+ "invocation_id": uuid.uuid4().hex,
268
+ "recorded_utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
269
+ "origin": "direct_runner",
270
+ "code": {
271
+ "git_commit": None,
272
+ "repo_root": None,
273
+ "runner_path": str(Path(__file__).resolve()),
274
+ "runner_sha256": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
275
+ },
276
+ "model": {
277
+ "checkpoint_path": str(Path(args.adapter_path).expanduser().resolve())
278
+ if args.adapter_path
279
+ else None,
280
+ "checkpoint_sha256": None,
281
+ "base_model": None,
282
+ },
283
+ "search": {
284
+ key: value
285
+ for key, value in vars(args).items()
286
+ if key != "execution_config_manifest"
287
+ },
288
+ "resources": {
289
+ "workers": int(args.gpu_workers or 0),
290
+ "gpu_ids": gpu_ids,
291
+ "worker_device_mapping": [
292
+ {
293
+ "worker_index": index,
294
+ "physical_gpu_index": gpu_id,
295
+ "gpu_uuid": None,
296
+ }
297
+ for index, gpu_id in enumerate(gpu_ids)
298
+ ],
299
+ "cuda_visible_devices": os.environ.get("CUDA_VISIBLE_DEVICES"),
300
+ "launch_gpu_snapshot": [],
301
+ },
302
+ "normalized_command": normalized,
303
+ }
304
+
305
+
306
+ def _load_execution_config(
307
+ manifest_path: str, args: argparse.Namespace
308
+ ) -> Dict[str, Any]:
309
+ if not manifest_path:
310
+ return _direct_execution_config(args)
311
+ path = Path(manifest_path).expanduser().resolve()
312
+ try:
313
+ manifest = json.loads(path.read_text(encoding="utf-8"))
314
+ except (OSError, json.JSONDecodeError) as exc:
315
+ raise SystemExit(f"Invalid --execution-config-manifest: {exc}") from exc
316
+ config = manifest.get("execution_config") if isinstance(manifest, dict) else None
317
+ if not isinstance(config, dict):
318
+ raise SystemExit("Execution-config manifest has no object execution_config")
319
+ if config.get("schema_version") != EXECUTION_CONFIG_SCHEMA_VERSION:
320
+ raise SystemExit(
321
+ "Unsupported execution_config schema: "
322
+ f"{config.get('schema_version')!r}"
323
+ )
324
+ required = {
325
+ "invocation_id",
326
+ "recorded_utc",
327
+ "origin",
328
+ "code",
329
+ "model",
330
+ "search",
331
+ "resources",
332
+ "normalized_command",
333
+ }
334
+ missing = sorted(required.difference(config))
335
+ if missing:
336
+ raise SystemExit(f"Execution config is missing fields: {missing}")
337
+ # Round-trip once so every row receives a detached, JSON-safe value.
338
+ return json.loads(json.dumps(config, ensure_ascii=False))
339
+
340
+
341
+ def _append_jsonl_row(path: Path, row: Dict[str, Any]) -> None:
342
+ """Append one complete row with one O_APPEND write, then fsync it."""
343
+
344
+ payload = (json.dumps(row, ensure_ascii=False) + "\n").encode("utf-8")
345
+ descriptor = os.open(
346
+ path,
347
+ os.O_WRONLY | os.O_CREAT | os.O_APPEND,
348
+ 0o600,
349
+ )
350
+ try:
351
+ written = os.write(descriptor, payload)
352
+ if written != len(payload):
353
+ raise OSError(
354
+ f"short atomic JSONL append: wrote {written} of {len(payload)} bytes"
355
+ )
356
+ os.fsync(descriptor)
357
+ finally:
358
+ os.close(descriptor)
359
+
360
+
361
+ def main() -> int:
362
+ ap = argparse.ArgumentParser()
363
+ ap.add_argument("--input", required=True, help="Input JSONL path")
364
+ ap.add_argument("--output", required=True, help="Output JSONL path")
365
+ ap.add_argument(
366
+ "--execution-config-manifest",
367
+ default="",
368
+ help="Live wrapper manifest containing the effective execution_config.",
369
+ )
370
+ ap.add_argument("--max-depth", type=int, default=10)
371
+ ap.add_argument("--rollouts", type=int, default=2)
372
+ ap.add_argument(
373
+ "--k-per-node",
374
+ type=int,
375
+ default=1,
376
+ help=(
377
+ "Number of sequential LLM generate+append steps per node before one LP check "
378
+ "(default: 1)."
379
+ ),
380
+ )
381
+ ap.add_argument(
382
+ "--llm-mode",
383
+ choices=["sequential", "batch"],
384
+ default="sequential",
385
+ help="LLM generation mode: sequential (k conditioned steps) or batch (one GPU call returns k candidates).",
386
+ )
387
+ ap.add_argument(
388
+ "--lp-workers",
389
+ type=int,
390
+ default=0,
391
+ help=(
392
+ "Reserved for command-line compatibility. Current v12 LP checks are "
393
+ "synchronous; this value does not create an LP process pool."
394
+ ),
395
+ )
396
+ ap.add_argument(
397
+ "--gpu-workers",
398
+ type=int,
399
+ default=0,
400
+ help=(
401
+ "If >0, run depth-parallel LLM expansion using this many GPU worker processes "
402
+ "(one model instance per process / per GPU)."
403
+ ),
404
+ )
405
+ ap.add_argument(
406
+ "--gpu-devices",
407
+ default="",
408
+ help=(
409
+ "Comma-separated list of GPU ids to use for LLM workers (e.g. '0,1,2,3'). "
410
+ "If omitted, will try to infer from CUDA_VISIBLE_DEVICES."
411
+ ),
412
+ )
413
+ ap.add_argument("--limit", type=int, default=0, help="If >0, only run first N samples")
414
+ ap.add_argument(
415
+ "--min-n",
416
+ type=int,
417
+ default=0,
418
+ help="If >0, skip samples with fewer than this many variables.",
419
+ )
420
+ ap.add_argument(
421
+ "--max-n",
422
+ type=int,
423
+ default=0,
424
+ help="If >0, skip samples with more than this many variables.",
425
+ )
426
+ ap.add_argument(
427
+ "--adapter-path",
428
+ default=os.environ.get("QWEN_ADAPTER_PATH", ""),
429
+ help="Path to PEFT adapter dir (defaults to env QWEN_ADAPTER_PATH)",
430
+ )
431
+ ap.add_argument("--use-dummy", action="store_true", help="Use dummy LLM outputs (no model load)")
432
+ ap.add_argument(
433
+ "--max-new-tokens",
434
+ type=int,
435
+ default=int(os.environ.get("QWEN_MAX_NEW_TOKENS", "1024")),
436
+ help="LLM generation cap",
437
+ )
438
+ ap.add_argument(
439
+ "--max-time",
440
+ type=float,
441
+ default=float(os.environ.get("QWEN_MAX_TIME", "20.0")),
442
+ help="LLM generation time cap (seconds)",
443
+ )
444
+ ap.add_argument(
445
+ "--sample-max-time-sec",
446
+ type=float,
447
+ default=0.0,
448
+ help=(
449
+ "Soft cap on elapsed search time. Work already in flight can finish after "
450
+ "the limit."
451
+ ),
452
+ )
453
+ ap.add_argument(
454
+ "--sample-max-tree-size",
455
+ type=int,
456
+ default=0,
457
+ help="Soft cap on explored tree size, checked between completed expansions.",
458
+ )
459
+ ap.add_argument(
460
+ "--sample-max-llm-calls",
461
+ type=int,
462
+ default=0,
463
+ help=(
464
+ "Soft cap on aggregated LLM calls. Concurrent or multi-step expansions can "
465
+ "overshoot it."
466
+ ),
467
+ )
468
+ ap.add_argument(
469
+ "--sample-max-llm-fails",
470
+ type=int,
471
+ default=0,
472
+ help="If >0, stop a sample once llm_usage.format_fail_total reaches this limit.",
473
+ )
474
+ ap.add_argument(
475
+ "--depth1-rollouts",
476
+ type=int,
477
+ default=0,
478
+ help="If >0, override the special depth=1 rollout count (default keeps existing value 32).",
479
+ )
480
+ ap.add_argument(
481
+ "--per-n-budget",
482
+ default="",
483
+ help="Per-n tree/time caps in the form '8:3134,1130;9:3192,1499'.",
484
+ )
485
+ ap.add_argument(
486
+ "--append",
487
+ action="store_true",
488
+ help="Append to the output JSONL instead of overwriting it.",
489
+ )
490
+ ap.add_argument(
491
+ "--resume-from-output",
492
+ action="store_true",
493
+ help="Resume by skipping the first N eligible samples, where N is the number of existing rows in the output JSONL.",
494
+ )
495
+ ap.add_argument(
496
+ "--log-every",
497
+ type=int,
498
+ default=1,
499
+ help="Print progress every N samples (default: 1)",
500
+ )
501
+ ap.add_argument(
502
+ "--print-llm",
503
+ action="store_true",
504
+ help="Print raw LLM generation and extracted inequalities each call (to stderr)",
505
+ )
506
+ ap.add_argument(
507
+ "--debug-search",
508
+ action="store_true",
509
+ help="Enable extra debug printing from tree_search (LP status / parse warnings)",
510
+ )
511
+ ap.add_argument(
512
+ "--beam-size",
513
+ type=int,
514
+ default=0,
515
+ help="If >0, keep at most this many nodes (paths) per depth to avoid explosion",
516
+ )
517
+ ap.add_argument(
518
+ "--random-prune",
519
+ action="store_true",
520
+ help="When beam-size is exceeded, uniformly sample nodes without replacement.",
521
+ )
522
+ ap.add_argument(
523
+ "--include-proof-trace",
524
+ action="store_true",
525
+ help="Include residual and score diagnostics for every selected proof step.",
526
+ )
527
+ ap.add_argument(
528
+ "--prune-seed",
529
+ type=int,
530
+ default=0,
531
+ help="Seed for random pruning (0 means nondeterministic)",
532
+ )
533
+ ap.add_argument(
534
+ "--seed",
535
+ type=int,
536
+ default=0,
537
+ help="Global seed for random, numpy, and torch (0 means nondeterministic)",
538
+ )
539
+ ap.add_argument(
540
+ "--log-wrong-pruned",
541
+ action="store_true",
542
+ help="Enable logging of pruned model outputs (invalid/negative/malformed, including multi-term expressions) to a JSONL file",
543
+ )
544
+ ap.add_argument(
545
+ "--wrong-pruned-path",
546
+ type=str,
547
+ default="",
548
+ help="Path to write the pruned malformed / wrong expressions JSONL dataset",
549
+ )
550
+ ap.add_argument(
551
+ "--enable-lp",
552
+ action="store_true",
553
+ help="Enable Shannon LP check.",
554
+ )
555
+ ap.add_argument(
556
+ "--max-vars-for-lp",
557
+ type=int,
558
+ default=6,
559
+ help="Maximum number of variables for full Shannon LP check (default: 6)",
560
+ )
561
+ args = ap.parse_args()
562
+ execution_config = _load_execution_config(args.execution_config_manifest, args)
563
+
564
+ if args.lp_workers and args.lp_workers > 0:
565
+ print(
566
+ "warning: --lp-workers is retained for compatibility but is not implemented "
567
+ "by the v12 search; LP checks remain synchronous",
568
+ file=sys.stderr,
569
+ )
570
+
571
+ if args.seed and args.seed != 0:
572
+ import random as _random
573
+ import numpy as _np
574
+ import torch as _torch
575
+ _random.seed(args.seed)
576
+ _np.random.seed(args.seed)
577
+ _torch.manual_seed(args.seed)
578
+ _torch.cuda.manual_seed_all(args.seed)
579
+ print(f"Global seed set to {args.seed}")
580
+
581
+ per_n_budget = _parse_per_n_budget(args.per_n_budget)
582
+ depth1_rollouts = args.depth1_rollouts if args.depth1_rollouts > 0 else None
583
+ budget_enabled = bool(per_n_budget) or any(
584
+ [
585
+ (args.sample_max_time_sec and args.sample_max_time_sec > 0),
586
+ (args.sample_max_tree_size and args.sample_max_tree_size > 0),
587
+ (args.sample_max_llm_calls and args.sample_max_llm_calls > 0),
588
+ (args.sample_max_llm_fails and args.sample_max_llm_fails > 0),
589
+ (depth1_rollouts is not None),
590
+ ]
591
+ )
592
+
593
+ gpu_devices = None
594
+ if args.gpu_workers and args.gpu_workers > 0:
595
+ dev_str = (args.gpu_devices or "").strip()
596
+ if not dev_str:
597
+ dev_str = (os.environ.get("CUDA_VISIBLE_DEVICES") or "").strip()
598
+ if dev_str:
599
+ parts = [p.strip() for p in dev_str.replace(" ", ",").split(",") if p.strip()]
600
+ try:
601
+ gpu_devices = [int(p) for p in parts]
602
+ except Exception:
603
+ raise SystemExit(f"Invalid --gpu-devices / CUDA_VISIBLE_DEVICES format: {dev_str!r}")
604
+ else:
605
+ gpu_devices = list(range(int(args.gpu_workers)))
606
+
607
+ if len(gpu_devices) < int(args.gpu_workers):
608
+ raise SystemExit(
609
+ f"--gpu-workers={args.gpu_workers} but only {len(gpu_devices)} devices provided: {gpu_devices}"
610
+ )
611
+ gpu_devices = gpu_devices[: int(args.gpu_workers)]
612
+
613
+ from Data_Generation_via_LLM_Guided_Search_v12 import Node, tree_search
614
+
615
+ in_path = Path(args.input)
616
+ out_path = Path(args.output)
617
+ out_path.parent.mkdir(parents=True, exist_ok=True)
618
+
619
+ if args.log_wrong_pruned and not args.wrong_pruned_path:
620
+ args.wrong_pruned_path = str(out_path.parent / (out_path.stem + "_wrong_pruned.jsonl"))
621
+
622
+ total = 0
623
+ proved_cnt = 0
624
+ t_start = time.perf_counter()
625
+ resume_skip_remaining = 0
626
+ if args.resume_from_output and out_path.exists():
627
+ with out_path.open("r", encoding="utf-8") as existing_f:
628
+ resume_skip_remaining = sum(1 for line in existing_f if line.strip())
629
+ if resume_skip_remaining:
630
+ print(
631
+ f"Resuming from existing output: skipping first {resume_skip_remaining} eligible samples from {out_path}"
632
+ )
633
+
634
+ try:
635
+ if not args.append:
636
+ descriptor = os.open(
637
+ out_path,
638
+ os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
639
+ 0o600,
640
+ )
641
+ try:
642
+ os.fsync(descriptor)
643
+ finally:
644
+ os.close(descriptor)
645
+ with in_path.open("r", encoding="utf-8") as fin:
646
+ for source_line_number, line in enumerate(fin, start=1):
647
+ line = line.strip()
648
+ if not line:
649
+ continue
650
+ input_sha256 = hashlib.sha256(line.encode("utf-8")).hexdigest()
651
+
652
+ obj: Dict[str, Any] = json.loads(line)
653
+ n = int(obj["n"])
654
+ if args.min_n and n < args.min_n:
655
+ continue
656
+ if args.max_n and n > args.max_n:
657
+ continue
658
+ if resume_skip_remaining > 0:
659
+ resume_skip_remaining -= 1
660
+ continue
661
+ ineq = str(obj["ineq"])
662
+ conditions = obj.get("conditions", [])
663
+
664
+ ineq_map, conds_map, mapping = ineq, conditions, {}
665
+ tree_cap, time_cap = _resolve_sample_budget(
666
+ n=n,
667
+ per_n_budget=per_n_budget,
668
+ fallback_tree_size=args.sample_max_tree_size,
669
+ fallback_time_sec=args.sample_max_time_sec,
670
+ )
671
+ llm_cap = int(args.sample_max_llm_calls or 0)
672
+ llm_fail_cap = int(args.sample_max_llm_fails or 0)
673
+
674
+ root = Node()
675
+ t0 = time.perf_counter()
676
+ try:
677
+ os.environ["QWEN_MAX_NEW_TOKENS"] = str(args.max_new_tokens)
678
+ os.environ["QWEN_MAX_TIME"] = str(args.max_time)
679
+ os.environ["QWEN_PRINT_OUTPUT"] = "1" if args.print_llm else "0"
680
+ os.environ["BEAM_DEBUG"] = "1" if args.debug_search else "0"
681
+ if args.lp_workers and args.lp_workers > 0:
682
+ os.environ.setdefault("OMP_NUM_THREADS", "1")
683
+ os.environ.setdefault("MKL_NUM_THREADS", "1")
684
+ os.environ.setdefault("OPENBLAS_NUM_THREADS", "1")
685
+
686
+ proved, tree, x, expr_str, llm_usage, stop_reason = tree_search(
687
+ root_node=root,
688
+ max_depth=args.max_depth,
689
+ rollout_count=args.rollouts,
690
+ target_inequality=ineq_map,
691
+ conditions=conds_map,
692
+ n=n,
693
+ adapter_path=(args.adapter_path or None),
694
+ use_dummy=args.use_dummy,
695
+ k_per_call=args.k_per_node,
696
+ llm_mode=args.llm_mode,
697
+ lp_workers=args.lp_workers,
698
+ gpu_devices=gpu_devices,
699
+ beam_size=args.beam_size,
700
+ random_prune=args.random_prune,
701
+ prune_seed=(args.prune_seed if args.prune_seed != 0 else None),
702
+ max_vars_for_lp=(args.max_vars_for_lp if args.enable_lp else 0),
703
+ log_wrong_pruned=args.log_wrong_pruned,
704
+ wrong_pruned_path=args.wrong_pruned_path,
705
+ sample_time_budget_sec=time_cap,
706
+ sample_tree_budget=tree_cap,
707
+ sample_llm_calls_budget=llm_cap,
708
+ sample_llm_fail_budget=llm_fail_cap,
709
+ depth1_rollouts=depth1_rollouts,
710
+ )
711
+ err = None
712
+ except Exception as e:
713
+ proved, tree, x, expr_str, llm_usage, stop_reason = False, None, None, None, None, None
714
+ err = f"{type(e).__name__}: {e}"
715
+ fatal_gpu_failure = _is_fatal_gpu_failure(e)
716
+ if fatal_gpu_failure:
717
+ stop_reason = "fatal_gpu_failure"
718
+ else:
719
+ fatal_gpu_failure = False
720
+
721
+ search_dt = time.perf_counter() - t0
722
+ total += 1
723
+ search_reported_proved = bool(proved)
724
+
725
+ out_obj = {
726
+ "source_line_number": source_line_number,
727
+ "input_sha256": input_sha256,
728
+ "input": obj,
729
+ "mapped": {"ineq": ineq_map, "n": n, "mapping": mapping},
730
+ "run": {
731
+ "proved": bool(proved),
732
+ "time_sec": search_dt,
733
+ "search_time_sec": search_dt,
734
+ "tree_size": len(tree) if tree is not None else 0,
735
+ "expr_str": expr_str,
736
+ "x": x.tolist() if hasattr(x, "tolist") else x,
737
+ "error": err,
738
+ "llm_usage": llm_usage or {},
739
+ "execution_config": execution_config,
740
+ },
741
+ }
742
+ apply_exact_verification_gate(
743
+ out_obj["run"],
744
+ ineq_map,
745
+ conds_map,
746
+ n,
747
+ expr_str,
748
+ search_reported_proved=search_reported_proved,
749
+ )
750
+ proved = bool(out_obj["run"]["proved"])
751
+ proved_cnt += int(proved)
752
+ if args.include_proof_trace:
753
+ attach_proof_trace(
754
+ out_obj["run"],
755
+ ineq_map,
756
+ n,
757
+ expr_str,
758
+ search_reported_proved=search_reported_proved,
759
+ )
760
+ if budget_enabled:
761
+ out_obj["run"]["stop_reason"] = stop_reason
762
+ out_obj["run"]["budget_applied"] = {
763
+ "tree_size": int(tree_cap or 0),
764
+ "time_sec": float(time_cap or 0.0),
765
+ "llm_calls": llm_cap,
766
+ "llm_fails": llm_fail_cap,
767
+ "depth1_rollouts": depth1_rollouts,
768
+ "enforcement": "soft_post_expansion",
769
+ }
770
+ elif fatal_gpu_failure:
771
+ out_obj["run"]["stop_reason"] = stop_reason
772
+ dt = time.perf_counter() - t0
773
+ out_obj["run"]["time_sec"] = dt
774
+ _append_jsonl_row(out_path, out_obj)
775
+
776
+ if fatal_gpu_failure:
777
+ sys.stderr.write(
778
+ "[runner] Fatal GPU OOM or worker crash; result prefix was "
779
+ "flushed and no further sample will be started.\n"
780
+ )
781
+ sys.stderr.flush()
782
+ return 12
783
+
784
+ if stop_reason in {"llm_fail_budget", "format_fail_budget"}:
785
+ sys.stderr.write(
786
+ "\n[runner] Sample hit format_fail_total budget. Exiting checkpoint evaluation.\n"
787
+ )
788
+ sys.exit(10)
789
+
790
+ if args.log_every and (total % args.log_every == 0):
791
+ now = time.perf_counter()
792
+ elapsed = now - t_start
793
+ avg = elapsed / max(total, 1)
794
+ msg = (
795
+ f"[progress] {total} samples | proved={proved_cnt} | "
796
+ f"last={dt:.2f}s avg={avg:.2f}s | "
797
+ f"max_depth={args.max_depth} rollouts={args.rollouts} "
798
+ f"max_new_tokens={args.max_new_tokens} max_time={args.max_time}"
799
+ )
800
+ if stop_reason:
801
+ msg += f" | stop_reason={stop_reason}"
802
+ meta = obj.get("meta") if isinstance(obj, dict) else None
803
+ if isinstance(meta, dict):
804
+ msg += f" | seed={meta.get('master_seed')} idx={meta.get('index_within_type')}"
805
+ msg += "\n"
806
+ sys.stderr.write(msg)
807
+ sys.stderr.flush()
808
+
809
+ if args.limit and total >= args.limit:
810
+ break
811
+ except KeyboardInterrupt:
812
+ print(f"\nInterrupted. partial output saved to {out_path}")
813
+
814
+ print(f"Done. total={total} proved={proved_cnt} output={out_path}")
815
+ if args.log_wrong_pruned:
816
+ print(f"Wrong expressions logged to: {args.wrong_pruned_path}")
817
+ return 0
818
+
819
+
820
+ if __name__ == "__main__":
821
+ raise SystemExit(main())
mce220_runner/beam_search_model2/verify_proof_certificate.py ADDED
@@ -0,0 +1,557 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Exact, standalone verifier for v12 entropy proof certificates.
3
+
4
+ The verifier deliberately does not import the search implementation. It parses
5
+ the emitted symbolic expressions into sparse maps whose keys are nonempty
6
+ subset masks and whose values are :class:`fractions.Fraction` objects.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import argparse
12
+ import json
13
+ import re
14
+ import sys
15
+ from dataclasses import dataclass
16
+ from fractions import Fraction
17
+ from pathlib import Path
18
+ from typing import Dict, List, Mapping, Optional, Sequence, Tuple
19
+
20
+
21
+ Vector = Dict[int, Fraction]
22
+
23
+ _NUMBER = r"(?:\d+(?:\.\d*)?|\.\d+)"
24
+ _NUMBER_RE = re.compile(_NUMBER)
25
+ _COMPARATOR_RE = re.compile(r"(>=|<=|=|>|<)\s*0(?:\.0*)?\s*$")
26
+ _VARIABLE_RE = re.compile(r"X_(?:\{(\d+(?:,\d+)*)\}|(\d+))")
27
+
28
+
29
+ class ProofCertificateError(ValueError):
30
+ """Base class for a malformed certificate or result row."""
31
+
32
+
33
+ class ProofSyntaxError(ProofCertificateError):
34
+ """An entropy expression is outside the verifier's accepted grammar."""
35
+
36
+
37
+ @dataclass(frozen=True)
38
+ class Atom:
39
+ kind: str
40
+ groups: Tuple[int, ...]
41
+
42
+
43
+ @dataclass(frozen=True)
44
+ class ParsedExpression:
45
+ vector: Vector
46
+ atom: Optional[Atom]
47
+ atom_coefficient: Optional[Fraction]
48
+
49
+
50
+ @dataclass(frozen=True)
51
+ class ParsedConstraint:
52
+ expression: ParsedExpression
53
+ comparator: str
54
+
55
+
56
+ @dataclass(frozen=True)
57
+ class VerifiedStep:
58
+ expression: str
59
+ kind: str
60
+ multiplier: Fraction
61
+ condition_index: Optional[int] = None
62
+
63
+
64
+ @dataclass(frozen=True)
65
+ class VerificationResult:
66
+ valid: bool
67
+ reason: str
68
+ steps: Tuple[VerifiedStep, ...]
69
+ residual: Vector
70
+ terminal_kind: Optional[str]
71
+
72
+
73
+ def _clean(vector: Mapping[int, Fraction]) -> Vector:
74
+ return {mask: value for mask, value in vector.items() if value}
75
+
76
+
77
+ def _add(target: Vector, source: Mapping[int, Fraction], scale: Fraction = Fraction(1)) -> None:
78
+ for mask, value in source.items():
79
+ new_value = target.get(mask, Fraction(0)) + scale * value
80
+ if new_value:
81
+ target[mask] = new_value
82
+ else:
83
+ target.pop(mask, None)
84
+
85
+
86
+ def _scaled(vector: Mapping[int, Fraction], scale: Fraction) -> Vector:
87
+ return _clean({mask: scale * value for mask, value in vector.items()})
88
+
89
+
90
+ def _parse_scalar(text: str) -> Fraction:
91
+ if text in ("", "+"):
92
+ return Fraction(1)
93
+ if text == "-":
94
+ return Fraction(-1)
95
+ if not re.fullmatch(rf"[+-]?{_NUMBER}", text):
96
+ raise ProofSyntaxError(f"invalid rational coefficient {text!r}")
97
+ # Fraction parses decimal strings exactly (for example, 0.1 as 1/10).
98
+ return Fraction(text)
99
+
100
+
101
+ def _matching_paren(text: str, opening: int) -> int:
102
+ depth = 0
103
+ for index in range(opening, len(text)):
104
+ char = text[index]
105
+ if char == "(":
106
+ depth += 1
107
+ elif char == ")":
108
+ depth -= 1
109
+ if depth == 0:
110
+ return index
111
+ if depth < 0:
112
+ break
113
+ raise ProofSyntaxError(f"unbalanced parentheses near {text[opening:]!r}")
114
+
115
+
116
+ def _outer_scalar(text: str) -> Optional[Tuple[Fraction, str]]:
117
+ opening = text.find("(")
118
+ if opening < 0 or _matching_paren(text, opening) != len(text) - 1:
119
+ return None
120
+ prefix = text[:opening]
121
+ # H(...) and I(...) are atoms, not grouping parentheses.
122
+ if prefix in ("H", "I"):
123
+ return None
124
+ if prefix.endswith("*"):
125
+ prefix = prefix[:-1]
126
+ if not prefix:
127
+ raise ProofSyntaxError("a '*' before a grouped expression needs a coefficient")
128
+ if prefix and not re.fullmatch(rf"[+-]?(?:{_NUMBER})?", prefix):
129
+ return None
130
+ return _parse_scalar(prefix), text[opening + 1 : -1]
131
+
132
+
133
+ def _parse_variable_group(text: str, n: int) -> int:
134
+ if not text:
135
+ raise ProofSyntaxError("an information term contains an empty variable group")
136
+ mask = 0
137
+ position = 0
138
+ found = False
139
+ for match in _VARIABLE_RE.finditer(text):
140
+ gap = text[position : match.start()]
141
+ if gap not in ("", ","):
142
+ raise ProofSyntaxError(f"invalid variable-group syntax {text!r}")
143
+ indices = (match.group(1) or match.group(2)).split(",")
144
+ for raw_index in indices:
145
+ index = int(raw_index)
146
+ if index < 1 or index > n:
147
+ raise ProofSyntaxError(
148
+ f"variable X_{index} is outside the declared range 1..{n}"
149
+ )
150
+ mask |= 1 << (index - 1)
151
+ position = match.end()
152
+ found = True
153
+ if not found or position != len(text):
154
+ raise ProofSyntaxError(f"invalid variable-group syntax {text!r}")
155
+ return mask
156
+
157
+
158
+ def _entropy(mask: int, coefficient: Fraction = Fraction(1)) -> Vector:
159
+ return {} if not mask or not coefficient else {mask: coefficient}
160
+
161
+
162
+ def _parse_atom(text: str, n: int) -> Tuple[Vector, Atom]:
163
+ if len(text) < 3 or text[0] not in "HI" or text[1] != "(":
164
+ raise ProofSyntaxError(f"expected H(...) or I(...), found {text!r}")
165
+ if _matching_paren(text, 1) != len(text) - 1:
166
+ raise ProofSyntaxError(f"unexpected text after information term {text!r}")
167
+
168
+ kind = text[0]
169
+ inside = text[2:-1]
170
+ if kind == "H":
171
+ parts = inside.split("|")
172
+ if len(parts) > 2:
173
+ raise ProofSyntaxError(f"too many conditioning bars in {text!r}")
174
+ left = _parse_variable_group(parts[0], n)
175
+ if len(parts) == 1:
176
+ return _entropy(left), Atom("entropy", (left,))
177
+ conditioning = _parse_variable_group(parts[1], n)
178
+ vector = _entropy(left | conditioning)
179
+ _add(vector, _entropy(conditioning), Fraction(-1))
180
+ return vector, Atom("conditional_entropy", (left, conditioning))
181
+
182
+ conditional_parts = inside.split("|")
183
+ if len(conditional_parts) > 2:
184
+ raise ProofSyntaxError(f"too many conditioning bars in {text!r}")
185
+ sides = conditional_parts[0].split(";")
186
+ if len(sides) != 2:
187
+ raise ProofSyntaxError(f"mutual information requires two ';'-separated groups in {text!r}")
188
+ left = _parse_variable_group(sides[0], n)
189
+ right = _parse_variable_group(sides[1], n)
190
+ conditioning = (
191
+ _parse_variable_group(conditional_parts[1], n)
192
+ if len(conditional_parts) == 2
193
+ else 0
194
+ )
195
+ vector: Vector = {}
196
+ _add(vector, _entropy(left | conditioning))
197
+ _add(vector, _entropy(right | conditioning))
198
+ _add(vector, _entropy(left | right | conditioning), Fraction(-1))
199
+ _add(vector, _entropy(conditioning), Fraction(-1))
200
+ atom_kind = "conditional_mutual_information" if conditioning else "mutual_information"
201
+ return vector, Atom(atom_kind, (left, right, conditioning))
202
+
203
+
204
+ def parse_expression(text: str, n: int) -> ParsedExpression:
205
+ """Parse a linear entropy expression into exact sparse coordinates."""
206
+ if not isinstance(text, str):
207
+ raise ProofSyntaxError("an entropy expression must be a string")
208
+ if not isinstance(n, int) or isinstance(n, bool) or n < 1:
209
+ raise ProofSyntaxError(f"n must be a positive integer, found {n!r}")
210
+ compact = re.sub(r"\s+", "", text)
211
+ if compact == "0":
212
+ return ParsedExpression({}, None, None)
213
+ if not compact:
214
+ raise ProofSyntaxError("empty entropy expression")
215
+
216
+ outer = _outer_scalar(compact)
217
+ outer_coefficient = Fraction(1)
218
+ if outer is not None:
219
+ outer_coefficient, compact = outer
220
+ if not compact:
221
+ raise ProofSyntaxError("empty parenthesized expression")
222
+
223
+ vector: Vector = {}
224
+ atoms: List[Tuple[Atom, Fraction]] = []
225
+ position = 0
226
+ while position < len(compact):
227
+ sign = Fraction(1)
228
+ if compact[position] in "+-":
229
+ sign = Fraction(-1) if compact[position] == "-" else Fraction(1)
230
+ position += 1
231
+ if position == len(compact):
232
+ raise ProofSyntaxError("expression ends with a sign")
233
+ elif position != 0:
234
+ raise ProofSyntaxError(f"expected '+' or '-' at offset {position}")
235
+
236
+ number_match = _NUMBER_RE.match(compact, position)
237
+ coefficient = Fraction(1)
238
+ if number_match:
239
+ coefficient = Fraction(number_match.group(0))
240
+ position = number_match.end()
241
+ if position < len(compact) and compact[position] == "*":
242
+ position += 1
243
+ elif position < len(compact) and compact[position] == "*":
244
+ raise ProofSyntaxError(f"unexpected '*' at offset {position}")
245
+
246
+ if position >= len(compact) or compact[position] not in "HI":
247
+ raise ProofSyntaxError(f"expected H(...) or I(...) at offset {position}")
248
+ opening = position + 1
249
+ if opening >= len(compact) or compact[opening] != "(":
250
+ raise ProofSyntaxError(f"expected '(' after {compact[position]!r}")
251
+ closing = _matching_paren(compact, opening)
252
+ atom_vector, atom = _parse_atom(compact[position : closing + 1], n)
253
+ term_coefficient = sign * coefficient * outer_coefficient
254
+ _add(vector, atom_vector, term_coefficient)
255
+ atoms.append((atom, term_coefficient))
256
+ position = closing + 1
257
+
258
+ single_atom = atoms[0][0] if len(atoms) == 1 else None
259
+ single_coefficient = atoms[0][1] if len(atoms) == 1 else None
260
+ return ParsedExpression(vector, single_atom, single_coefficient)
261
+
262
+
263
+ def parse_constraint(
264
+ text: str,
265
+ n: int,
266
+ *,
267
+ default_comparator: Optional[str] = None,
268
+ ) -> ParsedConstraint:
269
+ """Parse an expression followed by an optional comparison with zero."""
270
+ if not isinstance(text, str):
271
+ raise ProofSyntaxError("a constraint must be a string")
272
+ match = _COMPARATOR_RE.search(text)
273
+ if match:
274
+ lhs = text[: match.start()]
275
+ comparator = match.group(1)
276
+ if any(op in lhs for op in (">", "<", "=")):
277
+ raise ProofSyntaxError(f"unexpected comparator in {text!r}")
278
+ else:
279
+ if any(op in text for op in (">", "<", "=")):
280
+ raise ProofSyntaxError(f"constraints must compare against zero: {text!r}")
281
+ if default_comparator is None:
282
+ raise ProofSyntaxError(f"missing comparator in {text!r}")
283
+ lhs = text
284
+ comparator = default_comparator
285
+ if comparator not in (">=", "<=", "=", ">", "<"):
286
+ raise ProofSyntaxError(f"unsupported comparator {comparator!r}")
287
+ return ParsedConstraint(parse_expression(lhs, n), comparator)
288
+
289
+
290
+ def _oriented_vector(constraint: ParsedConstraint) -> Vector:
291
+ if constraint.comparator in ("<=", "<"):
292
+ return _scaled(constraint.expression.vector, Fraction(-1))
293
+ return dict(constraint.expression.vector)
294
+
295
+
296
+ def _multiple(left: Mapping[int, Fraction], right: Mapping[int, Fraction]) -> Optional[Fraction]:
297
+ """Return q when left == q * right, or None for nonmultiples/zero rows."""
298
+ left = _clean(left)
299
+ right = _clean(right)
300
+ if not left or not right or set(left) != set(right):
301
+ return None
302
+ first_mask = next(iter(right))
303
+ ratio = left[first_mask] / right[first_mask]
304
+ if ratio == 0:
305
+ return None
306
+ if all(left[mask] == ratio * right[mask] for mask in right):
307
+ return ratio
308
+ return None
309
+
310
+
311
+ @dataclass(frozen=True)
312
+ class _ConditionFact:
313
+ vector: Vector
314
+ equality: bool
315
+ source_index: int
316
+
317
+
318
+ def _condition_facts(conditions: Sequence[str], n: int) -> List[_ConditionFact]:
319
+ parsed: List[_ConditionFact] = []
320
+ for index, raw in enumerate(conditions):
321
+ sentinel = re.sub(r"\s+", "", raw).upper()
322
+ if sentinel in {"NONE", "NONE=0", "NONE>=0", "(NONE)", "(NONE)=0", "(NONE)>=0"}:
323
+ continue
324
+ condition = parse_constraint(raw, n, default_comparator=">=")
325
+ vector = _oriented_vector(condition)
326
+ if not vector:
327
+ raise ProofSyntaxError(f"condition {index + 1} has an identically zero left-hand side")
328
+ parsed.append(_ConditionFact(vector, condition.comparator == "=", index))
329
+
330
+ # Opposite one-sided rows jointly assert equality. Promote both, matching
331
+ # the v12 search's condition normalization.
332
+ promoted = set()
333
+ for left_index, left in enumerate(parsed):
334
+ if left.equality:
335
+ continue
336
+ for right_index in range(left_index + 1, len(parsed)):
337
+ right = parsed[right_index]
338
+ ratio = _multiple(left.vector, right.vector)
339
+ if not right.equality and ratio is not None and ratio < 0:
340
+ promoted.update((left_index, right_index))
341
+ return [
342
+ _ConditionFact(fact.vector, fact.equality or index in promoted, fact.source_index)
343
+ for index, fact in enumerate(parsed)
344
+ ]
345
+
346
+
347
+ def _classify_step(
348
+ raw_step: str,
349
+ n: int,
350
+ condition_facts: Sequence[_ConditionFact],
351
+ ) -> Tuple[Optional[VerifiedStep], Optional[str], Vector]:
352
+ step = parse_constraint(raw_step, n)
353
+ vector = dict(step.expression.vector)
354
+ if not vector:
355
+ return None, "step has an identically zero left-hand side", vector
356
+ if step.comparator in (">", "<"):
357
+ return None, "strict proof-step comparators are not accepted", vector
358
+ if step.comparator == "<=":
359
+ return None, "proof steps must be written in nonnegative (>=0) orientation", vector
360
+
361
+ if (
362
+ step.comparator == ">="
363
+ and step.expression.atom is not None
364
+ and step.expression.atom_coefficient is not None
365
+ and step.expression.atom_coefficient > 0
366
+ ):
367
+ return (
368
+ VerifiedStep(
369
+ raw_step,
370
+ "shannon_atom",
371
+ step.expression.atom_coefficient,
372
+ None,
373
+ ),
374
+ None,
375
+ vector,
376
+ )
377
+
378
+ for fact in condition_facts:
379
+ multiplier = _multiple(vector, fact.vector)
380
+ if multiplier is None:
381
+ continue
382
+ if fact.equality:
383
+ return (
384
+ VerifiedStep(raw_step, "equality_condition", multiplier, fact.source_index),
385
+ None,
386
+ vector,
387
+ )
388
+ if step.comparator == ">=" and multiplier > 0:
389
+ return (
390
+ VerifiedStep(raw_step, "inequality_condition", multiplier, fact.source_index),
391
+ None,
392
+ vector,
393
+ )
394
+
395
+ return None, "step is neither a positive Shannon atom nor an allowed condition multiple", vector
396
+
397
+
398
+ def verify_certificate(
399
+ n: int,
400
+ target: str,
401
+ conditions: Sequence[str],
402
+ steps: Sequence[str],
403
+ *,
404
+ terminal_residual: Optional[str] = None,
405
+ ) -> VerificationResult:
406
+ """Verify a certificate exactly, returning a diagnostic result.
407
+
408
+ The terminal residual is recomputed as ``target - sum(steps)``. When an
409
+ encoded terminal residual is supplied (for example from ``proof_trace``),
410
+ it must match that recomputation exactly.
411
+ """
412
+ target_constraint = parse_constraint(target, n, default_comparator=">=")
413
+ if target_constraint.comparator in (">", "<"):
414
+ return VerificationResult(False, "strict target comparators are not accepted", (), {}, None)
415
+ if target_constraint.comparator == "=":
416
+ return VerificationResult(False, "equality targets are not supported", (), {}, None)
417
+ target_vector = _oriented_vector(target_constraint)
418
+ facts = _condition_facts(list(conditions), n)
419
+
420
+ residual = dict(target_vector)
421
+ verified_steps: List[VerifiedStep] = []
422
+ for index, raw_step in enumerate(steps):
423
+ if not isinstance(raw_step, str):
424
+ raise ProofSyntaxError(f"step {index + 1} must be a string")
425
+ classified, reason, step_vector = _classify_step(raw_step, n, facts)
426
+ if classified is None:
427
+ return VerificationResult(
428
+ False,
429
+ f"step {index + 1}: {reason}",
430
+ tuple(verified_steps),
431
+ residual,
432
+ None,
433
+ )
434
+ verified_steps.append(classified)
435
+ _add(residual, step_vector, Fraction(-1))
436
+
437
+ residual = _clean(residual)
438
+ if terminal_residual is not None:
439
+ encoded = parse_expression(terminal_residual, n).vector
440
+ if encoded != residual:
441
+ return VerificationResult(
442
+ False,
443
+ "encoded terminal residual does not equal target minus accepted steps",
444
+ tuple(verified_steps),
445
+ residual,
446
+ None,
447
+ )
448
+
449
+ if not residual:
450
+ terminal_kind = "zero_residual"
451
+ elif all(coefficient >= 0 for coefficient in residual.values()):
452
+ terminal_kind = "coordinatewise_nonnegative"
453
+ else:
454
+ return VerificationResult(
455
+ False,
456
+ "terminal residual has a negative joint-entropy coordinate",
457
+ tuple(verified_steps),
458
+ residual,
459
+ None,
460
+ )
461
+
462
+ return VerificationResult(
463
+ True,
464
+ "certificate verified exactly",
465
+ tuple(verified_steps),
466
+ residual,
467
+ terminal_kind,
468
+ )
469
+
470
+
471
+ def verify_result_row(row: Mapping[str, object]) -> VerificationResult:
472
+ """Verify one claimed-proof row in the v12 JSONL output schema."""
473
+ if not isinstance(row, Mapping):
474
+ raise ProofCertificateError("result row must be a JSON object")
475
+ run = row.get("run")
476
+ input_data = row.get("input")
477
+ mapped = row.get("mapped")
478
+ if not isinstance(run, Mapping) or not isinstance(input_data, Mapping):
479
+ raise ProofCertificateError("result row requires object-valued 'input' and 'run' fields")
480
+ if run.get("proved") is not True:
481
+ return VerificationResult(False, "row does not claim a proof", (), {}, None)
482
+
483
+ mapped_data = mapped if isinstance(mapped, Mapping) else {}
484
+ raw_n = mapped_data.get("n", input_data.get("n"))
485
+ if not isinstance(raw_n, int) or isinstance(raw_n, bool) or raw_n < 1:
486
+ raise ProofCertificateError(f"result row has invalid n: {raw_n!r}")
487
+ target = mapped_data.get("ineq", input_data.get("ineq"))
488
+ if not isinstance(target, str):
489
+ raise ProofCertificateError("result row has no string target inequality")
490
+ raw_conditions = mapped_data.get("conditions", input_data.get("conditions", []))
491
+ if not isinstance(raw_conditions, list) or not all(isinstance(item, str) for item in raw_conditions):
492
+ raise ProofCertificateError("result row conditions must be a list of strings")
493
+ raw_steps = run.get("expr_str")
494
+ if not isinstance(raw_steps, list):
495
+ raise ProofCertificateError("a proved result row must contain a list-valued run.expr_str")
496
+
497
+ terminal_residual: Optional[str] = None
498
+ trace = run.get("proof_trace")
499
+ if isinstance(trace, Mapping):
500
+ terminal = trace.get("terminal")
501
+ if isinstance(terminal, Mapping) and "residual" in terminal:
502
+ value = terminal.get("residual")
503
+ if not isinstance(value, str):
504
+ raise ProofCertificateError("proof_trace terminal residual must be a string")
505
+ terminal_residual = value
506
+
507
+ return verify_certificate(
508
+ raw_n,
509
+ target,
510
+ raw_conditions,
511
+ raw_steps,
512
+ terminal_residual=terminal_residual,
513
+ )
514
+
515
+
516
+ def verify_jsonl(path: Path) -> Tuple[int, int, int, int]:
517
+ """Return (valid, invalid, error, skipped_unproved) counts for a v12 file."""
518
+ valid = invalid = errors = skipped = 0
519
+ with path.open("r", encoding="utf-8") as stream:
520
+ for line_number, line in enumerate(stream, start=1):
521
+ if not line.strip():
522
+ continue
523
+ try:
524
+ row = json.loads(line)
525
+ run = row.get("run") if isinstance(row, dict) else None
526
+ if isinstance(run, dict) and run.get("proved") is not True:
527
+ skipped += 1
528
+ continue
529
+ result = verify_result_row(row)
530
+ except (json.JSONDecodeError, ProofCertificateError) as exc:
531
+ errors += 1
532
+ print(f"line {line_number}: error: {exc}", file=sys.stderr)
533
+ continue
534
+ if result.valid:
535
+ valid += 1
536
+ else:
537
+ invalid += 1
538
+ print(f"line {line_number}: invalid: {result.reason}", file=sys.stderr)
539
+ return valid, invalid, errors, skipped
540
+
541
+
542
+ def main(argv: Optional[Sequence[str]] = None) -> int:
543
+ parser = argparse.ArgumentParser(description="Verify v12 symbolic proof certificates exactly.")
544
+ parser.add_argument("result_file", type=Path, help="v12 JSONL result file (read-only)")
545
+ args = parser.parse_args(argv)
546
+ if not args.result_file.is_file():
547
+ parser.error(f"not a file: {args.result_file}")
548
+ valid, invalid, errors, skipped = verify_jsonl(args.result_file)
549
+ print(f"valid: {valid}")
550
+ print(f"invalid: {invalid}")
551
+ print(f"error: {errors}")
552
+ print(f"skipped_unproved: {skipped}")
553
+ return 0 if invalid == 0 and errors == 0 else 1
554
+
555
+
556
+ if __name__ == "__main__":
557
+ raise SystemExit(main())
mce220_runner/data/__init__.py ADDED
File without changes
mce220_runner/data/entropy_vector_converter.py ADDED
@@ -0,0 +1,250 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Entropy Inequality Vector-String Converter
3
+
4
+ Converts between entropy inequality vector representation and compact string format.
5
+ Vector format: (H(X_1)'s coefficient, H(X_2)'s coefficient, H(X_{1,2})'s coefficient, ...)
6
+ String format: "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})" (compact, no spaces, subscript notation)
7
+ """
8
+
9
+ import numpy as np
10
+ from scipy.sparse import csr_matrix
11
+ import re
12
+
13
+
14
+ def vector_to_string(vec) -> str:
15
+ """
16
+ Convert entropy inequality vector to compact string format.
17
+
18
+ Format: "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})" (no spaces, subscript notation)
19
+
20
+ Args:
21
+ vec: Vector (numpy array, list, or csr_matrix) of length 2^n - 1
22
+
23
+ Returns:
24
+ String like "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})"
25
+
26
+ Example:
27
+ >>> vector_to_string([0, 0, 3, 0])
28
+ '3H(X_{1,2})'
29
+ >>> vector_to_string([1, -2, 0, 0])
30
+ 'H(X_{1}) - 2H(X_{2})'
31
+ """
32
+ # Convert to dense array
33
+ if isinstance(vec, csr_matrix):
34
+ vec_dense = vec.toarray().flatten()
35
+ else:
36
+ vec_dense = np.array(vec).flatten()
37
+
38
+ # Estimate n from vector length: M = 2^n - 1
39
+ n = int(np.log2(len(vec_dense) + 1))
40
+
41
+ terms = []
42
+ for i, coeff in enumerate(vec_dense):
43
+ if abs(coeff) > 1e-6:
44
+ # Convert index to subset representation
45
+ subset_idx = i + 1
46
+ # Convert to binary (low bit first)
47
+ subset_binary = [(subset_idx >> j) & 1 for j in range(n)]
48
+
49
+ # Build subset string in format X_{1,2,3}
50
+ vars_list = [f"{j+1}" for j, b in enumerate(subset_binary) if b == 1]
51
+ if not vars_list:
52
+ continue # Skip empty set
53
+
54
+ subset_str = ",".join(vars_list)
55
+
56
+ # Format coefficient and entropy term
57
+ if abs(coeff - 1.0) < 1e-6:
58
+ # Coefficient is 1
59
+ if coeff < 0:
60
+ terms.append(f"-H(X_{{{subset_str}}})")
61
+ else:
62
+ terms.append(f"H(X_{{{subset_str}}})")
63
+ elif abs(coeff + 1.0) < 1e-6:
64
+ # Coefficient is -1
65
+ terms.append(f"-H(X_{{{subset_str}}})")
66
+ else:
67
+ # Other coefficients
68
+ coeff_int = int(round(coeff))
69
+ if abs(coeff - coeff_int) < 1e-6:
70
+ # Integer coefficient
71
+ if coeff < 0:
72
+ terms.append(f"-{abs(coeff_int)}H(X_{{{subset_str}}})")
73
+ else:
74
+ terms.append(f"{coeff_int}H(X_{{{subset_str}}})")
75
+ else:
76
+ # Non-integer coefficient
77
+ if coeff < 0:
78
+ terms.append(f"-{abs(coeff)}H(X_{{{subset_str}}})")
79
+ else:
80
+ terms.append(f"{coeff}H(X_{{{subset_str}}})")
81
+
82
+ if not terms:
83
+ return "0"
84
+
85
+ # Join terms: positive terms with "+", negative terms already have "-"
86
+ result = terms[0]
87
+ for term in terms[1:]:
88
+ if term.startswith('-'):
89
+ result += term # Already has "-"
90
+ else:
91
+ result += "+" + term
92
+
93
+ return result
94
+
95
+
96
+ def string_to_vector(ineq_str: str, n_variables: int = None) -> csr_matrix:
97
+ """
98
+ Convert entropy inequality string to vector representation.
99
+
100
+ Parses strings like "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})"
101
+ and converts to CSR sparse matrix of length 2^n - 1.
102
+
103
+ Args:
104
+ ineq_str: String like "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})"
105
+ n_variables: Number of variables (if None, inferred from string)
106
+
107
+ Returns:
108
+ CSR sparse matrix (1 x (2^n - 1)) representing the inequality vector
109
+
110
+ Example:
111
+ >>> string_to_vector("3H(X_{1,2})").toarray()
112
+ array([[0., 0., 3.]])
113
+ >>> string_to_vector("H(X_{1})-2H(X_{2})").toarray()
114
+ array([[ 1., -2., 0.]])
115
+ """
116
+ # Remove spaces for easier parsing
117
+ ineq_str_clean = ineq_str.replace(' ', '')
118
+
119
+ # Pattern to match: optional sign at start of term, optional coefficient, H(X_{...})
120
+ # This pattern handles: "3H(X_{1,2})", "-H(X_{1})", "H(X_{1,2,3})", "-2H(X_{1,2})"
121
+ # Also handles cases like "+3H" or "-2H"
122
+ pattern = r'([+-]?)(\d*\.?\d*)H\(X_\{([^}]+)\}\)'
123
+
124
+ matches = re.findall(pattern, ineq_str_clean)
125
+
126
+ if not matches:
127
+ # Try to parse "0" or empty string
128
+ if ineq_str.strip() == "0" or not ineq_str.strip():
129
+ # Need n_variables to create zero vector
130
+ if n_variables is None:
131
+ raise ValueError("Cannot infer n_variables from '0'. Please specify n_variables.")
132
+ return csr_matrix((1, 2**n_variables - 1), dtype=np.float64)
133
+ else:
134
+ raise ValueError(f"Could not parse inequality string: {ineq_str}")
135
+
136
+ # Infer n_variables from the largest variable index found
137
+ max_var = 0
138
+ for sign, coeff, subset_str in matches:
139
+ vars_list = [int(x.strip()) for x in subset_str.split(',') if x.strip()]
140
+ if vars_list:
141
+ max_var = max(max_var, max(vars_list))
142
+
143
+ if n_variables is None:
144
+ n_variables = max_var if max_var > 0 else 1
145
+
146
+ # Create vector
147
+ M = 2**n_variables - 1
148
+ vec = np.zeros(M)
149
+
150
+ for sign, coeff_str, subset_str in matches:
151
+ # Parse coefficient
152
+ if not coeff_str or coeff_str == '.':
153
+ coeff = 1.0
154
+ else:
155
+ coeff = float(coeff_str)
156
+
157
+ # Apply sign
158
+ if sign == '-':
159
+ coeff = -coeff
160
+
161
+ # Parse subset
162
+ vars_list = [int(x.strip()) for x in subset_str.split(',') if x.strip()]
163
+
164
+ # Convert subset to index
165
+ # Index = sum(2^(i-1) for i in vars_list) - 1
166
+ subset_idx = 0
167
+ for var_idx in vars_list:
168
+ if 1 <= var_idx <= n_variables:
169
+ subset_idx += 2**(var_idx - 1)
170
+
171
+ vec_idx = subset_idx - 1
172
+
173
+ if 0 <= vec_idx < M:
174
+ vec[vec_idx] += coeff
175
+
176
+ # Convert to CSR sparse matrix (1 x M)
177
+ return csr_matrix(vec, shape=(1, M), dtype=np.float64)
178
+
179
+
180
+ # Example usage and test
181
+ if __name__ == "__main__":
182
+ # Test vector_to_string
183
+ print("Testing vector_to_string:")
184
+ vec1 = np.array([0, 0, 3, 0])
185
+ str1 = vector_to_string(vec1)
186
+ print(f" Vector {vec1} -> '{str1}'")
187
+ # Expected: "3H(X_{1,2})"
188
+
189
+ vec2 = np.array([1, -2, 0, 0])
190
+ str2 = vector_to_string(vec2)
191
+ print(f" Vector {vec2} -> '{str2}'")
192
+ # Expected: "H(X_{1}) - 2H(X_{2})"
193
+
194
+ vec3 = np.array([0, 0, 0, 1, 0, 0, -1])
195
+ str3 = vector_to_string(vec3)
196
+ print(f" Vector {vec3} -> '{str3}'")
197
+ # Expected: "H(X_{3}) - H(X_{1,2,3})" (for n=3)
198
+
199
+ print("\nTesting string_to_vector:")
200
+ # Test string_to_vector
201
+ str_test1 = "3H(X_{1,2})"
202
+ vec_back1 = string_to_vector(str_test1)
203
+ print(f" String '{str_test1}' -> CSR Matrix")
204
+ print(f" Type: {type(vec_back1)}")
205
+ print(f" Shape: {vec_back1.shape}")
206
+ print(f" Dense: {vec_back1.toarray().flatten()}")
207
+ print(f" Non-zero elements: {vec_back1.nnz}")
208
+
209
+ str_test2 = "H(X_{1}) - 2H(X_{2})"
210
+ vec_back2 = string_to_vector(str_test2)
211
+ print(f"\n String '{str_test2}' -> CSR Matrix")
212
+ print(f" Type: {type(vec_back2)}")
213
+ print(f" Shape: {vec_back2.shape}")
214
+ print(f" Dense: {vec_back2.toarray().flatten()}")
215
+ print(f" Non-zero elements: {vec_back2.nnz}")
216
+
217
+ str_test3 = "H(X_{1})-2H(X_{2})" # No spaces version
218
+ vec_back3 = string_to_vector(str_test3)
219
+ print(f"\n String '{str_test3}' -> CSR Matrix")
220
+ print(f" Dense: {vec_back3.toarray().flatten()}")
221
+
222
+ print("\nRound-trip test:")
223
+ # Round-trip test - use proper length vector (for n=2, length should be 3)
224
+ original_vec = np.array([0, 0, 3]) # n=2: [H(X1), H(X2), H(X1,X2)]
225
+ str_repr = vector_to_string(original_vec)
226
+ recovered_vec = string_to_vector(str_repr)
227
+ print(f" Original: {original_vec}")
228
+ print(f" String: '{str_repr}'")
229
+ print(f" Recovered (CSR): {recovered_vec.toarray().flatten()}")
230
+ print(f" Match: {np.allclose(original_vec, recovered_vec.toarray().flatten())}")
231
+
232
+ # Another round-trip test with n=2
233
+ original_vec2 = np.array([1, -2, 0])
234
+ str_repr2 = vector_to_string(original_vec2)
235
+ recovered_vec2 = string_to_vector(str_repr2)
236
+ print(f"\n Original: {original_vec2}")
237
+ print(f" String: '{str_repr2}'")
238
+ print(f" Recovered (CSR): {recovered_vec2.toarray().flatten()}")
239
+ print(f" Match: {np.allclose(original_vec2, recovered_vec2.toarray().flatten())}")
240
+
241
+ # Test CSR round-trip
242
+ print("\nCSR round-trip test:")
243
+ original_csr = csr_matrix([1, -2, 3])
244
+ str_repr_csr = vector_to_string(original_csr)
245
+ recovered_csr = string_to_vector(str_repr_csr)
246
+ print(f" Original CSR: {original_csr.toarray().flatten()}")
247
+ print(f" String: '{str_repr_csr}'")
248
+ print(f" Recovered CSR: {recovered_csr.toarray().flatten()}")
249
+ print(f" Match: {np.allclose(original_csr.toarray().flatten(), recovered_csr.toarray().flatten())}")
250
+ print(f" Both are CSR: {isinstance(original_csr, csr_matrix) and isinstance(recovered_csr, csr_matrix)}")
mce220_runner/dataset/README.md ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pretty_name: Multi-Construction Entropy-Proof Benchmark (MCE-220)
3
+ language:
4
+ - en
5
+ size_categories:
6
+ - n<1K
7
+ tags:
8
+ - benchmark
9
+ - information-theory
10
+ - entropy
11
+ - automated-theorem-proving
12
+ - synthetic
13
+ configs:
14
+ - config_name: default
15
+ data_files:
16
+ - split: test
17
+ path: data/questions.jsonl
18
+ ---
19
+
20
+ # Multi-Construction Entropy-Proof Benchmark (MCE-220)
21
+
22
+ MCE-220 is a frozen benchmark of 220 synthetic Shannon-type entropy-inequality proof problems. It spans 10 through 20 random variables with 20 questions at every value of `n`.
23
+
24
+ The benchmark is balanced across five construction families. At every `n`, it contains four questions from each family:
25
+
26
+ - weighted modular / hypergraphical constructions;
27
+ - finite-field linearly representable constructions over GF(61);
28
+ - directed-acyclic-graph constructions;
29
+ - deterministic finite-field functional constructions; and
30
+ - finite abelian 2-group quotient constructions.
31
+
32
+ It is also divided evenly into 110 condition-required questions and 110 Shannon-only questions. The benchmark is intended for evaluating automated entropy-inequality provers, proof-search systems, and exact symbolic verification pipelines.
33
+
34
+ ## Dataset structure
35
+
36
+ The Hub dataset exposes one `test` split from `data/questions.jsonl`. Each of its 220 records has:
37
+
38
+ - `id`: stable benchmark identifier;
39
+ - `n`: number of random variables;
40
+ - `ineq`: target entropy expression, interpreted as `ineq >= 0`;
41
+ - `conditions`: supplied equality conditions, represented as paired one-sided inequalities; and
42
+ - `sha256`: digest of the canonical semantic question representation.
43
+
44
+ The files under `audit/` are reproducibility evidence and are intentionally excluded from the dataset split:
45
+
46
+ - `provenance.jsonl`: construction witness, planted certificate, tokenization, seeds, and structural metadata for every question;
47
+ - `generation-report.jsonl`: per-cell generation and rejection accounting;
48
+ - `run-manifest.json`: frozen configuration, dependency hashes, tokenizer identity, output hashes, and completion summary;
49
+ - `validation-report.json`: exact certificate and artifact-integrity audit; and
50
+ - `diversity-audit.json`: balance, uniqueness, and structural-diversity audit.
51
+
52
+ The generation configuration is preserved as `generation/e2n_v2.toml`.
53
+
54
+ ## Generation and tokenization
55
+
56
+ MCE-220 was generated from a preregistered grid of 11 values of `n`, five construction families, and four proof recipes. Each selected problem retains a planted exact certificate and a construction witness. Token counts use the fast tokenizer for `Qwen/Qwen3-0.6B` at commit `c1899de289a04d12100db370d81485cdf75e47ca`, without special tokens.
57
+
58
+ The five construction families provide complementary entropic witnesses: modular component coverage, finite-field matrix ranks, rational Bayesian networks, deterministic polynomial transformations, and quotient profiles of finite abelian groups.
59
+
60
+ ## Validation and integrity
61
+
62
+ The frozen package contains 220 unique question records and 220 matching provenance records. All selected certificates pass exact sparse rational verification. The validation report records zero errors, and the diversity audit passes all declared balance and uniqueness checks.
63
+
64
+ Primary artifact SHA-256 values:
65
+
66
+ - `questions.jsonl`: `1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a`
67
+ - `provenance.jsonl`: `cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189`
68
+ - `generation-report.jsonl`: `50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c`
69
+
70
+ ## Intended use and limitations
71
+
72
+ MCE-220 is an evaluation benchmark, not a training corpus or a representative sample of every entropy inequality. Its questions are synthetic and constructed to have planted Shannon-type certificates. The controlled sparse construction does not establish performance on arbitrary dense inequalities or non-Shannon inequalities. Planted certificate length is not necessarily the minimum proof length, because a question may admit other valid proofs.
73
+
74
+ The benchmark supplies equality conditions because the evaluated AITIP workflow supports equality premises. Results should retain question-level accounting, exact proof verification, and resource or timeout settings.
75
+
76
+ ## Citation
77
+
78
+ The associated manuscript is forthcoming/unpublished:
79
+
80
+ ```bibtex
81
+ @unpublished{wong2026automated,
82
+ title = {Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search},
83
+ author = {Wong, Shing Yin and Liu, Shaocheng and Song, Linqi and Gohari, Amin and Li, Cheuk Ting},
84
+ year = {2026},
85
+ note = {Manuscript in preparation}
86
+ }
87
+ ```
88
+
89
+ No reuse license has been assigned to this private release.
mce220_runner/dataset/audit/diversity-audit.json ADDED
@@ -0,0 +1,422 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "counts": {
3
+ "by_method_n": {
4
+ "abelian_product_v2": {
5
+ "10": 4,
6
+ "11": 4,
7
+ "12": 4,
8
+ "13": 4,
9
+ "14": 4,
10
+ "15": 4,
11
+ "16": 4,
12
+ "17": 4,
13
+ "18": 4,
14
+ "19": 4,
15
+ "20": 4
16
+ },
17
+ "dag_v2": {
18
+ "10": 4,
19
+ "11": 4,
20
+ "12": 4,
21
+ "13": 4,
22
+ "14": 4,
23
+ "15": 4,
24
+ "16": 4,
25
+ "17": 4,
26
+ "18": 4,
27
+ "19": 4,
28
+ "20": 4
29
+ },
30
+ "functional_v2": {
31
+ "10": 4,
32
+ "11": 4,
33
+ "12": 4,
34
+ "13": 4,
35
+ "14": 4,
36
+ "15": 4,
37
+ "16": 4,
38
+ "17": 4,
39
+ "18": 4,
40
+ "19": 4,
41
+ "20": 4
42
+ },
43
+ "linear_rank_gf61_v2": {
44
+ "10": 4,
45
+ "11": 4,
46
+ "12": 4,
47
+ "13": 4,
48
+ "14": 4,
49
+ "15": 4,
50
+ "16": 4,
51
+ "17": 4,
52
+ "18": 4,
53
+ "19": 4,
54
+ "20": 4
55
+ },
56
+ "weighted_modular_v2": {
57
+ "10": 4,
58
+ "11": 4,
59
+ "12": 4,
60
+ "13": 4,
61
+ "14": 4,
62
+ "15": 4,
63
+ "16": 4,
64
+ "17": 4,
65
+ "18": 4,
66
+ "19": 4,
67
+ "20": 4
68
+ }
69
+ },
70
+ "by_n": {
71
+ "10": 20,
72
+ "11": 20,
73
+ "12": 20,
74
+ "13": 20,
75
+ "14": 20,
76
+ "15": 20,
77
+ "16": 20,
78
+ "17": 20,
79
+ "18": 20,
80
+ "19": 20,
81
+ "20": 20
82
+ },
83
+ "by_residual_mode": {
84
+ "condition": 110,
85
+ "shannon": 110
86
+ },
87
+ "by_variant_method_n": {
88
+ "abelian_product_v2": {
89
+ "10": {
90
+ "abelian_product_v2": 4
91
+ },
92
+ "11": {
93
+ "abelian_product_v2": 4
94
+ },
95
+ "12": {
96
+ "abelian_product_v2": 4
97
+ },
98
+ "13": {
99
+ "abelian_product_v2": 4
100
+ },
101
+ "14": {
102
+ "abelian_product_v2": 4
103
+ },
104
+ "15": {
105
+ "abelian_product_v2": 4
106
+ },
107
+ "16": {
108
+ "abelian_product_v2": 4
109
+ },
110
+ "17": {
111
+ "abelian_product_v2": 4
112
+ },
113
+ "18": {
114
+ "abelian_product_v2": 4
115
+ },
116
+ "19": {
117
+ "abelian_product_v2": 4
118
+ },
119
+ "20": {
120
+ "abelian_product_v2": 4
121
+ }
122
+ },
123
+ "dag_v2": {
124
+ "10": {
125
+ "dag_v2": 4
126
+ },
127
+ "11": {
128
+ "dag_v2": 4
129
+ },
130
+ "12": {
131
+ "dag_v2": 4
132
+ },
133
+ "13": {
134
+ "dag_v2": 4
135
+ },
136
+ "14": {
137
+ "dag_v2": 4
138
+ },
139
+ "15": {
140
+ "dag_v2": 4
141
+ },
142
+ "16": {
143
+ "dag_v2": 4
144
+ },
145
+ "17": {
146
+ "dag_v2": 4
147
+ },
148
+ "18": {
149
+ "dag_v2": 4
150
+ },
151
+ "19": {
152
+ "dag_v2": 4
153
+ },
154
+ "20": {
155
+ "dag_v2": 4
156
+ }
157
+ },
158
+ "functional_v2": {
159
+ "10": {
160
+ "linear": 1,
161
+ "mixed": 2,
162
+ "nonlinear": 1
163
+ },
164
+ "11": {
165
+ "linear": 1,
166
+ "mixed": 2,
167
+ "nonlinear": 1
168
+ },
169
+ "12": {
170
+ "linear": 1,
171
+ "mixed": 2,
172
+ "nonlinear": 1
173
+ },
174
+ "13": {
175
+ "linear": 1,
176
+ "mixed": 2,
177
+ "nonlinear": 1
178
+ },
179
+ "14": {
180
+ "linear": 1,
181
+ "mixed": 2,
182
+ "nonlinear": 1
183
+ },
184
+ "15": {
185
+ "linear": 1,
186
+ "mixed": 2,
187
+ "nonlinear": 1
188
+ },
189
+ "16": {
190
+ "linear": 1,
191
+ "mixed": 2,
192
+ "nonlinear": 1
193
+ },
194
+ "17": {
195
+ "linear": 1,
196
+ "mixed": 2,
197
+ "nonlinear": 1
198
+ },
199
+ "18": {
200
+ "linear": 1,
201
+ "mixed": 2,
202
+ "nonlinear": 1
203
+ },
204
+ "19": {
205
+ "linear": 1,
206
+ "mixed": 2,
207
+ "nonlinear": 1
208
+ },
209
+ "20": {
210
+ "linear": 1,
211
+ "mixed": 2,
212
+ "nonlinear": 1
213
+ }
214
+ },
215
+ "linear_rank_gf61_v2": {
216
+ "10": {
217
+ "linear_rank_gf61_v2": 4
218
+ },
219
+ "11": {
220
+ "linear_rank_gf61_v2": 4
221
+ },
222
+ "12": {
223
+ "linear_rank_gf61_v2": 4
224
+ },
225
+ "13": {
226
+ "linear_rank_gf61_v2": 4
227
+ },
228
+ "14": {
229
+ "linear_rank_gf61_v2": 4
230
+ },
231
+ "15": {
232
+ "linear_rank_gf61_v2": 4
233
+ },
234
+ "16": {
235
+ "linear_rank_gf61_v2": 4
236
+ },
237
+ "17": {
238
+ "linear_rank_gf61_v2": 4
239
+ },
240
+ "18": {
241
+ "linear_rank_gf61_v2": 4
242
+ },
243
+ "19": {
244
+ "linear_rank_gf61_v2": 4
245
+ },
246
+ "20": {
247
+ "linear_rank_gf61_v2": 4
248
+ }
249
+ },
250
+ "weighted_modular_v2": {
251
+ "10": {
252
+ "shared": 2,
253
+ "weighted": 2
254
+ },
255
+ "11": {
256
+ "shared": 2,
257
+ "weighted": 2
258
+ },
259
+ "12": {
260
+ "shared": 2,
261
+ "weighted": 2
262
+ },
263
+ "13": {
264
+ "shared": 2,
265
+ "weighted": 2
266
+ },
267
+ "14": {
268
+ "shared": 2,
269
+ "weighted": 2
270
+ },
271
+ "15": {
272
+ "shared": 2,
273
+ "weighted": 2
274
+ },
275
+ "16": {
276
+ "shared": 2,
277
+ "weighted": 2
278
+ },
279
+ "17": {
280
+ "shared": 2,
281
+ "weighted": 2
282
+ },
283
+ "18": {
284
+ "shared": 2,
285
+ "weighted": 2
286
+ },
287
+ "19": {
288
+ "shared": 2,
289
+ "weighted": 2
290
+ },
291
+ "20": {
292
+ "shared": 2,
293
+ "weighted": 2
294
+ }
295
+ }
296
+ },
297
+ "complete_cells": 220,
298
+ "generation_cells": 220,
299
+ "provenance": 220,
300
+ "questions": 220,
301
+ "structural_signature_count_by_method_n": {
302
+ "abelian_product_v2": {
303
+ "10": 4,
304
+ "11": 4,
305
+ "12": 4,
306
+ "13": 4,
307
+ "14": 4,
308
+ "15": 4,
309
+ "16": 4,
310
+ "17": 4,
311
+ "18": 4,
312
+ "19": 4,
313
+ "20": 4
314
+ },
315
+ "dag_v2": {
316
+ "10": 4,
317
+ "11": 4,
318
+ "12": 4,
319
+ "13": 4,
320
+ "14": 4,
321
+ "15": 4,
322
+ "16": 4,
323
+ "17": 4,
324
+ "18": 4,
325
+ "19": 4,
326
+ "20": 4
327
+ },
328
+ "functional_v2": {
329
+ "10": 4,
330
+ "11": 4,
331
+ "12": 4,
332
+ "13": 4,
333
+ "14": 4,
334
+ "15": 4,
335
+ "16": 4,
336
+ "17": 4,
337
+ "18": 4,
338
+ "19": 4,
339
+ "20": 4
340
+ },
341
+ "linear_rank_gf61_v2": {
342
+ "10": 4,
343
+ "11": 4,
344
+ "12": 4,
345
+ "13": 4,
346
+ "14": 4,
347
+ "15": 4,
348
+ "16": 4,
349
+ "17": 4,
350
+ "18": 4,
351
+ "19": 4,
352
+ "20": 4
353
+ },
354
+ "weighted_modular_v2": {
355
+ "10": 2,
356
+ "11": 2,
357
+ "12": 2,
358
+ "13": 2,
359
+ "14": 2,
360
+ "15": 2,
361
+ "16": 2,
362
+ "17": 2,
363
+ "18": 2,
364
+ "19": 2,
365
+ "20": 2
366
+ }
367
+ }
368
+ },
369
+ "error_count": 0,
370
+ "errors": [],
371
+ "expectations": {
372
+ "expected_per_method_n": 4,
373
+ "expected_per_n": 20,
374
+ "expected_total": 220,
375
+ "methods": [
376
+ "weighted_modular_v2",
377
+ "linear_rank_gf61_v2",
378
+ "dag_v2",
379
+ "functional_v2",
380
+ "abelian_product_v2"
381
+ ],
382
+ "n_values": [
383
+ 10,
384
+ 11,
385
+ 12,
386
+ 13,
387
+ 14,
388
+ 15,
389
+ 16,
390
+ 17,
391
+ 18,
392
+ 19,
393
+ 20
394
+ ],
395
+ "require_complete_cells": true,
396
+ "require_exact_uniqueness": true,
397
+ "residual_modes": [
398
+ "condition",
399
+ "shannon"
400
+ ]
401
+ },
402
+ "generation_report_rows": 220,
403
+ "pass": true,
404
+ "rejections": {
405
+ "by_reason": {
406
+ "token_window": 8
407
+ },
408
+ "total": 8
409
+ },
410
+ "schema_version": "reviewer-witness-diversity-audit-v1",
411
+ "uniqueness": {
412
+ "duplicate_condition_set_groups": [],
413
+ "duplicate_initial_target_groups": [],
414
+ "duplicate_selected_residual_groups": [],
415
+ "duplicate_witness_groups": [],
416
+ "unique_condition_sets": 220,
417
+ "unique_initial_targets": 220,
418
+ "unique_selected_residuals": 220,
419
+ "unique_witnesses": 220,
420
+ "witness_scope": "within each method and n"
421
+ }
422
+ }
mce220_runner/dataset/audit/generation-report.jsonl ADDED
@@ -0,0 +1,220 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3701457636437077952,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
2
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15246363881976218358,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
3
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3172063088685357728,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
4
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5056663783486606013,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
5
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9539850679019901972,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
6
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7955716649502997116,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
7
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4812851374726980080,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
8
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4284913358355807734,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
9
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8708808592976883946,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
10
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14394515690009120654,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
11
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5074899767269115176,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
12
+ {"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":8138142372704579071,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":3,"rejected":{"token_window":1},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
13
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11237412930374853913,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
14
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2622604060345787656,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
15
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1297526053728799686,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
16
+ {"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":12938408747832804044,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":3,"rejected":{"token_window":1},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
17
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15148184769047820,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
18
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11212742246525318633,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
19
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9952603600299452875,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
20
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17684431903806972992,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
21
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4358867344586403829,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
22
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13909308547342205491,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
23
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15975291380394989210,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
24
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9716087254387917306,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
25
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17539831590544119906,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
26
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11853313365675990415,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
27
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10445827094562493869,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
28
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4702628459963138141,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
29
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6338436637880042951,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
30
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4623525552571304476,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
31
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8389339039156670354,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
32
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16353085863076184214,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
33
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2196169065690155443,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
34
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15059909006415722323,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
35
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14583201521060589027,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
36
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5612252589744170130,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
37
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2728962570584640889,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
38
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9758846155311160259,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
39
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2720845152774216335,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
40
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5121022496212244148,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
41
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16436906769862432774,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
42
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14831573543779349003,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
43
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4480046151243417079,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
44
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9903418647447938583,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
45
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4936832681239338862,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
46
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15232824178863167110,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
47
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8186240528289594329,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
48
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17500194849731588113,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
49
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11832034070095118311,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
50
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8133750959736225035,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
51
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2273770823316088623,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
52
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1462948322211576970,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
53
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18067028359118754372,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
54
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11705998351666552518,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
55
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16254702609706164343,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
56
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2968902127481300008,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
57
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5936000122885923694,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
58
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15604122199806785487,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
59
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9337414342802015969,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
60
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16923520896556911049,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
61
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1464669209368047966,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
62
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10174962824566969569,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
63
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12100853506115398562,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
64
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5228631607116939670,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
65
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6393201095674814746,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
66
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15285263391928485372,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
67
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":549978287588786307,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
68
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5273822472737093431,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
69
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17505444870217075492,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
70
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12065121403697794235,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
71
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15963481462025769689,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
72
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9888338464856776367,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
73
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11743591167866989887,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
74
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5407253042649969843,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
75
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5124340889268399352,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
76
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7866954243082797305,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
77
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7141475136732988801,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
78
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16881613849503902662,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
79
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10510564794829486452,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
80
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13637027472625371970,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
81
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10865444810274626052,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
82
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16407715404674462704,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
83
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2967182459071344429,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
84
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8842815126698378794,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
85
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16996992628039077843,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
86
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11919690030700392770,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
87
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2310562788219762726,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
88
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2443067625292808296,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
89
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3184567625312884139,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
90
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6822340930228026675,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
91
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11359352071557003676,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
92
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11260656603170133794,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
93
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17431958280368089774,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
94
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":95470021134927501,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
95
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18173423042958114467,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
96
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10912102982755826622,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
97
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14647759189369950544,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
98
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17074290522542464048,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
99
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11051767266233656569,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
100
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1918497853187515936,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
101
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1907240975364747342,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
102
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17785309102802652106,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
103
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7181223247123743876,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
104
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2263938430705632913,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
105
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16433270489078283260,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
106
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15721389692426090446,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
107
+ {"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":16971901269964792975,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":14,"quota":1,"recipe_index":2,"rejected":{"token_window":1},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
108
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17994565298475579634,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
109
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8007488871997086332,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
110
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3719582981595767999,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
111
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6989838346124019281,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
112
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2665890865320848013,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
113
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14437300812821433667,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
114
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6577469201406442179,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
115
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13041507654325523808,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
116
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3570771541096614982,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
117
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11815261720096583648,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
118
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16648488661857383898,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
119
+ {"accepted":1,"attempted":3,"benchmark":"e2n-witness-first-v2","cell_seed":12258752852871151798,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":17,"quota":1,"recipe_index":2,"rejected":{"token_window":2},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
120
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6381643106495975100,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
121
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15423199845599069898,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
122
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13268465811212531448,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
123
+ {"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":4070565829160279811,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":18,"quota":1,"recipe_index":2,"rejected":{"token_window":1},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
124
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":105930261327445421,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
125
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10109908015276543465,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
126
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4956562969327863550,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
127
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5008331369838333772,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
128
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17827741488378490065,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
129
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11552473092704060,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
130
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17278969316211330178,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
131
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5863619077421464638,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
132
+ {"accepted":1,"attempted":3,"benchmark":"e2n-witness-first-v2","cell_seed":12216008678726813341,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":20,"quota":1,"recipe_index":3,"rejected":{"token_window":2},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
133
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12136638063071141276,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
134
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6986443780866752496,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
135
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3781574741278654854,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
136
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2343031434917169256,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
137
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":530243949934307503,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
138
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4147310449574325073,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
139
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13253479536535202397,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
140
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5481697251196607234,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
141
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3419907753431159668,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
142
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18349149685442145724,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
143
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12471696088800926995,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
144
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5514582684015399052,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
145
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14479113219010784219,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
146
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7511176911362227743,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
147
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11305014799087235417,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
148
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10456612778851741111,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
149
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17198548155075593163,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
150
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12054220294986305355,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
151
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6317325096853017294,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
152
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5557861230106346201,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
153
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15256452385745187341,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
154
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4984913849055553761,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
155
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1922809319832171575,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
156
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7290953918310180562,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
157
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2207934633545694466,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
158
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8865819739701358330,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
159
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8060021619136953608,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
160
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16572719200988242280,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
161
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4170973570952992437,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
162
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12513124999520014137,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
163
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14786551003661347752,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
164
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7575961521772144425,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
165
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5161629846671816961,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
166
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4552954781211765265,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
167
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12954501444229679227,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
168
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3734117315160446273,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
169
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15062695363987721571,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
170
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12101175090757923717,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
171
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3575707971365674080,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
172
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5105430873146479519,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
173
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15318398621928381177,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
174
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13391810045493605038,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
175
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16474088650880862895,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
176
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9697458621349092067,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
177
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14098164715461887531,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
178
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7413113805965221460,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
179
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8118303950565053392,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
180
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1018118677028597230,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
181
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14973580586272694633,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
182
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1362989527561847042,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
183
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14381275516087489947,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
184
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2369240884198343729,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
185
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":221228626003247890,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
186
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4312760315371342800,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
187
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15528857495560036646,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
188
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1186195495943774426,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
189
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10014313637976231552,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
190
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1552613301840816399,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
191
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":954177495762986129,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
192
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5525319345874424799,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
193
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18027165872156795930,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
194
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18098129184875256583,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
195
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13888201606725553085,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
196
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":919212854944883606,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
197
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8165437892884726391,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
198
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11934378534096084033,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
199
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12133732172664841221,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
200
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5360334427668528317,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
201
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5030154206041038858,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
202
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11647864275748426632,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
203
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4062663468625614856,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
204
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18407997404765423429,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
205
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13118711139945243757,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
206
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3758826947236076007,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
207
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11246400704042830426,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
208
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10213094060571213725,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
209
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9084829840437732960,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
210
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5453944936310748968,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
211
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8525833265372454032,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
212
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2691777673477642656,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
213
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6285703479023902455,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
214
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3138362092886322139,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
215
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1071160566897403333,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
216
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17567046285549651617,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
217
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6736118940594014540,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
218
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15176902604380923354,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
219
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8216170727549535292,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
220
+ {"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1392148745951243703,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
mce220_runner/dataset/audit/provenance.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
mce220_runner/dataset/audit/run-manifest.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"benchmark":"e2n-witness-first-v2","config":{"benchmark":{"coefficient_max":10,"max_attempts_per_cell":240,"name":"e2n-witness-first-v2","output_dir":"../outputs/e2n-v2","quota_per_cell":1,"require_complete":true,"require_unique_witness":true,"seed":20260825},"format":{"prompt_template":"TARGET:\n{target}>=0\n\nCONDITIONS:\n{conditions}\n\n"},"grid":{"methods":["weighted_modular_v2","linear_rank_gf61_v2","dag_v2","functional_v2","abelian_product_v2"],"n_values":[10,11,12,13,14,15,16,17,18,19,20],"recipes":[{"condition_width":8,"k":12,"m_eq":1,"residual_mode":"condition","witness_variants":{"functional_v2":"linear","weighted_modular_v2":"weighted"}},{"condition_width":12,"k":16,"m_eq":1,"residual_mode":"shannon","witness_variants":{"functional_v2":"nonlinear","weighted_modular_v2":"shared"}},{"condition_width":8,"k":20,"m_eq":2,"residual_mode":"condition","witness_variants":{"functional_v2":"mixed","weighted_modular_v2":"weighted"}},{"condition_width":8,"k":16,"m_eq":2,"residual_mode":"shannon","witness_variants":{"functional_v2":"mixed","weighted_modular_v2":"shared"}}]},"tokenization":{"counter":"huggingface","full_max":2200,"full_min":801,"hf_model":"Qwen/Qwen3-0.6B","hf_revision":"c1899de289a04d12100db370d81485cdf75e47ca","hf_use_fast":true,"target_max":799,"target_min":501}},"config_sha256":"1b1f298ed012071c85e9cfb286fe5dcf7c183383de92041754d8a7802f426421","generator_dependency_sha256":{"data/finite_field.py":"c9738c10a16ea2cfb3cde346a23673d1536dbfce491f8871346f441b9b4c2a6e","data/witness_first.py":"7af2a1d6096862a776e9c8530d0c6a3ae2b83aa7ecd8bb1026256d6f1cdc3560","data/witness_v2_algebraic.py":"ffbd48eb673a3f6f842327d00431285bbae8963a7e20f6956e37e544fa37165f","data/witness_v2_graph.py":"c81adcfefcf6ffc0e816f1b338af13323bc80e06293e8cac6b28a3b0ae2efc8a"},"generator_source_sha256":"7af2a1d6096862a776e9c8530d0c6a3ae2b83aa7ecd8bb1026256d6f1cdc3560","output_sha256":{"generation-report.jsonl":"50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c","provenance.jsonl":"cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189","questions.jsonl":"1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a"},"schema_version":"reviewer-benchmark-v1","selector_source_sha256":"83a328168eac42cedd3412a9c8b5624b8683a52a14042b5defbccf6d833efefe","summary":{"benchmark":"e2n-witness-first-v2","cells":220,"complete":true,"complete_cells":220,"questions":220},"tokenizer":{"add_special_tokens":false,"is_fast":true,"kind":"huggingface","model":"Qwen/Qwen3-0.6B","requested_revision":"c1899de289a04d12100db370d81485cdf75e47ca","resolved_commit":"c1899de289a04d12100db370d81485cdf75e47ca","tokenizer_class":"Qwen2TokenizerFast","transformers_version":"4.55.2","trust_remote_code":true,"use_fast":true}}
mce220_runner/dataset/audit/validation-report.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"benchmark":"e2n-witness-first-v2","benchmark_dir":"C:\\Users\\user\\Documents\\ChatGPT\\Entropy Inequality\\experiments\\reviewer_response\\outputs\\e2n-v2","counts":{"artifact_hashes_checked":3,"certificates_verified":220,"errors":0,"provenance_rows":220,"question_rows":220,"rows_checked":220},"errors":[],"schema_version":"reviewer-generated-benchmark-audit-v1","valid":true}
mce220_runner/dataset/data/questions.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
mce220_runner/dataset/generation/e2n_v2.toml ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [benchmark]
2
+ name = "e2n-witness-first-v2"
3
+ seed = 20260825
4
+ output_dir = "../outputs/e2n-v2"
5
+ quota_per_cell = 1
6
+ max_attempts_per_cell = 240
7
+ coefficient_max = 10
8
+ require_complete = true
9
+ require_unique_witness = true
10
+
11
+ [grid]
12
+ methods = [
13
+ "weighted_modular_v2",
14
+ "linear_rank_gf61_v2",
15
+ "dag_v2",
16
+ "functional_v2",
17
+ "abelian_product_v2",
18
+ ]
19
+ n_values = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
20
+
21
+ # Four questions per construction family and n. The model sees equality
22
+ # conditions only. "condition" means at least one equality action remains in
23
+ # the selected proof-completion state; "shannon" means none remains.
24
+ [[grid.recipes]]
25
+ k = 12
26
+ m_eq = 1
27
+ condition_width = 8
28
+ residual_mode = "condition"
29
+ witness_variants = { weighted_modular_v2 = "weighted", functional_v2 = "linear" }
30
+
31
+ [[grid.recipes]]
32
+ k = 16
33
+ m_eq = 1
34
+ condition_width = 12
35
+ residual_mode = "shannon"
36
+ witness_variants = { weighted_modular_v2 = "shared", functional_v2 = "nonlinear" }
37
+
38
+ [[grid.recipes]]
39
+ k = 20
40
+ m_eq = 2
41
+ condition_width = 8
42
+ residual_mode = "condition"
43
+ witness_variants = { weighted_modular_v2 = "weighted", functional_v2 = "mixed" }
44
+
45
+ [[grid.recipes]]
46
+ k = 16
47
+ m_eq = 2
48
+ condition_width = 8
49
+ residual_mode = "shannon"
50
+ witness_variants = { weighted_modular_v2 = "shared", functional_v2 = "mixed" }
51
+
52
+ [tokenization]
53
+ counter = "huggingface"
54
+ hf_model = "Qwen/Qwen3-0.6B"
55
+ hf_revision = "c1899de289a04d12100db370d81485cdf75e47ca"
56
+ hf_use_fast = true
57
+ target_min = 501
58
+ target_max = 799
59
+ full_min = 801
60
+ full_max = 2200
61
+
62
+ [format]
63
+ prompt_template = "TARGET:\n{target}>=0\n\nCONDITIONS:\n{conditions}\n\n"
mce220_runner/download_assets.py ADDED
@@ -0,0 +1,249 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Download and verify the pinned MCE-220 inference assets."""
3
+
4
+ from __future__ import annotations
5
+
6
+ import argparse
7
+ import hashlib
8
+ import json
9
+ from pathlib import Path
10
+
11
+ from huggingface_hub import snapshot_download
12
+
13
+ BUNDLE_DIR = Path(__file__).resolve().parent
14
+ DATASET_REPO = 'entropyinequalityprover/MCE-220'
15
+ DATASET_REVISION = '5650d7c8198a68116e8d12934aed41a37866ba92'
16
+ MODEL_REPO = 'entropyinequalityprover/models_and_scripts'
17
+ MODEL_REVISION = '7fcfe778e1c36cd67e88ea8aa2ee761a50a39127'
18
+ BASE_MODEL_REPO = 'Qwen/Qwen3-0.6B'
19
+ BASE_MODEL_REVISION = 'c1899de289a04d12100db370d81485cdf75e47ca'
20
+ CHECKPOINT_PREFIX = 'qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534'
21
+
22
+ DATASET_SHA256 = {
23
+ "README.md": "909828a43ace9c3d4e9aa2e2fe96812dadd1c08ece52c87cf4dc896ad4ce0452",
24
+ "audit/diversity-audit.json": "56721eb34f35fa9acc2d81bbb13b0ea263a7f29bc8838372339dbc38ecda44a4",
25
+ "audit/generation-report.jsonl": "50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c",
26
+ "audit/provenance.jsonl": "cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189",
27
+ "audit/run-manifest.json": "ac32fdaecf845714f9802c584c91b8743398898ff7e7c1f36c43014dea36d9a6",
28
+ "audit/validation-report.json": "3f86d1391c15a343cb938092a456ce2b5b88bb7f1ebcbf4089d0c8174c7e4d3c",
29
+ "data/questions.jsonl": "1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a",
30
+ "generation/e2n_v2.toml": "3346eb6fc6aa0bb70ca15f9a840d7b0f366418072c58cb76d32e0f8ac9faae4a"
31
+ }
32
+ CHECKPOINT_FILES = (
33
+ "adapter_config.json",
34
+ "adapter_model.safetensors",
35
+ "added_tokens.json",
36
+ "chat_template.jinja",
37
+ "merges.txt",
38
+ "special_tokens_map.json",
39
+ "tokenizer.json",
40
+ "tokenizer_config.json",
41
+ "vocab.json",
42
+ )
43
+ CHECKPOINT_CRITICAL_SHA256 = {
44
+ "adapter_model.safetensors": "813507e95124a06a4f8dc69d641b7fbcde791b4ec7d610f7c9403709588deb56",
45
+ "tokenizer.json": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4",
46
+ }
47
+ EXCLUDED_CHECKPOINT_FILES = (
48
+ "optimizer.pt",
49
+ "scheduler.pt",
50
+ "trainer_state.json",
51
+ "training_args.bin",
52
+ )
53
+ EXCLUDED_CHECKPOINT_PREFIXES = ("rng_state_",)
54
+ BASE_MODEL_FILES = (
55
+ "config.json",
56
+ "generation_config.json",
57
+ "merges.txt",
58
+ "model.safetensors",
59
+ "tokenizer.json",
60
+ "tokenizer_config.json",
61
+ "vocab.json",
62
+ )
63
+ BASE_MODEL_CRITICAL_SHA256 = {
64
+ "model.safetensors": "f47f71177f32bcd101b7573ec9171e6a57f4f4d31148d38e382306f42996874b",
65
+ "tokenizer.json": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4",
66
+ }
67
+
68
+
69
+ class AssetError(RuntimeError):
70
+ pass
71
+
72
+
73
+ def sha256_file(path: Path) -> str:
74
+ digest = hashlib.sha256()
75
+ with path.open("rb") as stream:
76
+ for chunk in iter(lambda: stream.read(1024 * 1024), b""):
77
+ digest.update(chunk)
78
+ return digest.hexdigest()
79
+
80
+
81
+ def visible_files(root: Path) -> set[str]:
82
+ if not root.is_dir():
83
+ return set()
84
+ return {
85
+ path.relative_to(root).as_posix()
86
+ for path in root.rglob("*")
87
+ if path.is_file() and ".cache" not in path.relative_to(root).parts
88
+ }
89
+
90
+
91
+ def require_hashes(root: Path, expected: dict[str, str]) -> None:
92
+ for relative, wanted in expected.items():
93
+ path = root / relative
94
+ if not path.is_file():
95
+ raise AssetError(f"missing required file: {path}")
96
+ actual = sha256_file(path)
97
+ if actual != wanted:
98
+ raise AssetError(
99
+ f"SHA-256 mismatch for {path}: expected {wanted}, got {actual}"
100
+ )
101
+
102
+
103
+ def verify_dataset_tree(root: Path) -> None:
104
+ expected_files = set(DATASET_SHA256)
105
+ actual_files = visible_files(root)
106
+ if actual_files != expected_files:
107
+ raise AssetError(
108
+ "dataset file set mismatch: missing="
109
+ f"{sorted(expected_files - actual_files)}, extra="
110
+ f"{sorted(actual_files - expected_files)}"
111
+ )
112
+ require_hashes(root, DATASET_SHA256)
113
+ question_path = root / "data/questions.jsonl"
114
+ rows = [
115
+ json.loads(line)
116
+ for line in question_path.read_text(encoding="utf-8").splitlines()
117
+ if line.strip()
118
+ ]
119
+ if len(rows) != 220 or len({row.get("id") for row in rows}) != 220:
120
+ raise AssetError("questions.jsonl must contain 220 unique question IDs")
121
+ if any(
122
+ set(row) != {"id", "n", "ineq", "conditions", "sha256"}
123
+ for row in rows
124
+ ):
125
+ raise AssetError("questions.jsonl has an unexpected schema")
126
+
127
+
128
+ def verify_checkpoint(root: Path) -> None:
129
+ expected = set(CHECKPOINT_FILES)
130
+ actual = visible_files(root)
131
+ if actual != expected:
132
+ raise AssetError(
133
+ "checkpoint file set mismatch: missing="
134
+ f"{sorted(expected - actual)}, extra={sorted(actual - expected)}"
135
+ )
136
+ for path in actual:
137
+ name = Path(path).name
138
+ if name in EXCLUDED_CHECKPOINT_FILES or any(
139
+ name.startswith(prefix) for prefix in EXCLUDED_CHECKPOINT_PREFIXES
140
+ ):
141
+ raise AssetError(f"training-only checkpoint file present: {path}")
142
+ require_hashes(root, CHECKPOINT_CRITICAL_SHA256)
143
+
144
+
145
+ def verify_base_model(root: Path) -> None:
146
+ expected = set(BASE_MODEL_FILES)
147
+ actual = visible_files(root)
148
+ if actual != expected:
149
+ raise AssetError(
150
+ "base-model file set mismatch: missing="
151
+ f"{sorted(expected - actual)}, extra={sorted(actual - expected)}"
152
+ )
153
+ require_hashes(root, BASE_MODEL_CRITICAL_SHA256)
154
+
155
+
156
+ def download_assets(asset_dir: Path) -> None:
157
+ asset_dir.mkdir(parents=True, exist_ok=True)
158
+ snapshot_download(
159
+ repo_id=DATASET_REPO,
160
+ repo_type="dataset",
161
+ revision=DATASET_REVISION,
162
+ allow_patterns=list(DATASET_SHA256),
163
+ local_dir=asset_dir / "dataset-source",
164
+ local_dir_use_symlinks=False,
165
+ )
166
+ snapshot_download(
167
+ repo_id=MODEL_REPO,
168
+ repo_type="model",
169
+ revision=MODEL_REVISION,
170
+ allow_patterns=[f"{CHECKPOINT_PREFIX}/{name}" for name in CHECKPOINT_FILES],
171
+ local_dir=asset_dir / "model-repo",
172
+ local_dir_use_symlinks=False,
173
+ )
174
+ snapshot_download(
175
+ repo_id=BASE_MODEL_REPO,
176
+ repo_type="model",
177
+ revision=BASE_MODEL_REVISION,
178
+ allow_patterns=list(BASE_MODEL_FILES),
179
+ local_dir=asset_dir / "qwen3-0.6b",
180
+ local_dir_use_symlinks=False,
181
+ )
182
+
183
+
184
+ def verify_all(asset_dir: Path) -> dict[str, object]:
185
+ bundled_dataset = BUNDLE_DIR / "dataset"
186
+ downloaded_dataset = asset_dir / "dataset-source"
187
+ checkpoint = asset_dir / "model-repo" / CHECKPOINT_PREFIX
188
+ base_model = asset_dir / "qwen3-0.6b"
189
+ verify_dataset_tree(bundled_dataset)
190
+ verify_dataset_tree(downloaded_dataset)
191
+ for relative in DATASET_SHA256:
192
+ if (
193
+ (bundled_dataset / relative).read_bytes()
194
+ != (downloaded_dataset / relative).read_bytes()
195
+ ):
196
+ raise AssetError(f"bundled dataset differs from pinned source: {relative}")
197
+ verify_checkpoint(checkpoint)
198
+ verify_base_model(base_model)
199
+ return {
200
+ "status": "verified",
201
+ "dataset": {
202
+ "repo": DATASET_REPO,
203
+ "revision": DATASET_REVISION,
204
+ "rows": 220,
205
+ },
206
+ "checkpoint": {
207
+ "repo": MODEL_REPO,
208
+ "revision": MODEL_REVISION,
209
+ "path": str(checkpoint),
210
+ },
211
+ "base_model": {
212
+ "repo": BASE_MODEL_REPO,
213
+ "revision": BASE_MODEL_REVISION,
214
+ "path": str(base_model),
215
+ },
216
+ }
217
+
218
+
219
+ def parser() -> argparse.ArgumentParser:
220
+ ap = argparse.ArgumentParser(description=__doc__)
221
+ ap.add_argument(
222
+ "--asset-dir",
223
+ type=Path,
224
+ default=BUNDLE_DIR / "assets",
225
+ help="Asset directory (default: mce220_runner/assets).",
226
+ )
227
+ ap.add_argument(
228
+ "--verify-only",
229
+ action="store_true",
230
+ help="Do not access the network; verify already downloaded assets.",
231
+ )
232
+ return ap
233
+
234
+
235
+ def main() -> int:
236
+ args = parser().parse_args()
237
+ asset_dir = args.asset_dir.expanduser().resolve()
238
+ try:
239
+ if not args.verify_only:
240
+ download_assets(asset_dir)
241
+ print(json.dumps(verify_all(asset_dir), indent=2, sort_keys=True))
242
+ return 0
243
+ except AssetError as exc:
244
+ print(f"asset verification failed: {exc}", file=__import__("sys").stderr)
245
+ return 2
246
+
247
+
248
+ if __name__ == "__main__":
249
+ raise SystemExit(main())
mce220_runner/experiments/__init__.py ADDED
File without changes
mce220_runner/experiments/reviewer_response/__init__.py ADDED
File without changes
mce220_runner/experiments/reviewer_response/configs/inference_policy.toml ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Frozen policy for all new reviewer proposed-model inference (E2-N, E6,
2
+ # E9-A/E9-B and E10). E2-R is a retrospective audit of the submitted run and
3
+ # is intentionally not governed by this new policy.
4
+
5
+ [policy]
6
+ schema_version = "reviewer-inference-policy-v1"
7
+ sample_max_time_sec = 1000
8
+ sample_max_tree_size = 6000
9
+ sample_max_llm_calls = 0 # No separate call cap; time and tree limits still apply.
10
+
11
+ [enforcement]
12
+ time = "cooperative_soft"
13
+ tree = "cooperative_soft"
14
+ hard_wall_time_enforced = false
15
+ external_hard_timeout_required = true
16
+ note = "The runner checks budgets between search operations; already submitted model generations may finish after a budget is reached."
mce220_runner/experiments/reviewer_response/run_search_benchmark.py ADDED
@@ -0,0 +1,1108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Reproducible wrapper for the submitted proposed-model search pipeline.
2
+
3
+ The wrapper deliberately does not import or modify the search runner. It
4
+ freezes the submitted search settings, records a machine-readable manifest,
5
+ executes the tracked runner as a child process, and verifies successful proof
6
+ artifacts with the standalone exact verifier.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import argparse
12
+ import csv
13
+ import hashlib
14
+ import importlib.metadata
15
+ import json
16
+ import os
17
+ import platform
18
+ import shlex
19
+ import subprocess
20
+ import sys
21
+ import time
22
+ import uuid
23
+ from datetime import datetime, timezone
24
+ from pathlib import Path
25
+ from typing import Any, Sequence
26
+
27
+
28
+ # Direct execution sets sys.path[0] to experiments/reviewer_response rather
29
+ # than the repository root. Add the root explicitly so resume verification can
30
+ # import beam_search_model2 in the same way as ``python -m`` execution.
31
+ _REPO_ROOT_FOR_IMPORTS = Path(__file__).resolve().parents[2]
32
+ if str(_REPO_ROOT_FOR_IMPORTS) not in sys.path:
33
+ sys.path.insert(0, str(_REPO_ROOT_FOR_IMPORTS))
34
+
35
+
36
+ SCHEMA_VERSION = "reviewer-search-run-v4"
37
+ EXECUTION_CONFIG_SCHEMA_VERSION = "e2n-execution-config-v1"
38
+ RESOURCE_POLICY_VERSION = "reviewer-inference-policy-v1"
39
+ SAMPLE_MAX_TIME_SEC = 1000.0
40
+ SAMPLE_MAX_TREE_SIZE = 6000
41
+ DEFAULT_RUNNER = Path("beam_search_model2/run_on_window_shear_jsonl_v12.py")
42
+ DEFAULT_GPU_IDS = "4,4,5,5,6,6,7,7"
43
+ PACKAGE_NAMES = (
44
+ "accelerate",
45
+ "numpy",
46
+ "peft",
47
+ "scipy",
48
+ "sympy",
49
+ "torch",
50
+ "transformers",
51
+ )
52
+ RELEVANT_INHERITED_ENV = (
53
+ "CUDA_VISIBLE_DEVICES",
54
+ "HF_HOME",
55
+ "HF_HUB_OFFLINE",
56
+ "MKL_NUM_THREADS",
57
+ "OMP_NUM_THREADS",
58
+ "OPENBLAS_NUM_THREADS",
59
+ "PYTHONHASHSEED",
60
+ "TOKENIZERS_PARALLELISM",
61
+ "TRANSFORMERS_CACHE",
62
+ )
63
+
64
+
65
+ class LauncherError(ValueError):
66
+ """Raised for a launcher preflight or artifact error."""
67
+
68
+
69
+ def _utc_now() -> str:
70
+ return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
71
+
72
+
73
+ def _sha256_file(path: Path) -> str:
74
+ digest = hashlib.sha256()
75
+ with path.open("rb") as stream:
76
+ for chunk in iter(lambda: stream.read(1024 * 1024), b""):
77
+ digest.update(chunk)
78
+ return digest.hexdigest()
79
+
80
+
81
+ def _file_artifact(path: Path) -> dict[str, Any]:
82
+ return {
83
+ "path": str(path),
84
+ "bytes": path.stat().st_size,
85
+ "sha256": _sha256_file(path),
86
+ }
87
+
88
+
89
+ def _checkpoint_artifact(path: Path) -> dict[str, Any]:
90
+ """Hash a checkpoint file or a checkpoint directory deterministically."""
91
+
92
+ if path.is_file():
93
+ return {"kind": "file", **_file_artifact(path)}
94
+ if not path.is_dir():
95
+ raise LauncherError(f"checkpoint does not exist: {path}")
96
+
97
+ files: dict[str, dict[str, Any]] = {}
98
+ tree_digest = hashlib.sha256()
99
+ total_bytes = 0
100
+ for child in sorted((item for item in path.rglob("*") if item.is_file())):
101
+ relative = child.relative_to(path).as_posix()
102
+ size = child.stat().st_size
103
+ sha256 = _sha256_file(child)
104
+ files[relative] = {"bytes": size, "sha256": sha256}
105
+ total_bytes += size
106
+ tree_digest.update(relative.encode("utf-8"))
107
+ tree_digest.update(b"\0")
108
+ tree_digest.update(sha256.encode("ascii"))
109
+ tree_digest.update(b"\0")
110
+ tree_digest.update(str(size).encode("ascii"))
111
+ tree_digest.update(b"\n")
112
+ if not files:
113
+ raise LauncherError(f"checkpoint directory contains no files: {path}")
114
+ return {
115
+ "kind": "directory",
116
+ "path": str(path),
117
+ "file_count": len(files),
118
+ "bytes": total_bytes,
119
+ "sha256": tree_digest.hexdigest(),
120
+ "files": files,
121
+ }
122
+
123
+
124
+ def _question_profile(path: Path) -> dict[str, Any]:
125
+ count = 0
126
+ ids = 0
127
+ variable_counts: list[int] = []
128
+ extra_fields: set[str] = set()
129
+ with path.open("r", encoding="utf-8") as stream:
130
+ for line_number, line in enumerate(stream, start=1):
131
+ if not line.strip():
132
+ continue
133
+ try:
134
+ row = json.loads(line)
135
+ except json.JSONDecodeError as exc:
136
+ raise LauncherError(
137
+ f"invalid questions JSON on line {line_number}: {exc}"
138
+ ) from exc
139
+ if not isinstance(row, dict):
140
+ raise LauncherError(f"questions line {line_number} is not an object")
141
+ missing = {"n", "ineq", "conditions"}.difference(row)
142
+ if missing:
143
+ raise LauncherError(
144
+ f"questions line {line_number} is missing: {sorted(missing)}"
145
+ )
146
+ if not isinstance(row["conditions"], list):
147
+ raise LauncherError(
148
+ f"questions line {line_number} conditions must be a list"
149
+ )
150
+ count += 1
151
+ ids += int("id" in row)
152
+ try:
153
+ variable_counts.append(int(row["n"]))
154
+ except (TypeError, ValueError) as exc:
155
+ raise LauncherError(
156
+ f"questions line {line_number} has invalid n={row['n']!r}"
157
+ ) from exc
158
+ extra_fields.update(set(row).difference({"n", "ineq", "conditions"}))
159
+ if count == 0:
160
+ raise LauncherError(f"questions file is empty: {path}")
161
+ return {
162
+ "rows": count,
163
+ "rows_with_id": ids,
164
+ "min_n": min(variable_counts),
165
+ "max_n": max(variable_counts),
166
+ "extra_fields_preserved_by_runner_input": sorted(extra_fields),
167
+ }
168
+
169
+
170
+ def _count_jsonl_rows(path: Path) -> int:
171
+ with path.open("r", encoding="utf-8") as stream:
172
+ return sum(1 for line in stream if line.strip())
173
+
174
+
175
+ def _read_jsonl(path: Path, label: str) -> list[dict[str, Any]]:
176
+ rows: list[dict[str, Any]] = []
177
+ with path.open("r", encoding="utf-8") as stream:
178
+ for line_number, line in enumerate(stream, start=1):
179
+ if not line.strip():
180
+ continue
181
+ try:
182
+ row = json.loads(line)
183
+ except json.JSONDecodeError as exc:
184
+ raise LauncherError(
185
+ f"invalid {label} JSON on line {line_number}: {exc}"
186
+ ) from exc
187
+ if not isinstance(row, dict):
188
+ raise LauncherError(f"{label} line {line_number} is not an object")
189
+ rows.append(row)
190
+ return rows
191
+
192
+
193
+ def _resume_artifact(
194
+ results: Path, questions: Path, expected_rows: int
195
+ ) -> dict[str, Any]:
196
+ """Validate and snapshot the immutable prefix used by v12 resume."""
197
+
198
+ if not results.is_file() or results.stat().st_size == 0:
199
+ raise LauncherError(
200
+ f"--resume requires an existing nonempty results file: {results}"
201
+ )
202
+ # Appending to an unterminated final line would corrupt that JSON object.
203
+ with results.open("rb") as stream:
204
+ stream.seek(-1, os.SEEK_END)
205
+ if stream.read(1) not in {b"\n", b"\r"}:
206
+ raise LauncherError("resume results must end with a newline")
207
+
208
+ prior_rows = _read_jsonl(results, "resume results")
209
+ if not prior_rows:
210
+ raise LauncherError(
211
+ f"--resume requires at least one existing result row: {results}"
212
+ )
213
+ if len(prior_rows) >= expected_rows:
214
+ raise LauncherError(
215
+ "--resume requires fewer existing result rows than questions: "
216
+ f"results={len(prior_rows)}, questions={expected_rows}"
217
+ )
218
+
219
+ question_rows = _read_jsonl(questions, "questions")
220
+ for index, (result, question) in enumerate(
221
+ zip(prior_rows, question_rows, strict=False), start=1
222
+ ):
223
+ if result.get("input") != question:
224
+ raise LauncherError(
225
+ "resume results are not the exact question-prefix at row "
226
+ f"{index}; refusing count-based resume"
227
+ )
228
+
229
+ artifact = _file_artifact(results)
230
+ artifact["rows"] = len(prior_rows)
231
+ return artifact
232
+
233
+
234
+ def _prefix_is_unchanged(path: Path, prior: dict[str, Any]) -> bool:
235
+ expected_bytes = int(prior["bytes"])
236
+ if not path.is_file() or path.stat().st_size < expected_bytes:
237
+ return False
238
+ digest = hashlib.sha256()
239
+ with path.open("rb") as stream:
240
+ remaining = expected_bytes
241
+ while remaining:
242
+ chunk = stream.read(min(1024 * 1024, remaining))
243
+ if not chunk:
244
+ return False
245
+ digest.update(chunk)
246
+ remaining -= len(chunk)
247
+ return digest.hexdigest() == prior["sha256"]
248
+
249
+
250
+ def _resolve_existing(path: Path, label: str) -> Path:
251
+ resolved = path.expanduser().resolve()
252
+ if not resolved.is_file():
253
+ raise LauncherError(f"{label} does not exist or is not a file: {resolved}")
254
+ return resolved
255
+
256
+
257
+ def _parse_gpu_ids(raw: str, workers: int) -> list[int]:
258
+ try:
259
+ ids = [int(part.strip()) for part in raw.split(",") if part.strip()]
260
+ except ValueError as exc:
261
+ raise LauncherError(f"invalid --gpu-ids value: {raw!r}") from exc
262
+ if workers <= 0:
263
+ raise LauncherError("--workers must be positive")
264
+ if len(ids) < workers:
265
+ raise LauncherError(
266
+ f"--workers={workers} requires at least {workers} GPU IDs; got {ids}"
267
+ )
268
+ return ids[:workers]
269
+
270
+
271
+ def _display_command(command: Sequence[str]) -> str:
272
+ if os.name == "nt":
273
+ return subprocess.list2cmdline(list(command))
274
+ return shlex.join(command)
275
+
276
+
277
+ def _git_metadata(repo_root: Path) -> dict[str, Any]:
278
+ def run(*args: str) -> str | None:
279
+ try:
280
+ completed = subprocess.run(
281
+ ["git", *args],
282
+ cwd=repo_root,
283
+ check=False,
284
+ capture_output=True,
285
+ text=True,
286
+ timeout=10,
287
+ )
288
+ except (OSError, subprocess.SubprocessError):
289
+ return None
290
+ return completed.stdout.strip() if completed.returncode == 0 else None
291
+
292
+ status = run("status", "--porcelain=v1", "--untracked-files=all")
293
+ return {
294
+ "root": str(repo_root),
295
+ "commit": run("rev-parse", "HEAD"),
296
+ "branch": run("branch", "--show-current"),
297
+ "status_porcelain": status,
298
+ "dirty": bool(status) if status is not None else None,
299
+ }
300
+
301
+
302
+ def _package_metadata() -> dict[str, str | None]:
303
+ versions: dict[str, str | None] = {}
304
+ for name in PACKAGE_NAMES:
305
+ try:
306
+ versions[name] = importlib.metadata.version(name)
307
+ except importlib.metadata.PackageNotFoundError:
308
+ versions[name] = None
309
+ return versions
310
+
311
+
312
+ def _hardware_metadata(gpu_ids: Sequence[int]) -> dict[str, Any]:
313
+ metadata: dict[str, Any] = {
314
+ "platform": platform.platform(),
315
+ "machine": platform.machine(),
316
+ "processor": platform.processor(),
317
+ "cpu_count": os.cpu_count(),
318
+ "requested_gpu_ids": list(gpu_ids),
319
+ }
320
+ query = [
321
+ "nvidia-smi",
322
+ "--query-gpu=index,name,uuid,memory.total,driver_version",
323
+ "--format=csv,noheader,nounits",
324
+ ]
325
+ try:
326
+ completed = subprocess.run(
327
+ query, check=False, capture_output=True, text=True, timeout=10
328
+ )
329
+ metadata["nvidia_smi_return_code"] = completed.returncode
330
+ metadata["nvidia_smi"] = [
331
+ line.strip() for line in completed.stdout.splitlines() if line.strip()
332
+ ]
333
+ if completed.stderr.strip():
334
+ metadata["nvidia_smi_stderr"] = completed.stderr.strip()
335
+ except (OSError, subprocess.SubprocessError) as exc:
336
+ metadata["nvidia_smi_error"] = f"{type(exc).__name__}: {exc}"
337
+ return metadata
338
+
339
+
340
+ def _launch_gpu_snapshot(gpu_ids: Sequence[int]) -> list[dict[str, Any]]:
341
+ """Capture physical-GPU state once, immediately before child launch."""
342
+
343
+ requested = list(dict.fromkeys(int(value) for value in gpu_ids))
344
+ inventory_command = [
345
+ "nvidia-smi",
346
+ "--query-gpu=index,uuid,memory.used,memory.free,utilization.gpu",
347
+ "--format=csv,noheader,nounits",
348
+ ]
349
+ apps_command = [
350
+ "nvidia-smi",
351
+ "--query-compute-apps=gpu_uuid,pid,process_name,used_gpu_memory",
352
+ "--format=csv,noheader,nounits",
353
+ ]
354
+ inventory: dict[int, dict[str, Any]] = {}
355
+ apps_by_uuid: dict[str, list[dict[str, Any]]] = {}
356
+ error: str | None = None
357
+ try:
358
+ inventory_result = subprocess.run(
359
+ inventory_command,
360
+ check=True,
361
+ capture_output=True,
362
+ text=True,
363
+ timeout=10,
364
+ )
365
+ for row in csv.reader(inventory_result.stdout.splitlines(), skipinitialspace=True):
366
+ if not row:
367
+ continue
368
+ if len(row) != 5:
369
+ raise ValueError(f"malformed GPU inventory row: {row!r}")
370
+ index = int(row[0].strip())
371
+ inventory[index] = {
372
+ "index": index,
373
+ "uuid": row[1].strip(),
374
+ "memory_used_mib": int(row[2].strip()),
375
+ "memory_free_mib": int(row[3].strip()),
376
+ "utilization_percent": int(row[4].strip()),
377
+ }
378
+ apps_result = subprocess.run(
379
+ apps_command,
380
+ check=True,
381
+ capture_output=True,
382
+ text=True,
383
+ timeout=10,
384
+ )
385
+ for row in csv.reader(apps_result.stdout.splitlines(), skipinitialspace=True):
386
+ if not row:
387
+ continue
388
+ if len(row) != 4:
389
+ raise ValueError(f"malformed compute-app row: {row!r}")
390
+ uuid_value = row[0].strip()
391
+ apps_by_uuid.setdefault(uuid_value, []).append(
392
+ {
393
+ "pid": row[1].strip(),
394
+ "process_name": row[2].strip(),
395
+ "used_gpu_memory_mib": row[3].strip(),
396
+ }
397
+ )
398
+ except (OSError, ValueError, subprocess.SubprocessError) as exc:
399
+ error = f"{type(exc).__name__}: {exc}"
400
+
401
+ snapshot: list[dict[str, Any]] = []
402
+ for index in requested:
403
+ gpu = inventory.get(
404
+ index,
405
+ {
406
+ "index": index,
407
+ "uuid": None,
408
+ "memory_used_mib": None,
409
+ "memory_free_mib": None,
410
+ "utilization_percent": None,
411
+ },
412
+ )
413
+ item = {**gpu, "compute_apps": apps_by_uuid.get(str(gpu["uuid"]), [])}
414
+ if error is not None:
415
+ item["snapshot_error"] = error
416
+ snapshot.append(item)
417
+ return snapshot
418
+
419
+
420
+ def _require_complete_launch_gpu_snapshot(
421
+ snapshot: Sequence[dict[str, Any]], gpu_ids: Sequence[int]
422
+ ) -> None:
423
+ """Fail before child start unless every selected physical GPU is identified."""
424
+
425
+ by_index = {
426
+ item.get("index"): item
427
+ for item in snapshot
428
+ if isinstance(item, dict) and isinstance(item.get("index"), int)
429
+ }
430
+ for gpu_id in dict.fromkeys(int(value) for value in gpu_ids):
431
+ item = by_index.get(gpu_id)
432
+ if item is None:
433
+ raise LauncherError(f"launch GPU snapshot is missing physical GPU {gpu_id}")
434
+ if item.get("snapshot_error"):
435
+ raise LauncherError(
436
+ f"launch GPU snapshot failed for physical GPU {gpu_id}: "
437
+ f"{item['snapshot_error']}"
438
+ )
439
+ if not isinstance(item.get("uuid"), str) or not item["uuid"]:
440
+ raise LauncherError(
441
+ f"launch GPU snapshot has no UUID for physical GPU {gpu_id}"
442
+ )
443
+
444
+
445
+ def _base_model(checkpoint: Path) -> str | None:
446
+ config_path = checkpoint / "adapter_config.json" if checkpoint.is_dir() else None
447
+ if config_path is None or not config_path.is_file():
448
+ return None
449
+ try:
450
+ config = json.loads(config_path.read_text(encoding="utf-8"))
451
+ except (OSError, json.JSONDecodeError):
452
+ return None
453
+ value = config.get("base_model_name_or_path") if isinstance(config, dict) else None
454
+ return str(value) if value is not None else None
455
+
456
+
457
+ def _write_manifest(path: Path, manifest: dict[str, Any]) -> None:
458
+ path.parent.mkdir(parents=True, exist_ok=True)
459
+ temporary = path.with_name(path.name + f".tmp-{os.getpid()}-{uuid.uuid4().hex}")
460
+ temporary.write_text(
461
+ json.dumps(manifest, ensure_ascii=False, indent=2, sort_keys=True) + "\n",
462
+ encoding="utf-8",
463
+ )
464
+ temporary.replace(path)
465
+
466
+
467
+ def _write_manifest_exclusive(path: Path, manifest: dict[str, Any]) -> None:
468
+ path.parent.mkdir(parents=True, exist_ok=True)
469
+ try:
470
+ with path.open("x", encoding="utf-8") as stream:
471
+ json.dump(manifest, stream, ensure_ascii=False, indent=2, sort_keys=True)
472
+ stream.write("\n")
473
+ except FileExistsError as exc:
474
+ raise LauncherError(f"manifest already exists: {path}") from exc
475
+
476
+
477
+ def build_command(
478
+ args: argparse.Namespace,
479
+ *,
480
+ questions: Path,
481
+ results: Path,
482
+ checkpoint: Path,
483
+ runner: Path,
484
+ gpu_ids: Sequence[int],
485
+ execution_config_manifest: Path | None = None,
486
+ ) -> tuple[list[str], dict[str, str]]:
487
+ """Build the exact child-process command and search-sensitive environment."""
488
+
489
+ command = [
490
+ sys.executable,
491
+ str(runner),
492
+ "--input",
493
+ str(questions),
494
+ "--output",
495
+ str(results),
496
+ "--min-n",
497
+ str(args.min_n),
498
+ "--max-n",
499
+ str(args.max_n),
500
+ "--max-depth",
501
+ str(args.max_depth),
502
+ "--rollouts",
503
+ str(args.rollouts),
504
+ "--k-per-node",
505
+ "1",
506
+ "--llm-mode",
507
+ "batch",
508
+ "--beam-size",
509
+ str(args.beam_size),
510
+ "--gpu-workers",
511
+ str(args.workers),
512
+ "--gpu-devices",
513
+ ",".join(str(value) for value in gpu_ids),
514
+ "--adapter-path",
515
+ str(checkpoint),
516
+ "--lp-workers",
517
+ str(args.lp_workers),
518
+ "--enable-lp",
519
+ "--max-vars-for-lp",
520
+ str(args.max_vars_for_lp),
521
+ "--max-new-tokens",
522
+ str(args.max_new_tokens),
523
+ "--max-time",
524
+ str(args.generation_max_time),
525
+ "--seed",
526
+ str(args.seed),
527
+ "--sample-max-tree-size",
528
+ str(args.sample_max_tree_size),
529
+ "--sample-max-time-sec",
530
+ str(args.sample_max_time_sec),
531
+ ]
532
+ if args.print_llm:
533
+ command.append("--print-llm")
534
+ if args.sample_max_llm_calls is not None:
535
+ command.extend(["--sample-max-llm-calls", str(args.sample_max_llm_calls)])
536
+ if execution_config_manifest is not None:
537
+ command.extend(
538
+ ["--execution-config-manifest", str(execution_config_manifest)]
539
+ )
540
+ if args.resume:
541
+ command.extend(["--append", "--resume-from-output"])
542
+
543
+ environment = {
544
+ "BEAM_PROMPT_FORMAT": args.prompt_format,
545
+ "BEAM_NORMALIZE_CANDIDATES": "1" if args.normalize_candidates else "0",
546
+ "BEAM_SORT_ALPHA_GROUPS": "0",
547
+ "BEAM_RANDOM_SCORE": "0",
548
+ "BEAM_MAX_TRIES_PER_STEP": "3",
549
+ "QWEN_TEMPERATURE": str(args.temperature),
550
+ "QWEN_TOP_P": "0.9",
551
+ "QWEN_MAX_NEW_TOKENS": str(args.max_new_tokens),
552
+ "QWEN_MAX_TIME": str(args.generation_max_time),
553
+ "QWEN_RAW_PROMPT": "1" if args.raw_prompt else "0",
554
+ "QWEN_PRINT_PROMPT": "0",
555
+ "QWEN_ADAPTER_PATH": str(checkpoint),
556
+ # Prefer the explicit pinned local snapshot and otherwise prevent
557
+ # unrelated shell state from replacing the adapter-declared base.
558
+ "QWEN_BASE_MODEL": str(args.base_model) if args.base_model else "",
559
+ "QWEN_WEIGHTS_PATH": "",
560
+ "GPU_DEVICES": ",".join(str(value) for value in gpu_ids),
561
+ }
562
+ return command, environment
563
+
564
+
565
+ def _settings(args: argparse.Namespace, gpu_ids: Sequence[int]) -> dict[str, Any]:
566
+ return {
567
+ "beam_size": args.beam_size,
568
+ "rollouts": args.rollouts,
569
+ "max_depth": args.max_depth,
570
+ "seed": args.seed,
571
+ "workers": args.workers,
572
+ "gpu_ids": list(gpu_ids),
573
+ "temperature": args.temperature,
574
+ "base_model": str(args.base_model) if args.base_model else None,
575
+ "max_new_tokens": args.max_new_tokens,
576
+ "generation_max_time_sec": args.generation_max_time,
577
+ "sample_max_tree_size": args.sample_max_tree_size,
578
+ "sample_max_time_sec": args.sample_max_time_sec,
579
+ "hard_question_timeout_sec": args.hard_question_timeout_sec,
580
+ "sample_max_llm_calls": int(args.sample_max_llm_calls or 0),
581
+ "max_vars_for_lp": args.max_vars_for_lp,
582
+ "lp_enabled": True,
583
+ "lp_workers": args.lp_workers,
584
+ "lp_execution": "synchronous",
585
+ "prompt_format": args.prompt_format,
586
+ "normalize_candidates": args.normalize_candidates,
587
+ "raw_prompt": args.raw_prompt,
588
+ "print_llm": args.print_llm,
589
+ "top_p": 0.9,
590
+ "sort_alpha_groups": False,
591
+ "random_score": False,
592
+ "max_tries_per_step": 3,
593
+ "min_n": args.min_n,
594
+ "max_n": args.max_n,
595
+ "llm_mode": "batch",
596
+ "k_per_node": 1,
597
+ "random_prune": False,
598
+ "prune_seed": None,
599
+ "sample_max_llm_fails": 0,
600
+ "depth1_rollouts": None,
601
+ "per_n_budget": {},
602
+ "include_proof_trace": False,
603
+ "limit": 0,
604
+ "use_dummy": False,
605
+ "debug_search": False,
606
+ "log_every": 1,
607
+ "log_wrong_pruned": False,
608
+ "wrong_pruned_path": "",
609
+ }
610
+
611
+
612
+ def _normalized_command(command: Sequence[str]) -> list[str]:
613
+ normalized = list(command)
614
+ if "--execution-config-manifest" in normalized:
615
+ index = normalized.index("--execution-config-manifest")
616
+ if index + 1 < len(normalized):
617
+ normalized[index + 1] = "<live-wrapper-manifest>"
618
+ return normalized
619
+
620
+
621
+ def _execution_config(
622
+ *,
623
+ invocation_id: str,
624
+ recorded_utc: str,
625
+ origin: str,
626
+ repo_root: Path,
627
+ repository: dict[str, Any],
628
+ runner: Path,
629
+ runner_artifact: dict[str, Any],
630
+ checkpoint: Path,
631
+ checkpoint_artifact: dict[str, Any],
632
+ settings: dict[str, Any],
633
+ gpu_ids: Sequence[int],
634
+ inherited_environment: dict[str, str | None],
635
+ gpu_snapshot: list[dict[str, Any]],
636
+ command: Sequence[str],
637
+ ) -> dict[str, Any]:
638
+ uuid_by_index = {
639
+ item.get("index"): item.get("uuid")
640
+ for item in gpu_snapshot
641
+ if isinstance(item, dict)
642
+ }
643
+ return {
644
+ "schema_version": EXECUTION_CONFIG_SCHEMA_VERSION,
645
+ "invocation_id": invocation_id,
646
+ "recorded_utc": recorded_utc,
647
+ "origin": origin,
648
+ "code": {
649
+ "git_commit": repository.get("commit"),
650
+ "repo_root": str(repo_root),
651
+ "runner_path": str(runner),
652
+ "runner_sha256": runner_artifact["sha256"],
653
+ },
654
+ "model": {
655
+ "checkpoint_path": str(checkpoint),
656
+ "checkpoint_sha256": checkpoint_artifact["sha256"],
657
+ "base_model": settings.get("base_model") or _base_model(checkpoint),
658
+ },
659
+ "search": settings,
660
+ "resources": {
661
+ "workers": len(gpu_ids),
662
+ "gpu_ids": list(gpu_ids),
663
+ "worker_device_mapping": [
664
+ {
665
+ "worker_index": worker_index,
666
+ "physical_gpu_index": gpu_id,
667
+ "gpu_uuid": uuid_by_index.get(gpu_id),
668
+ }
669
+ for worker_index, gpu_id in enumerate(gpu_ids)
670
+ ],
671
+ "cuda_visible_devices": inherited_environment.get(
672
+ "CUDA_VISIBLE_DEVICES"
673
+ ),
674
+ "launch_gpu_snapshot": gpu_snapshot,
675
+ },
676
+ "normalized_command": _normalized_command(command),
677
+ }
678
+
679
+
680
+ def _resource_policy(args: argparse.Namespace) -> dict[str, Any]:
681
+ """Describe the frozen reviewer budget and its enforcement boundary."""
682
+
683
+ return {
684
+ "policy_version": RESOURCE_POLICY_VERSION,
685
+ "sample_max_time_sec": args.sample_max_time_sec,
686
+ "hard_question_timeout_sec": args.hard_question_timeout_sec,
687
+ "sample_max_tree_size": args.sample_max_tree_size,
688
+ "sample_max_llm_calls": args.sample_max_llm_calls,
689
+ "time_enforcement": "cooperative_soft",
690
+ "tree_enforcement": "cooperative_soft",
691
+ "hard_wall_time_enforced": False,
692
+ "external_hard_timeout_required": True,
693
+ "interpretation": (
694
+ "The tracked runner checks elapsed time and tree size between search "
695
+ "operations. Already submitted model generations can finish after a "
696
+ "budget is reached, so neither limit is an external hard kill."
697
+ ),
698
+ }
699
+
700
+
701
+ def _parser() -> argparse.ArgumentParser:
702
+ parser = argparse.ArgumentParser(
703
+ description="Run and document the submitted proposed-model v12 search."
704
+ )
705
+ parser.add_argument("--questions", type=Path, required=True)
706
+ parser.add_argument("--results", type=Path, required=True)
707
+ parser.add_argument("--checkpoint", type=Path, required=True)
708
+ parser.add_argument(
709
+ "--base-model",
710
+ type=Path,
711
+ help=(
712
+ "Local base-model snapshot. When supplied, QWEN_BASE_MODEL is set "
713
+ "to this resolved directory instead of the adapter-declared Hub ID."
714
+ ),
715
+ )
716
+ parser.add_argument("--runner", type=Path, default=DEFAULT_RUNNER)
717
+ parser.add_argument(
718
+ "--manifest",
719
+ type=Path,
720
+ help="Run manifest path (default: RESULTS.run-manifest.json)",
721
+ )
722
+ parser.add_argument("--gpu-ids", default=DEFAULT_GPU_IDS)
723
+ parser.add_argument("--workers", type=int, default=8)
724
+ parser.add_argument("--beam-size", type=int, default=32)
725
+ parser.add_argument("--rollouts", type=int, default=6)
726
+ parser.add_argument("--max-depth", type=int, default=24)
727
+ parser.add_argument("--seed", type=int, default=42)
728
+ parser.add_argument("--temperature", type=float, default=1.2)
729
+ parser.add_argument("--max-new-tokens", type=int, default=2048)
730
+ parser.add_argument("--generation-max-time", type=float, default=20.0)
731
+ parser.add_argument(
732
+ "--sample-max-tree-size",
733
+ type=int,
734
+ default=SAMPLE_MAX_TREE_SIZE,
735
+ help=(
736
+ "Cooperative per-sample tree budget. The value is recorded in every "
737
+ "result row and may change on a later resume."
738
+ ),
739
+ )
740
+ parser.add_argument("--max-vars-for-lp", type=int, default=0)
741
+ parser.add_argument("--lp-workers", type=int, default=8)
742
+ parser.add_argument("--min-n", type=int, default=3)
743
+ parser.add_argument("--max-n", type=int, default=15)
744
+ parser.add_argument("--prompt-format", default="canonical")
745
+ parser.add_argument(
746
+ "--normalize-candidates",
747
+ action=argparse.BooleanOptionalAction,
748
+ default=True,
749
+ )
750
+ parser.add_argument("--raw-prompt", action=argparse.BooleanOptionalAction, default=True)
751
+ parser.add_argument("--print-llm", action=argparse.BooleanOptionalAction, default=True)
752
+ parser.add_argument(
753
+ "--sample-max-time-sec",
754
+ type=float,
755
+ default=SAMPLE_MAX_TIME_SEC,
756
+ help=(
757
+ "Cooperative per-sample time budget. The value is recorded in every "
758
+ "result row and may change on a later resume; it is not an external "
759
+ "hard kill."
760
+ ),
761
+ )
762
+ parser.add_argument(
763
+ "--hard-question-timeout-sec",
764
+ type=float,
765
+ default=None,
766
+ help=(
767
+ "External supervisor hard timeout recorded in per-row configuration. "
768
+ "This wrapper does not itself enforce the hard kill."
769
+ ),
770
+ )
771
+ parser.add_argument(
772
+ "--sample-max-llm-calls",
773
+ type=int,
774
+ default=None,
775
+ help="Optional per-sample LLM-call cap; omitted by default.",
776
+ )
777
+ parser.add_argument("--dry-run", action="store_true")
778
+ parser.add_argument(
779
+ "--resume",
780
+ action="store_true",
781
+ help=(
782
+ "Safely append after an existing nonempty result prefix. The wrapper "
783
+ "validates the prefix and passes v12 --append --resume-from-output."
784
+ ),
785
+ )
786
+ parser.add_argument(
787
+ "--resume-source-manifest",
788
+ type=Path,
789
+ help=argparse.SUPPRESS,
790
+ )
791
+ parser.add_argument(
792
+ "--allow-resume-resource-change",
793
+ action="store_true",
794
+ help=argparse.SUPPRESS,
795
+ )
796
+ parser.add_argument(
797
+ "--overwrite",
798
+ action="store_true",
799
+ help="Allow replacing an existing results file or manifest.",
800
+ )
801
+ return parser
802
+
803
+
804
+ def _validate_numeric_args(args: argparse.Namespace) -> None:
805
+ positive = {
806
+ "beam-size": args.beam_size,
807
+ "rollouts": args.rollouts,
808
+ "max-depth": args.max_depth,
809
+ "max-new-tokens": args.max_new_tokens,
810
+ "generation-max-time": args.generation_max_time,
811
+ "sample-max-tree-size": args.sample_max_tree_size,
812
+ }
813
+ for label, value in positive.items():
814
+ if value <= 0:
815
+ raise LauncherError(f"--{label} must be positive")
816
+ if args.sample_max_time_sec is None or args.sample_max_time_sec <= 0:
817
+ raise LauncherError("--sample-max-time-sec must be positive and bounded")
818
+ if (
819
+ args.hard_question_timeout_sec is not None
820
+ and args.hard_question_timeout_sec <= 0
821
+ ):
822
+ raise LauncherError("--hard-question-timeout-sec must be positive when set")
823
+ if args.sample_max_llm_calls is not None and args.sample_max_llm_calls <= 0:
824
+ raise LauncherError("--sample-max-llm-calls must be positive when supplied")
825
+
826
+
827
+ def _exact_verify(results: Path) -> dict[str, Any]:
828
+ from beam_search_model2.verify_proof_certificate import verify_jsonl
829
+
830
+ valid, invalid, errors, skipped = verify_jsonl(results)
831
+ return {
832
+ "valid": valid,
833
+ "invalid": invalid,
834
+ "errors": errors,
835
+ "skipped_unproved": skipped,
836
+ "passed": invalid == 0 and errors == 0,
837
+ "verifier": "beam_search_model2.verify_proof_certificate.verify_jsonl",
838
+ }
839
+
840
+
841
+ def main(argv: Sequence[str] | None = None) -> int:
842
+ args = _parser().parse_args(argv)
843
+ try:
844
+ _validate_numeric_args(args)
845
+ if args.resume and args.overwrite:
846
+ raise LauncherError("--resume and --overwrite are mutually exclusive")
847
+ repo_root = Path(__file__).resolve().parents[2]
848
+ questions = _resolve_existing(args.questions, "questions file")
849
+ runner_candidate = (
850
+ repo_root / args.runner if args.runner == DEFAULT_RUNNER else args.runner
851
+ )
852
+ runner = _resolve_existing(runner_candidate, "runner")
853
+ checkpoint = args.checkpoint.expanduser().resolve()
854
+ if args.base_model is not None:
855
+ args.base_model = args.base_model.expanduser().resolve()
856
+ if not args.base_model.is_dir():
857
+ raise LauncherError(
858
+ f"base-model snapshot does not exist or is not a directory: "
859
+ f"{args.base_model}"
860
+ )
861
+ results = args.results.expanduser().resolve()
862
+ gpu_ids = _parse_gpu_ids(args.gpu_ids, args.workers)
863
+ profile = _question_profile(questions)
864
+ resume_prior: dict[str, Any] | None = None
865
+ prior_verification: dict[str, Any] | None = None
866
+ if args.resume:
867
+ resume_prior = _resume_artifact(results, questions, profile["rows"])
868
+ try:
869
+ prior_verification = _exact_verify(results)
870
+ except Exception as exc:
871
+ raise LauncherError(
872
+ "could not exactly verify resume prefix: "
873
+ f"{type(exc).__name__}: {exc}"
874
+ ) from exc
875
+ if not prior_verification["passed"]:
876
+ raise LauncherError(
877
+ "resume prefix failed exact verification; refusing to append"
878
+ )
879
+
880
+ invocation_id = uuid.uuid4().hex
881
+ recorded_utc = _utc_now()
882
+ manifest_path = (
883
+ args.manifest.expanduser().resolve()
884
+ if args.manifest
885
+ else results.with_name(
886
+ f"{results.name}.{invocation_id}.run-manifest.json"
887
+ )
888
+ )
889
+
890
+ if results in {questions, runner, manifest_path}:
891
+ raise LauncherError("results path must be distinct from inputs and manifest")
892
+ if manifest_path in {questions, runner}:
893
+ raise LauncherError("manifest path must be distinct from input files")
894
+ if results.is_dir():
895
+ raise LauncherError(f"results path is a directory: {results}")
896
+ if manifest_path.is_dir():
897
+ raise LauncherError(f"manifest path is a directory: {manifest_path}")
898
+ if checkpoint.is_dir() and (
899
+ results.is_relative_to(checkpoint) or manifest_path.is_relative_to(checkpoint)
900
+ ):
901
+ raise LauncherError("results and manifest must not be written inside checkpoint")
902
+
903
+ if manifest_path.exists():
904
+ raise LauncherError(f"manifest already exists: {manifest_path}")
905
+ if (
906
+ not args.dry_run
907
+ and results.exists()
908
+ and not args.overwrite
909
+ and not args.resume
910
+ ):
911
+ raise LauncherError(
912
+ f"results already exist; pass --overwrite to replace them: {results}"
913
+ )
914
+
915
+ if profile["min_n"] < args.min_n or profile["max_n"] > args.max_n:
916
+ raise LauncherError(
917
+ "questions contain n outside the configured runner filter: "
918
+ f"questions={profile['min_n']}..{profile['max_n']}, "
919
+ f"filter={args.min_n}..{args.max_n}"
920
+ )
921
+ checkpoint_info = _checkpoint_artifact(checkpoint)
922
+ runner_info = _file_artifact(runner)
923
+ repository = _git_metadata(repo_root)
924
+ inherited_environment = {
925
+ name: os.environ.get(name) for name in RELEVANT_INHERITED_ENV
926
+ }
927
+ settings = _settings(args, gpu_ids)
928
+ runtime_environment: dict[str, Any] | None = None
929
+ if not args.dry_run:
930
+ runtime_environment = {
931
+ "python": {
932
+ "executable": sys.executable,
933
+ "version": sys.version,
934
+ "implementation": platform.python_implementation(),
935
+ },
936
+ "packages": _package_metadata(),
937
+ "hardware": _hardware_metadata(gpu_ids),
938
+ }
939
+ # Keep this as the final hardware query before manifest publication and
940
+ # child start; rows describe launch-time state, not an earlier preflight.
941
+ gpu_snapshot = _launch_gpu_snapshot(gpu_ids)
942
+ if not args.dry_run:
943
+ _require_complete_launch_gpu_snapshot(gpu_snapshot, gpu_ids)
944
+ command, environment = build_command(
945
+ args,
946
+ questions=questions,
947
+ results=results,
948
+ checkpoint=checkpoint,
949
+ runner=runner,
950
+ gpu_ids=gpu_ids,
951
+ execution_config_manifest=manifest_path,
952
+ )
953
+ execution_config = _execution_config(
954
+ invocation_id=invocation_id,
955
+ recorded_utc=recorded_utc,
956
+ origin="resume" if args.resume else "fresh",
957
+ repo_root=repo_root,
958
+ repository=repository,
959
+ runner=runner,
960
+ runner_artifact=runner_info,
961
+ checkpoint=checkpoint,
962
+ checkpoint_artifact=checkpoint_info,
963
+ settings=settings,
964
+ gpu_ids=gpu_ids,
965
+ inherited_environment=inherited_environment,
966
+ gpu_snapshot=gpu_snapshot,
967
+ command=command,
968
+ )
969
+ plan = {
970
+ "command": command,
971
+ "command_display": _display_command(command),
972
+ "cwd": str(repo_root),
973
+ "environment_overrides": environment,
974
+ "inherited_environment_relevant": inherited_environment,
975
+ "settings": settings,
976
+ }
977
+ manifest: dict[str, Any] = {
978
+ "schema_version": SCHEMA_VERSION,
979
+ "execution_config": execution_config,
980
+ "mode": "dry-run" if args.dry_run else "execute",
981
+ "status": "planned" if args.dry_run else "preparing",
982
+ "plan": plan,
983
+ "artifacts": {
984
+ "questions": _file_artifact(questions),
985
+ "runner": runner_info,
986
+ "checkpoint": checkpoint_info,
987
+ "results_path": str(results),
988
+ "manifest_path": str(manifest_path),
989
+ },
990
+ "questions": profile,
991
+ "resource_policy": _resource_policy(args),
992
+ }
993
+ if resume_prior is not None:
994
+ manifest["resume"] = {
995
+ "enabled": True,
996
+ "prior_results": resume_prior,
997
+ "prior_exact_verification": prior_verification,
998
+ "remaining_rows": profile["rows"] - int(resume_prior["rows"]),
999
+ "runner_flags": ["--append", "--resume-from-output"],
1000
+ "manifest_policy": (
1001
+ "Resume trusts no earlier configuration. The exact input prefix "
1002
+ "and proof certificates are validated, and this invocation is "
1003
+ "self-described by its own unique manifest."
1004
+ ),
1005
+ }
1006
+
1007
+ if args.dry_run:
1008
+ _write_manifest_exclusive(manifest_path, manifest)
1009
+ print(json.dumps(manifest, ensure_ascii=False, indent=2, sort_keys=True))
1010
+ return 0
1011
+
1012
+ results.parent.mkdir(parents=True, exist_ok=True)
1013
+ manifest["repository"] = repository
1014
+ manifest["runtime_environment"] = runtime_environment
1015
+ manifest["status"] = "running"
1016
+ manifest["runtime"] = {"started_utc": recorded_utc}
1017
+ # The supervisor reads this live manifest to stamp synthetic timeout
1018
+ # rows, so execution_config must be authoritative before child start.
1019
+ _write_manifest_exclusive(manifest_path, manifest)
1020
+
1021
+ child_environment = os.environ.copy()
1022
+ child_environment.update(environment)
1023
+ started = time.perf_counter()
1024
+ try:
1025
+ completed = subprocess.run(
1026
+ command,
1027
+ cwd=repo_root,
1028
+ env=child_environment,
1029
+ check=False,
1030
+ )
1031
+ return_code = completed.returncode
1032
+ except KeyboardInterrupt:
1033
+ return_code = 130
1034
+ manifest["status"] = "interrupted"
1035
+ except OSError as exc:
1036
+ return_code = 127
1037
+ manifest["status"] = "runner_failed"
1038
+ manifest["runtime"]["launch_error"] = f"{type(exc).__name__}: {exc}"
1039
+ wall_time = time.perf_counter() - started
1040
+ manifest["runtime"].update(
1041
+ {
1042
+ "ended_utc": _utc_now(),
1043
+ "wall_time_sec": wall_time,
1044
+ "return_code": return_code,
1045
+ }
1046
+ )
1047
+
1048
+ prefix_unchanged = (
1049
+ _prefix_is_unchanged(results, resume_prior)
1050
+ if resume_prior is not None
1051
+ else None
1052
+ )
1053
+ if resume_prior is not None:
1054
+ manifest["resume"]["prior_prefix_unchanged"] = prefix_unchanged
1055
+
1056
+ if return_code == 0:
1057
+ if not results.is_file():
1058
+ manifest["status"] = "verification_failed"
1059
+ manifest["exact_verification"] = {
1060
+ "passed": False,
1061
+ "error": "runner returned zero but did not create the results file",
1062
+ }
1063
+ final_code = 4
1064
+ else:
1065
+ manifest["artifacts"]["results"] = _file_artifact(results)
1066
+ result_rows = _count_jsonl_rows(results)
1067
+ manifest["artifacts"]["results"]["rows"] = result_rows
1068
+ try:
1069
+ verification = _exact_verify(results)
1070
+ except Exception as exc: # Preserve the run even if verifier setup fails.
1071
+ verification = {
1072
+ "passed": False,
1073
+ "error": f"{type(exc).__name__}: {exc}",
1074
+ }
1075
+ manifest["exact_verification"] = verification
1076
+ manifest["result_completeness"] = {
1077
+ "expected_rows": profile["rows"],
1078
+ "actual_rows": result_rows,
1079
+ "passed": result_rows == profile["rows"],
1080
+ }
1081
+ if (
1082
+ verification["passed"]
1083
+ and result_rows == profile["rows"]
1084
+ and prefix_unchanged is not False
1085
+ ):
1086
+ manifest["status"] = "succeeded"
1087
+ final_code = 0
1088
+ else:
1089
+ manifest["status"] = "verification_failed"
1090
+ final_code = 4
1091
+ else:
1092
+ manifest["status"] = (
1093
+ manifest["status"] if manifest["status"] == "interrupted" else "runner_failed"
1094
+ )
1095
+ if results.is_file():
1096
+ manifest["artifacts"]["partial_results"] = _file_artifact(results)
1097
+ final_code = return_code
1098
+
1099
+ _write_manifest(manifest_path, manifest)
1100
+ print(json.dumps(manifest, ensure_ascii=False, indent=2, sort_keys=True))
1101
+ return final_code
1102
+ except LauncherError as exc:
1103
+ print(f"error: {exc}", file=sys.stderr)
1104
+ return 2
1105
+
1106
+
1107
+ if __name__ == "__main__":
1108
+ raise SystemExit(main())
mce220_runner/experiments/reviewer_response/supervise_search_benchmark.py ADDED
@@ -0,0 +1,780 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Externally enforce a hard per-question limit for the v12 search launcher.
2
+
3
+ The submitted runner's time budget is cooperative: model calls already in flight
4
+ may finish after it expires. This supervisor leaves that runner and its
5
+ reproducibility wrapper unchanged. It watches the wrapper's streamed JSONL
6
+ output, kills the complete child process tree at a hard deadline, records a
7
+ schema-compatible timeout row for the exact pending input, and starts a new
8
+ wrapper resume segment at the following question.
9
+
10
+ Every invocation and process segment gets a collision-resistant identity and
11
+ new, never-overwritten logs and manifests. The result prefix is authoritative:
12
+ the mutable state file may be absent, stale, or left by a different invocation.
13
+ """
14
+
15
+ from __future__ import annotations
16
+
17
+ import argparse
18
+ import hashlib
19
+ import json
20
+ import os
21
+ import signal
22
+ import subprocess
23
+ import sys
24
+ import time
25
+ import uuid
26
+ from dataclasses import dataclass
27
+ from datetime import datetime, timezone
28
+ from pathlib import Path
29
+ from typing import Any, Sequence
30
+
31
+
32
+ SCHEMA_VERSION = "reviewer-hard-supervisor-v2"
33
+ ATTEMPT_SCHEMA_VERSION = "reviewer-hard-supervisor-attempt-v2"
34
+ EXECUTION_CONFIG_SCHEMA_VERSION = "e2n-execution-config-v1"
35
+ QUESTION_TIMEOUT_SEC = 1000.0
36
+ TREE_SIZE_CAP = 6000
37
+ TOTAL_JOB_STOP_SEC = 42.0 * 60.0 * 60.0
38
+ DEFAULT_LAUNCHER = Path("experiments/reviewer_response/run_search_benchmark.py")
39
+
40
+
41
+ class SupervisorError(ValueError):
42
+ """Raised when an artifact or resume prefix is unsafe."""
43
+
44
+
45
+ @dataclass(frozen=True)
46
+ class Question:
47
+ row: dict[str, Any]
48
+ source_line_number: int
49
+ input_sha256: str
50
+
51
+
52
+ def _utc_now() -> str:
53
+ return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
54
+
55
+
56
+ def _sha256_file(path: Path) -> str:
57
+ digest = hashlib.sha256()
58
+ with path.open("rb") as stream:
59
+ for chunk in iter(lambda: stream.read(1024 * 1024), b""):
60
+ digest.update(chunk)
61
+ return digest.hexdigest()
62
+
63
+
64
+ def _artifact(path: Path) -> dict[str, Any]:
65
+ return {
66
+ "path": str(path),
67
+ "bytes": path.stat().st_size,
68
+ "sha256": _sha256_file(path),
69
+ }
70
+
71
+
72
+ def _write_json_atomic(path: Path, value: dict[str, Any]) -> None:
73
+ path.parent.mkdir(parents=True, exist_ok=True)
74
+ temporary = path.with_name(path.name + f".tmp-{os.getpid()}")
75
+ temporary.write_text(
76
+ json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n",
77
+ encoding="utf-8",
78
+ )
79
+ temporary.replace(path)
80
+
81
+
82
+ def _write_json_exclusive(path: Path, value: dict[str, Any]) -> None:
83
+ path.parent.mkdir(parents=True, exist_ok=True)
84
+ with path.open("x", encoding="utf-8") as stream:
85
+ json.dump(value, stream, ensure_ascii=False, indent=2, sort_keys=True)
86
+ stream.write("\n")
87
+
88
+
89
+ def _read_questions(path: Path) -> list[Question]:
90
+ questions: list[Question] = []
91
+ ids: set[str] = set()
92
+ with path.open("rb") as stream:
93
+ for source_line_number, raw in enumerate(stream, start=1):
94
+ stripped = raw.strip()
95
+ if not stripped:
96
+ continue
97
+ try:
98
+ row = json.loads(stripped.decode("utf-8"))
99
+ except (UnicodeDecodeError, json.JSONDecodeError) as exc:
100
+ raise SupervisorError(
101
+ f"invalid question JSON on source line {source_line_number}: {exc}"
102
+ ) from exc
103
+ if not isinstance(row, dict):
104
+ raise SupervisorError(
105
+ f"question source line {source_line_number} is not an object"
106
+ )
107
+ missing = {"id", "n", "ineq", "conditions"}.difference(row)
108
+ if missing:
109
+ raise SupervisorError(
110
+ f"question source line {source_line_number} is missing {sorted(missing)}"
111
+ )
112
+ question_id = row["id"]
113
+ if not isinstance(question_id, str) or not question_id:
114
+ raise SupervisorError(
115
+ f"question source line {source_line_number} has an invalid id"
116
+ )
117
+ if question_id in ids:
118
+ raise SupervisorError(f"duplicate question id: {question_id}")
119
+ ids.add(question_id)
120
+ if not isinstance(row["conditions"], list):
121
+ raise SupervisorError(f"question {question_id} conditions is not a list")
122
+ questions.append(
123
+ Question(
124
+ row=row,
125
+ source_line_number=source_line_number,
126
+ input_sha256=hashlib.sha256(stripped).hexdigest(),
127
+ )
128
+ )
129
+ if not questions:
130
+ raise SupervisorError(f"questions file is empty: {path}")
131
+ return questions
132
+
133
+
134
+ def _read_complete_result_rows(path: Path) -> tuple[list[dict[str, Any]], int]:
135
+ """Read newline-terminated rows and return their safe byte boundary.
136
+
137
+ A final unterminated line is ignored while a child may be writing it. Any
138
+ malformed *terminated* row is an artifact error, never a reason to skip an
139
+ input by count.
140
+ """
141
+
142
+ if not path.exists():
143
+ return [], 0
144
+ rows: list[dict[str, Any]] = []
145
+ safe_bytes = 0
146
+ with path.open("rb") as stream:
147
+ while True:
148
+ raw = stream.readline()
149
+ if not raw:
150
+ break
151
+ if not raw.endswith((b"\n", b"\r")):
152
+ break
153
+ safe_bytes = stream.tell()
154
+ if not raw.strip():
155
+ continue
156
+ try:
157
+ row = json.loads(raw.decode("utf-8"))
158
+ except (UnicodeDecodeError, json.JSONDecodeError) as exc:
159
+ raise SupervisorError(
160
+ f"malformed terminated result row {len(rows) + 1}: {exc}"
161
+ ) from exc
162
+ if not isinstance(row, dict):
163
+ raise SupervisorError(f"result row {len(rows) + 1} is not an object")
164
+ rows.append(row)
165
+ return rows, safe_bytes
166
+
167
+
168
+ def _validate_prefix(
169
+ results: Path, questions: Sequence[Question], *, require_terminated: bool
170
+ ) -> list[dict[str, Any]]:
171
+ rows, safe_bytes = _read_complete_result_rows(results)
172
+ if results.exists() and require_terminated and results.stat().st_size != safe_bytes:
173
+ raise SupervisorError("results end in an incomplete JSONL row")
174
+ if len(rows) > len(questions):
175
+ raise SupervisorError(
176
+ f"results contain more rows than questions: {len(rows)} > {len(questions)}"
177
+ )
178
+ seen_ids: set[str] = set()
179
+ for index, (result, question) in enumerate(zip(rows, questions), start=1):
180
+ if result.get("input") != question.row:
181
+ raise SupervisorError(
182
+ f"result row {index} is not the exact frozen question prefix"
183
+ )
184
+ result_id = result["input"].get("id")
185
+ if result_id != question.row["id"]:
186
+ raise SupervisorError(f"result row {index} has a mismatched question id")
187
+ if result_id in seen_ids:
188
+ raise SupervisorError(f"duplicate result id: {result_id}")
189
+ seen_ids.add(result_id)
190
+ if "input_sha256" in result and result["input_sha256"] != question.input_sha256:
191
+ raise SupervisorError(f"result row {index} has a mismatched input_sha256")
192
+ return rows
193
+
194
+
195
+ def _timeout_row(
196
+ question: Question,
197
+ *,
198
+ timeout_sec: float,
199
+ observed_wall_sec: float,
200
+ attempt_number: int,
201
+ execution_config: dict[str, Any],
202
+ tree_size_cap: int = TREE_SIZE_CAP,
203
+ ) -> dict[str, Any]:
204
+ row = question.row
205
+ return {
206
+ "source_line_number": question.source_line_number,
207
+ "input_sha256": question.input_sha256,
208
+ "input": row,
209
+ "mapped": {
210
+ "ineq": row["ineq"],
211
+ "n": int(row["n"]),
212
+ "mapping": {},
213
+ },
214
+ "run": {
215
+ "proved": False,
216
+ "time_sec": float(timeout_sec),
217
+ "search_time_sec": float(timeout_sec),
218
+ "tree_size": 0,
219
+ "expr_str": None,
220
+ "x": None,
221
+ "error": (
222
+ f"ExternalHardTimeout: question exceeded {timeout_sec:g} seconds"
223
+ ),
224
+ "llm_usage": {},
225
+ "stop_reason": "external_hard_timeout",
226
+ "budget_applied": {
227
+ "tree_size": tree_size_cap,
228
+ "time_sec": float(timeout_sec),
229
+ "llm_calls": 0,
230
+ "llm_fails": 0,
231
+ "depth1_rollouts": None,
232
+ "enforcement": "external_hard_process_tree_kill",
233
+ },
234
+ "external_supervision": {
235
+ "schema_version": SCHEMA_VERSION,
236
+ "attempt_number": attempt_number,
237
+ "hard_timeout_sec": float(timeout_sec),
238
+ "observed_wall_sec": float(observed_wall_sec),
239
+ "synthetic_result": True,
240
+ },
241
+ "execution_config": execution_config,
242
+ },
243
+ }
244
+
245
+
246
+ def _append_row(path: Path, row: dict[str, Any]) -> None:
247
+ """Append one complete JSONL row with one O_APPEND write, then fsync it."""
248
+
249
+ path.parent.mkdir(parents=True, exist_ok=True)
250
+ payload = (json.dumps(row, ensure_ascii=False) + "\n").encode("utf-8")
251
+ descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_APPEND)
252
+ try:
253
+ written = os.write(descriptor, payload)
254
+ if written != len(payload):
255
+ raise OSError(
256
+ "short atomic JSONL append: "
257
+ f"wrote {written} of {len(payload)} bytes"
258
+ )
259
+ os.fsync(descriptor)
260
+ finally:
261
+ os.close(descriptor)
262
+
263
+
264
+ def _popen_group(command: Sequence[str], **kwargs: Any) -> subprocess.Popen[Any]:
265
+ if os.name == "nt":
266
+ kwargs["creationflags"] = kwargs.get("creationflags", 0) | subprocess.CREATE_NEW_PROCESS_GROUP
267
+ else:
268
+ kwargs["start_new_session"] = True
269
+ return subprocess.Popen(list(command), **kwargs)
270
+
271
+
272
+ def _kill_process_tree(process: subprocess.Popen[Any], *, grace_sec: float = 3.0) -> None:
273
+ """Terminate the exact child tree and wait until its root is reaped."""
274
+
275
+ # Do not return merely because the group leader has exited. A launcher can
276
+ # terminate between poll() and this call while one of its workers remains
277
+ # alive. The process group/tree operation must still be attempted.
278
+ root_already_exited = process.poll() is not None
279
+ if os.name == "nt":
280
+ try:
281
+ completed = subprocess.run(
282
+ ["taskkill", "/PID", str(process.pid), "/T", "/F"],
283
+ check=False,
284
+ stdout=subprocess.DEVNULL,
285
+ stderr=subprocess.DEVNULL,
286
+ timeout=max(grace_sec, 1.0),
287
+ )
288
+ except subprocess.TimeoutExpired as exc:
289
+ raise SupervisorError(
290
+ f"timed out killing launcher process tree rooted at pid {process.pid}"
291
+ ) from exc
292
+ if completed.returncode != 0:
293
+ raise SupervisorError(
294
+ "taskkill could not confirm termination of launcher process tree "
295
+ f"rooted at pid {process.pid} (exit {completed.returncode}); "
296
+ "refusing stale-lock removal or relaunch"
297
+ )
298
+ else:
299
+ try:
300
+ os.killpg(process.pid, signal.SIGTERM)
301
+ except ProcessLookupError:
302
+ pass
303
+ if not root_already_exited:
304
+ try:
305
+ process.wait(timeout=grace_sec)
306
+ except subprocess.TimeoutExpired:
307
+ pass
308
+ # The group leader can exit while a worker ignores SIGTERM. Always
309
+ # follow with SIGKILL for the same isolated process group.
310
+ try:
311
+ os.killpg(process.pid, signal.SIGKILL)
312
+ except ProcessLookupError:
313
+ pass
314
+ try:
315
+ process.wait(timeout=max(grace_sec, 1.0))
316
+ except subprocess.TimeoutExpired as exc:
317
+ raise SupervisorError(
318
+ f"could not reap killed launcher process tree rooted at pid {process.pid}"
319
+ ) from exc
320
+
321
+
322
+ def _protected_launcher_args(extra: Sequence[str]) -> None:
323
+ protected = {
324
+ "--questions",
325
+ "--results",
326
+ "--checkpoint",
327
+ "--manifest",
328
+ "--resume",
329
+ "--resume-source-manifest",
330
+ "--overwrite",
331
+ "--sample-max-time-sec",
332
+ "--hard-question-timeout-sec",
333
+ "--sample-max-tree-size",
334
+ }
335
+ used = sorted(
336
+ item
337
+ for item in extra
338
+ if item in protected or any(item.startswith(flag + "=") for flag in protected)
339
+ )
340
+ if used:
341
+ raise SupervisorError(
342
+ "launcher arguments may not override supervisor-controlled flags: "
343
+ + ", ".join(used)
344
+ )
345
+
346
+
347
+ def _timeout_execution_config(
348
+ wrapper_manifest: Path,
349
+ *,
350
+ invocation_id: str,
351
+ attempt_id: str,
352
+ ) -> dict[str, Any]:
353
+ """Copy and validate the live wrapper config for a synthetic timeout row."""
354
+
355
+ try:
356
+ manifest = json.loads(wrapper_manifest.read_text(encoding="utf-8"))
357
+ except (OSError, json.JSONDecodeError) as exc:
358
+ raise SupervisorError(
359
+ "cannot stamp synthetic timeout without the authoritative wrapper "
360
+ f"manifest {wrapper_manifest}: {exc}"
361
+ ) from exc
362
+ authoritative = manifest.get("execution_config")
363
+ if not isinstance(authoritative, dict):
364
+ raise SupervisorError(
365
+ "authoritative wrapper manifest has no execution_config object"
366
+ )
367
+ required = {
368
+ "invocation_id",
369
+ "recorded_utc",
370
+ "origin",
371
+ "code",
372
+ "model",
373
+ "search",
374
+ "resources",
375
+ "normalized_command",
376
+ }
377
+ missing = sorted(required.difference(authoritative))
378
+ if authoritative.get("schema_version") != EXECUTION_CONFIG_SCHEMA_VERSION:
379
+ raise SupervisorError(
380
+ "authoritative wrapper execution_config has unsupported schema "
381
+ f"{authoritative.get('schema_version')!r}"
382
+ )
383
+ if missing:
384
+ raise SupervisorError(
385
+ "authoritative wrapper execution_config is missing fields: "
386
+ + ", ".join(missing)
387
+ )
388
+ # JSON round-tripping detaches this row from the mutable manifest object.
389
+ config = json.loads(json.dumps(authoritative, ensure_ascii=False))
390
+ config["origin"] = "supervisor_synthetic_timeout"
391
+ config["supervision"] = {
392
+ "invocation_id": invocation_id,
393
+ "attempt_id": attempt_id,
394
+ }
395
+ return config
396
+
397
+
398
+ def _launcher_command(
399
+ *,
400
+ launcher: Path,
401
+ questions: Path,
402
+ results: Path,
403
+ checkpoint: Path,
404
+ wrapper_manifest: Path,
405
+ launcher_args: Sequence[str],
406
+ resume: bool,
407
+ question_timeout_sec: float,
408
+ tree_size_cap: int = TREE_SIZE_CAP,
409
+ ) -> list[str]:
410
+ command = [
411
+ sys.executable,
412
+ str(launcher),
413
+ "--questions",
414
+ str(questions),
415
+ "--results",
416
+ str(results),
417
+ "--checkpoint",
418
+ str(checkpoint),
419
+ "--manifest",
420
+ str(wrapper_manifest),
421
+ "--sample-max-time-sec",
422
+ str(float(question_timeout_sec)),
423
+ "--hard-question-timeout-sec",
424
+ str(float(question_timeout_sec)),
425
+ "--sample-max-tree-size",
426
+ str(tree_size_cap),
427
+ *launcher_args,
428
+ ]
429
+ if resume:
430
+ command.append("--resume")
431
+ return command
432
+
433
+
434
+ def run_supervisor(
435
+ *,
436
+ questions_path: Path,
437
+ results_path: Path,
438
+ checkpoint: Path,
439
+ launcher: Path,
440
+ artifact_dir: Path,
441
+ state_path: Path,
442
+ launcher_args: Sequence[str] = (),
443
+ question_timeout_sec: float = QUESTION_TIMEOUT_SEC,
444
+ tree_size_cap: int = TREE_SIZE_CAP,
445
+ startup_grace_sec: float = 0.0,
446
+ total_job_stop_sec: float | None = TOTAL_JOB_STOP_SEC,
447
+ poll_interval_sec: float = 0.25,
448
+ ) -> int:
449
+ """Run supervised wrapper segments; return 0 complete, 3 job stop, 4 failure."""
450
+
451
+ if question_timeout_sec <= 0:
452
+ raise SupervisorError("question timeout must be positive")
453
+ if tree_size_cap <= 0:
454
+ raise SupervisorError("tree size cap must be positive")
455
+ if startup_grace_sec < 0:
456
+ raise SupervisorError("startup grace must be nonnegative")
457
+ if total_job_stop_sec is not None and total_job_stop_sec <= 0:
458
+ raise SupervisorError("total job stop must be positive")
459
+ if poll_interval_sec <= 0:
460
+ raise SupervisorError("poll interval must be positive")
461
+ _protected_launcher_args(launcher_args)
462
+
463
+ questions_path = questions_path.expanduser().resolve()
464
+ results_path = results_path.expanduser().resolve()
465
+ checkpoint = checkpoint.expanduser().resolve()
466
+ launcher = launcher.expanduser().resolve()
467
+ artifact_dir = artifact_dir.expanduser().resolve()
468
+ state_path = state_path.expanduser().resolve()
469
+ for path, label in ((questions_path, "questions"), (launcher, "launcher")):
470
+ if not path.is_file():
471
+ raise SupervisorError(f"{label} file does not exist: {path}")
472
+ if not checkpoint.exists():
473
+ raise SupervisorError(f"checkpoint does not exist: {checkpoint}")
474
+ if len({questions_path, results_path, state_path}) != 3:
475
+ raise SupervisorError("questions, results, and state paths must be distinct")
476
+
477
+ questions = _read_questions(questions_path)
478
+ question_artifact = _artifact(questions_path)
479
+ launcher_artifact = _artifact(launcher)
480
+ supervisor_artifact = _artifact(Path(__file__).resolve())
481
+ policy = {
482
+ "hard_question_timeout_sec": float(question_timeout_sec),
483
+ "cooperative_tree_size_cap": int(tree_size_cap),
484
+ "startup_grace_sec": float(startup_grace_sec),
485
+ "total_job_stop_sec": (
486
+ float(total_job_stop_sec) if total_job_stop_sec is not None else None
487
+ ),
488
+ "startup_accounting": (
489
+ "The first question clock begins when the wrapper manifest is first "
490
+ "observed, after launcher preflight and before model loading, so model "
491
+ "startup is counted. startup_grace_sec is added only to that first "
492
+ "question's allowance. Later clocks begin when the preceding JSONL "
493
+ "row is observed."
494
+ ),
495
+ }
496
+ results_path.parent.mkdir(parents=True, exist_ok=True)
497
+ artifact_dir.mkdir(parents=True, exist_ok=True)
498
+ prefix = _validate_prefix(results_path, questions, require_terminated=True)
499
+ invocation_id = uuid.uuid4().hex
500
+ prior_state: dict[str, Any] | None = None
501
+ if state_path.is_file():
502
+ prior_state = _artifact(state_path)
503
+ try:
504
+ previous = json.loads(state_path.read_text(encoding="utf-8"))
505
+ if isinstance(previous, dict):
506
+ prior_state["status"] = previous.get("status")
507
+ prior_state["invocation_id"] = previous.get("invocation_id")
508
+ except (OSError, json.JSONDecodeError):
509
+ prior_state["status"] = "unreadable"
510
+ state: dict[str, Any] = {
511
+ "schema_version": SCHEMA_VERSION,
512
+ "status": "preparing",
513
+ "created_utc": _utc_now(),
514
+ "invocation_id": invocation_id,
515
+ "recovered_from_result_prefix_rows": len(prefix),
516
+ "prior_state": prior_state,
517
+ "artifacts": {
518
+ "questions": question_artifact,
519
+ "launcher": launcher_artifact,
520
+ "supervisor": supervisor_artifact,
521
+ "results_path": str(results_path),
522
+ "artifact_dir": str(artifact_dir),
523
+ },
524
+ "checkpoint": str(checkpoint),
525
+ "launcher_args": list(launcher_args),
526
+ "policy": policy,
527
+ "attempts": [],
528
+ }
529
+ _write_json_atomic(state_path, state)
530
+
531
+ job_started = time.monotonic()
532
+ try:
533
+ if len(prefix) == len(questions):
534
+ state["status"] = "succeeded"
535
+ state["completed_utc"] = _utc_now()
536
+ state["result_rows"] = len(prefix)
537
+ state["artifacts"]["results"] = _artifact(results_path)
538
+ _write_json_atomic(state_path, state)
539
+ return 0
540
+
541
+ while len(prefix) < len(questions):
542
+ attempt_number = len(state["attempts"]) + 1
543
+ attempt_id = f"{invocation_id}-{attempt_number:04d}-{uuid.uuid4().hex}"
544
+ stem = f"attempt-{attempt_id}"
545
+ wrapper_manifest = artifact_dir / f"{stem}.wrapper-manifest.json"
546
+ stdout_path = artifact_dir / f"{stem}.stdout.log"
547
+ stderr_path = artifact_dir / f"{stem}.stderr.log"
548
+ attempt_manifest = artifact_dir / f"{stem}.supervisor.json"
549
+ command = _launcher_command(
550
+ launcher=launcher,
551
+ questions=questions_path,
552
+ results=results_path,
553
+ checkpoint=checkpoint,
554
+ wrapper_manifest=wrapper_manifest,
555
+ launcher_args=launcher_args,
556
+ resume=bool(prefix),
557
+ question_timeout_sec=question_timeout_sec,
558
+ tree_size_cap=tree_size_cap,
559
+ )
560
+ rows_before = len(prefix)
561
+ started_utc = _utc_now()
562
+ attempt_started = time.monotonic()
563
+ first_pending = True
564
+ # The wrapper writes its manifest after launcher preflight and
565
+ # before model loading. Start the first question clock only then:
566
+ # preflight is excluded, while model startup is included.
567
+ current_started: float | None = None
568
+ termination = "unknown"
569
+ return_code: int | None = None
570
+ with stdout_path.open("xb") as stdout_stream, stderr_path.open("xb") as stderr_stream:
571
+ process = _popen_group(
572
+ command,
573
+ cwd=Path(__file__).resolve().parents[2],
574
+ stdout=stdout_stream,
575
+ stderr=stderr_stream,
576
+ )
577
+ while True:
578
+ now = time.monotonic()
579
+ if current_started is None and wrapper_manifest.is_file():
580
+ current_started = now
581
+ observed = _validate_prefix(
582
+ results_path, questions, require_terminated=False
583
+ )
584
+ if len(observed) > len(prefix):
585
+ prefix = observed
586
+ current_started = now
587
+ first_pending = False
588
+ if len(prefix) == len(questions):
589
+ # Let the wrapper finalize its own manifest cleanly.
590
+ try:
591
+ return_code = process.wait(timeout=30.0)
592
+ except subprocess.TimeoutExpired:
593
+ _kill_process_tree(process)
594
+ return_code = process.returncode
595
+ termination = (
596
+ "complete" if return_code == 0 else "launcher_exit"
597
+ )
598
+ break
599
+
600
+ return_code = process.poll()
601
+ if return_code is not None:
602
+ termination = "launcher_exit"
603
+ break
604
+
605
+ if total_job_stop_sec is not None and now - job_started >= total_job_stop_sec:
606
+ _kill_process_tree(process)
607
+ return_code = process.returncode
608
+ termination = "total_job_stop"
609
+ break
610
+
611
+ allowance = question_timeout_sec + (
612
+ startup_grace_sec if first_pending else 0.0
613
+ )
614
+ if (
615
+ current_started is not None
616
+ and now - current_started >= allowance
617
+ ):
618
+ _kill_process_tree(process)
619
+ return_code = process.returncode
620
+ # The row could have completed in the kill race. Re-read
621
+ # before appending any synthetic result.
622
+ observed = _validate_prefix(
623
+ results_path, questions, require_terminated=True
624
+ )
625
+ if len(observed) == len(prefix):
626
+ pending = questions[len(prefix)]
627
+ timeout_execution_config = _timeout_execution_config(
628
+ wrapper_manifest,
629
+ invocation_id=invocation_id,
630
+ attempt_id=attempt_id,
631
+ )
632
+ _append_row(
633
+ results_path,
634
+ _timeout_row(
635
+ pending,
636
+ timeout_sec=question_timeout_sec,
637
+ observed_wall_sec=now - current_started,
638
+ attempt_number=attempt_number,
639
+ execution_config=timeout_execution_config,
640
+ tree_size_cap=tree_size_cap,
641
+ ),
642
+ )
643
+ prefix = _validate_prefix(
644
+ results_path, questions, require_terminated=True
645
+ )
646
+ termination = "hard_question_timeout"
647
+ else:
648
+ prefix = observed
649
+ termination = "timeout_completion_race"
650
+ break
651
+ time.sleep(poll_interval_sec)
652
+
653
+ ended_utc = _utc_now()
654
+ frozen_wrapper_manifest = (
655
+ _artifact(wrapper_manifest) if wrapper_manifest.is_file() else None
656
+ )
657
+ attempt_record = {
658
+ "schema_version": ATTEMPT_SCHEMA_VERSION,
659
+ "attempt_number": attempt_number,
660
+ "attempt_id": attempt_id,
661
+ "invocation_id": invocation_id,
662
+ "started_utc": started_utc,
663
+ "ended_utc": ended_utc,
664
+ "wall_time_sec": time.monotonic() - attempt_started,
665
+ "rows_before": rows_before,
666
+ "rows_after": len(prefix),
667
+ "termination": termination,
668
+ "return_code": return_code,
669
+ "command": command,
670
+ "wrapper_manifest": frozen_wrapper_manifest,
671
+ "wrapper_manifest_path": str(wrapper_manifest),
672
+ "stdout_log": _artifact(stdout_path),
673
+ "stderr_log": _artifact(stderr_path),
674
+ }
675
+ _write_json_exclusive(attempt_manifest, attempt_record)
676
+ attempt_summary = dict(attempt_record)
677
+ attempt_summary["attempt_manifest"] = _artifact(attempt_manifest)
678
+ state["attempts"].append(attempt_summary)
679
+ state["status"] = (
680
+ "running" if termination != "total_job_stop" else "job_stopped"
681
+ )
682
+ state["result_rows"] = len(prefix)
683
+ state["updated_utc"] = ended_utc
684
+ if results_path.is_file():
685
+ state["artifacts"]["partial_results"] = _artifact(results_path)
686
+ _write_json_atomic(state_path, state)
687
+ if termination == "total_job_stop":
688
+ return 3
689
+ if termination == "launcher_exit" and return_code != 0:
690
+ state["status"] = "failed"
691
+ state["failure"] = (
692
+ f"launcher exited with code {return_code}; inspect immutable attempt logs"
693
+ )
694
+ _write_json_atomic(state_path, state)
695
+ return 4
696
+ if termination == "launcher_exit" and len(prefix) < len(questions):
697
+ state["status"] = "failed"
698
+ state["failure"] = "launcher exited successfully before producing all rows"
699
+ _write_json_atomic(state_path, state)
700
+ return 4
701
+
702
+ state["status"] = "succeeded"
703
+ state["completed_utc"] = _utc_now()
704
+ state["result_rows"] = len(prefix)
705
+ state["artifacts"].pop("partial_results", None)
706
+ state["artifacts"]["results"] = _artifact(results_path)
707
+ _write_json_atomic(state_path, state)
708
+ return 0
709
+ finally:
710
+ # This supervisor is deliberately lock-free. Exact-prefix validation and
711
+ # process-tree termination are its safety boundaries; callers must
712
+ # serialize supervisor invocations.
713
+ pass
714
+
715
+
716
+ def _parser() -> argparse.ArgumentParser:
717
+ parser = argparse.ArgumentParser(
718
+ description="Run v12 search with an external hard 1000-second question cap."
719
+ )
720
+ parser.add_argument("--questions", type=Path, required=True)
721
+ parser.add_argument("--results", type=Path, required=True)
722
+ parser.add_argument("--checkpoint", type=Path, required=True)
723
+ parser.add_argument("--launcher", type=Path, default=DEFAULT_LAUNCHER)
724
+ parser.add_argument("--artifact-dir", type=Path)
725
+ parser.add_argument("--state", type=Path)
726
+ parser.add_argument("--startup-grace-sec", type=float, default=0.0)
727
+ parser.add_argument(
728
+ "--question-timeout-sec", type=float, default=QUESTION_TIMEOUT_SEC
729
+ )
730
+ parser.add_argument("--tree-size-cap", type=int, default=TREE_SIZE_CAP)
731
+ parser.add_argument(
732
+ "--total-job-stop-sec", type=float, default=TOTAL_JOB_STOP_SEC
733
+ )
734
+ parser.add_argument("--poll-interval-sec", type=float, default=0.25)
735
+ parser.add_argument(
736
+ "launcher_args",
737
+ nargs=argparse.REMAINDER,
738
+ help="Arguments passed to run_search_benchmark.py after a standalone --.",
739
+ )
740
+ return parser
741
+
742
+
743
+ def main(argv: Sequence[str] | None = None) -> int:
744
+ args = _parser().parse_args(argv)
745
+ launcher_args = list(args.launcher_args)
746
+ if launcher_args[:1] == ["--"]:
747
+ launcher_args = launcher_args[1:]
748
+ results = args.results.expanduser().resolve()
749
+ artifact_dir = (
750
+ args.artifact_dir.expanduser().resolve()
751
+ if args.artifact_dir
752
+ else results.with_name(results.name + ".hard-supervisor-attempts")
753
+ )
754
+ state = (
755
+ args.state.expanduser().resolve()
756
+ if args.state
757
+ else results.with_name(results.name + ".hard-supervisor-state.json")
758
+ )
759
+ try:
760
+ return run_supervisor(
761
+ questions_path=args.questions,
762
+ results_path=results,
763
+ checkpoint=args.checkpoint,
764
+ launcher=args.launcher,
765
+ artifact_dir=artifact_dir,
766
+ state_path=state,
767
+ launcher_args=launcher_args,
768
+ question_timeout_sec=args.question_timeout_sec,
769
+ tree_size_cap=args.tree_size_cap,
770
+ startup_grace_sec=args.startup_grace_sec,
771
+ total_job_stop_sec=args.total_job_stop_sec,
772
+ poll_interval_sec=args.poll_interval_sec,
773
+ )
774
+ except SupervisorError as exc:
775
+ print(f"error: {exc}", file=sys.stderr)
776
+ return 2
777
+
778
+
779
+ if __name__ == "__main__":
780
+ raise SystemExit(main())
mce220_runner/grpo_pipeline/__init__.py ADDED
File without changes
mce220_runner/grpo_pipeline/modules/__init__.py ADDED
File without changes
mce220_runner/grpo_pipeline/modules/llm_api_interface.py ADDED
@@ -0,0 +1,427 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Local LLM interface for beam_search.
3
+
4
+ This replaces the previous OpenRouter/API-based implementation.
5
+
6
+ Public API:
7
+ - LLM(prompt, Gamma=None, M_current=None, k=5, adapter_path=None, use_dummy=False, **gen_kwargs) -> List[str]
8
+ """
9
+
10
+ from __future__ import annotations
11
+
12
+ import json
13
+ import os
14
+ import re
15
+ import sys
16
+ import hashlib
17
+ from dataclasses import dataclass
18
+ from typing import List, Optional, Any
19
+
20
+
21
+ _MODEL_CACHE: dict[tuple[str, str], Any] = {}
22
+ _DUMMY_CALL_IDX: int = 0
23
+ _LAST_LLM_STATS: dict[str, Any] = {}
24
+
25
+
26
+ def get_last_llm_stats(*, reset: bool = False) -> dict[str, Any]:
27
+ stats = dict(_LAST_LLM_STATS)
28
+ if reset:
29
+ _LAST_LLM_STATS.clear()
30
+ return stats
31
+
32
+
33
+ def _strip_code_fences(text: str) -> str:
34
+ text = re.sub(r"^```[a-zA-Z0-9_-]*\s*", "", text.strip())
35
+ text = re.sub(r"\s*```$", "", text.strip())
36
+ return text.strip()
37
+
38
+
39
+ def _extract_inequalities(text: str, k: int) -> List[str]:
40
+ """
41
+ Extract up to k candidate inequality strings from model output.
42
+ Accepts numbered lists, bullets, code fences, and JSON arrays.
43
+ """
44
+ text = _strip_code_fences(text)
45
+
46
+ # Try JSON first
47
+ try:
48
+ obj = json.loads(text)
49
+ if isinstance(obj, list):
50
+ out = []
51
+ for x in obj:
52
+ if isinstance(x, str):
53
+ s = x.strip()
54
+ if s:
55
+ out.append(s)
56
+ if out:
57
+ return out[:k]
58
+ except Exception:
59
+ pass
60
+
61
+ # Otherwise parse lines
62
+ lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
63
+ cands: List[str] = []
64
+ for ln in lines:
65
+ # remove common list prefixes: "1.", "1)", "*", and "-" ONLY if followed by space (bullet point)
66
+ # We must NOT strip a leading "-" if it's followed by a digit or "H("/"I(", as it's likely a negative coefficient.
67
+ ln = re.sub(r"^\s*(?:[*]|(\d+)[\.\)]|-(?:\s+|$))\s*", "", ln).strip()
68
+ ln = ln.strip("`").strip()
69
+ if not ln:
70
+ continue
71
+
72
+ # keep only plausible math constraints or explicit "wrong" signal
73
+ if ("H(" in ln) or ("I(" in ln):
74
+ cands.append(ln)
75
+ elif "wrong" in ln.lower():
76
+ cands.append("wrong")
77
+
78
+ if len(cands) >= k:
79
+ break
80
+
81
+ return cands[:k]
82
+
83
+
84
+ def _read_base_model_from_adapter(adapter_path: str) -> Optional[str]:
85
+ cfg_path = os.path.join(adapter_path, "adapter_config.json")
86
+ try:
87
+ with open(cfg_path, "r", encoding="utf-8") as f:
88
+ cfg = json.load(f)
89
+ base = cfg.get("base_model_name_or_path")
90
+ return base if isinstance(base, str) and base else None
91
+ except Exception:
92
+ return None
93
+
94
+
95
+ def _load_tokenizer_compat(tokenizer_src: str):
96
+ from transformers import AutoTokenizer
97
+
98
+ extra_kwargs = {}
99
+ cfg_path = os.path.join(tokenizer_src, "tokenizer_config.json")
100
+ if os.path.isdir(tokenizer_src) and os.path.exists(cfg_path):
101
+ try:
102
+ with open(cfg_path, "r", encoding="utf-8") as f:
103
+ cfg = json.load(f)
104
+ if isinstance(cfg.get("extra_special_tokens"), list):
105
+ sys.stderr.write(
106
+ f"[llm] Sanitizing tokenizer extra_special_tokens for: {tokenizer_src}\n"
107
+ )
108
+ extra_kwargs["extra_special_tokens"] = {}
109
+ except Exception:
110
+ pass
111
+
112
+ return AutoTokenizer.from_pretrained(
113
+ tokenizer_src,
114
+ trust_remote_code=True,
115
+ **extra_kwargs,
116
+ )
117
+
118
+
119
+ def _load_local_model(
120
+ adapter_path: Optional[str] = None,
121
+ *,
122
+ base_model: Optional[str] = None,
123
+ device_map: str = "auto",
124
+ ):
125
+ """
126
+ Load HF base model, optionally with a PEFT adapter.
127
+ Caches model/tokenizer to avoid re-loading each call.
128
+ """
129
+ from transformers import AutoTokenizer, AutoModelForCausalLM
130
+ import torch
131
+
132
+ # Only treat as adapter if path is truthy AND contains adapter_config.json
133
+ adapter_abs = ""
134
+ if adapter_path and adapter_path.strip():
135
+ abs_p = os.path.abspath(adapter_path.strip())
136
+ if os.path.exists(os.path.join(abs_p, "adapter_config.json")):
137
+ adapter_abs = abs_p
138
+ else:
139
+ sys.stderr.write(f"[llm] Warning: adapter_path '{adapter_path}' provided but no adapter_config.json found in {abs_p}. Ignoring adapter.\n")
140
+
141
+ resolved_base = (
142
+ base_model
143
+ or os.environ.get("QWEN_BASE_MODEL")
144
+ or (_read_base_model_from_adapter(adapter_abs) if adapter_abs else None)
145
+ or "Qwen/Qwen3-0.6B"
146
+ )
147
+
148
+ weights_path = os.environ.get("QWEN_WEIGHTS_PATH", "").strip()
149
+ cache_key = (adapter_abs, resolved_base, weights_path)
150
+ if cache_key in _MODEL_CACHE:
151
+ return _MODEL_CACHE[cache_key]
152
+
153
+ if weights_path:
154
+ from safetensors.torch import load_file
155
+ sys.stderr.write(f"[llm] Loading base arch from: {resolved_base}\n")
156
+ sys.stderr.write(f"[llm] Overriding weights from: {weights_path}\n")
157
+ sys.stderr.flush()
158
+ tokenizer = _load_tokenizer_compat(resolved_base)
159
+ model = AutoModelForCausalLM.from_pretrained(
160
+ resolved_base,
161
+ torch_dtype=torch.bfloat16 if torch.cuda.is_available() else "auto",
162
+ device_map=device_map,
163
+ trust_remote_code=True,
164
+ )
165
+ ckpt = load_file(os.path.join(weights_path, "model.safetensors"), device="cpu")
166
+ # The in-memory model state_dict includes lm_head.weight (tied to embed_tokens.weight).
167
+ # The GRPO checkpoint also has lm_head.weight serialized by DeepSpeed — load all keys.
168
+ model.load_state_dict(ckpt, strict=True)
169
+ model.eval()
170
+ _MODEL_CACHE[cache_key] = (model, tokenizer)
171
+ return model, tokenizer
172
+
173
+ if adapter_abs:
174
+ sys.stderr.write(f"[llm] Loading model: {resolved_base}\n")
175
+ sys.stderr.write(f"[llm] Loading adapter: {adapter_abs}\n")
176
+ else:
177
+ sys.stderr.write(f"[llm] Loading base model only: {resolved_base}\n")
178
+ sys.stderr.flush()
179
+
180
+ # Tokenizer should come from the adapter if present (it might include special tokens),
181
+ # otherwise from the base model.
182
+ tok_src = adapter_abs or resolved_base
183
+ tokenizer = _load_tokenizer_compat(tok_src)
184
+ model = AutoModelForCausalLM.from_pretrained(
185
+ resolved_base,
186
+ torch_dtype=torch.bfloat16 if torch.cuda.is_available() else "auto",
187
+ device_map=device_map,
188
+ trust_remote_code=True,
189
+ )
190
+
191
+ if adapter_abs:
192
+ from peft import PeftModel
193
+ model = PeftModel.from_pretrained(model, adapter_abs)
194
+
195
+ model.eval()
196
+
197
+ _MODEL_CACHE[cache_key] = (model, tokenizer)
198
+ return model, tokenizer
199
+
200
+
201
+ def _build_alpaca_prompt(input_text: str) -> str:
202
+ # Mirror the exact formatting observed in ft_using_Axolotl_1/last_run_prepared/* decoding
203
+ instruction = "You are proving the entropy inequality. Follow the instructions carefully."
204
+ preamble = (
205
+ "Below is an instruction that describes a task, paired with an input that provides further context. "
206
+ "Write a response that appropriately completes the request."
207
+ )
208
+ return (
209
+ f"{preamble}\n\n"
210
+ f"### Instruction:\n{instruction}\n\n"
211
+ f"### Input:\n{input_text}\n\n\n\n"
212
+ f"### Response:\n"
213
+ )
214
+
215
+
216
+ def LLM(
217
+ prompt: str,
218
+ Gamma=None, # kept for backward compatibility (ignored for local inference)
219
+ M_current=None, # kept for backward compatibility (ignored for local inference)
220
+ k: int = 5,
221
+ adapter_path: Optional[str] = None,
222
+ use_dummy: bool = False,
223
+ **gen_kwargs,
224
+ ) -> List[str]:
225
+ """
226
+ Local inference to propose inequalities.
227
+
228
+ Args:
229
+ prompt: natural language prompt
230
+ k: number of inequalities requested
231
+ adapter_path: local path to PEFT adapter directory (required unless use_dummy=True)
232
+ use_dummy: if True, returns a small hardcoded set for testing
233
+ gen_kwargs: passed to model.generate (e.g. max_new_tokens, temperature)
234
+ """
235
+ global _LAST_LLM_STATS
236
+
237
+ if use_dummy:
238
+ # Dummy mode should still generate *varied* valid inequalities; otherwise tree_search
239
+ # will keep adding the same single inequality and will never be able to prove even
240
+ # simple Shannon-type targets.
241
+ #
242
+ # We deterministically vary outputs based on the prompt content so different nodes
243
+ # (with different CURRENT INEQUALITIES M) receive different proposals.
244
+ def _parse_n_from_prompt(p: str) -> int:
245
+ # Try old format first
246
+ m = re.search(r"NUMBER OF VARIABLES:\s*\n(\d+)\b", p)
247
+ if m:
248
+ try:
249
+ return max(1, int(m.group(1)))
250
+ except Exception:
251
+ pass
252
+
253
+ # Fallback: Extract max index from X_{...}
254
+ idxs = []
255
+ for inside in re.findall(r"X_\{([^}]+)\}", p):
256
+ parts = [part.strip() for part in inside.split(",") if part.strip()]
257
+ for part in parts:
258
+ if part.isdigit():
259
+ idxs.append(int(part))
260
+ if idxs:
261
+ return max(idxs)
262
+ return 3
263
+
264
+ def _parse_allowed_range_from_prompt(p: str) -> Optional[tuple[int, int]]:
265
+ # Matches: "range 4..8" in IMPORTANT CONSTRAINT section.
266
+ m = re.search(r"range\s+(\d+)\.\.(\d+)", p)
267
+ if m:
268
+ try:
269
+ lo = int(m.group(1))
270
+ hi = int(m.group(2))
271
+ if lo > hi:
272
+ lo, hi = hi, lo
273
+ return lo, hi
274
+ except Exception:
275
+ pass
276
+
277
+ # Fallback: return (1, n)
278
+ return None
279
+
280
+ def _dummy_pool(n: int, lo: int, hi: int) -> List[str]:
281
+ cands: List[str] = []
282
+ lo = max(1, int(lo))
283
+ hi = min(int(n), int(hi))
284
+ if lo > hi:
285
+ lo, hi = 1, int(n)
286
+ vars_list = list(range(lo, hi + 1))
287
+ # Non-negativity
288
+ for i in vars_list:
289
+ cands.append(f"H(X_{{{i}}})>=0")
290
+ # Mutual information (unconditional)
291
+ for ii, i in enumerate(vars_list):
292
+ for j in vars_list[ii + 1 :]:
293
+ cands.append(f"I(X_{{{i}}};X_{{{j}}})>=0")
294
+ # Conditional mutual information with a single conditioning variable
295
+ if len(vars_list) >= 3:
296
+ for ii, i in enumerate(vars_list):
297
+ for j in vars_list[ii + 1 :]:
298
+ for c in vars_list:
299
+ if c != i and c != j:
300
+ cands.append(f"I(X_{{{i}}};X_{{{j}}}|X_{{{c}}})>=0")
301
+ # Conditional entropy with a single conditioning variable
302
+ for i in vars_list:
303
+ for j in vars_list:
304
+ if j != i:
305
+ cands.append(f"H(X_{{{i}}}|X_{{{j}}})>=0")
306
+ return cands
307
+
308
+ global _DUMMY_CALL_IDX
309
+ _DUMMY_CALL_IDX += 1
310
+
311
+ n = _parse_n_from_prompt(prompt)
312
+ rng = _parse_allowed_range_from_prompt(prompt) or (1, n)
313
+ pool = _dummy_pool(n, rng[0], rng[1])
314
+ # Stable seed from prompt content + call index, so repeated calls with the same
315
+ # prompt still yield different proposals (needed for rollout diversity).
316
+ seed_material = f"{prompt}\n[DUMMY_CALL_IDX]={_DUMMY_CALL_IDX}".encode("utf-8")
317
+ seed = int.from_bytes(hashlib.md5(seed_material).digest()[:8], "big", signed=False)
318
+ import random
319
+
320
+ rng = random.Random(seed)
321
+ rng.shuffle(pool)
322
+ out = pool[: max(0, int(k))]
323
+ if os.environ.get("QWEN_PRINT_OUTPUT", "0") == "1":
324
+ target_match = re.search(r"TARGET:\s*\n(.*?)\n\n", prompt, re.DOTALL)
325
+ target_val = target_match.group(1).strip() if target_match else "unknown"
326
+ sys.stderr.write(f"[llm] Target: {target_val}\n")
327
+ sys.stderr.write(f" Dummy Extracted: {out}\n")
328
+ sys.stderr.flush()
329
+ _LAST_LLM_STATS = {
330
+ "prompt_tokens": 0,
331
+ "generated_tokens_total": 0,
332
+ "generated_tokens_per_sequence": [0 for _ in out],
333
+ "num_sequences": len(out),
334
+ "extracted_candidates": len(out),
335
+ }
336
+ return out
337
+
338
+ # Use raw prompt when requested (matches alphabet SFT format).
339
+ if os.environ.get("QWEN_RAW_PROMPT", "0") == "1":
340
+ constrained_prompt = prompt
341
+ else:
342
+ # Fully adapt to the SFT training format: Alpaca template + caller-provided input block.
343
+ constrained_prompt = _build_alpaca_prompt(prompt.strip())
344
+ if os.environ.get("QWEN_PRINT_PROMPT", "0") == "1":
345
+ sys.stderr.write("[llm] Full prompt:\n")
346
+ sys.stderr.write(constrained_prompt)
347
+ sys.stderr.write("\n")
348
+ sys.stderr.flush()
349
+
350
+ # If adapter_path is omitted, we fall back to the baseline model (no PEFT).
351
+ model, tokenizer = _load_local_model(adapter_path=adapter_path)
352
+
353
+ import torch
354
+
355
+ if tokenizer.pad_token_id is None:
356
+ tokenizer.pad_token_id = tokenizer.eos_token_id
357
+
358
+ inputs = tokenizer(constrained_prompt, return_tensors="pt")
359
+ inputs = {k: v.to(model.device) for k, v in inputs.items()}
360
+ # Print prompt tokens alongside prompt for debugging alignment
361
+
362
+ default_kwargs = dict(
363
+ max_new_tokens=int(os.environ.get("QWEN_MAX_NEW_TOKENS", "512")),
364
+ # max_time=float(os.environ.get("QWEN_MAX_TIME", "40")),
365
+ do_sample=True,
366
+ temperature=float(os.environ.get("QWEN_TEMPERATURE", "0.7")),
367
+ top_p=float(os.environ.get("QWEN_TOP_P", "0.9")),
368
+ repetition_penalty=1.05,
369
+ pad_token_id=tokenizer.eos_token_id,
370
+ eos_token_id=tokenizer.eos_token_id,
371
+ )
372
+ default_kwargs.update(gen_kwargs or {})
373
+
374
+ with torch.inference_mode():
375
+ out = model.generate(
376
+ input_ids=inputs["input_ids"],
377
+ attention_mask=inputs.get("attention_mask", None),
378
+ **default_kwargs,
379
+ )
380
+
381
+ # Decode only the newly generated part when possible.
382
+ # If num_return_sequences > 1, generate() returns multiple sequences.
383
+ prompt_len = inputs["input_ids"].shape[-1]
384
+ cands: List[str] = []
385
+ decoded_texts: List[str] = []
386
+
387
+ if hasattr(out, "shape") and len(out.shape) >= 2 and out.shape[0] > 1:
388
+ # One inequality per returned sequence (preferred for "GPU batch generation")
389
+ generated_token_lengths: List[int] = []
390
+ for i in range(out.shape[0]):
391
+ gen_ids = out[i][prompt_len:]
392
+ generated_token_lengths.append(int(gen_ids.shape[-1]))
393
+ text_i = tokenizer.decode(gen_ids, skip_special_tokens=True).strip()
394
+ decoded_texts.append(text_i)
395
+ one = _extract_inequalities(text_i, k=1)
396
+ if one:
397
+ cands.append(one[0])
398
+ if len(cands) >= k:
399
+ # Keep decoding for printing if needed, but we have enough candidates
400
+ pass
401
+ else:
402
+ gen_ids = out[0][prompt_len:]
403
+ generated_token_lengths = [int(gen_ids.shape[-1])]
404
+ text = tokenizer.decode(gen_ids, skip_special_tokens=True).strip()
405
+ decoded_texts.append(text)
406
+ cands = _extract_inequalities(text, k=k)
407
+
408
+ _LAST_LLM_STATS = {
409
+ "prompt_tokens": int(prompt_len),
410
+ "generated_tokens_total": int(sum(generated_token_lengths)),
411
+ "generated_tokens_per_sequence": generated_token_lengths,
412
+ "num_sequences": len(generated_token_lengths),
413
+ "extracted_candidates": len(cands),
414
+ }
415
+
416
+ if os.environ.get("QWEN_PRINT_OUTPUT", "0") == "1":
417
+ # Simplified printing: only target, generated, and extracted.
418
+ target_match = re.search(r"TARGET:\s*\n(.*?)\n\n", prompt, re.DOTALL)
419
+ target_val = target_match.group(1).strip() if target_match else "unknown"
420
+ sys.stderr.write(f"[llm] Target: {target_val}\n")
421
+ for i, txt in enumerate(decoded_texts):
422
+ # Only print first line of generated text if it's long? No, user said generated inequality.
423
+ # Usually model outputs just the inequality.
424
+ sys.stderr.write(f" Gen {i}: {txt}\n")
425
+ sys.stderr.write(f" Extracted: {cands}\n")
426
+ sys.stderr.flush()
427
+ return cands
mce220_runner/requirements.txt ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ # Install a CUDA-enabled PyTorch wheel suitable for the host before this file if needed.
2
+ torch==2.3.1
3
+ transformers==4.55.2
4
+ peft==0.17.0
5
+ accelerate==1.10.0
6
+ huggingface_hub==0.36.2
7
+ safetensors==0.4.3
8
+ numpy==1.26.4
9
+ scipy==1.13.1
10
+ sympy==1.12.1
mce220_runner/run_mce220.sh ADDED
@@ -0,0 +1,182 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ usage() {
5
+ cat <<'EOF'
6
+ Usage: run_mce220.sh {dry-run|smoke|full} [--gpu-ids LIST] [--workers N] [--output-dir DIR]
7
+
8
+ Modes:
9
+ dry-run Verify all downloaded assets and print the frozen 220-row command.
10
+ smoke Evaluate the first question with one GPU and reduced search settings.
11
+ full Evaluate all 220 questions under the hard-timeout supervisor.
12
+ EOF
13
+ }
14
+
15
+ if [[ $# -lt 1 ]]; then
16
+ usage >&2
17
+ exit 2
18
+ fi
19
+
20
+ MODE="$1"
21
+ shift
22
+ case "$MODE" in
23
+ dry-run|smoke|full) ;;
24
+ *) usage >&2; exit 2 ;;
25
+ esac
26
+
27
+ SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
28
+ PYTHON_BIN="${PYTHON:-python3}"
29
+ ASSET_DIR="${MCE220_ASSET_DIR:-$SCRIPT_DIR/assets}"
30
+ GPU_IDS="0,1,2,3,4,5,6,7"
31
+ WORKERS=""
32
+ OUTPUT_DIR=""
33
+
34
+ while [[ $# -gt 0 ]]; do
35
+ case "$1" in
36
+ --gpu-ids)
37
+ [[ $# -ge 2 ]] || { echo "missing value for --gpu-ids" >&2; exit 2; }
38
+ GPU_IDS="$2"; shift 2 ;;
39
+ --workers)
40
+ [[ $# -ge 2 ]] || { echo "missing value for --workers" >&2; exit 2; }
41
+ WORKERS="$2"; shift 2 ;;
42
+ --output-dir)
43
+ [[ $# -ge 2 ]] || { echo "missing value for --output-dir" >&2; exit 2; }
44
+ OUTPUT_DIR="$2"; shift 2 ;;
45
+ -h|--help)
46
+ usage; exit 0 ;;
47
+ *) echo "unknown option: $1" >&2; usage >&2; exit 2 ;;
48
+ esac
49
+ done
50
+
51
+ if [[ -z "$WORKERS" ]]; then
52
+ if [[ "$MODE" == "smoke" ]]; then WORKERS=1; else WORKERS=8; fi
53
+ fi
54
+ [[ "$WORKERS" =~ ^[1-9][0-9]*$ ]] || {
55
+ echo "--workers must be a positive integer" >&2
56
+ exit 2
57
+ }
58
+ if [[ "$MODE" == "smoke" && "$WORKERS" != "1" ]]; then
59
+ echo "smoke mode is intentionally restricted to --workers 1" >&2
60
+ exit 2
61
+ fi
62
+
63
+ if [[ -z "$OUTPUT_DIR" ]]; then
64
+ OUTPUT_DIR="$SCRIPT_DIR/runs/$MODE"
65
+ fi
66
+ mkdir -p -- "$OUTPUT_DIR"
67
+
68
+ QUESTIONS="$SCRIPT_DIR/dataset/data/questions.jsonl"
69
+ CHECKPOINT="$ASSET_DIR/model-repo/qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534"
70
+ BASE_MODEL="$ASSET_DIR/qwen3-0.6b"
71
+ RUNNER="$SCRIPT_DIR/beam_search_model2/run_on_window_shear_jsonl_v12.py"
72
+ WRAPPER="$SCRIPT_DIR/experiments/reviewer_response/run_search_benchmark.py"
73
+ SUPERVISOR="$SCRIPT_DIR/experiments/reviewer_response/supervise_search_benchmark.py"
74
+
75
+ "$PYTHON_BIN" "$SCRIPT_DIR/download_assets.py" \
76
+ --asset-dir "$ASSET_DIR" \
77
+ --verify-only
78
+
79
+ common_full_args=(
80
+ --questions "$QUESTIONS"
81
+ --checkpoint "$CHECKPOINT"
82
+ --base-model "$BASE_MODEL"
83
+ --runner "$RUNNER"
84
+ --gpu-ids "$GPU_IDS"
85
+ --workers "$WORKERS"
86
+ --beam-size 32
87
+ --rollouts 6
88
+ --max-depth 24
89
+ --seed 42
90
+ --temperature 1.2
91
+ --max-new-tokens 2048
92
+ --generation-max-time 20
93
+ --max-vars-for-lp 0
94
+ --lp-workers "$WORKERS"
95
+ --min-n 10
96
+ --max-n 20
97
+ --prompt-format canonical
98
+ --normalize-candidates
99
+ --raw-prompt
100
+ --print-llm
101
+ )
102
+
103
+ case "$MODE" in
104
+ dry-run)
105
+ stamp="$(date -u +%Y%m%dT%H%M%SZ)-$$"
106
+ "$PYTHON_BIN" "$WRAPPER" \
107
+ "${common_full_args[@]}" \
108
+ --results "$OUTPUT_DIR/mce220-dry-run-results.jsonl" \
109
+ --manifest "$OUTPUT_DIR/mce220-dry-run-$stamp.manifest.json" \
110
+ --sample-max-time-sec 1000 \
111
+ --hard-question-timeout-sec 1000 \
112
+ --sample-max-tree-size 6000 \
113
+ --dry-run
114
+ ;;
115
+ smoke)
116
+ first_gpu="${GPU_IDS%%,*}"
117
+ smoke_questions="$OUTPUT_DIR/smoke-question.jsonl"
118
+ head -n 1 -- "$QUESTIONS" > "$smoke_questions"
119
+ smoke_results="$OUTPUT_DIR/smoke-results.jsonl"
120
+ resume_args=()
121
+ [[ -s "$smoke_results" ]] && resume_args+=(--resume)
122
+ "$PYTHON_BIN" "$WRAPPER" \
123
+ --questions "$smoke_questions" \
124
+ --results "$smoke_results" \
125
+ --checkpoint "$CHECKPOINT" \
126
+ --base-model "$BASE_MODEL" \
127
+ --runner "$RUNNER" \
128
+ --gpu-ids "$first_gpu" \
129
+ --workers 1 \
130
+ --beam-size 4 \
131
+ --rollouts 1 \
132
+ --max-depth 4 \
133
+ --seed 42 \
134
+ --temperature 1.2 \
135
+ --max-new-tokens 512 \
136
+ --generation-max-time 20 \
137
+ --sample-max-time-sec 300 \
138
+ --sample-max-tree-size 256 \
139
+ --max-vars-for-lp 0 \
140
+ --lp-workers 1 \
141
+ --min-n 10 \
142
+ --max-n 20 \
143
+ --prompt-format canonical \
144
+ --normalize-candidates \
145
+ --raw-prompt \
146
+ --print-llm \
147
+ "${resume_args[@]}"
148
+ ;;
149
+ full)
150
+ "$PYTHON_BIN" "$SUPERVISOR" \
151
+ --questions "$QUESTIONS" \
152
+ --results "$OUTPUT_DIR/results.jsonl" \
153
+ --checkpoint "$CHECKPOINT" \
154
+ --launcher "$WRAPPER" \
155
+ --artifact-dir "$OUTPUT_DIR/supervisor-attempts" \
156
+ --state "$OUTPUT_DIR/supervisor-state.json" \
157
+ --question-timeout-sec 1000 \
158
+ --tree-size-cap 6000 \
159
+ --total-job-stop-sec 151200 \
160
+ --poll-interval-sec 0.25 \
161
+ -- \
162
+ --base-model "$BASE_MODEL" \
163
+ --runner "$RUNNER" \
164
+ --gpu-ids "$GPU_IDS" \
165
+ --workers "$WORKERS" \
166
+ --beam-size 32 \
167
+ --rollouts 6 \
168
+ --max-depth 24 \
169
+ --seed 42 \
170
+ --temperature 1.2 \
171
+ --max-new-tokens 2048 \
172
+ --generation-max-time 20 \
173
+ --max-vars-for-lp 0 \
174
+ --lp-workers "$WORKERS" \
175
+ --min-n 10 \
176
+ --max-n 20 \
177
+ --prompt-format canonical \
178
+ --normalize-candidates \
179
+ --raw-prompt \
180
+ --print-llm
181
+ ;;
182
+ esac