Instructions to use entropyinequalityprover/Models_Scripts_MCE-220 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use entropyinequalityprover/Models_Scripts_MCE-220 with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Add reproducible MCE-220 Model A runner
Browse filesBundle the frozen MCE-220 dataset and audit trail, corrected Model A evaluation runner, pinned asset downloader, exact verifier, and hard-timeout supervisor.
- README.md +20 -0
- mce220_runner/README.md +54 -0
- mce220_runner/SOURCE_MANIFEST.json +252 -0
- mce220_runner/beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py +2232 -0
- mce220_runner/beam_search_model2/__init__.py +0 -0
- mce220_runner/beam_search_model2/llm_api_compat.py +26 -0
- mce220_runner/beam_search_model2/llmstr_to_vector.py +104 -0
- mce220_runner/beam_search_model2/run_on_window_shear_jsonl_v12.py +821 -0
- mce220_runner/beam_search_model2/verify_proof_certificate.py +557 -0
- mce220_runner/data/__init__.py +0 -0
- mce220_runner/data/entropy_vector_converter.py +250 -0
- mce220_runner/dataset/README.md +89 -0
- mce220_runner/dataset/audit/diversity-audit.json +422 -0
- mce220_runner/dataset/audit/generation-report.jsonl +220 -0
- mce220_runner/dataset/audit/provenance.jsonl +0 -0
- mce220_runner/dataset/audit/run-manifest.json +1 -0
- mce220_runner/dataset/audit/validation-report.json +1 -0
- mce220_runner/dataset/data/questions.jsonl +0 -0
- mce220_runner/dataset/generation/e2n_v2.toml +63 -0
- mce220_runner/download_assets.py +249 -0
- mce220_runner/experiments/__init__.py +0 -0
- mce220_runner/experiments/reviewer_response/__init__.py +0 -0
- mce220_runner/experiments/reviewer_response/configs/inference_policy.toml +16 -0
- mce220_runner/experiments/reviewer_response/run_search_benchmark.py +1108 -0
- mce220_runner/experiments/reviewer_response/supervise_search_benchmark.py +780 -0
- mce220_runner/grpo_pipeline/__init__.py +0 -0
- mce220_runner/grpo_pipeline/modules/__init__.py +0 -0
- mce220_runner/grpo_pipeline/modules/llm_api_interface.py +427 -0
- mce220_runner/requirements.txt +10 -0
- mce220_runner/run_mce220.sh +182 -0
README.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- peft
|
| 4 |
+
- qwen3
|
| 5 |
+
- entropy
|
| 6 |
+
- theorem-proving
|
| 7 |
+
- benchmark
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# Entropy Inequality Prover models and scripts
|
| 11 |
+
|
| 12 |
+
This repository contains the project checkpoints and inference resources. The new [MCE-220 Model A runner](./mce220_runner/) packages the corrected beam-search implementation, exact proof verifier, hard-timeout supervisor, frozen inference policy, and a complete local copy of the 220-question benchmark with its audit trail.
|
| 13 |
+
|
| 14 |
+
## MCE-220 release
|
| 15 |
+
|
| 16 |
+
- Runner and instructions: [`mce220_runner/`](./mce220_runner/)
|
| 17 |
+
- Model A adapter: [`qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534/`](./qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534/)
|
| 18 |
+
- Bundled benchmark: [`mce220_runner/dataset/`](./mce220_runner/dataset/)
|
| 19 |
+
|
| 20 |
+
The downloader pins this model repository at `7fcfe778e1c36cd67e88ea8aa2ee761a50a39127` and the base model at `c1899de289a04d12100db370d81485cdf75e47ca`. Existing checkpoint directories are unchanged. Model and benchmark reuse terms have not yet been assigned; this card does not add a license.
|
mce220_runner/README.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# MCE-220 Model A runner
|
| 2 |
+
|
| 3 |
+
This directory is a self-contained evaluation bundle for Model A checkpoint-9534 on the frozen 220-question MCE-220 benchmark. The benchmark, provenance, generation reports, validation evidence, diversity audit, generation configuration, runner, exact verifier, and supervision code are included here. Large model weights are downloaded on demand from immutable Hugging Face revisions.
|
| 4 |
+
|
| 5 |
+
## Pinned inputs
|
| 6 |
+
|
| 7 |
+
- Dataset source commit: `5650d7c8198a68116e8d12934aed41a37866ba92` (a byte-identical copy is under `dataset/`).
|
| 8 |
+
- Model repository commit: `7fcfe778e1c36cd67e88ea8aa2ee761a50a39127`.
|
| 9 |
+
- Model A adapter: `qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534`.
|
| 10 |
+
- Base model: `Qwen/Qwen3-0.6B` at `c1899de289a04d12100db370d81485cdf75e47ca`.
|
| 11 |
+
|
| 12 |
+
`download_assets.py` downloads only the nine inference-required checkpoint files. It rejects optimizer, scheduler, RNG, trainer-state, and training-argument artifacts, verifies the adapter and tokenizer SHA-256 values, downloads the seven required base-model files, and compares the bundled dataset with the pinned dataset source.
|
| 13 |
+
|
| 14 |
+
## Setup
|
| 15 |
+
|
| 16 |
+
Linux with Bash, Python 3.10 or newer, CUDA, and NVIDIA GPUs is expected. Create an environment, install the CUDA-enabled PyTorch build suitable for the host, then install the remaining pinned requirements:
|
| 17 |
+
|
| 18 |
+
```bash
|
| 19 |
+
python3 -m venv .venv
|
| 20 |
+
source .venv/bin/activate
|
| 21 |
+
pip install -r mce220_runner/requirements.txt
|
| 22 |
+
python mce220_runner/download_assets.py
|
| 23 |
+
```
|
| 24 |
+
|
| 25 |
+
If your platform needs a different PyTorch CUDA wheel, install that wheel first and then install the other pinned packages without replacing it. The base model is about 1.5 GB and the adapter about 162 MB. A worker holds an independent model instance; allow roughly 3–5 GiB of free VRAM per worker plus search overhead. The paper configuration uses eight workers. GPU IDs may be repeated only when a device has enough memory for multiple workers.
|
| 26 |
+
|
| 27 |
+
## Commands
|
| 28 |
+
|
| 29 |
+
```bash
|
| 30 |
+
# Validate every asset and print the frozen 220-row command without loading a model.
|
| 31 |
+
bash mce220_runner/run_mce220.sh dry-run --gpu-ids 0,1,2,3,4,5,6,7 --workers 8
|
| 32 |
+
|
| 33 |
+
# Run one question with beam 4, one rollout, depth 4, and one GPU.
|
| 34 |
+
bash mce220_runner/run_mce220.sh smoke --gpu-ids 0 --workers 1
|
| 35 |
+
|
| 36 |
+
# Run all questions with the published search and hard-timeout policy.
|
| 37 |
+
bash mce220_runner/run_mce220.sh full --gpu-ids 0,1,2,3,4,5,6,7 --workers 8 --output-dir ./mce220-results
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
Set `PYTHON` to choose the interpreter and `MCE220_ASSET_DIR` to place downloaded weights outside this directory.
|
| 41 |
+
|
| 42 |
+
## Frozen full-run policy
|
| 43 |
+
|
| 44 |
+
The full mode fixes beam size 32, six rollouts, maximum depth 24, seed 42, temperature 1.2, top-p 0.9, 2,048 generated tokens, 6,000 explored nodes, and a 1,000-second external hard timeout for every question. It uses the canonical raw prompt, candidate normalization, exact Shannon LP checks, and the pinned local base-model snapshot. The supervisor has a 42-hour total stop so interrupted jobs can be resumed deliberately.
|
| 45 |
+
|
| 46 |
+
## Outputs and resume behavior
|
| 47 |
+
|
| 48 |
+
`full` writes `results.jsonl`, `supervisor-state.json`, and immutable per-attempt manifests/stdout/stderr under `supervisor-attempts/`. Each successful or timed-out question produces one row, and proof-bearing rows are checked by the exact rational verifier. Re-running `full` with the same output directory validates the existing prefix and continues at the next question. It never silently replaces a nonempty result file. `smoke` similarly resumes a valid one-row prefix. Use a new output directory to start a clean run.
|
| 49 |
+
|
| 50 |
+
## Limitations
|
| 51 |
+
|
| 52 |
+
The benchmark contains constructed Shannon-type problems with supplied equality conditions; it does not cover arbitrary dense or non-Shannon inequalities. The base model and adapter are trusted local Hugging Face artifacts and the loader uses `trust_remote_code=True`, although the pinned Qwen revision contains no custom Python files. GPU sampling remains stochastic across kernels, devices, drivers, and library builds. A clean rerun follows the published protocol but is not guaranteed to reproduce exactly 155 proofs out of 220.
|
| 53 |
+
|
| 54 |
+
No AITIP/PSITIP baseline, training code, or historical segmented runner is included. No license is asserted by this bundle.
|
mce220_runner/SOURCE_MANIFEST.json
ADDED
|
@@ -0,0 +1,252 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-09-01T11:12:07.595478Z",
|
| 3 |
+
"files": {
|
| 4 |
+
"README.md": {
|
| 5 |
+
"bytes": 4249,
|
| 6 |
+
"sha256": "d7fb5484e254526cfca3476ef341fabcf14f34479f375ee8cb3416dc565aeaac",
|
| 7 |
+
"source": {
|
| 8 |
+
"kind": "generated-release-file"
|
| 9 |
+
}
|
| 10 |
+
},
|
| 11 |
+
"beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py": {
|
| 12 |
+
"bytes": 97133,
|
| 13 |
+
"sha256": "403370fcb05db7fb515109c76a87eb67ccd7033fdb06544fe6b239138a21b393",
|
| 14 |
+
"source": {
|
| 15 |
+
"kind": "local-source",
|
| 16 |
+
"path": "beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py"
|
| 17 |
+
}
|
| 18 |
+
},
|
| 19 |
+
"beam_search_model2/__init__.py": {
|
| 20 |
+
"bytes": 0,
|
| 21 |
+
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 22 |
+
"source": {
|
| 23 |
+
"kind": "generated-release-file"
|
| 24 |
+
}
|
| 25 |
+
},
|
| 26 |
+
"beam_search_model2/llm_api_compat.py": {
|
| 27 |
+
"bytes": 872,
|
| 28 |
+
"sha256": "8ed35fee37688c020051f13cf9945a402c8b1b969ca06acd204e629be9ed2cc0",
|
| 29 |
+
"source": {
|
| 30 |
+
"kind": "local-source",
|
| 31 |
+
"path": "beam_search_model2/llm_api_compat.py"
|
| 32 |
+
}
|
| 33 |
+
},
|
| 34 |
+
"beam_search_model2/llmstr_to_vector.py": {
|
| 35 |
+
"bytes": 3242,
|
| 36 |
+
"sha256": "0611db5d168460783755395b5a7aa4d95d78a8397139b53584eb5a17dc27c8ef",
|
| 37 |
+
"source": {
|
| 38 |
+
"kind": "local-source",
|
| 39 |
+
"path": "beam_search_model2/llmstr_to_vector.py"
|
| 40 |
+
}
|
| 41 |
+
},
|
| 42 |
+
"beam_search_model2/run_on_window_shear_jsonl_v12.py": {
|
| 43 |
+
"bytes": 30325,
|
| 44 |
+
"sha256": "197fd80b4dc39486e987b10547ed49a84fabf953e2ff6a2393b5c47cdadc95bf",
|
| 45 |
+
"source": {
|
| 46 |
+
"kind": "local-source",
|
| 47 |
+
"path": "beam_search_model2/run_on_window_shear_jsonl_v12.py"
|
| 48 |
+
}
|
| 49 |
+
},
|
| 50 |
+
"beam_search_model2/verify_proof_certificate.py": {
|
| 51 |
+
"bytes": 20711,
|
| 52 |
+
"sha256": "ba00337dac6e8715a8f4dd32166bb1393ec02e37ad931e9affdae7741807349a",
|
| 53 |
+
"source": {
|
| 54 |
+
"kind": "local-source",
|
| 55 |
+
"path": "beam_search_model2/verify_proof_certificate.py"
|
| 56 |
+
}
|
| 57 |
+
},
|
| 58 |
+
"data/__init__.py": {
|
| 59 |
+
"bytes": 0,
|
| 60 |
+
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 61 |
+
"source": {
|
| 62 |
+
"kind": "generated-release-file"
|
| 63 |
+
}
|
| 64 |
+
},
|
| 65 |
+
"data/entropy_vector_converter.py": {
|
| 66 |
+
"bytes": 9470,
|
| 67 |
+
"sha256": "3a1398661b146e661c91b2fa4d815a5e87b2a209751dba630ec6e88feb34d109",
|
| 68 |
+
"source": {
|
| 69 |
+
"kind": "local-source",
|
| 70 |
+
"path": "data/entropy_vector_converter.py"
|
| 71 |
+
}
|
| 72 |
+
},
|
| 73 |
+
"dataset/README.md": {
|
| 74 |
+
"bytes": 4651,
|
| 75 |
+
"sha256": "909828a43ace9c3d4e9aa2e2fe96812dadd1c08ece52c87cf4dc896ad4ce0452",
|
| 76 |
+
"source": {
|
| 77 |
+
"kind": "dataset-snapshot",
|
| 78 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 79 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 80 |
+
}
|
| 81 |
+
},
|
| 82 |
+
"dataset/audit/diversity-audit.json": {
|
| 83 |
+
"bytes": 8239,
|
| 84 |
+
"sha256": "56721eb34f35fa9acc2d81bbb13b0ea263a7f29bc8838372339dbc38ecda44a4",
|
| 85 |
+
"source": {
|
| 86 |
+
"kind": "dataset-snapshot",
|
| 87 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 88 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 89 |
+
}
|
| 90 |
+
},
|
| 91 |
+
"dataset/audit/generation-report.jsonl": {
|
| 92 |
+
"bytes": 69724,
|
| 93 |
+
"sha256": "50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c",
|
| 94 |
+
"source": {
|
| 95 |
+
"kind": "dataset-snapshot",
|
| 96 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 97 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 98 |
+
}
|
| 99 |
+
},
|
| 100 |
+
"dataset/audit/provenance.jsonl": {
|
| 101 |
+
"bytes": 4453377,
|
| 102 |
+
"sha256": "cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189",
|
| 103 |
+
"source": {
|
| 104 |
+
"kind": "dataset-snapshot",
|
| 105 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 106 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 107 |
+
}
|
| 108 |
+
},
|
| 109 |
+
"dataset/audit/run-manifest.json": {
|
| 110 |
+
"bytes": 2750,
|
| 111 |
+
"sha256": "ac32fdaecf845714f9802c584c91b8743398898ff7e7c1f36c43014dea36d9a6",
|
| 112 |
+
"source": {
|
| 113 |
+
"kind": "dataset-snapshot",
|
| 114 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 115 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 116 |
+
}
|
| 117 |
+
},
|
| 118 |
+
"dataset/audit/validation-report.json": {
|
| 119 |
+
"bytes": 381,
|
| 120 |
+
"sha256": "3f86d1391c15a343cb938092a456ce2b5b88bb7f1ebcbf4089d0c8174c7e4d3c",
|
| 121 |
+
"source": {
|
| 122 |
+
"kind": "dataset-snapshot",
|
| 123 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 124 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 125 |
+
}
|
| 126 |
+
},
|
| 127 |
+
"dataset/data/questions.jsonl": {
|
| 128 |
+
"bytes": 499119,
|
| 129 |
+
"sha256": "1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a",
|
| 130 |
+
"source": {
|
| 131 |
+
"kind": "dataset-snapshot",
|
| 132 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 133 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 134 |
+
}
|
| 135 |
+
},
|
| 136 |
+
"dataset/generation/e2n_v2.toml": {
|
| 137 |
+
"bytes": 1560,
|
| 138 |
+
"sha256": "3346eb6fc6aa0bb70ca15f9a840d7b0f366418072c58cb76d32e0f8ac9faae4a",
|
| 139 |
+
"source": {
|
| 140 |
+
"kind": "dataset-snapshot",
|
| 141 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 142 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 143 |
+
}
|
| 144 |
+
},
|
| 145 |
+
"download_assets.py": {
|
| 146 |
+
"bytes": 8434,
|
| 147 |
+
"sha256": "f308777bcf46500d4ce8fe2636e0b957ec0c361dd74fc06416204ccb17538610",
|
| 148 |
+
"source": {
|
| 149 |
+
"kind": "generated-release-file"
|
| 150 |
+
}
|
| 151 |
+
},
|
| 152 |
+
"experiments/__init__.py": {
|
| 153 |
+
"bytes": 0,
|
| 154 |
+
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 155 |
+
"source": {
|
| 156 |
+
"kind": "generated-release-file"
|
| 157 |
+
}
|
| 158 |
+
},
|
| 159 |
+
"experiments/reviewer_response/__init__.py": {
|
| 160 |
+
"bytes": 0,
|
| 161 |
+
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 162 |
+
"source": {
|
| 163 |
+
"kind": "generated-release-file"
|
| 164 |
+
}
|
| 165 |
+
},
|
| 166 |
+
"experiments/reviewer_response/configs/inference_policy.toml": {
|
| 167 |
+
"bytes": 671,
|
| 168 |
+
"sha256": "ba4e6497f6e688a0740cfbcc945488c20a9f447084b3e05bde4bb08ef51e07fe",
|
| 169 |
+
"source": {
|
| 170 |
+
"kind": "local-source",
|
| 171 |
+
"path": "experiments/reviewer_response/configs/inference_policy.toml"
|
| 172 |
+
}
|
| 173 |
+
},
|
| 174 |
+
"experiments/reviewer_response/run_search_benchmark.py": {
|
| 175 |
+
"bytes": 40875,
|
| 176 |
+
"sha256": "7c3a442548089419d0c1f1af24d764474aee7f540a818bd3bb27db74c997c0bd",
|
| 177 |
+
"source": {
|
| 178 |
+
"change": "portable --base-model argument and effective-path recording",
|
| 179 |
+
"kind": "derived-local-source",
|
| 180 |
+
"path": "experiments/reviewer_response/run_search_benchmark.py"
|
| 181 |
+
}
|
| 182 |
+
},
|
| 183 |
+
"experiments/reviewer_response/supervise_search_benchmark.py": {
|
| 184 |
+
"bytes": 30678,
|
| 185 |
+
"sha256": "3a748d5651de33aedb93d1ae9bf396ba9cbf3e87a69dce45fea1edcb52137d96",
|
| 186 |
+
"source": {
|
| 187 |
+
"kind": "local-source",
|
| 188 |
+
"path": "experiments/reviewer_response/supervise_search_benchmark.py"
|
| 189 |
+
}
|
| 190 |
+
},
|
| 191 |
+
"grpo_pipeline/__init__.py": {
|
| 192 |
+
"bytes": 0,
|
| 193 |
+
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 194 |
+
"source": {
|
| 195 |
+
"kind": "generated-release-file"
|
| 196 |
+
}
|
| 197 |
+
},
|
| 198 |
+
"grpo_pipeline/modules/__init__.py": {
|
| 199 |
+
"bytes": 0,
|
| 200 |
+
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 201 |
+
"source": {
|
| 202 |
+
"kind": "generated-release-file"
|
| 203 |
+
}
|
| 204 |
+
},
|
| 205 |
+
"grpo_pipeline/modules/llm_api_interface.py": {
|
| 206 |
+
"bytes": 16629,
|
| 207 |
+
"sha256": "4e7bd0f8c13be24d4e09ea90270799cbc440015beb0575e314b5d659b2a7544e",
|
| 208 |
+
"source": {
|
| 209 |
+
"kind": "local-source",
|
| 210 |
+
"path": "grpo_pipeline/modules/llm_api_interface.py"
|
| 211 |
+
}
|
| 212 |
+
},
|
| 213 |
+
"requirements.txt": {
|
| 214 |
+
"bytes": 240,
|
| 215 |
+
"sha256": "cbfac9b9d0f3a38e9c1a4c361856fe9678d95c31cf0b895ecab06de544a85a25",
|
| 216 |
+
"source": {
|
| 217 |
+
"kind": "generated-release-file"
|
| 218 |
+
}
|
| 219 |
+
},
|
| 220 |
+
"run_mce220.sh": {
|
| 221 |
+
"bytes": 5042,
|
| 222 |
+
"sha256": "fb3198e886fe53b09011344620da50b5734acd49e24ef147f8fe3e58f44f8959",
|
| 223 |
+
"source": {
|
| 224 |
+
"kind": "generated-release-file"
|
| 225 |
+
}
|
| 226 |
+
}
|
| 227 |
+
},
|
| 228 |
+
"pinned_revisions": {
|
| 229 |
+
"base_model": {
|
| 230 |
+
"repo": "Qwen/Qwen3-0.6B",
|
| 231 |
+
"revision": "c1899de289a04d12100db370d81485cdf75e47ca"
|
| 232 |
+
},
|
| 233 |
+
"dataset": {
|
| 234 |
+
"repo": "entropyinequalityprover/MCE-220",
|
| 235 |
+
"revision": "5650d7c8198a68116e8d12934aed41a37866ba92"
|
| 236 |
+
},
|
| 237 |
+
"model": {
|
| 238 |
+
"repo": "entropyinequalityprover/models_and_scripts",
|
| 239 |
+
"revision": "7fcfe778e1c36cd67e88ea8aa2ee761a50a39127"
|
| 240 |
+
}
|
| 241 |
+
},
|
| 242 |
+
"schema_version": "mce220-runner-source-manifest-v1",
|
| 243 |
+
"self_reference": {
|
| 244 |
+
"path": "SOURCE_MANIFEST.json",
|
| 245 |
+
"reason": "A file cannot contain its own cryptographic digest; verify this file from the release commit.",
|
| 246 |
+
"sha256": null
|
| 247 |
+
},
|
| 248 |
+
"source_workspace": {
|
| 249 |
+
"git_head": "985afadcface3883a66a1c0bcd39d141737abc1c",
|
| 250 |
+
"note": "Selected files were copied from the validated current worktree without modifying it."
|
| 251 |
+
}
|
| 252 |
+
}
|
mce220_runner/beam_search_model2/Data_Generation_via_LLM_Guided_Search_v12.py
ADDED
|
@@ -0,0 +1,2232 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# This is for llm inference on 4type data trained llm model2.
|
| 2 |
+
import random
|
| 3 |
+
import os
|
| 4 |
+
import sys
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
from concurrent.futures import ProcessPoolExecutor, as_completed
|
| 7 |
+
from concurrent.futures.process import BrokenProcessPool
|
| 8 |
+
import multiprocessing as mp
|
| 9 |
+
|
| 10 |
+
import numpy as np
|
| 11 |
+
import re
|
| 12 |
+
from scipy.sparse import csr_matrix, vstack, hstack, coo_matrix
|
| 13 |
+
from scipy.optimize import linprog
|
| 14 |
+
from typing import List, Dict, Tuple, Optional
|
| 15 |
+
import json
|
| 16 |
+
import time
|
| 17 |
+
import uuid
|
| 18 |
+
import pickle
|
| 19 |
+
# Make imports work when running from repo root (or anywhere)
|
| 20 |
+
_REPO_ROOT = Path(__file__).resolve().parents[1]
|
| 21 |
+
_DATA_DIR = _REPO_ROOT / "data"
|
| 22 |
+
_MODEL2_DIR = _REPO_ROOT / "beam_search_model2"
|
| 23 |
+
for _p in (str(_DATA_DIR), str(_MODEL2_DIR)):
|
| 24 |
+
if _p not in sys.path:
|
| 25 |
+
sys.path.append(_p)
|
| 26 |
+
|
| 27 |
+
try:
|
| 28 |
+
# Running as a script (local dir on sys.path)
|
| 29 |
+
from llmstr_to_vector import polym_ineq_to_vector
|
| 30 |
+
from entropy_vector_converter import string_to_vector, vector_to_string
|
| 31 |
+
from llm_api_compat import LLM as LLM_propose_inequalities
|
| 32 |
+
from llm_api_compat import get_last_llm_stats
|
| 33 |
+
except ModuleNotFoundError:
|
| 34 |
+
# Fallback to absolute/package-style imports if needed
|
| 35 |
+
from beam_search_model2.llmstr_to_vector import polym_ineq_to_vector
|
| 36 |
+
from entropy_vector_converter import string_to_vector, vector_to_string
|
| 37 |
+
from beam_search_model2.llm_api_compat import LLM as LLM_propose_inequalities
|
| 38 |
+
from beam_search_model2.llm_api_compat import get_last_llm_stats
|
| 39 |
+
|
| 40 |
+
# ---- Multi-GPU LLM worker pool support (one process per GPU; one model per process) ----
|
| 41 |
+
_GPU_ADAPTER_PATH = None
|
| 42 |
+
_GPU_USE_DUMMY = False
|
| 43 |
+
_GPU_READY = False
|
| 44 |
+
|
| 45 |
+
# Global cache for persistent GPU executors
|
| 46 |
+
_CACHED_GPU_EXECUTORS = None
|
| 47 |
+
_CACHED_GPU_PARAMS = None
|
| 48 |
+
_CACHED_GAMMA = {}
|
| 49 |
+
_COND_CACHE = {}
|
| 50 |
+
_LAST_RUN_ID = None
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def _is_fatal_gpu_worker_exception(exc: BaseException) -> bool:
|
| 54 |
+
"""Identify pool crashes and accelerator OOMs that must abort the run."""
|
| 55 |
+
|
| 56 |
+
current: BaseException | None = exc
|
| 57 |
+
seen: set[int] = set()
|
| 58 |
+
while current is not None and id(current) not in seen:
|
| 59 |
+
seen.add(id(current))
|
| 60 |
+
if isinstance(current, BrokenProcessPool):
|
| 61 |
+
return True
|
| 62 |
+
class_name = type(current).__name__.lower()
|
| 63 |
+
message = str(current).lower()
|
| 64 |
+
if "outofmemory" in class_name or any(
|
| 65 |
+
fragment in message
|
| 66 |
+
for fragment in (
|
| 67 |
+
"cuda out of memory",
|
| 68 |
+
"cuda error: out of memory",
|
| 69 |
+
"hip out of memory",
|
| 70 |
+
"cublas_status_alloc_failed",
|
| 71 |
+
"cudnn_status_alloc_failed",
|
| 72 |
+
"process in the process pool was terminated abruptly",
|
| 73 |
+
)
|
| 74 |
+
):
|
| 75 |
+
return True
|
| 76 |
+
current = current.__cause__ or current.__context__
|
| 77 |
+
return False
|
| 78 |
+
|
| 79 |
+
ALPHABET = list("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
|
| 80 |
+
_X_TOKEN_RE = re.compile(r"X_\{([^}]+)\}")
|
| 81 |
+
_MULTIPLIER_PAREN_RE = re.compile(r"(\b\d+)\*\(")
|
| 82 |
+
_HI_TERM_RE = re.compile(r"(?<![A-Za-z])[HI]\([^)]*\)")
|
| 83 |
+
_SINGLE_POSITIVE_STEP_RE = re.compile(
|
| 84 |
+
r"^\+?(?:(\d*\.?\d+)\*?)?([HI]\([^()]*\))$"
|
| 85 |
+
)
|
| 86 |
+
|
| 87 |
+
def _use_alpha_prompt() -> bool:
|
| 88 |
+
return os.environ.get("BEAM_PROMPT_FORMAT", "alphabet").strip().lower() != "canonical"
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def _sort_alpha_groups_enabled() -> bool:
|
| 92 |
+
return os.environ.get("BEAM_SORT_ALPHA_GROUPS", "0").strip() == "1"
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def _normalize_candidates_enabled() -> bool:
|
| 96 |
+
return os.environ.get("BEAM_NORMALIZE_CANDIDATES", "0").strip() == "1"
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def _format_scalar_text(value: float) -> str:
|
| 100 |
+
rounded = round(value)
|
| 101 |
+
if abs(value - rounded) <= 1e-12:
|
| 102 |
+
return str(int(rounded))
|
| 103 |
+
return f"{value:.12g}"
|
| 104 |
+
|
| 105 |
+
|
| 106 |
+
def _strip_wrapping_parens(expr: str) -> str:
|
| 107 |
+
s = (expr or "").strip()
|
| 108 |
+
while s.startswith("(") and s.endswith(")"):
|
| 109 |
+
depth = 0
|
| 110 |
+
wraps_all = True
|
| 111 |
+
for idx, ch in enumerate(s):
|
| 112 |
+
if ch == "(":
|
| 113 |
+
depth += 1
|
| 114 |
+
elif ch == ")":
|
| 115 |
+
depth -= 1
|
| 116 |
+
if depth < 0:
|
| 117 |
+
return s
|
| 118 |
+
if depth == 0 and idx != len(s) - 1:
|
| 119 |
+
wraps_all = False
|
| 120 |
+
break
|
| 121 |
+
if depth != 0 or not wraps_all:
|
| 122 |
+
break
|
| 123 |
+
s = s[1:-1].strip()
|
| 124 |
+
return s
|
| 125 |
+
|
| 126 |
+
|
| 127 |
+
def _split_top_level_linear_terms(expr: str) -> List[str]:
|
| 128 |
+
parts: List[str] = []
|
| 129 |
+
start = 0
|
| 130 |
+
depth = 0
|
| 131 |
+
for idx, ch in enumerate(expr):
|
| 132 |
+
if ch == "(":
|
| 133 |
+
depth += 1
|
| 134 |
+
elif ch == ")":
|
| 135 |
+
depth -= 1
|
| 136 |
+
elif depth == 0 and idx > start and ch in "+-":
|
| 137 |
+
parts.append(expr[start:idx])
|
| 138 |
+
start = idx
|
| 139 |
+
parts.append(expr[start:])
|
| 140 |
+
return [p for p in parts if p]
|
| 141 |
+
|
| 142 |
+
|
| 143 |
+
def _parse_linear_hi_term(term: str) -> Optional[Tuple[float, str]]:
|
| 144 |
+
s = _strip_wrapping_parens(term)
|
| 145 |
+
m = re.fullmatch(r"([+-]?)(?:(\d*\.?\d+)\*?)?([HI]\([^()]*\))", s)
|
| 146 |
+
if not m:
|
| 147 |
+
return None
|
| 148 |
+
sign_str, coeff_str, hi_term = m.groups()
|
| 149 |
+
coeff = 1.0
|
| 150 |
+
if coeff_str:
|
| 151 |
+
try:
|
| 152 |
+
coeff = float(coeff_str)
|
| 153 |
+
except ValueError:
|
| 154 |
+
return None
|
| 155 |
+
if sign_str == "-":
|
| 156 |
+
coeff *= -1.0
|
| 157 |
+
return coeff, hi_term
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
def _distribute_scalar_over_hi_expr(expr: str, scalar: float) -> Optional[str]:
|
| 161 |
+
inner = _strip_wrapping_parens(expr)
|
| 162 |
+
if not inner:
|
| 163 |
+
return None
|
| 164 |
+
pieces = _split_top_level_linear_terms(inner)
|
| 165 |
+
if not pieces:
|
| 166 |
+
return None
|
| 167 |
+
scaled_terms: List[str] = []
|
| 168 |
+
for piece in pieces:
|
| 169 |
+
parsed = _parse_linear_hi_term(piece)
|
| 170 |
+
if parsed is None:
|
| 171 |
+
return None
|
| 172 |
+
coeff, hi_term = parsed
|
| 173 |
+
scaled_terms.append(f"{_format_scalar_text(scalar * coeff)}*{hi_term}")
|
| 174 |
+
return "".join(
|
| 175 |
+
term if idx == 0 or term.startswith("-") else f"+{term}"
|
| 176 |
+
for idx, term in enumerate(scaled_terms)
|
| 177 |
+
)
|
| 178 |
+
|
| 179 |
+
|
| 180 |
+
def _collapse_leading_scalar_wrappers(expr: str) -> str:
|
| 181 |
+
s = _strip_wrapping_parens(expr)
|
| 182 |
+
while True:
|
| 183 |
+
m = re.fullmatch(r"([+-]?\d*\.?\d+)\*\((.*)\)", s)
|
| 184 |
+
if not m:
|
| 185 |
+
return s
|
| 186 |
+
try:
|
| 187 |
+
outer_coeff = float(m.group(1))
|
| 188 |
+
except ValueError:
|
| 189 |
+
return s
|
| 190 |
+
inner = _collapse_leading_scalar_wrappers(m.group(2))
|
| 191 |
+
distributed = _distribute_scalar_over_hi_expr(inner, outer_coeff)
|
| 192 |
+
if distributed is None:
|
| 193 |
+
return s
|
| 194 |
+
s = distributed
|
| 195 |
+
|
| 196 |
+
|
| 197 |
+
def _build_alpha_mapping(exprs: List[str]) -> Tuple[Optional[Dict[int, str]], Optional[Dict[str, int]]]:
|
| 198 |
+
"""
|
| 199 |
+
Build a deterministic mapping between indices and letters for alphabet syntax.
|
| 200 |
+
Returns (idx_to_letter, letter_to_idx) or (None, None) if not applicable.
|
| 201 |
+
"""
|
| 202 |
+
indices: List[int] = []
|
| 203 |
+
for expr in exprs:
|
| 204 |
+
indices.extend(_extract_var_indices(expr))
|
| 205 |
+
unique = sorted(set(indices))
|
| 206 |
+
if not unique:
|
| 207 |
+
return None, None
|
| 208 |
+
if len(unique) > len(ALPHABET):
|
| 209 |
+
sys.stderr.write(
|
| 210 |
+
f"[beam_debug] Too many unique indices ({len(unique)}) for alphabet mapping; "
|
| 211 |
+
"falling back to numeric syntax.\n"
|
| 212 |
+
)
|
| 213 |
+
sys.stderr.flush()
|
| 214 |
+
return None, None
|
| 215 |
+
idx_to_letter = {idx: ALPHABET[i] for i, idx in enumerate(unique)}
|
| 216 |
+
letter_to_idx = {v: k for k, v in idx_to_letter.items()}
|
| 217 |
+
return idx_to_letter, letter_to_idx
|
| 218 |
+
|
| 219 |
+
|
| 220 |
+
def _convert_old_to_alpha(expr: str, idx_to_letter: Optional[Dict[int, str]]) -> str:
|
| 221 |
+
if not expr or not idx_to_letter:
|
| 222 |
+
return expr
|
| 223 |
+
s = _normalize_to_braces(expr)
|
| 224 |
+
sort_groups = _sort_alpha_groups_enabled()
|
| 225 |
+
|
| 226 |
+
def repl(match: re.Match) -> str:
|
| 227 |
+
parts = [p.strip() for p in match.group(1).split(",")]
|
| 228 |
+
letters = []
|
| 229 |
+
for part in parts:
|
| 230 |
+
if not part.isdigit():
|
| 231 |
+
letters.append(part)
|
| 232 |
+
continue
|
| 233 |
+
letters.append(idx_to_letter.get(int(part), part))
|
| 234 |
+
if sort_groups:
|
| 235 |
+
letters = sorted(letters)
|
| 236 |
+
return "".join(letters)
|
| 237 |
+
|
| 238 |
+
s = _X_TOKEN_RE.sub(repl, s)
|
| 239 |
+
s = _MULTIPLIER_PAREN_RE.sub(r"\1(", s)
|
| 240 |
+
return s
|
| 241 |
+
|
| 242 |
+
|
| 243 |
+
def _convert_alpha_group(text: str, letter_to_idx: Dict[str, int]) -> str:
|
| 244 |
+
if "X_" in text:
|
| 245 |
+
return text
|
| 246 |
+
letters = re.findall(r"[A-Z]", text)
|
| 247 |
+
if not letters:
|
| 248 |
+
return text
|
| 249 |
+
if _sort_alpha_groups_enabled():
|
| 250 |
+
letters = sorted(letters)
|
| 251 |
+
indices = []
|
| 252 |
+
for ch in letters:
|
| 253 |
+
if ch not in letter_to_idx:
|
| 254 |
+
return text
|
| 255 |
+
indices.append(str(letter_to_idx[ch]))
|
| 256 |
+
return "X_{" + ",".join(indices) + "}"
|
| 257 |
+
|
| 258 |
+
|
| 259 |
+
def _convert_alpha_term(term: str, letter_to_idx: Dict[str, int]) -> str:
|
| 260 |
+
if term.startswith("H(") and term.endswith(")"):
|
| 261 |
+
inner = term[2:-1]
|
| 262 |
+
if "|" in inner:
|
| 263 |
+
left, right = inner.split("|", 1)
|
| 264 |
+
return f"H({_convert_alpha_group(left, letter_to_idx)}|{_convert_alpha_group(right, letter_to_idx)})"
|
| 265 |
+
return f"H({_convert_alpha_group(inner, letter_to_idx)})"
|
| 266 |
+
if term.startswith("I(") and term.endswith(")"):
|
| 267 |
+
inner = term[2:-1]
|
| 268 |
+
if "|" in inner:
|
| 269 |
+
main, cond = inner.split("|", 1)
|
| 270 |
+
else:
|
| 271 |
+
main, cond = inner, None
|
| 272 |
+
if ";" in main:
|
| 273 |
+
left, right = main.split(";", 1)
|
| 274 |
+
else:
|
| 275 |
+
left, right = main, ""
|
| 276 |
+
left_conv = _convert_alpha_group(left, letter_to_idx)
|
| 277 |
+
right_conv = _convert_alpha_group(right, letter_to_idx)
|
| 278 |
+
if cond is not None:
|
| 279 |
+
cond_conv = _convert_alpha_group(cond, letter_to_idx)
|
| 280 |
+
return f"I({left_conv};{right_conv}|{cond_conv})"
|
| 281 |
+
return f"I({left_conv};{right_conv})"
|
| 282 |
+
return term
|
| 283 |
+
|
| 284 |
+
|
| 285 |
+
def _convert_alpha_to_old(expr: str, letter_to_idx: Optional[Dict[str, int]]) -> str:
|
| 286 |
+
if not expr or not letter_to_idx:
|
| 287 |
+
return expr
|
| 288 |
+
s = _HI_TERM_RE.sub(lambda m: _convert_alpha_term(m.group(0), letter_to_idx), expr)
|
| 289 |
+
s = re.sub(r"(\b\d+)\(", r"\1*(", s)
|
| 290 |
+
return s
|
| 291 |
+
|
| 292 |
+
|
| 293 |
+
def _sort_alpha_group_text(text: str) -> str:
|
| 294 |
+
raw = text.strip()
|
| 295 |
+
if not raw or "X_" in raw:
|
| 296 |
+
return raw
|
| 297 |
+
letters = re.findall(r"[A-Z]", raw)
|
| 298 |
+
if not letters:
|
| 299 |
+
return raw
|
| 300 |
+
leftover = re.sub(r"[A-Z\s]", "", raw)
|
| 301 |
+
if leftover:
|
| 302 |
+
return raw
|
| 303 |
+
return "".join(sorted(letters))
|
| 304 |
+
|
| 305 |
+
|
| 306 |
+
def _canonicalize_alpha_term_text(term: str) -> str:
|
| 307 |
+
if term.startswith("H(") and term.endswith(")"):
|
| 308 |
+
inner = term[2:-1]
|
| 309 |
+
if "|" in inner:
|
| 310 |
+
left, right = inner.split("|", 1)
|
| 311 |
+
return f"H({_sort_alpha_group_text(left)}|{_sort_alpha_group_text(right)})"
|
| 312 |
+
return f"H({_sort_alpha_group_text(inner)})"
|
| 313 |
+
if term.startswith("I(") and term.endswith(")"):
|
| 314 |
+
inner = term[2:-1]
|
| 315 |
+
if "|" in inner:
|
| 316 |
+
main, cond = inner.split("|", 1)
|
| 317 |
+
else:
|
| 318 |
+
main, cond = inner, None
|
| 319 |
+
if ";" in main:
|
| 320 |
+
left, right = main.split(";", 1)
|
| 321 |
+
else:
|
| 322 |
+
left, right = main, ""
|
| 323 |
+
left = _sort_alpha_group_text(left)
|
| 324 |
+
right = _sort_alpha_group_text(right)
|
| 325 |
+
if cond is not None:
|
| 326 |
+
return f"I({left};{right}|{_sort_alpha_group_text(cond)})"
|
| 327 |
+
return f"I({left};{right})"
|
| 328 |
+
return term
|
| 329 |
+
|
| 330 |
+
|
| 331 |
+
def _normalize_candidate_text(cand_raw: str) -> str:
|
| 332 |
+
s = cand_raw.strip().strip("`").strip()
|
| 333 |
+
if not s:
|
| 334 |
+
return s
|
| 335 |
+
s = re.sub(r"^```[a-zA-Z0-9_-]*\s*", "", s)
|
| 336 |
+
s = re.sub(r"\s*```$", "", s)
|
| 337 |
+
s = re.sub(r"^\s*(?:try|candidate|answer|output|step)\s*:\s*", "", s, flags=re.IGNORECASE)
|
| 338 |
+
s = re.sub(r"^\s*(?:[*]|(\d+)[\.\)]|-(?:\s+|$))\s*", "", s).strip()
|
| 339 |
+
if s.lower() == "wrong":
|
| 340 |
+
return "wrong"
|
| 341 |
+
first_math = re.search(r"[HI]\(", s)
|
| 342 |
+
if first_math and first_math.start() > 0:
|
| 343 |
+
prefix = s[: first_math.start()]
|
| 344 |
+
if ":" in prefix or re.search(r"[A-Za-z]", prefix):
|
| 345 |
+
s = s[first_math.start() :].strip()
|
| 346 |
+
s = re.split(r"\bbecause\b", s, maxsplit=1, flags=re.IGNORECASE)[0].strip()
|
| 347 |
+
s = re.sub(r"\s+", "", s)
|
| 348 |
+
s = re.sub(r">\s*=\s*0\b", ">=0", s)
|
| 349 |
+
s = re.sub(r"<\s*=\s*0\b", "<=0", s)
|
| 350 |
+
s = re.sub(r"(?<![<>])=\s*0\b", "=0", s)
|
| 351 |
+
s = re.sub(r">\s*0\b", ">=0", s)
|
| 352 |
+
s = re.sub(r"\(\s*", "(", s)
|
| 353 |
+
s = re.sub(r"\s*\)", ")", s)
|
| 354 |
+
s = re.sub(r"\s*;\s*", ";", s)
|
| 355 |
+
s = re.sub(r"\s*\|\s*", "|", s)
|
| 356 |
+
s = re.sub(r"\s*,\s*", ",", s)
|
| 357 |
+
s = re.sub(r"\s*\+\s*", "+", s)
|
| 358 |
+
s = re.sub(r"\s*-\s*", "-", s)
|
| 359 |
+
s = re.sub(r"\s*\*\s*", "*", s)
|
| 360 |
+
# 若缺少比較運算子,自動補齊 >=0
|
| 361 |
+
if not any(op in s for op in [">=0", ">0", "<=0", "=0"]):
|
| 362 |
+
s += ">=0"
|
| 363 |
+
|
| 364 |
+
# 補全未閉合的括號 (Auto-balance parentheses before the comparator)
|
| 365 |
+
for op in [">=0", ">0", "<=0", "=0"]:
|
| 366 |
+
if op in s:
|
| 367 |
+
lhs, rhs = s.split(op, 1)
|
| 368 |
+
open_p = lhs.count("(")
|
| 369 |
+
close_p = lhs.count(")")
|
| 370 |
+
if open_p > close_p:
|
| 371 |
+
lhs += ")" * (open_p - close_p)
|
| 372 |
+
lhs = _collapse_leading_scalar_wrappers(lhs)
|
| 373 |
+
s = lhs + op + rhs
|
| 374 |
+
break
|
| 375 |
+
|
| 376 |
+
if _sort_alpha_groups_enabled():
|
| 377 |
+
s = _HI_TERM_RE.sub(lambda m: _canonicalize_alpha_term_text(m.group(0)), s)
|
| 378 |
+
return s
|
| 379 |
+
|
| 380 |
+
|
| 381 |
+
def _empty_llm_usage() -> Dict[str, int]:
|
| 382 |
+
return {
|
| 383 |
+
"llm_calls": 0,
|
| 384 |
+
"prompt_tokens_total": 0,
|
| 385 |
+
"generated_tokens_total": 0,
|
| 386 |
+
"generated_sequences_total": 0,
|
| 387 |
+
"extracted_candidates_total": 0,
|
| 388 |
+
"format_fail_total": 0,
|
| 389 |
+
}
|
| 390 |
+
|
| 391 |
+
|
| 392 |
+
def _merge_llm_usage(dst: Dict[str, int], src: Optional[Dict]) -> None:
|
| 393 |
+
if not src:
|
| 394 |
+
return
|
| 395 |
+
dst["llm_calls"] += 1
|
| 396 |
+
dst["prompt_tokens_total"] += int(src.get("prompt_tokens", 0) or 0)
|
| 397 |
+
dst["generated_tokens_total"] += int(src.get("generated_tokens_total", 0) or 0)
|
| 398 |
+
dst["generated_sequences_total"] += int(src.get("num_sequences", 0) or 0)
|
| 399 |
+
dst["extracted_candidates_total"] += int(src.get("extracted_candidates", 0) or 0)
|
| 400 |
+
|
| 401 |
+
|
| 402 |
+
def _split_candidate_comparator(cand: str) -> Tuple[str, Optional[str]]:
|
| 403 |
+
s = (cand or "").strip()
|
| 404 |
+
match = re.search(r"(>=|<=|>|<|=)\s*0\s*$", s)
|
| 405 |
+
if match:
|
| 406 |
+
comparator = {
|
| 407 |
+
">=": ">=0",
|
| 408 |
+
"<=": "<=0",
|
| 409 |
+
">": ">0",
|
| 410 |
+
"<": "<0",
|
| 411 |
+
"=": "=0",
|
| 412 |
+
}[match.group(1)]
|
| 413 |
+
return s[: match.start()].strip(), comparator
|
| 414 |
+
return s, None
|
| 415 |
+
|
| 416 |
+
|
| 417 |
+
def _normalize_constraint_comparator(expr: str, *, add_default: bool = False) -> str:
|
| 418 |
+
"""Canonicalize a comparator, weakening strict inequalities safely."""
|
| 419 |
+
raw = (expr or "").strip()
|
| 420 |
+
lhs, comparator = _split_candidate_comparator(raw)
|
| 421 |
+
if comparator == ">0":
|
| 422 |
+
comparator = ">=0"
|
| 423 |
+
elif comparator == "<0":
|
| 424 |
+
comparator = "<=0"
|
| 425 |
+
if comparator is None:
|
| 426 |
+
return f"{lhs}>=0" if add_default else lhs
|
| 427 |
+
return f"{lhs}{comparator}"
|
| 428 |
+
|
| 429 |
+
|
| 430 |
+
def _condition_comparator_allows_candidate(
|
| 431 |
+
cond_comparator: Optional[str],
|
| 432 |
+
cand_comparator: Optional[str],
|
| 433 |
+
) -> bool:
|
| 434 |
+
if cond_comparator == "=0":
|
| 435 |
+
return cand_comparator in {"=0", ">=0"}
|
| 436 |
+
return cond_comparator == ">=0" and cand_comparator == ">=0"
|
| 437 |
+
|
| 438 |
+
|
| 439 |
+
def _is_positive_scalar_multiple(vec_a: csr_matrix, vec_b: csr_matrix, tol: float = 1e-9) -> bool:
|
| 440 |
+
a = vec_a.tocsr(copy=True)
|
| 441 |
+
b = vec_b.tocsr(copy=True)
|
| 442 |
+
a.sum_duplicates()
|
| 443 |
+
b.sum_duplicates()
|
| 444 |
+
a.sort_indices()
|
| 445 |
+
b.sort_indices()
|
| 446 |
+
|
| 447 |
+
if a.nnz == 0 or b.nnz == 0 or a.nnz != b.nnz:
|
| 448 |
+
return False
|
| 449 |
+
if not np.array_equal(a.indices, b.indices):
|
| 450 |
+
return False
|
| 451 |
+
|
| 452 |
+
a_data = np.asarray(a.data, dtype=float)
|
| 453 |
+
b_data = np.asarray(b.data, dtype=float)
|
| 454 |
+
if np.any(np.abs(b_data) <= tol):
|
| 455 |
+
return False
|
| 456 |
+
|
| 457 |
+
ratio = a_data[0] / b_data[0]
|
| 458 |
+
if ratio <= tol:
|
| 459 |
+
return False
|
| 460 |
+
|
| 461 |
+
return np.allclose(a_data, ratio * b_data, rtol=1e-9, atol=tol)
|
| 462 |
+
|
| 463 |
+
|
| 464 |
+
def _is_scalar_multiple(vec_a: csr_matrix, vec_b: csr_matrix) -> bool:
|
| 465 |
+
return _is_positive_scalar_multiple(vec_a, vec_b) or _is_positive_scalar_multiple(vec_a, -vec_b)
|
| 466 |
+
|
| 467 |
+
|
| 468 |
+
def _vectors_are_opposites(vec_a: csr_matrix, vec_b: csr_matrix, tol: float = 1e-9) -> bool:
|
| 469 |
+
return _is_positive_scalar_multiple(vec_a, -vec_b, tol=tol)
|
| 470 |
+
|
| 471 |
+
|
| 472 |
+
def _parse_condition_constraints(
|
| 473 |
+
cond_lines: Optional[List[str]], n: int
|
| 474 |
+
) -> List[Tuple[str, csr_matrix]]:
|
| 475 |
+
"""Parse conditions once, promoting literal and two-sided equalities."""
|
| 476 |
+
equalities: List[csr_matrix] = []
|
| 477 |
+
inequalities: List[csr_matrix] = []
|
| 478 |
+
for raw in cond_lines or []:
|
| 479 |
+
condition = _normalize_constraint_comparator((raw or "").strip(), add_default=True)
|
| 480 |
+
if not condition or condition in {"(none)>=0", "NONE>=0", "NONE=0"}:
|
| 481 |
+
continue
|
| 482 |
+
_, comparator = _split_candidate_comparator(condition)
|
| 483 |
+
try:
|
| 484 |
+
vec = parse_cand_to_vector(condition, n)
|
| 485 |
+
except Exception:
|
| 486 |
+
continue
|
| 487 |
+
if vec is None or vec.nnz == 0:
|
| 488 |
+
continue
|
| 489 |
+
if comparator == "<=0":
|
| 490 |
+
vec = -vec
|
| 491 |
+
comparator = ">=0"
|
| 492 |
+
if comparator == "=0":
|
| 493 |
+
equalities.append(vec)
|
| 494 |
+
elif comparator == ">=0":
|
| 495 |
+
inequalities.append(vec)
|
| 496 |
+
|
| 497 |
+
paired = set()
|
| 498 |
+
for left in range(len(inequalities)):
|
| 499 |
+
if left in paired:
|
| 500 |
+
continue
|
| 501 |
+
for right in range(left + 1, len(inequalities)):
|
| 502 |
+
if right not in paired and _vectors_are_opposites(inequalities[left], inequalities[right]):
|
| 503 |
+
equalities.append(inequalities[left])
|
| 504 |
+
paired.update({left, right})
|
| 505 |
+
break
|
| 506 |
+
|
| 507 |
+
return [("eq", vec) for vec in equalities] + [
|
| 508 |
+
("ge", vec) for idx, vec in enumerate(inequalities) if idx not in paired
|
| 509 |
+
]
|
| 510 |
+
|
| 511 |
+
|
| 512 |
+
def _parse_condition_matrices(
|
| 513 |
+
cond_lines: Optional[List[str]], n: int
|
| 514 |
+
) -> Tuple[Optional[csr_matrix], Optional[csr_matrix]]:
|
| 515 |
+
constraints = _parse_condition_constraints(cond_lines, n)
|
| 516 |
+
eq_vecs = [vec for kind, vec in constraints if kind == "eq"]
|
| 517 |
+
ge_vecs = [vec for kind, vec in constraints if kind == "ge"]
|
| 518 |
+
return (
|
| 519 |
+
vstack(eq_vecs).tocsr() if eq_vecs else None,
|
| 520 |
+
vstack(ge_vecs).tocsr() if ge_vecs else None,
|
| 521 |
+
)
|
| 522 |
+
|
| 523 |
+
|
| 524 |
+
def _is_condition_positive_multiple_step(cand: str, n: Optional[int], cond_lines: Optional[List[str]]) -> bool:
|
| 525 |
+
if n is None or not cond_lines:
|
| 526 |
+
return False
|
| 527 |
+
|
| 528 |
+
cand = _normalize_constraint_comparator(cand)
|
| 529 |
+
_, cand_comparator = _split_candidate_comparator(cand)
|
| 530 |
+
if cand_comparator is None:
|
| 531 |
+
return False
|
| 532 |
+
|
| 533 |
+
try:
|
| 534 |
+
cand_vec = parse_cand_to_vector(cand, n)
|
| 535 |
+
except Exception:
|
| 536 |
+
return False
|
| 537 |
+
if cand_vec is None or cand_vec.nnz == 0:
|
| 538 |
+
return False
|
| 539 |
+
|
| 540 |
+
for kind, cond_vec in _parse_condition_constraints(cond_lines, n):
|
| 541 |
+
cond_comparator = "=0" if kind == "eq" else ">=0"
|
| 542 |
+
if not _condition_comparator_allows_candidate(cond_comparator, cand_comparator):
|
| 543 |
+
continue
|
| 544 |
+
if kind == "eq" and _is_scalar_multiple(cand_vec, cond_vec):
|
| 545 |
+
return True
|
| 546 |
+
if kind == "ge" and _is_positive_scalar_multiple(cand_vec, cond_vec):
|
| 547 |
+
return True
|
| 548 |
+
return False
|
| 549 |
+
|
| 550 |
+
|
| 551 |
+
def _strict_step_format_reason(
|
| 552 |
+
cand: str,
|
| 553 |
+
*,
|
| 554 |
+
n: Optional[int] = None,
|
| 555 |
+
cond_lines: Optional[List[str]] = None,
|
| 556 |
+
) -> Optional[str]:
|
| 557 |
+
"""
|
| 558 |
+
Accept only one positive H(...) or I(...) term with a trailing comparator.
|
| 559 |
+
Also accept multi-term outputs that are a positive scalar multiple of one
|
| 560 |
+
provided condition, e.g. 2*(cond)>=0 or cond>=0.
|
| 561 |
+
"""
|
| 562 |
+
s = _normalize_constraint_comparator((cand or "").strip())
|
| 563 |
+
if not s:
|
| 564 |
+
return "empty"
|
| 565 |
+
|
| 566 |
+
lhs, comparator = _split_candidate_comparator(s)
|
| 567 |
+
if comparator is None:
|
| 568 |
+
return "missing_comparator"
|
| 569 |
+
if comparator == "<=0":
|
| 570 |
+
return "nonpositive_comparator"
|
| 571 |
+
lhs = lhs.strip()
|
| 572 |
+
if not lhs:
|
| 573 |
+
return "empty_lhs"
|
| 574 |
+
|
| 575 |
+
hi_term_count = len(_HI_TERM_RE.findall(lhs))
|
| 576 |
+
if hi_term_count == 1:
|
| 577 |
+
m = _SINGLE_POSITIVE_STEP_RE.fullmatch(lhs)
|
| 578 |
+
if m:
|
| 579 |
+
coeff_str = m.group(1)
|
| 580 |
+
if coeff_str is not None:
|
| 581 |
+
try:
|
| 582 |
+
if float(coeff_str) <= 0:
|
| 583 |
+
return "nonpositive_coeff"
|
| 584 |
+
except ValueError:
|
| 585 |
+
return "bad_coeff"
|
| 586 |
+
if comparator == ">=0":
|
| 587 |
+
return None
|
| 588 |
+
# A Shannon atom is universally nonnegative, not universally zero.
|
| 589 |
+
# Equality output is allowed only when it matches a semantic
|
| 590 |
+
# equality condition in the condition-specific check below.
|
| 591 |
+
base_reason = "unjustified_equality"
|
| 592 |
+
else:
|
| 593 |
+
base_reason = "not_single_hi_term"
|
| 594 |
+
else:
|
| 595 |
+
base_reason = "multiple_terms"
|
| 596 |
+
|
| 597 |
+
if _is_condition_positive_multiple_step(s, n, cond_lines):
|
| 598 |
+
return None
|
| 599 |
+
return base_reason
|
| 600 |
+
|
| 601 |
+
|
| 602 |
+
def _get_cond_matrix_for_run(run_id, n, cond_lines):
|
| 603 |
+
"""
|
| 604 |
+
Worker-local cache.
|
| 605 |
+
Cache is valid only within the same run_id (i.e., one tree_search call).
|
| 606 |
+
"""
|
| 607 |
+
global _COND_CACHE, _LAST_RUN_ID
|
| 608 |
+
|
| 609 |
+
if run_id != _LAST_RUN_ID:
|
| 610 |
+
_COND_CACHE.clear()
|
| 611 |
+
_LAST_RUN_ID = run_id
|
| 612 |
+
|
| 613 |
+
if not cond_lines:
|
| 614 |
+
return None, None
|
| 615 |
+
|
| 616 |
+
key = (n, tuple(cond_lines))
|
| 617 |
+
cached = _COND_CACHE.get(key)
|
| 618 |
+
if cached is not None:
|
| 619 |
+
return cached
|
| 620 |
+
|
| 621 |
+
res = _parse_condition_matrices(cond_lines, n)
|
| 622 |
+
_COND_CACHE[key] = res
|
| 623 |
+
return res
|
| 624 |
+
|
| 625 |
+
def _generate_elemental_generators(n: int, dtype=np.int8) -> List[csr_matrix]:
|
| 626 |
+
"""
|
| 627 |
+
Generate elemental Shannon-type inequalities for n variables:
|
| 628 |
+
1) H(X | others) >= 0
|
| 629 |
+
2) I(X;Y | Z) >= 0 for all pairs (X,Y) and all Z subset of remaining vars.
|
| 630 |
+
Return format matches existing code: [A], where A is csr_matrix (#ineq, 2^n-1).
|
| 631 |
+
|
| 632 |
+
Number of inequalities = n + C(n,2)*2^(n-2)
|
| 633 |
+
Complexity ~ O(n^2 * 2^n), much smaller than O(5^n).
|
| 634 |
+
"""
|
| 635 |
+
if n in _CACHED_GAMMA:
|
| 636 |
+
return _CACHED_GAMMA[n]
|
| 637 |
+
|
| 638 |
+
# Use a different filename to avoid clashing with exhaustive ones
|
| 639 |
+
pkl_path = _MODEL2_DIR / f"elemental_generators_n{n}.pkl"
|
| 640 |
+
if pkl_path.exists():
|
| 641 |
+
try:
|
| 642 |
+
with open(pkl_path, "rb") as f:
|
| 643 |
+
gamma = pickle.load(f) # expects [A]
|
| 644 |
+
_CACHED_GAMMA[n] = gamma
|
| 645 |
+
sys.stderr.write(f"[beam_debug] Loaded precomputed elemental generators for n={n} from {pkl_path}\n")
|
| 646 |
+
sys.stderr.flush()
|
| 647 |
+
return gamma
|
| 648 |
+
except Exception as e:
|
| 649 |
+
sys.stderr.write(f"[beam_debug] Failed to load {pkl_path}: {e}\n")
|
| 650 |
+
sys.stderr.flush()
|
| 651 |
+
|
| 652 |
+
sys.stderr.write(f"[beam_debug] Generating elemental generators for n={n}...\n")
|
| 653 |
+
sys.stderr.flush()
|
| 654 |
+
|
| 655 |
+
A, _b = elemental_inequalities_sparse(n, dtype=dtype) # A is csr
|
| 656 |
+
gamma = [A]
|
| 657 |
+
_CACHED_GAMMA[n] = gamma
|
| 658 |
+
|
| 659 |
+
# Save to disk
|
| 660 |
+
try:
|
| 661 |
+
with open(pkl_path, "wb") as f:
|
| 662 |
+
pickle.dump(gamma, f, protocol=pickle.HIGHEST_PROTOCOL)
|
| 663 |
+
sys.stderr.write(f"[beam_debug] Saved elemental generators for n={n} to {pkl_path}\n")
|
| 664 |
+
sys.stderr.flush()
|
| 665 |
+
except Exception as e:
|
| 666 |
+
sys.stderr.write(f"[beam_debug] Failed to save {pkl_path}: {e}\n")
|
| 667 |
+
sys.stderr.flush()
|
| 668 |
+
|
| 669 |
+
return gamma
|
| 670 |
+
|
| 671 |
+
|
| 672 |
+
def elemental_inequalities_sparse(n: int, dtype=np.int8):
|
| 673 |
+
"""
|
| 674 |
+
Generate basic Shannon-type information measure inequality matrix.
|
| 675 |
+
"""
|
| 676 |
+
m = (1 << n) - 1 # Number of columns (entropy variables)
|
| 677 |
+
rows = []
|
| 678 |
+
cols = []
|
| 679 |
+
data = []
|
| 680 |
+
|
| 681 |
+
def add_coeff(row_id: int, mask: int, val: int):
|
| 682 |
+
cols.append(mask - 1) # 0-based
|
| 683 |
+
rows.append(row_id)
|
| 684 |
+
data.append(val)
|
| 685 |
+
|
| 686 |
+
row_id = 0
|
| 687 |
+
# H(X|Z) >= 0
|
| 688 |
+
for x in range(n):
|
| 689 |
+
# X mask
|
| 690 |
+
Xmask = 1 << x
|
| 691 |
+
# Z mask (everything else)
|
| 692 |
+
Zmask = ((1 << n) - 1) & (~Xmask)
|
| 693 |
+
|
| 694 |
+
# Add H(X|Z) >= 0 -> H(X U Z) - H(Z) >= 0
|
| 695 |
+
add_coeff(row_id, Xmask | Zmask, +1)
|
| 696 |
+
if Zmask != 0:
|
| 697 |
+
add_coeff(row_id, Zmask, -1)
|
| 698 |
+
row_id += 1
|
| 699 |
+
|
| 700 |
+
# I(X;Y|Z) >= 0
|
| 701 |
+
for x in range(n):
|
| 702 |
+
for y in range(x + 1, n):
|
| 703 |
+
zmask = 0
|
| 704 |
+
while zmask < (1 << n):
|
| 705 |
+
if (zmask & (1 << x)) == 0 and (zmask & (1 << y)) == 0:
|
| 706 |
+
XZ = zmask | (1 << x)
|
| 707 |
+
YZ = zmask | (1 << y)
|
| 708 |
+
XYZ = zmask | (1 << x) | (1 << y)
|
| 709 |
+
add_coeff(row_id, XZ, +1)
|
| 710 |
+
add_coeff(row_id, YZ, +1)
|
| 711 |
+
add_coeff(row_id, XYZ, -1)
|
| 712 |
+
if zmask != 0:
|
| 713 |
+
add_coeff(row_id, zmask, -1)
|
| 714 |
+
row_id += 1
|
| 715 |
+
zmask += 1
|
| 716 |
+
if (zmask & (1 << x)) != 0:
|
| 717 |
+
zmask += (1 << x)
|
| 718 |
+
if (zmask & (1 << y)) != 0:
|
| 719 |
+
zmask += (1 << y)
|
| 720 |
+
|
| 721 |
+
# Build sparse matrix A and b
|
| 722 |
+
A = coo_matrix((np.array(data, dtype=dtype), (np.array(rows), np.array(cols))),
|
| 723 |
+
shape=(max(rows) + 1, m)).tocsr()
|
| 724 |
+
b = np.zeros(A.shape[0], dtype=float)
|
| 725 |
+
return A, b
|
| 726 |
+
|
| 727 |
+
|
| 728 |
+
def _beam_debug_print_M(*, depth: int, rollout: int, path_exprs: List[str], residual_vec, action: str) -> None:
|
| 729 |
+
"""
|
| 730 |
+
Debug-print the current residual and path that will be used in the LP solve/enqueue.
|
| 731 |
+
"""
|
| 732 |
+
try:
|
| 733 |
+
m_sz = len(path_exprs) if path_exprs else 0
|
| 734 |
+
residual_str = vector_to_string(residual_vec)
|
| 735 |
+
|
| 736 |
+
sys.stderr.write(
|
| 737 |
+
f"[beam_debug] {action}: depth={depth} rollout={rollout} "
|
| 738 |
+
f"Path_size={m_sz} Residual={residual_str}\n"
|
| 739 |
+
)
|
| 740 |
+
sys.stderr.flush()
|
| 741 |
+
except Exception:
|
| 742 |
+
pass
|
| 743 |
+
|
| 744 |
+
|
| 745 |
+
# def _get_residual_score(vec: csr_matrix) -> float:
|
| 746 |
+
# """
|
| 747 |
+
# Score a residual vector: smaller is better (shorter/simpler expression).
|
| 748 |
+
# Primary: number of terms (nnz).
|
| 749 |
+
# Secondary: sum of absolute coefficients.
|
| 750 |
+
# """
|
| 751 |
+
# if vec is None:
|
| 752 |
+
# return 999999.0
|
| 753 |
+
# # nnz is the number of non-zero joint entropy terms
|
| 754 |
+
# num_terms = vec.nnz
|
| 755 |
+
# # sum of absolute coefficients
|
| 756 |
+
# coeff_sum = np.sum(np.abs(vec.data)) if vec.nnz > 0 else 0
|
| 757 |
+
# return float(num_terms) * 1000.0 + float(coeff_sum)
|
| 758 |
+
|
| 759 |
+
def _get_residual_score(vec: csr_matrix) -> float:
|
| 760 |
+
"""
|
| 761 |
+
Score a residual vector for beam pruning: smaller is better.
|
| 762 |
+
Priority:
|
| 763 |
+
1) fewer terms (sparser expression),
|
| 764 |
+
2) smaller absolute magnitude on negative coefficients,
|
| 765 |
+
3) smaller overall absolute magnitude.
|
| 766 |
+
"""
|
| 767 |
+
if os.environ.get("BEAM_RANDOM_SCORE", "0") == "1":
|
| 768 |
+
return random.random()
|
| 769 |
+
|
| 770 |
+
if vec is None or vec.nnz == 0:
|
| 771 |
+
return 0.0
|
| 772 |
+
|
| 773 |
+
coeffs = np.asarray(vec.data, dtype=np.float64)
|
| 774 |
+
neg_abs_sum = float(np.abs(coeffs[coeffs < 0]).sum()) if np.any(coeffs < 0) else 0.0
|
| 775 |
+
return neg_abs_sum * 1e3
|
| 776 |
+
|
| 777 |
+
|
| 778 |
+
def _select_beam_nodes(nodes, beam_size: int, *, random_prune: bool, rng=None):
|
| 779 |
+
"""Return uniformly sampled or score-ranked kept nodes and their complement."""
|
| 780 |
+
if beam_size <= 0 or len(nodes) <= beam_size:
|
| 781 |
+
return list(nodes), []
|
| 782 |
+
if random_prune:
|
| 783 |
+
rng = rng or random.Random()
|
| 784 |
+
kept_indices = set(rng.sample(range(len(nodes)), beam_size))
|
| 785 |
+
kept = [node for idx, node in enumerate(nodes) if idx in kept_indices]
|
| 786 |
+
pruned = [node for idx, node in enumerate(nodes) if idx not in kept_indices]
|
| 787 |
+
return kept, pruned
|
| 788 |
+
|
| 789 |
+
ranked = sorted(nodes, key=lambda node: _get_residual_score(node.residual_vec))
|
| 790 |
+
return ranked[:beam_size], ranked[beam_size:]
|
| 791 |
+
|
| 792 |
+
|
| 793 |
+
def _is_obviously_negative(vec: csr_matrix) -> bool:
|
| 794 |
+
"""
|
| 795 |
+
Check if all coefficients are non-positive and at least one is strictly negative.
|
| 796 |
+
Returns True if the residual can never reach 0 by subtracting non-negative inequalities.
|
| 797 |
+
"""
|
| 798 |
+
if vec is None or vec.nnz == 0:
|
| 799 |
+
return False
|
| 800 |
+
# Use a small epsilon for float precision
|
| 801 |
+
eps = 1e-9
|
| 802 |
+
# If any coefficient is significantly positive, it's not 'obviously negative'
|
| 803 |
+
if np.any(vec.data > eps):
|
| 804 |
+
return False
|
| 805 |
+
# If we are here, all coefficients are <= eps.
|
| 806 |
+
# Check if at least one is significantly negative.
|
| 807 |
+
return np.any(vec.data < -eps)
|
| 808 |
+
|
| 809 |
+
|
| 810 |
+
def _is_invalid_negative_ineq(
|
| 811 |
+
cand: str,
|
| 812 |
+
*,
|
| 813 |
+
n: Optional[int] = None,
|
| 814 |
+
cond_lines: Optional[List[str]] = None,
|
| 815 |
+
) -> bool:
|
| 816 |
+
"""
|
| 817 |
+
Check if the candidate is a negative term asserted as >= 0.
|
| 818 |
+
In Information Theory, terms like -I(X;Y) >= 0 are only valid if they are actually 0.
|
| 819 |
+
The training data uses -I(...) = 0.
|
| 820 |
+
"""
|
| 821 |
+
s = cand.strip()
|
| 822 |
+
if ">=" in s and s.lstrip().startswith("-"):
|
| 823 |
+
if n is not None and cond_lines and _is_condition_positive_multiple_step(s, n, cond_lines):
|
| 824 |
+
return False
|
| 825 |
+
return True
|
| 826 |
+
return False
|
| 827 |
+
|
| 828 |
+
|
| 829 |
+
def _is_in_row_space(vec: csr_matrix, matrix: Optional[csr_matrix]) -> bool:
|
| 830 |
+
"""
|
| 831 |
+
Check if vec is a linear combination of the rows of matrix.
|
| 832 |
+
Uses linprog to solve matrix.T @ y = vec.T
|
| 833 |
+
"""
|
| 834 |
+
if vec.nnz == 0:
|
| 835 |
+
return True
|
| 836 |
+
if matrix is None or matrix.shape[0] == 0:
|
| 837 |
+
return False
|
| 838 |
+
|
| 839 |
+
try:
|
| 840 |
+
# Solve A_eq @ y = b_eq
|
| 841 |
+
# where A_eq is matrix.T and b_eq is vec.T
|
| 842 |
+
A_eq = matrix.toarray().T
|
| 843 |
+
b_eq = vec.toarray().flatten()
|
| 844 |
+
c = np.zeros(A_eq.shape[1])
|
| 845 |
+
# Bayesian coefficients can be negative, so bounds=(None, None)
|
| 846 |
+
res = linprog(c, A_eq=A_eq, b_eq=b_eq, bounds=(None, None), method='highs')
|
| 847 |
+
return res.success
|
| 848 |
+
except Exception:
|
| 849 |
+
return False
|
| 850 |
+
|
| 851 |
+
|
| 852 |
+
def _get_involved_vars(vec: csr_matrix) -> set:
|
| 853 |
+
involved_vars = set()
|
| 854 |
+
if vec is None:
|
| 855 |
+
return involved_vars
|
| 856 |
+
for idx in vec.indices:
|
| 857 |
+
subset_idx = idx + 1
|
| 858 |
+
v = 1
|
| 859 |
+
temp = subset_idx
|
| 860 |
+
while temp > 0:
|
| 861 |
+
if temp & 1:
|
| 862 |
+
involved_vars.add(v)
|
| 863 |
+
temp >>= 1
|
| 864 |
+
v += 1
|
| 865 |
+
return involved_vars
|
| 866 |
+
|
| 867 |
+
|
| 868 |
+
def _reduce_vector(vec: csr_matrix, mapping: Dict[int, int], num_vars: int) -> csr_matrix:
|
| 869 |
+
new_indices = []
|
| 870 |
+
for idx in vec.indices:
|
| 871 |
+
subset_idx = idx + 1
|
| 872 |
+
new_subset_idx = 0
|
| 873 |
+
for old_v, new_v in mapping.items():
|
| 874 |
+
if subset_idx & (1 << (old_v - 1)):
|
| 875 |
+
new_subset_idx |= (1 << (new_v - 1))
|
| 876 |
+
new_indices.append(new_subset_idx - 1)
|
| 877 |
+
new_dim = (1 << num_vars) - 1
|
| 878 |
+
return csr_matrix((vec.data, new_indices, [0, len(vec.data)]), shape=(1, new_dim))
|
| 879 |
+
|
| 880 |
+
|
| 881 |
+
def _check_shannon_status(vec: csr_matrix, n_sample: int, Gamma: Optional[List[csr_matrix]] = None,
|
| 882 |
+
cond_eq_matrix: Optional[csr_matrix] = None,
|
| 883 |
+
cond_ge_matrix: Optional[csr_matrix] = None,
|
| 884 |
+
max_vars_for_lp: int = 0,
|
| 885 |
+
debug: bool = False) -> int:
|
| 886 |
+
"""
|
| 887 |
+
Check whether the residual has a direct certificate or is flagged by the
|
| 888 |
+
numerical LP false-pruning heuristic.
|
| 889 |
+
|
| 890 |
+
In exact arithmetic, an unbounded minimization supplies a counterexample
|
| 891 |
+
ray. This implementation trusts floating-point HiGHS status and does not
|
| 892 |
+
independently verify that ray, so the result can affect search completeness
|
| 893 |
+
but is never used to accept a proof. LP optimality likewise does not
|
| 894 |
+
terminate search because this routine does not reconstruct an explicit
|
| 895 |
+
conic certificate.
|
| 896 |
+
Returns:
|
| 897 |
+
1 if directly proved (nearly zero or all coefficients nonnegative)
|
| 898 |
+
0 if unknown, including an LP-true result without an explicit certificate
|
| 899 |
+
-1 if impossible (False/Prune) - only if n_sample <= max_vars_for_lp
|
| 900 |
+
"""
|
| 901 |
+
if vec is None:
|
| 902 |
+
return 0
|
| 903 |
+
|
| 904 |
+
eps = 1e-9
|
| 905 |
+
|
| 906 |
+
# 1. Quick checks (Proved)
|
| 907 |
+
if vec.nnz > 0 and np.all(vec.data >= -eps):
|
| 908 |
+
return 1
|
| 909 |
+
if vec.nnz == 0 or np.allclose(vec.toarray(), 0, atol=eps):
|
| 910 |
+
return 1
|
| 911 |
+
|
| 912 |
+
if max_vars_for_lp > 0:
|
| 913 |
+
involved_res = _get_involved_vars(vec)
|
| 914 |
+
|
| 915 |
+
# Determine if we can do a full check (with conditions) or only a partial check (proved-only)
|
| 916 |
+
can_prune = (n_sample <= max_vars_for_lp)
|
| 917 |
+
|
| 918 |
+
# involved_vars for the LP context
|
| 919 |
+
if can_prune:
|
| 920 |
+
involved = set(range(1, n_sample + 1))
|
| 921 |
+
else:
|
| 922 |
+
# For n_sample > max_vars_for_lp (e.g. 6), we can only prove it if the residual variables fit in max_vars_for_lp
|
| 923 |
+
involved = involved_res
|
| 924 |
+
if len(involved) > max_vars_for_lp:
|
| 925 |
+
return 0
|
| 926 |
+
# If we are here, we are doing a "no-conditions" check to see if it's Shannon-true
|
| 927 |
+
cond_eq_matrix = None
|
| 928 |
+
cond_ge_matrix = None
|
| 929 |
+
|
| 930 |
+
num_vars = len(involved)
|
| 931 |
+
if num_vars == 0: return 0
|
| 932 |
+
|
| 933 |
+
# Select or generate Gamma
|
| 934 |
+
global _CACHED_GAMMA
|
| 935 |
+
if num_vars not in _CACHED_GAMMA:
|
| 936 |
+
_generate_elemental_generators(num_vars)
|
| 937 |
+
gamma_data = _CACHED_GAMMA[num_vars]
|
| 938 |
+
# Handle cached tuple format: ([A], names)
|
| 939 |
+
current_gamma = gamma_data[0] if isinstance(gamma_data, tuple) else gamma_data
|
| 940 |
+
|
| 941 |
+
if debug:
|
| 942 |
+
try:
|
| 943 |
+
type_str = "full" if can_prune else "residual-only"
|
| 944 |
+
sys.stderr.write(
|
| 945 |
+
f"[beam_debug] LP_check_start: n={num_vars} ({type_str}) vec={vector_to_string(vec)}\n")
|
| 946 |
+
sys.stderr.flush()
|
| 947 |
+
except:
|
| 948 |
+
pass
|
| 949 |
+
|
| 950 |
+
sorted_vars = sorted(list(involved))
|
| 951 |
+
mapping = {old: new for new, old in enumerate(sorted_vars, 1)}
|
| 952 |
+
vec_small = _reduce_vector(vec, mapping, num_vars)
|
| 953 |
+
|
| 954 |
+
|
| 955 |
+
try:
|
| 956 |
+
G = vstack(current_gamma) # shape: (num_gen, m)
|
| 957 |
+
c = vec_small.toarray().ravel() # shape: (m,)
|
| 958 |
+
|
| 959 |
+
# A_ub @ y <= b_ub
|
| 960 |
+
# Shannon inequalities: G @ y >= 0 => -G @ y <= 0
|
| 961 |
+
A_ub_list = [-G]
|
| 962 |
+
|
| 963 |
+
if cond_ge_matrix is not None and cond_ge_matrix.shape[0] > 0:
|
| 964 |
+
ge_small_list = []
|
| 965 |
+
for i in range(cond_ge_matrix.shape[0]):
|
| 966 |
+
ge_reduced = _reduce_vector(cond_ge_matrix[i], mapping, num_vars)
|
| 967 |
+
ge_small_list.append(-ge_reduced) # v @ y >= 0 => -v @ y <= 0
|
| 968 |
+
A_ub_list.append(vstack(ge_small_list))
|
| 969 |
+
|
| 970 |
+
A_ub = vstack(A_ub_list)
|
| 971 |
+
b_ub = np.zeros(A_ub.shape[0], dtype=float)
|
| 972 |
+
|
| 973 |
+
A_eq = None
|
| 974 |
+
b_eq = None
|
| 975 |
+
if cond_eq_matrix is not None and cond_eq_matrix.shape[0] > 0:
|
| 976 |
+
cond_small_list = []
|
| 977 |
+
for i in range(cond_eq_matrix.shape[0]):
|
| 978 |
+
c_reduced = _reduce_vector(cond_eq_matrix[i], mapping, num_vars) # (1, m)
|
| 979 |
+
cond_small_list.append(c_reduced)
|
| 980 |
+
A_eq = vstack(cond_small_list) # shape: (num_cond, m)
|
| 981 |
+
b_eq = np.zeros(A_eq.shape[0], dtype=float)
|
| 982 |
+
|
| 983 |
+
# y is free (IMPORTANT)
|
| 984 |
+
bounds = [(None, None)] * G.shape[1]
|
| 985 |
+
|
| 986 |
+
res = linprog(c, A_ub=A_ub, b_ub=b_ub, A_eq=A_eq, b_eq=b_eq,
|
| 987 |
+
bounds=bounds, method="highs")
|
| 988 |
+
|
| 989 |
+
if res.status == 3:
|
| 990 |
+
# Numerically unbounded. In exact arithmetic this means the
|
| 991 |
+
# residual is false over the constrained Shannon outer cone.
|
| 992 |
+
if debug:
|
| 993 |
+
sys.stderr.write("[beam_debug] lp_dual: numerically UNBOUNDED => false-prune\n")
|
| 994 |
+
sys.stderr.flush()
|
| 995 |
+
return -1 if can_prune else 0
|
| 996 |
+
|
| 997 |
+
if res.status == 0 and res.success:
|
| 998 |
+
# optimal => should be ~0 => TRUE
|
| 999 |
+
if debug:
|
| 1000 |
+
sys.stderr.write(f"[beam_debug] lp_dual: OPTIMAL fun={res.fun}\n")
|
| 1001 |
+
sys.stderr.flush()
|
| 1002 |
+
|
| 1003 |
+
# res.fun might be -1e-12 numerically, treat as 0
|
| 1004 |
+
if res.fun < -eps:
|
| 1005 |
+
# Theoretically this means it should be unbounded but solver didn't report it
|
| 1006 |
+
return -1 if can_prune else 0
|
| 1007 |
+
|
| 1008 |
+
return 0
|
| 1009 |
+
|
| 1010 |
+
# Other cases: iteration limit/numerical issues (unsafe to prune)
|
| 1011 |
+
if debug:
|
| 1012 |
+
sys.stderr.write(f"[beam_debug] lp_dual: status={res.status} msg={res.message}\n")
|
| 1013 |
+
sys.stderr.flush()
|
| 1014 |
+
return 0
|
| 1015 |
+
|
| 1016 |
+
except Exception as e:
|
| 1017 |
+
if debug:
|
| 1018 |
+
sys.stderr.write(f"[beam_debug] LP error: {e}\n")
|
| 1019 |
+
return 0
|
| 1020 |
+
|
| 1021 |
+
return 0
|
| 1022 |
+
|
| 1023 |
+
|
| 1024 |
+
def _is_positive_polymatroid(vec: csr_matrix, n_sample: int, Gamma: Optional[List[csr_matrix]] = None,
|
| 1025 |
+
cond_eq_matrix: Optional[csr_matrix] = None,
|
| 1026 |
+
cond_ge_matrix: Optional[csr_matrix] = None,
|
| 1027 |
+
max_vars_for_lp: int = 0, debug: bool = False) -> bool:
|
| 1028 |
+
"""
|
| 1029 |
+
Check if the residual is a positive polymatroid (Shannon inequality).
|
| 1030 |
+
Returns True if vec >= 0 is already proved or follows from Gamma.
|
| 1031 |
+
"""
|
| 1032 |
+
return _check_shannon_status(vec, n_sample, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug) == 1
|
| 1033 |
+
|
| 1034 |
+
|
| 1035 |
+
def _normalize_to_braces(s: str) -> str:
|
| 1036 |
+
"""
|
| 1037 |
+
Ensure all H(...) and I(...) terms use X_{...} notation.
|
| 1038 |
+
Example: H(X_1,2) -> H(X_{1,2}), I(X_1;X_2) -> I(X_{1};X_{2}).
|
| 1039 |
+
"""
|
| 1040 |
+
def repl_X(m):
|
| 1041 |
+
inside = m.group(1)
|
| 1042 |
+
if inside.startswith('{') and inside.endswith('}'):
|
| 1043 |
+
return m.group(0)
|
| 1044 |
+
# Split by comma to handle X_1,2
|
| 1045 |
+
parts = [p.strip() for p in inside.split(',') if p.strip()]
|
| 1046 |
+
return "X_{" + ",".join(parts) + "}"
|
| 1047 |
+
|
| 1048 |
+
# Replace all occurrences of X_ followed by digits/commas OR X_{...}
|
| 1049 |
+
return re.sub(r'X_(\{?[0-9,]+\}?)', repl_X, s)
|
| 1050 |
+
|
| 1051 |
+
|
| 1052 |
+
def parse_cand_to_vector(cand: str, n: int) -> csr_matrix:
|
| 1053 |
+
"""
|
| 1054 |
+
Robustly parse an inequality candidate string from the LLM into a vector.
|
| 1055 |
+
Handles coefficients (including 8*H(...) or 8H(...)), signs, and conditional terms.
|
| 1056 |
+
"""
|
| 1057 |
+
# 1. Strip comparator and all whitespace/asterisks
|
| 1058 |
+
s = _strip_comparator(cand).replace(" ", "").replace("*", "")
|
| 1059 |
+
|
| 1060 |
+
# 2. Normalize braces to ensure consistency (X_1 -> X_{1})
|
| 1061 |
+
s = _normalize_to_braces(s)
|
| 1062 |
+
|
| 1063 |
+
def _parse_linear_expr(expr: str) -> csr_matrix:
|
| 1064 |
+
# Try string_to_vector first (handles linear combinations of joint entropies)
|
| 1065 |
+
try:
|
| 1066 |
+
return string_to_vector(expr, n)
|
| 1067 |
+
except Exception:
|
| 1068 |
+
pass
|
| 1069 |
+
|
| 1070 |
+
# Handle single term with potential coefficient for polym_ineq_to_vector
|
| 1071 |
+
# Pattern: optional sign and coefficient, followed by a term starting with H( or I(
|
| 1072 |
+
m = re.match(r'^([+-]?\d*\.?\d*)([HI]\(.*\))$', expr)
|
| 1073 |
+
if m:
|
| 1074 |
+
coeff_str, term_expr = m.groups()
|
| 1075 |
+
coeff = 1.0
|
| 1076 |
+
if coeff_str == "-":
|
| 1077 |
+
coeff = -1.0
|
| 1078 |
+
elif coeff_str and coeff_str != "+":
|
| 1079 |
+
try:
|
| 1080 |
+
coeff = float(coeff_str)
|
| 1081 |
+
except ValueError:
|
| 1082 |
+
coeff = 1.0
|
| 1083 |
+
try:
|
| 1084 |
+
# polym_ineq_to_vector handles H(A|B), I(A;B), I(A;B|C)
|
| 1085 |
+
return coeff * polym_ineq_to_vector(term_expr, n)
|
| 1086 |
+
except Exception:
|
| 1087 |
+
pass
|
| 1088 |
+
|
| 1089 |
+
# Final fallback: try polym_ineq_to_vector on the whole thing
|
| 1090 |
+
return polym_ineq_to_vector(expr, n)
|
| 1091 |
+
|
| 1092 |
+
# 3. Handle leading scalar times parenthesized expression, e.g. 2*(...), -3(...)
|
| 1093 |
+
m = re.match(r'^([+-]?\d*\.?\d*)\((.*)\)$', s)
|
| 1094 |
+
if m:
|
| 1095 |
+
coeff_str, inner_expr = m.groups()
|
| 1096 |
+
coeff = 1.0
|
| 1097 |
+
if coeff_str == "-":
|
| 1098 |
+
coeff = -1.0
|
| 1099 |
+
elif coeff_str and coeff_str != "+":
|
| 1100 |
+
try:
|
| 1101 |
+
coeff = float(coeff_str)
|
| 1102 |
+
except ValueError:
|
| 1103 |
+
coeff = 1.0
|
| 1104 |
+
inner_vec = _parse_linear_expr(inner_expr)
|
| 1105 |
+
return coeff * inner_vec
|
| 1106 |
+
|
| 1107 |
+
# 4. Normal linear expression path
|
| 1108 |
+
return _parse_linear_expr(s)
|
| 1109 |
+
|
| 1110 |
+
|
| 1111 |
+
def _extract_var_indices(expr: str) -> List[int]:
|
| 1112 |
+
"""
|
| 1113 |
+
Extract all variable indices appearing as X_{...} or X_... in an expression.
|
| 1114 |
+
"""
|
| 1115 |
+
idxs: List[int] = []
|
| 1116 |
+
if not expr:
|
| 1117 |
+
return idxs
|
| 1118 |
+
# X_{1,2,3}
|
| 1119 |
+
for inside in re.findall(r"X_\{([^}]+)\}", expr):
|
| 1120 |
+
parts = [p.strip() for p in inside.split(",") if p.strip()]
|
| 1121 |
+
for p in parts:
|
| 1122 |
+
if p.isdigit():
|
| 1123 |
+
idxs.append(int(p))
|
| 1124 |
+
# X_1 (avoid double counting those already in braces; ok if duplicates)
|
| 1125 |
+
for m in re.findall(r"X_(\d+)\b", expr):
|
| 1126 |
+
try:
|
| 1127 |
+
idxs.append(int(m))
|
| 1128 |
+
except Exception:
|
| 1129 |
+
pass
|
| 1130 |
+
return idxs
|
| 1131 |
+
|
| 1132 |
+
|
| 1133 |
+
def _strip_comparator(s: str) -> str:
|
| 1134 |
+
# Keep only LHS if input contains "...>=0", "...>0", "...<=0", or "...=0"
|
| 1135 |
+
s = s.strip()
|
| 1136 |
+
for token in (">=0", ">0", "<=0", "=0"):
|
| 1137 |
+
if token in s:
|
| 1138 |
+
s = s.split(token, 1)[0]
|
| 1139 |
+
# Also handle spaced variants like ">= 0"
|
| 1140 |
+
s = re.sub(r'(>=|>|<=|=)\s*0\s*$', '', s).strip()
|
| 1141 |
+
return s
|
| 1142 |
+
|
| 1143 |
+
|
| 1144 |
+
def _build_llm_prompt(
|
| 1145 |
+
*,
|
| 1146 |
+
n: int,
|
| 1147 |
+
min_idx: int,
|
| 1148 |
+
max_idx: int,
|
| 1149 |
+
target_inequality_normalized: str,
|
| 1150 |
+
cond_lines: List[str],
|
| 1151 |
+
m_exprs: List[str],
|
| 1152 |
+
idx_to_letter: Optional[Dict[int, str]] = None,
|
| 1153 |
+
) -> str:
|
| 1154 |
+
"""
|
| 1155 |
+
Build prompt matching the prepared DPO training data format.
|
| 1156 |
+
"""
|
| 1157 |
+
# 1. Format Target (ensure comparator)
|
| 1158 |
+
target = target_inequality_normalized.strip()
|
| 1159 |
+
if not any(op in target for op in [">=0", ">0", "<=0", "=0"]):
|
| 1160 |
+
target += ">=0"
|
| 1161 |
+
if _use_alpha_prompt():
|
| 1162 |
+
target = _convert_old_to_alpha(target, idx_to_letter)
|
| 1163 |
+
|
| 1164 |
+
# 2. Format Conditions (use NONE when empty)
|
| 1165 |
+
valid_conds = []
|
| 1166 |
+
for c in (cond_lines or []):
|
| 1167 |
+
c = c.strip()
|
| 1168 |
+
if not c:
|
| 1169 |
+
continue
|
| 1170 |
+
if c in {"(none)", "NONE", "NONE=0"}:
|
| 1171 |
+
continue
|
| 1172 |
+
valid_conds.append(_convert_old_to_alpha(c, idx_to_letter) if _use_alpha_prompt() else c)
|
| 1173 |
+
|
| 1174 |
+
cond_str = "NONE" if not valid_conds else "\n".join(valid_conds)
|
| 1175 |
+
|
| 1176 |
+
# 3. Assemble prompt in DPO 'input' format
|
| 1177 |
+
prompt = f"TARGET:\n{target}\n\n"
|
| 1178 |
+
prompt += f"CONDITIONS:\n{cond_str}\n\n"
|
| 1179 |
+
|
| 1180 |
+
return prompt
|
| 1181 |
+
|
| 1182 |
+
|
| 1183 |
+
def _gpu_worker_init(adapter_path: Optional[str], use_dummy: bool, gpu_id: int, max_vars_for_lp: int = 0, Gamma: Optional[List[csr_matrix]] = None):
|
| 1184 |
+
"""
|
| 1185 |
+
One worker process binds to one GPU and loads the model once.
|
| 1186 |
+
"""
|
| 1187 |
+
# Must be set before importing torch/transformers for correct device visibility.
|
| 1188 |
+
os.environ["CUDA_VISIBLE_DEVICES"] = str(gpu_id)
|
| 1189 |
+
os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
|
| 1190 |
+
|
| 1191 |
+
global _GPU_ADAPTER_PATH, _GPU_USE_DUMMY, _GPU_READY, _CACHED_GAMMA
|
| 1192 |
+
_GPU_ADAPTER_PATH = os.path.abspath(adapter_path) if adapter_path else None
|
| 1193 |
+
_GPU_USE_DUMMY = bool(use_dummy)
|
| 1194 |
+
|
| 1195 |
+
if not _GPU_USE_DUMMY:
|
| 1196 |
+
# Warm model load once per worker process.
|
| 1197 |
+
try:
|
| 1198 |
+
from llm_api_compat import _load_local_model
|
| 1199 |
+
except ModuleNotFoundError:
|
| 1200 |
+
from beam_search_model2.llm_api_compat import _load_local_model
|
| 1201 |
+
# adapter_path may be None to use the baseline model (no PEFT adapter).
|
| 1202 |
+
_load_local_model(adapter_path=_GPU_ADAPTER_PATH, device_map="auto")
|
| 1203 |
+
|
| 1204 |
+
if max_vars_for_lp > 0:
|
| 1205 |
+
if Gamma is not None:
|
| 1206 |
+
# Use the Gamma matrix passed from the main process
|
| 1207 |
+
_CACHED_GAMMA[max_vars_for_lp] = Gamma
|
| 1208 |
+
sys.stderr.write(f"[beam_debug] GPU worker {gpu_id} using Gamma matrix passed from main process (max_vars_for_lp={max_vars_for_lp}).\n")
|
| 1209 |
+
sys.stderr.flush()
|
| 1210 |
+
else:
|
| 1211 |
+
# Pre-generate Shannon inequalities once per worker
|
| 1212 |
+
sys.stderr.write(f"[beam_debug] GPU worker {gpu_id} pre-generating Shannon generators for max_vars_for_lp={max_vars_for_lp}...\n")
|
| 1213 |
+
sys.stderr.flush()
|
| 1214 |
+
gamma_list = _generate_elemental_generators(max_vars_for_lp)
|
| 1215 |
+
shape_str = f" Shape: {gamma_list[0].shape}" if gamma_list else ""
|
| 1216 |
+
sys.stderr.write(f"[beam_debug] GPU worker {gpu_id} done pre-generating Shannon generators.{shape_str}\n")
|
| 1217 |
+
sys.stderr.flush()
|
| 1218 |
+
|
| 1219 |
+
_GPU_READY = True
|
| 1220 |
+
|
| 1221 |
+
|
| 1222 |
+
def _expand_node_payload(payload: Dict) -> Dict:
|
| 1223 |
+
"""
|
| 1224 |
+
Worker-side expansion: starting from residual_vec, propose k inequalities and subtract them.
|
| 1225 |
+
Returns a dict with a list of potential child states.
|
| 1226 |
+
Note: All heavy modules (numpy, scipy, llm_api_interface) are imported at the top level
|
| 1227 |
+
of this file and will be already available in the worker process's memory.
|
| 1228 |
+
"""
|
| 1229 |
+
n = int(payload["n"])
|
| 1230 |
+
min_idx = int(payload["min_idx"])
|
| 1231 |
+
max_idx = int(payload["max_idx"])
|
| 1232 |
+
run_id = payload.get("run_id")
|
| 1233 |
+
cond_lines = payload["cond_lines"]
|
| 1234 |
+
base_path_exprs = list(payload.get("path_exprs") or [])
|
| 1235 |
+
base_residual_vec = payload.get("residual_vec", None)
|
| 1236 |
+
k_steps = max(0, int(payload.get("k_per_call", 1)))
|
| 1237 |
+
llm_mode = (payload.get("llm_mode") or "sequential").strip().lower()
|
| 1238 |
+
max_tries_per_step = int(payload.get("max_tries_per_step", 3))
|
| 1239 |
+
|
| 1240 |
+
adapter_path = payload.get("adapter_path", None)
|
| 1241 |
+
use_dummy = bool(payload.get("use_dummy", False))
|
| 1242 |
+
Gamma = payload.get("Gamma", None)
|
| 1243 |
+
max_vars_for_lp = int(payload.get("max_vars_for_lp", 0))
|
| 1244 |
+
idx_to_letter = payload.get("idx_to_letter", None)
|
| 1245 |
+
letter_to_idx = payload.get("letter_to_idx", None)
|
| 1246 |
+
|
| 1247 |
+
if Gamma is None and max_vars_for_lp > 0:
|
| 1248 |
+
# Use worker-local cached Gamma (already initialized in _gpu_worker_init)
|
| 1249 |
+
Gamma = _CACHED_GAMMA.get(max_vars_for_lp, None)
|
| 1250 |
+
|
| 1251 |
+
debug = os.environ.get("BEAM_DEBUG", "0") == "1"
|
| 1252 |
+
|
| 1253 |
+
log_wrong_pruned = bool(payload.get("log_wrong_pruned", False))
|
| 1254 |
+
wrong_data = [] # List of [input_str, output_str]
|
| 1255 |
+
|
| 1256 |
+
# Parse conditions into matrices for row-space and LP checks
|
| 1257 |
+
# Worker-local cache (_COND_CACHE) ensures we don't re-parse for every node.
|
| 1258 |
+
cond_eq_matrix, cond_ge_matrix = _get_cond_matrix_for_run(run_id, n, cond_lines)
|
| 1259 |
+
# if cond_lines and cond_lines != ["NONE"]:
|
| 1260 |
+
# try:
|
| 1261 |
+
# eq_vecs = []
|
| 1262 |
+
# ge_vecs = []
|
| 1263 |
+
# for cl in cond_lines:
|
| 1264 |
+
# try:
|
| 1265 |
+
# # Note: cand might already have >=0 or =0 from tree_search's cond_lines
|
| 1266 |
+
# # but parse_cand_to_vector calls _strip_comparator internally.
|
| 1267 |
+
# v = parse_cand_to_vector(cl, n)
|
| 1268 |
+
# if v.nnz == 0: continue
|
| 1269 |
+
# if ">=0" in cl:
|
| 1270 |
+
# ge_vecs.append(v)
|
| 1271 |
+
# elif "=0" in cl:
|
| 1272 |
+
# eq_vecs.append(v)
|
| 1273 |
+
# else:
|
| 1274 |
+
# # Fallback if no comparator found in string (shouldn't happen with latest logic)
|
| 1275 |
+
# eq_vecs.append(v)
|
| 1276 |
+
# except: continue
|
| 1277 |
+
# if eq_vecs:
|
| 1278 |
+
# cond_eq_matrix = vstack(eq_vecs)
|
| 1279 |
+
# if ge_vecs:
|
| 1280 |
+
# cond_ge_matrix = vstack(ge_vecs)
|
| 1281 |
+
# except:
|
| 1282 |
+
# pass
|
| 1283 |
+
|
| 1284 |
+
# Check status: 1=proved, -1=impossible, 0=unknown
|
| 1285 |
+
status = _check_shannon_status(base_residual_vec, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug=debug)
|
| 1286 |
+
if debug:
|
| 1287 |
+
sys.stderr.write(f"[beam_debug] worker_root_status: {status}\n")
|
| 1288 |
+
sys.stderr.flush()
|
| 1289 |
+
|
| 1290 |
+
if status == 1:
|
| 1291 |
+
return {
|
| 1292 |
+
"ok": True,
|
| 1293 |
+
"results": [{
|
| 1294 |
+
"proved": True,
|
| 1295 |
+
"path_exprs": base_path_exprs,
|
| 1296 |
+
"residual_vec": base_residual_vec,
|
| 1297 |
+
"new_ineq": None
|
| 1298 |
+
}],
|
| 1299 |
+
"parse_fail": 0,
|
| 1300 |
+
"out_of_range": 0,
|
| 1301 |
+
}
|
| 1302 |
+
# User requested: No pruning at root node. Let search proceed even if status is -1.
|
| 1303 |
+
|
| 1304 |
+
results = []
|
| 1305 |
+
pruned_results = []
|
| 1306 |
+
parse_fail = 0
|
| 1307 |
+
out_of_range = 0
|
| 1308 |
+
llm_usage = _empty_llm_usage()
|
| 1309 |
+
|
| 1310 |
+
if llm_mode == "sequential":
|
| 1311 |
+
# Sequential: Subtract k things one after another. Return ONE branch.
|
| 1312 |
+
current_residual_vec = base_residual_vec
|
| 1313 |
+
current_path_exprs = list(base_path_exprs)
|
| 1314 |
+
subtracted_any = False
|
| 1315 |
+
|
| 1316 |
+
for _step in range(k_steps):
|
| 1317 |
+
got_one = False
|
| 1318 |
+
current_target_norm = vector_to_string(current_residual_vec)
|
| 1319 |
+
|
| 1320 |
+
for _try in range(max_tries_per_step):
|
| 1321 |
+
prompt = _build_llm_prompt(
|
| 1322 |
+
n=n, min_idx=min_idx, max_idx=max_idx,
|
| 1323 |
+
target_inequality_normalized=current_target_norm,
|
| 1324 |
+
cond_lines=cond_lines, m_exprs=current_path_exprs,
|
| 1325 |
+
idx_to_letter=idx_to_letter,
|
| 1326 |
+
)
|
| 1327 |
+
cand_list = LLM_propose_inequalities(
|
| 1328 |
+
prompt=prompt, k=1, adapter_path=adapter_path, use_dummy=use_dummy,
|
| 1329 |
+
)
|
| 1330 |
+
_merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
|
| 1331 |
+
cand_raw = cand_list[0] if cand_list else ""
|
| 1332 |
+
cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
|
| 1333 |
+
cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
|
| 1334 |
+
format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
|
| 1335 |
+
if not cand or cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
|
| 1336 |
+
if log_wrong_pruned and cand_raw:
|
| 1337 |
+
wrong_data.append([prompt, cand_raw])
|
| 1338 |
+
if debug and cand_raw:
|
| 1339 |
+
sys.stderr.write(
|
| 1340 |
+
f"[beam_debug] invalid_or_wrong_cand: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
|
| 1341 |
+
)
|
| 1342 |
+
sys.stderr.flush()
|
| 1343 |
+
if format_reason is not None:
|
| 1344 |
+
llm_usage["format_fail_total"] += 1
|
| 1345 |
+
pruned_results.append({"is_pruned": True, "reason": "invalid_or_wrong_cand", "path_exprs": current_path_exprs + [cand]})
|
| 1346 |
+
continue
|
| 1347 |
+
|
| 1348 |
+
cand_indices = _extract_var_indices(cand)
|
| 1349 |
+
if cand_indices and (min(cand_indices) < min_idx or max(cand_indices) > max_idx):
|
| 1350 |
+
if log_wrong_pruned:
|
| 1351 |
+
wrong_data.append([prompt, cand_raw])
|
| 1352 |
+
out_of_range += 1
|
| 1353 |
+
pruned_results.append({"is_pruned": True, "reason": "out_of_range", "path_exprs": current_path_exprs + [cand]})
|
| 1354 |
+
continue
|
| 1355 |
+
try:
|
| 1356 |
+
vec = parse_cand_to_vector(cand, n)
|
| 1357 |
+
except (ValueError, Exception):
|
| 1358 |
+
if log_wrong_pruned:
|
| 1359 |
+
wrong_data.append([prompt, cand_raw])
|
| 1360 |
+
parse_fail += 1
|
| 1361 |
+
pruned_results.append({"is_pruned": True, "reason": "parse_fail", "path_exprs": current_path_exprs + [cand]})
|
| 1362 |
+
continue
|
| 1363 |
+
|
| 1364 |
+
# Validation for Bayesian equalities
|
| 1365 |
+
if re.search(r'(?<![<>])=\s*0', cand):
|
| 1366 |
+
if not _is_in_row_space(vec, cond_eq_matrix):
|
| 1367 |
+
if log_wrong_pruned:
|
| 1368 |
+
wrong_data.append([prompt, cand_raw])
|
| 1369 |
+
if debug:
|
| 1370 |
+
sys.stderr.write(f"[beam_debug] hallucinated_equality: step={cand_raw} not in row-space of conditions\n")
|
| 1371 |
+
sys.stderr.flush()
|
| 1372 |
+
pruned_results.append({"is_pruned": True, "reason": "hallucinated_equality", "path_exprs": current_path_exprs + [cand]})
|
| 1373 |
+
continue # Hallucinated equality
|
| 1374 |
+
else:
|
| 1375 |
+
if debug:
|
| 1376 |
+
sys.stderr.write(f"[beam_debug] equality_verified: step={cand_raw} is in row-space\n")
|
| 1377 |
+
sys.stderr.flush()
|
| 1378 |
+
|
| 1379 |
+
new_residual = current_residual_vec - vec
|
| 1380 |
+
# Check status: 1=proved, -1=impossible (false)
|
| 1381 |
+
status = _check_shannon_status(new_residual, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug=debug)
|
| 1382 |
+
|
| 1383 |
+
# Check if obviously negative (simple check)
|
| 1384 |
+
if _is_obviously_negative(new_residual):
|
| 1385 |
+
if log_wrong_pruned:
|
| 1386 |
+
wrong_data.append([prompt, cand_raw])
|
| 1387 |
+
if debug:
|
| 1388 |
+
res_str = vector_to_string(new_residual)
|
| 1389 |
+
sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
|
| 1390 |
+
sys.stderr.flush()
|
| 1391 |
+
pruned_results.append({"is_pruned": True, "reason": "obviously_negative", "residual_vec": new_residual, "path_exprs": current_path_exprs + [cand]})
|
| 1392 |
+
got_one = False # Don't continue this branch
|
| 1393 |
+
break
|
| 1394 |
+
|
| 1395 |
+
# Prune if impossible via LP
|
| 1396 |
+
if status == -1:
|
| 1397 |
+
if log_wrong_pruned:
|
| 1398 |
+
wrong_data.append([prompt, cand_raw])
|
| 1399 |
+
if debug:
|
| 1400 |
+
res_str = vector_to_string(new_residual)
|
| 1401 |
+
sys.stderr.write(f"[beam_debug] lp_false_prune: residual={res_str}\n")
|
| 1402 |
+
sys.stderr.flush()
|
| 1403 |
+
pruned_results.append({"is_pruned": True, "reason": "lp_false", "residual_vec": new_residual, "path_exprs": current_path_exprs + [cand]})
|
| 1404 |
+
got_one = False
|
| 1405 |
+
break
|
| 1406 |
+
|
| 1407 |
+
current_path_exprs.append(cand)
|
| 1408 |
+
current_residual_vec = new_residual
|
| 1409 |
+
subtracted_any = True
|
| 1410 |
+
got_one = True
|
| 1411 |
+
|
| 1412 |
+
if status == 1:
|
| 1413 |
+
break
|
| 1414 |
+
break
|
| 1415 |
+
if not got_one: break
|
| 1416 |
+
if _is_positive_polymatroid(current_residual_vec, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp): break
|
| 1417 |
+
|
| 1418 |
+
if subtracted_any:
|
| 1419 |
+
results.append({
|
| 1420 |
+
"proved": _is_positive_polymatroid(current_residual_vec, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp),
|
| 1421 |
+
"path_exprs": current_path_exprs,
|
| 1422 |
+
"residual_vec": current_residual_vec,
|
| 1423 |
+
"new_ineq": current_path_exprs[len(base_path_exprs):]
|
| 1424 |
+
})
|
| 1425 |
+
else:
|
| 1426 |
+
# Batch: Ask for k things. Each becomes a SEPARATE branch (subtracting only one).
|
| 1427 |
+
current_target_norm = vector_to_string(base_residual_vec)
|
| 1428 |
+
|
| 1429 |
+
tries = 0
|
| 1430 |
+
while len(results) < k_steps and tries < max_tries_per_step:
|
| 1431 |
+
tries += 1
|
| 1432 |
+
prompt = _build_llm_prompt(
|
| 1433 |
+
n=n, min_idx=min_idx, max_idx=max_idx,
|
| 1434 |
+
target_inequality_normalized=current_target_norm,
|
| 1435 |
+
cond_lines=cond_lines, m_exprs=base_path_exprs,
|
| 1436 |
+
idx_to_letter=idx_to_letter,
|
| 1437 |
+
)
|
| 1438 |
+
cand_list = LLM_propose_inequalities(
|
| 1439 |
+
prompt=prompt, k=k_steps, adapter_path=adapter_path, use_dummy=use_dummy,
|
| 1440 |
+
num_return_sequences=k_steps,
|
| 1441 |
+
)
|
| 1442 |
+
_merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
|
| 1443 |
+
for cand in cand_list:
|
| 1444 |
+
if len(results) >= k_steps: break
|
| 1445 |
+
cand_raw = cand
|
| 1446 |
+
cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
|
| 1447 |
+
cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
|
| 1448 |
+
format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
|
| 1449 |
+
if not cand or cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
|
| 1450 |
+
if log_wrong_pruned and cand_raw:
|
| 1451 |
+
wrong_data.append([prompt, cand_raw])
|
| 1452 |
+
if debug and cand_raw:
|
| 1453 |
+
sys.stderr.write(
|
| 1454 |
+
f"[beam_debug] invalid_or_wrong_cand: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
|
| 1455 |
+
)
|
| 1456 |
+
sys.stderr.flush()
|
| 1457 |
+
if format_reason is not None:
|
| 1458 |
+
llm_usage["format_fail_total"] += 1
|
| 1459 |
+
pruned_results.append({"is_pruned": True, "reason": "invalid_or_wrong_cand", "path_exprs": base_path_exprs + [cand]})
|
| 1460 |
+
continue
|
| 1461 |
+
|
| 1462 |
+
cand_indices = _extract_var_indices(cand)
|
| 1463 |
+
if cand_indices and (min(cand_indices) < min_idx or max(cand_indices) > max_idx):
|
| 1464 |
+
if log_wrong_pruned:
|
| 1465 |
+
wrong_data.append([prompt, cand_raw])
|
| 1466 |
+
out_of_range += 1
|
| 1467 |
+
pruned_results.append({"is_pruned": True, "reason": "out_of_range", "path_exprs": base_path_exprs + [cand]})
|
| 1468 |
+
continue
|
| 1469 |
+
try:
|
| 1470 |
+
vec = parse_cand_to_vector(cand, n)
|
| 1471 |
+
except (ValueError, Exception):
|
| 1472 |
+
if log_wrong_pruned:
|
| 1473 |
+
wrong_data.append([prompt, cand_raw])
|
| 1474 |
+
parse_fail += 1
|
| 1475 |
+
pruned_results.append({"is_pruned": True, "reason": "parse_fail", "path_exprs": base_path_exprs + [cand]})
|
| 1476 |
+
continue
|
| 1477 |
+
|
| 1478 |
+
# Validation for Bayesian equalities
|
| 1479 |
+
if re.search(r'(?<![<>])=\s*0', cand):
|
| 1480 |
+
if not _is_in_row_space(vec, cond_eq_matrix):
|
| 1481 |
+
if log_wrong_pruned:
|
| 1482 |
+
wrong_data.append([prompt, cand_raw])
|
| 1483 |
+
if debug:
|
| 1484 |
+
sys.stderr.write(f"[beam_debug] hallucinated_equality: step={cand_raw} not in row-space of conditions\n")
|
| 1485 |
+
sys.stderr.flush()
|
| 1486 |
+
pruned_results.append({"is_pruned": True, "reason": "hallucinated_equality", "path_exprs": base_path_exprs + [cand]})
|
| 1487 |
+
continue # Hallucinated equality
|
| 1488 |
+
else:
|
| 1489 |
+
if debug:
|
| 1490 |
+
sys.stderr.write(f"[beam_debug] equality_verified: step={cand_raw} is in row-space\n")
|
| 1491 |
+
sys.stderr.flush()
|
| 1492 |
+
|
| 1493 |
+
new_residual = base_residual_vec - vec
|
| 1494 |
+
# Check status: 1=proved, -1=impossible (false)
|
| 1495 |
+
status = _check_shannon_status(new_residual, n, Gamma, cond_eq_matrix, cond_ge_matrix, max_vars_for_lp, debug=debug)
|
| 1496 |
+
is_proved = (status == 1)
|
| 1497 |
+
|
| 1498 |
+
if not is_proved:
|
| 1499 |
+
# 1. Simple obviously negative check
|
| 1500 |
+
if _is_obviously_negative(new_residual):
|
| 1501 |
+
if log_wrong_pruned:
|
| 1502 |
+
wrong_data.append([prompt, cand_raw])
|
| 1503 |
+
if debug:
|
| 1504 |
+
res_str = vector_to_string(new_residual)
|
| 1505 |
+
sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
|
| 1506 |
+
sys.stderr.flush()
|
| 1507 |
+
pruned_results.append({"is_pruned": True, "reason": "obviously_negative", "residual_vec": new_residual, "path_exprs": base_path_exprs + [cand]})
|
| 1508 |
+
continue # Prune this candidate
|
| 1509 |
+
|
| 1510 |
+
# 2. LP impossible check
|
| 1511 |
+
if status == -1:
|
| 1512 |
+
if log_wrong_pruned:
|
| 1513 |
+
wrong_data.append([prompt, cand_raw])
|
| 1514 |
+
if debug:
|
| 1515 |
+
res_str = vector_to_string(new_residual)
|
| 1516 |
+
sys.stderr.write(f"[beam_debug] lp_false_prune: residual={res_str}\n")
|
| 1517 |
+
sys.stderr.flush()
|
| 1518 |
+
pruned_results.append({"is_pruned": True, "reason": "lp_false", "residual_vec": new_residual, "path_exprs": base_path_exprs + [cand]})
|
| 1519 |
+
continue # Prune this candidate
|
| 1520 |
+
|
| 1521 |
+
results.append({
|
| 1522 |
+
"proved": is_proved,
|
| 1523 |
+
"path_exprs": base_path_exprs + [cand],
|
| 1524 |
+
"residual_vec": new_residual,
|
| 1525 |
+
"new_ineq": [cand]
|
| 1526 |
+
})
|
| 1527 |
+
|
| 1528 |
+
return {
|
| 1529 |
+
"ok": len(results) > 0,
|
| 1530 |
+
"results": results,
|
| 1531 |
+
"pruned_results": pruned_results,
|
| 1532 |
+
"parse_fail": parse_fail,
|
| 1533 |
+
"out_of_range": out_of_range,
|
| 1534 |
+
"wrong_data": wrong_data,
|
| 1535 |
+
"llm_usage": llm_usage,
|
| 1536 |
+
}
|
| 1537 |
+
|
| 1538 |
+
# Define Node class
|
| 1539 |
+
|
| 1540 |
+
class Node:
|
| 1541 |
+
def __init__(
|
| 1542 |
+
self,
|
| 1543 |
+
residual_vec: Optional[csr_matrix] = None,
|
| 1544 |
+
path_exprs: Optional[List[str]] = None,
|
| 1545 |
+
proved: bool = False,
|
| 1546 |
+
children=None,
|
| 1547 |
+
parent=None,
|
| 1548 |
+
score: float = 0.0,
|
| 1549 |
+
is_pruned: bool = False,
|
| 1550 |
+
):
|
| 1551 |
+
if children is None:
|
| 1552 |
+
children = []
|
| 1553 |
+
if path_exprs is None:
|
| 1554 |
+
path_exprs = []
|
| 1555 |
+
|
| 1556 |
+
self.residual_vec = residual_vec
|
| 1557 |
+
self.path_exprs = list(path_exprs)
|
| 1558 |
+
self.proved = proved
|
| 1559 |
+
self.children = children
|
| 1560 |
+
self.parent = parent
|
| 1561 |
+
self.score = score
|
| 1562 |
+
self.is_pruned = is_pruned
|
| 1563 |
+
|
| 1564 |
+
def relabel_f_and_conditions(
|
| 1565 |
+
f_str: str,
|
| 1566 |
+
cond_list: List[str],
|
| 1567 |
+
) -> Tuple[str, List[str], Dict[int, int], int]:
|
| 1568 |
+
"""
|
| 1569 |
+
对 f_target 和多条 conditions 一起做变量重编号。
|
| 1570 |
+
要求输入里变量形式都是 X_{...},例如:
|
| 1571 |
+
H(X_{1,3}) + H(X_{1,23}) >= 0
|
| 1572 |
+
H(X_{1,2}) >= 0
|
| 1573 |
+
H(X_{3}) = 0 (X_{3} 也可以)
|
| 1574 |
+
|
| 1575 |
+
步骤:
|
| 1576 |
+
1. 从 f_str + cond_list 里找出所有出现的 index(1,3,23,...)
|
| 1577 |
+
2. 按升序排序,映射到 1..n
|
| 1578 |
+
3. 把所有 X_{...} 中的旧 index 换成新 index
|
| 1579 |
+
|
| 1580 |
+
返回:
|
| 1581 |
+
f_new: 重编号后的 f 字符串
|
| 1582 |
+
conds_new: 重编号后的 condition 字符串列表
|
| 1583 |
+
mapping: {旧 index : 新 index}
|
| 1584 |
+
n: 变量个数(新编号的最大值)
|
| 1585 |
+
"""
|
| 1586 |
+
|
| 1587 |
+
# 1) 收集所有出现的旧变量下标
|
| 1588 |
+
all_exprs = [f_str] + list(cond_list)
|
| 1589 |
+
old_indices = set()
|
| 1590 |
+
|
| 1591 |
+
# Normalize expressions to brace format first (H(X_1,X_2) -> H(X_{1,2}))
|
| 1592 |
+
def normalize_to_braces(s):
|
| 1593 |
+
"""Convert H(X_1,X_2) to H(X_{1,2}) format"""
|
| 1594 |
+
def repl_no_braces(match):
|
| 1595 |
+
prefix = match.group(1) or '' # Coefficient like "-35*"
|
| 1596 |
+
content = match.group(2) # X_1,X_2 or X_1
|
| 1597 |
+
if content.startswith('{') and content.endswith('}'):
|
| 1598 |
+
return match.group(0) # Already normalized
|
| 1599 |
+
if ',' in content:
|
| 1600 |
+
vars_list = [v.strip().replace('X_', '') for v in content.split(',')]
|
| 1601 |
+
normalized = ','.join(vars_list)
|
| 1602 |
+
return f"{prefix}H(X_{{{normalized}}})"
|
| 1603 |
+
else:
|
| 1604 |
+
var = content.replace('X_', '')
|
| 1605 |
+
return f"{prefix}H(X_{{{var}}})"
|
| 1606 |
+
return re.sub(r'([+-]?\d*\.?\d*\*?)H\(X_([^)]+)\)', repl_no_braces, s)
|
| 1607 |
+
|
| 1608 |
+
# Normalize all expressions first
|
| 1609 |
+
all_exprs_normalized = [normalize_to_braces(s) for s in all_exprs]
|
| 1610 |
+
|
| 1611 |
+
for s in all_exprs_normalized:
|
| 1612 |
+
# 匹配 X_{...},拿里面的 "1,3" 之类 (after normalization, all are in brace format)
|
| 1613 |
+
for inside in re.findall(r'X_\{([^}]+)\}', s):
|
| 1614 |
+
parts = [p.strip() for p in inside.split(',') if p.strip()]
|
| 1615 |
+
for p in parts:
|
| 1616 |
+
old_indices.add(int(p))
|
| 1617 |
+
|
| 1618 |
+
if not old_indices:
|
| 1619 |
+
# 没有变量,直接返回原表达式
|
| 1620 |
+
mapping: Dict[int, int] = {}
|
| 1621 |
+
return f_str, cond_list, mapping, 0
|
| 1622 |
+
|
| 1623 |
+
# 2) 排序后映射到 1..n
|
| 1624 |
+
sorted_old = sorted(old_indices) # 例如 [1,3,23]
|
| 1625 |
+
mapping: Dict[int, int] = {
|
| 1626 |
+
old: new for new, old in enumerate(sorted_old, start=1)
|
| 1627 |
+
}
|
| 1628 |
+
n = len(sorted_old)
|
| 1629 |
+
|
| 1630 |
+
# 3) 用 mapping 替换表达式中的 index
|
| 1631 |
+
def repl(match: re.Match) -> str:
|
| 1632 |
+
inside = match.group(1) # 原来的 "1,3" 或 "23"
|
| 1633 |
+
parts = [p.strip() for p in inside.split(',') if p.strip()]
|
| 1634 |
+
new_parts = [str(mapping[int(p)]) for p in parts]
|
| 1635 |
+
return f"X_{{{','.join(new_parts)}}}"
|
| 1636 |
+
|
| 1637 |
+
all_exprs_relabel = [
|
| 1638 |
+
re.sub(r'X_\{([^}]+)\}', repl, s)
|
| 1639 |
+
for s in all_exprs_normalized
|
| 1640 |
+
]
|
| 1641 |
+
|
| 1642 |
+
f_new = all_exprs_relabel[0]
|
| 1643 |
+
conds_new = all_exprs_relabel[1:]
|
| 1644 |
+
|
| 1645 |
+
return f_new, conds_new, mapping, n
|
| 1646 |
+
|
| 1647 |
+
# Define the algorithm
|
| 1648 |
+
def tree_search(root_node, max_depth, rollout_count, target_inequality, conditions, n,
|
| 1649 |
+
Gamma: Optional[List[csr_matrix]] = None,
|
| 1650 |
+
adapter_path: Optional[str] = None,
|
| 1651 |
+
use_dummy: bool = False,
|
| 1652 |
+
k_per_call: int = 1,
|
| 1653 |
+
llm_mode: str = "sequential",
|
| 1654 |
+
lp_workers: int = 0,
|
| 1655 |
+
gpu_devices: Optional[List[int]] = None,
|
| 1656 |
+
gpu_executors: Optional[List[ProcessPoolExecutor]] = None,
|
| 1657 |
+
beam_size: int = 0,
|
| 1658 |
+
random_prune: bool = False,
|
| 1659 |
+
prune_seed: Optional[int] = None,
|
| 1660 |
+
max_vars_for_lp: int = 0,
|
| 1661 |
+
log_wrong_pruned: bool = False,
|
| 1662 |
+
wrong_pruned_path: Optional[str] = None,
|
| 1663 |
+
sample_time_budget_sec: float = 0.0,
|
| 1664 |
+
sample_tree_budget: int = 0,
|
| 1665 |
+
sample_llm_calls_budget: int = 0,
|
| 1666 |
+
sample_llm_fail_budget: int = 0,
|
| 1667 |
+
depth1_rollouts: Optional[int] = None):
|
| 1668 |
+
"""
|
| 1669 |
+
Algorithm for Data Generation via LLM-Guided Tree Search (Model 2: Subtraction based).
|
| 1670 |
+
"""
|
| 1671 |
+
_seed_indices = _extract_var_indices(target_inequality)
|
| 1672 |
+
for _c in (conditions or []):
|
| 1673 |
+
_seed_indices.extend(_extract_var_indices(_c))
|
| 1674 |
+
if _seed_indices:
|
| 1675 |
+
min_idx = min(_seed_indices)
|
| 1676 |
+
max_idx = max(_seed_indices)
|
| 1677 |
+
else:
|
| 1678 |
+
min_idx, max_idx = 1, int(n)
|
| 1679 |
+
|
| 1680 |
+
min_idx = max(1, int(min_idx))
|
| 1681 |
+
max_idx = min(int(n), int(max_idx))
|
| 1682 |
+
if min_idx > max_idx:
|
| 1683 |
+
min_idx, max_idx = 1, int(n)
|
| 1684 |
+
|
| 1685 |
+
debug = os.environ.get("BEAM_DEBUG", "0") == "1"
|
| 1686 |
+
prune_rng = None
|
| 1687 |
+
if beam_size and beam_size > 0 and random_prune:
|
| 1688 |
+
prune_rng = random.Random(prune_seed)
|
| 1689 |
+
|
| 1690 |
+
target_inequality = _normalize_to_braces(target_inequality)
|
| 1691 |
+
conditions = [
|
| 1692 |
+
_normalize_constraint_comparator(_normalize_to_braces(c), add_default=True)
|
| 1693 |
+
for c in (conditions or [])
|
| 1694 |
+
]
|
| 1695 |
+
idx_to_letter, letter_to_idx = _build_alpha_mapping([target_inequality] + list(conditions or []))
|
| 1696 |
+
|
| 1697 |
+
if Gamma is None and max_vars_for_lp > 0:
|
| 1698 |
+
Gamma = _generate_elemental_generators(max_vars_for_lp)
|
| 1699 |
+
|
| 1700 |
+
f_csr = parse_cand_to_vector(target_inequality, n)
|
| 1701 |
+
cond_eq_csr, cond_ge_csr = _parse_condition_matrices(conditions, n)
|
| 1702 |
+
|
| 1703 |
+
root_node.residual_vec = f_csr
|
| 1704 |
+
root_node.path_exprs = []
|
| 1705 |
+
|
| 1706 |
+
seen_residuals = set()
|
| 1707 |
+
root_key = vector_to_string(root_node.residual_vec)
|
| 1708 |
+
seen_residuals.add(root_key)
|
| 1709 |
+
|
| 1710 |
+
nodes = [root_node]
|
| 1711 |
+
tree = [root_node]
|
| 1712 |
+
lp_calls = 0
|
| 1713 |
+
parse_fail = 0
|
| 1714 |
+
out_of_range = 0
|
| 1715 |
+
all_wrong_data = []
|
| 1716 |
+
llm_usage = _empty_llm_usage()
|
| 1717 |
+
sample_time_budget_sec = float(sample_time_budget_sec or 0.0)
|
| 1718 |
+
sample_tree_budget = int(sample_tree_budget or 0)
|
| 1719 |
+
sample_llm_calls_budget = int(sample_llm_calls_budget or 0)
|
| 1720 |
+
sample_llm_fail_budget = int(sample_llm_fail_budget or 0)
|
| 1721 |
+
depth1_rollouts = int(depth1_rollouts) if depth1_rollouts and int(depth1_rollouts) > 0 else None
|
| 1722 |
+
stop_reason = None
|
| 1723 |
+
t_start = time.monotonic()
|
| 1724 |
+
|
| 1725 |
+
llm_mode = (llm_mode or "sequential").strip().lower()
|
| 1726 |
+
run_id = uuid.uuid4().hex
|
| 1727 |
+
global _CACHED_GPU_EXECUTORS, _CACHED_GPU_PARAMS
|
| 1728 |
+
gpu_devices = list(gpu_devices) if gpu_devices else []
|
| 1729 |
+
|
| 1730 |
+
def _flush_wrong_data() -> None:
|
| 1731 |
+
if log_wrong_pruned and wrong_pruned_path and all_wrong_data:
|
| 1732 |
+
try:
|
| 1733 |
+
with open(wrong_pruned_path, "a", encoding="utf-8") as f:
|
| 1734 |
+
for inp_str, out_str in all_wrong_data:
|
| 1735 |
+
f.write(json.dumps({"input": inp_str, "output": out_str}, ensure_ascii=False) + "\n")
|
| 1736 |
+
all_wrong_data.clear()
|
| 1737 |
+
except Exception as e:
|
| 1738 |
+
if debug:
|
| 1739 |
+
sys.stderr.write(f"[beam_debug] error_writing_wrong_data: {e}\n")
|
| 1740 |
+
sys.stderr.flush()
|
| 1741 |
+
|
| 1742 |
+
def _budget_exceeded() -> Optional[str]:
|
| 1743 |
+
if sample_time_budget_sec > 0 and (time.monotonic() - t_start) > sample_time_budget_sec:
|
| 1744 |
+
return "time_budget"
|
| 1745 |
+
if sample_tree_budget > 0 and len(tree) >= sample_tree_budget:
|
| 1746 |
+
return "tree_budget"
|
| 1747 |
+
if sample_llm_calls_budget > 0 and llm_usage.get("llm_calls", 0) >= sample_llm_calls_budget:
|
| 1748 |
+
return "llm_calls_budget"
|
| 1749 |
+
if sample_llm_fail_budget > 0 and llm_usage.get("format_fail_total", 0) >= sample_llm_fail_budget:
|
| 1750 |
+
return "llm_fail_budget"
|
| 1751 |
+
return None
|
| 1752 |
+
|
| 1753 |
+
def _refresh_stop_reason() -> Optional[str]:
|
| 1754 |
+
nonlocal stop_reason
|
| 1755 |
+
if stop_reason is None:
|
| 1756 |
+
stop_reason = _budget_exceeded()
|
| 1757 |
+
return stop_reason
|
| 1758 |
+
|
| 1759 |
+
def _finalize(proved: bool, x, expr_str):
|
| 1760 |
+
_flush_wrong_data()
|
| 1761 |
+
return proved, tree, x, expr_str, llm_usage, stop_reason
|
| 1762 |
+
|
| 1763 |
+
current_params = (adapter_path, use_dummy, tuple(gpu_devices), max_vars_for_lp)
|
| 1764 |
+
|
| 1765 |
+
if gpu_executors is not None:
|
| 1766 |
+
pass
|
| 1767 |
+
elif _CACHED_GPU_EXECUTORS is not None and _CACHED_GPU_PARAMS == current_params:
|
| 1768 |
+
gpu_executors = _CACHED_GPU_EXECUTORS
|
| 1769 |
+
elif gpu_devices:
|
| 1770 |
+
if _CACHED_GPU_EXECUTORS is not None:
|
| 1771 |
+
for ex in _CACHED_GPU_EXECUTORS:
|
| 1772 |
+
try:
|
| 1773 |
+
ex.shutdown(wait=True, cancel_futures=True)
|
| 1774 |
+
except Exception:
|
| 1775 |
+
pass
|
| 1776 |
+
|
| 1777 |
+
gpu_executors = []
|
| 1778 |
+
try:
|
| 1779 |
+
gpu_ctx = mp.get_context("spawn")
|
| 1780 |
+
except Exception:
|
| 1781 |
+
gpu_ctx = None
|
| 1782 |
+
for gid in gpu_devices:
|
| 1783 |
+
gpu_executors.append(
|
| 1784 |
+
ProcessPoolExecutor(
|
| 1785 |
+
max_workers=1,
|
| 1786 |
+
mp_context=gpu_ctx,
|
| 1787 |
+
initializer=_gpu_worker_init,
|
| 1788 |
+
initargs=(adapter_path, use_dummy, int(gid), int(max_vars_for_lp), Gamma),
|
| 1789 |
+
)
|
| 1790 |
+
)
|
| 1791 |
+
_CACHED_GPU_EXECUTORS = gpu_executors
|
| 1792 |
+
_CACHED_GPU_PARAMS = current_params
|
| 1793 |
+
|
| 1794 |
+
if _is_positive_polymatroid(root_node.residual_vec, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug):
|
| 1795 |
+
if debug:
|
| 1796 |
+
sys.stderr.write(f"[beam_debug] proved_at_root: Target={target_inequality}\n")
|
| 1797 |
+
sys.stderr.flush()
|
| 1798 |
+
return _finalize(True, None, root_node.path_exprs)
|
| 1799 |
+
|
| 1800 |
+
try:
|
| 1801 |
+
cond_raw = conditions if conditions else []
|
| 1802 |
+
cond_lines = []
|
| 1803 |
+
for c in cond_raw:
|
| 1804 |
+
c_norm = _normalize_constraint_comparator(_normalize_to_braces(c.strip()), add_default=True)
|
| 1805 |
+
cond_lines.append(c_norm)
|
| 1806 |
+
|
| 1807 |
+
if not cond_lines:
|
| 1808 |
+
cond_lines = ["NONE=0"]
|
| 1809 |
+
|
| 1810 |
+
for depth in range(1, max_depth + 1):
|
| 1811 |
+
if _refresh_stop_reason():
|
| 1812 |
+
break
|
| 1813 |
+
next_level_nodes = []
|
| 1814 |
+
depth_rollouts = depth1_rollouts if depth == 1 and depth1_rollouts is not None else (32 if depth == 1 else rollout_count)
|
| 1815 |
+
|
| 1816 |
+
if gpu_executors:
|
| 1817 |
+
futures = []
|
| 1818 |
+
fut_meta = {}
|
| 1819 |
+
rr = 0
|
| 1820 |
+
max_tries_per_step = int(os.environ.get("BEAM_MAX_TRIES_PER_STEP", "3"))
|
| 1821 |
+
|
| 1822 |
+
for node in nodes:
|
| 1823 |
+
if _refresh_stop_reason():
|
| 1824 |
+
break
|
| 1825 |
+
for i in range(depth_rollouts):
|
| 1826 |
+
if _refresh_stop_reason():
|
| 1827 |
+
break
|
| 1828 |
+
payload = {
|
| 1829 |
+
"n": int(n),
|
| 1830 |
+
"min_idx": int(min_idx),
|
| 1831 |
+
"max_idx": int(max_idx),
|
| 1832 |
+
"run_id": run_id,
|
| 1833 |
+
"cond_lines": cond_lines,
|
| 1834 |
+
"path_exprs": list(node.path_exprs),
|
| 1835 |
+
"residual_vec": node.residual_vec,
|
| 1836 |
+
"k_per_call": int(k_per_call),
|
| 1837 |
+
"llm_mode": llm_mode,
|
| 1838 |
+
"max_tries_per_step": max_tries_per_step,
|
| 1839 |
+
"adapter_path": adapter_path,
|
| 1840 |
+
"use_dummy": bool(use_dummy),
|
| 1841 |
+
"Gamma": None,
|
| 1842 |
+
"max_vars_for_lp": int(max_vars_for_lp),
|
| 1843 |
+
"log_wrong_pruned": log_wrong_pruned,
|
| 1844 |
+
"idx_to_letter": idx_to_letter,
|
| 1845 |
+
"letter_to_idx": letter_to_idx,
|
| 1846 |
+
}
|
| 1847 |
+
ex = gpu_executors[rr % len(gpu_executors)]
|
| 1848 |
+
rr += 1
|
| 1849 |
+
fut = ex.submit(_expand_node_payload, payload)
|
| 1850 |
+
futures.append(fut)
|
| 1851 |
+
fut_meta[fut] = (node, i)
|
| 1852 |
+
if stop_reason:
|
| 1853 |
+
break
|
| 1854 |
+
|
| 1855 |
+
if not stop_reason:
|
| 1856 |
+
for fut in as_completed(futures):
|
| 1857 |
+
if stop_reason:
|
| 1858 |
+
break
|
| 1859 |
+
node, rollout_idx = fut_meta[fut]
|
| 1860 |
+
try:
|
| 1861 |
+
r = fut.result()
|
| 1862 |
+
except Exception as e:
|
| 1863 |
+
if _is_fatal_gpu_worker_exception(e):
|
| 1864 |
+
raise
|
| 1865 |
+
if debug:
|
| 1866 |
+
sys.stderr.write(f"[beam_debug] gpu_worker_error: {type(e).__name__}: {e}\n")
|
| 1867 |
+
sys.stderr.flush()
|
| 1868 |
+
continue
|
| 1869 |
+
parse_fail += int(r.get("parse_fail", 0))
|
| 1870 |
+
out_of_range += int(r.get("out_of_range", 0))
|
| 1871 |
+
worker_llm_usage = r.get("llm_usage")
|
| 1872 |
+
if worker_llm_usage:
|
| 1873 |
+
for key in llm_usage:
|
| 1874 |
+
llm_usage[key] += int(worker_llm_usage.get(key, 0))
|
| 1875 |
+
if log_wrong_pruned and "wrong_data" in r:
|
| 1876 |
+
all_wrong_data.extend(r["wrong_data"])
|
| 1877 |
+
if _refresh_stop_reason():
|
| 1878 |
+
break
|
| 1879 |
+
|
| 1880 |
+
for pr_data in r.get("pruned_results", []):
|
| 1881 |
+
pr_node = Node(
|
| 1882 |
+
proved=False,
|
| 1883 |
+
parent=node,
|
| 1884 |
+
path_exprs=list(pr_data.get("path_exprs") or []),
|
| 1885 |
+
residual_vec=pr_data.get("residual_vec"),
|
| 1886 |
+
is_pruned=True,
|
| 1887 |
+
)
|
| 1888 |
+
tree.append(pr_node)
|
| 1889 |
+
if _refresh_stop_reason():
|
| 1890 |
+
break
|
| 1891 |
+
if stop_reason:
|
| 1892 |
+
break
|
| 1893 |
+
|
| 1894 |
+
if not r.get("ok", False):
|
| 1895 |
+
continue
|
| 1896 |
+
|
| 1897 |
+
for res_data in r.get("results", []):
|
| 1898 |
+
if _refresh_stop_reason():
|
| 1899 |
+
break
|
| 1900 |
+
res_vec = res_data.get("residual_vec", None)
|
| 1901 |
+
if res_vec is None:
|
| 1902 |
+
continue
|
| 1903 |
+
|
| 1904 |
+
res_key = vector_to_string(res_vec)
|
| 1905 |
+
if res_key in seen_residuals:
|
| 1906 |
+
if debug:
|
| 1907 |
+
sys.stderr.write(f"[beam_debug] duplicate_prune: depth={depth} rollout={rollout_idx} Residual={res_key[:50]}...\n")
|
| 1908 |
+
sys.stderr.flush()
|
| 1909 |
+
pr_node = Node(
|
| 1910 |
+
proved=False,
|
| 1911 |
+
parent=node,
|
| 1912 |
+
path_exprs=list(res_data.get("path_exprs") or []),
|
| 1913 |
+
residual_vec=res_vec,
|
| 1914 |
+
is_pruned=True,
|
| 1915 |
+
)
|
| 1916 |
+
tree.append(pr_node)
|
| 1917 |
+
if _refresh_stop_reason():
|
| 1918 |
+
break
|
| 1919 |
+
continue
|
| 1920 |
+
seen_residuals.add(res_key)
|
| 1921 |
+
|
| 1922 |
+
child = Node(
|
| 1923 |
+
proved=res_data.get("proved", False),
|
| 1924 |
+
parent=node,
|
| 1925 |
+
path_exprs=list(res_data.get("path_exprs") or []),
|
| 1926 |
+
residual_vec=res_vec,
|
| 1927 |
+
)
|
| 1928 |
+
|
| 1929 |
+
if debug:
|
| 1930 |
+
_beam_debug_print_M(
|
| 1931 |
+
depth=depth,
|
| 1932 |
+
rollout=rollout_idx,
|
| 1933 |
+
path_exprs=child.path_exprs,
|
| 1934 |
+
residual_vec=child.residual_vec,
|
| 1935 |
+
action="expand_done",
|
| 1936 |
+
)
|
| 1937 |
+
|
| 1938 |
+
if child.proved:
|
| 1939 |
+
if debug:
|
| 1940 |
+
sys.stderr.write(f"[beam_debug] proved: path={child.path_exprs}\n")
|
| 1941 |
+
sys.stderr.flush()
|
| 1942 |
+
tree.append(child)
|
| 1943 |
+
return _finalize(True, None, child.path_exprs)
|
| 1944 |
+
|
| 1945 |
+
next_level_nodes.append(child)
|
| 1946 |
+
if stop_reason:
|
| 1947 |
+
break
|
| 1948 |
+
|
| 1949 |
+
if stop_reason:
|
| 1950 |
+
for fut in futures:
|
| 1951 |
+
try:
|
| 1952 |
+
fut.cancel()
|
| 1953 |
+
except Exception:
|
| 1954 |
+
pass
|
| 1955 |
+
else:
|
| 1956 |
+
for node in nodes:
|
| 1957 |
+
if _refresh_stop_reason():
|
| 1958 |
+
break
|
| 1959 |
+
for i in range(depth_rollouts):
|
| 1960 |
+
if _refresh_stop_reason():
|
| 1961 |
+
break
|
| 1962 |
+
current_residual_vec = node.residual_vec
|
| 1963 |
+
current_path_exprs = list(node.path_exprs)
|
| 1964 |
+
|
| 1965 |
+
if llm_mode == "sequential":
|
| 1966 |
+
subtracted_any = False
|
| 1967 |
+
for _step in range(k_per_call):
|
| 1968 |
+
if _refresh_stop_reason():
|
| 1969 |
+
break
|
| 1970 |
+
current_target_norm = vector_to_string(current_residual_vec)
|
| 1971 |
+
|
| 1972 |
+
prompt = _build_llm_prompt(
|
| 1973 |
+
n=n, min_idx=min_idx, max_idx=max_idx,
|
| 1974 |
+
target_inequality_normalized=current_target_norm,
|
| 1975 |
+
cond_lines=cond_lines, m_exprs=current_path_exprs,
|
| 1976 |
+
idx_to_letter=idx_to_letter,
|
| 1977 |
+
)
|
| 1978 |
+
cand_list = LLM_propose_inequalities(prompt, k=1, adapter_path=adapter_path, use_dummy=use_dummy)
|
| 1979 |
+
_merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
|
| 1980 |
+
if _refresh_stop_reason():
|
| 1981 |
+
break
|
| 1982 |
+
if not cand_list:
|
| 1983 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs, is_pruned=True))
|
| 1984 |
+
_refresh_stop_reason()
|
| 1985 |
+
break
|
| 1986 |
+
cand_raw = cand_list[0]
|
| 1987 |
+
cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
|
| 1988 |
+
cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
|
| 1989 |
+
format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
|
| 1990 |
+
if cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
|
| 1991 |
+
if log_wrong_pruned:
|
| 1992 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 1993 |
+
if debug:
|
| 1994 |
+
sys.stderr.write(
|
| 1995 |
+
f"[beam_debug] invalid_or_wrong_cand_manual: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
|
| 1996 |
+
)
|
| 1997 |
+
sys.stderr.flush()
|
| 1998 |
+
if format_reason is not None:
|
| 1999 |
+
llm_usage["format_fail_total"] += 1
|
| 2000 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
|
| 2001 |
+
_refresh_stop_reason()
|
| 2002 |
+
break
|
| 2003 |
+
try:
|
| 2004 |
+
vec = parse_cand_to_vector(cand, n)
|
| 2005 |
+
if re.search(r'(?<![<>])=\s*0', cand):
|
| 2006 |
+
if not _is_in_row_space(vec, cond_eq_csr):
|
| 2007 |
+
if log_wrong_pruned:
|
| 2008 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2009 |
+
if debug:
|
| 2010 |
+
sys.stderr.write(f"[beam_debug] hallucinated_equality_manual: step={cand_raw} not in row-space of conditions\n")
|
| 2011 |
+
sys.stderr.flush()
|
| 2012 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
|
| 2013 |
+
_refresh_stop_reason()
|
| 2014 |
+
break
|
| 2015 |
+
else:
|
| 2016 |
+
if debug:
|
| 2017 |
+
sys.stderr.write(f"[beam_debug] equality_verified_manual: step={cand_raw} is in row-space\n")
|
| 2018 |
+
sys.stderr.flush()
|
| 2019 |
+
|
| 2020 |
+
new_residual = current_residual_vec - vec
|
| 2021 |
+
status = _check_shannon_status(new_residual, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug)
|
| 2022 |
+
|
| 2023 |
+
if _is_obviously_negative(new_residual):
|
| 2024 |
+
if log_wrong_pruned:
|
| 2025 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2026 |
+
if debug:
|
| 2027 |
+
res_str = vector_to_string(new_residual)
|
| 2028 |
+
sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
|
| 2029 |
+
sys.stderr.flush()
|
| 2030 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
|
| 2031 |
+
subtracted_any = False
|
| 2032 |
+
_refresh_stop_reason()
|
| 2033 |
+
break
|
| 2034 |
+
|
| 2035 |
+
if status == -1:
|
| 2036 |
+
if log_wrong_pruned:
|
| 2037 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2038 |
+
if debug:
|
| 2039 |
+
res_str = vector_to_string(new_residual)
|
| 2040 |
+
sys.stderr.write(f"[beam_debug] lp_false_prune_manual: residual={res_str}\n")
|
| 2041 |
+
sys.stderr.flush()
|
| 2042 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
|
| 2043 |
+
subtracted_any = False
|
| 2044 |
+
_refresh_stop_reason()
|
| 2045 |
+
break
|
| 2046 |
+
|
| 2047 |
+
if status == 1:
|
| 2048 |
+
current_residual_vec = new_residual
|
| 2049 |
+
current_path_exprs.append(cand)
|
| 2050 |
+
subtracted_any = True
|
| 2051 |
+
break
|
| 2052 |
+
|
| 2053 |
+
res_key = vector_to_string(new_residual)
|
| 2054 |
+
if res_key in seen_residuals:
|
| 2055 |
+
if debug:
|
| 2056 |
+
sys.stderr.write(f"[beam_debug] duplicate_prune: depth={depth} rollout={i} Residual={res_key[:50]}...\n")
|
| 2057 |
+
sys.stderr.flush()
|
| 2058 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
|
| 2059 |
+
_refresh_stop_reason()
|
| 2060 |
+
break
|
| 2061 |
+
|
| 2062 |
+
seen_residuals.add(res_key)
|
| 2063 |
+
current_residual_vec = new_residual
|
| 2064 |
+
current_path_exprs.append(cand)
|
| 2065 |
+
subtracted_any = True
|
| 2066 |
+
except Exception:
|
| 2067 |
+
if log_wrong_pruned:
|
| 2068 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2069 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
|
| 2070 |
+
_refresh_stop_reason()
|
| 2071 |
+
continue
|
| 2072 |
+
|
| 2073 |
+
if stop_reason:
|
| 2074 |
+
break
|
| 2075 |
+
if subtracted_any:
|
| 2076 |
+
is_proved = _is_positive_polymatroid(current_residual_vec, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug)
|
| 2077 |
+
child = Node(proved=is_proved, parent=node, path_exprs=current_path_exprs, residual_vec=current_residual_vec)
|
| 2078 |
+
if debug:
|
| 2079 |
+
_beam_debug_print_M(depth=depth, rollout=i, path_exprs=child.path_exprs, residual_vec=child.residual_vec, action="manual_expand")
|
| 2080 |
+
if is_proved:
|
| 2081 |
+
if debug:
|
| 2082 |
+
sys.stderr.write(f"[beam_debug] proved: path={current_path_exprs}\n")
|
| 2083 |
+
sys.stderr.flush()
|
| 2084 |
+
tree.append(child)
|
| 2085 |
+
return _finalize(True, None, child.path_exprs)
|
| 2086 |
+
next_level_nodes.append(child)
|
| 2087 |
+
else:
|
| 2088 |
+
current_target_norm = vector_to_string(current_residual_vec)
|
| 2089 |
+
|
| 2090 |
+
prompt = _build_llm_prompt(
|
| 2091 |
+
n=n, min_idx=min_idx, max_idx=max_idx,
|
| 2092 |
+
target_inequality_normalized=current_target_norm,
|
| 2093 |
+
cond_lines=cond_lines, m_exprs=current_path_exprs,
|
| 2094 |
+
idx_to_letter=idx_to_letter,
|
| 2095 |
+
)
|
| 2096 |
+
cand_list = LLM_propose_inequalities(prompt, k=k_per_call, adapter_path=adapter_path, use_dummy=use_dummy)
|
| 2097 |
+
_merge_llm_usage(llm_usage, get_last_llm_stats(reset=True))
|
| 2098 |
+
if _refresh_stop_reason():
|
| 2099 |
+
break
|
| 2100 |
+
for cand in cand_list:
|
| 2101 |
+
if _refresh_stop_reason():
|
| 2102 |
+
break
|
| 2103 |
+
cand_raw = cand
|
| 2104 |
+
cand_text = _normalize_candidate_text(cand_raw) if _normalize_candidates_enabled() else cand_raw
|
| 2105 |
+
cand = _normalize_constraint_comparator(_convert_alpha_to_old(cand_text, letter_to_idx))
|
| 2106 |
+
format_reason = _strict_step_format_reason(cand, n=n, cond_lines=cond_lines)
|
| 2107 |
+
if not cand or cand == "wrong" or _is_invalid_negative_ineq(cand, n=n, cond_lines=cond_lines) or format_reason is not None:
|
| 2108 |
+
if log_wrong_pruned:
|
| 2109 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2110 |
+
if debug and cand_raw:
|
| 2111 |
+
sys.stderr.write(
|
| 2112 |
+
f"[beam_debug] invalid_or_wrong_cand_manual_batch: reason={format_reason or 'wrong_or_negative'} step={cand_raw}\n"
|
| 2113 |
+
)
|
| 2114 |
+
sys.stderr.flush()
|
| 2115 |
+
if format_reason is not None:
|
| 2116 |
+
llm_usage["format_fail_total"] += 1
|
| 2117 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
|
| 2118 |
+
_refresh_stop_reason()
|
| 2119 |
+
continue
|
| 2120 |
+
try:
|
| 2121 |
+
vec = parse_cand_to_vector(cand, n)
|
| 2122 |
+
if re.search(r'(?<![<>])=\s*0', cand):
|
| 2123 |
+
if not _is_in_row_space(vec, cond_eq_csr):
|
| 2124 |
+
if log_wrong_pruned:
|
| 2125 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2126 |
+
if debug:
|
| 2127 |
+
sys.stderr.write(f"[beam_debug] hallucinated_equality_manual: step={cand_raw} not in row-space of conditions\n")
|
| 2128 |
+
sys.stderr.flush()
|
| 2129 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
|
| 2130 |
+
_refresh_stop_reason()
|
| 2131 |
+
continue
|
| 2132 |
+
else:
|
| 2133 |
+
if debug:
|
| 2134 |
+
sys.stderr.write(f"[beam_debug] equality_verified_manual: step={cand_raw} is in row-space\n")
|
| 2135 |
+
sys.stderr.flush()
|
| 2136 |
+
|
| 2137 |
+
new_residual = current_residual_vec - vec
|
| 2138 |
+
status = _check_shannon_status(new_residual, n, Gamma, cond_eq_csr, cond_ge_csr, max_vars_for_lp, debug=debug)
|
| 2139 |
+
is_proved = (status == 1)
|
| 2140 |
+
|
| 2141 |
+
if not is_proved and _is_obviously_negative(new_residual):
|
| 2142 |
+
if log_wrong_pruned:
|
| 2143 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2144 |
+
if debug:
|
| 2145 |
+
res_str = vector_to_string(new_residual)
|
| 2146 |
+
sys.stderr.write(f"[beam_debug] obviously_negative_prune: residual={res_str}\n")
|
| 2147 |
+
sys.stderr.flush()
|
| 2148 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
|
| 2149 |
+
_refresh_stop_reason()
|
| 2150 |
+
continue
|
| 2151 |
+
|
| 2152 |
+
if not is_proved and status == -1:
|
| 2153 |
+
if log_wrong_pruned:
|
| 2154 |
+
all_wrong_data.append([prompt, cand_raw])
|
| 2155 |
+
if debug:
|
| 2156 |
+
res_str = vector_to_string(new_residual)
|
| 2157 |
+
sys.stderr.write(f"[beam_debug] lp_false_prune_manual_batch: residual={res_str}\n")
|
| 2158 |
+
sys.stderr.flush()
|
| 2159 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
|
| 2160 |
+
_refresh_stop_reason()
|
| 2161 |
+
continue
|
| 2162 |
+
|
| 2163 |
+
res_key = vector_to_string(new_residual)
|
| 2164 |
+
if res_key in seen_residuals:
|
| 2165 |
+
if debug:
|
| 2166 |
+
sys.stderr.write(f"[beam_debug] duplicate_prune: depth={depth} rollout={i} Residual={res_key[:50]}...\n")
|
| 2167 |
+
sys.stderr.flush()
|
| 2168 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual, is_pruned=True))
|
| 2169 |
+
_refresh_stop_reason()
|
| 2170 |
+
continue
|
| 2171 |
+
|
| 2172 |
+
seen_residuals.add(res_key)
|
| 2173 |
+
child = Node(proved=is_proved, parent=node, path_exprs=current_path_exprs + [cand], residual_vec=new_residual)
|
| 2174 |
+
if debug:
|
| 2175 |
+
_beam_debug_print_M(depth=depth, rollout=i, path_exprs=child.path_exprs, residual_vec=child.residual_vec, action="manual_expand_batch")
|
| 2176 |
+
if is_proved:
|
| 2177 |
+
if debug:
|
| 2178 |
+
sys.stderr.write(f"[beam_debug] proved: path={current_path_exprs + [cand]}\n")
|
| 2179 |
+
sys.stderr.flush()
|
| 2180 |
+
tree.append(child)
|
| 2181 |
+
return _finalize(True, None, child.path_exprs)
|
| 2182 |
+
next_level_nodes.append(child)
|
| 2183 |
+
except Exception:
|
| 2184 |
+
tree.append(Node(proved=False, parent=node, path_exprs=current_path_exprs + [cand], is_pruned=True))
|
| 2185 |
+
_refresh_stop_reason()
|
| 2186 |
+
continue
|
| 2187 |
+
if stop_reason:
|
| 2188 |
+
break
|
| 2189 |
+
if stop_reason:
|
| 2190 |
+
break
|
| 2191 |
+
|
| 2192 |
+
if stop_reason:
|
| 2193 |
+
break
|
| 2194 |
+
if not next_level_nodes:
|
| 2195 |
+
if debug:
|
| 2196 |
+
sys.stderr.write(f"[beam_debug] search_stuck: depth={depth} next_level_nodes is empty\n")
|
| 2197 |
+
sys.stderr.flush()
|
| 2198 |
+
break
|
| 2199 |
+
|
| 2200 |
+
nodes = next_level_nodes
|
| 2201 |
+
|
| 2202 |
+
if beam_size and len(nodes) > beam_size:
|
| 2203 |
+
before = len(nodes)
|
| 2204 |
+
nodes, pruned_by_beam = _select_beam_nodes(
|
| 2205 |
+
nodes,
|
| 2206 |
+
beam_size,
|
| 2207 |
+
random_prune=random_prune,
|
| 2208 |
+
rng=prune_rng,
|
| 2209 |
+
)
|
| 2210 |
+
for node in nodes:
|
| 2211 |
+
node.score = _get_residual_score(node.residual_vec)
|
| 2212 |
+
for pn in pruned_by_beam:
|
| 2213 |
+
pn.is_pruned = True
|
| 2214 |
+
tree.append(pn)
|
| 2215 |
+
if _refresh_stop_reason():
|
| 2216 |
+
break
|
| 2217 |
+
if stop_reason:
|
| 2218 |
+
break
|
| 2219 |
+
if debug:
|
| 2220 |
+
strategy = "seeded_uniform_random" if random_prune else "short_residual_greedy"
|
| 2221 |
+
sys.stderr.write(
|
| 2222 |
+
f"[beam_debug] prune: depth={depth} strategy={strategy} beam_size={beam_size} "
|
| 2223 |
+
f"before={before} after={len(nodes)}\n"
|
| 2224 |
+
)
|
| 2225 |
+
sys.stderr.flush()
|
| 2226 |
+
tree.extend(nodes)
|
| 2227 |
+
if _refresh_stop_reason():
|
| 2228 |
+
break
|
| 2229 |
+
|
| 2230 |
+
return _finalize(False, None, None)
|
| 2231 |
+
finally:
|
| 2232 |
+
pass
|
mce220_runner/beam_search_model2/__init__.py
ADDED
|
File without changes
|
mce220_runner/beam_search_model2/llm_api_compat.py
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Portable compatibility import for the shared local LLM interface.
|
| 2 |
+
|
| 3 |
+
The implementation lives in :mod:`grpo_pipeline.modules.llm_api_interface`.
|
| 4 |
+
This module replaces the repository's former absolute Linux symlink so the
|
| 5 |
+
beam-search entry points work from a fresh checkout on any platform.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
import importlib
|
| 11 |
+
import sys
|
| 12 |
+
from pathlib import Path
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
_REPO_ROOT = Path(__file__).resolve().parents[1]
|
| 16 |
+
if str(_REPO_ROOT) not in sys.path:
|
| 17 |
+
sys.path.insert(0, str(_REPO_ROOT))
|
| 18 |
+
|
| 19 |
+
_shared = importlib.import_module("grpo_pipeline.modules.llm_api_interface")
|
| 20 |
+
__all__ = [name for name in vars(_shared) if not name.startswith("_")]
|
| 21 |
+
globals().update({name: getattr(_shared, name) for name in __all__})
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def __getattr__(name: str):
|
| 25 |
+
"""Delegate private compatibility imports to the shared implementation."""
|
| 26 |
+
return getattr(_shared, name)
|
mce220_runner/beam_search_model2/llmstr_to_vector.py
ADDED
|
@@ -0,0 +1,104 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Parse common information-theoretic inequality shorthands into entropy vectors.
|
| 3 |
+
|
| 4 |
+
Supported:
|
| 5 |
+
- I(X_A;X_B|X_C)
|
| 6 |
+
- I(X_A;X_B)
|
| 7 |
+
- H(X_A|X_B)
|
| 8 |
+
- H(X_A) / H(X_{...}) linear combinations are handled by data/entropy_vector_converter.string_to_vector
|
| 9 |
+
|
| 10 |
+
This module provides polym_ineq_to_vector(expr, n) to match existing imports.
|
| 11 |
+
"""
|
| 12 |
+
|
| 13 |
+
from __future__ import annotations
|
| 14 |
+
|
| 15 |
+
import re
|
| 16 |
+
import numpy as np
|
| 17 |
+
from scipy.sparse import csr_matrix
|
| 18 |
+
from typing import List, Set
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def _subset_to_index(subset: Set[int], n: int) -> int:
|
| 22 |
+
# Convert subset {i} into 0-based index in length (2^n - 1) vector.
|
| 23 |
+
# index = sum(2^(i-1)) - 1
|
| 24 |
+
s = 0
|
| 25 |
+
for i in subset:
|
| 26 |
+
if 1 <= i <= n:
|
| 27 |
+
s += 2 ** (i - 1)
|
| 28 |
+
else:
|
| 29 |
+
raise ValueError(f"variable index out of range: {i} (n={n})")
|
| 30 |
+
if s <= 0:
|
| 31 |
+
raise ValueError("empty subset has no entropy coordinate")
|
| 32 |
+
return s - 1
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def _parse_X_term(s: str) -> Set[int]:
|
| 36 |
+
"""
|
| 37 |
+
Parse X_{1,2} or X_1 or {1,2} into a set of ints.
|
| 38 |
+
"""
|
| 39 |
+
s = s.strip()
|
| 40 |
+
# Allow raw "{1,2}" or "1,2"
|
| 41 |
+
s = s.strip("{}")
|
| 42 |
+
s = s.replace("X_", "").strip()
|
| 43 |
+
if not s:
|
| 44 |
+
return set()
|
| 45 |
+
parts = [p.strip() for p in s.split(",") if p.strip()]
|
| 46 |
+
out: Set[int] = set()
|
| 47 |
+
for p in parts:
|
| 48 |
+
out.add(int(p))
|
| 49 |
+
return out
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def _vec_for_H(subset: Set[int], n: int, coeff: float) -> csr_matrix:
|
| 53 |
+
M = 2 ** n - 1
|
| 54 |
+
arr = np.zeros(M, dtype=np.float64)
|
| 55 |
+
arr[_subset_to_index(subset, n)] = coeff
|
| 56 |
+
return csr_matrix(arr.reshape(1, -1))
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def polym_ineq_to_vector(expr: str, n: int) -> csr_matrix:
|
| 60 |
+
"""
|
| 61 |
+
Convert shorthand info-theoretic expressions into an entropy vector.
|
| 62 |
+
|
| 63 |
+
Note: caller is expected to strip any trailing ">=0" etc.
|
| 64 |
+
"""
|
| 65 |
+
s = expr.replace(" ", "").strip()
|
| 66 |
+
|
| 67 |
+
# H(X_A|X_B) = H(A∪B) - H(B)
|
| 68 |
+
m = re.fullmatch(r"H\(X_(\{[^}]+\}|\d+)\|X_(\{[^}]+\}|\d+)\)", s)
|
| 69 |
+
if m:
|
| 70 |
+
A = _parse_X_term(m.group(1))
|
| 71 |
+
B = _parse_X_term(m.group(2))
|
| 72 |
+
if not A:
|
| 73 |
+
raise ValueError(f"invalid conditional entropy: {expr}")
|
| 74 |
+
return _vec_for_H(A | B, n, 1.0) + _vec_for_H(B, n, -1.0)
|
| 75 |
+
|
| 76 |
+
# I(X_A;X_B|X_C) = H(A∪C)+H(B∪C)-H(C)-H(A∪B∪C)
|
| 77 |
+
m = re.fullmatch(
|
| 78 |
+
r"I\(X_(\{[^}]+\}|\d+);X_(\{[^}]+\}|\d+)\|X_(\{[^}]+\}|\d+)\)", s
|
| 79 |
+
)
|
| 80 |
+
if m:
|
| 81 |
+
A = _parse_X_term(m.group(1))
|
| 82 |
+
B = _parse_X_term(m.group(2))
|
| 83 |
+
C = _parse_X_term(m.group(3))
|
| 84 |
+
if not A or not B:
|
| 85 |
+
raise ValueError(f"invalid conditional mutual information: {expr}")
|
| 86 |
+
return (
|
| 87 |
+
_vec_for_H(A | C, n, 1.0)
|
| 88 |
+
+ _vec_for_H(B | C, n, 1.0)
|
| 89 |
+
+ _vec_for_H(C, n, -1.0)
|
| 90 |
+
+ _vec_for_H(A | B | C, n, -1.0)
|
| 91 |
+
)
|
| 92 |
+
|
| 93 |
+
# I(X_A;X_B) = H(A)+H(B)-H(A∪B)
|
| 94 |
+
m = re.fullmatch(r"I\(X_(\{[^}]+\}|\d+);X_(\{[^}]+\}|\d+)\)", s)
|
| 95 |
+
if m:
|
| 96 |
+
A = _parse_X_term(m.group(1))
|
| 97 |
+
B = _parse_X_term(m.group(2))
|
| 98 |
+
if not A or not B:
|
| 99 |
+
raise ValueError(f"invalid mutual information: {expr}")
|
| 100 |
+
return _vec_for_H(A, n, 1.0) + _vec_for_H(B, n, 1.0) + _vec_for_H(A | B, n, -1.0)
|
| 101 |
+
|
| 102 |
+
raise ValueError(f"Unsupported polymatroid shorthand: {expr}")
|
| 103 |
+
|
| 104 |
+
|
mce220_runner/beam_search_model2/run_on_window_shear_jsonl_v12.py
ADDED
|
@@ -0,0 +1,821 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Batch-run beam_search tree_search on a JSONL dataset using V12 logic.
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
from __future__ import annotations
|
| 7 |
+
|
| 8 |
+
import argparse
|
| 9 |
+
import hashlib
|
| 10 |
+
import json
|
| 11 |
+
import os
|
| 12 |
+
import sys
|
| 13 |
+
import time
|
| 14 |
+
import uuid
|
| 15 |
+
from datetime import datetime, timezone
|
| 16 |
+
from pathlib import Path
|
| 17 |
+
from typing import Any, Dict, Optional, Tuple
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
EXECUTION_CONFIG_SCHEMA_VERSION = "e2n-execution-config-v1"
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def _is_fatal_gpu_failure(exc: BaseException) -> bool:
|
| 24 |
+
"""Return true for accelerator OOMs or abruptly terminated model workers."""
|
| 25 |
+
|
| 26 |
+
current: BaseException | None = exc
|
| 27 |
+
seen: set[int] = set()
|
| 28 |
+
while current is not None and id(current) not in seen:
|
| 29 |
+
seen.add(id(current))
|
| 30 |
+
class_name = type(current).__name__.lower()
|
| 31 |
+
message = str(current).lower()
|
| 32 |
+
if "brokenprocesspool" in class_name or "outofmemory" in class_name:
|
| 33 |
+
return True
|
| 34 |
+
if any(
|
| 35 |
+
fragment in message
|
| 36 |
+
for fragment in (
|
| 37 |
+
"cuda out of memory",
|
| 38 |
+
"cuda error: out of memory",
|
| 39 |
+
"hip out of memory",
|
| 40 |
+
"cublas_status_alloc_failed",
|
| 41 |
+
"cudnn_status_alloc_failed",
|
| 42 |
+
"process in the process pool was terminated abruptly",
|
| 43 |
+
)
|
| 44 |
+
):
|
| 45 |
+
return True
|
| 46 |
+
current = current.__cause__ or current.__context__
|
| 47 |
+
return False
|
| 48 |
+
|
| 49 |
+
try:
|
| 50 |
+
from verify_proof_certificate import verify_certificate
|
| 51 |
+
except ModuleNotFoundError:
|
| 52 |
+
from beam_search_model2.verify_proof_certificate import verify_certificate
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def _proof_trace_metrics(vec, score_fn) -> Dict[str, Any]:
|
| 56 |
+
import numpy as np
|
| 57 |
+
|
| 58 |
+
data = np.asarray(vec.data, dtype=float) if vec is not None else np.asarray([], dtype=float)
|
| 59 |
+
return {
|
| 60 |
+
"negative_l1": float(np.abs(data[data < 0]).sum()) if data.size else 0.0,
|
| 61 |
+
"nnz": int(vec.nnz) if vec is not None else 0,
|
| 62 |
+
"score": float(score_fn(vec)),
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def build_proof_trace(
|
| 67 |
+
target_inequality: str,
|
| 68 |
+
n: int,
|
| 69 |
+
selected_steps,
|
| 70 |
+
*,
|
| 71 |
+
search_reported_proved: bool,
|
| 72 |
+
) -> Dict[str, Any]:
|
| 73 |
+
"""Reconstruct a compact, auditable residual trajectory from a selected path."""
|
| 74 |
+
try:
|
| 75 |
+
from Data_Generation_via_LLM_Guided_Search_v12 import (
|
| 76 |
+
_get_residual_score,
|
| 77 |
+
parse_cand_to_vector,
|
| 78 |
+
vector_to_string,
|
| 79 |
+
)
|
| 80 |
+
except ModuleNotFoundError:
|
| 81 |
+
from beam_search_model2.Data_Generation_via_LLM_Guided_Search_v12 import (
|
| 82 |
+
_get_residual_score,
|
| 83 |
+
parse_cand_to_vector,
|
| 84 |
+
vector_to_string,
|
| 85 |
+
)
|
| 86 |
+
|
| 87 |
+
residual = parse_cand_to_vector(target_inequality, n)
|
| 88 |
+
trace_steps = []
|
| 89 |
+
for index, inequality in enumerate(selected_steps or [], start=1):
|
| 90 |
+
before = residual
|
| 91 |
+
after = before - parse_cand_to_vector(str(inequality), n)
|
| 92 |
+
before_metrics = _proof_trace_metrics(before, _get_residual_score)
|
| 93 |
+
after_metrics = _proof_trace_metrics(after, _get_residual_score)
|
| 94 |
+
trace_steps.append(
|
| 95 |
+
{
|
| 96 |
+
"index": index,
|
| 97 |
+
"inequality": str(inequality),
|
| 98 |
+
"residual_before": vector_to_string(before),
|
| 99 |
+
"residual_after": vector_to_string(after),
|
| 100 |
+
"before": before_metrics,
|
| 101 |
+
"after": after_metrics,
|
| 102 |
+
"score_improvement": before_metrics["score"] - after_metrics["score"],
|
| 103 |
+
}
|
| 104 |
+
)
|
| 105 |
+
residual = after
|
| 106 |
+
|
| 107 |
+
terminal_metrics = _proof_trace_metrics(residual, _get_residual_score)
|
| 108 |
+
if terminal_metrics["nnz"] == 0:
|
| 109 |
+
certificate_kind = "zero_residual"
|
| 110 |
+
elif residual is not None and (residual.data >= -1e-9).all():
|
| 111 |
+
certificate_kind = "coefficientwise_nonnegative"
|
| 112 |
+
elif search_reported_proved:
|
| 113 |
+
certificate_kind = "reported_proved_without_explicit_certificate"
|
| 114 |
+
else:
|
| 115 |
+
certificate_kind = "not_certified"
|
| 116 |
+
return {
|
| 117 |
+
"steps": trace_steps,
|
| 118 |
+
"terminal": {
|
| 119 |
+
"residual": vector_to_string(residual),
|
| 120 |
+
**terminal_metrics,
|
| 121 |
+
"certificate_kind": certificate_kind,
|
| 122 |
+
},
|
| 123 |
+
}
|
| 124 |
+
|
| 125 |
+
|
| 126 |
+
def attach_proof_trace(
|
| 127 |
+
run_output: Dict[str, Any],
|
| 128 |
+
target_inequality: str,
|
| 129 |
+
n: int,
|
| 130 |
+
selected_steps,
|
| 131 |
+
*,
|
| 132 |
+
search_reported_proved: bool,
|
| 133 |
+
) -> None:
|
| 134 |
+
"""Attach a trace without allowing diagnostic failure to abort a batch."""
|
| 135 |
+
try:
|
| 136 |
+
run_output["proof_trace"] = build_proof_trace(
|
| 137 |
+
target_inequality,
|
| 138 |
+
n,
|
| 139 |
+
selected_steps,
|
| 140 |
+
search_reported_proved=search_reported_proved,
|
| 141 |
+
)
|
| 142 |
+
except Exception as exc:
|
| 143 |
+
run_output["proof_trace_error"] = f"{type(exc).__name__}: {exc}"
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
def _exact_verification_summary(result) -> Dict[str, Any]:
|
| 147 |
+
"""Convert an exact verifier result into stable JSON-compatible diagnostics."""
|
| 148 |
+
return {
|
| 149 |
+
"status": "valid" if result.valid else "invalid",
|
| 150 |
+
"valid": bool(result.valid),
|
| 151 |
+
"reason": result.reason,
|
| 152 |
+
"terminal_kind": result.terminal_kind,
|
| 153 |
+
"step_count": len(result.steps),
|
| 154 |
+
"steps": [
|
| 155 |
+
{
|
| 156 |
+
"kind": step.kind,
|
| 157 |
+
"multiplier": str(step.multiplier),
|
| 158 |
+
"condition_index": step.condition_index,
|
| 159 |
+
}
|
| 160 |
+
for step in result.steps
|
| 161 |
+
],
|
| 162 |
+
"residual": [
|
| 163 |
+
{"subset_mask": mask, "coefficient": str(coefficient)}
|
| 164 |
+
for mask, coefficient in sorted(result.residual.items())
|
| 165 |
+
],
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
|
| 169 |
+
def apply_exact_verification_gate(
|
| 170 |
+
run_output: Dict[str, Any],
|
| 171 |
+
target_inequality: str,
|
| 172 |
+
conditions,
|
| 173 |
+
n: int,
|
| 174 |
+
selected_steps,
|
| 175 |
+
*,
|
| 176 |
+
search_reported_proved: bool,
|
| 177 |
+
) -> None:
|
| 178 |
+
"""Require an exact symbolic certificate for every search-reported success."""
|
| 179 |
+
if not search_reported_proved:
|
| 180 |
+
return
|
| 181 |
+
verification_start = time.perf_counter()
|
| 182 |
+
try:
|
| 183 |
+
result = verify_certificate(
|
| 184 |
+
n,
|
| 185 |
+
target_inequality,
|
| 186 |
+
list(conditions or []),
|
| 187 |
+
list(selected_steps) if isinstance(selected_steps, list) else selected_steps,
|
| 188 |
+
)
|
| 189 |
+
summary = _exact_verification_summary(result)
|
| 190 |
+
except Exception as exc:
|
| 191 |
+
summary = {
|
| 192 |
+
"status": "error",
|
| 193 |
+
"valid": False,
|
| 194 |
+
"reason": f"{type(exc).__name__}: {exc}",
|
| 195 |
+
"terminal_kind": None,
|
| 196 |
+
"step_count": len(selected_steps) if isinstance(selected_steps, list) else 0,
|
| 197 |
+
"steps": [],
|
| 198 |
+
"residual": [],
|
| 199 |
+
}
|
| 200 |
+
summary["time_sec"] = max(time.perf_counter() - verification_start, 0.0)
|
| 201 |
+
run_output["exact_verification"] = summary
|
| 202 |
+
if summary["valid"]:
|
| 203 |
+
run_output["proved"] = True
|
| 204 |
+
return
|
| 205 |
+
|
| 206 |
+
run_output["proved"] = False
|
| 207 |
+
failure = f"Exact verification {summary['status']}: {summary['reason']}"
|
| 208 |
+
existing_error = run_output.get("error")
|
| 209 |
+
run_output["error"] = f"{existing_error}; {failure}" if existing_error else failure
|
| 210 |
+
|
| 211 |
+
|
| 212 |
+
def _parse_per_n_budget(spec: str) -> Dict[int, Tuple[int, float]]:
|
| 213 |
+
budgets: Dict[int, Tuple[int, float]] = {}
|
| 214 |
+
raw_spec = (spec or "").strip()
|
| 215 |
+
if not raw_spec:
|
| 216 |
+
return budgets
|
| 217 |
+
|
| 218 |
+
for chunk in raw_spec.split(";"):
|
| 219 |
+
chunk = chunk.strip()
|
| 220 |
+
if not chunk:
|
| 221 |
+
continue
|
| 222 |
+
try:
|
| 223 |
+
n_str, caps_str = chunk.split(":", 1)
|
| 224 |
+
tree_str, time_str = [part.strip() for part in caps_str.split(",", 1)]
|
| 225 |
+
budgets[int(n_str.strip())] = (int(tree_str), float(time_str))
|
| 226 |
+
except Exception as exc:
|
| 227 |
+
raise SystemExit(
|
| 228 |
+
"Invalid --per-n-budget format. Expected '8:3134,1130;9:3192,1499'. "
|
| 229 |
+
f"Failed at chunk {chunk!r}: {exc}"
|
| 230 |
+
)
|
| 231 |
+
return budgets
|
| 232 |
+
|
| 233 |
+
|
| 234 |
+
def _resolve_sample_budget(
|
| 235 |
+
n: int,
|
| 236 |
+
per_n_budget: Dict[int, Tuple[int, float]],
|
| 237 |
+
fallback_tree_size: int,
|
| 238 |
+
fallback_time_sec: float,
|
| 239 |
+
) -> Tuple[int, float]:
|
| 240 |
+
if n in per_n_budget:
|
| 241 |
+
return per_n_budget[n]
|
| 242 |
+
return fallback_tree_size, fallback_time_sec
|
| 243 |
+
|
| 244 |
+
|
| 245 |
+
def _direct_execution_config(args: argparse.Namespace) -> Dict[str, Any]:
|
| 246 |
+
"""Self-describe direct runner use when no reviewer wrapper is present."""
|
| 247 |
+
|
| 248 |
+
gpu_ids = []
|
| 249 |
+
raw_gpu_ids = str(
|
| 250 |
+
args.gpu_devices or os.environ.get("CUDA_VISIBLE_DEVICES") or ""
|
| 251 |
+
).strip()
|
| 252 |
+
if raw_gpu_ids:
|
| 253 |
+
gpu_ids = [
|
| 254 |
+
int(part.strip())
|
| 255 |
+
for part in raw_gpu_ids.replace(" ", ",").split(",")
|
| 256 |
+
if part.strip()
|
| 257 |
+
][: int(args.gpu_workers or 0)]
|
| 258 |
+
elif args.gpu_workers:
|
| 259 |
+
gpu_ids = list(range(int(args.gpu_workers)))
|
| 260 |
+
normalized = list(sys.argv)
|
| 261 |
+
if "--execution-config-manifest" in normalized:
|
| 262 |
+
index = normalized.index("--execution-config-manifest")
|
| 263 |
+
if index + 1 < len(normalized):
|
| 264 |
+
normalized[index + 1] = "<live-wrapper-manifest>"
|
| 265 |
+
return {
|
| 266 |
+
"schema_version": EXECUTION_CONFIG_SCHEMA_VERSION,
|
| 267 |
+
"invocation_id": uuid.uuid4().hex,
|
| 268 |
+
"recorded_utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
|
| 269 |
+
"origin": "direct_runner",
|
| 270 |
+
"code": {
|
| 271 |
+
"git_commit": None,
|
| 272 |
+
"repo_root": None,
|
| 273 |
+
"runner_path": str(Path(__file__).resolve()),
|
| 274 |
+
"runner_sha256": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
|
| 275 |
+
},
|
| 276 |
+
"model": {
|
| 277 |
+
"checkpoint_path": str(Path(args.adapter_path).expanduser().resolve())
|
| 278 |
+
if args.adapter_path
|
| 279 |
+
else None,
|
| 280 |
+
"checkpoint_sha256": None,
|
| 281 |
+
"base_model": None,
|
| 282 |
+
},
|
| 283 |
+
"search": {
|
| 284 |
+
key: value
|
| 285 |
+
for key, value in vars(args).items()
|
| 286 |
+
if key != "execution_config_manifest"
|
| 287 |
+
},
|
| 288 |
+
"resources": {
|
| 289 |
+
"workers": int(args.gpu_workers or 0),
|
| 290 |
+
"gpu_ids": gpu_ids,
|
| 291 |
+
"worker_device_mapping": [
|
| 292 |
+
{
|
| 293 |
+
"worker_index": index,
|
| 294 |
+
"physical_gpu_index": gpu_id,
|
| 295 |
+
"gpu_uuid": None,
|
| 296 |
+
}
|
| 297 |
+
for index, gpu_id in enumerate(gpu_ids)
|
| 298 |
+
],
|
| 299 |
+
"cuda_visible_devices": os.environ.get("CUDA_VISIBLE_DEVICES"),
|
| 300 |
+
"launch_gpu_snapshot": [],
|
| 301 |
+
},
|
| 302 |
+
"normalized_command": normalized,
|
| 303 |
+
}
|
| 304 |
+
|
| 305 |
+
|
| 306 |
+
def _load_execution_config(
|
| 307 |
+
manifest_path: str, args: argparse.Namespace
|
| 308 |
+
) -> Dict[str, Any]:
|
| 309 |
+
if not manifest_path:
|
| 310 |
+
return _direct_execution_config(args)
|
| 311 |
+
path = Path(manifest_path).expanduser().resolve()
|
| 312 |
+
try:
|
| 313 |
+
manifest = json.loads(path.read_text(encoding="utf-8"))
|
| 314 |
+
except (OSError, json.JSONDecodeError) as exc:
|
| 315 |
+
raise SystemExit(f"Invalid --execution-config-manifest: {exc}") from exc
|
| 316 |
+
config = manifest.get("execution_config") if isinstance(manifest, dict) else None
|
| 317 |
+
if not isinstance(config, dict):
|
| 318 |
+
raise SystemExit("Execution-config manifest has no object execution_config")
|
| 319 |
+
if config.get("schema_version") != EXECUTION_CONFIG_SCHEMA_VERSION:
|
| 320 |
+
raise SystemExit(
|
| 321 |
+
"Unsupported execution_config schema: "
|
| 322 |
+
f"{config.get('schema_version')!r}"
|
| 323 |
+
)
|
| 324 |
+
required = {
|
| 325 |
+
"invocation_id",
|
| 326 |
+
"recorded_utc",
|
| 327 |
+
"origin",
|
| 328 |
+
"code",
|
| 329 |
+
"model",
|
| 330 |
+
"search",
|
| 331 |
+
"resources",
|
| 332 |
+
"normalized_command",
|
| 333 |
+
}
|
| 334 |
+
missing = sorted(required.difference(config))
|
| 335 |
+
if missing:
|
| 336 |
+
raise SystemExit(f"Execution config is missing fields: {missing}")
|
| 337 |
+
# Round-trip once so every row receives a detached, JSON-safe value.
|
| 338 |
+
return json.loads(json.dumps(config, ensure_ascii=False))
|
| 339 |
+
|
| 340 |
+
|
| 341 |
+
def _append_jsonl_row(path: Path, row: Dict[str, Any]) -> None:
|
| 342 |
+
"""Append one complete row with one O_APPEND write, then fsync it."""
|
| 343 |
+
|
| 344 |
+
payload = (json.dumps(row, ensure_ascii=False) + "\n").encode("utf-8")
|
| 345 |
+
descriptor = os.open(
|
| 346 |
+
path,
|
| 347 |
+
os.O_WRONLY | os.O_CREAT | os.O_APPEND,
|
| 348 |
+
0o600,
|
| 349 |
+
)
|
| 350 |
+
try:
|
| 351 |
+
written = os.write(descriptor, payload)
|
| 352 |
+
if written != len(payload):
|
| 353 |
+
raise OSError(
|
| 354 |
+
f"short atomic JSONL append: wrote {written} of {len(payload)} bytes"
|
| 355 |
+
)
|
| 356 |
+
os.fsync(descriptor)
|
| 357 |
+
finally:
|
| 358 |
+
os.close(descriptor)
|
| 359 |
+
|
| 360 |
+
|
| 361 |
+
def main() -> int:
|
| 362 |
+
ap = argparse.ArgumentParser()
|
| 363 |
+
ap.add_argument("--input", required=True, help="Input JSONL path")
|
| 364 |
+
ap.add_argument("--output", required=True, help="Output JSONL path")
|
| 365 |
+
ap.add_argument(
|
| 366 |
+
"--execution-config-manifest",
|
| 367 |
+
default="",
|
| 368 |
+
help="Live wrapper manifest containing the effective execution_config.",
|
| 369 |
+
)
|
| 370 |
+
ap.add_argument("--max-depth", type=int, default=10)
|
| 371 |
+
ap.add_argument("--rollouts", type=int, default=2)
|
| 372 |
+
ap.add_argument(
|
| 373 |
+
"--k-per-node",
|
| 374 |
+
type=int,
|
| 375 |
+
default=1,
|
| 376 |
+
help=(
|
| 377 |
+
"Number of sequential LLM generate+append steps per node before one LP check "
|
| 378 |
+
"(default: 1)."
|
| 379 |
+
),
|
| 380 |
+
)
|
| 381 |
+
ap.add_argument(
|
| 382 |
+
"--llm-mode",
|
| 383 |
+
choices=["sequential", "batch"],
|
| 384 |
+
default="sequential",
|
| 385 |
+
help="LLM generation mode: sequential (k conditioned steps) or batch (one GPU call returns k candidates).",
|
| 386 |
+
)
|
| 387 |
+
ap.add_argument(
|
| 388 |
+
"--lp-workers",
|
| 389 |
+
type=int,
|
| 390 |
+
default=0,
|
| 391 |
+
help=(
|
| 392 |
+
"Reserved for command-line compatibility. Current v12 LP checks are "
|
| 393 |
+
"synchronous; this value does not create an LP process pool."
|
| 394 |
+
),
|
| 395 |
+
)
|
| 396 |
+
ap.add_argument(
|
| 397 |
+
"--gpu-workers",
|
| 398 |
+
type=int,
|
| 399 |
+
default=0,
|
| 400 |
+
help=(
|
| 401 |
+
"If >0, run depth-parallel LLM expansion using this many GPU worker processes "
|
| 402 |
+
"(one model instance per process / per GPU)."
|
| 403 |
+
),
|
| 404 |
+
)
|
| 405 |
+
ap.add_argument(
|
| 406 |
+
"--gpu-devices",
|
| 407 |
+
default="",
|
| 408 |
+
help=(
|
| 409 |
+
"Comma-separated list of GPU ids to use for LLM workers (e.g. '0,1,2,3'). "
|
| 410 |
+
"If omitted, will try to infer from CUDA_VISIBLE_DEVICES."
|
| 411 |
+
),
|
| 412 |
+
)
|
| 413 |
+
ap.add_argument("--limit", type=int, default=0, help="If >0, only run first N samples")
|
| 414 |
+
ap.add_argument(
|
| 415 |
+
"--min-n",
|
| 416 |
+
type=int,
|
| 417 |
+
default=0,
|
| 418 |
+
help="If >0, skip samples with fewer than this many variables.",
|
| 419 |
+
)
|
| 420 |
+
ap.add_argument(
|
| 421 |
+
"--max-n",
|
| 422 |
+
type=int,
|
| 423 |
+
default=0,
|
| 424 |
+
help="If >0, skip samples with more than this many variables.",
|
| 425 |
+
)
|
| 426 |
+
ap.add_argument(
|
| 427 |
+
"--adapter-path",
|
| 428 |
+
default=os.environ.get("QWEN_ADAPTER_PATH", ""),
|
| 429 |
+
help="Path to PEFT adapter dir (defaults to env QWEN_ADAPTER_PATH)",
|
| 430 |
+
)
|
| 431 |
+
ap.add_argument("--use-dummy", action="store_true", help="Use dummy LLM outputs (no model load)")
|
| 432 |
+
ap.add_argument(
|
| 433 |
+
"--max-new-tokens",
|
| 434 |
+
type=int,
|
| 435 |
+
default=int(os.environ.get("QWEN_MAX_NEW_TOKENS", "1024")),
|
| 436 |
+
help="LLM generation cap",
|
| 437 |
+
)
|
| 438 |
+
ap.add_argument(
|
| 439 |
+
"--max-time",
|
| 440 |
+
type=float,
|
| 441 |
+
default=float(os.environ.get("QWEN_MAX_TIME", "20.0")),
|
| 442 |
+
help="LLM generation time cap (seconds)",
|
| 443 |
+
)
|
| 444 |
+
ap.add_argument(
|
| 445 |
+
"--sample-max-time-sec",
|
| 446 |
+
type=float,
|
| 447 |
+
default=0.0,
|
| 448 |
+
help=(
|
| 449 |
+
"Soft cap on elapsed search time. Work already in flight can finish after "
|
| 450 |
+
"the limit."
|
| 451 |
+
),
|
| 452 |
+
)
|
| 453 |
+
ap.add_argument(
|
| 454 |
+
"--sample-max-tree-size",
|
| 455 |
+
type=int,
|
| 456 |
+
default=0,
|
| 457 |
+
help="Soft cap on explored tree size, checked between completed expansions.",
|
| 458 |
+
)
|
| 459 |
+
ap.add_argument(
|
| 460 |
+
"--sample-max-llm-calls",
|
| 461 |
+
type=int,
|
| 462 |
+
default=0,
|
| 463 |
+
help=(
|
| 464 |
+
"Soft cap on aggregated LLM calls. Concurrent or multi-step expansions can "
|
| 465 |
+
"overshoot it."
|
| 466 |
+
),
|
| 467 |
+
)
|
| 468 |
+
ap.add_argument(
|
| 469 |
+
"--sample-max-llm-fails",
|
| 470 |
+
type=int,
|
| 471 |
+
default=0,
|
| 472 |
+
help="If >0, stop a sample once llm_usage.format_fail_total reaches this limit.",
|
| 473 |
+
)
|
| 474 |
+
ap.add_argument(
|
| 475 |
+
"--depth1-rollouts",
|
| 476 |
+
type=int,
|
| 477 |
+
default=0,
|
| 478 |
+
help="If >0, override the special depth=1 rollout count (default keeps existing value 32).",
|
| 479 |
+
)
|
| 480 |
+
ap.add_argument(
|
| 481 |
+
"--per-n-budget",
|
| 482 |
+
default="",
|
| 483 |
+
help="Per-n tree/time caps in the form '8:3134,1130;9:3192,1499'.",
|
| 484 |
+
)
|
| 485 |
+
ap.add_argument(
|
| 486 |
+
"--append",
|
| 487 |
+
action="store_true",
|
| 488 |
+
help="Append to the output JSONL instead of overwriting it.",
|
| 489 |
+
)
|
| 490 |
+
ap.add_argument(
|
| 491 |
+
"--resume-from-output",
|
| 492 |
+
action="store_true",
|
| 493 |
+
help="Resume by skipping the first N eligible samples, where N is the number of existing rows in the output JSONL.",
|
| 494 |
+
)
|
| 495 |
+
ap.add_argument(
|
| 496 |
+
"--log-every",
|
| 497 |
+
type=int,
|
| 498 |
+
default=1,
|
| 499 |
+
help="Print progress every N samples (default: 1)",
|
| 500 |
+
)
|
| 501 |
+
ap.add_argument(
|
| 502 |
+
"--print-llm",
|
| 503 |
+
action="store_true",
|
| 504 |
+
help="Print raw LLM generation and extracted inequalities each call (to stderr)",
|
| 505 |
+
)
|
| 506 |
+
ap.add_argument(
|
| 507 |
+
"--debug-search",
|
| 508 |
+
action="store_true",
|
| 509 |
+
help="Enable extra debug printing from tree_search (LP status / parse warnings)",
|
| 510 |
+
)
|
| 511 |
+
ap.add_argument(
|
| 512 |
+
"--beam-size",
|
| 513 |
+
type=int,
|
| 514 |
+
default=0,
|
| 515 |
+
help="If >0, keep at most this many nodes (paths) per depth to avoid explosion",
|
| 516 |
+
)
|
| 517 |
+
ap.add_argument(
|
| 518 |
+
"--random-prune",
|
| 519 |
+
action="store_true",
|
| 520 |
+
help="When beam-size is exceeded, uniformly sample nodes without replacement.",
|
| 521 |
+
)
|
| 522 |
+
ap.add_argument(
|
| 523 |
+
"--include-proof-trace",
|
| 524 |
+
action="store_true",
|
| 525 |
+
help="Include residual and score diagnostics for every selected proof step.",
|
| 526 |
+
)
|
| 527 |
+
ap.add_argument(
|
| 528 |
+
"--prune-seed",
|
| 529 |
+
type=int,
|
| 530 |
+
default=0,
|
| 531 |
+
help="Seed for random pruning (0 means nondeterministic)",
|
| 532 |
+
)
|
| 533 |
+
ap.add_argument(
|
| 534 |
+
"--seed",
|
| 535 |
+
type=int,
|
| 536 |
+
default=0,
|
| 537 |
+
help="Global seed for random, numpy, and torch (0 means nondeterministic)",
|
| 538 |
+
)
|
| 539 |
+
ap.add_argument(
|
| 540 |
+
"--log-wrong-pruned",
|
| 541 |
+
action="store_true",
|
| 542 |
+
help="Enable logging of pruned model outputs (invalid/negative/malformed, including multi-term expressions) to a JSONL file",
|
| 543 |
+
)
|
| 544 |
+
ap.add_argument(
|
| 545 |
+
"--wrong-pruned-path",
|
| 546 |
+
type=str,
|
| 547 |
+
default="",
|
| 548 |
+
help="Path to write the pruned malformed / wrong expressions JSONL dataset",
|
| 549 |
+
)
|
| 550 |
+
ap.add_argument(
|
| 551 |
+
"--enable-lp",
|
| 552 |
+
action="store_true",
|
| 553 |
+
help="Enable Shannon LP check.",
|
| 554 |
+
)
|
| 555 |
+
ap.add_argument(
|
| 556 |
+
"--max-vars-for-lp",
|
| 557 |
+
type=int,
|
| 558 |
+
default=6,
|
| 559 |
+
help="Maximum number of variables for full Shannon LP check (default: 6)",
|
| 560 |
+
)
|
| 561 |
+
args = ap.parse_args()
|
| 562 |
+
execution_config = _load_execution_config(args.execution_config_manifest, args)
|
| 563 |
+
|
| 564 |
+
if args.lp_workers and args.lp_workers > 0:
|
| 565 |
+
print(
|
| 566 |
+
"warning: --lp-workers is retained for compatibility but is not implemented "
|
| 567 |
+
"by the v12 search; LP checks remain synchronous",
|
| 568 |
+
file=sys.stderr,
|
| 569 |
+
)
|
| 570 |
+
|
| 571 |
+
if args.seed and args.seed != 0:
|
| 572 |
+
import random as _random
|
| 573 |
+
import numpy as _np
|
| 574 |
+
import torch as _torch
|
| 575 |
+
_random.seed(args.seed)
|
| 576 |
+
_np.random.seed(args.seed)
|
| 577 |
+
_torch.manual_seed(args.seed)
|
| 578 |
+
_torch.cuda.manual_seed_all(args.seed)
|
| 579 |
+
print(f"Global seed set to {args.seed}")
|
| 580 |
+
|
| 581 |
+
per_n_budget = _parse_per_n_budget(args.per_n_budget)
|
| 582 |
+
depth1_rollouts = args.depth1_rollouts if args.depth1_rollouts > 0 else None
|
| 583 |
+
budget_enabled = bool(per_n_budget) or any(
|
| 584 |
+
[
|
| 585 |
+
(args.sample_max_time_sec and args.sample_max_time_sec > 0),
|
| 586 |
+
(args.sample_max_tree_size and args.sample_max_tree_size > 0),
|
| 587 |
+
(args.sample_max_llm_calls and args.sample_max_llm_calls > 0),
|
| 588 |
+
(args.sample_max_llm_fails and args.sample_max_llm_fails > 0),
|
| 589 |
+
(depth1_rollouts is not None),
|
| 590 |
+
]
|
| 591 |
+
)
|
| 592 |
+
|
| 593 |
+
gpu_devices = None
|
| 594 |
+
if args.gpu_workers and args.gpu_workers > 0:
|
| 595 |
+
dev_str = (args.gpu_devices or "").strip()
|
| 596 |
+
if not dev_str:
|
| 597 |
+
dev_str = (os.environ.get("CUDA_VISIBLE_DEVICES") or "").strip()
|
| 598 |
+
if dev_str:
|
| 599 |
+
parts = [p.strip() for p in dev_str.replace(" ", ",").split(",") if p.strip()]
|
| 600 |
+
try:
|
| 601 |
+
gpu_devices = [int(p) for p in parts]
|
| 602 |
+
except Exception:
|
| 603 |
+
raise SystemExit(f"Invalid --gpu-devices / CUDA_VISIBLE_DEVICES format: {dev_str!r}")
|
| 604 |
+
else:
|
| 605 |
+
gpu_devices = list(range(int(args.gpu_workers)))
|
| 606 |
+
|
| 607 |
+
if len(gpu_devices) < int(args.gpu_workers):
|
| 608 |
+
raise SystemExit(
|
| 609 |
+
f"--gpu-workers={args.gpu_workers} but only {len(gpu_devices)} devices provided: {gpu_devices}"
|
| 610 |
+
)
|
| 611 |
+
gpu_devices = gpu_devices[: int(args.gpu_workers)]
|
| 612 |
+
|
| 613 |
+
from Data_Generation_via_LLM_Guided_Search_v12 import Node, tree_search
|
| 614 |
+
|
| 615 |
+
in_path = Path(args.input)
|
| 616 |
+
out_path = Path(args.output)
|
| 617 |
+
out_path.parent.mkdir(parents=True, exist_ok=True)
|
| 618 |
+
|
| 619 |
+
if args.log_wrong_pruned and not args.wrong_pruned_path:
|
| 620 |
+
args.wrong_pruned_path = str(out_path.parent / (out_path.stem + "_wrong_pruned.jsonl"))
|
| 621 |
+
|
| 622 |
+
total = 0
|
| 623 |
+
proved_cnt = 0
|
| 624 |
+
t_start = time.perf_counter()
|
| 625 |
+
resume_skip_remaining = 0
|
| 626 |
+
if args.resume_from_output and out_path.exists():
|
| 627 |
+
with out_path.open("r", encoding="utf-8") as existing_f:
|
| 628 |
+
resume_skip_remaining = sum(1 for line in existing_f if line.strip())
|
| 629 |
+
if resume_skip_remaining:
|
| 630 |
+
print(
|
| 631 |
+
f"Resuming from existing output: skipping first {resume_skip_remaining} eligible samples from {out_path}"
|
| 632 |
+
)
|
| 633 |
+
|
| 634 |
+
try:
|
| 635 |
+
if not args.append:
|
| 636 |
+
descriptor = os.open(
|
| 637 |
+
out_path,
|
| 638 |
+
os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
|
| 639 |
+
0o600,
|
| 640 |
+
)
|
| 641 |
+
try:
|
| 642 |
+
os.fsync(descriptor)
|
| 643 |
+
finally:
|
| 644 |
+
os.close(descriptor)
|
| 645 |
+
with in_path.open("r", encoding="utf-8") as fin:
|
| 646 |
+
for source_line_number, line in enumerate(fin, start=1):
|
| 647 |
+
line = line.strip()
|
| 648 |
+
if not line:
|
| 649 |
+
continue
|
| 650 |
+
input_sha256 = hashlib.sha256(line.encode("utf-8")).hexdigest()
|
| 651 |
+
|
| 652 |
+
obj: Dict[str, Any] = json.loads(line)
|
| 653 |
+
n = int(obj["n"])
|
| 654 |
+
if args.min_n and n < args.min_n:
|
| 655 |
+
continue
|
| 656 |
+
if args.max_n and n > args.max_n:
|
| 657 |
+
continue
|
| 658 |
+
if resume_skip_remaining > 0:
|
| 659 |
+
resume_skip_remaining -= 1
|
| 660 |
+
continue
|
| 661 |
+
ineq = str(obj["ineq"])
|
| 662 |
+
conditions = obj.get("conditions", [])
|
| 663 |
+
|
| 664 |
+
ineq_map, conds_map, mapping = ineq, conditions, {}
|
| 665 |
+
tree_cap, time_cap = _resolve_sample_budget(
|
| 666 |
+
n=n,
|
| 667 |
+
per_n_budget=per_n_budget,
|
| 668 |
+
fallback_tree_size=args.sample_max_tree_size,
|
| 669 |
+
fallback_time_sec=args.sample_max_time_sec,
|
| 670 |
+
)
|
| 671 |
+
llm_cap = int(args.sample_max_llm_calls or 0)
|
| 672 |
+
llm_fail_cap = int(args.sample_max_llm_fails or 0)
|
| 673 |
+
|
| 674 |
+
root = Node()
|
| 675 |
+
t0 = time.perf_counter()
|
| 676 |
+
try:
|
| 677 |
+
os.environ["QWEN_MAX_NEW_TOKENS"] = str(args.max_new_tokens)
|
| 678 |
+
os.environ["QWEN_MAX_TIME"] = str(args.max_time)
|
| 679 |
+
os.environ["QWEN_PRINT_OUTPUT"] = "1" if args.print_llm else "0"
|
| 680 |
+
os.environ["BEAM_DEBUG"] = "1" if args.debug_search else "0"
|
| 681 |
+
if args.lp_workers and args.lp_workers > 0:
|
| 682 |
+
os.environ.setdefault("OMP_NUM_THREADS", "1")
|
| 683 |
+
os.environ.setdefault("MKL_NUM_THREADS", "1")
|
| 684 |
+
os.environ.setdefault("OPENBLAS_NUM_THREADS", "1")
|
| 685 |
+
|
| 686 |
+
proved, tree, x, expr_str, llm_usage, stop_reason = tree_search(
|
| 687 |
+
root_node=root,
|
| 688 |
+
max_depth=args.max_depth,
|
| 689 |
+
rollout_count=args.rollouts,
|
| 690 |
+
target_inequality=ineq_map,
|
| 691 |
+
conditions=conds_map,
|
| 692 |
+
n=n,
|
| 693 |
+
adapter_path=(args.adapter_path or None),
|
| 694 |
+
use_dummy=args.use_dummy,
|
| 695 |
+
k_per_call=args.k_per_node,
|
| 696 |
+
llm_mode=args.llm_mode,
|
| 697 |
+
lp_workers=args.lp_workers,
|
| 698 |
+
gpu_devices=gpu_devices,
|
| 699 |
+
beam_size=args.beam_size,
|
| 700 |
+
random_prune=args.random_prune,
|
| 701 |
+
prune_seed=(args.prune_seed if args.prune_seed != 0 else None),
|
| 702 |
+
max_vars_for_lp=(args.max_vars_for_lp if args.enable_lp else 0),
|
| 703 |
+
log_wrong_pruned=args.log_wrong_pruned,
|
| 704 |
+
wrong_pruned_path=args.wrong_pruned_path,
|
| 705 |
+
sample_time_budget_sec=time_cap,
|
| 706 |
+
sample_tree_budget=tree_cap,
|
| 707 |
+
sample_llm_calls_budget=llm_cap,
|
| 708 |
+
sample_llm_fail_budget=llm_fail_cap,
|
| 709 |
+
depth1_rollouts=depth1_rollouts,
|
| 710 |
+
)
|
| 711 |
+
err = None
|
| 712 |
+
except Exception as e:
|
| 713 |
+
proved, tree, x, expr_str, llm_usage, stop_reason = False, None, None, None, None, None
|
| 714 |
+
err = f"{type(e).__name__}: {e}"
|
| 715 |
+
fatal_gpu_failure = _is_fatal_gpu_failure(e)
|
| 716 |
+
if fatal_gpu_failure:
|
| 717 |
+
stop_reason = "fatal_gpu_failure"
|
| 718 |
+
else:
|
| 719 |
+
fatal_gpu_failure = False
|
| 720 |
+
|
| 721 |
+
search_dt = time.perf_counter() - t0
|
| 722 |
+
total += 1
|
| 723 |
+
search_reported_proved = bool(proved)
|
| 724 |
+
|
| 725 |
+
out_obj = {
|
| 726 |
+
"source_line_number": source_line_number,
|
| 727 |
+
"input_sha256": input_sha256,
|
| 728 |
+
"input": obj,
|
| 729 |
+
"mapped": {"ineq": ineq_map, "n": n, "mapping": mapping},
|
| 730 |
+
"run": {
|
| 731 |
+
"proved": bool(proved),
|
| 732 |
+
"time_sec": search_dt,
|
| 733 |
+
"search_time_sec": search_dt,
|
| 734 |
+
"tree_size": len(tree) if tree is not None else 0,
|
| 735 |
+
"expr_str": expr_str,
|
| 736 |
+
"x": x.tolist() if hasattr(x, "tolist") else x,
|
| 737 |
+
"error": err,
|
| 738 |
+
"llm_usage": llm_usage or {},
|
| 739 |
+
"execution_config": execution_config,
|
| 740 |
+
},
|
| 741 |
+
}
|
| 742 |
+
apply_exact_verification_gate(
|
| 743 |
+
out_obj["run"],
|
| 744 |
+
ineq_map,
|
| 745 |
+
conds_map,
|
| 746 |
+
n,
|
| 747 |
+
expr_str,
|
| 748 |
+
search_reported_proved=search_reported_proved,
|
| 749 |
+
)
|
| 750 |
+
proved = bool(out_obj["run"]["proved"])
|
| 751 |
+
proved_cnt += int(proved)
|
| 752 |
+
if args.include_proof_trace:
|
| 753 |
+
attach_proof_trace(
|
| 754 |
+
out_obj["run"],
|
| 755 |
+
ineq_map,
|
| 756 |
+
n,
|
| 757 |
+
expr_str,
|
| 758 |
+
search_reported_proved=search_reported_proved,
|
| 759 |
+
)
|
| 760 |
+
if budget_enabled:
|
| 761 |
+
out_obj["run"]["stop_reason"] = stop_reason
|
| 762 |
+
out_obj["run"]["budget_applied"] = {
|
| 763 |
+
"tree_size": int(tree_cap or 0),
|
| 764 |
+
"time_sec": float(time_cap or 0.0),
|
| 765 |
+
"llm_calls": llm_cap,
|
| 766 |
+
"llm_fails": llm_fail_cap,
|
| 767 |
+
"depth1_rollouts": depth1_rollouts,
|
| 768 |
+
"enforcement": "soft_post_expansion",
|
| 769 |
+
}
|
| 770 |
+
elif fatal_gpu_failure:
|
| 771 |
+
out_obj["run"]["stop_reason"] = stop_reason
|
| 772 |
+
dt = time.perf_counter() - t0
|
| 773 |
+
out_obj["run"]["time_sec"] = dt
|
| 774 |
+
_append_jsonl_row(out_path, out_obj)
|
| 775 |
+
|
| 776 |
+
if fatal_gpu_failure:
|
| 777 |
+
sys.stderr.write(
|
| 778 |
+
"[runner] Fatal GPU OOM or worker crash; result prefix was "
|
| 779 |
+
"flushed and no further sample will be started.\n"
|
| 780 |
+
)
|
| 781 |
+
sys.stderr.flush()
|
| 782 |
+
return 12
|
| 783 |
+
|
| 784 |
+
if stop_reason in {"llm_fail_budget", "format_fail_budget"}:
|
| 785 |
+
sys.stderr.write(
|
| 786 |
+
"\n[runner] Sample hit format_fail_total budget. Exiting checkpoint evaluation.\n"
|
| 787 |
+
)
|
| 788 |
+
sys.exit(10)
|
| 789 |
+
|
| 790 |
+
if args.log_every and (total % args.log_every == 0):
|
| 791 |
+
now = time.perf_counter()
|
| 792 |
+
elapsed = now - t_start
|
| 793 |
+
avg = elapsed / max(total, 1)
|
| 794 |
+
msg = (
|
| 795 |
+
f"[progress] {total} samples | proved={proved_cnt} | "
|
| 796 |
+
f"last={dt:.2f}s avg={avg:.2f}s | "
|
| 797 |
+
f"max_depth={args.max_depth} rollouts={args.rollouts} "
|
| 798 |
+
f"max_new_tokens={args.max_new_tokens} max_time={args.max_time}"
|
| 799 |
+
)
|
| 800 |
+
if stop_reason:
|
| 801 |
+
msg += f" | stop_reason={stop_reason}"
|
| 802 |
+
meta = obj.get("meta") if isinstance(obj, dict) else None
|
| 803 |
+
if isinstance(meta, dict):
|
| 804 |
+
msg += f" | seed={meta.get('master_seed')} idx={meta.get('index_within_type')}"
|
| 805 |
+
msg += "\n"
|
| 806 |
+
sys.stderr.write(msg)
|
| 807 |
+
sys.stderr.flush()
|
| 808 |
+
|
| 809 |
+
if args.limit and total >= args.limit:
|
| 810 |
+
break
|
| 811 |
+
except KeyboardInterrupt:
|
| 812 |
+
print(f"\nInterrupted. partial output saved to {out_path}")
|
| 813 |
+
|
| 814 |
+
print(f"Done. total={total} proved={proved_cnt} output={out_path}")
|
| 815 |
+
if args.log_wrong_pruned:
|
| 816 |
+
print(f"Wrong expressions logged to: {args.wrong_pruned_path}")
|
| 817 |
+
return 0
|
| 818 |
+
|
| 819 |
+
|
| 820 |
+
if __name__ == "__main__":
|
| 821 |
+
raise SystemExit(main())
|
mce220_runner/beam_search_model2/verify_proof_certificate.py
ADDED
|
@@ -0,0 +1,557 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Exact, standalone verifier for v12 entropy proof certificates.
|
| 3 |
+
|
| 4 |
+
The verifier deliberately does not import the search implementation. It parses
|
| 5 |
+
the emitted symbolic expressions into sparse maps whose keys are nonempty
|
| 6 |
+
subset masks and whose values are :class:`fractions.Fraction` objects.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import argparse
|
| 12 |
+
import json
|
| 13 |
+
import re
|
| 14 |
+
import sys
|
| 15 |
+
from dataclasses import dataclass
|
| 16 |
+
from fractions import Fraction
|
| 17 |
+
from pathlib import Path
|
| 18 |
+
from typing import Dict, List, Mapping, Optional, Sequence, Tuple
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
Vector = Dict[int, Fraction]
|
| 22 |
+
|
| 23 |
+
_NUMBER = r"(?:\d+(?:\.\d*)?|\.\d+)"
|
| 24 |
+
_NUMBER_RE = re.compile(_NUMBER)
|
| 25 |
+
_COMPARATOR_RE = re.compile(r"(>=|<=|=|>|<)\s*0(?:\.0*)?\s*$")
|
| 26 |
+
_VARIABLE_RE = re.compile(r"X_(?:\{(\d+(?:,\d+)*)\}|(\d+))")
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
class ProofCertificateError(ValueError):
|
| 30 |
+
"""Base class for a malformed certificate or result row."""
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
class ProofSyntaxError(ProofCertificateError):
|
| 34 |
+
"""An entropy expression is outside the verifier's accepted grammar."""
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
@dataclass(frozen=True)
|
| 38 |
+
class Atom:
|
| 39 |
+
kind: str
|
| 40 |
+
groups: Tuple[int, ...]
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
@dataclass(frozen=True)
|
| 44 |
+
class ParsedExpression:
|
| 45 |
+
vector: Vector
|
| 46 |
+
atom: Optional[Atom]
|
| 47 |
+
atom_coefficient: Optional[Fraction]
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
@dataclass(frozen=True)
|
| 51 |
+
class ParsedConstraint:
|
| 52 |
+
expression: ParsedExpression
|
| 53 |
+
comparator: str
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
@dataclass(frozen=True)
|
| 57 |
+
class VerifiedStep:
|
| 58 |
+
expression: str
|
| 59 |
+
kind: str
|
| 60 |
+
multiplier: Fraction
|
| 61 |
+
condition_index: Optional[int] = None
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
@dataclass(frozen=True)
|
| 65 |
+
class VerificationResult:
|
| 66 |
+
valid: bool
|
| 67 |
+
reason: str
|
| 68 |
+
steps: Tuple[VerifiedStep, ...]
|
| 69 |
+
residual: Vector
|
| 70 |
+
terminal_kind: Optional[str]
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def _clean(vector: Mapping[int, Fraction]) -> Vector:
|
| 74 |
+
return {mask: value for mask, value in vector.items() if value}
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def _add(target: Vector, source: Mapping[int, Fraction], scale: Fraction = Fraction(1)) -> None:
|
| 78 |
+
for mask, value in source.items():
|
| 79 |
+
new_value = target.get(mask, Fraction(0)) + scale * value
|
| 80 |
+
if new_value:
|
| 81 |
+
target[mask] = new_value
|
| 82 |
+
else:
|
| 83 |
+
target.pop(mask, None)
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def _scaled(vector: Mapping[int, Fraction], scale: Fraction) -> Vector:
|
| 87 |
+
return _clean({mask: scale * value for mask, value in vector.items()})
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
def _parse_scalar(text: str) -> Fraction:
|
| 91 |
+
if text in ("", "+"):
|
| 92 |
+
return Fraction(1)
|
| 93 |
+
if text == "-":
|
| 94 |
+
return Fraction(-1)
|
| 95 |
+
if not re.fullmatch(rf"[+-]?{_NUMBER}", text):
|
| 96 |
+
raise ProofSyntaxError(f"invalid rational coefficient {text!r}")
|
| 97 |
+
# Fraction parses decimal strings exactly (for example, 0.1 as 1/10).
|
| 98 |
+
return Fraction(text)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def _matching_paren(text: str, opening: int) -> int:
|
| 102 |
+
depth = 0
|
| 103 |
+
for index in range(opening, len(text)):
|
| 104 |
+
char = text[index]
|
| 105 |
+
if char == "(":
|
| 106 |
+
depth += 1
|
| 107 |
+
elif char == ")":
|
| 108 |
+
depth -= 1
|
| 109 |
+
if depth == 0:
|
| 110 |
+
return index
|
| 111 |
+
if depth < 0:
|
| 112 |
+
break
|
| 113 |
+
raise ProofSyntaxError(f"unbalanced parentheses near {text[opening:]!r}")
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
def _outer_scalar(text: str) -> Optional[Tuple[Fraction, str]]:
|
| 117 |
+
opening = text.find("(")
|
| 118 |
+
if opening < 0 or _matching_paren(text, opening) != len(text) - 1:
|
| 119 |
+
return None
|
| 120 |
+
prefix = text[:opening]
|
| 121 |
+
# H(...) and I(...) are atoms, not grouping parentheses.
|
| 122 |
+
if prefix in ("H", "I"):
|
| 123 |
+
return None
|
| 124 |
+
if prefix.endswith("*"):
|
| 125 |
+
prefix = prefix[:-1]
|
| 126 |
+
if not prefix:
|
| 127 |
+
raise ProofSyntaxError("a '*' before a grouped expression needs a coefficient")
|
| 128 |
+
if prefix and not re.fullmatch(rf"[+-]?(?:{_NUMBER})?", prefix):
|
| 129 |
+
return None
|
| 130 |
+
return _parse_scalar(prefix), text[opening + 1 : -1]
|
| 131 |
+
|
| 132 |
+
|
| 133 |
+
def _parse_variable_group(text: str, n: int) -> int:
|
| 134 |
+
if not text:
|
| 135 |
+
raise ProofSyntaxError("an information term contains an empty variable group")
|
| 136 |
+
mask = 0
|
| 137 |
+
position = 0
|
| 138 |
+
found = False
|
| 139 |
+
for match in _VARIABLE_RE.finditer(text):
|
| 140 |
+
gap = text[position : match.start()]
|
| 141 |
+
if gap not in ("", ","):
|
| 142 |
+
raise ProofSyntaxError(f"invalid variable-group syntax {text!r}")
|
| 143 |
+
indices = (match.group(1) or match.group(2)).split(",")
|
| 144 |
+
for raw_index in indices:
|
| 145 |
+
index = int(raw_index)
|
| 146 |
+
if index < 1 or index > n:
|
| 147 |
+
raise ProofSyntaxError(
|
| 148 |
+
f"variable X_{index} is outside the declared range 1..{n}"
|
| 149 |
+
)
|
| 150 |
+
mask |= 1 << (index - 1)
|
| 151 |
+
position = match.end()
|
| 152 |
+
found = True
|
| 153 |
+
if not found or position != len(text):
|
| 154 |
+
raise ProofSyntaxError(f"invalid variable-group syntax {text!r}")
|
| 155 |
+
return mask
|
| 156 |
+
|
| 157 |
+
|
| 158 |
+
def _entropy(mask: int, coefficient: Fraction = Fraction(1)) -> Vector:
|
| 159 |
+
return {} if not mask or not coefficient else {mask: coefficient}
|
| 160 |
+
|
| 161 |
+
|
| 162 |
+
def _parse_atom(text: str, n: int) -> Tuple[Vector, Atom]:
|
| 163 |
+
if len(text) < 3 or text[0] not in "HI" or text[1] != "(":
|
| 164 |
+
raise ProofSyntaxError(f"expected H(...) or I(...), found {text!r}")
|
| 165 |
+
if _matching_paren(text, 1) != len(text) - 1:
|
| 166 |
+
raise ProofSyntaxError(f"unexpected text after information term {text!r}")
|
| 167 |
+
|
| 168 |
+
kind = text[0]
|
| 169 |
+
inside = text[2:-1]
|
| 170 |
+
if kind == "H":
|
| 171 |
+
parts = inside.split("|")
|
| 172 |
+
if len(parts) > 2:
|
| 173 |
+
raise ProofSyntaxError(f"too many conditioning bars in {text!r}")
|
| 174 |
+
left = _parse_variable_group(parts[0], n)
|
| 175 |
+
if len(parts) == 1:
|
| 176 |
+
return _entropy(left), Atom("entropy", (left,))
|
| 177 |
+
conditioning = _parse_variable_group(parts[1], n)
|
| 178 |
+
vector = _entropy(left | conditioning)
|
| 179 |
+
_add(vector, _entropy(conditioning), Fraction(-1))
|
| 180 |
+
return vector, Atom("conditional_entropy", (left, conditioning))
|
| 181 |
+
|
| 182 |
+
conditional_parts = inside.split("|")
|
| 183 |
+
if len(conditional_parts) > 2:
|
| 184 |
+
raise ProofSyntaxError(f"too many conditioning bars in {text!r}")
|
| 185 |
+
sides = conditional_parts[0].split(";")
|
| 186 |
+
if len(sides) != 2:
|
| 187 |
+
raise ProofSyntaxError(f"mutual information requires two ';'-separated groups in {text!r}")
|
| 188 |
+
left = _parse_variable_group(sides[0], n)
|
| 189 |
+
right = _parse_variable_group(sides[1], n)
|
| 190 |
+
conditioning = (
|
| 191 |
+
_parse_variable_group(conditional_parts[1], n)
|
| 192 |
+
if len(conditional_parts) == 2
|
| 193 |
+
else 0
|
| 194 |
+
)
|
| 195 |
+
vector: Vector = {}
|
| 196 |
+
_add(vector, _entropy(left | conditioning))
|
| 197 |
+
_add(vector, _entropy(right | conditioning))
|
| 198 |
+
_add(vector, _entropy(left | right | conditioning), Fraction(-1))
|
| 199 |
+
_add(vector, _entropy(conditioning), Fraction(-1))
|
| 200 |
+
atom_kind = "conditional_mutual_information" if conditioning else "mutual_information"
|
| 201 |
+
return vector, Atom(atom_kind, (left, right, conditioning))
|
| 202 |
+
|
| 203 |
+
|
| 204 |
+
def parse_expression(text: str, n: int) -> ParsedExpression:
|
| 205 |
+
"""Parse a linear entropy expression into exact sparse coordinates."""
|
| 206 |
+
if not isinstance(text, str):
|
| 207 |
+
raise ProofSyntaxError("an entropy expression must be a string")
|
| 208 |
+
if not isinstance(n, int) or isinstance(n, bool) or n < 1:
|
| 209 |
+
raise ProofSyntaxError(f"n must be a positive integer, found {n!r}")
|
| 210 |
+
compact = re.sub(r"\s+", "", text)
|
| 211 |
+
if compact == "0":
|
| 212 |
+
return ParsedExpression({}, None, None)
|
| 213 |
+
if not compact:
|
| 214 |
+
raise ProofSyntaxError("empty entropy expression")
|
| 215 |
+
|
| 216 |
+
outer = _outer_scalar(compact)
|
| 217 |
+
outer_coefficient = Fraction(1)
|
| 218 |
+
if outer is not None:
|
| 219 |
+
outer_coefficient, compact = outer
|
| 220 |
+
if not compact:
|
| 221 |
+
raise ProofSyntaxError("empty parenthesized expression")
|
| 222 |
+
|
| 223 |
+
vector: Vector = {}
|
| 224 |
+
atoms: List[Tuple[Atom, Fraction]] = []
|
| 225 |
+
position = 0
|
| 226 |
+
while position < len(compact):
|
| 227 |
+
sign = Fraction(1)
|
| 228 |
+
if compact[position] in "+-":
|
| 229 |
+
sign = Fraction(-1) if compact[position] == "-" else Fraction(1)
|
| 230 |
+
position += 1
|
| 231 |
+
if position == len(compact):
|
| 232 |
+
raise ProofSyntaxError("expression ends with a sign")
|
| 233 |
+
elif position != 0:
|
| 234 |
+
raise ProofSyntaxError(f"expected '+' or '-' at offset {position}")
|
| 235 |
+
|
| 236 |
+
number_match = _NUMBER_RE.match(compact, position)
|
| 237 |
+
coefficient = Fraction(1)
|
| 238 |
+
if number_match:
|
| 239 |
+
coefficient = Fraction(number_match.group(0))
|
| 240 |
+
position = number_match.end()
|
| 241 |
+
if position < len(compact) and compact[position] == "*":
|
| 242 |
+
position += 1
|
| 243 |
+
elif position < len(compact) and compact[position] == "*":
|
| 244 |
+
raise ProofSyntaxError(f"unexpected '*' at offset {position}")
|
| 245 |
+
|
| 246 |
+
if position >= len(compact) or compact[position] not in "HI":
|
| 247 |
+
raise ProofSyntaxError(f"expected H(...) or I(...) at offset {position}")
|
| 248 |
+
opening = position + 1
|
| 249 |
+
if opening >= len(compact) or compact[opening] != "(":
|
| 250 |
+
raise ProofSyntaxError(f"expected '(' after {compact[position]!r}")
|
| 251 |
+
closing = _matching_paren(compact, opening)
|
| 252 |
+
atom_vector, atom = _parse_atom(compact[position : closing + 1], n)
|
| 253 |
+
term_coefficient = sign * coefficient * outer_coefficient
|
| 254 |
+
_add(vector, atom_vector, term_coefficient)
|
| 255 |
+
atoms.append((atom, term_coefficient))
|
| 256 |
+
position = closing + 1
|
| 257 |
+
|
| 258 |
+
single_atom = atoms[0][0] if len(atoms) == 1 else None
|
| 259 |
+
single_coefficient = atoms[0][1] if len(atoms) == 1 else None
|
| 260 |
+
return ParsedExpression(vector, single_atom, single_coefficient)
|
| 261 |
+
|
| 262 |
+
|
| 263 |
+
def parse_constraint(
|
| 264 |
+
text: str,
|
| 265 |
+
n: int,
|
| 266 |
+
*,
|
| 267 |
+
default_comparator: Optional[str] = None,
|
| 268 |
+
) -> ParsedConstraint:
|
| 269 |
+
"""Parse an expression followed by an optional comparison with zero."""
|
| 270 |
+
if not isinstance(text, str):
|
| 271 |
+
raise ProofSyntaxError("a constraint must be a string")
|
| 272 |
+
match = _COMPARATOR_RE.search(text)
|
| 273 |
+
if match:
|
| 274 |
+
lhs = text[: match.start()]
|
| 275 |
+
comparator = match.group(1)
|
| 276 |
+
if any(op in lhs for op in (">", "<", "=")):
|
| 277 |
+
raise ProofSyntaxError(f"unexpected comparator in {text!r}")
|
| 278 |
+
else:
|
| 279 |
+
if any(op in text for op in (">", "<", "=")):
|
| 280 |
+
raise ProofSyntaxError(f"constraints must compare against zero: {text!r}")
|
| 281 |
+
if default_comparator is None:
|
| 282 |
+
raise ProofSyntaxError(f"missing comparator in {text!r}")
|
| 283 |
+
lhs = text
|
| 284 |
+
comparator = default_comparator
|
| 285 |
+
if comparator not in (">=", "<=", "=", ">", "<"):
|
| 286 |
+
raise ProofSyntaxError(f"unsupported comparator {comparator!r}")
|
| 287 |
+
return ParsedConstraint(parse_expression(lhs, n), comparator)
|
| 288 |
+
|
| 289 |
+
|
| 290 |
+
def _oriented_vector(constraint: ParsedConstraint) -> Vector:
|
| 291 |
+
if constraint.comparator in ("<=", "<"):
|
| 292 |
+
return _scaled(constraint.expression.vector, Fraction(-1))
|
| 293 |
+
return dict(constraint.expression.vector)
|
| 294 |
+
|
| 295 |
+
|
| 296 |
+
def _multiple(left: Mapping[int, Fraction], right: Mapping[int, Fraction]) -> Optional[Fraction]:
|
| 297 |
+
"""Return q when left == q * right, or None for nonmultiples/zero rows."""
|
| 298 |
+
left = _clean(left)
|
| 299 |
+
right = _clean(right)
|
| 300 |
+
if not left or not right or set(left) != set(right):
|
| 301 |
+
return None
|
| 302 |
+
first_mask = next(iter(right))
|
| 303 |
+
ratio = left[first_mask] / right[first_mask]
|
| 304 |
+
if ratio == 0:
|
| 305 |
+
return None
|
| 306 |
+
if all(left[mask] == ratio * right[mask] for mask in right):
|
| 307 |
+
return ratio
|
| 308 |
+
return None
|
| 309 |
+
|
| 310 |
+
|
| 311 |
+
@dataclass(frozen=True)
|
| 312 |
+
class _ConditionFact:
|
| 313 |
+
vector: Vector
|
| 314 |
+
equality: bool
|
| 315 |
+
source_index: int
|
| 316 |
+
|
| 317 |
+
|
| 318 |
+
def _condition_facts(conditions: Sequence[str], n: int) -> List[_ConditionFact]:
|
| 319 |
+
parsed: List[_ConditionFact] = []
|
| 320 |
+
for index, raw in enumerate(conditions):
|
| 321 |
+
sentinel = re.sub(r"\s+", "", raw).upper()
|
| 322 |
+
if sentinel in {"NONE", "NONE=0", "NONE>=0", "(NONE)", "(NONE)=0", "(NONE)>=0"}:
|
| 323 |
+
continue
|
| 324 |
+
condition = parse_constraint(raw, n, default_comparator=">=")
|
| 325 |
+
vector = _oriented_vector(condition)
|
| 326 |
+
if not vector:
|
| 327 |
+
raise ProofSyntaxError(f"condition {index + 1} has an identically zero left-hand side")
|
| 328 |
+
parsed.append(_ConditionFact(vector, condition.comparator == "=", index))
|
| 329 |
+
|
| 330 |
+
# Opposite one-sided rows jointly assert equality. Promote both, matching
|
| 331 |
+
# the v12 search's condition normalization.
|
| 332 |
+
promoted = set()
|
| 333 |
+
for left_index, left in enumerate(parsed):
|
| 334 |
+
if left.equality:
|
| 335 |
+
continue
|
| 336 |
+
for right_index in range(left_index + 1, len(parsed)):
|
| 337 |
+
right = parsed[right_index]
|
| 338 |
+
ratio = _multiple(left.vector, right.vector)
|
| 339 |
+
if not right.equality and ratio is not None and ratio < 0:
|
| 340 |
+
promoted.update((left_index, right_index))
|
| 341 |
+
return [
|
| 342 |
+
_ConditionFact(fact.vector, fact.equality or index in promoted, fact.source_index)
|
| 343 |
+
for index, fact in enumerate(parsed)
|
| 344 |
+
]
|
| 345 |
+
|
| 346 |
+
|
| 347 |
+
def _classify_step(
|
| 348 |
+
raw_step: str,
|
| 349 |
+
n: int,
|
| 350 |
+
condition_facts: Sequence[_ConditionFact],
|
| 351 |
+
) -> Tuple[Optional[VerifiedStep], Optional[str], Vector]:
|
| 352 |
+
step = parse_constraint(raw_step, n)
|
| 353 |
+
vector = dict(step.expression.vector)
|
| 354 |
+
if not vector:
|
| 355 |
+
return None, "step has an identically zero left-hand side", vector
|
| 356 |
+
if step.comparator in (">", "<"):
|
| 357 |
+
return None, "strict proof-step comparators are not accepted", vector
|
| 358 |
+
if step.comparator == "<=":
|
| 359 |
+
return None, "proof steps must be written in nonnegative (>=0) orientation", vector
|
| 360 |
+
|
| 361 |
+
if (
|
| 362 |
+
step.comparator == ">="
|
| 363 |
+
and step.expression.atom is not None
|
| 364 |
+
and step.expression.atom_coefficient is not None
|
| 365 |
+
and step.expression.atom_coefficient > 0
|
| 366 |
+
):
|
| 367 |
+
return (
|
| 368 |
+
VerifiedStep(
|
| 369 |
+
raw_step,
|
| 370 |
+
"shannon_atom",
|
| 371 |
+
step.expression.atom_coefficient,
|
| 372 |
+
None,
|
| 373 |
+
),
|
| 374 |
+
None,
|
| 375 |
+
vector,
|
| 376 |
+
)
|
| 377 |
+
|
| 378 |
+
for fact in condition_facts:
|
| 379 |
+
multiplier = _multiple(vector, fact.vector)
|
| 380 |
+
if multiplier is None:
|
| 381 |
+
continue
|
| 382 |
+
if fact.equality:
|
| 383 |
+
return (
|
| 384 |
+
VerifiedStep(raw_step, "equality_condition", multiplier, fact.source_index),
|
| 385 |
+
None,
|
| 386 |
+
vector,
|
| 387 |
+
)
|
| 388 |
+
if step.comparator == ">=" and multiplier > 0:
|
| 389 |
+
return (
|
| 390 |
+
VerifiedStep(raw_step, "inequality_condition", multiplier, fact.source_index),
|
| 391 |
+
None,
|
| 392 |
+
vector,
|
| 393 |
+
)
|
| 394 |
+
|
| 395 |
+
return None, "step is neither a positive Shannon atom nor an allowed condition multiple", vector
|
| 396 |
+
|
| 397 |
+
|
| 398 |
+
def verify_certificate(
|
| 399 |
+
n: int,
|
| 400 |
+
target: str,
|
| 401 |
+
conditions: Sequence[str],
|
| 402 |
+
steps: Sequence[str],
|
| 403 |
+
*,
|
| 404 |
+
terminal_residual: Optional[str] = None,
|
| 405 |
+
) -> VerificationResult:
|
| 406 |
+
"""Verify a certificate exactly, returning a diagnostic result.
|
| 407 |
+
|
| 408 |
+
The terminal residual is recomputed as ``target - sum(steps)``. When an
|
| 409 |
+
encoded terminal residual is supplied (for example from ``proof_trace``),
|
| 410 |
+
it must match that recomputation exactly.
|
| 411 |
+
"""
|
| 412 |
+
target_constraint = parse_constraint(target, n, default_comparator=">=")
|
| 413 |
+
if target_constraint.comparator in (">", "<"):
|
| 414 |
+
return VerificationResult(False, "strict target comparators are not accepted", (), {}, None)
|
| 415 |
+
if target_constraint.comparator == "=":
|
| 416 |
+
return VerificationResult(False, "equality targets are not supported", (), {}, None)
|
| 417 |
+
target_vector = _oriented_vector(target_constraint)
|
| 418 |
+
facts = _condition_facts(list(conditions), n)
|
| 419 |
+
|
| 420 |
+
residual = dict(target_vector)
|
| 421 |
+
verified_steps: List[VerifiedStep] = []
|
| 422 |
+
for index, raw_step in enumerate(steps):
|
| 423 |
+
if not isinstance(raw_step, str):
|
| 424 |
+
raise ProofSyntaxError(f"step {index + 1} must be a string")
|
| 425 |
+
classified, reason, step_vector = _classify_step(raw_step, n, facts)
|
| 426 |
+
if classified is None:
|
| 427 |
+
return VerificationResult(
|
| 428 |
+
False,
|
| 429 |
+
f"step {index + 1}: {reason}",
|
| 430 |
+
tuple(verified_steps),
|
| 431 |
+
residual,
|
| 432 |
+
None,
|
| 433 |
+
)
|
| 434 |
+
verified_steps.append(classified)
|
| 435 |
+
_add(residual, step_vector, Fraction(-1))
|
| 436 |
+
|
| 437 |
+
residual = _clean(residual)
|
| 438 |
+
if terminal_residual is not None:
|
| 439 |
+
encoded = parse_expression(terminal_residual, n).vector
|
| 440 |
+
if encoded != residual:
|
| 441 |
+
return VerificationResult(
|
| 442 |
+
False,
|
| 443 |
+
"encoded terminal residual does not equal target minus accepted steps",
|
| 444 |
+
tuple(verified_steps),
|
| 445 |
+
residual,
|
| 446 |
+
None,
|
| 447 |
+
)
|
| 448 |
+
|
| 449 |
+
if not residual:
|
| 450 |
+
terminal_kind = "zero_residual"
|
| 451 |
+
elif all(coefficient >= 0 for coefficient in residual.values()):
|
| 452 |
+
terminal_kind = "coordinatewise_nonnegative"
|
| 453 |
+
else:
|
| 454 |
+
return VerificationResult(
|
| 455 |
+
False,
|
| 456 |
+
"terminal residual has a negative joint-entropy coordinate",
|
| 457 |
+
tuple(verified_steps),
|
| 458 |
+
residual,
|
| 459 |
+
None,
|
| 460 |
+
)
|
| 461 |
+
|
| 462 |
+
return VerificationResult(
|
| 463 |
+
True,
|
| 464 |
+
"certificate verified exactly",
|
| 465 |
+
tuple(verified_steps),
|
| 466 |
+
residual,
|
| 467 |
+
terminal_kind,
|
| 468 |
+
)
|
| 469 |
+
|
| 470 |
+
|
| 471 |
+
def verify_result_row(row: Mapping[str, object]) -> VerificationResult:
|
| 472 |
+
"""Verify one claimed-proof row in the v12 JSONL output schema."""
|
| 473 |
+
if not isinstance(row, Mapping):
|
| 474 |
+
raise ProofCertificateError("result row must be a JSON object")
|
| 475 |
+
run = row.get("run")
|
| 476 |
+
input_data = row.get("input")
|
| 477 |
+
mapped = row.get("mapped")
|
| 478 |
+
if not isinstance(run, Mapping) or not isinstance(input_data, Mapping):
|
| 479 |
+
raise ProofCertificateError("result row requires object-valued 'input' and 'run' fields")
|
| 480 |
+
if run.get("proved") is not True:
|
| 481 |
+
return VerificationResult(False, "row does not claim a proof", (), {}, None)
|
| 482 |
+
|
| 483 |
+
mapped_data = mapped if isinstance(mapped, Mapping) else {}
|
| 484 |
+
raw_n = mapped_data.get("n", input_data.get("n"))
|
| 485 |
+
if not isinstance(raw_n, int) or isinstance(raw_n, bool) or raw_n < 1:
|
| 486 |
+
raise ProofCertificateError(f"result row has invalid n: {raw_n!r}")
|
| 487 |
+
target = mapped_data.get("ineq", input_data.get("ineq"))
|
| 488 |
+
if not isinstance(target, str):
|
| 489 |
+
raise ProofCertificateError("result row has no string target inequality")
|
| 490 |
+
raw_conditions = mapped_data.get("conditions", input_data.get("conditions", []))
|
| 491 |
+
if not isinstance(raw_conditions, list) or not all(isinstance(item, str) for item in raw_conditions):
|
| 492 |
+
raise ProofCertificateError("result row conditions must be a list of strings")
|
| 493 |
+
raw_steps = run.get("expr_str")
|
| 494 |
+
if not isinstance(raw_steps, list):
|
| 495 |
+
raise ProofCertificateError("a proved result row must contain a list-valued run.expr_str")
|
| 496 |
+
|
| 497 |
+
terminal_residual: Optional[str] = None
|
| 498 |
+
trace = run.get("proof_trace")
|
| 499 |
+
if isinstance(trace, Mapping):
|
| 500 |
+
terminal = trace.get("terminal")
|
| 501 |
+
if isinstance(terminal, Mapping) and "residual" in terminal:
|
| 502 |
+
value = terminal.get("residual")
|
| 503 |
+
if not isinstance(value, str):
|
| 504 |
+
raise ProofCertificateError("proof_trace terminal residual must be a string")
|
| 505 |
+
terminal_residual = value
|
| 506 |
+
|
| 507 |
+
return verify_certificate(
|
| 508 |
+
raw_n,
|
| 509 |
+
target,
|
| 510 |
+
raw_conditions,
|
| 511 |
+
raw_steps,
|
| 512 |
+
terminal_residual=terminal_residual,
|
| 513 |
+
)
|
| 514 |
+
|
| 515 |
+
|
| 516 |
+
def verify_jsonl(path: Path) -> Tuple[int, int, int, int]:
|
| 517 |
+
"""Return (valid, invalid, error, skipped_unproved) counts for a v12 file."""
|
| 518 |
+
valid = invalid = errors = skipped = 0
|
| 519 |
+
with path.open("r", encoding="utf-8") as stream:
|
| 520 |
+
for line_number, line in enumerate(stream, start=1):
|
| 521 |
+
if not line.strip():
|
| 522 |
+
continue
|
| 523 |
+
try:
|
| 524 |
+
row = json.loads(line)
|
| 525 |
+
run = row.get("run") if isinstance(row, dict) else None
|
| 526 |
+
if isinstance(run, dict) and run.get("proved") is not True:
|
| 527 |
+
skipped += 1
|
| 528 |
+
continue
|
| 529 |
+
result = verify_result_row(row)
|
| 530 |
+
except (json.JSONDecodeError, ProofCertificateError) as exc:
|
| 531 |
+
errors += 1
|
| 532 |
+
print(f"line {line_number}: error: {exc}", file=sys.stderr)
|
| 533 |
+
continue
|
| 534 |
+
if result.valid:
|
| 535 |
+
valid += 1
|
| 536 |
+
else:
|
| 537 |
+
invalid += 1
|
| 538 |
+
print(f"line {line_number}: invalid: {result.reason}", file=sys.stderr)
|
| 539 |
+
return valid, invalid, errors, skipped
|
| 540 |
+
|
| 541 |
+
|
| 542 |
+
def main(argv: Optional[Sequence[str]] = None) -> int:
|
| 543 |
+
parser = argparse.ArgumentParser(description="Verify v12 symbolic proof certificates exactly.")
|
| 544 |
+
parser.add_argument("result_file", type=Path, help="v12 JSONL result file (read-only)")
|
| 545 |
+
args = parser.parse_args(argv)
|
| 546 |
+
if not args.result_file.is_file():
|
| 547 |
+
parser.error(f"not a file: {args.result_file}")
|
| 548 |
+
valid, invalid, errors, skipped = verify_jsonl(args.result_file)
|
| 549 |
+
print(f"valid: {valid}")
|
| 550 |
+
print(f"invalid: {invalid}")
|
| 551 |
+
print(f"error: {errors}")
|
| 552 |
+
print(f"skipped_unproved: {skipped}")
|
| 553 |
+
return 0 if invalid == 0 and errors == 0 else 1
|
| 554 |
+
|
| 555 |
+
|
| 556 |
+
if __name__ == "__main__":
|
| 557 |
+
raise SystemExit(main())
|
mce220_runner/data/__init__.py
ADDED
|
File without changes
|
mce220_runner/data/entropy_vector_converter.py
ADDED
|
@@ -0,0 +1,250 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Entropy Inequality Vector-String Converter
|
| 3 |
+
|
| 4 |
+
Converts between entropy inequality vector representation and compact string format.
|
| 5 |
+
Vector format: (H(X_1)'s coefficient, H(X_2)'s coefficient, H(X_{1,2})'s coefficient, ...)
|
| 6 |
+
String format: "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})" (compact, no spaces, subscript notation)
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
import numpy as np
|
| 10 |
+
from scipy.sparse import csr_matrix
|
| 11 |
+
import re
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def vector_to_string(vec) -> str:
|
| 15 |
+
"""
|
| 16 |
+
Convert entropy inequality vector to compact string format.
|
| 17 |
+
|
| 18 |
+
Format: "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})" (no spaces, subscript notation)
|
| 19 |
+
|
| 20 |
+
Args:
|
| 21 |
+
vec: Vector (numpy array, list, or csr_matrix) of length 2^n - 1
|
| 22 |
+
|
| 23 |
+
Returns:
|
| 24 |
+
String like "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})"
|
| 25 |
+
|
| 26 |
+
Example:
|
| 27 |
+
>>> vector_to_string([0, 0, 3, 0])
|
| 28 |
+
'3H(X_{1,2})'
|
| 29 |
+
>>> vector_to_string([1, -2, 0, 0])
|
| 30 |
+
'H(X_{1}) - 2H(X_{2})'
|
| 31 |
+
"""
|
| 32 |
+
# Convert to dense array
|
| 33 |
+
if isinstance(vec, csr_matrix):
|
| 34 |
+
vec_dense = vec.toarray().flatten()
|
| 35 |
+
else:
|
| 36 |
+
vec_dense = np.array(vec).flatten()
|
| 37 |
+
|
| 38 |
+
# Estimate n from vector length: M = 2^n - 1
|
| 39 |
+
n = int(np.log2(len(vec_dense) + 1))
|
| 40 |
+
|
| 41 |
+
terms = []
|
| 42 |
+
for i, coeff in enumerate(vec_dense):
|
| 43 |
+
if abs(coeff) > 1e-6:
|
| 44 |
+
# Convert index to subset representation
|
| 45 |
+
subset_idx = i + 1
|
| 46 |
+
# Convert to binary (low bit first)
|
| 47 |
+
subset_binary = [(subset_idx >> j) & 1 for j in range(n)]
|
| 48 |
+
|
| 49 |
+
# Build subset string in format X_{1,2,3}
|
| 50 |
+
vars_list = [f"{j+1}" for j, b in enumerate(subset_binary) if b == 1]
|
| 51 |
+
if not vars_list:
|
| 52 |
+
continue # Skip empty set
|
| 53 |
+
|
| 54 |
+
subset_str = ",".join(vars_list)
|
| 55 |
+
|
| 56 |
+
# Format coefficient and entropy term
|
| 57 |
+
if abs(coeff - 1.0) < 1e-6:
|
| 58 |
+
# Coefficient is 1
|
| 59 |
+
if coeff < 0:
|
| 60 |
+
terms.append(f"-H(X_{{{subset_str}}})")
|
| 61 |
+
else:
|
| 62 |
+
terms.append(f"H(X_{{{subset_str}}})")
|
| 63 |
+
elif abs(coeff + 1.0) < 1e-6:
|
| 64 |
+
# Coefficient is -1
|
| 65 |
+
terms.append(f"-H(X_{{{subset_str}}})")
|
| 66 |
+
else:
|
| 67 |
+
# Other coefficients
|
| 68 |
+
coeff_int = int(round(coeff))
|
| 69 |
+
if abs(coeff - coeff_int) < 1e-6:
|
| 70 |
+
# Integer coefficient
|
| 71 |
+
if coeff < 0:
|
| 72 |
+
terms.append(f"-{abs(coeff_int)}H(X_{{{subset_str}}})")
|
| 73 |
+
else:
|
| 74 |
+
terms.append(f"{coeff_int}H(X_{{{subset_str}}})")
|
| 75 |
+
else:
|
| 76 |
+
# Non-integer coefficient
|
| 77 |
+
if coeff < 0:
|
| 78 |
+
terms.append(f"-{abs(coeff)}H(X_{{{subset_str}}})")
|
| 79 |
+
else:
|
| 80 |
+
terms.append(f"{coeff}H(X_{{{subset_str}}})")
|
| 81 |
+
|
| 82 |
+
if not terms:
|
| 83 |
+
return "0"
|
| 84 |
+
|
| 85 |
+
# Join terms: positive terms with "+", negative terms already have "-"
|
| 86 |
+
result = terms[0]
|
| 87 |
+
for term in terms[1:]:
|
| 88 |
+
if term.startswith('-'):
|
| 89 |
+
result += term # Already has "-"
|
| 90 |
+
else:
|
| 91 |
+
result += "+" + term
|
| 92 |
+
|
| 93 |
+
return result
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def string_to_vector(ineq_str: str, n_variables: int = None) -> csr_matrix:
|
| 97 |
+
"""
|
| 98 |
+
Convert entropy inequality string to vector representation.
|
| 99 |
+
|
| 100 |
+
Parses strings like "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})"
|
| 101 |
+
and converts to CSR sparse matrix of length 2^n - 1.
|
| 102 |
+
|
| 103 |
+
Args:
|
| 104 |
+
ineq_str: String like "3H(X_{1,2})" or "H(X_{1}) - 2H(X_{2,3})"
|
| 105 |
+
n_variables: Number of variables (if None, inferred from string)
|
| 106 |
+
|
| 107 |
+
Returns:
|
| 108 |
+
CSR sparse matrix (1 x (2^n - 1)) representing the inequality vector
|
| 109 |
+
|
| 110 |
+
Example:
|
| 111 |
+
>>> string_to_vector("3H(X_{1,2})").toarray()
|
| 112 |
+
array([[0., 0., 3.]])
|
| 113 |
+
>>> string_to_vector("H(X_{1})-2H(X_{2})").toarray()
|
| 114 |
+
array([[ 1., -2., 0.]])
|
| 115 |
+
"""
|
| 116 |
+
# Remove spaces for easier parsing
|
| 117 |
+
ineq_str_clean = ineq_str.replace(' ', '')
|
| 118 |
+
|
| 119 |
+
# Pattern to match: optional sign at start of term, optional coefficient, H(X_{...})
|
| 120 |
+
# This pattern handles: "3H(X_{1,2})", "-H(X_{1})", "H(X_{1,2,3})", "-2H(X_{1,2})"
|
| 121 |
+
# Also handles cases like "+3H" or "-2H"
|
| 122 |
+
pattern = r'([+-]?)(\d*\.?\d*)H\(X_\{([^}]+)\}\)'
|
| 123 |
+
|
| 124 |
+
matches = re.findall(pattern, ineq_str_clean)
|
| 125 |
+
|
| 126 |
+
if not matches:
|
| 127 |
+
# Try to parse "0" or empty string
|
| 128 |
+
if ineq_str.strip() == "0" or not ineq_str.strip():
|
| 129 |
+
# Need n_variables to create zero vector
|
| 130 |
+
if n_variables is None:
|
| 131 |
+
raise ValueError("Cannot infer n_variables from '0'. Please specify n_variables.")
|
| 132 |
+
return csr_matrix((1, 2**n_variables - 1), dtype=np.float64)
|
| 133 |
+
else:
|
| 134 |
+
raise ValueError(f"Could not parse inequality string: {ineq_str}")
|
| 135 |
+
|
| 136 |
+
# Infer n_variables from the largest variable index found
|
| 137 |
+
max_var = 0
|
| 138 |
+
for sign, coeff, subset_str in matches:
|
| 139 |
+
vars_list = [int(x.strip()) for x in subset_str.split(',') if x.strip()]
|
| 140 |
+
if vars_list:
|
| 141 |
+
max_var = max(max_var, max(vars_list))
|
| 142 |
+
|
| 143 |
+
if n_variables is None:
|
| 144 |
+
n_variables = max_var if max_var > 0 else 1
|
| 145 |
+
|
| 146 |
+
# Create vector
|
| 147 |
+
M = 2**n_variables - 1
|
| 148 |
+
vec = np.zeros(M)
|
| 149 |
+
|
| 150 |
+
for sign, coeff_str, subset_str in matches:
|
| 151 |
+
# Parse coefficient
|
| 152 |
+
if not coeff_str or coeff_str == '.':
|
| 153 |
+
coeff = 1.0
|
| 154 |
+
else:
|
| 155 |
+
coeff = float(coeff_str)
|
| 156 |
+
|
| 157 |
+
# Apply sign
|
| 158 |
+
if sign == '-':
|
| 159 |
+
coeff = -coeff
|
| 160 |
+
|
| 161 |
+
# Parse subset
|
| 162 |
+
vars_list = [int(x.strip()) for x in subset_str.split(',') if x.strip()]
|
| 163 |
+
|
| 164 |
+
# Convert subset to index
|
| 165 |
+
# Index = sum(2^(i-1) for i in vars_list) - 1
|
| 166 |
+
subset_idx = 0
|
| 167 |
+
for var_idx in vars_list:
|
| 168 |
+
if 1 <= var_idx <= n_variables:
|
| 169 |
+
subset_idx += 2**(var_idx - 1)
|
| 170 |
+
|
| 171 |
+
vec_idx = subset_idx - 1
|
| 172 |
+
|
| 173 |
+
if 0 <= vec_idx < M:
|
| 174 |
+
vec[vec_idx] += coeff
|
| 175 |
+
|
| 176 |
+
# Convert to CSR sparse matrix (1 x M)
|
| 177 |
+
return csr_matrix(vec, shape=(1, M), dtype=np.float64)
|
| 178 |
+
|
| 179 |
+
|
| 180 |
+
# Example usage and test
|
| 181 |
+
if __name__ == "__main__":
|
| 182 |
+
# Test vector_to_string
|
| 183 |
+
print("Testing vector_to_string:")
|
| 184 |
+
vec1 = np.array([0, 0, 3, 0])
|
| 185 |
+
str1 = vector_to_string(vec1)
|
| 186 |
+
print(f" Vector {vec1} -> '{str1}'")
|
| 187 |
+
# Expected: "3H(X_{1,2})"
|
| 188 |
+
|
| 189 |
+
vec2 = np.array([1, -2, 0, 0])
|
| 190 |
+
str2 = vector_to_string(vec2)
|
| 191 |
+
print(f" Vector {vec2} -> '{str2}'")
|
| 192 |
+
# Expected: "H(X_{1}) - 2H(X_{2})"
|
| 193 |
+
|
| 194 |
+
vec3 = np.array([0, 0, 0, 1, 0, 0, -1])
|
| 195 |
+
str3 = vector_to_string(vec3)
|
| 196 |
+
print(f" Vector {vec3} -> '{str3}'")
|
| 197 |
+
# Expected: "H(X_{3}) - H(X_{1,2,3})" (for n=3)
|
| 198 |
+
|
| 199 |
+
print("\nTesting string_to_vector:")
|
| 200 |
+
# Test string_to_vector
|
| 201 |
+
str_test1 = "3H(X_{1,2})"
|
| 202 |
+
vec_back1 = string_to_vector(str_test1)
|
| 203 |
+
print(f" String '{str_test1}' -> CSR Matrix")
|
| 204 |
+
print(f" Type: {type(vec_back1)}")
|
| 205 |
+
print(f" Shape: {vec_back1.shape}")
|
| 206 |
+
print(f" Dense: {vec_back1.toarray().flatten()}")
|
| 207 |
+
print(f" Non-zero elements: {vec_back1.nnz}")
|
| 208 |
+
|
| 209 |
+
str_test2 = "H(X_{1}) - 2H(X_{2})"
|
| 210 |
+
vec_back2 = string_to_vector(str_test2)
|
| 211 |
+
print(f"\n String '{str_test2}' -> CSR Matrix")
|
| 212 |
+
print(f" Type: {type(vec_back2)}")
|
| 213 |
+
print(f" Shape: {vec_back2.shape}")
|
| 214 |
+
print(f" Dense: {vec_back2.toarray().flatten()}")
|
| 215 |
+
print(f" Non-zero elements: {vec_back2.nnz}")
|
| 216 |
+
|
| 217 |
+
str_test3 = "H(X_{1})-2H(X_{2})" # No spaces version
|
| 218 |
+
vec_back3 = string_to_vector(str_test3)
|
| 219 |
+
print(f"\n String '{str_test3}' -> CSR Matrix")
|
| 220 |
+
print(f" Dense: {vec_back3.toarray().flatten()}")
|
| 221 |
+
|
| 222 |
+
print("\nRound-trip test:")
|
| 223 |
+
# Round-trip test - use proper length vector (for n=2, length should be 3)
|
| 224 |
+
original_vec = np.array([0, 0, 3]) # n=2: [H(X1), H(X2), H(X1,X2)]
|
| 225 |
+
str_repr = vector_to_string(original_vec)
|
| 226 |
+
recovered_vec = string_to_vector(str_repr)
|
| 227 |
+
print(f" Original: {original_vec}")
|
| 228 |
+
print(f" String: '{str_repr}'")
|
| 229 |
+
print(f" Recovered (CSR): {recovered_vec.toarray().flatten()}")
|
| 230 |
+
print(f" Match: {np.allclose(original_vec, recovered_vec.toarray().flatten())}")
|
| 231 |
+
|
| 232 |
+
# Another round-trip test with n=2
|
| 233 |
+
original_vec2 = np.array([1, -2, 0])
|
| 234 |
+
str_repr2 = vector_to_string(original_vec2)
|
| 235 |
+
recovered_vec2 = string_to_vector(str_repr2)
|
| 236 |
+
print(f"\n Original: {original_vec2}")
|
| 237 |
+
print(f" String: '{str_repr2}'")
|
| 238 |
+
print(f" Recovered (CSR): {recovered_vec2.toarray().flatten()}")
|
| 239 |
+
print(f" Match: {np.allclose(original_vec2, recovered_vec2.toarray().flatten())}")
|
| 240 |
+
|
| 241 |
+
# Test CSR round-trip
|
| 242 |
+
print("\nCSR round-trip test:")
|
| 243 |
+
original_csr = csr_matrix([1, -2, 3])
|
| 244 |
+
str_repr_csr = vector_to_string(original_csr)
|
| 245 |
+
recovered_csr = string_to_vector(str_repr_csr)
|
| 246 |
+
print(f" Original CSR: {original_csr.toarray().flatten()}")
|
| 247 |
+
print(f" String: '{str_repr_csr}'")
|
| 248 |
+
print(f" Recovered CSR: {recovered_csr.toarray().flatten()}")
|
| 249 |
+
print(f" Match: {np.allclose(original_csr.toarray().flatten(), recovered_csr.toarray().flatten())}")
|
| 250 |
+
print(f" Both are CSR: {isinstance(original_csr, csr_matrix) and isinstance(recovered_csr, csr_matrix)}")
|
mce220_runner/dataset/README.md
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pretty_name: Multi-Construction Entropy-Proof Benchmark (MCE-220)
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
size_categories:
|
| 6 |
+
- n<1K
|
| 7 |
+
tags:
|
| 8 |
+
- benchmark
|
| 9 |
+
- information-theory
|
| 10 |
+
- entropy
|
| 11 |
+
- automated-theorem-proving
|
| 12 |
+
- synthetic
|
| 13 |
+
configs:
|
| 14 |
+
- config_name: default
|
| 15 |
+
data_files:
|
| 16 |
+
- split: test
|
| 17 |
+
path: data/questions.jsonl
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# Multi-Construction Entropy-Proof Benchmark (MCE-220)
|
| 21 |
+
|
| 22 |
+
MCE-220 is a frozen benchmark of 220 synthetic Shannon-type entropy-inequality proof problems. It spans 10 through 20 random variables with 20 questions at every value of `n`.
|
| 23 |
+
|
| 24 |
+
The benchmark is balanced across five construction families. At every `n`, it contains four questions from each family:
|
| 25 |
+
|
| 26 |
+
- weighted modular / hypergraphical constructions;
|
| 27 |
+
- finite-field linearly representable constructions over GF(61);
|
| 28 |
+
- directed-acyclic-graph constructions;
|
| 29 |
+
- deterministic finite-field functional constructions; and
|
| 30 |
+
- finite abelian 2-group quotient constructions.
|
| 31 |
+
|
| 32 |
+
It is also divided evenly into 110 condition-required questions and 110 Shannon-only questions. The benchmark is intended for evaluating automated entropy-inequality provers, proof-search systems, and exact symbolic verification pipelines.
|
| 33 |
+
|
| 34 |
+
## Dataset structure
|
| 35 |
+
|
| 36 |
+
The Hub dataset exposes one `test` split from `data/questions.jsonl`. Each of its 220 records has:
|
| 37 |
+
|
| 38 |
+
- `id`: stable benchmark identifier;
|
| 39 |
+
- `n`: number of random variables;
|
| 40 |
+
- `ineq`: target entropy expression, interpreted as `ineq >= 0`;
|
| 41 |
+
- `conditions`: supplied equality conditions, represented as paired one-sided inequalities; and
|
| 42 |
+
- `sha256`: digest of the canonical semantic question representation.
|
| 43 |
+
|
| 44 |
+
The files under `audit/` are reproducibility evidence and are intentionally excluded from the dataset split:
|
| 45 |
+
|
| 46 |
+
- `provenance.jsonl`: construction witness, planted certificate, tokenization, seeds, and structural metadata for every question;
|
| 47 |
+
- `generation-report.jsonl`: per-cell generation and rejection accounting;
|
| 48 |
+
- `run-manifest.json`: frozen configuration, dependency hashes, tokenizer identity, output hashes, and completion summary;
|
| 49 |
+
- `validation-report.json`: exact certificate and artifact-integrity audit; and
|
| 50 |
+
- `diversity-audit.json`: balance, uniqueness, and structural-diversity audit.
|
| 51 |
+
|
| 52 |
+
The generation configuration is preserved as `generation/e2n_v2.toml`.
|
| 53 |
+
|
| 54 |
+
## Generation and tokenization
|
| 55 |
+
|
| 56 |
+
MCE-220 was generated from a preregistered grid of 11 values of `n`, five construction families, and four proof recipes. Each selected problem retains a planted exact certificate and a construction witness. Token counts use the fast tokenizer for `Qwen/Qwen3-0.6B` at commit `c1899de289a04d12100db370d81485cdf75e47ca`, without special tokens.
|
| 57 |
+
|
| 58 |
+
The five construction families provide complementary entropic witnesses: modular component coverage, finite-field matrix ranks, rational Bayesian networks, deterministic polynomial transformations, and quotient profiles of finite abelian groups.
|
| 59 |
+
|
| 60 |
+
## Validation and integrity
|
| 61 |
+
|
| 62 |
+
The frozen package contains 220 unique question records and 220 matching provenance records. All selected certificates pass exact sparse rational verification. The validation report records zero errors, and the diversity audit passes all declared balance and uniqueness checks.
|
| 63 |
+
|
| 64 |
+
Primary artifact SHA-256 values:
|
| 65 |
+
|
| 66 |
+
- `questions.jsonl`: `1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a`
|
| 67 |
+
- `provenance.jsonl`: `cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189`
|
| 68 |
+
- `generation-report.jsonl`: `50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c`
|
| 69 |
+
|
| 70 |
+
## Intended use and limitations
|
| 71 |
+
|
| 72 |
+
MCE-220 is an evaluation benchmark, not a training corpus or a representative sample of every entropy inequality. Its questions are synthetic and constructed to have planted Shannon-type certificates. The controlled sparse construction does not establish performance on arbitrary dense inequalities or non-Shannon inequalities. Planted certificate length is not necessarily the minimum proof length, because a question may admit other valid proofs.
|
| 73 |
+
|
| 74 |
+
The benchmark supplies equality conditions because the evaluated AITIP workflow supports equality premises. Results should retain question-level accounting, exact proof verification, and resource or timeout settings.
|
| 75 |
+
|
| 76 |
+
## Citation
|
| 77 |
+
|
| 78 |
+
The associated manuscript is forthcoming/unpublished:
|
| 79 |
+
|
| 80 |
+
```bibtex
|
| 81 |
+
@unpublished{wong2026automated,
|
| 82 |
+
title = {Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search},
|
| 83 |
+
author = {Wong, Shing Yin and Liu, Shaocheng and Song, Linqi and Gohari, Amin and Li, Cheuk Ting},
|
| 84 |
+
year = {2026},
|
| 85 |
+
note = {Manuscript in preparation}
|
| 86 |
+
}
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
No reuse license has been assigned to this private release.
|
mce220_runner/dataset/audit/diversity-audit.json
ADDED
|
@@ -0,0 +1,422 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"counts": {
|
| 3 |
+
"by_method_n": {
|
| 4 |
+
"abelian_product_v2": {
|
| 5 |
+
"10": 4,
|
| 6 |
+
"11": 4,
|
| 7 |
+
"12": 4,
|
| 8 |
+
"13": 4,
|
| 9 |
+
"14": 4,
|
| 10 |
+
"15": 4,
|
| 11 |
+
"16": 4,
|
| 12 |
+
"17": 4,
|
| 13 |
+
"18": 4,
|
| 14 |
+
"19": 4,
|
| 15 |
+
"20": 4
|
| 16 |
+
},
|
| 17 |
+
"dag_v2": {
|
| 18 |
+
"10": 4,
|
| 19 |
+
"11": 4,
|
| 20 |
+
"12": 4,
|
| 21 |
+
"13": 4,
|
| 22 |
+
"14": 4,
|
| 23 |
+
"15": 4,
|
| 24 |
+
"16": 4,
|
| 25 |
+
"17": 4,
|
| 26 |
+
"18": 4,
|
| 27 |
+
"19": 4,
|
| 28 |
+
"20": 4
|
| 29 |
+
},
|
| 30 |
+
"functional_v2": {
|
| 31 |
+
"10": 4,
|
| 32 |
+
"11": 4,
|
| 33 |
+
"12": 4,
|
| 34 |
+
"13": 4,
|
| 35 |
+
"14": 4,
|
| 36 |
+
"15": 4,
|
| 37 |
+
"16": 4,
|
| 38 |
+
"17": 4,
|
| 39 |
+
"18": 4,
|
| 40 |
+
"19": 4,
|
| 41 |
+
"20": 4
|
| 42 |
+
},
|
| 43 |
+
"linear_rank_gf61_v2": {
|
| 44 |
+
"10": 4,
|
| 45 |
+
"11": 4,
|
| 46 |
+
"12": 4,
|
| 47 |
+
"13": 4,
|
| 48 |
+
"14": 4,
|
| 49 |
+
"15": 4,
|
| 50 |
+
"16": 4,
|
| 51 |
+
"17": 4,
|
| 52 |
+
"18": 4,
|
| 53 |
+
"19": 4,
|
| 54 |
+
"20": 4
|
| 55 |
+
},
|
| 56 |
+
"weighted_modular_v2": {
|
| 57 |
+
"10": 4,
|
| 58 |
+
"11": 4,
|
| 59 |
+
"12": 4,
|
| 60 |
+
"13": 4,
|
| 61 |
+
"14": 4,
|
| 62 |
+
"15": 4,
|
| 63 |
+
"16": 4,
|
| 64 |
+
"17": 4,
|
| 65 |
+
"18": 4,
|
| 66 |
+
"19": 4,
|
| 67 |
+
"20": 4
|
| 68 |
+
}
|
| 69 |
+
},
|
| 70 |
+
"by_n": {
|
| 71 |
+
"10": 20,
|
| 72 |
+
"11": 20,
|
| 73 |
+
"12": 20,
|
| 74 |
+
"13": 20,
|
| 75 |
+
"14": 20,
|
| 76 |
+
"15": 20,
|
| 77 |
+
"16": 20,
|
| 78 |
+
"17": 20,
|
| 79 |
+
"18": 20,
|
| 80 |
+
"19": 20,
|
| 81 |
+
"20": 20
|
| 82 |
+
},
|
| 83 |
+
"by_residual_mode": {
|
| 84 |
+
"condition": 110,
|
| 85 |
+
"shannon": 110
|
| 86 |
+
},
|
| 87 |
+
"by_variant_method_n": {
|
| 88 |
+
"abelian_product_v2": {
|
| 89 |
+
"10": {
|
| 90 |
+
"abelian_product_v2": 4
|
| 91 |
+
},
|
| 92 |
+
"11": {
|
| 93 |
+
"abelian_product_v2": 4
|
| 94 |
+
},
|
| 95 |
+
"12": {
|
| 96 |
+
"abelian_product_v2": 4
|
| 97 |
+
},
|
| 98 |
+
"13": {
|
| 99 |
+
"abelian_product_v2": 4
|
| 100 |
+
},
|
| 101 |
+
"14": {
|
| 102 |
+
"abelian_product_v2": 4
|
| 103 |
+
},
|
| 104 |
+
"15": {
|
| 105 |
+
"abelian_product_v2": 4
|
| 106 |
+
},
|
| 107 |
+
"16": {
|
| 108 |
+
"abelian_product_v2": 4
|
| 109 |
+
},
|
| 110 |
+
"17": {
|
| 111 |
+
"abelian_product_v2": 4
|
| 112 |
+
},
|
| 113 |
+
"18": {
|
| 114 |
+
"abelian_product_v2": 4
|
| 115 |
+
},
|
| 116 |
+
"19": {
|
| 117 |
+
"abelian_product_v2": 4
|
| 118 |
+
},
|
| 119 |
+
"20": {
|
| 120 |
+
"abelian_product_v2": 4
|
| 121 |
+
}
|
| 122 |
+
},
|
| 123 |
+
"dag_v2": {
|
| 124 |
+
"10": {
|
| 125 |
+
"dag_v2": 4
|
| 126 |
+
},
|
| 127 |
+
"11": {
|
| 128 |
+
"dag_v2": 4
|
| 129 |
+
},
|
| 130 |
+
"12": {
|
| 131 |
+
"dag_v2": 4
|
| 132 |
+
},
|
| 133 |
+
"13": {
|
| 134 |
+
"dag_v2": 4
|
| 135 |
+
},
|
| 136 |
+
"14": {
|
| 137 |
+
"dag_v2": 4
|
| 138 |
+
},
|
| 139 |
+
"15": {
|
| 140 |
+
"dag_v2": 4
|
| 141 |
+
},
|
| 142 |
+
"16": {
|
| 143 |
+
"dag_v2": 4
|
| 144 |
+
},
|
| 145 |
+
"17": {
|
| 146 |
+
"dag_v2": 4
|
| 147 |
+
},
|
| 148 |
+
"18": {
|
| 149 |
+
"dag_v2": 4
|
| 150 |
+
},
|
| 151 |
+
"19": {
|
| 152 |
+
"dag_v2": 4
|
| 153 |
+
},
|
| 154 |
+
"20": {
|
| 155 |
+
"dag_v2": 4
|
| 156 |
+
}
|
| 157 |
+
},
|
| 158 |
+
"functional_v2": {
|
| 159 |
+
"10": {
|
| 160 |
+
"linear": 1,
|
| 161 |
+
"mixed": 2,
|
| 162 |
+
"nonlinear": 1
|
| 163 |
+
},
|
| 164 |
+
"11": {
|
| 165 |
+
"linear": 1,
|
| 166 |
+
"mixed": 2,
|
| 167 |
+
"nonlinear": 1
|
| 168 |
+
},
|
| 169 |
+
"12": {
|
| 170 |
+
"linear": 1,
|
| 171 |
+
"mixed": 2,
|
| 172 |
+
"nonlinear": 1
|
| 173 |
+
},
|
| 174 |
+
"13": {
|
| 175 |
+
"linear": 1,
|
| 176 |
+
"mixed": 2,
|
| 177 |
+
"nonlinear": 1
|
| 178 |
+
},
|
| 179 |
+
"14": {
|
| 180 |
+
"linear": 1,
|
| 181 |
+
"mixed": 2,
|
| 182 |
+
"nonlinear": 1
|
| 183 |
+
},
|
| 184 |
+
"15": {
|
| 185 |
+
"linear": 1,
|
| 186 |
+
"mixed": 2,
|
| 187 |
+
"nonlinear": 1
|
| 188 |
+
},
|
| 189 |
+
"16": {
|
| 190 |
+
"linear": 1,
|
| 191 |
+
"mixed": 2,
|
| 192 |
+
"nonlinear": 1
|
| 193 |
+
},
|
| 194 |
+
"17": {
|
| 195 |
+
"linear": 1,
|
| 196 |
+
"mixed": 2,
|
| 197 |
+
"nonlinear": 1
|
| 198 |
+
},
|
| 199 |
+
"18": {
|
| 200 |
+
"linear": 1,
|
| 201 |
+
"mixed": 2,
|
| 202 |
+
"nonlinear": 1
|
| 203 |
+
},
|
| 204 |
+
"19": {
|
| 205 |
+
"linear": 1,
|
| 206 |
+
"mixed": 2,
|
| 207 |
+
"nonlinear": 1
|
| 208 |
+
},
|
| 209 |
+
"20": {
|
| 210 |
+
"linear": 1,
|
| 211 |
+
"mixed": 2,
|
| 212 |
+
"nonlinear": 1
|
| 213 |
+
}
|
| 214 |
+
},
|
| 215 |
+
"linear_rank_gf61_v2": {
|
| 216 |
+
"10": {
|
| 217 |
+
"linear_rank_gf61_v2": 4
|
| 218 |
+
},
|
| 219 |
+
"11": {
|
| 220 |
+
"linear_rank_gf61_v2": 4
|
| 221 |
+
},
|
| 222 |
+
"12": {
|
| 223 |
+
"linear_rank_gf61_v2": 4
|
| 224 |
+
},
|
| 225 |
+
"13": {
|
| 226 |
+
"linear_rank_gf61_v2": 4
|
| 227 |
+
},
|
| 228 |
+
"14": {
|
| 229 |
+
"linear_rank_gf61_v2": 4
|
| 230 |
+
},
|
| 231 |
+
"15": {
|
| 232 |
+
"linear_rank_gf61_v2": 4
|
| 233 |
+
},
|
| 234 |
+
"16": {
|
| 235 |
+
"linear_rank_gf61_v2": 4
|
| 236 |
+
},
|
| 237 |
+
"17": {
|
| 238 |
+
"linear_rank_gf61_v2": 4
|
| 239 |
+
},
|
| 240 |
+
"18": {
|
| 241 |
+
"linear_rank_gf61_v2": 4
|
| 242 |
+
},
|
| 243 |
+
"19": {
|
| 244 |
+
"linear_rank_gf61_v2": 4
|
| 245 |
+
},
|
| 246 |
+
"20": {
|
| 247 |
+
"linear_rank_gf61_v2": 4
|
| 248 |
+
}
|
| 249 |
+
},
|
| 250 |
+
"weighted_modular_v2": {
|
| 251 |
+
"10": {
|
| 252 |
+
"shared": 2,
|
| 253 |
+
"weighted": 2
|
| 254 |
+
},
|
| 255 |
+
"11": {
|
| 256 |
+
"shared": 2,
|
| 257 |
+
"weighted": 2
|
| 258 |
+
},
|
| 259 |
+
"12": {
|
| 260 |
+
"shared": 2,
|
| 261 |
+
"weighted": 2
|
| 262 |
+
},
|
| 263 |
+
"13": {
|
| 264 |
+
"shared": 2,
|
| 265 |
+
"weighted": 2
|
| 266 |
+
},
|
| 267 |
+
"14": {
|
| 268 |
+
"shared": 2,
|
| 269 |
+
"weighted": 2
|
| 270 |
+
},
|
| 271 |
+
"15": {
|
| 272 |
+
"shared": 2,
|
| 273 |
+
"weighted": 2
|
| 274 |
+
},
|
| 275 |
+
"16": {
|
| 276 |
+
"shared": 2,
|
| 277 |
+
"weighted": 2
|
| 278 |
+
},
|
| 279 |
+
"17": {
|
| 280 |
+
"shared": 2,
|
| 281 |
+
"weighted": 2
|
| 282 |
+
},
|
| 283 |
+
"18": {
|
| 284 |
+
"shared": 2,
|
| 285 |
+
"weighted": 2
|
| 286 |
+
},
|
| 287 |
+
"19": {
|
| 288 |
+
"shared": 2,
|
| 289 |
+
"weighted": 2
|
| 290 |
+
},
|
| 291 |
+
"20": {
|
| 292 |
+
"shared": 2,
|
| 293 |
+
"weighted": 2
|
| 294 |
+
}
|
| 295 |
+
}
|
| 296 |
+
},
|
| 297 |
+
"complete_cells": 220,
|
| 298 |
+
"generation_cells": 220,
|
| 299 |
+
"provenance": 220,
|
| 300 |
+
"questions": 220,
|
| 301 |
+
"structural_signature_count_by_method_n": {
|
| 302 |
+
"abelian_product_v2": {
|
| 303 |
+
"10": 4,
|
| 304 |
+
"11": 4,
|
| 305 |
+
"12": 4,
|
| 306 |
+
"13": 4,
|
| 307 |
+
"14": 4,
|
| 308 |
+
"15": 4,
|
| 309 |
+
"16": 4,
|
| 310 |
+
"17": 4,
|
| 311 |
+
"18": 4,
|
| 312 |
+
"19": 4,
|
| 313 |
+
"20": 4
|
| 314 |
+
},
|
| 315 |
+
"dag_v2": {
|
| 316 |
+
"10": 4,
|
| 317 |
+
"11": 4,
|
| 318 |
+
"12": 4,
|
| 319 |
+
"13": 4,
|
| 320 |
+
"14": 4,
|
| 321 |
+
"15": 4,
|
| 322 |
+
"16": 4,
|
| 323 |
+
"17": 4,
|
| 324 |
+
"18": 4,
|
| 325 |
+
"19": 4,
|
| 326 |
+
"20": 4
|
| 327 |
+
},
|
| 328 |
+
"functional_v2": {
|
| 329 |
+
"10": 4,
|
| 330 |
+
"11": 4,
|
| 331 |
+
"12": 4,
|
| 332 |
+
"13": 4,
|
| 333 |
+
"14": 4,
|
| 334 |
+
"15": 4,
|
| 335 |
+
"16": 4,
|
| 336 |
+
"17": 4,
|
| 337 |
+
"18": 4,
|
| 338 |
+
"19": 4,
|
| 339 |
+
"20": 4
|
| 340 |
+
},
|
| 341 |
+
"linear_rank_gf61_v2": {
|
| 342 |
+
"10": 4,
|
| 343 |
+
"11": 4,
|
| 344 |
+
"12": 4,
|
| 345 |
+
"13": 4,
|
| 346 |
+
"14": 4,
|
| 347 |
+
"15": 4,
|
| 348 |
+
"16": 4,
|
| 349 |
+
"17": 4,
|
| 350 |
+
"18": 4,
|
| 351 |
+
"19": 4,
|
| 352 |
+
"20": 4
|
| 353 |
+
},
|
| 354 |
+
"weighted_modular_v2": {
|
| 355 |
+
"10": 2,
|
| 356 |
+
"11": 2,
|
| 357 |
+
"12": 2,
|
| 358 |
+
"13": 2,
|
| 359 |
+
"14": 2,
|
| 360 |
+
"15": 2,
|
| 361 |
+
"16": 2,
|
| 362 |
+
"17": 2,
|
| 363 |
+
"18": 2,
|
| 364 |
+
"19": 2,
|
| 365 |
+
"20": 2
|
| 366 |
+
}
|
| 367 |
+
}
|
| 368 |
+
},
|
| 369 |
+
"error_count": 0,
|
| 370 |
+
"errors": [],
|
| 371 |
+
"expectations": {
|
| 372 |
+
"expected_per_method_n": 4,
|
| 373 |
+
"expected_per_n": 20,
|
| 374 |
+
"expected_total": 220,
|
| 375 |
+
"methods": [
|
| 376 |
+
"weighted_modular_v2",
|
| 377 |
+
"linear_rank_gf61_v2",
|
| 378 |
+
"dag_v2",
|
| 379 |
+
"functional_v2",
|
| 380 |
+
"abelian_product_v2"
|
| 381 |
+
],
|
| 382 |
+
"n_values": [
|
| 383 |
+
10,
|
| 384 |
+
11,
|
| 385 |
+
12,
|
| 386 |
+
13,
|
| 387 |
+
14,
|
| 388 |
+
15,
|
| 389 |
+
16,
|
| 390 |
+
17,
|
| 391 |
+
18,
|
| 392 |
+
19,
|
| 393 |
+
20
|
| 394 |
+
],
|
| 395 |
+
"require_complete_cells": true,
|
| 396 |
+
"require_exact_uniqueness": true,
|
| 397 |
+
"residual_modes": [
|
| 398 |
+
"condition",
|
| 399 |
+
"shannon"
|
| 400 |
+
]
|
| 401 |
+
},
|
| 402 |
+
"generation_report_rows": 220,
|
| 403 |
+
"pass": true,
|
| 404 |
+
"rejections": {
|
| 405 |
+
"by_reason": {
|
| 406 |
+
"token_window": 8
|
| 407 |
+
},
|
| 408 |
+
"total": 8
|
| 409 |
+
},
|
| 410 |
+
"schema_version": "reviewer-witness-diversity-audit-v1",
|
| 411 |
+
"uniqueness": {
|
| 412 |
+
"duplicate_condition_set_groups": [],
|
| 413 |
+
"duplicate_initial_target_groups": [],
|
| 414 |
+
"duplicate_selected_residual_groups": [],
|
| 415 |
+
"duplicate_witness_groups": [],
|
| 416 |
+
"unique_condition_sets": 220,
|
| 417 |
+
"unique_initial_targets": 220,
|
| 418 |
+
"unique_selected_residuals": 220,
|
| 419 |
+
"unique_witnesses": 220,
|
| 420 |
+
"witness_scope": "within each method and n"
|
| 421 |
+
}
|
| 422 |
+
}
|
mce220_runner/dataset/audit/generation-report.jsonl
ADDED
|
@@ -0,0 +1,220 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3701457636437077952,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 2 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15246363881976218358,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 3 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3172063088685357728,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 4 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5056663783486606013,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 5 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9539850679019901972,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 6 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7955716649502997116,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 7 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4812851374726980080,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 8 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4284913358355807734,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 9 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8708808592976883946,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 10 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14394515690009120654,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 11 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5074899767269115176,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 12 |
+
{"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":8138142372704579071,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":12,"quota":1,"recipe_index":3,"rejected":{"token_window":1},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 13 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11237412930374853913,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 14 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2622604060345787656,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 15 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1297526053728799686,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 16 |
+
{"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":12938408747832804044,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":13,"quota":1,"recipe_index":3,"rejected":{"token_window":1},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 17 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15148184769047820,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 18 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11212742246525318633,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 19 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9952603600299452875,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 20 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17684431903806972992,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 21 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4358867344586403829,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 22 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13909308547342205491,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 23 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15975291380394989210,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 24 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9716087254387917306,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 25 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17539831590544119906,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 26 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11853313365675990415,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 27 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10445827094562493869,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 28 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4702628459963138141,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 29 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6338436637880042951,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 30 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4623525552571304476,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 31 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8389339039156670354,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 32 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16353085863076184214,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 33 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2196169065690155443,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 34 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15059909006415722323,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 35 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14583201521060589027,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 36 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5612252589744170130,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 37 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2728962570584640889,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 38 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9758846155311160259,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 39 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2720845152774216335,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 40 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5121022496212244148,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 41 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16436906769862432774,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 42 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14831573543779349003,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 43 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4480046151243417079,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"weighted"}
|
| 44 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9903418647447938583,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"weighted_modular_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"shared"}
|
| 45 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4936832681239338862,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 46 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15232824178863167110,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 47 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8186240528289594329,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 48 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17500194849731588113,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 49 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11832034070095118311,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 50 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8133750959736225035,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 51 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2273770823316088623,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 52 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1462948322211576970,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 53 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18067028359118754372,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 54 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11705998351666552518,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 55 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16254702609706164343,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 56 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2968902127481300008,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 57 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5936000122885923694,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 58 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15604122199806785487,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 59 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9337414342802015969,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 60 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16923520896556911049,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 61 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1464669209368047966,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 62 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10174962824566969569,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 63 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12100853506115398562,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 64 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5228631607116939670,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 65 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6393201095674814746,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 66 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15285263391928485372,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 67 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":549978287588786307,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 68 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5273822472737093431,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 69 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17505444870217075492,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 70 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12065121403697794235,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 71 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15963481462025769689,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 72 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9888338464856776367,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 73 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11743591167866989887,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 74 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5407253042649969843,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 75 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5124340889268399352,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 76 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7866954243082797305,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 77 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7141475136732988801,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 78 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16881613849503902662,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 79 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10510564794829486452,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 80 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13637027472625371970,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 81 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10865444810274626052,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 82 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16407715404674462704,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 83 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2967182459071344429,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 84 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8842815126698378794,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 85 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16996992628039077843,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 86 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11919690030700392770,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 87 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2310562788219762726,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 88 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2443067625292808296,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"linear_rank_gf61_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 89 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3184567625312884139,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 90 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6822340930228026675,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 91 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11359352071557003676,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 92 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11260656603170133794,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 93 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17431958280368089774,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 94 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":95470021134927501,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 95 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18173423042958114467,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 96 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10912102982755826622,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 97 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14647759189369950544,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 98 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17074290522542464048,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 99 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11051767266233656569,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 100 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1918497853187515936,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 101 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1907240975364747342,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 102 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17785309102802652106,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 103 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7181223247123743876,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 104 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2263938430705632913,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 105 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16433270489078283260,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 106 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15721389692426090446,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 107 |
+
{"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":16971901269964792975,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":14,"quota":1,"recipe_index":2,"rejected":{"token_window":1},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 108 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17994565298475579634,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 109 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8007488871997086332,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 110 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3719582981595767999,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 111 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6989838346124019281,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 112 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2665890865320848013,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 113 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14437300812821433667,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 114 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6577469201406442179,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 115 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13041507654325523808,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 116 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3570771541096614982,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 117 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11815261720096583648,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 118 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16648488661857383898,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 119 |
+
{"accepted":1,"attempted":3,"benchmark":"e2n-witness-first-v2","cell_seed":12258752852871151798,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":17,"quota":1,"recipe_index":2,"rejected":{"token_window":2},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 120 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6381643106495975100,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 121 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15423199845599069898,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 122 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13268465811212531448,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 123 |
+
{"accepted":1,"attempted":2,"benchmark":"e2n-witness-first-v2","cell_seed":4070565829160279811,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":18,"quota":1,"recipe_index":2,"rejected":{"token_window":1},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 124 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":105930261327445421,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 125 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10109908015276543465,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 126 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4956562969327863550,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 127 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5008331369838333772,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 128 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17827741488378490065,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 129 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11552473092704060,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"dag_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 130 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17278969316211330178,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"dag_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 131 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5863619077421464638,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"dag_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 132 |
+
{"accepted":1,"attempted":3,"benchmark":"e2n-witness-first-v2","cell_seed":12216008678726813341,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"dag_v2","n":20,"quota":1,"recipe_index":3,"rejected":{"token_window":2},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 133 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12136638063071141276,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 134 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6986443780866752496,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 135 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3781574741278654854,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 136 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2343031434917169256,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 137 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":530243949934307503,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 138 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4147310449574325073,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 139 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13253479536535202397,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 140 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5481697251196607234,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 141 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3419907753431159668,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 142 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18349149685442145724,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 143 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12471696088800926995,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 144 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5514582684015399052,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 145 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14479113219010784219,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 146 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7511176911362227743,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 147 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11305014799087235417,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 148 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10456612778851741111,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 149 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17198548155075593163,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 150 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12054220294986305355,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 151 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6317325096853017294,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 152 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5557861230106346201,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 153 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15256452385745187341,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 154 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4984913849055553761,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 155 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1922809319832171575,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 156 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7290953918310180562,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 157 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2207934633545694466,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 158 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8865819739701358330,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 159 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8060021619136953608,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 160 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16572719200988242280,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 161 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4170973570952992437,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 162 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12513124999520014137,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 163 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14786551003661347752,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 164 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7575961521772144425,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 165 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5161629846671816961,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 166 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4552954781211765265,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 167 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12954501444229679227,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 168 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3734117315160446273,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 169 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15062695363987721571,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 170 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12101175090757923717,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 171 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3575707971365674080,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 172 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5105430873146479519,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 173 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15318398621928381177,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"functional_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"linear"}
|
| 174 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13391810045493605038,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"functional_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"nonlinear"}
|
| 175 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":16474088650880862895,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"functional_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 176 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9697458621349092067,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"functional_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":"mixed"}
|
| 177 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14098164715461887531,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 178 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":7413113805965221460,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 179 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8118303950565053392,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 180 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1018118677028597230,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":10,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 181 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14973580586272694633,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 182 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1362989527561847042,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 183 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":14381275516087489947,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 184 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2369240884198343729,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":11,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 185 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":221228626003247890,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 186 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4312760315371342800,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 187 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15528857495560036646,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 188 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1186195495943774426,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":12,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 189 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10014313637976231552,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 190 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1552613301840816399,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 191 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":954177495762986129,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 192 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5525319345874424799,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":13,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 193 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18027165872156795930,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 194 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18098129184875256583,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 195 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13888201606725553085,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 196 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":919212854944883606,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":14,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 197 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8165437892884726391,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 198 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11934378534096084033,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 199 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":12133732172664841221,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 200 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5360334427668528317,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":15,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 201 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5030154206041038858,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 202 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11647864275748426632,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 203 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":4062663468625614856,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 204 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":18407997404765423429,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":16,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 205 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":13118711139945243757,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 206 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3758826947236076007,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 207 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":11246400704042830426,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 208 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":10213094060571213725,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":17,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 209 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":9084829840437732960,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 210 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":5453944936310748968,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 211 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8525833265372454032,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 212 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":2691777673477642656,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":18,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 213 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6285703479023902455,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 214 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":3138362092886322139,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 215 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1071160566897403333,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 216 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":17567046285549651617,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":19,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 217 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":6736118940594014540,"complete":true,"condition_width":8,"k":12,"m_eq":1,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":0,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 218 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":15176902604380923354,"complete":true,"condition_width":12,"k":16,"m_eq":1,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":1,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 219 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":8216170727549535292,"complete":true,"condition_width":8,"k":20,"m_eq":2,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":2,"rejected":{},"residual_mode":"condition","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
| 220 |
+
{"accepted":1,"attempted":1,"benchmark":"e2n-witness-first-v2","cell_seed":1392148745951243703,"complete":true,"condition_width":8,"k":16,"m_eq":2,"method":"abelian_product_v2","n":20,"quota":1,"recipe_index":3,"rejected":{},"residual_mode":"shannon","schema_version":"reviewer-benchmark-v1","witness_variant":null}
|
mce220_runner/dataset/audit/provenance.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
mce220_runner/dataset/audit/run-manifest.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"benchmark":"e2n-witness-first-v2","config":{"benchmark":{"coefficient_max":10,"max_attempts_per_cell":240,"name":"e2n-witness-first-v2","output_dir":"../outputs/e2n-v2","quota_per_cell":1,"require_complete":true,"require_unique_witness":true,"seed":20260825},"format":{"prompt_template":"TARGET:\n{target}>=0\n\nCONDITIONS:\n{conditions}\n\n"},"grid":{"methods":["weighted_modular_v2","linear_rank_gf61_v2","dag_v2","functional_v2","abelian_product_v2"],"n_values":[10,11,12,13,14,15,16,17,18,19,20],"recipes":[{"condition_width":8,"k":12,"m_eq":1,"residual_mode":"condition","witness_variants":{"functional_v2":"linear","weighted_modular_v2":"weighted"}},{"condition_width":12,"k":16,"m_eq":1,"residual_mode":"shannon","witness_variants":{"functional_v2":"nonlinear","weighted_modular_v2":"shared"}},{"condition_width":8,"k":20,"m_eq":2,"residual_mode":"condition","witness_variants":{"functional_v2":"mixed","weighted_modular_v2":"weighted"}},{"condition_width":8,"k":16,"m_eq":2,"residual_mode":"shannon","witness_variants":{"functional_v2":"mixed","weighted_modular_v2":"shared"}}]},"tokenization":{"counter":"huggingface","full_max":2200,"full_min":801,"hf_model":"Qwen/Qwen3-0.6B","hf_revision":"c1899de289a04d12100db370d81485cdf75e47ca","hf_use_fast":true,"target_max":799,"target_min":501}},"config_sha256":"1b1f298ed012071c85e9cfb286fe5dcf7c183383de92041754d8a7802f426421","generator_dependency_sha256":{"data/finite_field.py":"c9738c10a16ea2cfb3cde346a23673d1536dbfce491f8871346f441b9b4c2a6e","data/witness_first.py":"7af2a1d6096862a776e9c8530d0c6a3ae2b83aa7ecd8bb1026256d6f1cdc3560","data/witness_v2_algebraic.py":"ffbd48eb673a3f6f842327d00431285bbae8963a7e20f6956e37e544fa37165f","data/witness_v2_graph.py":"c81adcfefcf6ffc0e816f1b338af13323bc80e06293e8cac6b28a3b0ae2efc8a"},"generator_source_sha256":"7af2a1d6096862a776e9c8530d0c6a3ae2b83aa7ecd8bb1026256d6f1cdc3560","output_sha256":{"generation-report.jsonl":"50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c","provenance.jsonl":"cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189","questions.jsonl":"1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a"},"schema_version":"reviewer-benchmark-v1","selector_source_sha256":"83a328168eac42cedd3412a9c8b5624b8683a52a14042b5defbccf6d833efefe","summary":{"benchmark":"e2n-witness-first-v2","cells":220,"complete":true,"complete_cells":220,"questions":220},"tokenizer":{"add_special_tokens":false,"is_fast":true,"kind":"huggingface","model":"Qwen/Qwen3-0.6B","requested_revision":"c1899de289a04d12100db370d81485cdf75e47ca","resolved_commit":"c1899de289a04d12100db370d81485cdf75e47ca","tokenizer_class":"Qwen2TokenizerFast","transformers_version":"4.55.2","trust_remote_code":true,"use_fast":true}}
|
mce220_runner/dataset/audit/validation-report.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"benchmark":"e2n-witness-first-v2","benchmark_dir":"C:\\Users\\user\\Documents\\ChatGPT\\Entropy Inequality\\experiments\\reviewer_response\\outputs\\e2n-v2","counts":{"artifact_hashes_checked":3,"certificates_verified":220,"errors":0,"provenance_rows":220,"question_rows":220,"rows_checked":220},"errors":[],"schema_version":"reviewer-generated-benchmark-audit-v1","valid":true}
|
mce220_runner/dataset/data/questions.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
mce220_runner/dataset/generation/e2n_v2.toml
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[benchmark]
|
| 2 |
+
name = "e2n-witness-first-v2"
|
| 3 |
+
seed = 20260825
|
| 4 |
+
output_dir = "../outputs/e2n-v2"
|
| 5 |
+
quota_per_cell = 1
|
| 6 |
+
max_attempts_per_cell = 240
|
| 7 |
+
coefficient_max = 10
|
| 8 |
+
require_complete = true
|
| 9 |
+
require_unique_witness = true
|
| 10 |
+
|
| 11 |
+
[grid]
|
| 12 |
+
methods = [
|
| 13 |
+
"weighted_modular_v2",
|
| 14 |
+
"linear_rank_gf61_v2",
|
| 15 |
+
"dag_v2",
|
| 16 |
+
"functional_v2",
|
| 17 |
+
"abelian_product_v2",
|
| 18 |
+
]
|
| 19 |
+
n_values = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
|
| 20 |
+
|
| 21 |
+
# Four questions per construction family and n. The model sees equality
|
| 22 |
+
# conditions only. "condition" means at least one equality action remains in
|
| 23 |
+
# the selected proof-completion state; "shannon" means none remains.
|
| 24 |
+
[[grid.recipes]]
|
| 25 |
+
k = 12
|
| 26 |
+
m_eq = 1
|
| 27 |
+
condition_width = 8
|
| 28 |
+
residual_mode = "condition"
|
| 29 |
+
witness_variants = { weighted_modular_v2 = "weighted", functional_v2 = "linear" }
|
| 30 |
+
|
| 31 |
+
[[grid.recipes]]
|
| 32 |
+
k = 16
|
| 33 |
+
m_eq = 1
|
| 34 |
+
condition_width = 12
|
| 35 |
+
residual_mode = "shannon"
|
| 36 |
+
witness_variants = { weighted_modular_v2 = "shared", functional_v2 = "nonlinear" }
|
| 37 |
+
|
| 38 |
+
[[grid.recipes]]
|
| 39 |
+
k = 20
|
| 40 |
+
m_eq = 2
|
| 41 |
+
condition_width = 8
|
| 42 |
+
residual_mode = "condition"
|
| 43 |
+
witness_variants = { weighted_modular_v2 = "weighted", functional_v2 = "mixed" }
|
| 44 |
+
|
| 45 |
+
[[grid.recipes]]
|
| 46 |
+
k = 16
|
| 47 |
+
m_eq = 2
|
| 48 |
+
condition_width = 8
|
| 49 |
+
residual_mode = "shannon"
|
| 50 |
+
witness_variants = { weighted_modular_v2 = "shared", functional_v2 = "mixed" }
|
| 51 |
+
|
| 52 |
+
[tokenization]
|
| 53 |
+
counter = "huggingface"
|
| 54 |
+
hf_model = "Qwen/Qwen3-0.6B"
|
| 55 |
+
hf_revision = "c1899de289a04d12100db370d81485cdf75e47ca"
|
| 56 |
+
hf_use_fast = true
|
| 57 |
+
target_min = 501
|
| 58 |
+
target_max = 799
|
| 59 |
+
full_min = 801
|
| 60 |
+
full_max = 2200
|
| 61 |
+
|
| 62 |
+
[format]
|
| 63 |
+
prompt_template = "TARGET:\n{target}>=0\n\nCONDITIONS:\n{conditions}\n\n"
|
mce220_runner/download_assets.py
ADDED
|
@@ -0,0 +1,249 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Download and verify the pinned MCE-220 inference assets."""
|
| 3 |
+
|
| 4 |
+
from __future__ import annotations
|
| 5 |
+
|
| 6 |
+
import argparse
|
| 7 |
+
import hashlib
|
| 8 |
+
import json
|
| 9 |
+
from pathlib import Path
|
| 10 |
+
|
| 11 |
+
from huggingface_hub import snapshot_download
|
| 12 |
+
|
| 13 |
+
BUNDLE_DIR = Path(__file__).resolve().parent
|
| 14 |
+
DATASET_REPO = 'entropyinequalityprover/MCE-220'
|
| 15 |
+
DATASET_REVISION = '5650d7c8198a68116e8d12934aed41a37866ba92'
|
| 16 |
+
MODEL_REPO = 'entropyinequalityprover/models_and_scripts'
|
| 17 |
+
MODEL_REVISION = '7fcfe778e1c36cd67e88ea8aa2ee761a50a39127'
|
| 18 |
+
BASE_MODEL_REPO = 'Qwen/Qwen3-0.6B'
|
| 19 |
+
BASE_MODEL_REVISION = 'c1899de289a04d12100db370d81485cdf75e47ca'
|
| 20 |
+
CHECKPOINT_PREFIX = 'qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534'
|
| 21 |
+
|
| 22 |
+
DATASET_SHA256 = {
|
| 23 |
+
"README.md": "909828a43ace9c3d4e9aa2e2fe96812dadd1c08ece52c87cf4dc896ad4ce0452",
|
| 24 |
+
"audit/diversity-audit.json": "56721eb34f35fa9acc2d81bbb13b0ea263a7f29bc8838372339dbc38ecda44a4",
|
| 25 |
+
"audit/generation-report.jsonl": "50304ef59e263296d675bf63f79ceb04857deaba339bf7b491dca5d0752ca12c",
|
| 26 |
+
"audit/provenance.jsonl": "cbd708a699899127a254d7367df6f129fcc6eec73b50f05db86f01b5a732e189",
|
| 27 |
+
"audit/run-manifest.json": "ac32fdaecf845714f9802c584c91b8743398898ff7e7c1f36c43014dea36d9a6",
|
| 28 |
+
"audit/validation-report.json": "3f86d1391c15a343cb938092a456ce2b5b88bb7f1ebcbf4089d0c8174c7e4d3c",
|
| 29 |
+
"data/questions.jsonl": "1ff45298389fa9a0cc6000b833302d610c489334c88d40a07b28b45ea76c762a",
|
| 30 |
+
"generation/e2n_v2.toml": "3346eb6fc6aa0bb70ca15f9a840d7b0f366418072c58cb76d32e0f8ac9faae4a"
|
| 31 |
+
}
|
| 32 |
+
CHECKPOINT_FILES = (
|
| 33 |
+
"adapter_config.json",
|
| 34 |
+
"adapter_model.safetensors",
|
| 35 |
+
"added_tokens.json",
|
| 36 |
+
"chat_template.jinja",
|
| 37 |
+
"merges.txt",
|
| 38 |
+
"special_tokens_map.json",
|
| 39 |
+
"tokenizer.json",
|
| 40 |
+
"tokenizer_config.json",
|
| 41 |
+
"vocab.json",
|
| 42 |
+
)
|
| 43 |
+
CHECKPOINT_CRITICAL_SHA256 = {
|
| 44 |
+
"adapter_model.safetensors": "813507e95124a06a4f8dc69d641b7fbcde791b4ec7d610f7c9403709588deb56",
|
| 45 |
+
"tokenizer.json": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4",
|
| 46 |
+
}
|
| 47 |
+
EXCLUDED_CHECKPOINT_FILES = (
|
| 48 |
+
"optimizer.pt",
|
| 49 |
+
"scheduler.pt",
|
| 50 |
+
"trainer_state.json",
|
| 51 |
+
"training_args.bin",
|
| 52 |
+
)
|
| 53 |
+
EXCLUDED_CHECKPOINT_PREFIXES = ("rng_state_",)
|
| 54 |
+
BASE_MODEL_FILES = (
|
| 55 |
+
"config.json",
|
| 56 |
+
"generation_config.json",
|
| 57 |
+
"merges.txt",
|
| 58 |
+
"model.safetensors",
|
| 59 |
+
"tokenizer.json",
|
| 60 |
+
"tokenizer_config.json",
|
| 61 |
+
"vocab.json",
|
| 62 |
+
)
|
| 63 |
+
BASE_MODEL_CRITICAL_SHA256 = {
|
| 64 |
+
"model.safetensors": "f47f71177f32bcd101b7573ec9171e6a57f4f4d31148d38e382306f42996874b",
|
| 65 |
+
"tokenizer.json": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4",
|
| 66 |
+
}
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
class AssetError(RuntimeError):
|
| 70 |
+
pass
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def sha256_file(path: Path) -> str:
|
| 74 |
+
digest = hashlib.sha256()
|
| 75 |
+
with path.open("rb") as stream:
|
| 76 |
+
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
| 77 |
+
digest.update(chunk)
|
| 78 |
+
return digest.hexdigest()
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def visible_files(root: Path) -> set[str]:
|
| 82 |
+
if not root.is_dir():
|
| 83 |
+
return set()
|
| 84 |
+
return {
|
| 85 |
+
path.relative_to(root).as_posix()
|
| 86 |
+
for path in root.rglob("*")
|
| 87 |
+
if path.is_file() and ".cache" not in path.relative_to(root).parts
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def require_hashes(root: Path, expected: dict[str, str]) -> None:
|
| 92 |
+
for relative, wanted in expected.items():
|
| 93 |
+
path = root / relative
|
| 94 |
+
if not path.is_file():
|
| 95 |
+
raise AssetError(f"missing required file: {path}")
|
| 96 |
+
actual = sha256_file(path)
|
| 97 |
+
if actual != wanted:
|
| 98 |
+
raise AssetError(
|
| 99 |
+
f"SHA-256 mismatch for {path}: expected {wanted}, got {actual}"
|
| 100 |
+
)
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def verify_dataset_tree(root: Path) -> None:
|
| 104 |
+
expected_files = set(DATASET_SHA256)
|
| 105 |
+
actual_files = visible_files(root)
|
| 106 |
+
if actual_files != expected_files:
|
| 107 |
+
raise AssetError(
|
| 108 |
+
"dataset file set mismatch: missing="
|
| 109 |
+
f"{sorted(expected_files - actual_files)}, extra="
|
| 110 |
+
f"{sorted(actual_files - expected_files)}"
|
| 111 |
+
)
|
| 112 |
+
require_hashes(root, DATASET_SHA256)
|
| 113 |
+
question_path = root / "data/questions.jsonl"
|
| 114 |
+
rows = [
|
| 115 |
+
json.loads(line)
|
| 116 |
+
for line in question_path.read_text(encoding="utf-8").splitlines()
|
| 117 |
+
if line.strip()
|
| 118 |
+
]
|
| 119 |
+
if len(rows) != 220 or len({row.get("id") for row in rows}) != 220:
|
| 120 |
+
raise AssetError("questions.jsonl must contain 220 unique question IDs")
|
| 121 |
+
if any(
|
| 122 |
+
set(row) != {"id", "n", "ineq", "conditions", "sha256"}
|
| 123 |
+
for row in rows
|
| 124 |
+
):
|
| 125 |
+
raise AssetError("questions.jsonl has an unexpected schema")
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def verify_checkpoint(root: Path) -> None:
|
| 129 |
+
expected = set(CHECKPOINT_FILES)
|
| 130 |
+
actual = visible_files(root)
|
| 131 |
+
if actual != expected:
|
| 132 |
+
raise AssetError(
|
| 133 |
+
"checkpoint file set mismatch: missing="
|
| 134 |
+
f"{sorted(expected - actual)}, extra={sorted(actual - expected)}"
|
| 135 |
+
)
|
| 136 |
+
for path in actual:
|
| 137 |
+
name = Path(path).name
|
| 138 |
+
if name in EXCLUDED_CHECKPOINT_FILES or any(
|
| 139 |
+
name.startswith(prefix) for prefix in EXCLUDED_CHECKPOINT_PREFIXES
|
| 140 |
+
):
|
| 141 |
+
raise AssetError(f"training-only checkpoint file present: {path}")
|
| 142 |
+
require_hashes(root, CHECKPOINT_CRITICAL_SHA256)
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
def verify_base_model(root: Path) -> None:
|
| 146 |
+
expected = set(BASE_MODEL_FILES)
|
| 147 |
+
actual = visible_files(root)
|
| 148 |
+
if actual != expected:
|
| 149 |
+
raise AssetError(
|
| 150 |
+
"base-model file set mismatch: missing="
|
| 151 |
+
f"{sorted(expected - actual)}, extra={sorted(actual - expected)}"
|
| 152 |
+
)
|
| 153 |
+
require_hashes(root, BASE_MODEL_CRITICAL_SHA256)
|
| 154 |
+
|
| 155 |
+
|
| 156 |
+
def download_assets(asset_dir: Path) -> None:
|
| 157 |
+
asset_dir.mkdir(parents=True, exist_ok=True)
|
| 158 |
+
snapshot_download(
|
| 159 |
+
repo_id=DATASET_REPO,
|
| 160 |
+
repo_type="dataset",
|
| 161 |
+
revision=DATASET_REVISION,
|
| 162 |
+
allow_patterns=list(DATASET_SHA256),
|
| 163 |
+
local_dir=asset_dir / "dataset-source",
|
| 164 |
+
local_dir_use_symlinks=False,
|
| 165 |
+
)
|
| 166 |
+
snapshot_download(
|
| 167 |
+
repo_id=MODEL_REPO,
|
| 168 |
+
repo_type="model",
|
| 169 |
+
revision=MODEL_REVISION,
|
| 170 |
+
allow_patterns=[f"{CHECKPOINT_PREFIX}/{name}" for name in CHECKPOINT_FILES],
|
| 171 |
+
local_dir=asset_dir / "model-repo",
|
| 172 |
+
local_dir_use_symlinks=False,
|
| 173 |
+
)
|
| 174 |
+
snapshot_download(
|
| 175 |
+
repo_id=BASE_MODEL_REPO,
|
| 176 |
+
repo_type="model",
|
| 177 |
+
revision=BASE_MODEL_REVISION,
|
| 178 |
+
allow_patterns=list(BASE_MODEL_FILES),
|
| 179 |
+
local_dir=asset_dir / "qwen3-0.6b",
|
| 180 |
+
local_dir_use_symlinks=False,
|
| 181 |
+
)
|
| 182 |
+
|
| 183 |
+
|
| 184 |
+
def verify_all(asset_dir: Path) -> dict[str, object]:
|
| 185 |
+
bundled_dataset = BUNDLE_DIR / "dataset"
|
| 186 |
+
downloaded_dataset = asset_dir / "dataset-source"
|
| 187 |
+
checkpoint = asset_dir / "model-repo" / CHECKPOINT_PREFIX
|
| 188 |
+
base_model = asset_dir / "qwen3-0.6b"
|
| 189 |
+
verify_dataset_tree(bundled_dataset)
|
| 190 |
+
verify_dataset_tree(downloaded_dataset)
|
| 191 |
+
for relative in DATASET_SHA256:
|
| 192 |
+
if (
|
| 193 |
+
(bundled_dataset / relative).read_bytes()
|
| 194 |
+
!= (downloaded_dataset / relative).read_bytes()
|
| 195 |
+
):
|
| 196 |
+
raise AssetError(f"bundled dataset differs from pinned source: {relative}")
|
| 197 |
+
verify_checkpoint(checkpoint)
|
| 198 |
+
verify_base_model(base_model)
|
| 199 |
+
return {
|
| 200 |
+
"status": "verified",
|
| 201 |
+
"dataset": {
|
| 202 |
+
"repo": DATASET_REPO,
|
| 203 |
+
"revision": DATASET_REVISION,
|
| 204 |
+
"rows": 220,
|
| 205 |
+
},
|
| 206 |
+
"checkpoint": {
|
| 207 |
+
"repo": MODEL_REPO,
|
| 208 |
+
"revision": MODEL_REVISION,
|
| 209 |
+
"path": str(checkpoint),
|
| 210 |
+
},
|
| 211 |
+
"base_model": {
|
| 212 |
+
"repo": BASE_MODEL_REPO,
|
| 213 |
+
"revision": BASE_MODEL_REVISION,
|
| 214 |
+
"path": str(base_model),
|
| 215 |
+
},
|
| 216 |
+
}
|
| 217 |
+
|
| 218 |
+
|
| 219 |
+
def parser() -> argparse.ArgumentParser:
|
| 220 |
+
ap = argparse.ArgumentParser(description=__doc__)
|
| 221 |
+
ap.add_argument(
|
| 222 |
+
"--asset-dir",
|
| 223 |
+
type=Path,
|
| 224 |
+
default=BUNDLE_DIR / "assets",
|
| 225 |
+
help="Asset directory (default: mce220_runner/assets).",
|
| 226 |
+
)
|
| 227 |
+
ap.add_argument(
|
| 228 |
+
"--verify-only",
|
| 229 |
+
action="store_true",
|
| 230 |
+
help="Do not access the network; verify already downloaded assets.",
|
| 231 |
+
)
|
| 232 |
+
return ap
|
| 233 |
+
|
| 234 |
+
|
| 235 |
+
def main() -> int:
|
| 236 |
+
args = parser().parse_args()
|
| 237 |
+
asset_dir = args.asset_dir.expanduser().resolve()
|
| 238 |
+
try:
|
| 239 |
+
if not args.verify_only:
|
| 240 |
+
download_assets(asset_dir)
|
| 241 |
+
print(json.dumps(verify_all(asset_dir), indent=2, sort_keys=True))
|
| 242 |
+
return 0
|
| 243 |
+
except AssetError as exc:
|
| 244 |
+
print(f"asset verification failed: {exc}", file=__import__("sys").stderr)
|
| 245 |
+
return 2
|
| 246 |
+
|
| 247 |
+
|
| 248 |
+
if __name__ == "__main__":
|
| 249 |
+
raise SystemExit(main())
|
mce220_runner/experiments/__init__.py
ADDED
|
File without changes
|
mce220_runner/experiments/reviewer_response/__init__.py
ADDED
|
File without changes
|
mce220_runner/experiments/reviewer_response/configs/inference_policy.toml
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Frozen policy for all new reviewer proposed-model inference (E2-N, E6,
|
| 2 |
+
# E9-A/E9-B and E10). E2-R is a retrospective audit of the submitted run and
|
| 3 |
+
# is intentionally not governed by this new policy.
|
| 4 |
+
|
| 5 |
+
[policy]
|
| 6 |
+
schema_version = "reviewer-inference-policy-v1"
|
| 7 |
+
sample_max_time_sec = 1000
|
| 8 |
+
sample_max_tree_size = 6000
|
| 9 |
+
sample_max_llm_calls = 0 # No separate call cap; time and tree limits still apply.
|
| 10 |
+
|
| 11 |
+
[enforcement]
|
| 12 |
+
time = "cooperative_soft"
|
| 13 |
+
tree = "cooperative_soft"
|
| 14 |
+
hard_wall_time_enforced = false
|
| 15 |
+
external_hard_timeout_required = true
|
| 16 |
+
note = "The runner checks budgets between search operations; already submitted model generations may finish after a budget is reached."
|
mce220_runner/experiments/reviewer_response/run_search_benchmark.py
ADDED
|
@@ -0,0 +1,1108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Reproducible wrapper for the submitted proposed-model search pipeline.
|
| 2 |
+
|
| 3 |
+
The wrapper deliberately does not import or modify the search runner. It
|
| 4 |
+
freezes the submitted search settings, records a machine-readable manifest,
|
| 5 |
+
executes the tracked runner as a child process, and verifies successful proof
|
| 6 |
+
artifacts with the standalone exact verifier.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import argparse
|
| 12 |
+
import csv
|
| 13 |
+
import hashlib
|
| 14 |
+
import importlib.metadata
|
| 15 |
+
import json
|
| 16 |
+
import os
|
| 17 |
+
import platform
|
| 18 |
+
import shlex
|
| 19 |
+
import subprocess
|
| 20 |
+
import sys
|
| 21 |
+
import time
|
| 22 |
+
import uuid
|
| 23 |
+
from datetime import datetime, timezone
|
| 24 |
+
from pathlib import Path
|
| 25 |
+
from typing import Any, Sequence
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
# Direct execution sets sys.path[0] to experiments/reviewer_response rather
|
| 29 |
+
# than the repository root. Add the root explicitly so resume verification can
|
| 30 |
+
# import beam_search_model2 in the same way as ``python -m`` execution.
|
| 31 |
+
_REPO_ROOT_FOR_IMPORTS = Path(__file__).resolve().parents[2]
|
| 32 |
+
if str(_REPO_ROOT_FOR_IMPORTS) not in sys.path:
|
| 33 |
+
sys.path.insert(0, str(_REPO_ROOT_FOR_IMPORTS))
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
SCHEMA_VERSION = "reviewer-search-run-v4"
|
| 37 |
+
EXECUTION_CONFIG_SCHEMA_VERSION = "e2n-execution-config-v1"
|
| 38 |
+
RESOURCE_POLICY_VERSION = "reviewer-inference-policy-v1"
|
| 39 |
+
SAMPLE_MAX_TIME_SEC = 1000.0
|
| 40 |
+
SAMPLE_MAX_TREE_SIZE = 6000
|
| 41 |
+
DEFAULT_RUNNER = Path("beam_search_model2/run_on_window_shear_jsonl_v12.py")
|
| 42 |
+
DEFAULT_GPU_IDS = "4,4,5,5,6,6,7,7"
|
| 43 |
+
PACKAGE_NAMES = (
|
| 44 |
+
"accelerate",
|
| 45 |
+
"numpy",
|
| 46 |
+
"peft",
|
| 47 |
+
"scipy",
|
| 48 |
+
"sympy",
|
| 49 |
+
"torch",
|
| 50 |
+
"transformers",
|
| 51 |
+
)
|
| 52 |
+
RELEVANT_INHERITED_ENV = (
|
| 53 |
+
"CUDA_VISIBLE_DEVICES",
|
| 54 |
+
"HF_HOME",
|
| 55 |
+
"HF_HUB_OFFLINE",
|
| 56 |
+
"MKL_NUM_THREADS",
|
| 57 |
+
"OMP_NUM_THREADS",
|
| 58 |
+
"OPENBLAS_NUM_THREADS",
|
| 59 |
+
"PYTHONHASHSEED",
|
| 60 |
+
"TOKENIZERS_PARALLELISM",
|
| 61 |
+
"TRANSFORMERS_CACHE",
|
| 62 |
+
)
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
class LauncherError(ValueError):
|
| 66 |
+
"""Raised for a launcher preflight or artifact error."""
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def _utc_now() -> str:
|
| 70 |
+
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def _sha256_file(path: Path) -> str:
|
| 74 |
+
digest = hashlib.sha256()
|
| 75 |
+
with path.open("rb") as stream:
|
| 76 |
+
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
| 77 |
+
digest.update(chunk)
|
| 78 |
+
return digest.hexdigest()
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def _file_artifact(path: Path) -> dict[str, Any]:
|
| 82 |
+
return {
|
| 83 |
+
"path": str(path),
|
| 84 |
+
"bytes": path.stat().st_size,
|
| 85 |
+
"sha256": _sha256_file(path),
|
| 86 |
+
}
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
def _checkpoint_artifact(path: Path) -> dict[str, Any]:
|
| 90 |
+
"""Hash a checkpoint file or a checkpoint directory deterministically."""
|
| 91 |
+
|
| 92 |
+
if path.is_file():
|
| 93 |
+
return {"kind": "file", **_file_artifact(path)}
|
| 94 |
+
if not path.is_dir():
|
| 95 |
+
raise LauncherError(f"checkpoint does not exist: {path}")
|
| 96 |
+
|
| 97 |
+
files: dict[str, dict[str, Any]] = {}
|
| 98 |
+
tree_digest = hashlib.sha256()
|
| 99 |
+
total_bytes = 0
|
| 100 |
+
for child in sorted((item for item in path.rglob("*") if item.is_file())):
|
| 101 |
+
relative = child.relative_to(path).as_posix()
|
| 102 |
+
size = child.stat().st_size
|
| 103 |
+
sha256 = _sha256_file(child)
|
| 104 |
+
files[relative] = {"bytes": size, "sha256": sha256}
|
| 105 |
+
total_bytes += size
|
| 106 |
+
tree_digest.update(relative.encode("utf-8"))
|
| 107 |
+
tree_digest.update(b"\0")
|
| 108 |
+
tree_digest.update(sha256.encode("ascii"))
|
| 109 |
+
tree_digest.update(b"\0")
|
| 110 |
+
tree_digest.update(str(size).encode("ascii"))
|
| 111 |
+
tree_digest.update(b"\n")
|
| 112 |
+
if not files:
|
| 113 |
+
raise LauncherError(f"checkpoint directory contains no files: {path}")
|
| 114 |
+
return {
|
| 115 |
+
"kind": "directory",
|
| 116 |
+
"path": str(path),
|
| 117 |
+
"file_count": len(files),
|
| 118 |
+
"bytes": total_bytes,
|
| 119 |
+
"sha256": tree_digest.hexdigest(),
|
| 120 |
+
"files": files,
|
| 121 |
+
}
|
| 122 |
+
|
| 123 |
+
|
| 124 |
+
def _question_profile(path: Path) -> dict[str, Any]:
|
| 125 |
+
count = 0
|
| 126 |
+
ids = 0
|
| 127 |
+
variable_counts: list[int] = []
|
| 128 |
+
extra_fields: set[str] = set()
|
| 129 |
+
with path.open("r", encoding="utf-8") as stream:
|
| 130 |
+
for line_number, line in enumerate(stream, start=1):
|
| 131 |
+
if not line.strip():
|
| 132 |
+
continue
|
| 133 |
+
try:
|
| 134 |
+
row = json.loads(line)
|
| 135 |
+
except json.JSONDecodeError as exc:
|
| 136 |
+
raise LauncherError(
|
| 137 |
+
f"invalid questions JSON on line {line_number}: {exc}"
|
| 138 |
+
) from exc
|
| 139 |
+
if not isinstance(row, dict):
|
| 140 |
+
raise LauncherError(f"questions line {line_number} is not an object")
|
| 141 |
+
missing = {"n", "ineq", "conditions"}.difference(row)
|
| 142 |
+
if missing:
|
| 143 |
+
raise LauncherError(
|
| 144 |
+
f"questions line {line_number} is missing: {sorted(missing)}"
|
| 145 |
+
)
|
| 146 |
+
if not isinstance(row["conditions"], list):
|
| 147 |
+
raise LauncherError(
|
| 148 |
+
f"questions line {line_number} conditions must be a list"
|
| 149 |
+
)
|
| 150 |
+
count += 1
|
| 151 |
+
ids += int("id" in row)
|
| 152 |
+
try:
|
| 153 |
+
variable_counts.append(int(row["n"]))
|
| 154 |
+
except (TypeError, ValueError) as exc:
|
| 155 |
+
raise LauncherError(
|
| 156 |
+
f"questions line {line_number} has invalid n={row['n']!r}"
|
| 157 |
+
) from exc
|
| 158 |
+
extra_fields.update(set(row).difference({"n", "ineq", "conditions"}))
|
| 159 |
+
if count == 0:
|
| 160 |
+
raise LauncherError(f"questions file is empty: {path}")
|
| 161 |
+
return {
|
| 162 |
+
"rows": count,
|
| 163 |
+
"rows_with_id": ids,
|
| 164 |
+
"min_n": min(variable_counts),
|
| 165 |
+
"max_n": max(variable_counts),
|
| 166 |
+
"extra_fields_preserved_by_runner_input": sorted(extra_fields),
|
| 167 |
+
}
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
def _count_jsonl_rows(path: Path) -> int:
|
| 171 |
+
with path.open("r", encoding="utf-8") as stream:
|
| 172 |
+
return sum(1 for line in stream if line.strip())
|
| 173 |
+
|
| 174 |
+
|
| 175 |
+
def _read_jsonl(path: Path, label: str) -> list[dict[str, Any]]:
|
| 176 |
+
rows: list[dict[str, Any]] = []
|
| 177 |
+
with path.open("r", encoding="utf-8") as stream:
|
| 178 |
+
for line_number, line in enumerate(stream, start=1):
|
| 179 |
+
if not line.strip():
|
| 180 |
+
continue
|
| 181 |
+
try:
|
| 182 |
+
row = json.loads(line)
|
| 183 |
+
except json.JSONDecodeError as exc:
|
| 184 |
+
raise LauncherError(
|
| 185 |
+
f"invalid {label} JSON on line {line_number}: {exc}"
|
| 186 |
+
) from exc
|
| 187 |
+
if not isinstance(row, dict):
|
| 188 |
+
raise LauncherError(f"{label} line {line_number} is not an object")
|
| 189 |
+
rows.append(row)
|
| 190 |
+
return rows
|
| 191 |
+
|
| 192 |
+
|
| 193 |
+
def _resume_artifact(
|
| 194 |
+
results: Path, questions: Path, expected_rows: int
|
| 195 |
+
) -> dict[str, Any]:
|
| 196 |
+
"""Validate and snapshot the immutable prefix used by v12 resume."""
|
| 197 |
+
|
| 198 |
+
if not results.is_file() or results.stat().st_size == 0:
|
| 199 |
+
raise LauncherError(
|
| 200 |
+
f"--resume requires an existing nonempty results file: {results}"
|
| 201 |
+
)
|
| 202 |
+
# Appending to an unterminated final line would corrupt that JSON object.
|
| 203 |
+
with results.open("rb") as stream:
|
| 204 |
+
stream.seek(-1, os.SEEK_END)
|
| 205 |
+
if stream.read(1) not in {b"\n", b"\r"}:
|
| 206 |
+
raise LauncherError("resume results must end with a newline")
|
| 207 |
+
|
| 208 |
+
prior_rows = _read_jsonl(results, "resume results")
|
| 209 |
+
if not prior_rows:
|
| 210 |
+
raise LauncherError(
|
| 211 |
+
f"--resume requires at least one existing result row: {results}"
|
| 212 |
+
)
|
| 213 |
+
if len(prior_rows) >= expected_rows:
|
| 214 |
+
raise LauncherError(
|
| 215 |
+
"--resume requires fewer existing result rows than questions: "
|
| 216 |
+
f"results={len(prior_rows)}, questions={expected_rows}"
|
| 217 |
+
)
|
| 218 |
+
|
| 219 |
+
question_rows = _read_jsonl(questions, "questions")
|
| 220 |
+
for index, (result, question) in enumerate(
|
| 221 |
+
zip(prior_rows, question_rows, strict=False), start=1
|
| 222 |
+
):
|
| 223 |
+
if result.get("input") != question:
|
| 224 |
+
raise LauncherError(
|
| 225 |
+
"resume results are not the exact question-prefix at row "
|
| 226 |
+
f"{index}; refusing count-based resume"
|
| 227 |
+
)
|
| 228 |
+
|
| 229 |
+
artifact = _file_artifact(results)
|
| 230 |
+
artifact["rows"] = len(prior_rows)
|
| 231 |
+
return artifact
|
| 232 |
+
|
| 233 |
+
|
| 234 |
+
def _prefix_is_unchanged(path: Path, prior: dict[str, Any]) -> bool:
|
| 235 |
+
expected_bytes = int(prior["bytes"])
|
| 236 |
+
if not path.is_file() or path.stat().st_size < expected_bytes:
|
| 237 |
+
return False
|
| 238 |
+
digest = hashlib.sha256()
|
| 239 |
+
with path.open("rb") as stream:
|
| 240 |
+
remaining = expected_bytes
|
| 241 |
+
while remaining:
|
| 242 |
+
chunk = stream.read(min(1024 * 1024, remaining))
|
| 243 |
+
if not chunk:
|
| 244 |
+
return False
|
| 245 |
+
digest.update(chunk)
|
| 246 |
+
remaining -= len(chunk)
|
| 247 |
+
return digest.hexdigest() == prior["sha256"]
|
| 248 |
+
|
| 249 |
+
|
| 250 |
+
def _resolve_existing(path: Path, label: str) -> Path:
|
| 251 |
+
resolved = path.expanduser().resolve()
|
| 252 |
+
if not resolved.is_file():
|
| 253 |
+
raise LauncherError(f"{label} does not exist or is not a file: {resolved}")
|
| 254 |
+
return resolved
|
| 255 |
+
|
| 256 |
+
|
| 257 |
+
def _parse_gpu_ids(raw: str, workers: int) -> list[int]:
|
| 258 |
+
try:
|
| 259 |
+
ids = [int(part.strip()) for part in raw.split(",") if part.strip()]
|
| 260 |
+
except ValueError as exc:
|
| 261 |
+
raise LauncherError(f"invalid --gpu-ids value: {raw!r}") from exc
|
| 262 |
+
if workers <= 0:
|
| 263 |
+
raise LauncherError("--workers must be positive")
|
| 264 |
+
if len(ids) < workers:
|
| 265 |
+
raise LauncherError(
|
| 266 |
+
f"--workers={workers} requires at least {workers} GPU IDs; got {ids}"
|
| 267 |
+
)
|
| 268 |
+
return ids[:workers]
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
def _display_command(command: Sequence[str]) -> str:
|
| 272 |
+
if os.name == "nt":
|
| 273 |
+
return subprocess.list2cmdline(list(command))
|
| 274 |
+
return shlex.join(command)
|
| 275 |
+
|
| 276 |
+
|
| 277 |
+
def _git_metadata(repo_root: Path) -> dict[str, Any]:
|
| 278 |
+
def run(*args: str) -> str | None:
|
| 279 |
+
try:
|
| 280 |
+
completed = subprocess.run(
|
| 281 |
+
["git", *args],
|
| 282 |
+
cwd=repo_root,
|
| 283 |
+
check=False,
|
| 284 |
+
capture_output=True,
|
| 285 |
+
text=True,
|
| 286 |
+
timeout=10,
|
| 287 |
+
)
|
| 288 |
+
except (OSError, subprocess.SubprocessError):
|
| 289 |
+
return None
|
| 290 |
+
return completed.stdout.strip() if completed.returncode == 0 else None
|
| 291 |
+
|
| 292 |
+
status = run("status", "--porcelain=v1", "--untracked-files=all")
|
| 293 |
+
return {
|
| 294 |
+
"root": str(repo_root),
|
| 295 |
+
"commit": run("rev-parse", "HEAD"),
|
| 296 |
+
"branch": run("branch", "--show-current"),
|
| 297 |
+
"status_porcelain": status,
|
| 298 |
+
"dirty": bool(status) if status is not None else None,
|
| 299 |
+
}
|
| 300 |
+
|
| 301 |
+
|
| 302 |
+
def _package_metadata() -> dict[str, str | None]:
|
| 303 |
+
versions: dict[str, str | None] = {}
|
| 304 |
+
for name in PACKAGE_NAMES:
|
| 305 |
+
try:
|
| 306 |
+
versions[name] = importlib.metadata.version(name)
|
| 307 |
+
except importlib.metadata.PackageNotFoundError:
|
| 308 |
+
versions[name] = None
|
| 309 |
+
return versions
|
| 310 |
+
|
| 311 |
+
|
| 312 |
+
def _hardware_metadata(gpu_ids: Sequence[int]) -> dict[str, Any]:
|
| 313 |
+
metadata: dict[str, Any] = {
|
| 314 |
+
"platform": platform.platform(),
|
| 315 |
+
"machine": platform.machine(),
|
| 316 |
+
"processor": platform.processor(),
|
| 317 |
+
"cpu_count": os.cpu_count(),
|
| 318 |
+
"requested_gpu_ids": list(gpu_ids),
|
| 319 |
+
}
|
| 320 |
+
query = [
|
| 321 |
+
"nvidia-smi",
|
| 322 |
+
"--query-gpu=index,name,uuid,memory.total,driver_version",
|
| 323 |
+
"--format=csv,noheader,nounits",
|
| 324 |
+
]
|
| 325 |
+
try:
|
| 326 |
+
completed = subprocess.run(
|
| 327 |
+
query, check=False, capture_output=True, text=True, timeout=10
|
| 328 |
+
)
|
| 329 |
+
metadata["nvidia_smi_return_code"] = completed.returncode
|
| 330 |
+
metadata["nvidia_smi"] = [
|
| 331 |
+
line.strip() for line in completed.stdout.splitlines() if line.strip()
|
| 332 |
+
]
|
| 333 |
+
if completed.stderr.strip():
|
| 334 |
+
metadata["nvidia_smi_stderr"] = completed.stderr.strip()
|
| 335 |
+
except (OSError, subprocess.SubprocessError) as exc:
|
| 336 |
+
metadata["nvidia_smi_error"] = f"{type(exc).__name__}: {exc}"
|
| 337 |
+
return metadata
|
| 338 |
+
|
| 339 |
+
|
| 340 |
+
def _launch_gpu_snapshot(gpu_ids: Sequence[int]) -> list[dict[str, Any]]:
|
| 341 |
+
"""Capture physical-GPU state once, immediately before child launch."""
|
| 342 |
+
|
| 343 |
+
requested = list(dict.fromkeys(int(value) for value in gpu_ids))
|
| 344 |
+
inventory_command = [
|
| 345 |
+
"nvidia-smi",
|
| 346 |
+
"--query-gpu=index,uuid,memory.used,memory.free,utilization.gpu",
|
| 347 |
+
"--format=csv,noheader,nounits",
|
| 348 |
+
]
|
| 349 |
+
apps_command = [
|
| 350 |
+
"nvidia-smi",
|
| 351 |
+
"--query-compute-apps=gpu_uuid,pid,process_name,used_gpu_memory",
|
| 352 |
+
"--format=csv,noheader,nounits",
|
| 353 |
+
]
|
| 354 |
+
inventory: dict[int, dict[str, Any]] = {}
|
| 355 |
+
apps_by_uuid: dict[str, list[dict[str, Any]]] = {}
|
| 356 |
+
error: str | None = None
|
| 357 |
+
try:
|
| 358 |
+
inventory_result = subprocess.run(
|
| 359 |
+
inventory_command,
|
| 360 |
+
check=True,
|
| 361 |
+
capture_output=True,
|
| 362 |
+
text=True,
|
| 363 |
+
timeout=10,
|
| 364 |
+
)
|
| 365 |
+
for row in csv.reader(inventory_result.stdout.splitlines(), skipinitialspace=True):
|
| 366 |
+
if not row:
|
| 367 |
+
continue
|
| 368 |
+
if len(row) != 5:
|
| 369 |
+
raise ValueError(f"malformed GPU inventory row: {row!r}")
|
| 370 |
+
index = int(row[0].strip())
|
| 371 |
+
inventory[index] = {
|
| 372 |
+
"index": index,
|
| 373 |
+
"uuid": row[1].strip(),
|
| 374 |
+
"memory_used_mib": int(row[2].strip()),
|
| 375 |
+
"memory_free_mib": int(row[3].strip()),
|
| 376 |
+
"utilization_percent": int(row[4].strip()),
|
| 377 |
+
}
|
| 378 |
+
apps_result = subprocess.run(
|
| 379 |
+
apps_command,
|
| 380 |
+
check=True,
|
| 381 |
+
capture_output=True,
|
| 382 |
+
text=True,
|
| 383 |
+
timeout=10,
|
| 384 |
+
)
|
| 385 |
+
for row in csv.reader(apps_result.stdout.splitlines(), skipinitialspace=True):
|
| 386 |
+
if not row:
|
| 387 |
+
continue
|
| 388 |
+
if len(row) != 4:
|
| 389 |
+
raise ValueError(f"malformed compute-app row: {row!r}")
|
| 390 |
+
uuid_value = row[0].strip()
|
| 391 |
+
apps_by_uuid.setdefault(uuid_value, []).append(
|
| 392 |
+
{
|
| 393 |
+
"pid": row[1].strip(),
|
| 394 |
+
"process_name": row[2].strip(),
|
| 395 |
+
"used_gpu_memory_mib": row[3].strip(),
|
| 396 |
+
}
|
| 397 |
+
)
|
| 398 |
+
except (OSError, ValueError, subprocess.SubprocessError) as exc:
|
| 399 |
+
error = f"{type(exc).__name__}: {exc}"
|
| 400 |
+
|
| 401 |
+
snapshot: list[dict[str, Any]] = []
|
| 402 |
+
for index in requested:
|
| 403 |
+
gpu = inventory.get(
|
| 404 |
+
index,
|
| 405 |
+
{
|
| 406 |
+
"index": index,
|
| 407 |
+
"uuid": None,
|
| 408 |
+
"memory_used_mib": None,
|
| 409 |
+
"memory_free_mib": None,
|
| 410 |
+
"utilization_percent": None,
|
| 411 |
+
},
|
| 412 |
+
)
|
| 413 |
+
item = {**gpu, "compute_apps": apps_by_uuid.get(str(gpu["uuid"]), [])}
|
| 414 |
+
if error is not None:
|
| 415 |
+
item["snapshot_error"] = error
|
| 416 |
+
snapshot.append(item)
|
| 417 |
+
return snapshot
|
| 418 |
+
|
| 419 |
+
|
| 420 |
+
def _require_complete_launch_gpu_snapshot(
|
| 421 |
+
snapshot: Sequence[dict[str, Any]], gpu_ids: Sequence[int]
|
| 422 |
+
) -> None:
|
| 423 |
+
"""Fail before child start unless every selected physical GPU is identified."""
|
| 424 |
+
|
| 425 |
+
by_index = {
|
| 426 |
+
item.get("index"): item
|
| 427 |
+
for item in snapshot
|
| 428 |
+
if isinstance(item, dict) and isinstance(item.get("index"), int)
|
| 429 |
+
}
|
| 430 |
+
for gpu_id in dict.fromkeys(int(value) for value in gpu_ids):
|
| 431 |
+
item = by_index.get(gpu_id)
|
| 432 |
+
if item is None:
|
| 433 |
+
raise LauncherError(f"launch GPU snapshot is missing physical GPU {gpu_id}")
|
| 434 |
+
if item.get("snapshot_error"):
|
| 435 |
+
raise LauncherError(
|
| 436 |
+
f"launch GPU snapshot failed for physical GPU {gpu_id}: "
|
| 437 |
+
f"{item['snapshot_error']}"
|
| 438 |
+
)
|
| 439 |
+
if not isinstance(item.get("uuid"), str) or not item["uuid"]:
|
| 440 |
+
raise LauncherError(
|
| 441 |
+
f"launch GPU snapshot has no UUID for physical GPU {gpu_id}"
|
| 442 |
+
)
|
| 443 |
+
|
| 444 |
+
|
| 445 |
+
def _base_model(checkpoint: Path) -> str | None:
|
| 446 |
+
config_path = checkpoint / "adapter_config.json" if checkpoint.is_dir() else None
|
| 447 |
+
if config_path is None or not config_path.is_file():
|
| 448 |
+
return None
|
| 449 |
+
try:
|
| 450 |
+
config = json.loads(config_path.read_text(encoding="utf-8"))
|
| 451 |
+
except (OSError, json.JSONDecodeError):
|
| 452 |
+
return None
|
| 453 |
+
value = config.get("base_model_name_or_path") if isinstance(config, dict) else None
|
| 454 |
+
return str(value) if value is not None else None
|
| 455 |
+
|
| 456 |
+
|
| 457 |
+
def _write_manifest(path: Path, manifest: dict[str, Any]) -> None:
|
| 458 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 459 |
+
temporary = path.with_name(path.name + f".tmp-{os.getpid()}-{uuid.uuid4().hex}")
|
| 460 |
+
temporary.write_text(
|
| 461 |
+
json.dumps(manifest, ensure_ascii=False, indent=2, sort_keys=True) + "\n",
|
| 462 |
+
encoding="utf-8",
|
| 463 |
+
)
|
| 464 |
+
temporary.replace(path)
|
| 465 |
+
|
| 466 |
+
|
| 467 |
+
def _write_manifest_exclusive(path: Path, manifest: dict[str, Any]) -> None:
|
| 468 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 469 |
+
try:
|
| 470 |
+
with path.open("x", encoding="utf-8") as stream:
|
| 471 |
+
json.dump(manifest, stream, ensure_ascii=False, indent=2, sort_keys=True)
|
| 472 |
+
stream.write("\n")
|
| 473 |
+
except FileExistsError as exc:
|
| 474 |
+
raise LauncherError(f"manifest already exists: {path}") from exc
|
| 475 |
+
|
| 476 |
+
|
| 477 |
+
def build_command(
|
| 478 |
+
args: argparse.Namespace,
|
| 479 |
+
*,
|
| 480 |
+
questions: Path,
|
| 481 |
+
results: Path,
|
| 482 |
+
checkpoint: Path,
|
| 483 |
+
runner: Path,
|
| 484 |
+
gpu_ids: Sequence[int],
|
| 485 |
+
execution_config_manifest: Path | None = None,
|
| 486 |
+
) -> tuple[list[str], dict[str, str]]:
|
| 487 |
+
"""Build the exact child-process command and search-sensitive environment."""
|
| 488 |
+
|
| 489 |
+
command = [
|
| 490 |
+
sys.executable,
|
| 491 |
+
str(runner),
|
| 492 |
+
"--input",
|
| 493 |
+
str(questions),
|
| 494 |
+
"--output",
|
| 495 |
+
str(results),
|
| 496 |
+
"--min-n",
|
| 497 |
+
str(args.min_n),
|
| 498 |
+
"--max-n",
|
| 499 |
+
str(args.max_n),
|
| 500 |
+
"--max-depth",
|
| 501 |
+
str(args.max_depth),
|
| 502 |
+
"--rollouts",
|
| 503 |
+
str(args.rollouts),
|
| 504 |
+
"--k-per-node",
|
| 505 |
+
"1",
|
| 506 |
+
"--llm-mode",
|
| 507 |
+
"batch",
|
| 508 |
+
"--beam-size",
|
| 509 |
+
str(args.beam_size),
|
| 510 |
+
"--gpu-workers",
|
| 511 |
+
str(args.workers),
|
| 512 |
+
"--gpu-devices",
|
| 513 |
+
",".join(str(value) for value in gpu_ids),
|
| 514 |
+
"--adapter-path",
|
| 515 |
+
str(checkpoint),
|
| 516 |
+
"--lp-workers",
|
| 517 |
+
str(args.lp_workers),
|
| 518 |
+
"--enable-lp",
|
| 519 |
+
"--max-vars-for-lp",
|
| 520 |
+
str(args.max_vars_for_lp),
|
| 521 |
+
"--max-new-tokens",
|
| 522 |
+
str(args.max_new_tokens),
|
| 523 |
+
"--max-time",
|
| 524 |
+
str(args.generation_max_time),
|
| 525 |
+
"--seed",
|
| 526 |
+
str(args.seed),
|
| 527 |
+
"--sample-max-tree-size",
|
| 528 |
+
str(args.sample_max_tree_size),
|
| 529 |
+
"--sample-max-time-sec",
|
| 530 |
+
str(args.sample_max_time_sec),
|
| 531 |
+
]
|
| 532 |
+
if args.print_llm:
|
| 533 |
+
command.append("--print-llm")
|
| 534 |
+
if args.sample_max_llm_calls is not None:
|
| 535 |
+
command.extend(["--sample-max-llm-calls", str(args.sample_max_llm_calls)])
|
| 536 |
+
if execution_config_manifest is not None:
|
| 537 |
+
command.extend(
|
| 538 |
+
["--execution-config-manifest", str(execution_config_manifest)]
|
| 539 |
+
)
|
| 540 |
+
if args.resume:
|
| 541 |
+
command.extend(["--append", "--resume-from-output"])
|
| 542 |
+
|
| 543 |
+
environment = {
|
| 544 |
+
"BEAM_PROMPT_FORMAT": args.prompt_format,
|
| 545 |
+
"BEAM_NORMALIZE_CANDIDATES": "1" if args.normalize_candidates else "0",
|
| 546 |
+
"BEAM_SORT_ALPHA_GROUPS": "0",
|
| 547 |
+
"BEAM_RANDOM_SCORE": "0",
|
| 548 |
+
"BEAM_MAX_TRIES_PER_STEP": "3",
|
| 549 |
+
"QWEN_TEMPERATURE": str(args.temperature),
|
| 550 |
+
"QWEN_TOP_P": "0.9",
|
| 551 |
+
"QWEN_MAX_NEW_TOKENS": str(args.max_new_tokens),
|
| 552 |
+
"QWEN_MAX_TIME": str(args.generation_max_time),
|
| 553 |
+
"QWEN_RAW_PROMPT": "1" if args.raw_prompt else "0",
|
| 554 |
+
"QWEN_PRINT_PROMPT": "0",
|
| 555 |
+
"QWEN_ADAPTER_PATH": str(checkpoint),
|
| 556 |
+
# Prefer the explicit pinned local snapshot and otherwise prevent
|
| 557 |
+
# unrelated shell state from replacing the adapter-declared base.
|
| 558 |
+
"QWEN_BASE_MODEL": str(args.base_model) if args.base_model else "",
|
| 559 |
+
"QWEN_WEIGHTS_PATH": "",
|
| 560 |
+
"GPU_DEVICES": ",".join(str(value) for value in gpu_ids),
|
| 561 |
+
}
|
| 562 |
+
return command, environment
|
| 563 |
+
|
| 564 |
+
|
| 565 |
+
def _settings(args: argparse.Namespace, gpu_ids: Sequence[int]) -> dict[str, Any]:
|
| 566 |
+
return {
|
| 567 |
+
"beam_size": args.beam_size,
|
| 568 |
+
"rollouts": args.rollouts,
|
| 569 |
+
"max_depth": args.max_depth,
|
| 570 |
+
"seed": args.seed,
|
| 571 |
+
"workers": args.workers,
|
| 572 |
+
"gpu_ids": list(gpu_ids),
|
| 573 |
+
"temperature": args.temperature,
|
| 574 |
+
"base_model": str(args.base_model) if args.base_model else None,
|
| 575 |
+
"max_new_tokens": args.max_new_tokens,
|
| 576 |
+
"generation_max_time_sec": args.generation_max_time,
|
| 577 |
+
"sample_max_tree_size": args.sample_max_tree_size,
|
| 578 |
+
"sample_max_time_sec": args.sample_max_time_sec,
|
| 579 |
+
"hard_question_timeout_sec": args.hard_question_timeout_sec,
|
| 580 |
+
"sample_max_llm_calls": int(args.sample_max_llm_calls or 0),
|
| 581 |
+
"max_vars_for_lp": args.max_vars_for_lp,
|
| 582 |
+
"lp_enabled": True,
|
| 583 |
+
"lp_workers": args.lp_workers,
|
| 584 |
+
"lp_execution": "synchronous",
|
| 585 |
+
"prompt_format": args.prompt_format,
|
| 586 |
+
"normalize_candidates": args.normalize_candidates,
|
| 587 |
+
"raw_prompt": args.raw_prompt,
|
| 588 |
+
"print_llm": args.print_llm,
|
| 589 |
+
"top_p": 0.9,
|
| 590 |
+
"sort_alpha_groups": False,
|
| 591 |
+
"random_score": False,
|
| 592 |
+
"max_tries_per_step": 3,
|
| 593 |
+
"min_n": args.min_n,
|
| 594 |
+
"max_n": args.max_n,
|
| 595 |
+
"llm_mode": "batch",
|
| 596 |
+
"k_per_node": 1,
|
| 597 |
+
"random_prune": False,
|
| 598 |
+
"prune_seed": None,
|
| 599 |
+
"sample_max_llm_fails": 0,
|
| 600 |
+
"depth1_rollouts": None,
|
| 601 |
+
"per_n_budget": {},
|
| 602 |
+
"include_proof_trace": False,
|
| 603 |
+
"limit": 0,
|
| 604 |
+
"use_dummy": False,
|
| 605 |
+
"debug_search": False,
|
| 606 |
+
"log_every": 1,
|
| 607 |
+
"log_wrong_pruned": False,
|
| 608 |
+
"wrong_pruned_path": "",
|
| 609 |
+
}
|
| 610 |
+
|
| 611 |
+
|
| 612 |
+
def _normalized_command(command: Sequence[str]) -> list[str]:
|
| 613 |
+
normalized = list(command)
|
| 614 |
+
if "--execution-config-manifest" in normalized:
|
| 615 |
+
index = normalized.index("--execution-config-manifest")
|
| 616 |
+
if index + 1 < len(normalized):
|
| 617 |
+
normalized[index + 1] = "<live-wrapper-manifest>"
|
| 618 |
+
return normalized
|
| 619 |
+
|
| 620 |
+
|
| 621 |
+
def _execution_config(
|
| 622 |
+
*,
|
| 623 |
+
invocation_id: str,
|
| 624 |
+
recorded_utc: str,
|
| 625 |
+
origin: str,
|
| 626 |
+
repo_root: Path,
|
| 627 |
+
repository: dict[str, Any],
|
| 628 |
+
runner: Path,
|
| 629 |
+
runner_artifact: dict[str, Any],
|
| 630 |
+
checkpoint: Path,
|
| 631 |
+
checkpoint_artifact: dict[str, Any],
|
| 632 |
+
settings: dict[str, Any],
|
| 633 |
+
gpu_ids: Sequence[int],
|
| 634 |
+
inherited_environment: dict[str, str | None],
|
| 635 |
+
gpu_snapshot: list[dict[str, Any]],
|
| 636 |
+
command: Sequence[str],
|
| 637 |
+
) -> dict[str, Any]:
|
| 638 |
+
uuid_by_index = {
|
| 639 |
+
item.get("index"): item.get("uuid")
|
| 640 |
+
for item in gpu_snapshot
|
| 641 |
+
if isinstance(item, dict)
|
| 642 |
+
}
|
| 643 |
+
return {
|
| 644 |
+
"schema_version": EXECUTION_CONFIG_SCHEMA_VERSION,
|
| 645 |
+
"invocation_id": invocation_id,
|
| 646 |
+
"recorded_utc": recorded_utc,
|
| 647 |
+
"origin": origin,
|
| 648 |
+
"code": {
|
| 649 |
+
"git_commit": repository.get("commit"),
|
| 650 |
+
"repo_root": str(repo_root),
|
| 651 |
+
"runner_path": str(runner),
|
| 652 |
+
"runner_sha256": runner_artifact["sha256"],
|
| 653 |
+
},
|
| 654 |
+
"model": {
|
| 655 |
+
"checkpoint_path": str(checkpoint),
|
| 656 |
+
"checkpoint_sha256": checkpoint_artifact["sha256"],
|
| 657 |
+
"base_model": settings.get("base_model") or _base_model(checkpoint),
|
| 658 |
+
},
|
| 659 |
+
"search": settings,
|
| 660 |
+
"resources": {
|
| 661 |
+
"workers": len(gpu_ids),
|
| 662 |
+
"gpu_ids": list(gpu_ids),
|
| 663 |
+
"worker_device_mapping": [
|
| 664 |
+
{
|
| 665 |
+
"worker_index": worker_index,
|
| 666 |
+
"physical_gpu_index": gpu_id,
|
| 667 |
+
"gpu_uuid": uuid_by_index.get(gpu_id),
|
| 668 |
+
}
|
| 669 |
+
for worker_index, gpu_id in enumerate(gpu_ids)
|
| 670 |
+
],
|
| 671 |
+
"cuda_visible_devices": inherited_environment.get(
|
| 672 |
+
"CUDA_VISIBLE_DEVICES"
|
| 673 |
+
),
|
| 674 |
+
"launch_gpu_snapshot": gpu_snapshot,
|
| 675 |
+
},
|
| 676 |
+
"normalized_command": _normalized_command(command),
|
| 677 |
+
}
|
| 678 |
+
|
| 679 |
+
|
| 680 |
+
def _resource_policy(args: argparse.Namespace) -> dict[str, Any]:
|
| 681 |
+
"""Describe the frozen reviewer budget and its enforcement boundary."""
|
| 682 |
+
|
| 683 |
+
return {
|
| 684 |
+
"policy_version": RESOURCE_POLICY_VERSION,
|
| 685 |
+
"sample_max_time_sec": args.sample_max_time_sec,
|
| 686 |
+
"hard_question_timeout_sec": args.hard_question_timeout_sec,
|
| 687 |
+
"sample_max_tree_size": args.sample_max_tree_size,
|
| 688 |
+
"sample_max_llm_calls": args.sample_max_llm_calls,
|
| 689 |
+
"time_enforcement": "cooperative_soft",
|
| 690 |
+
"tree_enforcement": "cooperative_soft",
|
| 691 |
+
"hard_wall_time_enforced": False,
|
| 692 |
+
"external_hard_timeout_required": True,
|
| 693 |
+
"interpretation": (
|
| 694 |
+
"The tracked runner checks elapsed time and tree size between search "
|
| 695 |
+
"operations. Already submitted model generations can finish after a "
|
| 696 |
+
"budget is reached, so neither limit is an external hard kill."
|
| 697 |
+
),
|
| 698 |
+
}
|
| 699 |
+
|
| 700 |
+
|
| 701 |
+
def _parser() -> argparse.ArgumentParser:
|
| 702 |
+
parser = argparse.ArgumentParser(
|
| 703 |
+
description="Run and document the submitted proposed-model v12 search."
|
| 704 |
+
)
|
| 705 |
+
parser.add_argument("--questions", type=Path, required=True)
|
| 706 |
+
parser.add_argument("--results", type=Path, required=True)
|
| 707 |
+
parser.add_argument("--checkpoint", type=Path, required=True)
|
| 708 |
+
parser.add_argument(
|
| 709 |
+
"--base-model",
|
| 710 |
+
type=Path,
|
| 711 |
+
help=(
|
| 712 |
+
"Local base-model snapshot. When supplied, QWEN_BASE_MODEL is set "
|
| 713 |
+
"to this resolved directory instead of the adapter-declared Hub ID."
|
| 714 |
+
),
|
| 715 |
+
)
|
| 716 |
+
parser.add_argument("--runner", type=Path, default=DEFAULT_RUNNER)
|
| 717 |
+
parser.add_argument(
|
| 718 |
+
"--manifest",
|
| 719 |
+
type=Path,
|
| 720 |
+
help="Run manifest path (default: RESULTS.run-manifest.json)",
|
| 721 |
+
)
|
| 722 |
+
parser.add_argument("--gpu-ids", default=DEFAULT_GPU_IDS)
|
| 723 |
+
parser.add_argument("--workers", type=int, default=8)
|
| 724 |
+
parser.add_argument("--beam-size", type=int, default=32)
|
| 725 |
+
parser.add_argument("--rollouts", type=int, default=6)
|
| 726 |
+
parser.add_argument("--max-depth", type=int, default=24)
|
| 727 |
+
parser.add_argument("--seed", type=int, default=42)
|
| 728 |
+
parser.add_argument("--temperature", type=float, default=1.2)
|
| 729 |
+
parser.add_argument("--max-new-tokens", type=int, default=2048)
|
| 730 |
+
parser.add_argument("--generation-max-time", type=float, default=20.0)
|
| 731 |
+
parser.add_argument(
|
| 732 |
+
"--sample-max-tree-size",
|
| 733 |
+
type=int,
|
| 734 |
+
default=SAMPLE_MAX_TREE_SIZE,
|
| 735 |
+
help=(
|
| 736 |
+
"Cooperative per-sample tree budget. The value is recorded in every "
|
| 737 |
+
"result row and may change on a later resume."
|
| 738 |
+
),
|
| 739 |
+
)
|
| 740 |
+
parser.add_argument("--max-vars-for-lp", type=int, default=0)
|
| 741 |
+
parser.add_argument("--lp-workers", type=int, default=8)
|
| 742 |
+
parser.add_argument("--min-n", type=int, default=3)
|
| 743 |
+
parser.add_argument("--max-n", type=int, default=15)
|
| 744 |
+
parser.add_argument("--prompt-format", default="canonical")
|
| 745 |
+
parser.add_argument(
|
| 746 |
+
"--normalize-candidates",
|
| 747 |
+
action=argparse.BooleanOptionalAction,
|
| 748 |
+
default=True,
|
| 749 |
+
)
|
| 750 |
+
parser.add_argument("--raw-prompt", action=argparse.BooleanOptionalAction, default=True)
|
| 751 |
+
parser.add_argument("--print-llm", action=argparse.BooleanOptionalAction, default=True)
|
| 752 |
+
parser.add_argument(
|
| 753 |
+
"--sample-max-time-sec",
|
| 754 |
+
type=float,
|
| 755 |
+
default=SAMPLE_MAX_TIME_SEC,
|
| 756 |
+
help=(
|
| 757 |
+
"Cooperative per-sample time budget. The value is recorded in every "
|
| 758 |
+
"result row and may change on a later resume; it is not an external "
|
| 759 |
+
"hard kill."
|
| 760 |
+
),
|
| 761 |
+
)
|
| 762 |
+
parser.add_argument(
|
| 763 |
+
"--hard-question-timeout-sec",
|
| 764 |
+
type=float,
|
| 765 |
+
default=None,
|
| 766 |
+
help=(
|
| 767 |
+
"External supervisor hard timeout recorded in per-row configuration. "
|
| 768 |
+
"This wrapper does not itself enforce the hard kill."
|
| 769 |
+
),
|
| 770 |
+
)
|
| 771 |
+
parser.add_argument(
|
| 772 |
+
"--sample-max-llm-calls",
|
| 773 |
+
type=int,
|
| 774 |
+
default=None,
|
| 775 |
+
help="Optional per-sample LLM-call cap; omitted by default.",
|
| 776 |
+
)
|
| 777 |
+
parser.add_argument("--dry-run", action="store_true")
|
| 778 |
+
parser.add_argument(
|
| 779 |
+
"--resume",
|
| 780 |
+
action="store_true",
|
| 781 |
+
help=(
|
| 782 |
+
"Safely append after an existing nonempty result prefix. The wrapper "
|
| 783 |
+
"validates the prefix and passes v12 --append --resume-from-output."
|
| 784 |
+
),
|
| 785 |
+
)
|
| 786 |
+
parser.add_argument(
|
| 787 |
+
"--resume-source-manifest",
|
| 788 |
+
type=Path,
|
| 789 |
+
help=argparse.SUPPRESS,
|
| 790 |
+
)
|
| 791 |
+
parser.add_argument(
|
| 792 |
+
"--allow-resume-resource-change",
|
| 793 |
+
action="store_true",
|
| 794 |
+
help=argparse.SUPPRESS,
|
| 795 |
+
)
|
| 796 |
+
parser.add_argument(
|
| 797 |
+
"--overwrite",
|
| 798 |
+
action="store_true",
|
| 799 |
+
help="Allow replacing an existing results file or manifest.",
|
| 800 |
+
)
|
| 801 |
+
return parser
|
| 802 |
+
|
| 803 |
+
|
| 804 |
+
def _validate_numeric_args(args: argparse.Namespace) -> None:
|
| 805 |
+
positive = {
|
| 806 |
+
"beam-size": args.beam_size,
|
| 807 |
+
"rollouts": args.rollouts,
|
| 808 |
+
"max-depth": args.max_depth,
|
| 809 |
+
"max-new-tokens": args.max_new_tokens,
|
| 810 |
+
"generation-max-time": args.generation_max_time,
|
| 811 |
+
"sample-max-tree-size": args.sample_max_tree_size,
|
| 812 |
+
}
|
| 813 |
+
for label, value in positive.items():
|
| 814 |
+
if value <= 0:
|
| 815 |
+
raise LauncherError(f"--{label} must be positive")
|
| 816 |
+
if args.sample_max_time_sec is None or args.sample_max_time_sec <= 0:
|
| 817 |
+
raise LauncherError("--sample-max-time-sec must be positive and bounded")
|
| 818 |
+
if (
|
| 819 |
+
args.hard_question_timeout_sec is not None
|
| 820 |
+
and args.hard_question_timeout_sec <= 0
|
| 821 |
+
):
|
| 822 |
+
raise LauncherError("--hard-question-timeout-sec must be positive when set")
|
| 823 |
+
if args.sample_max_llm_calls is not None and args.sample_max_llm_calls <= 0:
|
| 824 |
+
raise LauncherError("--sample-max-llm-calls must be positive when supplied")
|
| 825 |
+
|
| 826 |
+
|
| 827 |
+
def _exact_verify(results: Path) -> dict[str, Any]:
|
| 828 |
+
from beam_search_model2.verify_proof_certificate import verify_jsonl
|
| 829 |
+
|
| 830 |
+
valid, invalid, errors, skipped = verify_jsonl(results)
|
| 831 |
+
return {
|
| 832 |
+
"valid": valid,
|
| 833 |
+
"invalid": invalid,
|
| 834 |
+
"errors": errors,
|
| 835 |
+
"skipped_unproved": skipped,
|
| 836 |
+
"passed": invalid == 0 and errors == 0,
|
| 837 |
+
"verifier": "beam_search_model2.verify_proof_certificate.verify_jsonl",
|
| 838 |
+
}
|
| 839 |
+
|
| 840 |
+
|
| 841 |
+
def main(argv: Sequence[str] | None = None) -> int:
|
| 842 |
+
args = _parser().parse_args(argv)
|
| 843 |
+
try:
|
| 844 |
+
_validate_numeric_args(args)
|
| 845 |
+
if args.resume and args.overwrite:
|
| 846 |
+
raise LauncherError("--resume and --overwrite are mutually exclusive")
|
| 847 |
+
repo_root = Path(__file__).resolve().parents[2]
|
| 848 |
+
questions = _resolve_existing(args.questions, "questions file")
|
| 849 |
+
runner_candidate = (
|
| 850 |
+
repo_root / args.runner if args.runner == DEFAULT_RUNNER else args.runner
|
| 851 |
+
)
|
| 852 |
+
runner = _resolve_existing(runner_candidate, "runner")
|
| 853 |
+
checkpoint = args.checkpoint.expanduser().resolve()
|
| 854 |
+
if args.base_model is not None:
|
| 855 |
+
args.base_model = args.base_model.expanduser().resolve()
|
| 856 |
+
if not args.base_model.is_dir():
|
| 857 |
+
raise LauncherError(
|
| 858 |
+
f"base-model snapshot does not exist or is not a directory: "
|
| 859 |
+
f"{args.base_model}"
|
| 860 |
+
)
|
| 861 |
+
results = args.results.expanduser().resolve()
|
| 862 |
+
gpu_ids = _parse_gpu_ids(args.gpu_ids, args.workers)
|
| 863 |
+
profile = _question_profile(questions)
|
| 864 |
+
resume_prior: dict[str, Any] | None = None
|
| 865 |
+
prior_verification: dict[str, Any] | None = None
|
| 866 |
+
if args.resume:
|
| 867 |
+
resume_prior = _resume_artifact(results, questions, profile["rows"])
|
| 868 |
+
try:
|
| 869 |
+
prior_verification = _exact_verify(results)
|
| 870 |
+
except Exception as exc:
|
| 871 |
+
raise LauncherError(
|
| 872 |
+
"could not exactly verify resume prefix: "
|
| 873 |
+
f"{type(exc).__name__}: {exc}"
|
| 874 |
+
) from exc
|
| 875 |
+
if not prior_verification["passed"]:
|
| 876 |
+
raise LauncherError(
|
| 877 |
+
"resume prefix failed exact verification; refusing to append"
|
| 878 |
+
)
|
| 879 |
+
|
| 880 |
+
invocation_id = uuid.uuid4().hex
|
| 881 |
+
recorded_utc = _utc_now()
|
| 882 |
+
manifest_path = (
|
| 883 |
+
args.manifest.expanduser().resolve()
|
| 884 |
+
if args.manifest
|
| 885 |
+
else results.with_name(
|
| 886 |
+
f"{results.name}.{invocation_id}.run-manifest.json"
|
| 887 |
+
)
|
| 888 |
+
)
|
| 889 |
+
|
| 890 |
+
if results in {questions, runner, manifest_path}:
|
| 891 |
+
raise LauncherError("results path must be distinct from inputs and manifest")
|
| 892 |
+
if manifest_path in {questions, runner}:
|
| 893 |
+
raise LauncherError("manifest path must be distinct from input files")
|
| 894 |
+
if results.is_dir():
|
| 895 |
+
raise LauncherError(f"results path is a directory: {results}")
|
| 896 |
+
if manifest_path.is_dir():
|
| 897 |
+
raise LauncherError(f"manifest path is a directory: {manifest_path}")
|
| 898 |
+
if checkpoint.is_dir() and (
|
| 899 |
+
results.is_relative_to(checkpoint) or manifest_path.is_relative_to(checkpoint)
|
| 900 |
+
):
|
| 901 |
+
raise LauncherError("results and manifest must not be written inside checkpoint")
|
| 902 |
+
|
| 903 |
+
if manifest_path.exists():
|
| 904 |
+
raise LauncherError(f"manifest already exists: {manifest_path}")
|
| 905 |
+
if (
|
| 906 |
+
not args.dry_run
|
| 907 |
+
and results.exists()
|
| 908 |
+
and not args.overwrite
|
| 909 |
+
and not args.resume
|
| 910 |
+
):
|
| 911 |
+
raise LauncherError(
|
| 912 |
+
f"results already exist; pass --overwrite to replace them: {results}"
|
| 913 |
+
)
|
| 914 |
+
|
| 915 |
+
if profile["min_n"] < args.min_n or profile["max_n"] > args.max_n:
|
| 916 |
+
raise LauncherError(
|
| 917 |
+
"questions contain n outside the configured runner filter: "
|
| 918 |
+
f"questions={profile['min_n']}..{profile['max_n']}, "
|
| 919 |
+
f"filter={args.min_n}..{args.max_n}"
|
| 920 |
+
)
|
| 921 |
+
checkpoint_info = _checkpoint_artifact(checkpoint)
|
| 922 |
+
runner_info = _file_artifact(runner)
|
| 923 |
+
repository = _git_metadata(repo_root)
|
| 924 |
+
inherited_environment = {
|
| 925 |
+
name: os.environ.get(name) for name in RELEVANT_INHERITED_ENV
|
| 926 |
+
}
|
| 927 |
+
settings = _settings(args, gpu_ids)
|
| 928 |
+
runtime_environment: dict[str, Any] | None = None
|
| 929 |
+
if not args.dry_run:
|
| 930 |
+
runtime_environment = {
|
| 931 |
+
"python": {
|
| 932 |
+
"executable": sys.executable,
|
| 933 |
+
"version": sys.version,
|
| 934 |
+
"implementation": platform.python_implementation(),
|
| 935 |
+
},
|
| 936 |
+
"packages": _package_metadata(),
|
| 937 |
+
"hardware": _hardware_metadata(gpu_ids),
|
| 938 |
+
}
|
| 939 |
+
# Keep this as the final hardware query before manifest publication and
|
| 940 |
+
# child start; rows describe launch-time state, not an earlier preflight.
|
| 941 |
+
gpu_snapshot = _launch_gpu_snapshot(gpu_ids)
|
| 942 |
+
if not args.dry_run:
|
| 943 |
+
_require_complete_launch_gpu_snapshot(gpu_snapshot, gpu_ids)
|
| 944 |
+
command, environment = build_command(
|
| 945 |
+
args,
|
| 946 |
+
questions=questions,
|
| 947 |
+
results=results,
|
| 948 |
+
checkpoint=checkpoint,
|
| 949 |
+
runner=runner,
|
| 950 |
+
gpu_ids=gpu_ids,
|
| 951 |
+
execution_config_manifest=manifest_path,
|
| 952 |
+
)
|
| 953 |
+
execution_config = _execution_config(
|
| 954 |
+
invocation_id=invocation_id,
|
| 955 |
+
recorded_utc=recorded_utc,
|
| 956 |
+
origin="resume" if args.resume else "fresh",
|
| 957 |
+
repo_root=repo_root,
|
| 958 |
+
repository=repository,
|
| 959 |
+
runner=runner,
|
| 960 |
+
runner_artifact=runner_info,
|
| 961 |
+
checkpoint=checkpoint,
|
| 962 |
+
checkpoint_artifact=checkpoint_info,
|
| 963 |
+
settings=settings,
|
| 964 |
+
gpu_ids=gpu_ids,
|
| 965 |
+
inherited_environment=inherited_environment,
|
| 966 |
+
gpu_snapshot=gpu_snapshot,
|
| 967 |
+
command=command,
|
| 968 |
+
)
|
| 969 |
+
plan = {
|
| 970 |
+
"command": command,
|
| 971 |
+
"command_display": _display_command(command),
|
| 972 |
+
"cwd": str(repo_root),
|
| 973 |
+
"environment_overrides": environment,
|
| 974 |
+
"inherited_environment_relevant": inherited_environment,
|
| 975 |
+
"settings": settings,
|
| 976 |
+
}
|
| 977 |
+
manifest: dict[str, Any] = {
|
| 978 |
+
"schema_version": SCHEMA_VERSION,
|
| 979 |
+
"execution_config": execution_config,
|
| 980 |
+
"mode": "dry-run" if args.dry_run else "execute",
|
| 981 |
+
"status": "planned" if args.dry_run else "preparing",
|
| 982 |
+
"plan": plan,
|
| 983 |
+
"artifacts": {
|
| 984 |
+
"questions": _file_artifact(questions),
|
| 985 |
+
"runner": runner_info,
|
| 986 |
+
"checkpoint": checkpoint_info,
|
| 987 |
+
"results_path": str(results),
|
| 988 |
+
"manifest_path": str(manifest_path),
|
| 989 |
+
},
|
| 990 |
+
"questions": profile,
|
| 991 |
+
"resource_policy": _resource_policy(args),
|
| 992 |
+
}
|
| 993 |
+
if resume_prior is not None:
|
| 994 |
+
manifest["resume"] = {
|
| 995 |
+
"enabled": True,
|
| 996 |
+
"prior_results": resume_prior,
|
| 997 |
+
"prior_exact_verification": prior_verification,
|
| 998 |
+
"remaining_rows": profile["rows"] - int(resume_prior["rows"]),
|
| 999 |
+
"runner_flags": ["--append", "--resume-from-output"],
|
| 1000 |
+
"manifest_policy": (
|
| 1001 |
+
"Resume trusts no earlier configuration. The exact input prefix "
|
| 1002 |
+
"and proof certificates are validated, and this invocation is "
|
| 1003 |
+
"self-described by its own unique manifest."
|
| 1004 |
+
),
|
| 1005 |
+
}
|
| 1006 |
+
|
| 1007 |
+
if args.dry_run:
|
| 1008 |
+
_write_manifest_exclusive(manifest_path, manifest)
|
| 1009 |
+
print(json.dumps(manifest, ensure_ascii=False, indent=2, sort_keys=True))
|
| 1010 |
+
return 0
|
| 1011 |
+
|
| 1012 |
+
results.parent.mkdir(parents=True, exist_ok=True)
|
| 1013 |
+
manifest["repository"] = repository
|
| 1014 |
+
manifest["runtime_environment"] = runtime_environment
|
| 1015 |
+
manifest["status"] = "running"
|
| 1016 |
+
manifest["runtime"] = {"started_utc": recorded_utc}
|
| 1017 |
+
# The supervisor reads this live manifest to stamp synthetic timeout
|
| 1018 |
+
# rows, so execution_config must be authoritative before child start.
|
| 1019 |
+
_write_manifest_exclusive(manifest_path, manifest)
|
| 1020 |
+
|
| 1021 |
+
child_environment = os.environ.copy()
|
| 1022 |
+
child_environment.update(environment)
|
| 1023 |
+
started = time.perf_counter()
|
| 1024 |
+
try:
|
| 1025 |
+
completed = subprocess.run(
|
| 1026 |
+
command,
|
| 1027 |
+
cwd=repo_root,
|
| 1028 |
+
env=child_environment,
|
| 1029 |
+
check=False,
|
| 1030 |
+
)
|
| 1031 |
+
return_code = completed.returncode
|
| 1032 |
+
except KeyboardInterrupt:
|
| 1033 |
+
return_code = 130
|
| 1034 |
+
manifest["status"] = "interrupted"
|
| 1035 |
+
except OSError as exc:
|
| 1036 |
+
return_code = 127
|
| 1037 |
+
manifest["status"] = "runner_failed"
|
| 1038 |
+
manifest["runtime"]["launch_error"] = f"{type(exc).__name__}: {exc}"
|
| 1039 |
+
wall_time = time.perf_counter() - started
|
| 1040 |
+
manifest["runtime"].update(
|
| 1041 |
+
{
|
| 1042 |
+
"ended_utc": _utc_now(),
|
| 1043 |
+
"wall_time_sec": wall_time,
|
| 1044 |
+
"return_code": return_code,
|
| 1045 |
+
}
|
| 1046 |
+
)
|
| 1047 |
+
|
| 1048 |
+
prefix_unchanged = (
|
| 1049 |
+
_prefix_is_unchanged(results, resume_prior)
|
| 1050 |
+
if resume_prior is not None
|
| 1051 |
+
else None
|
| 1052 |
+
)
|
| 1053 |
+
if resume_prior is not None:
|
| 1054 |
+
manifest["resume"]["prior_prefix_unchanged"] = prefix_unchanged
|
| 1055 |
+
|
| 1056 |
+
if return_code == 0:
|
| 1057 |
+
if not results.is_file():
|
| 1058 |
+
manifest["status"] = "verification_failed"
|
| 1059 |
+
manifest["exact_verification"] = {
|
| 1060 |
+
"passed": False,
|
| 1061 |
+
"error": "runner returned zero but did not create the results file",
|
| 1062 |
+
}
|
| 1063 |
+
final_code = 4
|
| 1064 |
+
else:
|
| 1065 |
+
manifest["artifacts"]["results"] = _file_artifact(results)
|
| 1066 |
+
result_rows = _count_jsonl_rows(results)
|
| 1067 |
+
manifest["artifacts"]["results"]["rows"] = result_rows
|
| 1068 |
+
try:
|
| 1069 |
+
verification = _exact_verify(results)
|
| 1070 |
+
except Exception as exc: # Preserve the run even if verifier setup fails.
|
| 1071 |
+
verification = {
|
| 1072 |
+
"passed": False,
|
| 1073 |
+
"error": f"{type(exc).__name__}: {exc}",
|
| 1074 |
+
}
|
| 1075 |
+
manifest["exact_verification"] = verification
|
| 1076 |
+
manifest["result_completeness"] = {
|
| 1077 |
+
"expected_rows": profile["rows"],
|
| 1078 |
+
"actual_rows": result_rows,
|
| 1079 |
+
"passed": result_rows == profile["rows"],
|
| 1080 |
+
}
|
| 1081 |
+
if (
|
| 1082 |
+
verification["passed"]
|
| 1083 |
+
and result_rows == profile["rows"]
|
| 1084 |
+
and prefix_unchanged is not False
|
| 1085 |
+
):
|
| 1086 |
+
manifest["status"] = "succeeded"
|
| 1087 |
+
final_code = 0
|
| 1088 |
+
else:
|
| 1089 |
+
manifest["status"] = "verification_failed"
|
| 1090 |
+
final_code = 4
|
| 1091 |
+
else:
|
| 1092 |
+
manifest["status"] = (
|
| 1093 |
+
manifest["status"] if manifest["status"] == "interrupted" else "runner_failed"
|
| 1094 |
+
)
|
| 1095 |
+
if results.is_file():
|
| 1096 |
+
manifest["artifacts"]["partial_results"] = _file_artifact(results)
|
| 1097 |
+
final_code = return_code
|
| 1098 |
+
|
| 1099 |
+
_write_manifest(manifest_path, manifest)
|
| 1100 |
+
print(json.dumps(manifest, ensure_ascii=False, indent=2, sort_keys=True))
|
| 1101 |
+
return final_code
|
| 1102 |
+
except LauncherError as exc:
|
| 1103 |
+
print(f"error: {exc}", file=sys.stderr)
|
| 1104 |
+
return 2
|
| 1105 |
+
|
| 1106 |
+
|
| 1107 |
+
if __name__ == "__main__":
|
| 1108 |
+
raise SystemExit(main())
|
mce220_runner/experiments/reviewer_response/supervise_search_benchmark.py
ADDED
|
@@ -0,0 +1,780 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Externally enforce a hard per-question limit for the v12 search launcher.
|
| 2 |
+
|
| 3 |
+
The submitted runner's time budget is cooperative: model calls already in flight
|
| 4 |
+
may finish after it expires. This supervisor leaves that runner and its
|
| 5 |
+
reproducibility wrapper unchanged. It watches the wrapper's streamed JSONL
|
| 6 |
+
output, kills the complete child process tree at a hard deadline, records a
|
| 7 |
+
schema-compatible timeout row for the exact pending input, and starts a new
|
| 8 |
+
wrapper resume segment at the following question.
|
| 9 |
+
|
| 10 |
+
Every invocation and process segment gets a collision-resistant identity and
|
| 11 |
+
new, never-overwritten logs and manifests. The result prefix is authoritative:
|
| 12 |
+
the mutable state file may be absent, stale, or left by a different invocation.
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
import argparse
|
| 18 |
+
import hashlib
|
| 19 |
+
import json
|
| 20 |
+
import os
|
| 21 |
+
import signal
|
| 22 |
+
import subprocess
|
| 23 |
+
import sys
|
| 24 |
+
import time
|
| 25 |
+
import uuid
|
| 26 |
+
from dataclasses import dataclass
|
| 27 |
+
from datetime import datetime, timezone
|
| 28 |
+
from pathlib import Path
|
| 29 |
+
from typing import Any, Sequence
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
SCHEMA_VERSION = "reviewer-hard-supervisor-v2"
|
| 33 |
+
ATTEMPT_SCHEMA_VERSION = "reviewer-hard-supervisor-attempt-v2"
|
| 34 |
+
EXECUTION_CONFIG_SCHEMA_VERSION = "e2n-execution-config-v1"
|
| 35 |
+
QUESTION_TIMEOUT_SEC = 1000.0
|
| 36 |
+
TREE_SIZE_CAP = 6000
|
| 37 |
+
TOTAL_JOB_STOP_SEC = 42.0 * 60.0 * 60.0
|
| 38 |
+
DEFAULT_LAUNCHER = Path("experiments/reviewer_response/run_search_benchmark.py")
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
class SupervisorError(ValueError):
|
| 42 |
+
"""Raised when an artifact or resume prefix is unsafe."""
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
@dataclass(frozen=True)
|
| 46 |
+
class Question:
|
| 47 |
+
row: dict[str, Any]
|
| 48 |
+
source_line_number: int
|
| 49 |
+
input_sha256: str
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def _utc_now() -> str:
|
| 53 |
+
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def _sha256_file(path: Path) -> str:
|
| 57 |
+
digest = hashlib.sha256()
|
| 58 |
+
with path.open("rb") as stream:
|
| 59 |
+
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
| 60 |
+
digest.update(chunk)
|
| 61 |
+
return digest.hexdigest()
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def _artifact(path: Path) -> dict[str, Any]:
|
| 65 |
+
return {
|
| 66 |
+
"path": str(path),
|
| 67 |
+
"bytes": path.stat().st_size,
|
| 68 |
+
"sha256": _sha256_file(path),
|
| 69 |
+
}
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def _write_json_atomic(path: Path, value: dict[str, Any]) -> None:
|
| 73 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 74 |
+
temporary = path.with_name(path.name + f".tmp-{os.getpid()}")
|
| 75 |
+
temporary.write_text(
|
| 76 |
+
json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n",
|
| 77 |
+
encoding="utf-8",
|
| 78 |
+
)
|
| 79 |
+
temporary.replace(path)
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def _write_json_exclusive(path: Path, value: dict[str, Any]) -> None:
|
| 83 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 84 |
+
with path.open("x", encoding="utf-8") as stream:
|
| 85 |
+
json.dump(value, stream, ensure_ascii=False, indent=2, sort_keys=True)
|
| 86 |
+
stream.write("\n")
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
def _read_questions(path: Path) -> list[Question]:
|
| 90 |
+
questions: list[Question] = []
|
| 91 |
+
ids: set[str] = set()
|
| 92 |
+
with path.open("rb") as stream:
|
| 93 |
+
for source_line_number, raw in enumerate(stream, start=1):
|
| 94 |
+
stripped = raw.strip()
|
| 95 |
+
if not stripped:
|
| 96 |
+
continue
|
| 97 |
+
try:
|
| 98 |
+
row = json.loads(stripped.decode("utf-8"))
|
| 99 |
+
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
| 100 |
+
raise SupervisorError(
|
| 101 |
+
f"invalid question JSON on source line {source_line_number}: {exc}"
|
| 102 |
+
) from exc
|
| 103 |
+
if not isinstance(row, dict):
|
| 104 |
+
raise SupervisorError(
|
| 105 |
+
f"question source line {source_line_number} is not an object"
|
| 106 |
+
)
|
| 107 |
+
missing = {"id", "n", "ineq", "conditions"}.difference(row)
|
| 108 |
+
if missing:
|
| 109 |
+
raise SupervisorError(
|
| 110 |
+
f"question source line {source_line_number} is missing {sorted(missing)}"
|
| 111 |
+
)
|
| 112 |
+
question_id = row["id"]
|
| 113 |
+
if not isinstance(question_id, str) or not question_id:
|
| 114 |
+
raise SupervisorError(
|
| 115 |
+
f"question source line {source_line_number} has an invalid id"
|
| 116 |
+
)
|
| 117 |
+
if question_id in ids:
|
| 118 |
+
raise SupervisorError(f"duplicate question id: {question_id}")
|
| 119 |
+
ids.add(question_id)
|
| 120 |
+
if not isinstance(row["conditions"], list):
|
| 121 |
+
raise SupervisorError(f"question {question_id} conditions is not a list")
|
| 122 |
+
questions.append(
|
| 123 |
+
Question(
|
| 124 |
+
row=row,
|
| 125 |
+
source_line_number=source_line_number,
|
| 126 |
+
input_sha256=hashlib.sha256(stripped).hexdigest(),
|
| 127 |
+
)
|
| 128 |
+
)
|
| 129 |
+
if not questions:
|
| 130 |
+
raise SupervisorError(f"questions file is empty: {path}")
|
| 131 |
+
return questions
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
def _read_complete_result_rows(path: Path) -> tuple[list[dict[str, Any]], int]:
|
| 135 |
+
"""Read newline-terminated rows and return their safe byte boundary.
|
| 136 |
+
|
| 137 |
+
A final unterminated line is ignored while a child may be writing it. Any
|
| 138 |
+
malformed *terminated* row is an artifact error, never a reason to skip an
|
| 139 |
+
input by count.
|
| 140 |
+
"""
|
| 141 |
+
|
| 142 |
+
if not path.exists():
|
| 143 |
+
return [], 0
|
| 144 |
+
rows: list[dict[str, Any]] = []
|
| 145 |
+
safe_bytes = 0
|
| 146 |
+
with path.open("rb") as stream:
|
| 147 |
+
while True:
|
| 148 |
+
raw = stream.readline()
|
| 149 |
+
if not raw:
|
| 150 |
+
break
|
| 151 |
+
if not raw.endswith((b"\n", b"\r")):
|
| 152 |
+
break
|
| 153 |
+
safe_bytes = stream.tell()
|
| 154 |
+
if not raw.strip():
|
| 155 |
+
continue
|
| 156 |
+
try:
|
| 157 |
+
row = json.loads(raw.decode("utf-8"))
|
| 158 |
+
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
| 159 |
+
raise SupervisorError(
|
| 160 |
+
f"malformed terminated result row {len(rows) + 1}: {exc}"
|
| 161 |
+
) from exc
|
| 162 |
+
if not isinstance(row, dict):
|
| 163 |
+
raise SupervisorError(f"result row {len(rows) + 1} is not an object")
|
| 164 |
+
rows.append(row)
|
| 165 |
+
return rows, safe_bytes
|
| 166 |
+
|
| 167 |
+
|
| 168 |
+
def _validate_prefix(
|
| 169 |
+
results: Path, questions: Sequence[Question], *, require_terminated: bool
|
| 170 |
+
) -> list[dict[str, Any]]:
|
| 171 |
+
rows, safe_bytes = _read_complete_result_rows(results)
|
| 172 |
+
if results.exists() and require_terminated and results.stat().st_size != safe_bytes:
|
| 173 |
+
raise SupervisorError("results end in an incomplete JSONL row")
|
| 174 |
+
if len(rows) > len(questions):
|
| 175 |
+
raise SupervisorError(
|
| 176 |
+
f"results contain more rows than questions: {len(rows)} > {len(questions)}"
|
| 177 |
+
)
|
| 178 |
+
seen_ids: set[str] = set()
|
| 179 |
+
for index, (result, question) in enumerate(zip(rows, questions), start=1):
|
| 180 |
+
if result.get("input") != question.row:
|
| 181 |
+
raise SupervisorError(
|
| 182 |
+
f"result row {index} is not the exact frozen question prefix"
|
| 183 |
+
)
|
| 184 |
+
result_id = result["input"].get("id")
|
| 185 |
+
if result_id != question.row["id"]:
|
| 186 |
+
raise SupervisorError(f"result row {index} has a mismatched question id")
|
| 187 |
+
if result_id in seen_ids:
|
| 188 |
+
raise SupervisorError(f"duplicate result id: {result_id}")
|
| 189 |
+
seen_ids.add(result_id)
|
| 190 |
+
if "input_sha256" in result and result["input_sha256"] != question.input_sha256:
|
| 191 |
+
raise SupervisorError(f"result row {index} has a mismatched input_sha256")
|
| 192 |
+
return rows
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
def _timeout_row(
|
| 196 |
+
question: Question,
|
| 197 |
+
*,
|
| 198 |
+
timeout_sec: float,
|
| 199 |
+
observed_wall_sec: float,
|
| 200 |
+
attempt_number: int,
|
| 201 |
+
execution_config: dict[str, Any],
|
| 202 |
+
tree_size_cap: int = TREE_SIZE_CAP,
|
| 203 |
+
) -> dict[str, Any]:
|
| 204 |
+
row = question.row
|
| 205 |
+
return {
|
| 206 |
+
"source_line_number": question.source_line_number,
|
| 207 |
+
"input_sha256": question.input_sha256,
|
| 208 |
+
"input": row,
|
| 209 |
+
"mapped": {
|
| 210 |
+
"ineq": row["ineq"],
|
| 211 |
+
"n": int(row["n"]),
|
| 212 |
+
"mapping": {},
|
| 213 |
+
},
|
| 214 |
+
"run": {
|
| 215 |
+
"proved": False,
|
| 216 |
+
"time_sec": float(timeout_sec),
|
| 217 |
+
"search_time_sec": float(timeout_sec),
|
| 218 |
+
"tree_size": 0,
|
| 219 |
+
"expr_str": None,
|
| 220 |
+
"x": None,
|
| 221 |
+
"error": (
|
| 222 |
+
f"ExternalHardTimeout: question exceeded {timeout_sec:g} seconds"
|
| 223 |
+
),
|
| 224 |
+
"llm_usage": {},
|
| 225 |
+
"stop_reason": "external_hard_timeout",
|
| 226 |
+
"budget_applied": {
|
| 227 |
+
"tree_size": tree_size_cap,
|
| 228 |
+
"time_sec": float(timeout_sec),
|
| 229 |
+
"llm_calls": 0,
|
| 230 |
+
"llm_fails": 0,
|
| 231 |
+
"depth1_rollouts": None,
|
| 232 |
+
"enforcement": "external_hard_process_tree_kill",
|
| 233 |
+
},
|
| 234 |
+
"external_supervision": {
|
| 235 |
+
"schema_version": SCHEMA_VERSION,
|
| 236 |
+
"attempt_number": attempt_number,
|
| 237 |
+
"hard_timeout_sec": float(timeout_sec),
|
| 238 |
+
"observed_wall_sec": float(observed_wall_sec),
|
| 239 |
+
"synthetic_result": True,
|
| 240 |
+
},
|
| 241 |
+
"execution_config": execution_config,
|
| 242 |
+
},
|
| 243 |
+
}
|
| 244 |
+
|
| 245 |
+
|
| 246 |
+
def _append_row(path: Path, row: dict[str, Any]) -> None:
|
| 247 |
+
"""Append one complete JSONL row with one O_APPEND write, then fsync it."""
|
| 248 |
+
|
| 249 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 250 |
+
payload = (json.dumps(row, ensure_ascii=False) + "\n").encode("utf-8")
|
| 251 |
+
descriptor = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_APPEND)
|
| 252 |
+
try:
|
| 253 |
+
written = os.write(descriptor, payload)
|
| 254 |
+
if written != len(payload):
|
| 255 |
+
raise OSError(
|
| 256 |
+
"short atomic JSONL append: "
|
| 257 |
+
f"wrote {written} of {len(payload)} bytes"
|
| 258 |
+
)
|
| 259 |
+
os.fsync(descriptor)
|
| 260 |
+
finally:
|
| 261 |
+
os.close(descriptor)
|
| 262 |
+
|
| 263 |
+
|
| 264 |
+
def _popen_group(command: Sequence[str], **kwargs: Any) -> subprocess.Popen[Any]:
|
| 265 |
+
if os.name == "nt":
|
| 266 |
+
kwargs["creationflags"] = kwargs.get("creationflags", 0) | subprocess.CREATE_NEW_PROCESS_GROUP
|
| 267 |
+
else:
|
| 268 |
+
kwargs["start_new_session"] = True
|
| 269 |
+
return subprocess.Popen(list(command), **kwargs)
|
| 270 |
+
|
| 271 |
+
|
| 272 |
+
def _kill_process_tree(process: subprocess.Popen[Any], *, grace_sec: float = 3.0) -> None:
|
| 273 |
+
"""Terminate the exact child tree and wait until its root is reaped."""
|
| 274 |
+
|
| 275 |
+
# Do not return merely because the group leader has exited. A launcher can
|
| 276 |
+
# terminate between poll() and this call while one of its workers remains
|
| 277 |
+
# alive. The process group/tree operation must still be attempted.
|
| 278 |
+
root_already_exited = process.poll() is not None
|
| 279 |
+
if os.name == "nt":
|
| 280 |
+
try:
|
| 281 |
+
completed = subprocess.run(
|
| 282 |
+
["taskkill", "/PID", str(process.pid), "/T", "/F"],
|
| 283 |
+
check=False,
|
| 284 |
+
stdout=subprocess.DEVNULL,
|
| 285 |
+
stderr=subprocess.DEVNULL,
|
| 286 |
+
timeout=max(grace_sec, 1.0),
|
| 287 |
+
)
|
| 288 |
+
except subprocess.TimeoutExpired as exc:
|
| 289 |
+
raise SupervisorError(
|
| 290 |
+
f"timed out killing launcher process tree rooted at pid {process.pid}"
|
| 291 |
+
) from exc
|
| 292 |
+
if completed.returncode != 0:
|
| 293 |
+
raise SupervisorError(
|
| 294 |
+
"taskkill could not confirm termination of launcher process tree "
|
| 295 |
+
f"rooted at pid {process.pid} (exit {completed.returncode}); "
|
| 296 |
+
"refusing stale-lock removal or relaunch"
|
| 297 |
+
)
|
| 298 |
+
else:
|
| 299 |
+
try:
|
| 300 |
+
os.killpg(process.pid, signal.SIGTERM)
|
| 301 |
+
except ProcessLookupError:
|
| 302 |
+
pass
|
| 303 |
+
if not root_already_exited:
|
| 304 |
+
try:
|
| 305 |
+
process.wait(timeout=grace_sec)
|
| 306 |
+
except subprocess.TimeoutExpired:
|
| 307 |
+
pass
|
| 308 |
+
# The group leader can exit while a worker ignores SIGTERM. Always
|
| 309 |
+
# follow with SIGKILL for the same isolated process group.
|
| 310 |
+
try:
|
| 311 |
+
os.killpg(process.pid, signal.SIGKILL)
|
| 312 |
+
except ProcessLookupError:
|
| 313 |
+
pass
|
| 314 |
+
try:
|
| 315 |
+
process.wait(timeout=max(grace_sec, 1.0))
|
| 316 |
+
except subprocess.TimeoutExpired as exc:
|
| 317 |
+
raise SupervisorError(
|
| 318 |
+
f"could not reap killed launcher process tree rooted at pid {process.pid}"
|
| 319 |
+
) from exc
|
| 320 |
+
|
| 321 |
+
|
| 322 |
+
def _protected_launcher_args(extra: Sequence[str]) -> None:
|
| 323 |
+
protected = {
|
| 324 |
+
"--questions",
|
| 325 |
+
"--results",
|
| 326 |
+
"--checkpoint",
|
| 327 |
+
"--manifest",
|
| 328 |
+
"--resume",
|
| 329 |
+
"--resume-source-manifest",
|
| 330 |
+
"--overwrite",
|
| 331 |
+
"--sample-max-time-sec",
|
| 332 |
+
"--hard-question-timeout-sec",
|
| 333 |
+
"--sample-max-tree-size",
|
| 334 |
+
}
|
| 335 |
+
used = sorted(
|
| 336 |
+
item
|
| 337 |
+
for item in extra
|
| 338 |
+
if item in protected or any(item.startswith(flag + "=") for flag in protected)
|
| 339 |
+
)
|
| 340 |
+
if used:
|
| 341 |
+
raise SupervisorError(
|
| 342 |
+
"launcher arguments may not override supervisor-controlled flags: "
|
| 343 |
+
+ ", ".join(used)
|
| 344 |
+
)
|
| 345 |
+
|
| 346 |
+
|
| 347 |
+
def _timeout_execution_config(
|
| 348 |
+
wrapper_manifest: Path,
|
| 349 |
+
*,
|
| 350 |
+
invocation_id: str,
|
| 351 |
+
attempt_id: str,
|
| 352 |
+
) -> dict[str, Any]:
|
| 353 |
+
"""Copy and validate the live wrapper config for a synthetic timeout row."""
|
| 354 |
+
|
| 355 |
+
try:
|
| 356 |
+
manifest = json.loads(wrapper_manifest.read_text(encoding="utf-8"))
|
| 357 |
+
except (OSError, json.JSONDecodeError) as exc:
|
| 358 |
+
raise SupervisorError(
|
| 359 |
+
"cannot stamp synthetic timeout without the authoritative wrapper "
|
| 360 |
+
f"manifest {wrapper_manifest}: {exc}"
|
| 361 |
+
) from exc
|
| 362 |
+
authoritative = manifest.get("execution_config")
|
| 363 |
+
if not isinstance(authoritative, dict):
|
| 364 |
+
raise SupervisorError(
|
| 365 |
+
"authoritative wrapper manifest has no execution_config object"
|
| 366 |
+
)
|
| 367 |
+
required = {
|
| 368 |
+
"invocation_id",
|
| 369 |
+
"recorded_utc",
|
| 370 |
+
"origin",
|
| 371 |
+
"code",
|
| 372 |
+
"model",
|
| 373 |
+
"search",
|
| 374 |
+
"resources",
|
| 375 |
+
"normalized_command",
|
| 376 |
+
}
|
| 377 |
+
missing = sorted(required.difference(authoritative))
|
| 378 |
+
if authoritative.get("schema_version") != EXECUTION_CONFIG_SCHEMA_VERSION:
|
| 379 |
+
raise SupervisorError(
|
| 380 |
+
"authoritative wrapper execution_config has unsupported schema "
|
| 381 |
+
f"{authoritative.get('schema_version')!r}"
|
| 382 |
+
)
|
| 383 |
+
if missing:
|
| 384 |
+
raise SupervisorError(
|
| 385 |
+
"authoritative wrapper execution_config is missing fields: "
|
| 386 |
+
+ ", ".join(missing)
|
| 387 |
+
)
|
| 388 |
+
# JSON round-tripping detaches this row from the mutable manifest object.
|
| 389 |
+
config = json.loads(json.dumps(authoritative, ensure_ascii=False))
|
| 390 |
+
config["origin"] = "supervisor_synthetic_timeout"
|
| 391 |
+
config["supervision"] = {
|
| 392 |
+
"invocation_id": invocation_id,
|
| 393 |
+
"attempt_id": attempt_id,
|
| 394 |
+
}
|
| 395 |
+
return config
|
| 396 |
+
|
| 397 |
+
|
| 398 |
+
def _launcher_command(
|
| 399 |
+
*,
|
| 400 |
+
launcher: Path,
|
| 401 |
+
questions: Path,
|
| 402 |
+
results: Path,
|
| 403 |
+
checkpoint: Path,
|
| 404 |
+
wrapper_manifest: Path,
|
| 405 |
+
launcher_args: Sequence[str],
|
| 406 |
+
resume: bool,
|
| 407 |
+
question_timeout_sec: float,
|
| 408 |
+
tree_size_cap: int = TREE_SIZE_CAP,
|
| 409 |
+
) -> list[str]:
|
| 410 |
+
command = [
|
| 411 |
+
sys.executable,
|
| 412 |
+
str(launcher),
|
| 413 |
+
"--questions",
|
| 414 |
+
str(questions),
|
| 415 |
+
"--results",
|
| 416 |
+
str(results),
|
| 417 |
+
"--checkpoint",
|
| 418 |
+
str(checkpoint),
|
| 419 |
+
"--manifest",
|
| 420 |
+
str(wrapper_manifest),
|
| 421 |
+
"--sample-max-time-sec",
|
| 422 |
+
str(float(question_timeout_sec)),
|
| 423 |
+
"--hard-question-timeout-sec",
|
| 424 |
+
str(float(question_timeout_sec)),
|
| 425 |
+
"--sample-max-tree-size",
|
| 426 |
+
str(tree_size_cap),
|
| 427 |
+
*launcher_args,
|
| 428 |
+
]
|
| 429 |
+
if resume:
|
| 430 |
+
command.append("--resume")
|
| 431 |
+
return command
|
| 432 |
+
|
| 433 |
+
|
| 434 |
+
def run_supervisor(
|
| 435 |
+
*,
|
| 436 |
+
questions_path: Path,
|
| 437 |
+
results_path: Path,
|
| 438 |
+
checkpoint: Path,
|
| 439 |
+
launcher: Path,
|
| 440 |
+
artifact_dir: Path,
|
| 441 |
+
state_path: Path,
|
| 442 |
+
launcher_args: Sequence[str] = (),
|
| 443 |
+
question_timeout_sec: float = QUESTION_TIMEOUT_SEC,
|
| 444 |
+
tree_size_cap: int = TREE_SIZE_CAP,
|
| 445 |
+
startup_grace_sec: float = 0.0,
|
| 446 |
+
total_job_stop_sec: float | None = TOTAL_JOB_STOP_SEC,
|
| 447 |
+
poll_interval_sec: float = 0.25,
|
| 448 |
+
) -> int:
|
| 449 |
+
"""Run supervised wrapper segments; return 0 complete, 3 job stop, 4 failure."""
|
| 450 |
+
|
| 451 |
+
if question_timeout_sec <= 0:
|
| 452 |
+
raise SupervisorError("question timeout must be positive")
|
| 453 |
+
if tree_size_cap <= 0:
|
| 454 |
+
raise SupervisorError("tree size cap must be positive")
|
| 455 |
+
if startup_grace_sec < 0:
|
| 456 |
+
raise SupervisorError("startup grace must be nonnegative")
|
| 457 |
+
if total_job_stop_sec is not None and total_job_stop_sec <= 0:
|
| 458 |
+
raise SupervisorError("total job stop must be positive")
|
| 459 |
+
if poll_interval_sec <= 0:
|
| 460 |
+
raise SupervisorError("poll interval must be positive")
|
| 461 |
+
_protected_launcher_args(launcher_args)
|
| 462 |
+
|
| 463 |
+
questions_path = questions_path.expanduser().resolve()
|
| 464 |
+
results_path = results_path.expanduser().resolve()
|
| 465 |
+
checkpoint = checkpoint.expanduser().resolve()
|
| 466 |
+
launcher = launcher.expanduser().resolve()
|
| 467 |
+
artifact_dir = artifact_dir.expanduser().resolve()
|
| 468 |
+
state_path = state_path.expanduser().resolve()
|
| 469 |
+
for path, label in ((questions_path, "questions"), (launcher, "launcher")):
|
| 470 |
+
if not path.is_file():
|
| 471 |
+
raise SupervisorError(f"{label} file does not exist: {path}")
|
| 472 |
+
if not checkpoint.exists():
|
| 473 |
+
raise SupervisorError(f"checkpoint does not exist: {checkpoint}")
|
| 474 |
+
if len({questions_path, results_path, state_path}) != 3:
|
| 475 |
+
raise SupervisorError("questions, results, and state paths must be distinct")
|
| 476 |
+
|
| 477 |
+
questions = _read_questions(questions_path)
|
| 478 |
+
question_artifact = _artifact(questions_path)
|
| 479 |
+
launcher_artifact = _artifact(launcher)
|
| 480 |
+
supervisor_artifact = _artifact(Path(__file__).resolve())
|
| 481 |
+
policy = {
|
| 482 |
+
"hard_question_timeout_sec": float(question_timeout_sec),
|
| 483 |
+
"cooperative_tree_size_cap": int(tree_size_cap),
|
| 484 |
+
"startup_grace_sec": float(startup_grace_sec),
|
| 485 |
+
"total_job_stop_sec": (
|
| 486 |
+
float(total_job_stop_sec) if total_job_stop_sec is not None else None
|
| 487 |
+
),
|
| 488 |
+
"startup_accounting": (
|
| 489 |
+
"The first question clock begins when the wrapper manifest is first "
|
| 490 |
+
"observed, after launcher preflight and before model loading, so model "
|
| 491 |
+
"startup is counted. startup_grace_sec is added only to that first "
|
| 492 |
+
"question's allowance. Later clocks begin when the preceding JSONL "
|
| 493 |
+
"row is observed."
|
| 494 |
+
),
|
| 495 |
+
}
|
| 496 |
+
results_path.parent.mkdir(parents=True, exist_ok=True)
|
| 497 |
+
artifact_dir.mkdir(parents=True, exist_ok=True)
|
| 498 |
+
prefix = _validate_prefix(results_path, questions, require_terminated=True)
|
| 499 |
+
invocation_id = uuid.uuid4().hex
|
| 500 |
+
prior_state: dict[str, Any] | None = None
|
| 501 |
+
if state_path.is_file():
|
| 502 |
+
prior_state = _artifact(state_path)
|
| 503 |
+
try:
|
| 504 |
+
previous = json.loads(state_path.read_text(encoding="utf-8"))
|
| 505 |
+
if isinstance(previous, dict):
|
| 506 |
+
prior_state["status"] = previous.get("status")
|
| 507 |
+
prior_state["invocation_id"] = previous.get("invocation_id")
|
| 508 |
+
except (OSError, json.JSONDecodeError):
|
| 509 |
+
prior_state["status"] = "unreadable"
|
| 510 |
+
state: dict[str, Any] = {
|
| 511 |
+
"schema_version": SCHEMA_VERSION,
|
| 512 |
+
"status": "preparing",
|
| 513 |
+
"created_utc": _utc_now(),
|
| 514 |
+
"invocation_id": invocation_id,
|
| 515 |
+
"recovered_from_result_prefix_rows": len(prefix),
|
| 516 |
+
"prior_state": prior_state,
|
| 517 |
+
"artifacts": {
|
| 518 |
+
"questions": question_artifact,
|
| 519 |
+
"launcher": launcher_artifact,
|
| 520 |
+
"supervisor": supervisor_artifact,
|
| 521 |
+
"results_path": str(results_path),
|
| 522 |
+
"artifact_dir": str(artifact_dir),
|
| 523 |
+
},
|
| 524 |
+
"checkpoint": str(checkpoint),
|
| 525 |
+
"launcher_args": list(launcher_args),
|
| 526 |
+
"policy": policy,
|
| 527 |
+
"attempts": [],
|
| 528 |
+
}
|
| 529 |
+
_write_json_atomic(state_path, state)
|
| 530 |
+
|
| 531 |
+
job_started = time.monotonic()
|
| 532 |
+
try:
|
| 533 |
+
if len(prefix) == len(questions):
|
| 534 |
+
state["status"] = "succeeded"
|
| 535 |
+
state["completed_utc"] = _utc_now()
|
| 536 |
+
state["result_rows"] = len(prefix)
|
| 537 |
+
state["artifacts"]["results"] = _artifact(results_path)
|
| 538 |
+
_write_json_atomic(state_path, state)
|
| 539 |
+
return 0
|
| 540 |
+
|
| 541 |
+
while len(prefix) < len(questions):
|
| 542 |
+
attempt_number = len(state["attempts"]) + 1
|
| 543 |
+
attempt_id = f"{invocation_id}-{attempt_number:04d}-{uuid.uuid4().hex}"
|
| 544 |
+
stem = f"attempt-{attempt_id}"
|
| 545 |
+
wrapper_manifest = artifact_dir / f"{stem}.wrapper-manifest.json"
|
| 546 |
+
stdout_path = artifact_dir / f"{stem}.stdout.log"
|
| 547 |
+
stderr_path = artifact_dir / f"{stem}.stderr.log"
|
| 548 |
+
attempt_manifest = artifact_dir / f"{stem}.supervisor.json"
|
| 549 |
+
command = _launcher_command(
|
| 550 |
+
launcher=launcher,
|
| 551 |
+
questions=questions_path,
|
| 552 |
+
results=results_path,
|
| 553 |
+
checkpoint=checkpoint,
|
| 554 |
+
wrapper_manifest=wrapper_manifest,
|
| 555 |
+
launcher_args=launcher_args,
|
| 556 |
+
resume=bool(prefix),
|
| 557 |
+
question_timeout_sec=question_timeout_sec,
|
| 558 |
+
tree_size_cap=tree_size_cap,
|
| 559 |
+
)
|
| 560 |
+
rows_before = len(prefix)
|
| 561 |
+
started_utc = _utc_now()
|
| 562 |
+
attempt_started = time.monotonic()
|
| 563 |
+
first_pending = True
|
| 564 |
+
# The wrapper writes its manifest after launcher preflight and
|
| 565 |
+
# before model loading. Start the first question clock only then:
|
| 566 |
+
# preflight is excluded, while model startup is included.
|
| 567 |
+
current_started: float | None = None
|
| 568 |
+
termination = "unknown"
|
| 569 |
+
return_code: int | None = None
|
| 570 |
+
with stdout_path.open("xb") as stdout_stream, stderr_path.open("xb") as stderr_stream:
|
| 571 |
+
process = _popen_group(
|
| 572 |
+
command,
|
| 573 |
+
cwd=Path(__file__).resolve().parents[2],
|
| 574 |
+
stdout=stdout_stream,
|
| 575 |
+
stderr=stderr_stream,
|
| 576 |
+
)
|
| 577 |
+
while True:
|
| 578 |
+
now = time.monotonic()
|
| 579 |
+
if current_started is None and wrapper_manifest.is_file():
|
| 580 |
+
current_started = now
|
| 581 |
+
observed = _validate_prefix(
|
| 582 |
+
results_path, questions, require_terminated=False
|
| 583 |
+
)
|
| 584 |
+
if len(observed) > len(prefix):
|
| 585 |
+
prefix = observed
|
| 586 |
+
current_started = now
|
| 587 |
+
first_pending = False
|
| 588 |
+
if len(prefix) == len(questions):
|
| 589 |
+
# Let the wrapper finalize its own manifest cleanly.
|
| 590 |
+
try:
|
| 591 |
+
return_code = process.wait(timeout=30.0)
|
| 592 |
+
except subprocess.TimeoutExpired:
|
| 593 |
+
_kill_process_tree(process)
|
| 594 |
+
return_code = process.returncode
|
| 595 |
+
termination = (
|
| 596 |
+
"complete" if return_code == 0 else "launcher_exit"
|
| 597 |
+
)
|
| 598 |
+
break
|
| 599 |
+
|
| 600 |
+
return_code = process.poll()
|
| 601 |
+
if return_code is not None:
|
| 602 |
+
termination = "launcher_exit"
|
| 603 |
+
break
|
| 604 |
+
|
| 605 |
+
if total_job_stop_sec is not None and now - job_started >= total_job_stop_sec:
|
| 606 |
+
_kill_process_tree(process)
|
| 607 |
+
return_code = process.returncode
|
| 608 |
+
termination = "total_job_stop"
|
| 609 |
+
break
|
| 610 |
+
|
| 611 |
+
allowance = question_timeout_sec + (
|
| 612 |
+
startup_grace_sec if first_pending else 0.0
|
| 613 |
+
)
|
| 614 |
+
if (
|
| 615 |
+
current_started is not None
|
| 616 |
+
and now - current_started >= allowance
|
| 617 |
+
):
|
| 618 |
+
_kill_process_tree(process)
|
| 619 |
+
return_code = process.returncode
|
| 620 |
+
# The row could have completed in the kill race. Re-read
|
| 621 |
+
# before appending any synthetic result.
|
| 622 |
+
observed = _validate_prefix(
|
| 623 |
+
results_path, questions, require_terminated=True
|
| 624 |
+
)
|
| 625 |
+
if len(observed) == len(prefix):
|
| 626 |
+
pending = questions[len(prefix)]
|
| 627 |
+
timeout_execution_config = _timeout_execution_config(
|
| 628 |
+
wrapper_manifest,
|
| 629 |
+
invocation_id=invocation_id,
|
| 630 |
+
attempt_id=attempt_id,
|
| 631 |
+
)
|
| 632 |
+
_append_row(
|
| 633 |
+
results_path,
|
| 634 |
+
_timeout_row(
|
| 635 |
+
pending,
|
| 636 |
+
timeout_sec=question_timeout_sec,
|
| 637 |
+
observed_wall_sec=now - current_started,
|
| 638 |
+
attempt_number=attempt_number,
|
| 639 |
+
execution_config=timeout_execution_config,
|
| 640 |
+
tree_size_cap=tree_size_cap,
|
| 641 |
+
),
|
| 642 |
+
)
|
| 643 |
+
prefix = _validate_prefix(
|
| 644 |
+
results_path, questions, require_terminated=True
|
| 645 |
+
)
|
| 646 |
+
termination = "hard_question_timeout"
|
| 647 |
+
else:
|
| 648 |
+
prefix = observed
|
| 649 |
+
termination = "timeout_completion_race"
|
| 650 |
+
break
|
| 651 |
+
time.sleep(poll_interval_sec)
|
| 652 |
+
|
| 653 |
+
ended_utc = _utc_now()
|
| 654 |
+
frozen_wrapper_manifest = (
|
| 655 |
+
_artifact(wrapper_manifest) if wrapper_manifest.is_file() else None
|
| 656 |
+
)
|
| 657 |
+
attempt_record = {
|
| 658 |
+
"schema_version": ATTEMPT_SCHEMA_VERSION,
|
| 659 |
+
"attempt_number": attempt_number,
|
| 660 |
+
"attempt_id": attempt_id,
|
| 661 |
+
"invocation_id": invocation_id,
|
| 662 |
+
"started_utc": started_utc,
|
| 663 |
+
"ended_utc": ended_utc,
|
| 664 |
+
"wall_time_sec": time.monotonic() - attempt_started,
|
| 665 |
+
"rows_before": rows_before,
|
| 666 |
+
"rows_after": len(prefix),
|
| 667 |
+
"termination": termination,
|
| 668 |
+
"return_code": return_code,
|
| 669 |
+
"command": command,
|
| 670 |
+
"wrapper_manifest": frozen_wrapper_manifest,
|
| 671 |
+
"wrapper_manifest_path": str(wrapper_manifest),
|
| 672 |
+
"stdout_log": _artifact(stdout_path),
|
| 673 |
+
"stderr_log": _artifact(stderr_path),
|
| 674 |
+
}
|
| 675 |
+
_write_json_exclusive(attempt_manifest, attempt_record)
|
| 676 |
+
attempt_summary = dict(attempt_record)
|
| 677 |
+
attempt_summary["attempt_manifest"] = _artifact(attempt_manifest)
|
| 678 |
+
state["attempts"].append(attempt_summary)
|
| 679 |
+
state["status"] = (
|
| 680 |
+
"running" if termination != "total_job_stop" else "job_stopped"
|
| 681 |
+
)
|
| 682 |
+
state["result_rows"] = len(prefix)
|
| 683 |
+
state["updated_utc"] = ended_utc
|
| 684 |
+
if results_path.is_file():
|
| 685 |
+
state["artifacts"]["partial_results"] = _artifact(results_path)
|
| 686 |
+
_write_json_atomic(state_path, state)
|
| 687 |
+
if termination == "total_job_stop":
|
| 688 |
+
return 3
|
| 689 |
+
if termination == "launcher_exit" and return_code != 0:
|
| 690 |
+
state["status"] = "failed"
|
| 691 |
+
state["failure"] = (
|
| 692 |
+
f"launcher exited with code {return_code}; inspect immutable attempt logs"
|
| 693 |
+
)
|
| 694 |
+
_write_json_atomic(state_path, state)
|
| 695 |
+
return 4
|
| 696 |
+
if termination == "launcher_exit" and len(prefix) < len(questions):
|
| 697 |
+
state["status"] = "failed"
|
| 698 |
+
state["failure"] = "launcher exited successfully before producing all rows"
|
| 699 |
+
_write_json_atomic(state_path, state)
|
| 700 |
+
return 4
|
| 701 |
+
|
| 702 |
+
state["status"] = "succeeded"
|
| 703 |
+
state["completed_utc"] = _utc_now()
|
| 704 |
+
state["result_rows"] = len(prefix)
|
| 705 |
+
state["artifacts"].pop("partial_results", None)
|
| 706 |
+
state["artifacts"]["results"] = _artifact(results_path)
|
| 707 |
+
_write_json_atomic(state_path, state)
|
| 708 |
+
return 0
|
| 709 |
+
finally:
|
| 710 |
+
# This supervisor is deliberately lock-free. Exact-prefix validation and
|
| 711 |
+
# process-tree termination are its safety boundaries; callers must
|
| 712 |
+
# serialize supervisor invocations.
|
| 713 |
+
pass
|
| 714 |
+
|
| 715 |
+
|
| 716 |
+
def _parser() -> argparse.ArgumentParser:
|
| 717 |
+
parser = argparse.ArgumentParser(
|
| 718 |
+
description="Run v12 search with an external hard 1000-second question cap."
|
| 719 |
+
)
|
| 720 |
+
parser.add_argument("--questions", type=Path, required=True)
|
| 721 |
+
parser.add_argument("--results", type=Path, required=True)
|
| 722 |
+
parser.add_argument("--checkpoint", type=Path, required=True)
|
| 723 |
+
parser.add_argument("--launcher", type=Path, default=DEFAULT_LAUNCHER)
|
| 724 |
+
parser.add_argument("--artifact-dir", type=Path)
|
| 725 |
+
parser.add_argument("--state", type=Path)
|
| 726 |
+
parser.add_argument("--startup-grace-sec", type=float, default=0.0)
|
| 727 |
+
parser.add_argument(
|
| 728 |
+
"--question-timeout-sec", type=float, default=QUESTION_TIMEOUT_SEC
|
| 729 |
+
)
|
| 730 |
+
parser.add_argument("--tree-size-cap", type=int, default=TREE_SIZE_CAP)
|
| 731 |
+
parser.add_argument(
|
| 732 |
+
"--total-job-stop-sec", type=float, default=TOTAL_JOB_STOP_SEC
|
| 733 |
+
)
|
| 734 |
+
parser.add_argument("--poll-interval-sec", type=float, default=0.25)
|
| 735 |
+
parser.add_argument(
|
| 736 |
+
"launcher_args",
|
| 737 |
+
nargs=argparse.REMAINDER,
|
| 738 |
+
help="Arguments passed to run_search_benchmark.py after a standalone --.",
|
| 739 |
+
)
|
| 740 |
+
return parser
|
| 741 |
+
|
| 742 |
+
|
| 743 |
+
def main(argv: Sequence[str] | None = None) -> int:
|
| 744 |
+
args = _parser().parse_args(argv)
|
| 745 |
+
launcher_args = list(args.launcher_args)
|
| 746 |
+
if launcher_args[:1] == ["--"]:
|
| 747 |
+
launcher_args = launcher_args[1:]
|
| 748 |
+
results = args.results.expanduser().resolve()
|
| 749 |
+
artifact_dir = (
|
| 750 |
+
args.artifact_dir.expanduser().resolve()
|
| 751 |
+
if args.artifact_dir
|
| 752 |
+
else results.with_name(results.name + ".hard-supervisor-attempts")
|
| 753 |
+
)
|
| 754 |
+
state = (
|
| 755 |
+
args.state.expanduser().resolve()
|
| 756 |
+
if args.state
|
| 757 |
+
else results.with_name(results.name + ".hard-supervisor-state.json")
|
| 758 |
+
)
|
| 759 |
+
try:
|
| 760 |
+
return run_supervisor(
|
| 761 |
+
questions_path=args.questions,
|
| 762 |
+
results_path=results,
|
| 763 |
+
checkpoint=args.checkpoint,
|
| 764 |
+
launcher=args.launcher,
|
| 765 |
+
artifact_dir=artifact_dir,
|
| 766 |
+
state_path=state,
|
| 767 |
+
launcher_args=launcher_args,
|
| 768 |
+
question_timeout_sec=args.question_timeout_sec,
|
| 769 |
+
tree_size_cap=args.tree_size_cap,
|
| 770 |
+
startup_grace_sec=args.startup_grace_sec,
|
| 771 |
+
total_job_stop_sec=args.total_job_stop_sec,
|
| 772 |
+
poll_interval_sec=args.poll_interval_sec,
|
| 773 |
+
)
|
| 774 |
+
except SupervisorError as exc:
|
| 775 |
+
print(f"error: {exc}", file=sys.stderr)
|
| 776 |
+
return 2
|
| 777 |
+
|
| 778 |
+
|
| 779 |
+
if __name__ == "__main__":
|
| 780 |
+
raise SystemExit(main())
|
mce220_runner/grpo_pipeline/__init__.py
ADDED
|
File without changes
|
mce220_runner/grpo_pipeline/modules/__init__.py
ADDED
|
File without changes
|
mce220_runner/grpo_pipeline/modules/llm_api_interface.py
ADDED
|
@@ -0,0 +1,427 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Local LLM interface for beam_search.
|
| 3 |
+
|
| 4 |
+
This replaces the previous OpenRouter/API-based implementation.
|
| 5 |
+
|
| 6 |
+
Public API:
|
| 7 |
+
- LLM(prompt, Gamma=None, M_current=None, k=5, adapter_path=None, use_dummy=False, **gen_kwargs) -> List[str]
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import json
|
| 13 |
+
import os
|
| 14 |
+
import re
|
| 15 |
+
import sys
|
| 16 |
+
import hashlib
|
| 17 |
+
from dataclasses import dataclass
|
| 18 |
+
from typing import List, Optional, Any
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
_MODEL_CACHE: dict[tuple[str, str], Any] = {}
|
| 22 |
+
_DUMMY_CALL_IDX: int = 0
|
| 23 |
+
_LAST_LLM_STATS: dict[str, Any] = {}
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def get_last_llm_stats(*, reset: bool = False) -> dict[str, Any]:
|
| 27 |
+
stats = dict(_LAST_LLM_STATS)
|
| 28 |
+
if reset:
|
| 29 |
+
_LAST_LLM_STATS.clear()
|
| 30 |
+
return stats
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def _strip_code_fences(text: str) -> str:
|
| 34 |
+
text = re.sub(r"^```[a-zA-Z0-9_-]*\s*", "", text.strip())
|
| 35 |
+
text = re.sub(r"\s*```$", "", text.strip())
|
| 36 |
+
return text.strip()
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def _extract_inequalities(text: str, k: int) -> List[str]:
|
| 40 |
+
"""
|
| 41 |
+
Extract up to k candidate inequality strings from model output.
|
| 42 |
+
Accepts numbered lists, bullets, code fences, and JSON arrays.
|
| 43 |
+
"""
|
| 44 |
+
text = _strip_code_fences(text)
|
| 45 |
+
|
| 46 |
+
# Try JSON first
|
| 47 |
+
try:
|
| 48 |
+
obj = json.loads(text)
|
| 49 |
+
if isinstance(obj, list):
|
| 50 |
+
out = []
|
| 51 |
+
for x in obj:
|
| 52 |
+
if isinstance(x, str):
|
| 53 |
+
s = x.strip()
|
| 54 |
+
if s:
|
| 55 |
+
out.append(s)
|
| 56 |
+
if out:
|
| 57 |
+
return out[:k]
|
| 58 |
+
except Exception:
|
| 59 |
+
pass
|
| 60 |
+
|
| 61 |
+
# Otherwise parse lines
|
| 62 |
+
lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
|
| 63 |
+
cands: List[str] = []
|
| 64 |
+
for ln in lines:
|
| 65 |
+
# remove common list prefixes: "1.", "1)", "*", and "-" ONLY if followed by space (bullet point)
|
| 66 |
+
# We must NOT strip a leading "-" if it's followed by a digit or "H("/"I(", as it's likely a negative coefficient.
|
| 67 |
+
ln = re.sub(r"^\s*(?:[*]|(\d+)[\.\)]|-(?:\s+|$))\s*", "", ln).strip()
|
| 68 |
+
ln = ln.strip("`").strip()
|
| 69 |
+
if not ln:
|
| 70 |
+
continue
|
| 71 |
+
|
| 72 |
+
# keep only plausible math constraints or explicit "wrong" signal
|
| 73 |
+
if ("H(" in ln) or ("I(" in ln):
|
| 74 |
+
cands.append(ln)
|
| 75 |
+
elif "wrong" in ln.lower():
|
| 76 |
+
cands.append("wrong")
|
| 77 |
+
|
| 78 |
+
if len(cands) >= k:
|
| 79 |
+
break
|
| 80 |
+
|
| 81 |
+
return cands[:k]
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
def _read_base_model_from_adapter(adapter_path: str) -> Optional[str]:
|
| 85 |
+
cfg_path = os.path.join(adapter_path, "adapter_config.json")
|
| 86 |
+
try:
|
| 87 |
+
with open(cfg_path, "r", encoding="utf-8") as f:
|
| 88 |
+
cfg = json.load(f)
|
| 89 |
+
base = cfg.get("base_model_name_or_path")
|
| 90 |
+
return base if isinstance(base, str) and base else None
|
| 91 |
+
except Exception:
|
| 92 |
+
return None
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def _load_tokenizer_compat(tokenizer_src: str):
|
| 96 |
+
from transformers import AutoTokenizer
|
| 97 |
+
|
| 98 |
+
extra_kwargs = {}
|
| 99 |
+
cfg_path = os.path.join(tokenizer_src, "tokenizer_config.json")
|
| 100 |
+
if os.path.isdir(tokenizer_src) and os.path.exists(cfg_path):
|
| 101 |
+
try:
|
| 102 |
+
with open(cfg_path, "r", encoding="utf-8") as f:
|
| 103 |
+
cfg = json.load(f)
|
| 104 |
+
if isinstance(cfg.get("extra_special_tokens"), list):
|
| 105 |
+
sys.stderr.write(
|
| 106 |
+
f"[llm] Sanitizing tokenizer extra_special_tokens for: {tokenizer_src}\n"
|
| 107 |
+
)
|
| 108 |
+
extra_kwargs["extra_special_tokens"] = {}
|
| 109 |
+
except Exception:
|
| 110 |
+
pass
|
| 111 |
+
|
| 112 |
+
return AutoTokenizer.from_pretrained(
|
| 113 |
+
tokenizer_src,
|
| 114 |
+
trust_remote_code=True,
|
| 115 |
+
**extra_kwargs,
|
| 116 |
+
)
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
def _load_local_model(
|
| 120 |
+
adapter_path: Optional[str] = None,
|
| 121 |
+
*,
|
| 122 |
+
base_model: Optional[str] = None,
|
| 123 |
+
device_map: str = "auto",
|
| 124 |
+
):
|
| 125 |
+
"""
|
| 126 |
+
Load HF base model, optionally with a PEFT adapter.
|
| 127 |
+
Caches model/tokenizer to avoid re-loading each call.
|
| 128 |
+
"""
|
| 129 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 130 |
+
import torch
|
| 131 |
+
|
| 132 |
+
# Only treat as adapter if path is truthy AND contains adapter_config.json
|
| 133 |
+
adapter_abs = ""
|
| 134 |
+
if adapter_path and adapter_path.strip():
|
| 135 |
+
abs_p = os.path.abspath(adapter_path.strip())
|
| 136 |
+
if os.path.exists(os.path.join(abs_p, "adapter_config.json")):
|
| 137 |
+
adapter_abs = abs_p
|
| 138 |
+
else:
|
| 139 |
+
sys.stderr.write(f"[llm] Warning: adapter_path '{adapter_path}' provided but no adapter_config.json found in {abs_p}. Ignoring adapter.\n")
|
| 140 |
+
|
| 141 |
+
resolved_base = (
|
| 142 |
+
base_model
|
| 143 |
+
or os.environ.get("QWEN_BASE_MODEL")
|
| 144 |
+
or (_read_base_model_from_adapter(adapter_abs) if adapter_abs else None)
|
| 145 |
+
or "Qwen/Qwen3-0.6B"
|
| 146 |
+
)
|
| 147 |
+
|
| 148 |
+
weights_path = os.environ.get("QWEN_WEIGHTS_PATH", "").strip()
|
| 149 |
+
cache_key = (adapter_abs, resolved_base, weights_path)
|
| 150 |
+
if cache_key in _MODEL_CACHE:
|
| 151 |
+
return _MODEL_CACHE[cache_key]
|
| 152 |
+
|
| 153 |
+
if weights_path:
|
| 154 |
+
from safetensors.torch import load_file
|
| 155 |
+
sys.stderr.write(f"[llm] Loading base arch from: {resolved_base}\n")
|
| 156 |
+
sys.stderr.write(f"[llm] Overriding weights from: {weights_path}\n")
|
| 157 |
+
sys.stderr.flush()
|
| 158 |
+
tokenizer = _load_tokenizer_compat(resolved_base)
|
| 159 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 160 |
+
resolved_base,
|
| 161 |
+
torch_dtype=torch.bfloat16 if torch.cuda.is_available() else "auto",
|
| 162 |
+
device_map=device_map,
|
| 163 |
+
trust_remote_code=True,
|
| 164 |
+
)
|
| 165 |
+
ckpt = load_file(os.path.join(weights_path, "model.safetensors"), device="cpu")
|
| 166 |
+
# The in-memory model state_dict includes lm_head.weight (tied to embed_tokens.weight).
|
| 167 |
+
# The GRPO checkpoint also has lm_head.weight serialized by DeepSpeed — load all keys.
|
| 168 |
+
model.load_state_dict(ckpt, strict=True)
|
| 169 |
+
model.eval()
|
| 170 |
+
_MODEL_CACHE[cache_key] = (model, tokenizer)
|
| 171 |
+
return model, tokenizer
|
| 172 |
+
|
| 173 |
+
if adapter_abs:
|
| 174 |
+
sys.stderr.write(f"[llm] Loading model: {resolved_base}\n")
|
| 175 |
+
sys.stderr.write(f"[llm] Loading adapter: {adapter_abs}\n")
|
| 176 |
+
else:
|
| 177 |
+
sys.stderr.write(f"[llm] Loading base model only: {resolved_base}\n")
|
| 178 |
+
sys.stderr.flush()
|
| 179 |
+
|
| 180 |
+
# Tokenizer should come from the adapter if present (it might include special tokens),
|
| 181 |
+
# otherwise from the base model.
|
| 182 |
+
tok_src = adapter_abs or resolved_base
|
| 183 |
+
tokenizer = _load_tokenizer_compat(tok_src)
|
| 184 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 185 |
+
resolved_base,
|
| 186 |
+
torch_dtype=torch.bfloat16 if torch.cuda.is_available() else "auto",
|
| 187 |
+
device_map=device_map,
|
| 188 |
+
trust_remote_code=True,
|
| 189 |
+
)
|
| 190 |
+
|
| 191 |
+
if adapter_abs:
|
| 192 |
+
from peft import PeftModel
|
| 193 |
+
model = PeftModel.from_pretrained(model, adapter_abs)
|
| 194 |
+
|
| 195 |
+
model.eval()
|
| 196 |
+
|
| 197 |
+
_MODEL_CACHE[cache_key] = (model, tokenizer)
|
| 198 |
+
return model, tokenizer
|
| 199 |
+
|
| 200 |
+
|
| 201 |
+
def _build_alpaca_prompt(input_text: str) -> str:
|
| 202 |
+
# Mirror the exact formatting observed in ft_using_Axolotl_1/last_run_prepared/* decoding
|
| 203 |
+
instruction = "You are proving the entropy inequality. Follow the instructions carefully."
|
| 204 |
+
preamble = (
|
| 205 |
+
"Below is an instruction that describes a task, paired with an input that provides further context. "
|
| 206 |
+
"Write a response that appropriately completes the request."
|
| 207 |
+
)
|
| 208 |
+
return (
|
| 209 |
+
f"{preamble}\n\n"
|
| 210 |
+
f"### Instruction:\n{instruction}\n\n"
|
| 211 |
+
f"### Input:\n{input_text}\n\n\n\n"
|
| 212 |
+
f"### Response:\n"
|
| 213 |
+
)
|
| 214 |
+
|
| 215 |
+
|
| 216 |
+
def LLM(
|
| 217 |
+
prompt: str,
|
| 218 |
+
Gamma=None, # kept for backward compatibility (ignored for local inference)
|
| 219 |
+
M_current=None, # kept for backward compatibility (ignored for local inference)
|
| 220 |
+
k: int = 5,
|
| 221 |
+
adapter_path: Optional[str] = None,
|
| 222 |
+
use_dummy: bool = False,
|
| 223 |
+
**gen_kwargs,
|
| 224 |
+
) -> List[str]:
|
| 225 |
+
"""
|
| 226 |
+
Local inference to propose inequalities.
|
| 227 |
+
|
| 228 |
+
Args:
|
| 229 |
+
prompt: natural language prompt
|
| 230 |
+
k: number of inequalities requested
|
| 231 |
+
adapter_path: local path to PEFT adapter directory (required unless use_dummy=True)
|
| 232 |
+
use_dummy: if True, returns a small hardcoded set for testing
|
| 233 |
+
gen_kwargs: passed to model.generate (e.g. max_new_tokens, temperature)
|
| 234 |
+
"""
|
| 235 |
+
global _LAST_LLM_STATS
|
| 236 |
+
|
| 237 |
+
if use_dummy:
|
| 238 |
+
# Dummy mode should still generate *varied* valid inequalities; otherwise tree_search
|
| 239 |
+
# will keep adding the same single inequality and will never be able to prove even
|
| 240 |
+
# simple Shannon-type targets.
|
| 241 |
+
#
|
| 242 |
+
# We deterministically vary outputs based on the prompt content so different nodes
|
| 243 |
+
# (with different CURRENT INEQUALITIES M) receive different proposals.
|
| 244 |
+
def _parse_n_from_prompt(p: str) -> int:
|
| 245 |
+
# Try old format first
|
| 246 |
+
m = re.search(r"NUMBER OF VARIABLES:\s*\n(\d+)\b", p)
|
| 247 |
+
if m:
|
| 248 |
+
try:
|
| 249 |
+
return max(1, int(m.group(1)))
|
| 250 |
+
except Exception:
|
| 251 |
+
pass
|
| 252 |
+
|
| 253 |
+
# Fallback: Extract max index from X_{...}
|
| 254 |
+
idxs = []
|
| 255 |
+
for inside in re.findall(r"X_\{([^}]+)\}", p):
|
| 256 |
+
parts = [part.strip() for part in inside.split(",") if part.strip()]
|
| 257 |
+
for part in parts:
|
| 258 |
+
if part.isdigit():
|
| 259 |
+
idxs.append(int(part))
|
| 260 |
+
if idxs:
|
| 261 |
+
return max(idxs)
|
| 262 |
+
return 3
|
| 263 |
+
|
| 264 |
+
def _parse_allowed_range_from_prompt(p: str) -> Optional[tuple[int, int]]:
|
| 265 |
+
# Matches: "range 4..8" in IMPORTANT CONSTRAINT section.
|
| 266 |
+
m = re.search(r"range\s+(\d+)\.\.(\d+)", p)
|
| 267 |
+
if m:
|
| 268 |
+
try:
|
| 269 |
+
lo = int(m.group(1))
|
| 270 |
+
hi = int(m.group(2))
|
| 271 |
+
if lo > hi:
|
| 272 |
+
lo, hi = hi, lo
|
| 273 |
+
return lo, hi
|
| 274 |
+
except Exception:
|
| 275 |
+
pass
|
| 276 |
+
|
| 277 |
+
# Fallback: return (1, n)
|
| 278 |
+
return None
|
| 279 |
+
|
| 280 |
+
def _dummy_pool(n: int, lo: int, hi: int) -> List[str]:
|
| 281 |
+
cands: List[str] = []
|
| 282 |
+
lo = max(1, int(lo))
|
| 283 |
+
hi = min(int(n), int(hi))
|
| 284 |
+
if lo > hi:
|
| 285 |
+
lo, hi = 1, int(n)
|
| 286 |
+
vars_list = list(range(lo, hi + 1))
|
| 287 |
+
# Non-negativity
|
| 288 |
+
for i in vars_list:
|
| 289 |
+
cands.append(f"H(X_{{{i}}})>=0")
|
| 290 |
+
# Mutual information (unconditional)
|
| 291 |
+
for ii, i in enumerate(vars_list):
|
| 292 |
+
for j in vars_list[ii + 1 :]:
|
| 293 |
+
cands.append(f"I(X_{{{i}}};X_{{{j}}})>=0")
|
| 294 |
+
# Conditional mutual information with a single conditioning variable
|
| 295 |
+
if len(vars_list) >= 3:
|
| 296 |
+
for ii, i in enumerate(vars_list):
|
| 297 |
+
for j in vars_list[ii + 1 :]:
|
| 298 |
+
for c in vars_list:
|
| 299 |
+
if c != i and c != j:
|
| 300 |
+
cands.append(f"I(X_{{{i}}};X_{{{j}}}|X_{{{c}}})>=0")
|
| 301 |
+
# Conditional entropy with a single conditioning variable
|
| 302 |
+
for i in vars_list:
|
| 303 |
+
for j in vars_list:
|
| 304 |
+
if j != i:
|
| 305 |
+
cands.append(f"H(X_{{{i}}}|X_{{{j}}})>=0")
|
| 306 |
+
return cands
|
| 307 |
+
|
| 308 |
+
global _DUMMY_CALL_IDX
|
| 309 |
+
_DUMMY_CALL_IDX += 1
|
| 310 |
+
|
| 311 |
+
n = _parse_n_from_prompt(prompt)
|
| 312 |
+
rng = _parse_allowed_range_from_prompt(prompt) or (1, n)
|
| 313 |
+
pool = _dummy_pool(n, rng[0], rng[1])
|
| 314 |
+
# Stable seed from prompt content + call index, so repeated calls with the same
|
| 315 |
+
# prompt still yield different proposals (needed for rollout diversity).
|
| 316 |
+
seed_material = f"{prompt}\n[DUMMY_CALL_IDX]={_DUMMY_CALL_IDX}".encode("utf-8")
|
| 317 |
+
seed = int.from_bytes(hashlib.md5(seed_material).digest()[:8], "big", signed=False)
|
| 318 |
+
import random
|
| 319 |
+
|
| 320 |
+
rng = random.Random(seed)
|
| 321 |
+
rng.shuffle(pool)
|
| 322 |
+
out = pool[: max(0, int(k))]
|
| 323 |
+
if os.environ.get("QWEN_PRINT_OUTPUT", "0") == "1":
|
| 324 |
+
target_match = re.search(r"TARGET:\s*\n(.*?)\n\n", prompt, re.DOTALL)
|
| 325 |
+
target_val = target_match.group(1).strip() if target_match else "unknown"
|
| 326 |
+
sys.stderr.write(f"[llm] Target: {target_val}\n")
|
| 327 |
+
sys.stderr.write(f" Dummy Extracted: {out}\n")
|
| 328 |
+
sys.stderr.flush()
|
| 329 |
+
_LAST_LLM_STATS = {
|
| 330 |
+
"prompt_tokens": 0,
|
| 331 |
+
"generated_tokens_total": 0,
|
| 332 |
+
"generated_tokens_per_sequence": [0 for _ in out],
|
| 333 |
+
"num_sequences": len(out),
|
| 334 |
+
"extracted_candidates": len(out),
|
| 335 |
+
}
|
| 336 |
+
return out
|
| 337 |
+
|
| 338 |
+
# Use raw prompt when requested (matches alphabet SFT format).
|
| 339 |
+
if os.environ.get("QWEN_RAW_PROMPT", "0") == "1":
|
| 340 |
+
constrained_prompt = prompt
|
| 341 |
+
else:
|
| 342 |
+
# Fully adapt to the SFT training format: Alpaca template + caller-provided input block.
|
| 343 |
+
constrained_prompt = _build_alpaca_prompt(prompt.strip())
|
| 344 |
+
if os.environ.get("QWEN_PRINT_PROMPT", "0") == "1":
|
| 345 |
+
sys.stderr.write("[llm] Full prompt:\n")
|
| 346 |
+
sys.stderr.write(constrained_prompt)
|
| 347 |
+
sys.stderr.write("\n")
|
| 348 |
+
sys.stderr.flush()
|
| 349 |
+
|
| 350 |
+
# If adapter_path is omitted, we fall back to the baseline model (no PEFT).
|
| 351 |
+
model, tokenizer = _load_local_model(adapter_path=adapter_path)
|
| 352 |
+
|
| 353 |
+
import torch
|
| 354 |
+
|
| 355 |
+
if tokenizer.pad_token_id is None:
|
| 356 |
+
tokenizer.pad_token_id = tokenizer.eos_token_id
|
| 357 |
+
|
| 358 |
+
inputs = tokenizer(constrained_prompt, return_tensors="pt")
|
| 359 |
+
inputs = {k: v.to(model.device) for k, v in inputs.items()}
|
| 360 |
+
# Print prompt tokens alongside prompt for debugging alignment
|
| 361 |
+
|
| 362 |
+
default_kwargs = dict(
|
| 363 |
+
max_new_tokens=int(os.environ.get("QWEN_MAX_NEW_TOKENS", "512")),
|
| 364 |
+
# max_time=float(os.environ.get("QWEN_MAX_TIME", "40")),
|
| 365 |
+
do_sample=True,
|
| 366 |
+
temperature=float(os.environ.get("QWEN_TEMPERATURE", "0.7")),
|
| 367 |
+
top_p=float(os.environ.get("QWEN_TOP_P", "0.9")),
|
| 368 |
+
repetition_penalty=1.05,
|
| 369 |
+
pad_token_id=tokenizer.eos_token_id,
|
| 370 |
+
eos_token_id=tokenizer.eos_token_id,
|
| 371 |
+
)
|
| 372 |
+
default_kwargs.update(gen_kwargs or {})
|
| 373 |
+
|
| 374 |
+
with torch.inference_mode():
|
| 375 |
+
out = model.generate(
|
| 376 |
+
input_ids=inputs["input_ids"],
|
| 377 |
+
attention_mask=inputs.get("attention_mask", None),
|
| 378 |
+
**default_kwargs,
|
| 379 |
+
)
|
| 380 |
+
|
| 381 |
+
# Decode only the newly generated part when possible.
|
| 382 |
+
# If num_return_sequences > 1, generate() returns multiple sequences.
|
| 383 |
+
prompt_len = inputs["input_ids"].shape[-1]
|
| 384 |
+
cands: List[str] = []
|
| 385 |
+
decoded_texts: List[str] = []
|
| 386 |
+
|
| 387 |
+
if hasattr(out, "shape") and len(out.shape) >= 2 and out.shape[0] > 1:
|
| 388 |
+
# One inequality per returned sequence (preferred for "GPU batch generation")
|
| 389 |
+
generated_token_lengths: List[int] = []
|
| 390 |
+
for i in range(out.shape[0]):
|
| 391 |
+
gen_ids = out[i][prompt_len:]
|
| 392 |
+
generated_token_lengths.append(int(gen_ids.shape[-1]))
|
| 393 |
+
text_i = tokenizer.decode(gen_ids, skip_special_tokens=True).strip()
|
| 394 |
+
decoded_texts.append(text_i)
|
| 395 |
+
one = _extract_inequalities(text_i, k=1)
|
| 396 |
+
if one:
|
| 397 |
+
cands.append(one[0])
|
| 398 |
+
if len(cands) >= k:
|
| 399 |
+
# Keep decoding for printing if needed, but we have enough candidates
|
| 400 |
+
pass
|
| 401 |
+
else:
|
| 402 |
+
gen_ids = out[0][prompt_len:]
|
| 403 |
+
generated_token_lengths = [int(gen_ids.shape[-1])]
|
| 404 |
+
text = tokenizer.decode(gen_ids, skip_special_tokens=True).strip()
|
| 405 |
+
decoded_texts.append(text)
|
| 406 |
+
cands = _extract_inequalities(text, k=k)
|
| 407 |
+
|
| 408 |
+
_LAST_LLM_STATS = {
|
| 409 |
+
"prompt_tokens": int(prompt_len),
|
| 410 |
+
"generated_tokens_total": int(sum(generated_token_lengths)),
|
| 411 |
+
"generated_tokens_per_sequence": generated_token_lengths,
|
| 412 |
+
"num_sequences": len(generated_token_lengths),
|
| 413 |
+
"extracted_candidates": len(cands),
|
| 414 |
+
}
|
| 415 |
+
|
| 416 |
+
if os.environ.get("QWEN_PRINT_OUTPUT", "0") == "1":
|
| 417 |
+
# Simplified printing: only target, generated, and extracted.
|
| 418 |
+
target_match = re.search(r"TARGET:\s*\n(.*?)\n\n", prompt, re.DOTALL)
|
| 419 |
+
target_val = target_match.group(1).strip() if target_match else "unknown"
|
| 420 |
+
sys.stderr.write(f"[llm] Target: {target_val}\n")
|
| 421 |
+
for i, txt in enumerate(decoded_texts):
|
| 422 |
+
# Only print first line of generated text if it's long? No, user said generated inequality.
|
| 423 |
+
# Usually model outputs just the inequality.
|
| 424 |
+
sys.stderr.write(f" Gen {i}: {txt}\n")
|
| 425 |
+
sys.stderr.write(f" Extracted: {cands}\n")
|
| 426 |
+
sys.stderr.flush()
|
| 427 |
+
return cands
|
mce220_runner/requirements.txt
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Install a CUDA-enabled PyTorch wheel suitable for the host before this file if needed.
|
| 2 |
+
torch==2.3.1
|
| 3 |
+
transformers==4.55.2
|
| 4 |
+
peft==0.17.0
|
| 5 |
+
accelerate==1.10.0
|
| 6 |
+
huggingface_hub==0.36.2
|
| 7 |
+
safetensors==0.4.3
|
| 8 |
+
numpy==1.26.4
|
| 9 |
+
scipy==1.13.1
|
| 10 |
+
sympy==1.12.1
|
mce220_runner/run_mce220.sh
ADDED
|
@@ -0,0 +1,182 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
set -euo pipefail
|
| 3 |
+
|
| 4 |
+
usage() {
|
| 5 |
+
cat <<'EOF'
|
| 6 |
+
Usage: run_mce220.sh {dry-run|smoke|full} [--gpu-ids LIST] [--workers N] [--output-dir DIR]
|
| 7 |
+
|
| 8 |
+
Modes:
|
| 9 |
+
dry-run Verify all downloaded assets and print the frozen 220-row command.
|
| 10 |
+
smoke Evaluate the first question with one GPU and reduced search settings.
|
| 11 |
+
full Evaluate all 220 questions under the hard-timeout supervisor.
|
| 12 |
+
EOF
|
| 13 |
+
}
|
| 14 |
+
|
| 15 |
+
if [[ $# -lt 1 ]]; then
|
| 16 |
+
usage >&2
|
| 17 |
+
exit 2
|
| 18 |
+
fi
|
| 19 |
+
|
| 20 |
+
MODE="$1"
|
| 21 |
+
shift
|
| 22 |
+
case "$MODE" in
|
| 23 |
+
dry-run|smoke|full) ;;
|
| 24 |
+
*) usage >&2; exit 2 ;;
|
| 25 |
+
esac
|
| 26 |
+
|
| 27 |
+
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 28 |
+
PYTHON_BIN="${PYTHON:-python3}"
|
| 29 |
+
ASSET_DIR="${MCE220_ASSET_DIR:-$SCRIPT_DIR/assets}"
|
| 30 |
+
GPU_IDS="0,1,2,3,4,5,6,7"
|
| 31 |
+
WORKERS=""
|
| 32 |
+
OUTPUT_DIR=""
|
| 33 |
+
|
| 34 |
+
while [[ $# -gt 0 ]]; do
|
| 35 |
+
case "$1" in
|
| 36 |
+
--gpu-ids)
|
| 37 |
+
[[ $# -ge 2 ]] || { echo "missing value for --gpu-ids" >&2; exit 2; }
|
| 38 |
+
GPU_IDS="$2"; shift 2 ;;
|
| 39 |
+
--workers)
|
| 40 |
+
[[ $# -ge 2 ]] || { echo "missing value for --workers" >&2; exit 2; }
|
| 41 |
+
WORKERS="$2"; shift 2 ;;
|
| 42 |
+
--output-dir)
|
| 43 |
+
[[ $# -ge 2 ]] || { echo "missing value for --output-dir" >&2; exit 2; }
|
| 44 |
+
OUTPUT_DIR="$2"; shift 2 ;;
|
| 45 |
+
-h|--help)
|
| 46 |
+
usage; exit 0 ;;
|
| 47 |
+
*) echo "unknown option: $1" >&2; usage >&2; exit 2 ;;
|
| 48 |
+
esac
|
| 49 |
+
done
|
| 50 |
+
|
| 51 |
+
if [[ -z "$WORKERS" ]]; then
|
| 52 |
+
if [[ "$MODE" == "smoke" ]]; then WORKERS=1; else WORKERS=8; fi
|
| 53 |
+
fi
|
| 54 |
+
[[ "$WORKERS" =~ ^[1-9][0-9]*$ ]] || {
|
| 55 |
+
echo "--workers must be a positive integer" >&2
|
| 56 |
+
exit 2
|
| 57 |
+
}
|
| 58 |
+
if [[ "$MODE" == "smoke" && "$WORKERS" != "1" ]]; then
|
| 59 |
+
echo "smoke mode is intentionally restricted to --workers 1" >&2
|
| 60 |
+
exit 2
|
| 61 |
+
fi
|
| 62 |
+
|
| 63 |
+
if [[ -z "$OUTPUT_DIR" ]]; then
|
| 64 |
+
OUTPUT_DIR="$SCRIPT_DIR/runs/$MODE"
|
| 65 |
+
fi
|
| 66 |
+
mkdir -p -- "$OUTPUT_DIR"
|
| 67 |
+
|
| 68 |
+
QUESTIONS="$SCRIPT_DIR/dataset/data/questions.jsonl"
|
| 69 |
+
CHECKPOINT="$ASSET_DIR/model-repo/qwen-06b-sft-4type-old-format-lora-dataset-a-4096-balanced-higher-maxm/checkpoint-9534"
|
| 70 |
+
BASE_MODEL="$ASSET_DIR/qwen3-0.6b"
|
| 71 |
+
RUNNER="$SCRIPT_DIR/beam_search_model2/run_on_window_shear_jsonl_v12.py"
|
| 72 |
+
WRAPPER="$SCRIPT_DIR/experiments/reviewer_response/run_search_benchmark.py"
|
| 73 |
+
SUPERVISOR="$SCRIPT_DIR/experiments/reviewer_response/supervise_search_benchmark.py"
|
| 74 |
+
|
| 75 |
+
"$PYTHON_BIN" "$SCRIPT_DIR/download_assets.py" \
|
| 76 |
+
--asset-dir "$ASSET_DIR" \
|
| 77 |
+
--verify-only
|
| 78 |
+
|
| 79 |
+
common_full_args=(
|
| 80 |
+
--questions "$QUESTIONS"
|
| 81 |
+
--checkpoint "$CHECKPOINT"
|
| 82 |
+
--base-model "$BASE_MODEL"
|
| 83 |
+
--runner "$RUNNER"
|
| 84 |
+
--gpu-ids "$GPU_IDS"
|
| 85 |
+
--workers "$WORKERS"
|
| 86 |
+
--beam-size 32
|
| 87 |
+
--rollouts 6
|
| 88 |
+
--max-depth 24
|
| 89 |
+
--seed 42
|
| 90 |
+
--temperature 1.2
|
| 91 |
+
--max-new-tokens 2048
|
| 92 |
+
--generation-max-time 20
|
| 93 |
+
--max-vars-for-lp 0
|
| 94 |
+
--lp-workers "$WORKERS"
|
| 95 |
+
--min-n 10
|
| 96 |
+
--max-n 20
|
| 97 |
+
--prompt-format canonical
|
| 98 |
+
--normalize-candidates
|
| 99 |
+
--raw-prompt
|
| 100 |
+
--print-llm
|
| 101 |
+
)
|
| 102 |
+
|
| 103 |
+
case "$MODE" in
|
| 104 |
+
dry-run)
|
| 105 |
+
stamp="$(date -u +%Y%m%dT%H%M%SZ)-$$"
|
| 106 |
+
"$PYTHON_BIN" "$WRAPPER" \
|
| 107 |
+
"${common_full_args[@]}" \
|
| 108 |
+
--results "$OUTPUT_DIR/mce220-dry-run-results.jsonl" \
|
| 109 |
+
--manifest "$OUTPUT_DIR/mce220-dry-run-$stamp.manifest.json" \
|
| 110 |
+
--sample-max-time-sec 1000 \
|
| 111 |
+
--hard-question-timeout-sec 1000 \
|
| 112 |
+
--sample-max-tree-size 6000 \
|
| 113 |
+
--dry-run
|
| 114 |
+
;;
|
| 115 |
+
smoke)
|
| 116 |
+
first_gpu="${GPU_IDS%%,*}"
|
| 117 |
+
smoke_questions="$OUTPUT_DIR/smoke-question.jsonl"
|
| 118 |
+
head -n 1 -- "$QUESTIONS" > "$smoke_questions"
|
| 119 |
+
smoke_results="$OUTPUT_DIR/smoke-results.jsonl"
|
| 120 |
+
resume_args=()
|
| 121 |
+
[[ -s "$smoke_results" ]] && resume_args+=(--resume)
|
| 122 |
+
"$PYTHON_BIN" "$WRAPPER" \
|
| 123 |
+
--questions "$smoke_questions" \
|
| 124 |
+
--results "$smoke_results" \
|
| 125 |
+
--checkpoint "$CHECKPOINT" \
|
| 126 |
+
--base-model "$BASE_MODEL" \
|
| 127 |
+
--runner "$RUNNER" \
|
| 128 |
+
--gpu-ids "$first_gpu" \
|
| 129 |
+
--workers 1 \
|
| 130 |
+
--beam-size 4 \
|
| 131 |
+
--rollouts 1 \
|
| 132 |
+
--max-depth 4 \
|
| 133 |
+
--seed 42 \
|
| 134 |
+
--temperature 1.2 \
|
| 135 |
+
--max-new-tokens 512 \
|
| 136 |
+
--generation-max-time 20 \
|
| 137 |
+
--sample-max-time-sec 300 \
|
| 138 |
+
--sample-max-tree-size 256 \
|
| 139 |
+
--max-vars-for-lp 0 \
|
| 140 |
+
--lp-workers 1 \
|
| 141 |
+
--min-n 10 \
|
| 142 |
+
--max-n 20 \
|
| 143 |
+
--prompt-format canonical \
|
| 144 |
+
--normalize-candidates \
|
| 145 |
+
--raw-prompt \
|
| 146 |
+
--print-llm \
|
| 147 |
+
"${resume_args[@]}"
|
| 148 |
+
;;
|
| 149 |
+
full)
|
| 150 |
+
"$PYTHON_BIN" "$SUPERVISOR" \
|
| 151 |
+
--questions "$QUESTIONS" \
|
| 152 |
+
--results "$OUTPUT_DIR/results.jsonl" \
|
| 153 |
+
--checkpoint "$CHECKPOINT" \
|
| 154 |
+
--launcher "$WRAPPER" \
|
| 155 |
+
--artifact-dir "$OUTPUT_DIR/supervisor-attempts" \
|
| 156 |
+
--state "$OUTPUT_DIR/supervisor-state.json" \
|
| 157 |
+
--question-timeout-sec 1000 \
|
| 158 |
+
--tree-size-cap 6000 \
|
| 159 |
+
--total-job-stop-sec 151200 \
|
| 160 |
+
--poll-interval-sec 0.25 \
|
| 161 |
+
-- \
|
| 162 |
+
--base-model "$BASE_MODEL" \
|
| 163 |
+
--runner "$RUNNER" \
|
| 164 |
+
--gpu-ids "$GPU_IDS" \
|
| 165 |
+
--workers "$WORKERS" \
|
| 166 |
+
--beam-size 32 \
|
| 167 |
+
--rollouts 6 \
|
| 168 |
+
--max-depth 24 \
|
| 169 |
+
--seed 42 \
|
| 170 |
+
--temperature 1.2 \
|
| 171 |
+
--max-new-tokens 2048 \
|
| 172 |
+
--generation-max-time 20 \
|
| 173 |
+
--max-vars-for-lp 0 \
|
| 174 |
+
--lp-workers "$WORKERS" \
|
| 175 |
+
--min-n 10 \
|
| 176 |
+
--max-n 20 \
|
| 177 |
+
--prompt-format canonical \
|
| 178 |
+
--normalize-candidates \
|
| 179 |
+
--raw-prompt \
|
| 180 |
+
--print-llm
|
| 181 |
+
;;
|
| 182 |
+
esac
|