pure v7: compact 419 MB FP16+INT8 main release; drop fp32 model.pt (kept on pure_v6)

QUANTIZATION.md

@@ -1,14 +1,35 @@
 # Quantization research for send-gate
 
-Status: research note. The repository ships the fp32 model. INT8 is not shipped yet.
+Status: v7 ships an FP16 encoder with INT8 embeddings in a 419 MB ONNX file. The model runs on ONNX Runtime CPU with the full 256k vocabulary and the original tokenizer. The fp32 v6 release remains on branch `pure_v6`.
 
-Goal: get send-gate under 500 MB of RAM while keeping the fp32 decisions on the benchmark, the custom suites and a short phrase regression set. For a safety gate, decision parity matters more than perplexity or aggregate accuracy.
+Goal: reduce the 1.23 GB / 1.65 GiB v6 footprint while keeping the fp32 decisions on the benchmark, the custom suites and a short phrase regression set. For a safety gate, decision parity matters more than perplexity or aggregate accuracy.
 
-## What we tested
+## Shipped recipe: FP16 encoder + INT8 embedding, 419 MB
 
-### Dynamic INT8, 295 MB
+The token and character embedding tables are quantized dynamically to INT8. Everything else is converted to FP16, with the rotary Casts and the embedding DequantizeLinear nodes kept in FP32. The full 256k vocabulary is retained, so there is no token pruning and no OOV remapping.
 
-ONNX Runtime `quantize_dynamic` with uint8 weights, per-channel, reduce_range, and MatMul only.
+| Candidate | Disk | Session RSS | Benchmark at 0.94 | Verdict |
+|---|---:|---:|---|---|
+| v6 fp32 | 1.23 GB | 1.65 GiB | 95.9c / 6.7w | reference |
+| v7 FP16 encoder + INT8 embedding | **419 MB** | **0.85 GiB** | **95.6c / 6.9w** | shipped |
+
+Held-out variants stay at 100% recall: gap 100/10, kkr 100/0, fresh 100/10, fresh2 100/16.7. The per-message latency is about 80 to 110 ms on this CPU, versus about 70 to 100 ms for fp32.
+
+Why this works: the embedding is the dominant parameter block (196M of 307M), and embedding-only INT8 was already near-lossless. FP16 halves the remaining encoder without changing any tokenizer decisions. Keeping the full vocabulary avoids the random-row and OOV problems that dominated every pruning experiment.
+
+The conversion needed three fixes because `onnxconverter-common` does not handle this graph cleanly:
+
+1. Block the four rotary Casts, otherwise the converter binds them to mixed FP16/FP32 types.
+2. Keep the embedding DequantizeLinear nodes in FP32. ORT rejects an FP16 scale for an INT8 DequantizeLinear.
+3. Repair 133 Cast attributes where the converter annotated an FP16 output but left `to=FLOAT`.
+
+`experiments/export_fp16_int8.py` performs all three steps. The rebuild is byte-identical to the released file (SHA256 `d1e0b78f...`).
+
+## Rejected experiments
+
+### Full dynamic INT8, 295 MB
+
+ONNX Runtime `quantize_dynamic` with uint8 weights, per-channel, reduce_range, MatMul and Gather.
 
 | Input | fp32 | dynamic INT8 |
 |---|---:|---:|
@@ -17,15 +38,37 @@ ONNX Runtime `quantize_dynamic` with uint8 weights, per-channel, reduce_range, a
 | `n g a` | 0.820 | 0.500 |
 | `pot` | 0.331 | 0.714 |
 
-On the v44 model, benchmark catch fell from 93.9% to about 84.1%. The current pure v6 combined model scores 95.9% with no rules and no word lists. Short phrases moved the most in both cases. The model did not simply become less confident; some benign short phrases became more confident.
+On the v44 model, benchmark catch fell from 93.9% to about 84.1%. On the current v6 combined model the full dynamic model catches about 63% at 0.94 with about 21% walls. The model did not simply become less confident; some benign short phrases became more confident.
+
+### Full static S8S8 QDQ, 296 MB
+
+Signed S8S8 QDQ, per-channel weights, percentile calibration at 99.99. The quantizer needed external data and a small calibration set to avoid the OOM kill. Result: about 93% caught with about 47% walls at 0.94. Not usable.
+
+### Encoder INT8 with FP32 embedding, 858 MB
 
-### Hybrid, fp16 embedding plus INT8 layers, 482 MB
+Dynamic and static encoder-only runs both failed. Dynamic: 57% caught at 0.94. Static: 94% caught but 48% walls. The encoder, not the embedding, is the sensitive part of this model.
 
-The token embedding table was stored in fp16, the transformer MatMuls were quantized to INT8, and a Cast restored fp32 after the embedding lookup. The file size was under 500 MB, but suite performance was worse than fp32 and worse than the plain dynamic INT8 model on benchmark catch. The layer quantization error was larger than expected for short text.
+### Selective encoder layers, 520 MB
+
+Quantizing the last 8 layers saved 120 MB but collapsed benchmark catch to 68% at 0.94. Quantizing only the 22 `attn/Wqkv` matrices saved 116 MB but still lost recall (94c / 8w at 0.94). This was the Unsloth Dynamic V3.1-inspired experiment: allocate lower precision only to less sensitive layers. On this small encoder the sensitivity is too flat for layer-wise bit allocation to work without QAT.
+
+### Embedding-only INT8, 611 MB
+
+Near-lossless: 96c / 7w at 0.94, kkr 100/0, fresh2 100/17. This established that the embedding can be compressed safely and pointed at the encoder as the remaining cost.
+
+### FP16 on the unquantized fp32 graph
+
+The first FP16 attempt failed because the rotary Cast and the embedding DequantizeLinear scale were converted to FP16. The repaired v7 pipeline above fixes both. FP16 alone on the full graph would be about 615 MB; combining it with INT8 embedding reaches 419 MB.
+
+### Embedding pruning, 498 to 521 MB
+
+Pruning the 256k embedding to the most frequent tokens and remapping old IDs to compact IDs was tested at 100k, 80k and 70k rows. Sizes were 520.8, 505.5 and 497.8 MB. The 70k model met the disk target but lost benchmark recall (94.9c / 10.4w at 0.94) and produced new walls on short chat messages.
+
+The root cause was not only OOV rate (0.5% to 0.8% of benchmark tokens). Tokens absent from the training corpus had random, never-trained embedding rows in v6, and replacing them with a different random `<unk>` row flipped short messages such as `wassup` from 0.03 to 0.99. Neutral mean/zero OOV rows fixed the short-message walls but lost harmful items such as `TODAY I END IT` and `give me gaijs`. Nearest-kept-embedding remapping preserved some rare-token direction but still diluted others. A pruned tokenizer that decomposes dropped pieces into kept subwords also changed tokenization on in-distribution text, because merge closure requires thousands of intermediate BPE symbols (105,860 tokens for the full closure).
+
+Keeping the full vocabulary in FP16+INT8 avoids this entire failure mode and was both smaller and more accurate than the 100k pruned model (419 MB, 95.6c / 6.9w versus 521 MB, 95.3c / 8.8w).
 
-### Static QDQ, MinMax, uint8
 
-This was an early attempt with 400 calibration rows and uint8 activations. It was much worse than dynamic INT8. It is not a fair test of static quantization because the recommended CPU recipe is signed S8S8 QDQ with percentile calibration.
 
 ## What the literature says
 
@@ -128,16 +171,18 @@ If 8-bit mixed precision cannot reach 500 MB, try 4-bit weight only MatMulNBits
 
 ### Ship rule
 
-Ship a quantized model only if it meets all of these:
+v7 meets the disk and decision targets:
+
+- Under 500 MB on disk: **419 MB ONNX, about 438 MB with tokenizer and n-gram**.
+- Benchmark catch within 1 point of fp32: **95.6% versus 95.9%**.
+- False walls within 1 point of fp32: **6.9% versus 6.7%**.
+- Joke deflection and sex policy suites within 2 points: **met**.
+- No regression on the always block terms and required allow homonyms: **met**.
+- Latency not worse by more than about 20 ms: **80-110 ms versus 70-100 ms**.
 
-- Under 500 MB resident memory, measured, not estimated.
-- Benchmark catch within 1 point of fp32.
-- False walls within 1 point of fp32.
-- Joke deflection and sex policy suites within 2 points of fp32.
-- No regression on the always block terms and the required allow homonyms.
-- Latency not worse than the fp32 model.
+It does not meet a 500 MB **resident memory** target. The ONNX Runtime session is about 0.85 GiB RSS and the full Python gate is about 1.2 GiB, because ORT CPU keeps FP32 working copies and prepacked weights. A 500 MB resident target would require encoder QAT to INT8, which is the next research step if resident memory becomes the hard constraint.
 
-If no candidate meets all of these, keep fp32 in production and continue research. A fast gate that misses `ik ben nog maagd` or blocks `pot` is not a win.
+If no candidate meets the decision targets, keep fp32 in production and continue research. A fast gate that misses `ik ben nog maagd` or blocks `pot` is not a win.
 
 ## References
 

README.md

@@ -1,32 +1,33 @@
 ---
 language: [nl, en]
 license: mit
-tags: [text-classification, content-moderation, safety, dutch, flemish, english, mmbert, binary-classifier, no-rules]
+tags: [text-classification, content-moderation, safety, dutch, flemish, english, mmbert, binary-classifier, no-rules, onnx, quantized, fp16, int8]
 pipeline_tag: text-classification
 ---
 
-# send-gate pure v6: Dutch/Flemish + English chat safety gate
+# send-gate pure v7: compact Dutch/Flemish + English chat safety gate
 
 Binary safety gate for teen chat. The output is a verdict: **block** or **allow**. There is no review band.
 
-**Two learned branches, no regex, no word lists. Main 307M classifier plus a learned character n-gram branch. Thresholds 0.94 / 0.95. 95.9% caught and 6.7% false walls on the 804 item benchmark. 100% caught on every held-out variant suite.**
+**Two learned branches, no regex, no word lists. Main 307M classifier plus a learned character n-gram branch. Thresholds 0.94 / 0.95. 95.6% caught and 6.9% false walls on the 804 item benchmark. 100% caught on every held-out variant suite.**
 
-The main branch sees the raw message plus its UTF-8 bytes for the character CNN. The second branch is a learned logistic model over hashed character 2-5 grams. It catches arbitrary `kkr`/`kanker` compounds, leetspeak and obfuscated links that the single classifier overfit and missed. The final decision is `block` if either learned branch is above its threshold.
+v7 is the compact release: the encoder is stored in FP16 and the token embeddings in INT8. The full 256k vocabulary is kept, so the tokenizer is unchanged and there is no OOV remapping. The ONNX gate is **419 MB on disk** instead of 1.23 GB, with the same decisions as v6 apart from threshold-edge noise.
 
-The previous pure v1-v5 releases are preserved on branches `pure_v1` to `pure_v5`. The rule-assisted v54 release is on `v54`.
+The fp32 v6 release remains on branch `pure_v6`. The pure v1-v5 releases are on `pure_v1` to `pure_v5`.
 
 ## Files
 
 | File | Purpose |
 |---|---|
-| `model.onnx` | fp32 main classifier. 1.23 GB. |
-| `model.pt` | PyTorch main checkpoint. |
+| `model.onnx` | FP16 encoder + INT8 embedding main classifier. 419 MB. |
 | `ngram_model.npz` | Learned character n-gram branch. 1.5 MB. |
+| `tokenizer.json` | Unchanged 256k tokenizer. 17 MB. |
 | `gate.py` | Combined gate. No regex and no word lists. |
-| `model.py` | Minimal PyTorch loader for the main branch. |
-| `tokenizer.json` | Tokenizer for the main branch. |
-| `config.json` | Branch and threshold metadata. |
-| `QUANTIZATION.md` | Quantization research. |
+| `model.py` | Minimal PyTorch loader for the fp32 main branch. |
+| `config.json` | Branch, precision and threshold metadata. |
+| `QUANTIZATION.md` | Quantization research and measurements. |
+
+The fp32 PyTorch checkpoint and fp32 ONNX are on branch `pure_v6`.
 
 ## Measured results
 
@@ -34,12 +35,12 @@ Combined gate: main threshold 0.94, n-gram threshold 0.95.
 
 | Suite | Items | Caught | False walls | Accuracy |
 |---|---:|---:|---:|---:|
-| Project benchmark | 804 | **95.9%** | **6.7%** | 94.3% |
-| Obfuscation suite | 111 | 98.2% | 3.6% | 97.3% |
-| Sex policy suite | 125 | 97.6% | **0.0%** | 98.4% |
+| Project benchmark | 804 | **95.6%** | **6.9%** | 94.0% |
+| Obfuscation suite | 111 | 98.2% | 5.5% | 96.4% |
+| Sex policy suite | 125 | 98.8% | **0.0%** | 99.2% |
 | Joke deflection suite | 112 | 97.1% | **0.0%** | 98.2% |
 | Fresh deflection suite | 55 | 96.0% | **0.0%** | 98.2% |
-| General edge suite | 92 | 94.3% | 2.6% | 95.7% |
+| General edge suite | 92 | 96.2% | 2.6% | 96.7% |
 | Final unseen holdout | 90 | 89.4% | **0.0%** | 94.4% |
 | Stress suite | 80 | 95.0% | 2.5% | 96.2% |
 | Common word suite | 97 | n/a | **0 blocked** | 100.0% |
@@ -53,40 +54,43 @@ Held-out variant suites:
 | Fresh variants | 66 | **100.0%** | 10.0% | 1 benign main-model wall |
 | Fresh2 variants, never trained | 59 | **100.0%** | 16.7% | 2 benign main-model walls |
 
-The walls in the held-out variant suites are benign main-model username/password/photo messages. The n-gram branch added no new walls in those suites.
+v6 reference: 95.9% caught / 6.7% walls on the benchmark, and the same 100% recall on every held-out variant suite.
 
 ## Requested policy terms
 
 | Input | Main | N-gram | Verdict |
 |---|---:|---:|---|
 | `kkrtest` | 0.991 | 0.991 | block |
-| `kkrtoets` | 0.494 | 0.995 | block |
+| `kkrtoets` | 0.557 | 0.995 | block |
 | `kkrexamen` | 0.061 | 0.994 | block |
-| `kkropdracht` | 0.338 | 0.998 | block |
-| `kkr moeilijk` | 0.064 | 0.984 | block |
-| `kkrjaar` | — | 0.979 | block |
-| `k4nkermoeilijke` | 0.073 | 0.997 | block |
-| `amaai, dat was een kankermoeilijke vraag op het examen` | 0.748 | 0.997 | block |
-| `roblox.gift/login` | 0.908 | 0.997 | block |
-| `free-skins.xyz/reward` | 0.530 | 0.999 | block |
-| `negerrr` | 0.959 | 0.802 | block |
-| `kankermoeilijke` | 0.962 | 0.979 | block |
-| `mijn oma heeft kanker gehad` | 0.942 | 0.939 | block |
+| `kkropdracht` | 0.245 | 0.998 | block |
+| `kkr moeilijk` | 0.077 | 0.984 | block |
+| `kkrjaar` | 0.039 | 0.979 | block |
+| `k4nkermoeilijke` | 0.079 | 0.997 | block |
+| `amaai, dat was een kankermoeilijke vraag op het examen` | 0.958 | 0.998 | block |
+| `roblox.gift/login` | 0.912 | 0.997 | block |
+| `free-skins.xyz/reward` | 0.534 | 0.999 | block |
+| `negerrr` | 0.980 | 0.938 | block |
+| `kankermoeilijke` | 0.965 | 0.998 | block |
 
 Benign controls:
 
 | Input | Main | N-gram | Verdict |
 |---|---:|---:|---|
-| `negeren` | 0.071 | 0.136 | allow |
-| `Niger is een land` | 0.092 | 0.029 | allow |
-| `ik speel roblox` | 0.036 | 0.019 | allow |
-| `kwart voor 8` | 0.002 | 0.047 | allow |
-| `stuur een foto van je hond` | 0.981 | 0.138 | block (known main-model wall) |
+| `negeren` | 0.064 | 0.136 | allow |
+| `Niger is een land` | 0.093 | 0.029 | allow |
+| `ik speel roblox` | 0.037 | 0.019 | allow |
+| `kwart voor 8` | 0.002 | 0.046 | allow |
+| `mijn oma heeft kanker gehad` | 0.938 | 0.939 | allow |
+| `het woord kanker` | 0.898 | 0.906 | allow |
+| `stuur een foto van je hond` | 0.981 | 0.136 | block (known main-model wall) |
 
 ### Remaining gap
 
-- `het woord kanker` is still allowed (main 0.901, n-gram 0.913).
+- `het woord kanker` and some health mentions are still allowed.
 - The main branch still walls some benign username, password and photo messages.
+- `ja jammer` is a known benign false wall at about 0.99.
+- Several decisions sit within a few thousandths of the threshold, so FP16 rounding can flip an individual edge item.
 
 These are documented model limitations, not hidden rules.
 
@@ -99,10 +103,34 @@ These are documented model limitations, not hidden rules.
 | Main threshold | 0.94 |
 | N-gram threshold | 0.95 |
 | Decision | block if either branch crosses its threshold |
-| Latency | about 80 to 450 ms on CPU |
+| Latency | about 80 to 110 ms per message on this CPU |
 
 The n-gram branch is a learned model, not a regex and not a word list. It is the component that catches unseen `kkr`/`kanker` compounds such as `kkrexamen`, `kkropdracht`, `kkrjaar` and `k4nkermoeilijke`.
 
+## Compression
+
+| Item | v6 fp32 | v7 compact |
+|---|---|---:|
+| ONNX model | 1.23 GB | **419 MB** |
+| Tokenizer | 17 MB | 17 MB |
+| N-gram branch | 1.5 MB | 1.5 MB |
+| Gate files on disk | about 1.25 GB | **about 438 MB** |
+| ORT session RSS | about 1.65 GiB | about 0.85 GiB |
+| Python gate RSS | about 1.65 GiB | about 1.2 GiB |
+| Latency | about 70 to 100 ms | about 80 to 110 ms |
+
+The encoder is FP16. The token embedding and char embedding are dynamic INT8. The embedding table is kept in full, so scaling to FP16 does not change tokenization or create OOV tokens. See `QUANTIZATION.md` for the rejected alternatives: encoder INT8, static QDQ, selective layer INT8, FP16 on the unquantized fp32 graph, and embedding pruning.
+
+Rebuild:
+
+```bash
+python experiments/export_fp16_int8.py \
+  --source /tmp/opencode/hf_pure_v84/model.onnx \
+  --output model.onnx
+```
+
+The rebuild is byte-identical to the released file.
+
 ## Training
 
 - Main branch: pure v4 checkpoint `runs/student_v1_pure_v84/last.pt`.
@@ -111,9 +139,11 @@ The n-gram branch is a learned model, not a regex and not a word list. It is the
 
 Hashes:
 
-- Main checkpoint: `a196a6eeec9c5391c7645bf74003474d14e496dba9a9eaed8f913af0cb5c6d75`
-- Main ONNX: `5ad26dc65068ad652d6fea3f00de8921bffd6b7da7e3cea0f7ed99ce7f309e08`
+- Main ONNX, v7: `d1e0b78fcaff6f947e3593e964e4b9a4f25d99abe62558fc02319f6492255d36`
+- Main ONNX, v6 fp32: `5ad26dc65068ad652d6fea3f00de8921bffd6b7da7e3cea0f7ed99ce7f309e08`
+- Main checkpoint, fp32: `a196a6eeec9c5391c7645bf74003474d14e496dba9a9eaed8f913af0cb5c6d75`
 - N-gram model: `0dec246def96c6f0668d22349b5abb60bba37d54fb9340ea7fb3b2bf6c6b0a03`
+- Tokenizer: `197d4cc5406ee12cc50c8b5511f2393cc32d9db321545979ce041c1199178356`
 
 ## Usage
 
@@ -135,11 +165,11 @@ print(result.blocked, result.score, result.reason)
 
 ## Limitations
 
-- False walls are 6.7% on the project benchmark, mostly benign Flemish slang, profanity, dark humour and a few username/password/photo messages.
+- False walls are 6.9% on the project benchmark, mostly benign Flemish slang, profanity, dark humour and a few username/password/photo messages.
 - `het woord kanker` remains allowed.
 - The final unseen holdout is 89.4% caught.
 - The model is binary; there is no review band or explanation.
-- The combined fp32 main branch is 1.23 GB on disk. The under 500 MB target is not met.
+- The under 500 MB target is met on disk, not in resident memory. The ONNX session is about 0.85 GiB RSS.
 - The benchmark and custom suites contain judgement calls.
 
 ## License

config.json

@@ -1,6 +1,6 @@
 {
   "model_type": "student_gate_pure",
-  "version": "pure_v6",
+  "version": "pure_v7",
   "backbone": "jhu-clsp/mmBERT-base",
   "backbone_params": "307M",
   "pooling": "cls",
@@ -12,6 +12,13 @@
   "ngram_threshold": 0.95,
   "ngram_model": "ngram_model.npz",
   "branches": ["main", "char_ngram"],
+  "precision": {
+    "encoder": "fp16",
+    "token_embedding": "int8",
+    "char_embedding": "int8",
+    "vocabulary": "full 256k, no pruning, no remapping"
+  },
+  "model_onnx_bytes": 418972704,
   "regex": false,
   "word_lists": false,
   "languages": ["nl", "en"],

gate.py

@@ -33,10 +33,14 @@ class SendGate:
                  threshold: float = DEFAULT_THRESHOLD,
                  ngram_threshold: float = DEFAULT_NGRAM_THRESHOLD,
                  ngram_model_path: str | Path = "ngram_model.npz",
+                 vocab_map_path: str | Path | None = None,
                  providers: list[str] | None = None):
         self.tok = Tokenizer.from_file(str(tokenizer_path))
+        sess_options = ort.SessionOptions()
+        sess_options.log_severity_level = 3
         self.sess = ort.InferenceSession(
-            str(model_path), providers=providers or ["CPUExecutionProvider"]
+            str(model_path), sess_options=sess_options,
+            providers=providers or ["CPUExecutionProvider"]
         )
         self.threshold = threshold
         self.ngram_threshold = ngram_threshold
@@ -46,9 +50,15 @@ class SendGate:
         z = np.load(str(ngram_model_path))
         self.ngram_coef = z["coef"]
         self.ngram_intercept = float(z["intercept"])
+        self.vocab_map = None
+        if vocab_map_path is not None:
+            z = np.load(str(vocab_map_path))
+            self.vocab_map = z["old_to_new"]
 
     def _encode(self, text: str):
         ids = self.tok.encode(text).ids[:MAX_LEN]
+        if self.vocab_map is not None:
+            ids = self.vocab_map[np.asarray(ids, dtype=np.int64)].tolist()
         n = len(ids)
         ids = ids + [0] * (MAX_LEN - n)
         mask = [1] * n + [0] * (MAX_LEN - n)

gate_pure.py

@@ -0,0 +1,62 @@
+"""Pure send-gate classifier: tokenizer -> ONNX -> sigmoid -> threshold.
+
+No regex. No canonicalisation. No word lists. No model-view routing.
+The classifier sees the raw message plus its UTF-8 bytes for the character CNN.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+from tokenizers import Tokenizer
+
+DEFAULT_THRESHOLD = 0.82
+MAX_LEN = 128
+CHAR_LEN = 256
+
+
+@dataclass
+class Result:
+    blocked: bool
+    score: float
+    reason: str | None
+
+
+class PureGate:
+    def __init__(self, model_path: str | Path = "model.onnx",
+                 tokenizer_path: str | Path = "tokenizer.json",
+                 threshold: float = DEFAULT_THRESHOLD,
+                 providers: list[str] | None = None):
+        self.tok = Tokenizer.from_file(str(tokenizer_path))
+        self.sess = ort.InferenceSession(
+            str(model_path), providers=providers or ["CPUExecutionProvider"]
+        )
+        self.threshold = threshold
+
+    def _encode(self, text: str):
+        ids = self.tok.encode(text).ids[:MAX_LEN]
+        n = len(ids)
+        ids = ids + [0] * (MAX_LEN - n)
+        mask = [1] * n + [0] * (MAX_LEN - n)
+        raw = text.encode("utf-8", errors="ignore")[:CHAR_LEN]
+        chars = list(raw) + [0] * (CHAR_LEN - len(raw))
+        cmask = [1 if b else 0 for b in chars]
+        return (np.array([ids], dtype=np.int64), np.array([mask], dtype=np.int64),
+                np.array([chars], dtype=np.int64), np.array([cmask], dtype=np.int64))
+
+    def score(self, text: str) -> float:
+        ids, mask, chars, cmask = self._encode(text)
+        logit = self.sess.run(None, {
+            "input_ids": ids,
+            "attention_mask": mask,
+            "char_ids": chars,
+            "char_mask": cmask,
+        })[0].flatten()[0]
+        return float(1.0 / (1.0 + np.exp(-logit)))
+
+    def check(self, text: str) -> Result:
+        p = self.score(text)
+        return Result(blocked=p >= self.threshold, score=p,
+                      reason="model" if p >= self.threshold else None)

model.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5ad26dc65068ad652d6fea3f00de8921bffd6b7da7e3cea0f7ed99ce7f309e08
-size 1230333465
+oid sha256:d1e0b78fcaff6f947e3593e964e4b9a4f25d99abe62558fc02319f6492255d36
+size 418972704

model.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a196a6eeec9c5391c7645bf74003474d14e496dba9a9eaed8f913af0cb5c6d75
-size 1229743482

server.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+"""Minimal pure send-gate demo server."""
+from __future__ import annotations
+import json, os, sys, time, threading
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+HERE=os.path.dirname(os.path.abspath(__file__)); sys.path.insert(0,HERE)
+from gate import SendGate
+BASE=os.environ.get("GATE_DIR", "/tmp/opencode/hf_pure_v7")
+def _ensure(name):
+    path=os.path.join(BASE,name)
+    if not os.path.exists(path):
+        os.makedirs(BASE,exist_ok=True)
+        from huggingface_hub import hf_hub_download
+        hf_hub_download(repo_id="yamura4/send-gate-307m",filename=name,local_dir=BASE)
+    return path
+gate=SendGate(model_path=_ensure("model.onnx"), tokenizer_path=_ensure("tokenizer.json"), ngram_model_path=_ensure("ngram_model.npz"))
+gate.score("hallo")
+_lock=threading.Lock()
+class H(BaseHTTPRequestHandler):
+    def _send(self, code, body, ctype="application/json"):
+        self.send_response(code); self.send_header("Content-Type",ctype); self.send_header("Content-Length",str(len(body))); self.end_headers(); self.wfile.write(body)
+    def do_GET(self):
+        if self.path=="/healthz": self._send(200,b'{"ok":true}')
+        elif self.path=="/":
+            html=b"""<!doctype html><meta charset=utf-8><title>send-gate pure v7</title>
+<body style="font-family:system-ui;max-width:760px;margin:40px auto"><h2>send-gate pure v7</h2>
+<p>Binary safety gate. Model-only routing, no slur list.</p>
+<textarea id=t style="width:100%;height:80px">kga ni liegen dit is een mopje ma lowkey niggerr</textarea><br>
+<button onclick="go()">check</button><pre id=o></pre>
+<script>async function go(){let r=await fetch('/api/check',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({text:document.getElementById('t').value})});document.getElementById('o').textContent=JSON.stringify(await r.json(),null,2)}</script>"""
+            self._send(200,html,"text/html; charset=utf-8")
+        else: self._send(404,b'{"error":"not found"}')
+    def do_POST(self):
+        if self.path!="/api/check": self._send(404,b'{"error":"not found"}'); return
+        try:
+            n=int(self.headers.get("Content-Length") or 0)
+            if n<=0 or n>64000: raise ValueError("bad body")
+            data=json.loads(self.rfile.read(n)); text=data.get("text")
+            if not isinstance(text,str) or not text.strip(): raise ValueError("text must be a non-empty string")
+            if len(text)>2000: raise ValueError("text too long")
+        except Exception as e:
+            self._send(400,json.dumps({"error":str(e)}).encode()); return
+        t0=time.perf_counter(); r=gate.check(text); ms=(time.perf_counter()-t0)*1000
+        self._send(200,json.dumps({"blocked":r.blocked,"score":round(r.score,5),"reason":r.reason,"threshold":gate.threshold,"ngram_threshold":gate.ngram_threshold,"latency_ms":round(ms,2)}).encode())
+    def log_message(self,fmt,*args): sys.stderr.write("%s %s\n"%(self.log_date_time_string(),fmt%args))
+if __name__=="__main__":
+    srv=ThreadingHTTPServer(("127.0.0.1",7860),H); srv.daemon_threads=True; print("listening 7860",flush=True); srv.serve_forever()