Instructions to use webAI-Official/TwIL-LM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use webAI-Official/TwIL-LM with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="webAI-Official/TwIL-LM")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("webAI-Official/TwIL-LM")
model = AutoModelForCausalLM.from_pretrained("webAI-Official/TwIL-LM", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use webAI-Official/TwIL-LM with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webAI-Official/TwIL-LM:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf webAI-Official/TwIL-LM:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webAI-Official/TwIL-LM:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf webAI-Official/TwIL-LM:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf webAI-Official/TwIL-LM:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf webAI-Official/TwIL-LM:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf webAI-Official/TwIL-LM:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf webAI-Official/TwIL-LM:Q4_K_M

Use Docker

docker model run hf.co/webAI-Official/TwIL-LM:Q4_K_M

LM Studio
Jan

vLLM

How to use webAI-Official/TwIL-LM with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webAI-Official/TwIL-LM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webAI-Official/TwIL-LM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/webAI-Official/TwIL-LM:Q4_K_M

SGLang

How to use webAI-Official/TwIL-LM with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "webAI-Official/TwIL-LM" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webAI-Official/TwIL-LM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "webAI-Official/TwIL-LM" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webAI-Official/TwIL-LM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use webAI-Official/TwIL-LM with Ollama:
```
ollama run hf.co/webAI-Official/TwIL-LM:Q4_K_M
```

Unsloth Studio

How to use webAI-Official/TwIL-LM with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for webAI-Official/TwIL-LM to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for webAI-Official/TwIL-LM to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for webAI-Official/TwIL-LM to start chatting

Docker Model Runner
How to use webAI-Official/TwIL-LM with Docker Model Runner:
```
docker model run hf.co/webAI-Official/TwIL-LM:Q4_K_M
```

Lemonade

How to use webAI-Official/TwIL-LM with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull webAI-Official/TwIL-LM:Q4_K_M

Run and chat with the model

lemonade run user.TwIL-LM-Q4_K_M

List all available models

lemonade list

Atomic Chat

Add highlights and model details, refresh comparison tables

by anurag051194 - opened 1 day ago

base: refs/heads/main

←

from: refs/pr/3

Discussion Files changed

+258

-66

Files changed (1) hide show

README.md +258 -66

README.md CHANGED Viewed

@@ -27,60 +27,189 @@ through LoRA supervised fine-tuning, checkpoint fusion, WiSE-FT weight interpola
 entropy-weighted GRPO reinforcement learning.
 It raises in-domain formal-logic performance by **+49% relative** over its base model
-(macro gate 0.263 → 0.393) while holding held-out general capability roughly flat.
-Its larger sibling, **TwIL-LM3** (3B, from SmolLM3), trades a smaller in-domain gain for
-strictly better held-out retention. If you care about not regressing on general benchmarks,
-prefer that one.
 ## Results
 ### Track A — in-domain formal logic
-The **macro gate** is the mean of five objective scores: entailment labelling, multiple-choice
-answering, procedural reasoning, Lean proof critique, and rule induction (scored by its
-continuous derivation score). MCQ and procedural are credited as `max(exact_match, loose_match)`.
-`n = 200` prompts per objective, greedy decoding, 2048 max new tokens.
-| objective | SmolLM2-1.7B-Instruct | TwIL-LM2 | Δ |
-|---|---:|---:|---:|
-| entailment_label | 0.245 | **0.585** | +0.340 |
-| rule_induction | 0.135 | **0.514** | +0.379 |
-| lean_critic | 0.490 | **0.525** | +0.035 |
-| mcq_answer | 0.290 | 0.270 | −0.020 |
-| procedural | 0.155 | 0.070 | −0.085 |
-| **macro gate** | **0.2630** | **0.3927** | **+0.1297** |
-The gain is concentrated in entailment labelling and rule induction. MCQ answering and
-procedural reasoning **regressed**, and that is not hidden by the macro — it is averaged into
-the number above.
 ### Track B — held-out benchmarks
-Nothing in this suite was trained on. Scores are re-derived from saved generations with
-delimiter-aware answer extractors rather than read from harness metrics.
-| | SmolLM2-1.7B-Instruct | TwIL-LM2 | Δ |
-|---|---:|---:|---:|
-| core average | 0.499 | **0.508** | +0.009 |
-| suite average (14 datasets) | 0.384 | 0.374 | −0.010 |
-Per-dataset, largest moves in each direction:
-| dataset | base | TwIL-LM2 | Δ |
-|---|---:|---:|---:|
-| GSM-Symbolic | 0.220 | 0.260 | +0.040 |
-| CommonsenseQA | 0.397 | 0.433 | +0.037 |
-| LogicBench BQA | 0.507 | 0.540 | +0.033 |
-| MATH-500 | 0.190 | 0.210 | +0.020 |
-| IFEval (strict) | 0.470 | 0.430 | −0.040 |
-| SVAMP | 0.487 | 0.383 | −0.103 |
-| MuSR | 0.422 | 0.313 | −0.109 |
-**This model does not pass a no-regression bar on held-out tasks.** MuSR and SVAMP lose about
-ten points each. The suite average is slightly negative. The honest summary is that in-domain
-logic improves substantially and general capability is approximately preserved *on average*,
-with real losses on multi-step narrative and word-problem reasoning.
 ## Usage
@@ -109,18 +238,73 @@ print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=Tru
 `return_dict=True` matters on transformers 5.x, where `apply_chat_template` returns a
 `BatchEncoding` rather than a bare tensor; the above works on both 4.x and 5.x.
-The model was trained and evaluated with **greedy decoding** (`do_sample=False`) and a
-**2048-token** generation budget. It usually opens a `<think>...</think>` reasoning block before
-answering, so give it room — the reported numbers assume at least 2048 new tokens, and a shorter
-budget truncates reasoning and scores far worse.
 ## Limitations and caveats
-**Truncation.** At a 2048-token budget, 6.9% of Track A generations hit the cap (down from
-11.7% for the base). Our protocol marks a comparison `rankable` only below 2% truncation, so
-both the base and this model are formally **not rankable** on Track A and the macro gate should
-be read as indicative rather than exact. A truncated response scores zero regardless of whether
-its reasoning was sound, so both numbers are pessimistic — the base more so.
 **Scope.** Tuned for formal logic. The Track B suite does not cover code generation or tool use
 (HumanEval, LiveCodeBench and BFCL were not run for this model or its base), so this release
@@ -130,9 +314,9 @@ makes no claim about those.
 safety tuning beyond whatever the base model carries, and no instruction-following alignment
 work — IFEval in fact regressed.
-**Failed consolidation stage.** A post-RL self-distillation round (SDFT) was attempted to recover
-held-out capability and made both tracks worse at every budget tried. It is not part of this
-model. See the accompanying `SDFT_RESULT.md` in the project repository.
 ## Evaluation protocol
@@ -141,25 +325,33 @@ model. See the accompanying `SDFT_RESULT.md` in the project repository.
 - Track B: 300 examples per task, greedy, `max_gen_toks = 4096`, `max_model_len = 8192`,
   `repetition_penalty = 1.0`, chat template applied, vLLM backend.
 - Both tracks use the same protocol for the model and its base, in a paired run over identical
-  sampled rows.
 `repetition_penalty = 1.0` is load-bearing. A 1.1 penalty produced apparent 20-point swings on
 Track B that were pure decoding artefact; the decoding kwargs are hashed into the protocol
 identity so a mismatched runner fails loudly instead of quietly producing a different number.
 ## Relationship to prior releases
-`main` holds **TwIL-LM2**: a **full merged model** from later in the pipeline — after fusion,
-WiSE-FT interpolation and MGPO reinforcement learning — so it is loaded directly with
-`AutoModelForCausalLM`, with no adapter and no base checkpoint required.
-The original TwIL-LM (v1) release — a PEFT **LoRA adapter** plus GGUF builds for the supervised
-fine-tuning stage only — is archived on the [`TwIL-LM1`](https://huggingface.co/webAI-Official/TwIL-LM/tree/TwIL-LM1)
-branch (and matching tag). Load it with `revision="TwIL-LM1"`.
 The two are scored on different protocols and their headline numbers are not directly
 comparable: v1 reports a macro-*primary* average, while this card reports the five-component
-macro *gate* described above.
 ## License and attribution

 entropy-weighted GRPO reinforcement learning.
 It raises in-domain formal-logic performance by **+49% relative** over its base model
+(macro gate 0.263 → 0.393), and on the strict, no-partial-credit reading of Track A it is the
+strongest model we have measured at any size — ahead of Qwen3-8B and of a 26B Gemma-4 MoE.
+Its larger sibling, [**TwIL-LM3**](https://huggingface.co/webAI-Official/TwIL-LM3) (3B, from SmolLM3),
+trades a smaller in-domain gain for strictly better held-out retention. If you care about not
+regressing on general benchmarks, prefer that one.
+## Highlights
+* **Best strict-7 score of any model we have evaluated** — 0.2386, against 0.2093 for Qwen3-8B
+  and 0.2050 for Gemma-4-26B-A4B-it. Strict-7 gives no loose-match or partial credit anywhere,
+  so it rewards emitting the exactly-requested form rather than merely reasoning near it.
+* **Structured-output accuracy is where the gain lands.** Semantic parsing token-F1 0.8428 and
+  Lean formalisation token-F1 0.6199 are both the highest in the comparison table below, by
+  margins of roughly 0.39 and 0.21 over the next model.
+* **Rule induction nearly quadruples and entailment more than doubles** over the base model
+  (0.135 → 0.514 derivation score, 0.245 → 0.585 accuracy).
+* **Formatted answering becomes possible at all.** Strict MCQ accuracy moves 0.000 → 0.160,
+  where every other model in the comparison table scores 0.020 or below — including both
+  8B-and-larger arms, which answer the question correctly but almost never in the requested form.
+* **Lowest perplexity in the table on both held-out corpora** (2.2981 language, 3.0390 maths),
+  including against models up to fifteen times its size.
+* **Short answers.** Track A generations average 460 tokens against the base model's 719, at
+  14,963 tok/s decode on one H100 — roughly 32 completed answers per second.
+* **Runs anywhere.** 1.7B parameters in bf16, with Q4\_K\_M GGUF at 0.98 GiB for CPU or 2 GB of
+  VRAM.
+Two things this model is **not**: it is not a general assistant (see
+[Limitations](#limitations-and-caveats)), and it does not preserve held-out benchmark
+performance — it gives back about a point of Track B macro relative to its base, which is the
+trade TwIL-LM3 was built to avoid.
+## Model Details
+| Property                  | Value                                                                                               |
+| ------------------------- | --------------------------------------------------------------------------------------------------- |
+| Model ID                  | `webAI-Official/TwIL-LM` (weights on `main`)                                                                 |
+| Base model                | [`HuggingFaceTB/SmolLM2-1.7B-Instruct`](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)  |
+| Total parameters          | 1.71B                                                                                               |
+| Architecture              | Llama-style decoder-only transformer; 24 layers, hidden size 2048                                    |
+| Input / output            | Text / text                                                                                         |
+| Language                  | English                                                                                             |
+| Tokenizer vocabulary size | 49,152                                                                                              |
+| Context window            | 8,192 tokens                                                                                        |
+| Checkpoint precision      | bfloat16 (3.19 GiB), plus Q4\_K\_M / Q5\_K\_M / Q8\_0 / F16 GGUF builds                              |
+| Post-training             | LoRA SFT → checkpoint fusion → WiSE-FT (λ = 0.75) → MGPO reinforcement learning (step 1680)          |
+| Reasoning format          | Emits a `<think>…</think>` block before the answer                                                   |
+| Evaluated decoding        | Greedy, 2048 new tokens, `max_seq_len` 8192                                                          |
+| Specialisation            | Formal logic: FOL translation, entailment, semantic parsing, Lean formalisation and critique          |
+| License                   | webAI Non-Commercial License ver. 1.0                                                               |
+The base model's 8,192-token context is carried through unchanged; nothing in this pipeline
+extends or reduces it, and every reported score was measured inside that window.
 ## Results
 ### Track A — in-domain formal logic
+Every arm below ran through the same harness, prompts and decoding settings described under
+[Evaluation protocol](#evaluation-protocol) — 200 prompts per objective, greedy, 2048 new
+tokens.
+| lane / metric              | TwIL-LM2   | SmolLM2-1.7B base | LFM2.5-1.2B-Thinking | LFM2-2.6B | Granite-4.1-3B | Llama-3.2-3B | Qwen3-8B   | Gemma-4-26B-A4B-it |
+| -------------------------- | ---------- | ----------------- | -------------------- | --------- | -------------- | ------------ | ---------- | ------------------ |
+| parameters                 | 1.7B       | 1.7B              | 1.2B                 | 2.6B      | 3B             | 3B           | 8B         | 26B (4B active)    |
+| lean\_formalize token\_f1  | **0.6199** | 0.1087            | 0.1890               | 0.1321    | 0.2652         | 0.3690       | 0.4022     | 0.4107             |
+| rule\_induction derivation | 0.5136     | 0.1350            | 0.0837               | 0.0615    | 0.2476         | 0.0825       | 0.3680     | **0.7319**         |
+| entailment\_label accuracy | 0.5850     | 0.2450            | 0.4700               | 0.4700    | 0.4900         | 0.3300       | 0.5800     | **0.6200**         |
+| mcq\_answer accuracy       | **0.1600** | 0.0000            | 0.0000               | 0.0150    | 0.0100         | 0.0000       | 0.0000     | 0.0200             |
+| semantic\_parse token\_f1  | **0.8428** | 0.2155            | 0.4439               | 0.3665    | 0.1953         | 0.3102       | 0.4257     | 0.4567             |
+| lean\_critic accuracy      | 0.5250     | 0.4950            | 0.5450               | 0.5900    | 0.5150         | 0.5300       | **0.7950** | 0.7500             |
+| lm\_corpus perplexity ↓    | **2.2981** | 2.5845            | 5.0065               | 4.3815    | 2.4736         | 2.8478       | 2.5440     | 16.1145            |
+| math\_corpus perplexity ↓  | **3.0390** | 3.2670            | 7.7402               | 6.7472    | 4.1162         | 4.7531       | 4.0083     | 59.7838            |
+| average, 6 lanes           | **0.5410** | 0.1999            | 0.2886               | 0.2725    | 0.2872         | 0.2703       | 0.4285     | 0.4982             |
+| **strict-7**               | **0.2386** | 0.1071            | 0.1450               | 0.1579    | 0.1507         | 0.1229       | 0.2093     | 0.2050             |
+| **macro gate**             | 0.3927     | 0.2590 †          | 0.3067               | 0.3473    | 0.3435         | 0.2925       | 0.5336     | **0.6344**         |
+| macro\_primary             | 0.3625     | 0.2900            | 0.3625               | 0.4188    | 0.3675         | 0.3450       | 0.5750     | **0.6100**         |
+| mean generation length ↓   | 460        | 719               | 2464                 | 2296      | **246**        | 696          | 2094       | 1183               |
+† The base column comes from the external-comparison run rather than the paired base-vs-TwIL
+run, hence 0.2590 against the 0.2630 quoted in the summary at the top of this card —
+run-to-run variation of the same checkpoint. The paired run is the correct basis for the
+improvement claim.
+**`average, 6 lanes`** is the plain mean of the six objective rows above it, each at whatever
+scoring that row reports. It mixes token-F1 with accuracy, so it is coarse, but it is the
+broadest summary every arm can be compared on.
+The three rows after it aggregate more carefully, and none of them include the perplexity lanes
+or the token-F1 scorings, which are not on a common 0–1 accuracy scale.
+**`strict-7`** is the mean of seven lanes scored under strict metrics only (`fol_translation`,
+`entailment_label`, `mcq_answer`, `semantic_parse` and `lean_formalize` exact match,
+`lean_critic` and `procedural` accuracy), with no loose-match credit anywhere. Exact match on
+generative lanes is near zero for every model, so it is a harsh scale — useful for ranking
+models against each other rather than as an absolute capability measure.
+**`macro gate`** is the metric the training pipeline gates on: the equal-weight mean of the four
+bounded classification lanes (`entailment_label`, `mcq_answer`, `procedural`, `lean_critic`)
+plus `rule_induction`, scored by its continuous derivation score. Rule induction is included
+specifically so a fine-tune cannot pass the gate while quietly regressing inductive reasoning.
+In the gate, `mcq_answer` and `procedural` are credited as `max(exact_match, loose_match)`: for
+free-text answer lanes, a response that is correct but differently formatted is a formatting
+artefact rather than a reasoning failure. This affects the aggregate only — the per-lane rows
+above stay strict.
+**`macro_primary`** is the same mean over the four classification lanes alone, without
+`rule_induction`. It is kept for comparability with earlier reports, and it is the one summary
+where TwIL-LM2 looks unremarkable: it excludes all three lanes this model is strongest on
+(`semantic_parse`, `lean_formalize`, `rule_induction`) and it credits loose matches, which is
+where the larger models recover most of their score.
+Read against models at its own scale, TwIL-LM2 wins outright. It beats its own base on all six
+objective lanes and all four summary rows, and it beats every 1–3B arm here on strict-7 by at
+least 0.08.
+The more interesting comparison is upward. On **strict-7 it leads the entire table** — 0.2386
+against 0.2093 for Qwen3-8B (4.7x the parameters) and 0.2050 for Gemma-4-26B-A4B-it — and it
+holds the best six-lane average at 0.5410 against Gemma's 0.4982. It also has the lowest
+perplexity in the table on both corpora.
+It does not lead the macro gate, where Gemma-4-26B-A4B-it reaches 0.6344 and Qwen3-8B 0.5336
+against 0.3927. Most of that gap is partial credit rather than capability: the gate credits
+`mcq_answer` and `procedural` at `max(exact_match, loose_match)`, and both larger models answer
+those lanes correctly while almost never producing the requested form — Qwen3-8B's strict MCQ
+accuracy is 0.0000 against TwIL-LM2's 0.1600. Gemma also genuinely leads rule induction
+(0.7319) and entailment (0.6200), which no amount of scoring convention explains away.
+So the honest reading is a split one. If what you need is a model that emits exactly the
+demanded formal object — a parse, a Lean statement, a bare label — this is the strongest option
+in the table and by some distance the smallest. If what you need is a model that gets the answer
+approximately right in free text, the 8B and 26B arms are better.
 ### Track B — held-out benchmarks
+Nothing in this suite was trained on. All arms are scored by the same aggregation over 300
+randomly sampled, model-identical examples per dataset.
+| dataset                     | TwIL-LM2 | SmolLM2-1.7B base | LFM2.5-1.2B-Thinking | LFM2-2.6B  | Granite-4.1-3B | Llama-3.2-3B | Qwen3-8B   | Gemma-4-26B-A4B-it |
+| --------------------------- | -------- | ----------------- | -------------------- | ---------- | -------------- | ------------ | ---------- | ------------------ |
+| gsm8k                       | 0.4633   | 0.4800            | 0.8400               | 0.8767     | 0.9100         | 0.8300       | 0.9567     | **0.9733**         |
+| svamp                       | 0.3833   | 0.4867            | 0.9167               | 0.9000     | 0.9000         | 0.8200       | 0.9367     | **0.9500**         |
+| gsm\_symbolic               | 0.2600   | 0.2200            | 0.6867               | 0.9767     | 0.9533         | 0.8067       | 0.8133     | **0.9967**         |
+| arc\_cot                    | 0.5200   | 0.5100            | 0.8300               | 0.8667     | 0.8633         | 0.7967       | 0.9633     | **0.9767**         |
+| logicbench                  | 0.5400   | 0.5067            | 0.6700               | 0.6267     | 0.7367         | 0.5733       | 0.8567     | **0.8667**         |
+| strategyqa                  | 0.5900   | 0.6000            | 0.5933               | 0.6433     | 0.6333         | 0.6533       | 0.7400     | **0.7700**         |
+| drop                        | 0.4367   | 0.4233            | 0.6667               | 0.6900     | 0.7600         | 0.6733       | **0.8833** | 0.7933             |
+| csqa                        | 0.4333   | 0.3967            | 0.6100               | 0.7433     | 0.7633         | 0.7500       | **0.8633** | **0.8633**         |
+| musr                        | 0.3131   | 0.4223            | 0.5227               | 0.4867     | 0.5669         | 0.4932       | 0.6301     | **0.6369**         |
+| mmlu\_redux                 | 0.3933   | 0.4100            | 0.6400               | 0.7133     | 0.6800         | 0.6000       | 0.8500     | **0.9633**         |
+| ifeval                      | 0.4300   | 0.4700            | 0.8233               | 0.7300     | 0.7967         | 0.7167       | 0.8400     | **0.8733**         |
+| rudas\_ood                  | 0.0289   | 0.0128            | 0.0089               | 0.0017     | 0.0355         | 0.0733       | 0.0468     | **0.1547**         |
+| bbh\_logic                  | 0.2373   | 0.2447            | 0.5327               | 0.5713     | 0.7727         | 0.5333       | 0.6367     | **0.9940**         |
+| math500                     | 0.2100   | 0.1900            | 0.6867               | 0.7133     | 0.6067         | 0.4233       | 0.6100     | **0.9000**         |
+| **macro (10 CoT datasets)** | 0.4333   | 0.4456            | 0.6976               | 0.7523     | 0.7767         | 0.6997       | 0.8493     | **0.8790**         |
+| **macro (all 14)**          | 0.3742   | 0.3838            | 0.6448               | 0.6814     | 0.7127         | 0.6245       | 0.7591     | **0.8366**         |
+The 10-dataset macro covers the chain-of-thought reasoning and QA sets (`gsm8k`, `svamp`,
+`gsm_symbolic`, `arc_cot`, `logicbench`, `strategyqa`, `drop`, `csqa`, `musr`, `mmlu_redux`);
+the 14-dataset macro adds `ifeval`, `rudas_ood`, `bbh_logic` and `math500`.
+**TwIL-LM2 is last in this table, and slightly below its own base.** The 10-dataset macro moves
+0.4456 → 0.4333 and the 14-dataset macro 0.3838 → 0.3742, so roughly one point is given back on
+both. Every other arm is larger, and the ordering is close to a size ordering, so the only
+like-for-like comparison here is against SmolLM2-1.7B — and that comparison is mildly negative.
+Per dataset, the moves against the base go in both directions:
+| dataset       | base   | TwIL-LM2 | Δ      |
+| ------------- | ------ | -------- | ------ |
+| gsm\_symbolic | 0.2200 | 0.2600   | +0.040 |
+| csqa          | 0.3967 | 0.4333   | +0.037 |
+| logicbench    | 0.5067 | 0.5400   | +0.033 |
+| math500       | 0.1900 | 0.2100   | +0.020 |
+| ifeval        | 0.4700 | 0.4300   | −0.040 |
+| svamp         | 0.4867 | 0.3833   | −0.103 |
+| musr          | 0.4223 | 0.3131   | −0.109 |
+The pattern is coherent: the sets that reward committing to a discrete, checkable answer improve
+(symbolic arithmetic, commonsense MCQ, propositional logic), and the sets that reward
+open-ended multi-step narrative reasoning lose (MuSR, SVAMP word problems). Instruction
+following also regresses, which is expected of a model tuned against verifiers rather than
+preferences. **This model does not pass a no-regression bar on held-out tasks.**
 ## Usage
 `return_dict=True` matters on transformers 5.x, where `apply_chat_template` returns a
 `BatchEncoding` rather than a bare tensor; the above works on both 4.x and 5.x.
+The reported numbers use **greedy decoding** (`do_sample=False`) and a **2048-token** generation
+budget. The shipped `generation_config.json` carries no sampling defaults, so greedy is what you
+get unless you ask for otherwise. The model opens a `<think>...</think>` reasoning block before
+answering, so give it room — a short budget truncates the reasoning and scores far worse.
+### GGUF / llama.cpp
+Quantized GGUF builds ship alongside the safetensors weights. The `llama` architecture is fully
+supported by llama.cpp, and the chat template, `<|im_end|>` EOS and BOS are carried into the
+GGUF metadata, so chat mode works without extra flags.
+| file                   | quant    | size     | bits/weight | notes                                             |
+| ---------------------- | -------- | -------- | ----------- | ------------------------------------------------- |
+| TwIL-LM2-Q4\_K\_M.gguf | Q4\_K\_M | 0.98 GiB | 4.93        | recommended default; runs on CPU or 2 GB of VRAM  |
+| TwIL-LM2-Q5\_K\_M.gguf | Q5\_K\_M | 1.14 GiB | 5.73        | a little more headroom than Q4\_K\_M              |
+| TwIL-LM2-Q8\_0.gguf    | Q8\_0    | 1.70 GiB | 8.51        | near-lossless, for quality-sensitive use          |
+| TwIL-LM2-F16.gguf      | F16      | 3.19 GiB | 16.01       | unquantized, for requantization or reference runs |
+```bash
+llama-cli -m TwIL-LM2-Q4_K_M.gguf -cnv --temp 0 -n 2048
+```
+Pass `--temp 0` and leave the generation budget at 2048 tokens or more: the model emits a
+`<think>` block before answering, and truncating it costs far more accuracy than the
+quantization does.
+F16 was produced directly by `convert_hf_to_gguf.py` from the released bf16 weights; the
+K-quants were quantized from the F16 build with `llama-quantize`, without an importance matrix.
+Note that F16 is not bit-identical to the released weights: bf16 and f16 carry the same 16 bits
+but trade exponent range against mantissa precision, so the conversion is a narrowing one, in
+practice negligible for inference.
+The published Track A and Track B numbers were measured on the **bf16** weights through vLLM,
+not on any of these GGUF builds, so expect small deviations — most likely at Q4\_K\_M — that
+have not been quantified here.
+## How it was built
+Four stages on top of the base model:
+1. **LoRA supervised fine-tuning** on a synthetic formal-logic corpus covering the Track A
+   objectives (first-order-logic translation, entailment labelling, semantic parsing, Lean
+   formalisation and critique, procedural reasoning, rule induction).
+2. **Checkpoint fusion** — parameter-space averaging of intermediate SFT checkpoints selected
+   by a diversity probe, rather than taking the final checkpoint.
+3. **WiSE-FT interpolation** toward the pretrained base, `W = (1 − λ)·W_base + λ·W_finetuned`
+   with **λ = 0.75** — three quarters of the fine-tuned delta is retained. λ was chosen by
+   constrained optimisation: maximise in-domain score subject to minimal degradation on held-out
+   benchmarks. TwIL-LM3 keeps only a quarter of its delta, and that difference is most of why it
+   holds Track B where this model does not.
+4. **MGPO** — entropy-weighted GRPO reinforcement learning against a programmatic verifier,
+   with partial credit for loose matches and token-F1 so that all-fail prompt groups still
+   produce gradient. Published checkpoint is **step 1680**.
 ## Limitations and caveats
+**Held-out regression.** The 10-dataset Track B macro moves 0.4456 → 0.4333 against the base.
+An earlier revision of this card quoted a narrower five-dataset "core average" that showed a
+small gain; the canonical 10- and 14-dataset macros in the table above are the numbers to use,
+and both are slightly negative.
+**Truncation.** At a 2048-token budget, 6.9% of Track A generations hit the cap, down from 11.7%
+for the base. Our protocol marks a comparison `rankable` only below 2% truncation, so both the
+base and this model are formally **not rankable** on Track A and the macro gate should be read
+as indicative rather than exact. A truncated response scores zero regardless of whether its
+reasoning was sound, so both numbers are pessimistic — the base more so, meaning the true gap is
+probably narrower than +0.130.
 **Scope.** Tuned for formal logic. The Track B suite does not cover code generation or tool use
 (HumanEval, LiveCodeBench and BFCL were not run for this model or its base), so this release
 safety tuning beyond whatever the base model carries, and no instruction-following alignment
 work — IFEval in fact regressed.
+**Failed consolidation stage.** A post-RL self-distillation round (SDFT) was attempted to
+recover held-out capability and made both tracks worse at every budget tried. It is not part of
+this model. See the accompanying `SDFT_RESULT.md` in the project repository.
 ## Evaluation protocol
 - Track B: 300 examples per task, greedy, `max_gen_toks = 4096`, `max_model_len = 8192`,
   `repetition_penalty = 1.0`, chat template applied, vLLM backend.
 - Both tracks use the same protocol for the model and its base, in a paired run over identical
+  sampled rows. The comparison arms are scored on the same sampled rows as well.
 `repetition_penalty = 1.0` is load-bearing. A 1.1 penalty produced apparent 20-point swings on
 Track B that were pure decoding artefact; the decoding kwargs are hashed into the protocol
 identity so a mismatched runner fails loudly instead of quietly producing a different number.
+Track B is sampled at 300 examples per dataset for compute reasons. Absolute scores can shift on
+the full sets, but the comparative ordering across models is stable.
 ## Relationship to prior releases
+The `main` branch of this repository holds **TwIL-LM2**: a **full merged model** from a later
+point in the pipeline — after fusion, WiSE-FT interpolation and MGPO reinforcement learning — so
+it loads directly with `AutoModelForCausalLM`, with no adapter and no base checkpoint required.
+It is also mirrored on the `TwIL-LM2` branch.
+The original TwIL-LM (v1) release — a PEFT **LoRA adapter** for the supervised fine-tuning stage
+only — is archived on the `TwIL-LM1` branch and matching tag. Load it with
+`revision="TwIL-LM1"`.
 The two are scored on different protocols and their headline numbers are not directly
 comparable: v1 reports a macro-*primary* average, while this card reports the five-component
+macro *gate* and the seven-lane strict mean described above.
+[**TwIL-LM3**](https://huggingface.co/webAI-Official/TwIL-LM3) is the 3B member of the family, built from
+SmolLM3 by the same pipeline. It gains less in-domain than this model but improves its held-out
+scores at the same time, which this model does not.
 ## License and attribution