Add ARC-Easy and HellaSwag benchmark results

Publish the full epoch-3 zero-shot benchmark panel alongside PIQA, with machine-readable MLX FP16 reports.

README.md

@@ -38,10 +38,21 @@ and lowest held-out SFT loss across the three epochs.
 | 2 | 3,988 | 0.963912 | 2.62 | 67.85% | 68.82% |
 | 3 | 5,982 | 0.959617 | 2.61 | 68.01% | 69.10% |
 
-PIQA uses the complete 1,838-example validation split, zero-shot causal
-continuation log-likelihood, no chat template, maximum length 2,048 and FP16
-PyTorch with the custom Triton path. Behavioral reports and exact prompts are
-published under `training/sft-v2-300k/evaluations/`.
+### Released checkpoint benchmark panel
+
+| Benchmark | Split | Examples | Accuracy | Accuracy (length-normalized) | Evaluation backend |
+|---|---|---:|---:|---:|---|
+| PIQA | validation | 1,838 | **68.01%** | **69.10%** | PyTorch FP16, custom Triton |
+| ARC-Easy | test | 2,376 | **57.24%** | **52.86%** | MLX FP16 |
+| HellaSwag | validation | 10,042 | **33.21%** | **38.74%** | MLX FP16 |
+
+All three benchmarks use zero-shot causal continuation log-likelihood, no
+chat template and a maximum sequence length of 2,048. Accuracy selects the
+choice with the highest total continuation log-likelihood; the normalized
+metric selects by mean continuation log-likelihood per scored token. PIQA was
+evaluated from the native epoch-3 checkpoint. ARC-Easy and HellaSwag were
+evaluated from an FP16 MLX conversion of the same root F32 SafeTensors weights.
+Machine-readable reports are published under `reports/sft-v2-300k/`.
 
 ## Training recipe
 

release_manifest.json

@@ -1,5 +1,29 @@
 {
   "architecture": "tr_hash_moe",
+  "benchmarks": {
+    "arc_easy": {
+      "acc": 0.5723905723905723,
+      "acc_norm": 0.5286195286195287,
+      "backend": "mlx-fp16",
+      "examples": 2376,
+      "split": "test"
+    },
+    "hellaswag": {
+      "acc": 0.33210515833499304,
+      "acc_norm": 0.3873730332603067,
+      "backend": "mlx-fp16",
+      "examples": 10042,
+      "split": "validation"
+    },
+    "piqa": {
+      "acc": 0.6800870511425462,
+      "acc_norm": 0.690968443960827,
+      "backend": "pytorch-fp16-custom-triton",
+      "examples": 1838,
+      "split": "validation"
+    }
+  },
+  "benchmark_protocol": "zero-shot causal choice log-likelihood; no chat template; max_length=2048",
   "behavior_gate_passed": false,
   "dataset": "AETHORIA-AI/TR-HASH-MoE-200M-SFT-v2-300K",
   "dataset_revision": "084a658ec47e4ee872f6d67fdbad3602f599424b",

reports/sft-v2-300k/arc_easy_epoch3.json

@@ -0,0 +1,20 @@
+{
+  "backend": "mlx",
+  "benchmark": "arc_easy",
+  "chat_template_applied": false,
+  "checkpoint_epoch": 3,
+  "checkpoint_step": 5982,
+  "dtype": "float16",
+  "max_length": 2048,
+  "model": "AETHORIA-AI/TR-HASH-MoE-200M-160B-SFT",
+  "results": {
+    "acc": 0.5723905723905723,
+    "acc_norm": 0.5286195286195287,
+    "correct": 1360,
+    "correct_norm": 1256,
+    "examples": 2376
+  },
+  "scoring": "causal_choice_loglikelihood",
+  "split": "test",
+  "zero_shot": true
+}

reports/sft-v2-300k/hellaswag_epoch3.json

@@ -0,0 +1,20 @@
+{
+  "backend": "mlx",
+  "benchmark": "hellaswag",
+  "chat_template_applied": false,
+  "checkpoint_epoch": 3,
+  "checkpoint_step": 5982,
+  "dtype": "float16",
+  "max_length": 2048,
+  "model": "AETHORIA-AI/TR-HASH-MoE-200M-160B-SFT",
+  "results": {
+    "acc": 0.33210515833499304,
+    "acc_norm": 0.3873730332603067,
+    "correct": 3335,
+    "correct_norm": 3890,
+    "examples": 10042
+  },
+  "scoring": "causal_choice_loglikelihood",
+  "split": "validation",
+  "zero_shot": true
+}