Rename Fock-Attention to 'Fock Attention (MLP V_theta)', fix stale pre-causal-leak-fix 9.30 PPL comparisons, add V_theta-vs-exchange-mechanism insight

README.md

@@ -42,11 +42,13 @@ model-index:
             name: Validation Perplexity
 ---
 
-# Fock Attention PARFLM (Direct Token-to-Token Exchange Force Language Model)
+# Fock Attention with MLP V_theta (Direct Token-to-Token Exchange Force Language Model)
 
-The **Fock Attention PARFLM** implements the Section 5.1 Feynman diagram as a literal non-conservative force: each token emits a virtual photon carrying a key and payload, and each token absorbs with a query. The exchange coupling \\(\alpha_{ij} = \text{softmax}_j(q_i \cdot k_j / \sqrt{d_k})\\) and the force \\(F_i = \sum_j \alpha_{ij} \cdot v_j\\). This is the \\(\lambda = 0\\) **(instantaneous exchange) limit** of the Fock mechanism -- no registers, no persistence, no creation/destruction gates.
+The **Fock Attention with MLP \\(V_\theta\\)** model (previously presented on this page simply as "Fock Attention") implements the Section 5.1 Feynman diagram as a literal non-conservative force: each token emits a virtual photon carrying a key and payload, and each token absorbs with a query. The exchange coupling \\(\alpha_{ij} = \text{softmax}_j(q_i \cdot k_j / \sqrt{d_k})\\) and the force \\(F_i = \sum_j \alpha_{ij} \cdot v_j\\). This is the \\(\lambda = 0\\) **(instantaneous exchange) limit** of the Fock mechanism -- no registers, no persistence, no creation/destruction gates.
 
-This "Route 2" across the Conservative Obstruction is the \\(O(T^2)\\) counterpart to the Fock register pool's \\(O(1)\\) "Route 1". It achieves **9.42 PPL** on TinyStories -- slightly behind the register-based Fock-PARFLM v2.1 (9.30), confirming that register persistence provides a small but consistent advantage once routing is fixed.
+This "Route 2" across the Conservative Obstruction is the \\(O(T^2)\\) counterpart to the Fock register pool's \\(O(1)\\) "Route 1". It achieves **9.42 PPL** on TinyStories, honest and leak-free. After correcting an earlier comparison that used the leaky-checkpoint Fock-PARFLM v2.1 number (9.30), this model is actually **0.28 PPL better** than the honest, leak-fixed register-based Fock-PARFLM v2.1 (9.70) -- see [Evaluation Results](#evaluation-results) for the full, corrected table.
+
+The card name now spells out **"with MLP \\(V_\theta\\)"** on purpose: as the [V_theta vs. Exchange Mechanism](#v_theta-vs-exchange-mechanism-what-actually-drives-ppl-on-tinystories) section below shows, on TinyStories the single-body potential's *shape* -- not the choice between register-mediated and direct-exchange Fock mechanisms -- is the dominant factor separating this model's 9.42 PPL from the family-best 9.04 (depth-conditioned anisotropic-Gaussian \\(V_\theta\\)).
 
 Part of the [Semantic Simulation](https://doi.org/10.5281/zenodo.19712427) framework.
 
@@ -146,11 +148,28 @@ Input tokens x_1, ..., x_T
 
 ### Key Design Properties
 
-- **Two routes across the obstruction:** This model takes "Route 2" (direct exchange, \\(O(T^2)\\)) while the Fock-PARFLM takes "Route 1" (register-mediated, \\(O(1)\\)). Both achieve comparable PPL.
+- **Two routes across the obstruction:** This model takes "Route 2" (direct exchange, \\(O(T^2)\\)) while the Fock-PARFLM takes "Route 1" (register-mediated, \\(O(1)\\)). Both achieve comparable, honest PPL (9.42 vs. 9.70).
 - **Repulsive exchange:** The learned scale is negative, meaning the exchange force *diversifies* token representations rather than collapsing them.
 - **Minimal overhead:** Only ~131K parameters for the exchange mechanism (Q/K/V projections + scale scalar).
 - **Otherwise conservative:** The core dynamics \\(V_\theta + V_\phi\\) remain fully conservative; the exchange force is the only non-conservative component.
 
+### V_theta vs. Exchange Mechanism: What Actually Drives PPL on TinyStories
+
+This model's single-body potential \\(V_\theta\\) is a plain 3-layer MLP -- the same unbounded, black-box shape used in the original [Fock-PARFLM](https://huggingface.co/dimitarpg13/semsimula-fock-parflm). Its most visible design feature is the direct exchange force above, so it is tempting to credit that Feynman-diagram mechanism for most of the PPL gain over the no-Fock [Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi) baseline (12.06 -> 9.42). The family's own ablations, holding one factor fixed at a time, point the other way:
+
+| \\(V_\theta\\) shape | Exchange / Fock mechanism | Honest PPL |
+|---|---|---|
+| MLP | none ([Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi)) | 12.06 |
+| MLP | register-based Fock v2.1 ([Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm)) | 9.70 |
+| MLP | direct exchange (**this model**) | 9.42 |
+| Isotropic Gaussian | register-based Fock v2.1 ([sibling](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-depthcond-vtheta)) | 16.33 |
+| Anisotropic Gaussian | register-based Fock v2.1 ([sibling](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta)) | 9.04 |
+| Anisotropic Gaussian | register-based Fock v2.1 + first-order integrator ([Fock-G1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-fock-g1)) | 8.95 |
+
+Holding the exchange mechanism fixed at "register-based Fock v2.1" and varying only \\(V_\theta\\)'s shape swings PPL by more than 7 points (16.33 down to 8.95). Holding \\(V_\theta\\) fixed at MLP and varying only the exchange mechanism (none -> registers -> direct exchange) swings PPL by about 2.6 points -- but the choice *between* the two Fock-mechanism flavours (registers vs. direct exchange) is worth only \\(\approx\\)0.3 points, and direct exchange (this model) actually wins that comparison.
+
+**On TinyStories, \\(V_\theta\\)'s profile is doing most of the work; the specific exchange mechanism is a second-order effect.** Whether the anisotropic-Gaussian-beats-MLP ordering seen here survives at OpenWebText scale is still an open question: only the anisotropic-Gaussian \\(V_\theta\\) has been scaled up to OpenWebText so far (see the [gamma-sweep family](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep)), so there is currently no MLP-\\(V_\theta\\) OpenWebText-scale checkpoint to compare against directly.
+
 ## Why Not a Transformer?
 
 The Fock Attention PARFLM is **not based on the Transformer architecture**. There are no Transformer-style FFN towers. The conservative core is driven by two small scalar-potential MLPs — \\(V_\theta\\) (≈3.4M params) and \\(V_\phi\\) (≈19K params). The direct exchange force adds only ≈131K parameters (Q/K/V projections).
@@ -165,7 +184,7 @@ Key structural differences from Transformers:
 | Total parameters | 124M | 16.7M |
 | Pairwise token interaction | \\(O(T^2)\\) dense attention | \\(O(T^2)\\) direct exchange + \\(O(Tk)\\) sparse PARF |
 
-**Note:** Because this model uses a direct token-to-token exchange force (the "Route 2" across the Conservative Obstruction), its runtime cost is \\(O(T^2)\\) like attention. For fully \\(O(1)\\) inference with comparable PPL, see the register-based [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) ("Route 1"), which achieves 9.30 PPL vs this model's 9.42 PPL.
+**Note:** Because this model uses a direct token-to-token exchange force (the "Route 2" across the Conservative Obstruction), its runtime cost is \\(O(T^2)\\) like attention. For fully \\(O(1)\\) inference, see the register-based [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) ("Route 1"), which achieves an honest 9.70 PPL vs. this model's 9.42 PPL -- i.e. \\(O(1)\\) inference here comes at a small PPL cost relative to this model's \\(O(T^2)\\) direct exchange, not the other way around (see the [causal-leak notice](https://huggingface.co/dimitarpg13/semsimula-fock-parflm#readme) on that card for why an earlier version of this comparison had the direction reversed).
 
 ![Runtime information capacity vs sequence length](runtime_information_capacity.png)
 
@@ -305,12 +324,12 @@ model.eval()
 |---|---|---|---|
 | Matched Attention (baseline) | **7.81** | 19.5M | -- |
 | [Hybrid SPLM+Attn](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) | **8.50** | ~19.0M | +0.69 |
-| [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) | **9.30** | 17.4M | +1.49 |
-| **Fock Attention** (this model) | **9.42** | 16.7M | +1.61 |
-| [Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi) | **12.06** | 17.6M | +4.25 |
+| **Fock Attention (MLP V_theta)** (this model) | **9.42** | 16.7M | +1.61 |
+| [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) | **9.70** | 17.4M | +1.89 |
 | [Multi-Xi SPLM](https://huggingface.co/dimitarpg13/semsimula-splm-multixi) | **11.51** | 16.5M | +3.70 |
+| [Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi) | **12.06** | 17.6M | +4.25 |
 
-The Fock Attention is 0.12 PPL behind the register-based Fock-PARFLM v2.1, confirming that register persistence provides a small advantage. However, Fock Attention has the smallest parameter count in the family (16.7M) and the simplest Fock mechanism (no creation/destruction gates, no stack discipline).
+**Corrected comparison:** earlier versions of this card compared against Fock-PARFLM v2.1's leaky-checkpoint PPL of 9.30, concluding that "register persistence provides a small advantage." That checkpoint has since been re-trained with the causal-leak fix; its honest PPL is **9.70**, which means Fock Attention (this model) is actually **0.28 PPL better**, not worse (see the [causal-leak notice](https://huggingface.co/dimitarpg13/semsimula-fock-parflm#readme) on the Fock-PARFLM v2.1 card for the full account). Fock Attention also has the smallest parameter count in the family (16.7M) and the simplest Fock mechanism (no creation/destruction gates, no stack discipline). As the [V_theta vs. Exchange Mechanism](#v_theta-vs-exchange-mechanism-what-actually-drives-ppl-on-tinystories) section above argues, the 0.62 PPL gap between this model and the family-best anisotropic-Gaussian variant (9.04) is better explained by \\(V_\theta\\)'s shape than by the exchange mechanism.
 
 ## SPLM Family Overview
 
@@ -327,7 +346,7 @@ This model is part of the **Semantic Simulation SPLM family**:
 | Fock-PARFLM v2.1 (depth-cond. isotropic Gaussian) | Bounded multi-context + pairwise + Fock | TinyStories | 16.33 | [semsimula-fock-parflm-depthcond-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-depthcond-vtheta) |
 | Fock-PARFLM v2.1 (depth-cond. anisotropic Gaussian + fock-reg) | Bounded, ellipsoidal multi-context + pairwise + Fock | TinyStories | 9.04 | [semsimula-fock-parflm-anisogaussian-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta) |
 | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, first-order/Fock-G1) | Same as above, gradient-flow integrator | TinyStories | 8.95 | [semsimula-fock-parflm-anisogaussian-vtheta-fock-g1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-fock-g1) |
-| **Fock-Attention** | **Fock + attention** | **TinyStories** | **9.42** | **this model** |
+| **Fock Attention (MLP V_theta)** | **Fock + attention** | **TinyStories** | **9.42** | **this model** |
 | Hybrid SPLM+Attn | Attention + SPLM refinement | TinyStories | 8.50 | [semsimula-hybrid-splm](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) |
 | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=384) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 278.27 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep) |
 | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=768) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 326.97 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep) |