README.md · dimitarpg13/semsimula-fock-attention at main

Correct the conservativity claim and scope the geodesic-residual reading
8e6d74f verified about 1 month ago
25.4 kB
---
pipeline_tag: text-generation
library_name: pytorch
language:
  - en
license: cc-by-4.0
tags:
  - research
  - semsimula
  - conservative-language-model
  - scalar-potential
  - lagrangian-mechanics
  - energy-based-model
  - physics-informed
  - fock-attention
  - parflm
  - parf
  - multi-channel-xi
  - direct-exchange
  - feynman-diagram
  - damped-riemannian-geometry
  - riemannian-geometry
  - riemannian-geodesics
  - tinystories
  - non-transformer
datasets:
  - roneneldan/TinyStories
metrics:
  - perplexity
model-index:
  - name: semsimula-fock-attention
    results:
      - task:
          type: text-generation
        dataset:
          type: roneneldan/TinyStories
          name: TinyStories
          split: validation
        metrics:
          - type: perplexity
            value: 9.42
            name: Validation Perplexity
---
# Fock Attention with MLP V_theta (Direct Token-to-Token Exchange Force Language Model)
The **Fock Attention with MLP \\(V_\theta\\)** model (previously presented on this page simply as "Fock Attention") implements the Section 5.1 Feynman diagram as a literal non-conservative force: each token emits a virtual photon carrying a key and payload, and each token absorbs with a query. The exchange coupling \\(\alpha_{ij} = \text{softmax}_j(q_i \cdot k_j / \sqrt{d_k})\\) and the force \\(F_i = \sum_j \alpha_{ij} \cdot v_j\\). This is the \\(\lambda = 0\\) **(instantaneous exchange) limit** of the Fock mechanism -- no registers, no persistence, no creation/destruction gates.

This "Route 2" across the Conservative Obstruction is the \\(O(T^2)\\) counterpart to the Fock register pool's \\(O(1)\\) "Route 1". It achieves **9.42 PPL** on TinyStories, honest and leak-free. After correcting an earlier comparison that used the leaky-checkpoint Fock-PARFLM v2.1 number (9.30), this model is actually **0.28 PPL better** than the honest, leak-fixed register-based Fock-PARFLM v2.1 (9.70) -- see [Evaluation Results](#evaluation-results) for the full, corrected table.

The card name now spells out **"with MLP \\(V_\theta\\)"** on purpose: as the [V_theta vs. Exchange Mechanism](#v_theta-vs-exchange-mechanism-what-actually-drives-ppl-on-tinystories) section below shows, on TinyStories the single-body potential's *shape* -- not the choice between register-mediated and direct-exchange Fock mechanisms -- is the dominant factor separating this model's 9.42 PPL from the family-best 9.04 (depth-conditioned anisotropic-Gaussian \\(V_\theta\\)).
Part of the [Semantic Simulation](https://doi.org/10.5281/zenodo.19712427) framework.
## Table of Contents
- [Model Details](#model-details)
- [Architecture](#architecture)
- [Why Not a Transformer?](#why-not-a-transformer)
- [Geometric Capabilities](#geometric-capabilities)
- [How to Get Started](#how-to-get-started)
- [Training Details](#training-details)
- [Evaluation Results](#evaluation-results)
- [SPLM Family Overview](#splm-family-overview)
- [Bias, Risks, and Limitations](#bias-risks-and-limitations)
- [Citation](#citation)
- [Environmental Impact](#environmental-impact)
## Model Details
### Model Description
The Fock Attention PARFLM extends the Multi-Xi PARFLM with a **direct token-to-token exchange force** inspired by Feynman's virtual particle exchange diagram. Unlike the register-based Fock-PARFLM which uses persistent auxiliary state, this variant implements instantaneous exchange:

1. Token \\(j\\) emits: key \\(k_j = W_K h_j\\), payload \\(v_j = W_V h_j\\)
2. Token \\(i\\) absorbs: query \\(q_i = W_Q h_i\\)
3. Coupling: \\(\alpha_{ij} = \text{softmax}_j(q_i \cdot k_j / \sqrt{d_k})\\)
4. Exchange force on token i: \\(F_i = \sum_j \alpha_{ij} \cdot v_j\\)

The force is injected post-Verlet step as a non-conservative addition:


$$h \mathrel{+}= \frac{\Delta t^2}{m} \cdot \tanh(s_{\text{ex}}) \cdot F_{\text{exchange}}$$


The learned exchange scale \\(\tanh(s_{\text{ex}})\\) converges to approximately -0.32 (repulsive), meaning the exchange force *pushes tokens apart* in hidden-state space rather than attracting them.
- **Developed by:** Dimitar P. Gueorguiev (Independent Researcher)
- **Model type:** Conservative autoregressive LM with direct exchange force
- **Language:** English
- **License:** [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)
### Model Sources
- **Paper:** [Semantic Simulation: A Prescriptive Lagrangian Framework for Efficient Semantic Inference](https://doi.org/10.5281/zenodo.19712427) (Section 17c)
- **Repository:** [github.com/dimitarpg13/semsimula-paper](https://github.com/dimitarpg13/semsimula-paper)
- **Model source code:** [`notebooks/conservative_arch/parf/model_fock_attention.py`](https://github.com/dimitarpg13/semsimula-paper/blob/main/notebooks/conservative_arch/parf/model_fock_attention.py)
## Architecture
```
Input tokens x_1, ..., x_T
       |
   Embedding E[x] + positional encoding
       |
   For each of L=8 integration steps:
       |
       +-- K-EMA channels: xi^(k)_t = causal_ema(h, alpha_k)   [K=4 channels]
       |
       +-- Single-body: V_theta([xi_1..xi_K, h]) -> R           [3-layer MLP]
       |
       +-- Sparse PARF: V_phi(h_t, h_s)                         [top-k=8 routing]
       |
       +-- Conservative force: f = -grad_h(V_theta + V_phi)     [autograd]
       |
       +-- Damped Stormer-Verlet step: h += (h-h_prev)/(1+dt*gamma) + dt^2*f/(m*(1+dt*gamma))
       |
       +-- Direct exchange force (Feynman diagram):
       |     q_i = W_Q * h_t     [4 heads, d_k=32]
       |     k_j = W_K * h_s
       |     v_j = W_V * h_s
       |     alpha_ij = softmax_j(q_i . k_j / sqrt(d_k))        [causal mask]
       |     F_i = sum_j alpha_ij * v_j
       |     h += (dt^2/m) * tanh(s_ex) * F_i                   [non-conservative]
       |
       +-- LayerNorm(h)
       |
   Logits = h @ E^T                                            [tied embeddings]
```
| Parameter | Value |
|---|---|
| Hidden dim (d) | 256 |
| Layers (L) | 8 |
| \\(V_\theta\\) hidden / depth | 1024 / 3 |
| Xi channels (K) | 4 |
| \\(V_\phi\\) kind | structural_competitive |
| \\(V_\phi\\) hidden (H) | 128 |
| Sparse routing top_k | 8 |
| Exchange heads | 4 |
| Exchange d_k | 32 |
| Learned exchange scale | \\(\tanh(s_{\text{ex}})\\) ~ -0.32 (repulsive) |
| Mass model | logfreq |
| Damping \\(\gamma\\) | 0.30 nominal, ~0.033 effective (LayerNorm prevents compounding; see note below) |
| Total parameters | **16,714,708** |
**Effective damping.** The nominal \\(\gamma = 0.30\\) overstates the true dissipation. The LayerNorm applied after each integration step rescales the hidden state, absorbing most of the velocity decay. The dynamics are therefore **heavily underdamped** even at this nominal value. Gamma-sweep experiments on the OpenWebText-scale Fock-PARFLM variant confirm that the effective damping γ<sub>eff</sub> is much smaller than the nominal coefficient.
### Key Design Properties
- **Two routes across the obstruction:** This model takes "Route 2" (direct exchange, \\(O(T^2)\\)) while the Fock-PARFLM takes "Route 1" (register-mediated, \\(O(1)\\)). Both achieve comparable, honest PPL (9.42 vs. 9.70).
- **Repulsive exchange:** The learned scale is negative, meaning the exchange force *diversifies* token representations rather than collapsing them.
- **Minimal overhead:** Only ~131K parameters for the exchange mechanism (Q/K/V projections + scale scalar).
- **Otherwise conservative:** The core dynamics \\(V_\theta + V_\phi\\) remain fully conservative; the exchange force is the only non-conservative component.
### V_theta vs. Exchange Mechanism: What Actually Drives PPL on TinyStories

This model's single-body potential \\(V_\theta\\) is a plain 3-layer MLP -- the same unbounded, black-box shape used in the original [Fock-PARFLM](https://huggingface.co/dimitarpg13/semsimula-fock-parflm). Its most visible design feature is the direct exchange force above, so it is tempting to credit that Feynman-diagram mechanism for most of the PPL gain over the no-Fock [Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi) baseline (12.06 -> 9.42). The family's own ablations, holding one factor fixed at a time, point the other way:
| \\(V_\theta\\) shape | Exchange / Fock mechanism | Honest PPL |
|---|---|---|
| MLP | none ([Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi)) | 12.06 |
| MLP | register-based Fock v2.1 ([Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm)) | 9.70 |
| MLP | direct exchange (**this model**) | 9.42 |
| Isotropic Gaussian | register-based Fock v2.1 ([sibling](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-depthcond-vtheta)) | 16.33 |
| Anisotropic Gaussian | register-based Fock v2.1 ([sibling](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta)) | 9.04 |
| Anisotropic Gaussian | register-based Fock v2.1 + first-order integrator ([Fock-G1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-fock-g1)) | 8.95 |

Holding the exchange mechanism fixed at "register-based Fock v2.1" and varying only \\(V_\theta\\)'s shape swings PPL by more than 7 points (16.33 down to 8.95). Holding \\(V_\theta\\) fixed at MLP and varying only the exchange mechanism (none -> registers -> direct exchange) swings PPL by about 2.6 points -- but the choice *between* the two Fock-mechanism flavours (registers vs. direct exchange) is worth only \\(\approx\\)0.3 points, and direct exchange (this model) actually wins that comparison.

**On TinyStories, \\(V_\theta\\)'s profile is doing most of the work; the specific exchange mechanism is a second-order effect.** Whether the anisotropic-Gaussian-beats-MLP ordering seen here survives at OpenWebText scale is still an open question: only the anisotropic-Gaussian \\(V_\theta\\) has been scaled up to OpenWebText so far (see the [gamma-sweep family](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep)), so there is currently no MLP-\\(V_\theta\\) OpenWebText-scale checkpoint to compare against directly.
## Why Not a Transformer?
The Fock Attention PARFLM is **not based on the Transformer architecture**. There are no Transformer-style FFN towers. The conservative core is driven by two small scalar-potential MLPs — \\(V_\theta\\) (≈3.4M params) and \\(V_\phi\\) (≈19K params). The direct exchange force adds only ≈131K parameters (Q/K/V projections).
Key structural differences from Transformers:
| Property | Transformer (GPT-2 small) | Fock Attention (this model) |
|---|---|---|
| Architecture | Self-attention + FFN blocks | Scalar-potential gradient flow + direct exchange |
| Core computation | 50.3M (MLP) + 28.3M (attention) | 3.4M \\(V_\theta\\) + 19K \\(V_\phi\\) + 131K (exchange) |
| Runtime state per token | \\(O(T)\\) — KV-cache grows linearly | \\(O(T)\\) — exchange force is \\(O(T^2)\\) |
| Total parameters | 124M | 16.7M |
| Pairwise token interaction | \\(O(T^2)\\) dense attention | \\(O(T^2)\\) direct exchange + \\(O(Tk)\\) sparse PARF |
**Note:** Because this model uses a direct token-to-token exchange force (the "Route 2" across the Conservative Obstruction), its runtime cost is \\(O(T^2)\\) like attention. For fully \\(O(1)\\) inference, see the register-based [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) ("Route 1"), which achieves an honest 9.70 PPL vs. this model's 9.42 PPL -- i.e. \\(O(1)\\) inference here comes at a small PPL cost relative to this model's \\(O(T^2)\\) direct exchange, not the other way around (see the [causal-leak notice](https://huggingface.co/dimitarpg13/semsimula-fock-parflm#readme) on that card for why an earlier version of this comparison had the direction reversed).
![Runtime information capacity vs sequence length](runtime_information_capacity.png)
## Geometric Capabilities
**Note:** This model uses a direct \\(O(T^2)\\) exchange force that is non-conservative, breaking the full Riemannian guarantee. The full Riemannian geometry — Jacobi metric, computable geodesics, curvature-based hallucination detection, native chain-of-thought — is available only in the purely conservative variants: [Multi-Xi SPLM](https://huggingface.co/dimitarpg13/semsimula-splm-multixi) and [Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi). ([Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) was listed here until 2026-09-28 and has been removed: it ships with the reverse channel enabled, which is not a gradient of anything in \\(h\\).). See Section 18d and Section 23 of the [paper](https://doi.org/10.5281/zenodo.19712427) for details.
### Damped Riemannian Geometry (June 2026 update)
A [Riemannian Geometry Diagnostic Battery](https://github.com/dimitarpg13/semsimula-paper/blob/main/notebooks/conservative_arch/scaleup/colab_riemannian_diagnostic.ipynb) run on all three SPLM-family checkpoints confirmed that while the force field \\(f = -\nabla V_\theta\\) is conservative, the full dynamics are dominated by the **damping term** \\(\gamma\\) in the integrator. Key findings:

- **Metric validity (Arm 1):** The layer-dependent conformal factor \\(\Omega^2 = 2T_\ell \cdot m > 0\\) at 100% of positions — the Riemannian metric is well-defined everywhere.
- **Geodesic compliance (Arm 2):** Undamped geodesics predict the *wrong direction* (compliance \\(\approx -0.4\\)); the damped geodesic equation (with friction \\(-\gamma \dot{\gamma}^k\\)) is required.
- **Energy dissipation (Arm 4):** Energy decays monotonically across layers (SPLM: 172% drift), consistent with the designed damping — not a numerical artefact.
- **Asymmetry (Arm 5):** The layer map is strongly asymmetric (\\(R^2_{\text{sym}} \ll 0\\)), expected for damped dynamics + LayerNorm.

The theoretical framework has been updated from undamped Maupertuis-Jacobi to **damped Riemannian geometry** with a contact Hamiltonian interpretation. See the [companion note](https://github.com/dimitarpg13/semsimula-paper/blob/main/companion_notes/Exploiting_the_Riemannian_geometry_of_conservative_language_models.md) for full details.

## How to Get Started

```python
# Clone the companion repository for full source code
# git clone https://github.com/dimitarpg13/semsimula-paper.git
# cd semsimula-paper/notebooks/conservative_arch
import torch
import sys
sys.path.insert(0, "parf")
sys.path.insert(0, "multixi")
sys.path.insert(0, "energetic_minima")
sys.path.insert(0, "sarf_mass_variant")
from parf.model_fock_attention import FockAttentionPARFLM, FockAttentionConfig
config = FockAttentionConfig(
    vocab_size=50257,
    d=256,
    n_layers=8,
    v_hidden=1024,
    v_depth=3,
    max_len=1024,
    block_size=512,
    gamma=0.30,
    xi_channels=4,
    v_phi_kind="structural_competitive",
    v_phi_hidden=128,
    top_k=8,
    exchange_n_heads=4,
    exchange_d_k=32,
)
model = FockAttentionPARFLM(config)
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")
# Forward pass
x = torch.randint(0, 50257, (1, 64))
logits, loss = model(x, targets=x)
```

### Available Checkpoint

A trained checkpoint (PPL 9.42, 16k steps) is included in this repository:

| File | Description |
|---|---|
| `checkpoint/model.pt` | Full model state dict (64 MB) |
| `training_log.jsonl` | Per-step training metrics |
| `loss_curve.png` | Training/validation loss plot |
| `training_summary.md` | Hyperparameters and final metrics |

To load the checkpoint:

```python
from huggingface_hub import hf_hub_download
import torch
# Download checkpoint
ckpt_path = hf_hub_download(
    repo_id="dimitarpg13/semsimula-fock-attention",
    filename="checkpoint/model.pt",
)
# Load into model (after creating model as above)
state = torch.load(ckpt_path, map_location="cpu")
model.load_state_dict(state["model_state_dict"])
model.eval()
```

## Training Details

### Training Data

[TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) -- GPT-2 BPE tokenization. Training cap: 5M tokens.

### Training Procedure

| Hyperparameter | Value |
|---|---|
| Optimizer | AdamW |
| Learning rate | 5e-4 (cosine decay) |
| Warmup steps | 400 |
| Weight decay | 0.01 |
| Gradient clipping | 1.0 |
| Batch size | 16 |
| Block size | 512 |
| Training steps | 16,000 |
| Memory optimisation | Level-2 grad checkpoint |
| Hardware | A100/H100 (Google Colab) |

### Training Script

[`notebooks/conservative_arch/scaleup/train_fock_attention_scaleup.py`](https://github.com/dimitarpg13/semsimula-paper/blob/main/notebooks/conservative_arch/scaleup/train_fock_attention_scaleup.py)

### Colab Notebook

[`notebooks/conservative_arch/scaleup/colab_fock_attention_h128.ipynb`](https://github.com/dimitarpg13/semsimula-paper/blob/main/notebooks/conservative_arch/scaleup/colab_fock_attention_h128.ipynb) — 4-arm sweep over head count and schedule with live progress display, saves results to Google Drive.

### Training Results

[`notebooks/conservative_arch/scaleup/results/semsimula_fock_attention_h128/`](https://github.com/dimitarpg13/semsimula-paper/tree/main/notebooks/conservative_arch/scaleup/results/semsimula_fock_attention_h128) — training logs, loss curves, and experiment report.

### Head / d_k Sweep
| Arm | Heads | d_k | Steps | PPL |
|---|---|---|---|---|
| direct_K4_h1_8k | 1 | 64 | 8k | 11.48 |
| direct_K4_h4_8k | 4 | 32 | 8k | 10.93 |
| direct_K8_h4_8k | 8 | 32 | 8k | 11.65 |
| **direct_K4_h4_16k** | **4** | **32** | **16k** | **9.42** |
## Evaluation Results
### TinyStories Validation Perplexity
| Model | PPL | Params | Gap vs Attention |
|---|---|---|---|
| Matched Attention (baseline) | **7.81** | 19.5M | -- |
| [Hybrid SPLM+Attn](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) | **8.50** | ~19.0M | +0.69 |
| **Fock Attention (MLP V_theta)** (this model) | **9.42** | 16.7M | +1.61 |
| [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) | **9.70** | 17.4M | +1.89 |
| [Multi-Xi SPLM](https://huggingface.co/dimitarpg13/semsimula-splm-multixi) | **11.51** | 16.5M | +3.70 |
| [Multi-Xi PARFLM](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi) | **12.06** | 17.6M | +4.25 |
**Corrected comparison:** earlier versions of this card compared against Fock-PARFLM v2.1's leaky-checkpoint PPL of 9.30, concluding that "register persistence provides a small advantage." That checkpoint has since been re-trained with the causal-leak fix; its honest PPL is **9.70**, which means Fock Attention (this model) is actually **0.28 PPL better**, not worse (see the [causal-leak notice](https://huggingface.co/dimitarpg13/semsimula-fock-parflm#readme) on the Fock-PARFLM v2.1 card for the full account). Fock Attention also has the smallest parameter count in the family (16.7M) and the simplest Fock mechanism (no creation/destruction gates, no stack discipline). As the [V_theta vs. Exchange Mechanism](#v_theta-vs-exchange-mechanism-what-actually-drives-ppl-on-tinystories) section above argues, the 0.62 PPL gap between this model and the family-best anisotropic-Gaussian variant (9.04) is better explained by \\(V_\theta\\)'s shape than by the exchange mechanism.

## SPLM Family Overview

This model is part of the **Semantic Simulation SPLM family**:

| Model | Design | Corpus | PPL | HuggingFace |
|---|---|---|---|---|
| Multi-Xi SPLM (MLP) | Pure scalar potential | TinyStories | 11.51 | [semsimula-splm-multixi](https://huggingface.co/dimitarpg13/semsimula-splm-multixi) |
| Multi-Xi SPLM (SQ3) | Structured scalar potential | TinyStories | 13.33 | [semsimula-splm-multixi-structured-vtheta](https://huggingface.co/dimitarpg13/semsimula-splm-multixi-structured-vtheta) |
| Multi-Xi PARFLM (MLP) | Scalar + pairwise forces | TinyStories | 12.06 | [semsimula-parflm-multixi](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi) |
| Multi-Xi PARFLM (SQ3) | Structured scalar + pairwise | TinyStories | 12.27 | [semsimula-parflm-multixi-structured-vtheta](https://huggingface.co/dimitarpg13/semsimula-parflm-multixi-structured-vtheta) |
| Fock-PARFLM v2.1 (MLP) | PARFLM + Fock registers | TinyStories | 9.70 | [semsimula-fock-parflm](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) |
| Fock-PARFLM v2.1 (SQ3) | Structured + pairwise + Fock | TinyStories | 10.90 | [semsimula-fock-parflm-structured-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-structured-vtheta) |
| Fock-PARFLM v2.1 (depth-cond. isotropic Gaussian) | Bounded multi-context + pairwise + Fock | TinyStories | 16.33 | [semsimula-fock-parflm-depthcond-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-depthcond-vtheta) |
| Fock-PARFLM v2.1 (depth-cond. anisotropic Gaussian + fock-reg) | Bounded, ellipsoidal multi-context + pairwise + Fock | TinyStories | 9.04 | [semsimula-fock-parflm-anisogaussian-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta) |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, first-order/Fock-G1) | Same as above, gradient-flow integrator | TinyStories | 8.95 | [semsimula-fock-parflm-anisogaussian-vtheta-fock-g1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-fock-g1) |
| **Fock Attention (MLP V_theta)** | **Fock + attention** | **TinyStories** | **9.42** | **this model** |
| Hybrid SPLM+Attn | Attention + SPLM refinement | TinyStories | 8.50 | [semsimula-hybrid-splm](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=384) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 278.27 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep) |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=768) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 326.97 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep) |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=1024) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 244.23 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d1024-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d1024-gammasweep) |
| Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, Verlet instability, d=384) | Same architecture, two full-run attempts — SCAF stiffness audit identifies structural Verlet instability | OpenWebText | 184.11 / 211.63 (both runs stalled, not final) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-verlet-instability](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-verlet-instability) |
**Collection:** [Semantic Simulation SPLM Model Family](https://huggingface.co/collections/dimitarpg13/semantic-simulation-splm-model-family)
## Bias, Risks, and Limitations
- **Research checkpoint only.** Proof-of-concept for direct exchange force as a non-conservative complement to conservative dynamics.
- **TinyStories only.** Trained exclusively on synthetic children's stories (~5M tokens).
- **English only.** No multilingual capability.
- **Small scale.** 16.7M parameters, 256-dim hidden states.
- **No safety training.** No RLHF, DPO, or safety filtering has been applied.
## Citation
```bibtex
@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Efficient Semantic Inference --- A Conservative-by-
               Construction Language Model and the Shared-Potential
               Separator, with a Correspondence to Joint Embedding
               Predictive Architectures},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v15 (Jun 7, 2026).
               Companion code repository (DOI 10.5281/zenodo.20579561):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}
```
## Environmental Impact
- **Hardware:** NVIDIA A100/H100 (Google Colab)
- **Training time:** ~8 hours (16,000 steps with gradient checkpointing)
- **Carbon footprint:** Estimated < 3 kg CO2
> **Correction (2026-09-27) — integrator name.** This model's layer step was described here as a *damped Euler* step. It is not. The update is
> `h_new = h + (h - h_prev)/(1 + dt*gamma) + dt^2*f/(m*(1 + dt*gamma))`, which carries no velocity state at all: the velocity is the position difference `h - h_prev`. Undamped this is `h_{n+1} = 2*h_n - h_{n-1} + dt^2*f/m`, i.e. **Störmer–Verlet in position form**. It is also not *velocity*-Verlet, which carries an explicit velocity through half-kick/drift/half-kick.
> The distinction matters because the \\(\omega \cdot dt < 2\\) stability wall is the Störmer/leapfrog bound and applies to this lineage; the semi-implicit Euler models in the same collection were never subject to it. Only the name was wrong — no measurement on this card changes.