arturoornelasb/triadic-gpt2-medium-v8

Triadic GPT-2 Medium (V8)

A 72-bit triadic projection head trained on top of frozen GPT-2 Medium (345M params).

What is this?

This model adds a triadic head to GPT-2 Medium that projects language model embeddings into 72-bit binary codes representing ontological primitives from the Emergent Duality framework.

Each bit corresponds to one of 72 semantic primitives organized in 6 algebraic layers (Boolean, Fuzzy, Ordinal, Modal, Trivalent, Probabilistic). The head learns to map continuous embeddings to discrete binary codes that preserve ontological structure.

Files

File	Description
best.pt	Final trained weights (best checkpoint at 250K steps)
step_30000.pt	Base checkpoint before phase transition (for X-ray reproducibility)
run_config.json	Full training configuration
gold_extended_v7.json	Gold standard: 72 primitives + extended concept targets

X-ray Reproducibility

step_30000.pt is the base checkpoint used for the telescopic X-ray analysis that revealed the cascade phase transition. To reproduce:

1. Resume training from step_30000.pt with save-every-5-steps
2. Run reptimeline on the resulting checkpoints
3. Observe the cascade activation at step ~31K (65/72 bits change in waves over ~145 steps)

See train_v8_xray.bat for exact commands.

Training

• Base model: GPT-2 Medium (345M params, frozen)
• Head: Deep triadic head (561K trainable params)
• Activation: IFSQ (Integer Finite Scalar Quantization)
• Dataset: WikiText-103
• Steps: 250,000
• Bit accuracy: 85.3% (test)
• Subsumption: 95.7% (test)
• Dead bits: 25/72
• GPU: NVIDIA RTX 4060 Ti 16 GB

Key findings

• Phase transition (cascade) at step ~31K with wave propagation over ~145 steps
• 65/72 bits change in waves of 1-7 bits; 7 scaffold bits remain stable throughout
• Wave-FEP fit: R²=0.943, overdamped regime (d/w=1.34), dAIC=+92.2 (wave preferred over exponential)
• Cross-architecture cascade pattern confirmed with v9 (GPT-Neo 125M)

Usage

import torch
from huggingface_hub import hf_hub_download

# Download checkpoint
ckpt_path = hf_hub_download("arturoornelasb/triadic-gpt2-medium-v8", "best.pt")
checkpoint = torch.load(ckpt_path, map_location="cpu")

# Or use with triadic_extractor (pip install reptimeline)
from triadic_extractor import TriadicExtractor
extractor = TriadicExtractor(n_bits=72)
snap = extractor.extract(ckpt_path, concepts, device='cuda')

Links

• Paper: Triadic Duality (Zenodo)
• Code: triadic-emergent-duality (GitHub)
• V9 weights: triadic-gptneo-125m-v9 (HuggingFace)
• reptimeline: PyPI | GitHub
• Triadic Neurosymbolic Engine: Zenodo

Citation

@misc{ornelas2026triadic,
  author       = {Ornelas Brand, J. Arturo},
  title        = {Triadic Duality: Ontological Primitives as Emergent Algebraic Structure in Neural Representations},
  year         = {2026},
  doi          = {10.5281/zenodo.19375167},
  publisher    = {Zenodo},
}