arturoornelasb/triadic-gpt2-medium

GitHub: arturoornelasb/triadic-microgpt | Paper: Emergent Duality (2026) | PyPI: triadic-head

Triadic GPT-2 Medium (355M + 72-bit triadic head)

GPT-2 Medium (355M parameters) augmented with a triadic projection head that produces discrete prime-factor semantic signatures alongside standard next-token predictions.

The backbone is frozen during training -- only the 73,728-parameter triadic head is trained, preserving the base model's language ability (PPL identical to baseline).

Checkpoints

Two training variants are included:

Variant	Bit Accuracy	Dead Bits	Description
ternary_v2/model_best.pt	91.3%	26	Ternary activation (v2)
sparsity_v2/model_best.pt	91.2%	26	Sparsity-regularized (v2)

Architecture

• Base model: GPT-2 Medium (355M params, frozen)
• Triadic head: Linear projection 1024 -> 72 bits
• Activation: iFSQ (2*sigmoid(1.6x) - 1)
• Trainable params: 73,728 (0.02% of model)

Training

• Data: WikiText-103 (~100M tokens)
• Loss: Language + 0.05 * (triadic + 2supervised + 5subsumption)
• Optimizer: AdamW (3e-4, beta 0.9/0.95)
• Schedule: Cosine decay, 50% triadic warmup
• Steps: 50,000
• Hardware: NVIDIA RTX 4060 Ti 16GB

Key Results

• Bit accuracy: ~91% (72-bit semantic signatures)
• Perplexity: 31.95 (identical to GPT-2 Medium baseline)
• Regla de Tres: 0.996 mean cosine (near-perfect relational structure)
• Phase transition: Significant at step ~6000 (p=0.0004)

Intended Use

Research on neurosymbolic AI, discrete representation learning, and interpretable semantic structures. The triadic head maps continuous embeddings to algebraic prime-factor signatures enabling exact logical operations (subsumption, composition, gap analysis) impossible under cosine similarity.

Limitations

• D-A-N classifications used for training were self-evaluated by the author; independent rater validation is pending.
• Only 5 of 13 dual axes show significant anti-correlation after FDR correction.
• Layer ordering is not recovered in neural representations (p=0.71).

Files

ternary_v2/
  model_best.pt          # Checkpoint (1.4GB)
  training_log.csv       # Per-step metrics
sparsity_v2/
  model_best.pt          # Checkpoint (1.4GB)
  training_log.csv       # Per-step metrics
data/
  primitivos.json        # 72 semantic primitives (DAG, primes, bits)
  anclas.json            # Training anchors
  anclas_v2.json         # Extended anchors
LICENSE                  # BUSL-1.1

Related Models

• triadic-gpt-40m -- End-to-end 40M param model (unfrozen backbone)

Related Papers

1. Prime Factorization as a Neurosymbolic Bridge (Ornelas Brand, 2026a) -- Core algebraic framework
2. TriadicGPT (Ornelas Brand, 2026b) -- End-to-end learning
3. reptimeline (Ornelas Brand, 2026c) -- Representation lifecycle tracking
4. Emergent Duality (Ornelas Brand, 2026d) -- 14+ dualities across 6 algebraic layers (in progress)

Usage

import torch
checkpoint = torch.load("ternary_v2/model_best.pt", map_location="cpu")
# Contains model state dict + triadic head weights

For the full training/inference pipeline, install triadic-head:

pip install triadic-head

Citation

@article{ornelas2026emergent,
  title={Emergent Duality: 14+ Fundamental Dualities Across 6 Algebraic Layers},
  author={Ornelas Brand, J. Arturo},
  year={2026}
}

License

BUSL-1.1 -- free for individuals, academic, and non-profit use. Commercial production use requires a participation agreement. See COMMERCIAL.md.