⚡ DeepNarra Hiraya-1-Base 1.0 (118M Serialized / 80.5M Tied Dense)

Phase 1 Foundation Base Model — Apex Tri-Hybrid (75% Mamba-2 SSD + 25% Differential Attention + BitNet b1.58 + SwiGLU)

Organization: DeepNarra
Pipeline Stage: Phase 1 — Foundation Base Pre-Training
Architecture: Apex-TriHybrid-Mamba2-DiffAttn-BitNet158 (d_model=768, num_layers=8, vocab_size=48,760)
Training Convergence: Pre-trained for 5,000 steps with Kahan 64-bit Compensated Muon + AdamW Optimizer (Loss converged from 33.9651 ➔ 0.63)
Evaluation Governance: Zero Fake Scores Policy — 100% Verifiable Engineering Facts Only

---

🏛️ Verified Technical Specifications (Exact Repository Match)

Architectural Metric	Exact Verified Value in This Repository
Official Model Name	DeepNarra Hiraya-1-Base 1.0 (118M Serialized / 80.5M Tied)
Exact Safetensors Parameter Count	117,993,216 serialized elements (450.12 MB in model.safetensors)
Unique Weight-Tied Parameters	80,545,538 active parameters (token_embeddings and lm_head decoupled by .clone() of 48,760 × 768 = 37,447,680 params for safetensors compliance)
Sovereign Omni-BPE Tokenizer	48,760 active vocabulary (tokenizer.json 3.39 MB, includes native <think>, <click>, <touch>, <audio>, <video>, <box3d> tokens)
Hidden Dimension & Layers	d_model = 768, intermediate_dim = 1,376, num_layers = 8, num_heads = 8
Tri-Hybrid Layer Ratio	75% Mamba-2 State Space Duality (SSD) + 25% Microsoft Differential Attention
Weight Quantization Readiness	BitNet b1.58 Ternary Projections + SwiGLU FFN
Optimizer & Numerical Precision	Muon Optimizer (Newton-Schulz orthogonalization) with Kahan 64-bit Compensated Summation

---

🧬 DeepNarra Cumulative Training Lineage

1. 👉 DeepNarra/Hiraya-1-Base (118M / 450.12 MB) — You are here (Phase 1 Foundation Pre-Training)
2. ⬆️ DeepNarra/Hiraya-1-H-MoE (908.6M / 3.46 GB) — Phase 2 Expansion into 92 Hierarchical Sub-Experts
3. ⬆️ DeepNarra/Hiraya-1-Instruct (908.6M / 3.32 GB) — Phase 3 14-Horizon Streaming SFT Super-Agent
4. ⬆️ DeepNarra/Hiraya-1-Reasoning (908.6M / 1.73 GB FP16) — Phase 4 Autonomous SymPy CAS & DeepSeek-R1 GRPO Reasoning

---

🚀 Quickstart Usage (Python)

from transformers import AutoTokenizer
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download

repo_id = "DeepNarra/Hiraya-1-Base"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
weights_path = hf_hub_download(repo_id=repo_id, filename="model.safetensors")
state_dict = load_file(weights_path)
print(f"Vocab Size: {len(tokenizer):,} | Tensors: {len(state_dict):,}")

---

⚖️ License, Dataset Attributions & Responsible AI Compliance

1. Model License (Apache-2.0)

This repository, including all model weights (.safetensors and .gguf), configuration manifests, and custom architectural code, is licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0). See the LICENSE and NOTICE files in the root of this repository for full legal terms.

2. Upstream Dataset & Open-Data Attributions (ODC-By-1.0 & Apache-2.0)

In strict compliance with open-data attribution requirements:

• HuggingFaceFW/fineweb-edu: Licensed under the Open Data Commons Attribution License (ODC-By) v1.0.
• CohereForAI/aya_collection: Licensed under the Apache License 2.0.
• m-a-p/CodeFeedback-Filtered-Instruction: Licensed under the Apache License 2.0.
• HuggingFaceH4/no_robots: Attributed under CC-BY-NC-4.0 for instruction-following research evaluation.
• SymPy CAS Procedural Reasoning Stream: Synthetically generated and formally verified via SymPy (BSD-3-Clause).

3. Intended Use, Safety & Limitations

• Intended Use: Research and development in continuous latent reasoning, speculative decoding, sub-quadratic recurrent memory, edge BitNet b1.58 GGUF deployment, and sandboxed autonomous agents.
• Safety & Execution Airlock: Code execution and tool-calling capabilities are designed to run strictly inside the DeepNarra Airlock Sandbox with AST static verification and SSRF network firewalling (169.254.169.254 and RFC-1918 private subnets blocked).
• Limitations & Disclaimer: As an experimental sovereign architecture trained on compact compute budgets, outputs should be independently verified for high-stakes medical, legal, financial, or safety-critical systems. Provided "AS IS" without warranties of any kind under Section 7 of the Apache 2.0 License.