Deploy Helios Open-Weight Foundation Model Ecosystem (Weights, SafeTensors, Tokenizer, ONNX, TFLite)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+helios_model.onnx.data filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,122 @@
 ---
 license: apache-2.0
+tags:
+  - helios
+  - open-weight
+  - pytorch
+  - causal-lm
+  - rotary-embeddings
+  - grouped-query-attention
+  - swiglu
+  - hierarchical-memory
+datasets:
+  - HuggingFaceFW/fineweb-edu
+  - HuggingFaceFW/fineweb
+  - mlfoundations/dCLM-baseline
+  - HuggingFaceTB/cosmopedia-v2
+  - open-web-math/open-web-math
+  - EleutherAI/proof-pile-2
+  - bigcode/the-stack-v2
+  - bigcode/starcoderdata
+  - uonlp/CulturaX
+  - oscar-corpus/OSCAR-2301
+  - statmt/cc-100
+  - wikimedia/wikipedia
+pipeline_tag: text-generation
+language:
+  - en
+  - tl
 ---
+
+# Helios Open-Weight Foundation Model (Helios)
+
+**Developed by: Arron Kian Parejas** (`arronkianparejas@gmail.com`)  
+**GitHub Repository**: [https://github.com/darknecrocities/Helios](https://github.com/darknecrocities/Helios)  
+**Hugging Face Hub**: [https://huggingface.co/Darknecrocities/Helios](https://huggingface.co/Darknecrocities/Helios)  
+**Ollama Registry**: `parejasarronkian/Helios`  
+**License**: Apache-2.0 / MIT Dual License  
+
+---
+
+## 1. Overview & Architecture
+
+Helios is a modular, transparent, open-weight foundation model ecosystem
+engineered for efficient inference on consumer edge hardware and clusters.
+
+### Core Architectural Specifications:
+- **Architecture**: Decoder-only Causal Transformer
+- **Parameters**: ~17.84M (7 layers, 320 hidden dim)
+- **Attention**: Grouped-Query Attention (GQA, 8 Q : 2 KV)
+- **Position Representation**: Rotary Position Embeddings (RoPE, theta=10000.0)
+- **Normalization**: Pre-RMSNorm (eps=1e-05)
+- **Feed-Forward**: SwiGLU Gated MLP (intermediate dim=864)
+- **Context Length**: 512 tokens
+- **Memory Augmentation**: HMA with Working, Episodic, Semantic, and Procedural tiers.
+
+---
+
+## 2. Parameter Scaling Roadmap & Exit Gates
+
+| Stage | Target Size | Hardware Target | Status | Exit Gates |
+| :--- | :--- | :--- | :--- | :--- |
+| **Nano** | 10M–40M | Apple Silicon M3 | **Active** | Deterministic resume, overfit tiny corpus |
+| **Alpha** | 100M | Single GPU / MPS | Planned | Perplexity < 25, fertility validated |
+| **Beta** | 300M | Single GPU (24GB) | Planned | GQA KV-cache efficiency, zero regressions |
+| **Core** | 1B | Multi-GPU Node | Roadmap | FSDP2 sharding, cluster recipe |
+| **Pro** | 3B | Cluster (8x H100) | Long-Term | Multi-node recovery, optimizer sharding |
+| **Flagship**| 6B | Compute Cluster | Long-Term | Full training provenance, safety review |
+
+---
+
+## 3. Pretraining Datasets & Domain Allocation
+
+Helios models are trained on an overtraining schedule using 12 high-quality datasets:
+
+| Domain | Dataset Name | Hugging Face Dataset ID | Ratio |
+| :--- | :--- | :--- | :--- |
+| **Educational** | FineWeb-Edu | `HuggingFaceFW/fineweb-edu` | 30% |
+| **General Web** | FineWeb & DCLM | `HuggingFaceFW/fineweb` | 15% |
+| **Synthetic** | Cosmopedia v2 | `HuggingFaceTB/cosmopedia-v2` | 15% |
+| **Code** | The Stack v2 | `bigcode/the-stack-v2` | 20% |
+| **STEM/Math** | OpenWebMath | `open-web-math/open-web-math` | 10% |
+| **Filipino/TL** | CulturaX (tl) | `uonlp/CulturaX` (tl) | 10% |
+
+---
+
+## 4. Quickstart Usage
+
+### Using Helios CLI:
+```bash
+# Inspect model parameters
+helios inspect -m nano-20m
+
+# Run interactive generation
+helios generate --prompt "Write a Python function to compute Fibonacci numbers."
+```
+
+### Using Transformers:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("Darknecrocities/Helios")
+model = AutoModelForCausalLM.from_pretrained("Darknecrocities/Helios")
+
+inputs = tokenizer("Hello Helios! Who developed you?", return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=50)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+## 5. Citation & Attribution
+
+```bibtex
+@misc{helios2026,
+  author = {Parejas, Arron Kian},
+  title = {Helios: Transparent Open-Weight Foundation Model Ecosystem},
+  year = {2026},
+  publisher = {Hugging Face},
+  journal = {Hugging Face Hub Repository},
+  howpublished = {\url{https://huggingface.co/Darknecrocities/Helios}}
+}
+```

config.json

@@ -0,0 +1,28 @@
+{
+  "architectures": [
+    "HeliosForCausalLM"
+  ],
+  "model_type": "helios",
+  "auto_map": {
+    "AutoConfig": "helios.model.config.HeliosConfig",
+    "AutoModelForCausalLM": "helios.model.transformer.HeliosForCausalLM",
+    "AutoTokenizer": "helios.tokenizer.base.HeliosTokenizer"
+  },
+  "vocab_size": 32000,
+  "hidden_size": 320,
+  "num_hidden_layers": 7,
+  "num_attention_heads": 8,
+  "num_key_value_heads": 2,
+  "intermediate_size": 864,
+  "max_position_embeddings": 512,
+  "rope_theta": 10000.0,
+  "rms_norm_eps": 1e-05,
+  "tie_word_embeddings": true,
+  "initializer_range": 0.02,
+  "dropout": 0.0,
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.0",
+  "model_family": "Helios",
+  "developer": "Arron Kian Parejas",
+  "developer_email": "arronkianparejas@gmail.com"
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "bos_token_id": 256,
+  "eos_token_id": 257,
+  "pad_token_id": 258,
+  "max_length": 512,
+  "temperature": 0.7,
+  "top_p": 0.9,
+  "top_k": 40,
+  "do_sample": true,
+  "transformers_version": "4.40.0"
+}

helios_model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:964b8e0585778a8a7d728273bf83e4bc4cf4bb898acab09a4b4ac0abe4008a0b
+size 906229

helios_model.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afea2c34d7852634cb79674695cac64652f9d67b8ac6da68aed8e7f8b41aeb8d
+size 72518400

helios_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:673ce70ed9f3162c1311fa4bb695b3a9b417d4b2a7982e3b9091f3f1a50c229f
+size 72729448

helios_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2da61fff29f95ab9592a98196410dcc1c5f73ff0f95d792c879ad94aabf4b6f
+size 112338432

helios_model.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4982c57ec7c55ea30de0492caa79d6569bd2253966de3aa60d1fd991a290ba32
+size 71384256

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2da61fff29f95ab9592a98196410dcc1c5f73ff0f95d792c879ad94aabf4b6f
+size 112338432

physical_manifest.json

@@ -0,0 +1,27 @@
+{
+  "model_family": "Helios",
+  "developer": "Arron Kian Parejas",
+  "output_directory": "/Users/arronkianparejas/helios/dist/physical_models",
+  "formats": {
+    "safetensors": {
+      "file": "/Users/arronkianparejas/helios/dist/physical_models/helios_model.safetensors",
+      "bytes": 112338432,
+      "sha256": "d2da61fff29f95ab9592a98196410dcc1c5f73ff0f95d792c879ad94aabf4b6f"
+    },
+    "torchscript": {
+      "file": "/Users/arronkianparejas/helios/dist/physical_models/helios_model.pt",
+      "bytes": 72729448,
+      "sha256": "673ce70ed9f3162c1311fa4bb695b3a9b417d4b2a7982e3b9091f3f1a50c229f"
+    },
+    "onnx": {
+      "file": "/Users/arronkianparejas/helios/dist/physical_models/helios_model.onnx",
+      "bytes": 906229,
+      "sha256": "964b8e0585778a8a7d728273bf83e4bc4cf4bb898acab09a4b4ac0abe4008a0b"
+    },
+    "tflite": {
+      "file": "/Users/arronkianparejas/helios/dist/physical_models/helios_model.tflite",
+      "bytes": 71384256,
+      "sha256": "4982c57ec7c55ea30de0492caa79d6569bd2253966de3aa60d1fd991a290ba32"
+    }
+  }
+}

special_tokens_map.json

@@ -0,0 +1,12 @@
+{
+  "bos_token": "<|begin_of_text|>",
+  "eos_token": "<|end_of_text|>",
+  "pad_token": "<|pad|>",
+  "additional_special_tokens": [
+    "<|start_header_id|>",
+    "<|end_header_id|>",
+    "<|call:tool|>",
+    "<|return:tool|>",
+    "<|refusal|>"
+  ]
+}

tokenizer_config.json

@@ -0,0 +1,81 @@
+{
+  "tokenizer_class": "HeliosTokenizer",
+  "auto_map": {
+    "AutoTokenizer": [
+      "helios.tokenizer.base.HeliosTokenizer",
+      null
+    ]
+  },
+  "model_max_length": 1024,
+  "bos_token": "<|begin_of_text|>",
+  "eos_token": "<|end_of_text|>",
+  "pad_token": "<|pad|>",
+  "clean_up_tokenization_spaces": false,
+  "added_tokens_decoder": {
+    "256": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "257": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "258": {
+      "content": "<|pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "259": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "260": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "261": {
+      "content": "<|call:tool|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "262": {
+      "content": "<|return:tool|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "263": {
+      "content": "<|refusal|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "chat_template": "{% for message in messages %}{{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' + message['content'] + '<|end_of_text|>' }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}"
+}