Add LOREA V3-RE: Qwen3-Coder-30B-A3B LoRA for Wine/RE + Linux/macOS internals

NOTICE

@@ -1,28 +1,26 @@
 LOREA
 Copyright (c) 2026 Soaperloafidksum
-
-This model is licensed under the Apache License, Version 2.0.
-Full text: https://www.apache.org/licenses/LICENSE-2.0
+Licensed under the Apache License, Version 2.0 — https://www.apache.org/licenses/LICENSE-2.0
 
 ------------------------------------------------------------------------
 Attribution
 ------------------------------------------------------------------------
-Built with Qwen2.5-Coder-14B-Instruct
+Built with Qwen3-Coder-30B-A3B-Instruct
   © Alibaba Cloud — Apache License 2.0
-  https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct
-
-Quantized base: mlx-community/Qwen2.5-Coder-14B-Instruct-4bit
+  https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct
+Quantized base: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit
 
 ------------------------------------------------------------------------
-Fine-tuning data sources and their licenses
+Fine-tuning data sources and licenses
 ------------------------------------------------------------------------
-- Wine            (https://www.winehq.org)            LGPL-2.1-or-later
-- DXVK            (https://github.com/doitsujin/dxvk)  zlib
-- DXMT            (https://github.com/3Shain/dxmt)     see upstream LICENSE
-- Windows / PE / reverse-engineering notes             author's own work
-- Tool-calling instruction examples                    original (synthetic)
+- Wine            (winehq.org)                         LGPL-2.1-or-later
+- DXVK            (github.com/doitsujin/dxvk)           zlib
+- DXMT            (github.com/3Shain/dxmt)              see upstream
+- XNU / dyld / objc4 / libdispatch (apple-oss-distributions)  APSL 2.0 / Apache-2.0
+- Linux kernel    (kernel.org)                          GPL-2.0
+- RE / Linux / macOS Q&A, x86-64 disasm<->source pairs  original (synthetic)
+- Broad multi-language code sample                      respective upstream licenses
 
 The legal status of model weights trained on copyleft-licensed source code
-(e.g. Wine, LGPL-2.1) is unsettled. This model is provided as-is, without
-warranty. Users are responsible for compliance with applicable licenses in
-their own use.
+(e.g. Wine LGPL-2.1, Linux GPL-2.0) is unsettled. Provided as-is, without
+warranty. Users are responsible for license compliance in their own use.

README.md

@@ -1,6 +1,6 @@
 ---
 license: apache-2.0
-base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct
 tags:
 - mlx
 - lora
@@ -9,8 +9,9 @@ tags:
 - dxvk
 - dxmt
 - reverse-engineering
-- tool-calling
-- qwen2.5-coder
+- linux-internals
+- macos-internals
+- qwen3-coder
 language:
 - en
 library_name: mlx
@@ -19,72 +20,65 @@ pipeline_tag: text-generation
 
 # LOREA 🧀🤖
 
-**LOREA** is a LoRA fine-tune of **Qwen2.5-Coder-14B-Instruct** specialized for
-**Wine development, DXVK/DXMT (D3D→Vulkan / D3D→Metal), Windows internals, and
-reverse engineering**, with native support for the [OCLI](https://github.com/)
-agent's `<tools>{...}</tools>` tool-calling format.
+**LOREA** is a coding agent fine-tuned for **Wine development, DXVK/DXMT
+(D3D→Vulkan / D3D→Metal), Windows / Linux / macOS internals, and reverse
+engineering**, designed to drive the LOREA/OCLI agent CLI's `<tools>{...}</tools>`
+tool-calling. It runs **fully locally on Apple Silicon** via
+[MLX](https://github.com/ml-explore/mlx).
 
-It runs **fully locally on Apple Silicon** via [MLX](https://github.com/ml-explore/mlx).
-
-## Details
+## Flagship: `v3-re-30b/` — the RE model
 
 | | |
 |---|---|
-| Base model | `Qwen/Qwen2.5-Coder-14B-Instruct` (Apache-2.0) |
-| Quantized base used | `mlx-community/Qwen2.5-Coder-14B-Instruct-4bit` |
-| Method | LoRA (16 layers, rank-default), 2048-token sequences |
-| Framework | MLX (`mlx_lm`) |
-| Adapter size | ~46 MB |
-
-## What it's good at
-
-- Wine / DXVK / DXMT code (C / C++ / Objective-C / Rust)
-- Windows API, PE format, Win32 internals
-- Reverse-engineering context and workflows
-- Emitting tool calls in the OCLI format:
-  `<tools>{"name": "grep", "arguments": {"pattern": "vkCreateDevice", "path": "."}}</tools>`
-
-## Use it (MLX)
-
+| Base | `Qwen/Qwen3-Coder-30B-A3B-Instruct` (Apache-2.0, MoE — 30B total / ~3B active) |
+| Quantized base | `mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit` |
+| Method | LoRA (8 layers), 1000 iters, seq 1536, lr 5e-5 (MLX) |
+| Val loss | 1.77 → 1.25 |
+| Adapter | `v3-re-30b/adapters.safetensors` (~269 MB) |
+
+### What it knows
+- Wine / DXVK / DXMT code (C / C++ / Rust / Objective-C / Metal)
+- **Reverse engineering**: x86-64 disassembly → C, PE / ELF / Mach-O formats, calling conventions
+- **Linux internals**: syscalls, ELF loading, ptrace, the Wine architecture
+- **macOS internals**: Mach exception ports, IOKit matching (`IOServiceAddMatchingNotification`), dyld, the Obj-C runtime, **libdispatch** (`_dispatch_assert_queue_fail`), **Rosetta 2** (`%gs` / TLS)
+- Emits tool calls: `<tools>{"name":"grep","arguments":{"pattern":"vkCreateDevice","path":"."}}</tools>`
+
+### Use it (MLX)
 ```bash
 pip install mlx-lm
-# apply the LoRA adapter on top of the base:
 python -m mlx_lm generate \
-  --model mlx-community/Qwen2.5-Coder-14B-Instruct-4bit \
-  --adapter-path ./ \
-  --prompt "Explain how DXVK creates a D3D11 device."
-```
-
-Or fuse it into a standalone model:
-
-```bash
+  --model mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit \
+  --adapter-path ./v3-re-30b \
+  --prompt "Explain how a macOS IOKit matching notification can fire on the wrong dispatch queue."
+# or fuse into a standalone model:
 python -m mlx_lm fuse \
-  --model mlx-community/Qwen2.5-Coder-14B-Instruct-4bit \
-  --adapter-path ./ --save-path ./lorea-coder-14b
+  --model mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit \
+  --adapter-path ./v3-re-30b --save-path ./lorea-30b
 ```
 
 ## Training data
-
-Fine-tuned on a blend of:
+Wine / DXVK / DXMT source; Linux kernel + macOS open-source internals (XNU, dyld,
+objc4, libdispatch); synthesized x86-64 disasm↔source pairs; ~380 expert,
+adversarially fact-checked RE/Linux/macOS Q&A; and a broad multi-language code
+sample. Plus conversational + tool-calling data so it chats normally and
+finalizes after a tool result (no loops).
 
 | Source | License |
 |---|---|
-| Wine source + docs | LGPL-2.1+ |
+| Wine | LGPL-2.1+ |
 | DXVK | zlib |
-| DXMT | see upstream repo |
-| Windows / PE / RE analysis notes | author's own |
-| Tool-calling instruction examples | original (synthetic) |
+| DXMT | upstream |
+| XNU / dyld / objc4 / libdispatch | Apple OSS (APSL / Apache-2.0) |
+| Linux kernel | GPL-2.0 |
+| RE/Linux/macOS Q&A, disasm pairs | original (synthetic) |
 
-## License & attribution
+## License & limitations
+Apache-2.0, matching the base. *Built with Qwen3-Coder-30B-A3B (© Alibaba,
+Apache-2.0).* See `NOTICE`.
 
-LOREA is released under **Apache-2.0**, matching the base model.
+It's a LoRA on a 30B — strong at domain **vocabulary, concepts, and tool-driving**,
+but it can be wrong on deep specifics. **Review its output**, especially for
+reverse engineering. Provided **as-is, no warranty.**
 
-> Built with **Qwen2.5-Coder-14B** (© Alibaba, Apache-2.0). Fine-tuned on Wine
-> (LGPL-2.1), DXVK (zlib), DXMT, and original instruction data. See `NOTICE`.
-
-## Limitations
-
-A 14B model fine-tuned with a small LoRA — it improves *domain fluency, code
-style, and tool-calling*, but it is **not** an authoritative source: it can
-produce incorrect code and hallucinate APIs. It does not autonomously reverse-
-engineer binaries. Always review its output. Provided **as-is, no warranty**.
+---
+*An earlier 14B adapter (Qwen2.5-Coder-14B) lives in this repo's history.*

v3-re-30b/adapter_config.json

@@ -0,0 +1,41 @@
+{
+    "adapter_path": "/Users/teoballesteros/lorea-ft/adapters_v3_30b",
+    "batch_size": 1,
+    "clear_cache_threshold": 0,
+    "config": null,
+    "data": "/Users/teoballesteros/lorea-ft/data_v3",
+    "fine_tune_type": "lora",
+    "grad_accumulation_steps": 1,
+    "grad_checkpoint": true,
+    "iters": 1000,
+    "learning_rate": 5e-05,
+    "lora_parameters": {
+        "rank": 8,
+        "dropout": 0.0,
+        "scale": 20.0
+    },
+    "lr_schedule": null,
+    "mask_prompt": false,
+    "max_seq_length": 1536,
+    "model": "mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit",
+    "num_layers": 8,
+    "optimizer": "adam",
+    "optimizer_config": {
+        "adam": {},
+        "adamw": {},
+        "muon": {},
+        "sgd": {},
+        "adafactor": {}
+    },
+    "project_name": null,
+    "report_to": null,
+    "resume_adapter_file": null,
+    "save_every": 100,
+    "seed": 0,
+    "steps_per_eval": 200,
+    "steps_per_report": 20,
+    "test": false,
+    "test_batches": 500,
+    "train": true,
+    "val_batches": 25
+}

v3-re-30b/adapters.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18b89d8f74400e292f5134d984fd9175329f4d7006840a94f16156177d4aa592
+size 281852227