zyr-AGENT/zyr3-v1

ZYR3 v1

ZYR3 v1 is an agentic coding assistant built as a LoRA adapter on Qwen/Qwen2.5-Coder-7B-Instruct. It behaves as an AI coding assistant rather than only a code generator: it explains code, edits, refactors, debugs, and works through multi-step coding tasks iteratively.

There is no hosted ZYR3-V1 API. ZYR3-V1 is a self-hosted model: you load the 7B base model and this adapter on your own GPU.

Agent layer (Planner + Coder)

The repo ships an in-process multi-agent mode. POST /v1/agents/plan-code with {"task": "...", "context": "..."} runs two agents on the loaded model:

1. Planner (PG) - splits the task into ordered steps (<PLAN> JSON).
2. Coder (CG) - implements each step, emitting one <CODE file="..."> block per file, with already-written files kept in context for consistency.

Response: {task, steps, files, summary}. Set ZYR3_AGENT_URL (and optional ZYR3_AGENT_MODEL/ZYR3_AGENT_KEY) to route the agents at a remote endpoint instead - e.g. a zyr3-3.1 MoE (Qwen3-30B-A3B) build can act as the stronger coder while zyr3-v1 stays the planner. Run the same orchestration standalone with python agents.py "write a python hello world with tests".

What you need

• Base model: Qwen/Qwen2.5-Coder-7B-Instruct (downloads automatically from Hugging Face, ~15 GB fp16)
• Adapter: zyr-AGENT/zyr3-v1 (this repo, ~80 MB)

Path	VRAM	Speed
Transformers + PEFT, fp16	~16 GB	full speed
Transformers + PEFT, 4-bit load	~8-9 GB	slightly slower
GGUF quantized via Ollama / llama.cpp	6-8 GB (Q4_K_M ~5 GB)	fast, low VRAM

NVIDIA CUDA GPU recommended. CPU works for quantized GGUF but is slow.

Capabilities

• Coding
• Code explanation
• Code editing
• Debugging
• Bug finding
• Refactoring
• Iterative problem solving
• Programming reasoning
• Multi-step coding tasks
• Edge-case handling

Path A — Transformers + PEFT (GPU)

Install:

pip install torch transformers accelerate peft safetensors

Load and chat (fp16, ~16 GB VRAM):

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen2.5-Coder-7B-Instruct"
adapter = "zyr-AGENT/zyr3-v1"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

messages = [
    {"role": "system", "content": "You are ZYR3, an agentic coding assistant."},
    {"role": "user", "content": "Write a Flask web server."},
]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Low-VRAM alternative: add 4-bit loading via bitsandbytes:

from transformers import BitsAndBytesConfig

quant = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16)
model = AutoModelForCausalLM.from_pretrained(
    base_model, quantization_config=quant, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter)

CLI helper (env-driven):

python infer.py "def add(a, b):"

Path B — GGUF / Ollama (low VRAM)

Ollama cannot load a PEFT adapter directly, so merge first, convert to GGUF, then quantize.

# Merge adapter into the base model (needs ~20 GB RAM or a 16 GB GPU)
python merge_for_ollama.py          # writes ./merged_zyr3

# Convert to GGUF (llama.cpp: https://github.com/ggerganov/llama.cpp)
python llama.cpp/convert_hf_to_gguf.py merged_zyr3 \
    --outfile zyr3-v1-f16.gguf --outtype f16

# Quantize to 4-bit for 6-8 GB GPUs
llama.cpp/build/bin/llama-quantize zyr3-v1-f16.gguf \
    zyr3-v1-q4_k_m.gguf q4_k_m

# Create and run the Ollama model
ollama create zyr3-v1 -f ollama/Modelfile
ollama run zyr3-v1 "Write a Flask web server"

ollama/Modelfile uses ChatML formatting (<|im_start|> / <|im_end|>) and a ZYR3 system prompt. The FROM line points at zyr3-v1-q4_k_m.gguf; adjust it if you skip quantization (use ./zyr3-v1-f16.gguf).

Serve it as an OpenAI-compatible API (/v1) for VS Code anywhere

Once you have Path B working on any GPU machine (desktop, VPS, or cloud VM), expose ZYR3-V1 as an OpenAI-compatible /v1 endpoint that Cline, Continue, Roo Code, or any coding agent can connect to.

1. Run the server on your GPU host

Option A — Ollama (simplest). Ollama exposes OpenAI-compatible endpoints automatically at /v1/chat/completions:

ollama create zyr3-v1 -f ollama/Modelfile

# listen on all interfaces so other machines can reach it
OLLAMA_HOST=0.0.0.0:11434 ollama serve

Verify locally:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"zyr3-v1","messages":[{"role":"user","content":"hi"}]}'

Option B — vLLM (higher throughput, one server, needs the merged model):

vllm serve merged_zyr3 --served-model-name zyr3-v1 --port 8000

Endpoint: http://HOST:8000/v1.

2. Reach it from anywhere

• Fastest/safest for personal use: Tailscale on the host and your laptop, then use http://<tailscale-ip>:11434/v1.
• For a public endpoint put a reverse proxy (nginx / Cloudflare Tunnel) in front with a bearer token; Ollama itself has no auth.

3. Point VS Code at it

Cline / Roo Code (OpenAI Compatible provider):

{
  "provider": "OpenAI Compatible",
  "baseUrl": "http://HOST:11434/v1",
  "apiKey": "none",
  "model": "zyr3-v1"
}

Continue (config.yaml):

models:
  - name: ZYR3-V1
    provider: openai
    model: zyr3-v1
    apiBase: http://HOST:11434/v1
    apiKey: none

Any OpenAI-compatible client works the same way: base URL http://HOST:11434/v1 (or :8000/v1), model id zyr3-v1.

Files

• adapter_config.json / adapter_model.safetensors — the PEFT LoRA weights
• infer.py — GPU chat CLI (Path A)
• merge_for_ollama.py — merge adapter into the base model (Path B)
• ollama/Modelfile — Ollama template (Path B)
• config.json + modeling_zyr3.py — optional custom-code transformers wrapper (for HF "servable model" listing / API-style loading via trust_remote_code=True); the real inference is the base + adapter above.

Project

ZYR3

Version: v1