Runtime: Osaurus 0.25.24 or newer.

OsaurusAI/Ornith-1.5-122B-JANG_4M

Ornith 1.5 122B in 73.9 GB, with 94.6% top-1 agreement with the full-precision model. Vision, video and the native multi-token-prediction (MTP) head are included.

This is a JANG bundle of ornith-ai/Ornith-1.5-122B: a 48-layer hybrid MoE (36 gated-DeltaNet linear-attention layers and 12 full-attention layers; 256 routed experts, top-8, plus a shared expert) with a vision/video tower, one MTP step and a 262k context.

• Routed experts: affine 4-bit (group 64), rounded with GPTQ on per-expert Hessians after activation-aware (AWQ) scaling.
• Everything else: affine 8-bit (group 64), GPTQ-rounded. That covers the linear-attention and attention projections, shared experts, embeddings and lm_head. Routers and shared-expert gates stay bf16.
• Vision tower: affine 8-bit (group 64). The MLP output projections (width 4304, which no group size divides) stay bf16.
• Native MTP head: experts 4-bit, the rest 8-bit, calibrated the way the main model is.
• Size: 73.88 GB of model shards, 4.73 bits per weight over the 125.1B source parameters.

Quality vs the original model

The test set is 240 held-out sequences (109,748 teacher-forced positions), scored against the BF16 release run layer by layer from disk, with top-128 KL. The assistant turns were written by Ornith itself. They span both reasoning modes, 22+ human languages, ~35 coding languages, multi-round agent episodes with tool calls, terminal, math, science, security, chat, prose and images. None of these sequences were used for calibration.

	median KL ↓	mean KL ↓	p99 KL ↓	top-1 ↑
plain round-to-nearest 4-bit experts (same 8-bit rest)	0.0002	0.0726	1.35	94.3%
Ornith-1.5-122B-JANG_4M (73.9 GB)	0.0001	0.0651	1.21	94.6%

Change in negative log-likelihood of the actual held-out tokens: +0.030 (plain 4-bit: +0.035).

By reasoning mode (mean KL · top-1):

mode	positions	JANG_4M	plain 4-bit
thinking on (default)	88,680	0.063 · 94.7%	0.070 · 94.4%
thinking off	21,068	0.075 · 94.3%	0.085 · 93.9%

By domain (mean KL · top-1):

domain	positions	JANG_4M	plain 4-bit
multilingual	35,381	0.095 · 93.7%	0.101 · 93.4%
coding	18,836	0.053 · 95.0%	0.061 · 94.5%
multi-round tool conversations	11,812	0.060 · 95.4%	0.070 · 95.1%
math / reasoning	10,473	0.015 · 97.5%	0.016 · 97.4%
images	9,775	0.037 · 95.2%	0.044 · 94.7%
terminal	5,321	0.035 · 95.5%	0.045 · 95.2%
chat	4,265	0.111 · 90.1%	0.121 · 90.0%
science	3,539	0.075 · 93.8%	0.080 · 93.6%
cybersecurity	3,439	0.042 · 95.6%	0.057 · 95.1%
prose	3,080	0.058 · 95.0%	0.073 · 93.5%
Chinese	2,005	0.127 · 90.0%	0.131 · 89.5%
agentic	1,822	0.032 · 97.4%	0.044 · 96.6%

Tool decisions and reasoning

• Tool calls: the original model's top choice is to start a tool call at 75 points. JANG_4M starts the call at 73 of 75. Across all 79 decision points, it flips between calling and answering at 2.
• Closing reasoning: there are 249 points where the transcript closes the model's reasoning (</think>). The median P(</think>) is 0.9999, the same as the original. 16 positions are below 0.5, against 15 for the original model.

Native MTP head

The bundle keeps Ornith's one-step MTP head, which predicts the token after next. On the held-out positions:

• greedy draft acceptance against the main model is 59.9% (the original BF16 head: 60.3%);
• top-1 agreement with the original MTP head is 83.8%.

What's in the bundle

All bit widths below were measured from the shard bytes.

• Experts (48 MoE layers + the MTP MoE): 4-bit group 64. 37,593 of 37,632 projections were GPTQ-rounded; 39 rarely-routed ones were rounded to nearest.
• 8-bit group 64:
  • linear-attention in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / out_proj (×36 each);
  • attention q/k/v/o (×12 each);
  • shared-expert gate/up/down (×48 each);
  • embeddings and lm_head;
  • vision qkv / proj / fc1 (×27 each) and the merger.
• bf16: routers, shared-expert gates, norms, linear-attention conv1d, vision MLP output projections, and the patch and position embeddings.
• MTP: in the model shards (mtp.*). A small stamp file records the measured lm_head layout (8-bit group 64, untied) for draft proposals.

Serving contract

• Reasoning: enable_thinking on (default) or off. There are no effort levels. The chat template opens <think> and the model closes it with </think> before the answer.
• Tool calls: the qwen3_coder XML format (<tool_call><function=…><parameter=…>).
• Sampling: the default is Ornith's coding preset (generation_config.json): temperature 0.6, top-p 0.95, top-k 20, min-p 0, presence penalty 0, repetition penalty 1.0. For general tasks Ornith recommends temperature 1.0, top-p 0.95, top-k 20, min-p 0, presence penalty 1.5, repetition penalty 1.0 (jang_config.json sampling_modes.general).
• EOS: <|im_end|>, <|endoftext|>.

Build details

• Calibration: 650k tokens over 760 sequences of Ornith's own outputs, both reasoning modes (two-thirds thinking on).
  • Text: coding (~35 languages), multi-round agent and tool episodes, terminal, math, 22+ human languages, Chinese, science, security, chat, prose.
  • 100 images, run through the vision tower on their real pixels.
  • The 240 held-out sequences are disjoint from calibration.
• Hessians: per-expert (router-weight² weighted) and per-linear input Hessians from a layer-streamed run of the BF16 model, including the vision tower and the MTP step.
• AWQ with exact folds:
  • Expert input scales are folded into the post-attention RMSNorm, with the bf16-rounded scale compensated exactly.
  • Down-projection input scales are folded through each expert's own up rows.
  • Channels the model zeroes out are left untouched.
  • Fold strength is chosen per layer from the Hessian-weighted error; a strength of zero (no fold) is allowed.
• GPTQ:
  • Expert Hessians shrink toward the layer's pooled statistics. This beat each expert's own Hessian on held-out KL (0.0651 vs 0.0669).
  • A never-worse-than-round-to-nearest guard applies per projection.
• Verification: checked from the shards. Every tensor file is aligned, there is one index, every module's bits match the config, and every source tensor is accounted for.