---
library_name: transformers
license: mit
base_model:
  - ornith-ai/Ornith-1.5-9B
pipeline_tag: image-text-to-text
tags:
  - ornith
  - qwen
  - qwen3.5
  - autoround
  - quantized
  - int4
  - mixed-precision
  - multimodal
  - reasoning
  - vllm
---

# Ornith-1.5-9B-MixedInt4-AutoRound

A mixed-precision AutoRound quantized version of **ornith-ai/Ornith-1.5-9B**, optimized to reduce memory requirements while preserving the capabilities and quality of the original model.

## Base Model

**Base model:** `ornith-ai/Ornith-1.5-9B`

This model is a quantized version of the original Ornith-1.5-9B checkpoint. It is not a fine-tune, merge, or distillation.

Ornith-1.5-9B is a dense multimodal reasoning model based on the Qwen3.5 architecture.

## Quantization

Quantization was performed using **Intel AutoRound** with a custom mixed-precision quantization configuration.

The quantization recipe was designed to balance:

- Model quality
- Memory efficiency
- Inference performance
- vLLM compatibility

Some model components are intentionally retained at higher precision where appropriate.

The calibration process was primarily optimized for **instruction-following performance[no-think]**, focusing on maintaining high-quality responses for user instructions and conversational interactions after quantization.

| Property | Value |
|---|---|
| Quantization framework | Intel AutoRound |
| Quantization type | Custom Mixed-Precision INT4 |
| Base model | ornith-ai/Ornith-1.5-9B |
| Architecture | Qwen3.5 |
| Language layers | 32 |
| Vision encoder depth | 27 |
| Native context length | 262,144 tokens |
| Vision tower | Preserved at original precision |

The detailed mixed-precision allocation strategy is not included in this model card.

## Evaluation

Evaluation was performed using **AutoRound's evaluation interface with LM Evaluation Harness**.

The following results compare the original **Ornith-1.5-9B** model against **Ornith-1.5-9B-MixedInt4-AutoRound**.

| Benchmark | Metric | Ornith-1.5-9B | Ornith-1.5-9B-MixedInt4-AutoRound | Difference | Recovery Rate |
|---|---:|---:|---:|---:|---:|
| MMLU | acc | 77.30% | 76.80% | -0.50 pp | 99.35% |
| GSM8K | exact_match (flexible) | 62.09% | 70.51% | +8.42 pp | 113.56% |
| **Average** | — | 69.70% | 73.66% | +3.96 pp | 105.68% |

### MMLU Category Breakdown

| MMLU Category | Ornith-1.5-9B | Ornith-1.5-9B-MixedInt4-AutoRound | Difference | Recovery Rate |
|---|---:|---:|---:|---:|
| Humanities | 69.12% | 68.80% | -0.32 pp | 99.54% |
| Other | 81.62% | 81.43% | -0.19 pp | 99.77% |
| Social Sciences | 85.93% | 85.05% | -0.88 pp | 98.98% |
| STEM | 76.82% | 76.12% | -0.70 pp | 99.09% |

### GSM8K

| Metric | Ornith-1.5-9B | Ornith-1.5-9B-MixedInt4-AutoRound | Difference | Recovery Rate |
|---|---:|---:|---:|---:|
| Flexible Exact Match | 62.09% | 70.51% | +8.42 pp | 113.56% |
| Strict Exact Match | 73.09% | 73.69% | +0.60 pp | 100.82% |

> Recovery Rate represents benchmark performance relative to the original Ornith-1.5-9B checkpoint. A recovery rate above 100% indicates that the quantized model scored higher than the original model in that particular evaluation. Benchmark preservation does not imply identical behavior for every prompt, multimodal workload, long-context workload, agentic workload, or generation setting.

## Usage

This checkpoint is intended for inference engines with AutoRound quantization support, including compatible versions of vLLM.

Ornith-1.5-9B is a reasoning model and supports tool calling. Recent versions of the serving runtime are recommended.

### vLLM

```bash
vllm serve Pilcothink/Ornith-1.5-9B-MixedInt4-AutoRound \
  --served-model-name Ornith-1.5-9B-MixedInt4-AutoRound \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --enable-prefix-caching \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90
```

### Recommended Sampling Parameters

For general tasks:

```text
temperature=1.0
top_p=0.95
top_k=20
min_p=0.0
presence_penalty=1.5
repetition_penalty=1.0
```

For precise coding tasks:

```text
temperature=0.6
top_p=0.95
top_k=20
min_p=0.0
presence_penalty=0.0
repetition_penalty=1.0
```

## Long-Context Serving

The native context length of Ornith-1.5-9B is **262,144 tokens**.

For workloads requiring a context window beyond the native limit, Ornith recommends extending the effective context using YaRN RoPE scaling.

Example for approximately 1M tokens:

```bash
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
vllm serve Pilcothink/Ornith-1.5-9B-MixedInt4-AutoRound \
  --served-model-name Ornith-1.5-9B-MixedInt4-AutoRound \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.90 \
  --hf-overrides '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}}' \
  --max-model-len 1000000
```

Static YaRN scaling is applied to all requests and may slightly affect quality on shorter contexts. It should therefore only be enabled when workloads genuinely require context lengths beyond the native 262,144-token window.

## Notes

- This is a quantized derivative of `ornith-ai/Ornith-1.5-9B`.
- The model uses a custom mixed-precision INT4 quantization recipe.
- The vision components are preserved at their original precision.
- The model retains the original Ornith reasoning and tool-calling format.
- The native context length is 262,144 tokens.
- Quantization may introduce small behavioral differences compared with the original checkpoint.
- Long-context, multimodal, reasoning, and agentic workloads may respond differently from the original model even when benchmark recovery is high.

## Acknowledgements

- **Base model:** Ornith / DeepReinforce Team
- **Underlying architecture:** Qwen3.5
- **Quantization framework:** Intel AutoRound

Please refer to the original `ornith-ai/Ornith-1.5-9B` model card for licensing, intended usage, limitations, recommended sampling parameters, and other information applicable to the base model.