tiny-random/unlimited-ocr-2

This is a randomly initialized tiny model based on baidu/Unlimited-OCR-2 for testing and debugging.

Feature	Source	This repo
Checkpoint tensor elements	3.34B BF16, 257 I64	9.04M BF16, 257 I64
Safetensors weight files	6.69 GB (2 files)	18.12 MB (model.safetensors)
Decoder layers / hidden width / dense MLP	12 / 1,280 / 6,848	4 / 32 / 256
Q / KV heads / head dimension	10 / 10 / 128	4 / 4 / 32
Routed experts / top-k / shared experts	64 / 6 / 2	16 / 6 / 2
Expert intermediate width / initial dense layers	896 / 1	64 / 1
DASH index width / sparse block size	128 / 32	32 / 32
Page and token RoPE / sparse layer coverage	page2d / all layers	Same
SAM layers / width / global attention layers	12 / 768 / 2,5,8,11	6 / 32 / 2,5
CLIP layers / width	24 / 1,024	4 / 32
SAM neck / downsampling channels	256 / 512,1,024	8 / 16,32
Vision projector input / output	2,048 / 1,280	64 / 32
Global / crop image size / page visual tokens	1,024 / 640 / 273	Same
Vocabulary rows / tied embeddings	129,280 / no	Same
Serialized precision	BF16 weights + I64 position IDs	Same

Residual width 32 is independent of the four 32-dimensional attention heads: Q/K/V project to 128 channels and the output projection returns to the residual width. The included runtime exposes these config dimensions while preserving the upstream attention kernels.

File path	Size
model.safetensors	18.1MB

Example usage

Transformers (✅ verified)

Install Transformers 4.52.1, torchvision, Pillow, addict, easydict and einops and load the included upstream remote code. The source registers only mha_eager; its OCR API runs DASH Triton sparse prefill, the reference masked sparse decode, and SDPA in SAM/CLIP.

def run_transformers(model_id: str):
    import torch
    from PIL import Image, ImageDraw
    from transformers import AutoModel, AutoTokenizer

    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModel.from_pretrained(model_id, trust_remote_code=True, torch_dtype=torch.bfloat16).eval().cuda()
    page = Image.new("RGB", (800, 1000), "white")
    ImageDraw.Draw(page).text((50, 50), "Hello OCR. Page one.", fill="black")
    text = model.infer(tokenizer, prompt="Read this document.", image_file=page, max_new_tokens=10)
    print(text)
    text, tokens = model.infer_multi(tokenizer, image_files=[page, page.copy()], max_new_tokens=10)
    print(text, tokens)


if __name__ == "__main__":
    run_transformers(model_id="tiny-random/unlimited-ocr-2")

SGLang (✅ verified)

Install Baidu's pinned sglang-0.5.20.dev13+g75b9b06b02 wheel from the source repo. The included external-model package reads the reduced vision config, separates residual width from attention head dimension, and gathers the TP-sharded DASH index projections before normalization. The upstream attention and routing kernels are retained.

hf download tiny-random/unlimited-ocr-2 --local-dir ./unlimited-ocr-2
pip install --no-deps ./unlimited-ocr-2

export SGLANG_EXTERNAL_MODEL_PACKAGE=unlimited_ocr_runtime
export SGLANG_UOCR_EANABLE_DOC_LIMIT=0
python -m sglang.launch_server \
    --model-path ./unlimited-ocr-2 \
    --tp-size 4 \
    --trust-remote-code \
    --chat-template unlimited-ocr \
    --context-length 4096 \
    --max-total-tokens 8192 \
    --max-running-requests 4 \
    --cuda-graph-backend-decode full \
    --cuda-graph-max-bs-decode 4 \
    --cuda-graph-backend-prefill full \
    --cuda-graph-max-bs-prefill 2048 \
    --cuda-graph-bs-prefill 2048 \
    --max-prefill-tokens 2048 \
    --chunked-prefill-size 2048 \
    --cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 4}}' \
    --mem-fraction-static 0.3 \
    --host 0.0.0.0 --port 8000

import base64
from io import BytesIO

import requests
from PIL import Image, ImageDraw

page = Image.new("RGB", (800, 1000), "white")
ImageDraw.Draw(page).text((50, 50), "Hello OCR", fill="black")
buffer = BytesIO()
page.save(buffer, format="PNG")
image = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
session = requests.Session()
session.trust_env = False
response = session.post("http://127.0.0.1:8000/v1/chat/completions", json={
    "model": "./unlimited-ocr-2",
    "messages": [{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": image}},
        {"type": "text", "text": "<PAGE><image>Read this document."},
    ]}],
    "max_tokens": 10, "temperature": 0,
}, timeout=120)
response.raise_for_status()
print(response.json()["choices"][0]["message"]["content"])

Creating this repo

• Code: .meta/create_model.py
• Model structure: .meta/model.log
• Environment: .meta/env.json

Change log

• 2026-10-31: Initial tiny Unlimited-OCR 2 checkpoint with the upstream SR-SWA runtime.
• 2026-11-01: Separate residual width from the 32-dimensional attention heads to reduce checkpoint size.

This repo is generated by GPT-6.1-Sol.