cwnd/Kimi-K3-MoonViT-V2-INT8-W8A16

Kimi-K3 MoonViT-V2 — INT8 W8A16

Standalone Kimi-K3 MoonViT-V2 vision tower with all 108 Transformer nn.Linear layers quantized to bitsandbytes INT8 weight-only format. Patch Conv2d, learned position embedding, RMSNorm and activations remain FP16.

Size and fidelity

Version	Checkpoint size	Pooled feature cosine vs FP16
Original BF16	802.45 MB	1.0
This INT8 W8A16	407.37 MB	0.99617 mean

The INT8 checkpoint was tested on 60 internet images across 16 groups, including photos, anime, manga, UI screenshots, charts, medical imagery, artwork, adult glamour/boudoir, night scenes, and satellite imagery. All outputs were finite. See INT8_STRESS_REPORT.md and the raw JSON results.

Files

• moonvit_v2_bnb_int8_w8a16.pt — packed INT8 checkpoint
• load_int8_moonvit.py — standalone loader
• moonvit.py — MoonViT-V2 architecture
• vision_config.json — configuration
• BUILD_MANIFEST.json — exact size, SHA-256 and smoke test
• QUANTIZATION_REPORT.md, INT8_STRESS_REPORT.md — evaluation reports
• Raw result JSON files
• LICENSE — upstream Kimi K3 License

Requirements

pip install torch bitsandbytes>=0.50

A CUDA GPU supported by bitsandbytes is required by this loader.

Loading

from huggingface_hub import snapshot_download
import sys

repo_dir = snapshot_download("cwnd/Kimi-K3-MoonViT-V2-INT8-W8A16")
sys.path.insert(0, repo_dir)

from load_int8_moonvit import load_int8_moonvit

model = load_int8_moonvit(
    f"{repo_dir}/moonvit_v2_bnb_int8_w8a16.pt",
    device="cuda",
)

The model accepts pre-patchified pixel values and a [T,H,W] grid:

outputs = model(pixel_values, grid_thws)
# For one 224×224 image:
# outputs[0].shape == [64, 4, 1024]

pixel_values has shape [num_patches, 3, 14, 14]. grid_thws contains one row per image, with height and width expressed in patch units.

Verification

Built and tested on a Tesla T4:

• 108 linear layers quantized
• Smoke output shape: [1,4,1024]
• No NaN/Inf
• Saved checkpoint reloaded successfully with the included loader
• Broad stress-set pooled feature cosine vs FP16: 0.99617 mean, 0.99294 minimum
• Pairwise similarity Pearson correlation vs FP16: 0.99791
• Top-1 nearest-neighbor agreement: 95%

See BUILD_MANIFEST.json for the checkpoint SHA-256.

Limitations

• This is the vision tower only; it does not generate text.
• Input preprocessing must match Kimi-K3/MoonViT-V2.
• On a Tesla T4, bitsandbytes INT8 was smaller but slower than FP16; the main benefit is storage/VRAM reduction.
• End-to-end Kimi-K3 VQA/OCR benchmarks were not run.

License

This is a derivative of Moonshot AI's Kimi K3 weights and remains subject to the Kimi K3 License included in this repository.