Hugging Face's logo

CMSManhattan
/
JiRackCoderQwen_7b

ONNX
English
qwen2
code
coding
qwen2.5
int8
web-ui
Model card Files
xet
 Community

JiRack Coder 7B INT8

A fast and efficient coding assistant with a clean built-in web UI, powered by Qwen2.5-Coder-7B-Instruct base and optimized using Microsoft ONNX Runtime.

Quick Start

Watch the JiRack Coder 7B in action: DEMO: JiRack Coder 7B Web UI

Run with Docker

--Default CPU--

  • docker run -d
    --name jirack_coder_7b
    -p 7869:7869
    --restart unless-stopped
    cmsmanhattan/jirack_coder_7b_int8_qwenbase:latest

--Multi CPU--

  • docker run -d
    --name jirack_coder_7b
    -p 7869:7869
    --restart unless-stopped
    --memory=20g
    --cpus=12
    cmsmanhattan/jirack_coder_7b_int8_qwenbase:latest

---GPU--

  • docker run -d
    --name jirack_coder_7b
    -p 7869:7869
    --gpus all
    --restart unless-stopped
    cmsmanhattan/jirack_coder_7b_int8:1.0.2

Access the UI

Once the container is running, open your browser and navigate to:

http://localhost:7869

This opens the JiRack Coder UI — a clean web interface designed for coding.

Changing the Port

The listening port can be easily modified directly from the Settings panel within the JiRack Coder UI.

Licensing

  • The JiRack Coder 7B model is released under the GNU General Public License v3.0 (GPL-3.0).
  • All JiRack UI clients are provided under a commercial license.
  • However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.

JiRack Coder 32B is available exclusively under a commercial enterprise license.

For commercial licensing, cluster deployment, or enterprise use of the JiRack Coder 32B and JiRack Coder 14B , please contact us.

Hardware Recommendations for AMD Systems

Recommended Hardware for JiRack Coder 7B INT8 . It is one dcoker container

Use Case CPU GPU (ROCm) VRAM / RAM Expected Speed Recommendation
Recommended Ryzen 7 7700 / 9700X RX 7900 XTX / 7900 XT 24GB VRAM 50-75 tokens/s Best choice
High Performance Ryzen 9 7950X / 9950X RX 7900 XTX 24GB+ VRAM 65-90 tokens/s Excellent
Enterprise EPYC 7003/9004 series MI300X or 2x RX 7900 XTX 48GB+ VRAM 90-140 tokens/s For 32B model
Budget Option Ryzen 5 7600 / 9600X RX 7800 XT (16GB) 16GB VRAM 35-50 tokens/s Acceptable

Important Memory Notes

Even though the 7B INT8 model itself takes approximately 8–9 GB, we recommend at least 24GB VRAM for the following reasons:

  • KV-cache consumption during generation (especially with long context)
  • ONNX Runtime overhead and temporary buffers
  • System stability and to avoid Out of Memory errors
  • Room for larger context windows

Minimum recommended: 24GB VRAM (RX 7900 series)
Ideal: 24–32GB VRAM

For pure CPU inference (no GPU), we recommend at least 64GB system RAM (Ryzen 9 7950X/9950X).

I added the default model in full FP32 precision, which is approximately 62 GB in size. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.

📧 Contact & Licensing

For joint venture opportunities, hardware integration, or licensing inquiries:

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support