Qwen3.8-27B-Turkish-CPT-Beta (Full Merged BF16)

A standalone continued-pretrained model for Qwen3.8-27B, with Turkish language adaptation weights merged directly into full precision bfloat16 base weights.

Model Details

• Base Model: Qwen/Qwen3.8-27B (Text-only causal LM backbone)
• Architecture: Qwen3_5ForCausalLM
• Training Method: NF4 QLoRA (Rank 16, Alpha 16) continued pretraining on the Kumru Turkish corpus followed by standalone BF16 weight merge.
• Precision: bfloat16
• Format: HuggingFace safetensors (12 shards, ~51.5 GB total)

Evaluation & Benchmark Performance

Metric	Base Model (Qwen3.8-27B)	Turkish-CPT-Beta	Improvement
Turkish Validation Perplexity	7.5838	6.9360	-8.55%
Turkish Test Perplexity	8.0534	7.3303	-8.98%
TurkishMMLU (90-Q Stratified)	60.00%	75.56%	+15.56%
Belebele Reading Comprehension (TR)	83.33%	90.00%	+6.67%
Belebele Reading Comprehension (EN)	96.67%	96.67%	100% Retained

Usage with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "UgurI/Qwen3.8-27B-Turkish-CPT-Beta"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "Türkiye'de bilimsel araştırmaların geliştirilmesi için önerilerin nelerdir?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Usage with vLLM

vllm serve UgurI/Qwen3.8-27B-Turkish-CPT-Beta --tensor-parallel-size 2 --dtype bfloat16

Citation & Attribution

• Training Corpus: VNGRS-AI (vngrs-ai/vngrs-web-corpus)
• Base Model: Qwen Team (Qwen/Qwen3.8-27B)
• Author: Ugur Inanc