Kimi-K3-0.40B-Kazakh-CPT-step200

Instructions to use AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200", trust_remote_code=True)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200

SGLang

How to use AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200 with Docker Model Runner:
```
docker model run hf.co/AMAImedia/Kimi-K3-0.40B-Kazakh-CPT-step200
```

Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).

Founder: Ilia Bolotnikov
Organization: AMAImedia.com
X (Twitter): @AMAImediacom
LinkedIn: Ilia Bolotnikov
Telegram: @djbionicl
NOESIS version: v16.1
Release date: 2026-08

Kimi-K3-0.40B Kazakh CPT — Step 200

An experimental Kazakh continued-pretraining checkpoint based on inference-optimization/Kimi-K3-0.40B.

This is an early smoke-test checkpoint created to validate a complete Kazakh language-model pretraining pipeline on Kaggle.

Important limitation

The base checkpoint is a tiny architecture-development version of Kimi K3. Its weights were initialized from scratch and trained only on a toy copypasta dataset. It is not a distilled or compressed version of the trained 2.8T Kimi K3 model.

This checkpoint is therefore an educational research artifact, not a production language model.

Training

Dataset: kz-transformers/multidomain-kazakh-dataset
Objective: causal language modeling
Optimizer steps: 200
Tokens processed: 1,638,400
Sequence length: 512
Effective batch: 8,192 tokens
Initial learning rate: 3e-4
Hardware: 1× NVIDIA T4
Runtime: approximately 12.7 minutes
Tokenization: native Kimi tokenizer accelerated with Gigatoken

Results

Metric	Before	After
Validation loss	16.7615	4.9539
Validation perplexity	19,030,254	141.72

Example generations

The model has started learning Kazakh Cyrillic, morphology, punctuation, and sentence structure, but its generations remain semantically incoherent and contain invented words. This is expected after only 1.64M training tokens.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "Eraly-ml/Kimi-K3-0.40B-Kazakh-CPT-step200"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.float16,
    device_map="auto",
)

inputs = tokenizer(
    "Қазақстан —",
    return_tensors="pt",
).to(model.device)

output = model.language_model.generate(
    **inputs,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.8,
    top_p=0.95,
)

print(tokenizer.decode(output[0], skip_special_tokens=True))

Intended use

This checkpoint is intended for:

studying Kazakh language adaptation; inspecting Kimi K3's tiny MoE architecture; testing dataset and pretraining pipelines; comparing intermediate language acquisition checkpoints.

It should not be used for factual, safety-critical, or production tasks.