KIEFERSA/Sophea-Qwen3.6-v1.1

Sophea-Qwen3.6-v1.1

RLVR refinement of Sophea-Qwen3.6-v1: verifiable rewards, same thinking model, vision retained, MTP speculative decoding

📄 Released with the paper Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See (arXiv:2608.17744). Every number on this card is measured, contextualized, and caveated in the paper; read it for the full evaluation protocol.

• Creator: Kiefer SA (Sophea AI Lab, Athens)
• Base model: Qwen3.6-35B-A3B via Sophea-Qwen3.6-v1 (sparse MoE transformer, 36.0B total / 3.97B active)
• Ships with: the base's full multimodal (vision) stack, untouched, and the multi-token-prediction draft head for speculative decoding
• Languages: Greek and English (the reasoning trace follows the question's language)
• Decoding: thinking (enable_thinking=true); the trace is the product and follows the question's language

Serve thinking. Identical serving to Sophea-Qwen3.6-v1: leave enable_thinking=true and, on vLLM, add --reasoning-parser qwen3. The trace follows the question's language.

Intended use

• Same envelope as Sophea-Qwen3.6-v1: Greek and English reasoning with auditable traces
• Preferred over v1 where answer-format compliance or a clean answer channel matters (2.8% fallback, 0.02% leak)

The instructed el→EN override reached 53.9%, up from 44.8% but short of the pre-registered 60% trainability target: improved, not solved. Not evaluated for safety-critical, legal, or medical use.

---

Evaluation

Arrows mark the desirable direction for every metric: ↑ higher is better, ↓ lower is better.

What v1.1 changes over v1

Axis (same instrument, same day)	Sophea-Qwen3.6-v1 (SFT)	Sophea-Qwen3.6-v1.1 (RLVR)
Greek-trace fidelity, Greek questions (n=5,156) ↑	97.98%	98.27%
Answer-format fallback ↓	24.1%	2.8%
Answer-channel leak ↓	3.53%	0.02%
Instructed override, el→EN ("think in English") ↑	44.8%	53.9%
Instructed override, en→EL (Greek suffix) ↑	83.7%	85.7%
Agreeing instruction (el→EL) ↑	98.5%	98.5% (unchanged)
In-question switches / 100 sentences ↓	0.0	0.0
Pooled Greek reasoning accuracy ↑	73.7	77.0 (inside the 7.7 pp seed floor, not claimed as a gain)

Every movement above was adjudicated against a random-reward control arm trained on the same data, steps and seed: the control reproduced the v1 baseline on every axis (fallback 22.1%, leak 3.61%, override 44.1%), so none of the movements are elicitation artifacts. The arm trained with no language protection ended at 98.22% fidelity, above its starting point: the Greek reasoning habit survived an accuracy-only gradient.

Per-benchmark detail

The pre-registered RLVR round, all four arms

Arm (all: same data, steps, seed, hardware)	Fidelity %	Acc	Fallback %	Leak %	el→EN %	en→EL %
SFT checkpoint (before)	97.98	73.7	24.1	3.53	44.8	83.7
Random reward (control)	98.06	74.0	22.1	3.61	44.1	83.8
Correctness + format + termination	98.22	77.2	2.5	0.00	48.8	84.7
+ language consistency	98.02	77.2	5.6	0.04	45.9	84.5
+ override (released as v1.1)	98.27	77.0	2.8	0.02	53.9	85.7

Accuracy is shown for completeness only: +3.3 is inside the 7.7 pp seed floor and is claimed by nobody. The forgetting check (Titan-1 suite, 14 bilingual NLU benchmarks) was not re-run for v1.1: the SFT release it refines measured −0.01 Greek / +0.08 English against its own base, statistically flat, i.e. no forgetting, and the RLVR round's own gates held fidelity and both language directions throughout training.

Method

Sophea-Qwen3.6-v1.1 is a reinforcement-learning refinement of Sophea-Qwen3.6-v1 using verifiable rewards, pre-registered before training and adjudicated against a random-reward control. Full method, defenses and decision rules are in the paper.

Stated honestly: the pre-registered override target of 60% was missed; 53.9% was reached, so the paper words the steerability result as reward-responsive, not trainable.

Usage

Serve with vLLM (OpenAI-compatible; the fused-expert LoRA is already merged into dense weights):

vllm serve KIEFERSA/Sophea-Qwen3.6-v1.1 --served-model-name sophea-qwen3.6-v1_1 --reasoning-parser qwen3

Client (OpenAI SDK), thinking left ON:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="sophea-qwen3.6-v1_1",
    messages=[{"role": "user", "content": "Ένα κατάστημα πουλά στυλό προς 1,50 € το ένα. Πόσο κοστίζουν 12 στυλό;"}],
    temperature=0.7,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},   # thinking model
)
print(resp.choices[0].message.reasoning_content)  # the trace, in the question's language
print(resp.choices[0].message.content)            # the final answer

Transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("KIEFERSA/Sophea-Qwen3.6-v1.1")
model = AutoModelForCausalLM.from_pretrained(
    "KIEFERSA/Sophea-Qwen3.6-v1.1", torch_dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Ένα τρένο διανύει 240 χλμ σε 3 ώρες. Ποια είναι η μέση ταχύτητά του;"}]
text = tok.apply_chat_template(messages, tokenize=False,
                               add_generation_prompt=True, enable_thinking=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=2048, temperature=0.7, do_sample=True)
print(tok.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=False))

Steering the trace language (partial, measured per checkpoint): append "Think step by step IN ENGLISH." to a Greek question, or "Σκέψου βήμα προς βήμα ΣΤΑ ΕΛΛΗΝΙΚΑ." to an English one. This checkpoint follows the English steering suffix on 53.9% of Greek items and the Greek suffix on 85.7% of English items.

Speculative decoding (MTP)

This model ships the multi-token-prediction head: 19 mtp.* tensors (~1.7 GB, bf16) in model-mtp.safetensors, the single-layer draft stack that config.json declares through mtp_num_hidden_layers: 1. The head is the base model's own; speculative decoding verifies every drafted token against the main model, so it changes throughput only, never the output distribution.

Enable it with vLLM:

vllm serve KIEFERSA/Sophea-Qwen3.6-v1.1 --served-model-name sophea-qwen3.6-v1.1 --trust-remote-code \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":2}'

Vision

The full vision stack of the multimodal base is included, byte-identical to the base: image inputs work exactly as on Qwen3.6-35B-A3B. The adaptation touched only the language decoder; vision performance has not been separately re-evaluated for this release.

License

Apache-2.0. The base model, Qwen/Qwen3.6-35B-A3B, is released under Apache-2.0 and this fine-tune inherits those terms. Verify base-model terms before use.

Citation

This model is a release artifact of arXiv:2608.17744. If you use it, please cite:

@misc{kirouane2026thinking,
      title={Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See},
      author={Ayoub Kirouane and Christos Petrocheilos},
      year={2026},
      eprint={2608.17744},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.17744},
}