QwenBend: LoRA adapter

Qwen3.5-0.8B writes a Bend program. Then the same model type-checks it and runs it inside its own forward pass. Code, design notes and the 15-second demo: https://github.com/voidash/qwenbend

This repository holds the LoRA adapter that teaches Qwen3.5-0.8B to write BendLite, a subset of Bend2.

The engine half of QwenBend is not trained. Its weights are set by construction, from code in the GitHub repo:

• a 16-register CPU with 8,192 words of memory held in Gated DeltaNet heads;
• the CPU applies Bend2's static rules (types, affine use, termination) and executes the program.

python bendvm/qwenbend.py "your task" there builds the full 3.97B model in memory in about 15 s from Qwen3.5-0.8B and this adapter:

• 0.753B nonzero weights are Qwen's;
• 52.9M are the engine's;
• the rest is zero padding from widening the hidden size to 4096.

The adapter

• adapter/adapters.safetensors + adapter/adapter_config.json: mlx_lm LoRA format.
  • rank 8, scale 20, the last 16 layers;
  • trained with lr 1e-4, batch 2;
  • the 400-step checkpoint was best by validation loss (training to 1,200 steps overfit).
• qwen_fused.py in the GitHub repo fuses it into the HF weights in torch: W += 20·(A·B)ᵀ.
• sha256 of adapters.safetensors: 4577f177fdaabd10ae0f920f71a93d8f89603236b05484822bf0393e4caf315c.

Measured

On 200 test items: greedy, each program checked and run with real Bend2.

model	valid	correct	in-distribution correct	held-out families correct
Qwen3.5-0.8B (no adapter)	0%	0%	0%	0%
+ this adapter	58%	42%	80%	4%

End to end in the merged model (Apple M4 Pro, MPS, bf16): 5/5 programs correct, and every execution trace equals the reference interpreter's token for token. About 20 tokens/s.

Limits

The writer is a 0.8B model: it often writes the wrong program for unfamiliar tasks. The engine guarantees the answer is exactly what the program computes, not that the program fits the question.

License: Apache-2.0 (same as the base model). Built at Ampixa.