Script2Play-4B

The 4B vision-language-action policy of From Game Generation to Game Playing: Fully Automated Ability Training for Vision-Language-Action Models (project page). It plays games from the screen: one frame and a task in, the next 200 ms of keyboard and mouse out.

Qwen3.5-4B-Base post-trained by behavior cloning on 150.3 h of verified scripted-teacher demonstrations in generated games: 26.9 h of Minecraft and 123.4 h of Roblox. No ViZDoom or UE5 gameplay is in the training data, and no human gameplay, RL or DAgger is used.

Usage

Serve it with vLLM 0.26 and query its OpenAI-compatible endpoint; the inference client, the exact request the scores were measured with, and the benchmark harnesses are in AssetTool-Labs/script2play-inference.

vllm serve PATH_TO_THIS_REPO --served-model-name script2play --trust-remote-code \
  --max-model-len 8192 --max-num-batched-tokens 8192 --limit-mm-per-prompt '{"image":1,"video":0}' \
  --attention-backend FLASHINFER --mm-encoder-attn-backend TORCH_SDPA

Each call sends the brief system prompt with the task, and one frame (832 px wide for Doom-like and Roblox games, 640x360 as is for Minecraft) as a JPEG. The reply is one line, dx dy dz ; K1 ; K2 ; K3 ; K4: the mouse displacement in px over the 200 ms (0.15 degrees of view turn per px) and the scroll steps, then the keys held in each 50 ms step. Sample at temperature 0.8, top-p 0.99, at most 48 tokens, stopping at <|action_end|> or <|im_end|>, with thinking disabled.

Downloads last month
5
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for GameIntelligenceLab/Script2Play-4B

Finetuned
(330)
this model

Paper for GameIntelligenceLab/Script2Play-4B