vvennelakanti/samgo-glimmer-lora-r11

SAMgo Glimmer (run 11): a LoRA that builds working LEGO mechanisms

The builder model of SAMgo: one photo of a machine becomes a LEGO model whose mechanisms work (wheels roll, doors swing, rotors spin) and that you can actually build. Made for the Meta Global AI Developer Hackathon (Oct 27 - Nov 4, 2026).

This is a LoRA adapter (PEFT, rank 32, alpha 32, on q/k/v/o and the MLP projections, BF16) on Muse Glimmer 30B, trained with serverless SFT on Fireworks. Training data, script and recipe: vvennelakanti/samgo-glimmer-train-r11.

What it does

Given a LEGO model as a numbered parts list (studs in x/y, plates in z) and one generic sentence ("add the running gear", "one edge of a flat structure should swing", "set a rotor into this face"), it answers with a JSON edit list (clear, remove, set, add) that builds the mechanism: Technic bricks and axles for wheels, hinge plates for doors and flaps, axles and blade crosses for rotors, turntables for upright rotors. If SAMgo's checker rejects the answer, it receives the checker's exact feedback ("input part 86 overlaps your bar plate: remove it first", "this clip holds nothing") and fixes its answer, up to 3 rounds.

In SAMgo, Muse Spark decides what moves and where; this model decides how to build it; code verifies every answer (real parts only, no overlaps, every part held by studs, the mechanism moves clear, the model stands).

Training data

15,680 rows, all generated and verified by SAMgo's own checker, no object names: generic bodies (blocks, panels, lids, fins, shelves, car-like shells) with generic skills (running gear, hinge a strip or a sheet's edge, a leg with a wheel, a side rotor, an upright rotor on a turntable). About half the skill rows are mistake -> checker feedback -> fix, including the model's own earlier mistakes. Airplanes, helicopters, fans and turbines were never in the data: they were held-out exams.

Recipe: LoRA rank 32, batch 64, 2 epochs, max sequence length 24,576.

Results (frozen test sets, never trained on)

First try / within 3 feedback rounds.

Test (unseen bodies)	Run 11
Side rotor set into a face, spins 360° clear (30)	28
Upright rotor: blades on a turntable, turn 360° clear (30)	19 / 24
Hinge a flat sheet's edge (30)	14 / 20
Hinge a strip on a block (20)	14 / 15
Add a leg with a wheel (20)	20
Car running gear, held-out cars (50)	48 / 50
Door hinges on unseen vehicles (10)	10
Objects: place or repair a mechanism (112)	92

Earlier runs of this project scored 0 of 10 on the door-hinge test (run 4) and 0 of 20 on strip hinges (run 5); run 11 scores 10/10 and 15/20.

Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("meta-models/Muse-Glimmer-30B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "vvennelakanti/samgo-glimmer-lora-r11")
tok = AutoTokenizer.from_pretrained("vvennelakanti/samgo-glimmer-lora-r11")

The prompt format, edit format and checker are in the SAMgo repository (samgo/audit_objects.py messages, samgo/edits.py, samgo/free_edit_test.py); the pipeline calls the model through samgo/assembly.py.