sana_rwm_policy_from_pretrained_robotwin_epoch1_step21500

RoboTwin2 Aloha-AgileX joint-only policy SFT of the unified RWM world model. Weights only, epoch 1 / step 21500 of the run sft_robotwin_aloha_unified_jointonly_f33_480px_2node_s57032.

metadata.pth self-reports {'epoch': 1, 'step': 21500}; the repo name is derived from it, not from the directory name.

Lineage

Warm-started from logits/sana_rwm_pretrained_epoch3_step57032 — the unified world model, pretraining run rwm_unified_action_mode_aw1_spatial_rope, epoch 3 / step 57032. Weights only; the donor's optimizer state and RNG were deliberately not inherited (it was a 64-rank run and an FSDP-sharded optimizer pins its world size).

This is NOT the same line as logits/sana_rwm_policy_from_video_robotwin. That one warm-started from a video-only donor that carried no robot modules, so its SFT had to waive six missing keys. This donor carries action_head.*, action_embed.*, state_embed.* and plucker_embed.* already, so nothing is waived — SANA_SFT_ALLOWED_MISSING_KEYS must stay UNSET when loading.

Model class SanaRWMVideoQwenNextSubAttnResV2SelfFlowWorldModelCameraConditionMultiViewPolicy_5B_P1_D36 — 805 tensors, 4.469 B parameters. pos_embed is force-dropped by load_checkpoint on every load (it is resolution-dependent); a single Missing keys: ['pos_embed'] warning is expected and correct. Anything else missing, or any unexpected key, is a real mismatch.

Held-out validation

50 RoboTwin2 benchmark tasks, one episode per task that the checkpoint has never seen (holdout_mode: eval, 5 episodes reserved per task/split). Action MSE in the normalized 80-slot space, over the 14 supervised joint slots.

ckpt step	windows seen	mean	median	max	tasks > 0.2
0 (donor)	0	0.1489	0.0674	0.7118	12
4000	512,000	0.1125	0.0297	1.0384	8
5000	640,000	0.0887	0.0286	0.5911	8
10000	1,280,000	0.0558	0.0158	0.3679	6
15000	1,920,000	0.0430	0.0147	0.3479	2
20000	2,560,000	0.0323	0.0128	0.1957	0
21500 (this)	2,752,000	not yet measured

Step 21500 falls between milestones — the curve is sampled every 5000 steps. The nearest measured point is 20000, 1500 steps earlier.

Read the x-axis carefully: the step counter is dataloader steps of 128 windows, not optimizer steps. gradient_accumulation_steps: 2, so one optimizer step is two of these and the effective batch is 256 windows. One epoch is 32,155 such steps; step 21500 is 67% of the first epoch, of three.

Training

• Corpus: RoboTwin2-Aloha-AgileX, 50 tasks x {clean, randomized}. 27,500 episodes, 500 held out (5 per task/split), 27,000 used.
• Windows: stride-1 exhaustive expansion to 4,115,952 windows of 33 frames at 30 fps (1.07 s each), one dataset entry per window, one visit per epoch.
• Views: 3 (head / left / right), 480 px. front_camera is in the dataset's view registry but NOT in the policy plan: SanaVideoActionPolicyDataset.deterministic_view_plan resolves every RoboTwin2-* tree through _ROBOTWIN2_POLICY_VIEWS, which is ('head_camera', 'left_camera', 'right_camera'). The cached latents' own identity payload agrees: view_keys: [head_camera, left_camera, right_camera], view_latent_shapes: [[15,20],[15,20],[15,20]], giving a (128, 5, 15, 60) fp16 window. (An earlier revision of this file said 4 views; that was read off the module-level videoaction_view_plan, which is not what the policy line uses.)
• Action mode: joint_only (action_mode_sample_ratio: [0.0, 0.0, 1.0]). 14 of the 80 slots are supervised — 12 arm joints (0-5, 29-34) plus 2 grippers (16, 45). The EEF slots are neither supervised nor fed as state (include_eef = False on this path).
• Joint targets: window-anchored delta, joint_target_mode: anchor_delta. q_t - q_0 is formed on RAW RADIANS and normalized afterwards — delta first, then the affine map. Not the other way round.
• lr: 1.0e-5 constant, action_loss_weight: 5.0, use_mobile_base: false, flow_shift 3.0 train.
• 2 nodes x 8 GPU, train_batch_size: 8 per device, accumulation 2.

The frozen run config is included as config.yaml.

Normalization — REQUIRED, and not interchangeable

normalization/robotwin2_aloha_agilex_model_fps_30_f33_normalization.json

This is the affine map the action head was fit to. Copy it to

<json_cache_dir>/action_mode/robot_base_eef/robotwin2_aloha_agilex_model_fps_30_f33_normalization.json

It declares joint_target_mode: anchor_delta, num_frames: 33, model_fps: 30, eef_origin: flange, quantiles [0.01, 0.99], and formula = normalized=where(mask,(value-center80)/scale80,value); no clipping.

Relationship to the file in logits/sana_rwm_policy_from_video_robotwin (..._f33_jointdelta_normalization.json): numerically identical — center80 and scale80 agree on all 80 slots for both the action and state blocks. They differ only in how the delta convention is stamped. The older file encoded it in action_representation: robot_base_qwen_jointdelta and left joint_target_mode absent; this one uses the explicit joint_target_mode: anchor_delta field with action_representation: robot_base_qwen. Either loads the same map, but this checkpoint's loader expects the explicit field, so use THIS file.

Do NOT substitute current_anchor_qwen_multiview/.... That artifact carries absolute joint targets (joint_action_delta: false) and will silently produce wrong actions with a delta-trained head.

Contents

metadata.pth                     epoch/step bookkeeping ({'epoch': 1, 'step': 21500})
model/pytorch_model_fsdp.bin     17,876,172,578 bytes, 805 tensors, fp32
model/scheduler.bin              LR scheduler state
normalization/...json            the affine map above
config.yaml                      the run's frozen config

Deliberately omitted:

• optimizer.bin (34 GB) — FSDP-sharded to this run's 16 ranks, so it pins the world size. Useless for anything but resuming this exact topology.
• random_states_*.pkl — per-rank RNG of this run.

This snapshot therefore cannot be resumed from; it is sized for --model.load_from, which reads metadata.pth and model/pytorch_model_fsdp.bin and nothing else.

Evaluation note

The RoboTwin evaluation server refuses a training config whose scheduler.inference_cfg_scale is anything but 6.0, so config.yaml still declares 6.0 — but every measurement above was taken at cfg 1.0, which is what the server actually runs. The sampler applies one scale to both heads (pred = uncond + s*(cond - uncond)), so s=6 would extrapolate the action regression 6x along the cond/uncond direction, a regime nothing downstream uses.