SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling
Paper • 2608.24334 • Published
SeMoCo couples a semantically supervised motion codec with body-structured flow matching for text-to-motion generation. An autoregressive model predicts a semantic code sequence from text, and a conditional flow generates continuous kinematic features for the root, torso, and limbs. The semantic codes and kinematic features are decoded together into motion.
This repository contains the pretrained HumanML3D and Ω-MotionVerse models for the main results.
| File | Contents |
|---|---|
humanml3d/tokenizer.pt |
HumanML3D codec |
humanml3d/semantic_ar.pt |
HumanML3D semantic model |
humanml3d/flow.pt |
HumanML3D six-part flow |
humanml3d/text_encoder.pt |
HumanML3D frozen sentence encoder |
humanml3d/kin_stats.npz, humanml3d/feature_stats.npz |
Latent and Guo feature normalization |
omega/tokenizer.pt |
Ω-MotionVerse codec |
omega/semantic_ar.pt |
Ω-MotionVerse semantic model |
omega/flow.pt |
Ω-MotionVerse six-part flow |
omega/text_encoder.pt |
Ω-MotionVerse frozen sentence encoder |
omega/kin_stats.npz, omega/feature_stats.npz |
Latent and UMR499 normalization |
omega/temporal_parent40k.pt |
Ω-MotionVerse pretrained temporal flow backbone |
omega/evaluator_v2.pt, omega/norm_stats.json |
Ω-MotionVerse V2 evaluator and 30 Hz statistics |
| Dataset | FID | R@1 | R@2 | R@3 | MM-Dist |
|---|---|---|---|---|---|
| HumanML3D | 0.0327 | 0.5561 | 0.7473 | 0.8365 | 2.750 |
| Ω-MotionVerse | 0.01238 | 0.7325 | 0.8143 | 0.8446 | 0.963 |
HumanML3D reports the mean of 20 repeats with the checkpoint held fixed. Ω-MotionVerse is one evaluation of the full test set.
hf download poisonousID/SeMoCo --local-dir checkpoints
Encode prompts and generate motion with the code repository:
semoco-encode-text --dataset humanml3d --prompts examples/prompts.json \
--output runs/prompts.npz
semoco-generate --config configs/humanml3d_inference.yaml \
--inputs runs/prompts.npz --output runs/motion.npz
Use --dataset omega and configs/omega_inference.yaml for Ω-MotionVerse.
Apache-2.0.
@article{semoco,
title = {SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching},
author = {Huang, Tianlv and Guo, Hetian and Cai, Ziyi and Liu, Yu and Peng, Yingzhi and Wang, Song and Fan, Zipei and Song, Xuan and Wu, Guangming and Zheng, Xin},
journal = {arXiv preprint arXiv:2608.24334},
year = {2026}
}