SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching

Code arXiv Project

Introduction

SeMoCo couples a semantically supervised motion codec with body-structured flow matching for text-to-motion generation. An autoregressive model predicts a semantic code sequence from text, and a conditional flow generates continuous kinematic features for the root, torso, and limbs. The semantic codes and kinematic features are decoded together into motion.

This repository contains the pretrained HumanML3D and Ω-MotionVerse models for the main results.

Model Files

File Contents
humanml3d/tokenizer.pt HumanML3D codec
humanml3d/semantic_ar.pt HumanML3D semantic model
humanml3d/flow.pt HumanML3D six-part flow
humanml3d/text_encoder.pt HumanML3D frozen sentence encoder
humanml3d/kin_stats.npz, humanml3d/feature_stats.npz Latent and Guo feature normalization
omega/tokenizer.pt Ω-MotionVerse codec
omega/semantic_ar.pt Ω-MotionVerse semantic model
omega/flow.pt Ω-MotionVerse six-part flow
omega/text_encoder.pt Ω-MotionVerse frozen sentence encoder
omega/kin_stats.npz, omega/feature_stats.npz Latent and UMR499 normalization
omega/temporal_parent40k.pt Ω-MotionVerse pretrained temporal flow backbone
omega/evaluator_v2.pt, omega/norm_stats.json Ω-MotionVerse V2 evaluator and 30 Hz statistics

Evaluation

Dataset FID R@1 R@2 R@3 MM-Dist
HumanML3D 0.0327 0.5561 0.7473 0.8365 2.750
Ω-MotionVerse 0.01238 0.7325 0.8143 0.8446 0.963

HumanML3D reports the mean of 20 repeats with the checkpoint held fixed. Ω-MotionVerse is one evaluation of the full test set.

Usage

hf download poisonousID/SeMoCo --local-dir checkpoints

Encode prompts and generate motion with the code repository:

semoco-encode-text --dataset humanml3d --prompts examples/prompts.json \
  --output runs/prompts.npz
semoco-generate --config configs/humanml3d_inference.yaml \
  --inputs runs/prompts.npz --output runs/motion.npz

Use --dataset omega and configs/omega_inference.yaml for Ω-MotionVerse.

License

Apache-2.0.

Citation

@article{semoco,
  title   = {SeMoCo: Semantic Motion Tokens with Body-Structured Flow Matching},
  author  = {Huang, Tianlv and Guo, Hetian and Cai, Ziyi and Liu, Yu and Peng, Yingzhi and Wang, Song and Fan, Zipei and Song, Xuan and Wu, Guangming and Zheng, Xin},
  journal = {arXiv preprint arXiv:2608.24334},
  year    = {2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for poisonousID/SeMoCo