ZeyuLing commited on
Commit
50f7b46
·
verified ·
1 Parent(s): f2b9cd8

Normalize Motius Pipeline.from_pretrained metadata

Browse files
Files changed (2) hide show
  1. README.md +31 -23
  2. model_index.json +34 -13
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- library_name: hftrainer
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
@@ -14,18 +14,18 @@ license: other
14
  # Go to Zero (MotionMillion)
15
 
16
  Million-scale, 3B/7B-parameter autoregressive text-to-motion model ("Go to Zero",
17
- ICCV 2025 **Highlight**) integrated into the hftrainer Model Zoo. Our
18
  reproduction is **fully self-contained and independent of the original
19
  repository** at runtime: the HumanVQVAE (FSQ tokenizer) and the LLaMA
20
  autoregressive transformer live in
21
- `hftrainer.models.motion.motionmillion.network`. Only the T2M inference path is
22
  exercised.
23
 
24
  | | |
25
  |---|---|
26
  | **Task** | Text-to-Motion (T2M), zero-shot |
27
  | **Bundle / Pipeline** | `MotionMillionBundle` / `MotionMillionPipeline` |
28
- | **Processed HF artifact** | [`7B-train`](https://huggingface.co/ZeyuLing/hftrainer-gotozero-7b-train-humanml272), [`3B-train`](https://huggingface.co/ZeyuLing/hftrainer-gotozero-3b-train-humanml272) |
29
  | **Motion representation** | **humanml3d_272** (272-dim, 30 fps) — *identical layout to MotionStreamer-272* |
30
  | **Tokenizer** | HumanVQVAE + **FSQ** (levels `[8,8,8,5,5,5]`, codebook 64000) |
31
  | **AR model** | LLaMA-style 3B / 7B AR transformer (RoPE, length-causal text cross-attn) |
@@ -39,14 +39,14 @@ exercised.
39
 
40
  ## Weights
41
 
42
- Current hftrainer artifact (diffusers-style `from_pretrained`):
43
 
44
  | Artifact | Location | Contents | Status |
45
  |---|---|---|---|
46
- | Go-to-Zero 7B train-only HumanML3D-272 | [`ZeyuLing/hftrainer-gotozero-7b-train-humanml272`](https://huggingface.co/ZeyuLing/hftrainer-gotozero-7b-train-humanml272) | `fsq.safetensors` + `ar.safetensors` + `mm_config.json` + `model_index.json` + `mean.npy` / `std.npy` + `text_encoder/` | public Hub artifact; Flan-T5-XL is packaged as safetensors |
47
- | Go-to-Zero 3B train-only HumanML3D-272 | [`ZeyuLing/hftrainer-gotozero-3b-train-humanml272`](https://huggingface.co/ZeyuLing/hftrainer-gotozero-3b-train-humanml272) | same layout | public Hub artifact; Flan-T5-XL is packaged as safetensors |
48
- | local mirror, 7B | `checkpoints/gotozero/hftrainer_7b_train_humanml272` | same layout | optional local cache |
49
- | local mirror, 3B | `checkpoints/gotozero/hftrainer_3b_train_humanml272` | same layout | optional local cache |
50
 
51
  The public leaderboard uses the **train-only** checkpoints for both 7B and 3B.
52
  The older `t2m_*_all.zip` checkpoints were trained with all HumanML3D splits and
@@ -55,13 +55,13 @@ should not be used for HumanML3D test-set leaderboard rows.
55
  **Use directly from the Hub (recommended):**
56
 
57
  ```python
58
- from hftrainer.pipelines.motionmillion import MotionMillionPipeline
59
 
60
  pipe = MotionMillionPipeline.from_pretrained(
61
- "ZeyuLing/hftrainer-gotozero-7b-train-humanml272",
62
  device="cuda",
63
  )
64
- # Use "ZeyuLing/hftrainer-gotozero-3b-train-humanml272" for the 3B train-only artifact.
65
  # Cast the AR to bf16 for memory-efficient inference:
66
  import torch
67
  pipe.bundle.ar.to(dtype=torch.bfloat16)
@@ -79,20 +79,20 @@ bundle = MotionMillionBundle(
79
  )
80
  ```
81
 
82
- Package the hftrainer artifact from local upstream weights:
83
 
84
  ```bash
85
  python3 scripts/eval/convert_motionmillion_checkpoint.py \
86
  --ar checkpoints/motionmillion/pretrained_models/motionmillion_7B.pth \
87
- --out_dir checkpoints/gotozero/hftrainer_7b_train_humanml272 \
88
- --repo_id ZeyuLing/hftrainer-gotozero-7b-train-humanml272 \
89
  --text_model_source checkpoints/flan-t5-xl \
90
  --verify
91
  ```
92
 
93
  Use the same converter with the 3B train-only checkpoint and
94
- `--out_dir checkpoints/gotozero/hftrainer_3b_train_humanml272 --repo_id
95
- ZeyuLing/hftrainer-gotozero-3b-train-humanml272` for the 3B artifact.
96
 
97
  ---
98
 
@@ -112,10 +112,10 @@ text -> Flan-T5-XL -> LLaMA-style AR (greedy, EOS-stopped, ≤150 tokens)
112
 
113
  The upstream demo sampler caps generation at 50 motion tokens (about 100 frames
114
  at 30 fps), which truncates many HumanML3D test clips. The model block size
115
- supports the full benchmark range, so the hftrainer reproduction uses
116
  `max_sample_steps=150` and writes exact-GT-length MS272 predictions.
117
 
118
- Convert to HumanML3D-263 with `hftrainer.motion.representation.convert`
119
  (`motion272_to_hml263`).
120
 
121
  ---
@@ -136,12 +136,12 @@ Reproduce generation with the packaged artifact:
136
  ```bash
137
  NGPU=6 TOTAL_SHARDS=6 LIMIT=0 \
138
  OUT=outputs/evaluation/t2m/humanml3d_official_test/ms272/gotozero_7b_train \
139
- ARTIFACT=checkpoints/gotozero/hftrainer_7b_train_humanml272 \
140
  STEPS=150 DTYPE=bf16 \
141
  bash scripts/eval/run_motionmillion_official272_exactlen_genonly.sh
142
  ```
143
 
144
- Swap `OUT` and `ARTIFACT` to the `gotozero_3b_train` / `hftrainer_3b_train_humanml272`
145
  paths for the 3B train-only row.
146
 
147
  ### MotionStreamer-272 evaluator (native space)
@@ -209,7 +209,7 @@ use it as a secondary diagnostic.
209
 
210
  ## Implementation notes
211
 
212
- - **hftrainer-native runtime**: `hftrainer/models/motion/motionmillion/network/` holds
213
  `fsq.py` (Finite Scalar Quantization), `resnet.py` / `modules.py` (Haar
214
  patch/unpatch) / `encdec.py` / `vqvae.py` (HumanVQVAE tokenizer) and
215
  `llama.py` (LLaMA AR). The `args` namespace was refactored into explicit
@@ -220,8 +220,16 @@ use it as a secondary diagnostic.
220
  state next to the `trans` weights; a tolerant unpickler stubs the unimportable
221
  classes so only the tensor `state_dict` is read (repo-independent).
222
  - **Complete text encoder artifact**: Flan-T5-XL is frozen and stored under
223
- `text_encoder/` in the hftrainer artifact. `MotionMillionBundle.from_pretrained`
224
  resolves that artifact-local directory automatically.
225
  - **Representation parity verified**: on smoke pairs the de-normalised prediction
226
  matches GT per-block scale almost exactly (e.g. position-block std 0.514 vs
227
  0.514; rot6d-block std 0.495 vs 0.489), confirming MotionMillion-272 ≡ MS-272.
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: motius
3
  pipeline_tag: other
4
  tags:
5
  - motion-generation
 
14
  # Go to Zero (MotionMillion)
15
 
16
  Million-scale, 3B/7B-parameter autoregressive text-to-motion model ("Go to Zero",
17
+ ICCV 2025 **Highlight**) integrated into the motius Model Zoo. Our
18
  reproduction is **fully self-contained and independent of the original
19
  repository** at runtime: the HumanVQVAE (FSQ tokenizer) and the LLaMA
20
  autoregressive transformer live in
21
+ `motius.models.motion.motionmillion.network`. Only the T2M inference path is
22
  exercised.
23
 
24
  | | |
25
  |---|---|
26
  | **Task** | Text-to-Motion (T2M), zero-shot |
27
  | **Bundle / Pipeline** | `MotionMillionBundle` / `MotionMillionPipeline` |
28
+ | **Processed HF artifact** | [`7B-train`](https://huggingface.co/ZeyuLing/Motius-MotionMillion-7B-HumanML272), [`3B-train`](https://huggingface.co/ZeyuLing/Motius-MotionMillion-3B-HumanML272) |
29
  | **Motion representation** | **humanml3d_272** (272-dim, 30 fps) — *identical layout to MotionStreamer-272* |
30
  | **Tokenizer** | HumanVQVAE + **FSQ** (levels `[8,8,8,5,5,5]`, codebook 64000) |
31
  | **AR model** | LLaMA-style 3B / 7B AR transformer (RoPE, length-causal text cross-attn) |
 
39
 
40
  ## Weights
41
 
42
+ Current motius artifact (diffusers-style `from_pretrained`):
43
 
44
  | Artifact | Location | Contents | Status |
45
  |---|---|---|---|
46
+ | Go-to-Zero 7B train-only HumanML3D-272 | [`ZeyuLing/Motius-MotionMillion-7B-HumanML272`](https://huggingface.co/ZeyuLing/Motius-MotionMillion-7B-HumanML272) | `fsq.safetensors` + `ar.safetensors` + `mm_config.json` + `model_index.json` + `mean.npy` / `std.npy` + `text_encoder/` | public Hub artifact; Flan-T5-XL is packaged as safetensors |
47
+ | Go-to-Zero 3B train-only HumanML3D-272 | [`ZeyuLing/Motius-MotionMillion-3B-HumanML272`](https://huggingface.co/ZeyuLing/Motius-MotionMillion-3B-HumanML272) | same layout | public Hub artifact; Flan-T5-XL is packaged as safetensors |
48
+ | local mirror, 7B | `checkpoints/gotozero/motius_7b_train_humanml272` | same layout | optional local cache |
49
+ | local mirror, 3B | `checkpoints/gotozero/motius_3b_train_humanml272` | same layout | optional local cache |
50
 
51
  The public leaderboard uses the **train-only** checkpoints for both 7B and 3B.
52
  The older `t2m_*_all.zip` checkpoints were trained with all HumanML3D splits and
 
55
  **Use directly from the Hub (recommended):**
56
 
57
  ```python
58
+ from motius.pipelines.motionmillion import MotionMillionPipeline
59
 
60
  pipe = MotionMillionPipeline.from_pretrained(
61
+ "ZeyuLing/Motius-MotionMillion-7B-HumanML272",
62
  device="cuda",
63
  )
64
+ # Use "ZeyuLing/Motius-MotionMillion-3B-HumanML272" for the 3B train-only artifact.
65
  # Cast the AR to bf16 for memory-efficient inference:
66
  import torch
67
  pipe.bundle.ar.to(dtype=torch.bfloat16)
 
79
  )
80
  ```
81
 
82
+ Package the motius artifact from local upstream weights:
83
 
84
  ```bash
85
  python3 scripts/eval/convert_motionmillion_checkpoint.py \
86
  --ar checkpoints/motionmillion/pretrained_models/motionmillion_7B.pth \
87
+ --out_dir checkpoints/gotozero/motius_7b_train_humanml272 \
88
+ --repo_id ZeyuLing/Motius-MotionMillion-7B-HumanML272 \
89
  --text_model_source checkpoints/flan-t5-xl \
90
  --verify
91
  ```
92
 
93
  Use the same converter with the 3B train-only checkpoint and
94
+ `--out_dir checkpoints/gotozero/motius_3b_train_humanml272 --repo_id
95
+ ZeyuLing/Motius-MotionMillion-3B-HumanML272` for the 3B artifact.
96
 
97
  ---
98
 
 
112
 
113
  The upstream demo sampler caps generation at 50 motion tokens (about 100 frames
114
  at 30 fps), which truncates many HumanML3D test clips. The model block size
115
+ supports the full benchmark range, so the motius reproduction uses
116
  `max_sample_steps=150` and writes exact-GT-length MS272 predictions.
117
 
118
+ Convert to HumanML3D-263 with `motius.motion.representation.convert`
119
  (`motion272_to_hml263`).
120
 
121
  ---
 
136
  ```bash
137
  NGPU=6 TOTAL_SHARDS=6 LIMIT=0 \
138
  OUT=outputs/evaluation/t2m/humanml3d_official_test/ms272/gotozero_7b_train \
139
+ ARTIFACT=checkpoints/gotozero/motius_7b_train_humanml272 \
140
  STEPS=150 DTYPE=bf16 \
141
  bash scripts/eval/run_motionmillion_official272_exactlen_genonly.sh
142
  ```
143
 
144
+ Swap `OUT` and `ARTIFACT` to the `gotozero_3b_train` / `motius_3b_train_humanml272`
145
  paths for the 3B train-only row.
146
 
147
  ### MotionStreamer-272 evaluator (native space)
 
209
 
210
  ## Implementation notes
211
 
212
+ - **motius-native runtime**: `motius/models/motion/motionmillion/network/` holds
213
  `fsq.py` (Finite Scalar Quantization), `resnet.py` / `modules.py` (Haar
214
  patch/unpatch) / `encdec.py` / `vqvae.py` (HumanVQVAE tokenizer) and
215
  `llama.py` (LLaMA AR). The `args` namespace was refactored into explicit
 
220
  state next to the `trans` weights; a tolerant unpickler stubs the unimportable
221
  classes so only the tensor `state_dict` is read (repo-independent).
222
  - **Complete text encoder artifact**: Flan-T5-XL is frozen and stored under
223
+ `text_encoder/` in the motius artifact. `MotionMillionBundle.from_pretrained`
224
  resolves that artifact-local directory automatically.
225
  - **Representation parity verified**: on smoke pairs the de-normalised prediction
226
  matches GT per-block scale almost exactly (e.g. position-block std 0.514 vs
227
  0.514; rot6d-block std 0.495 vs 0.489), confirming MotionMillion-272 ≡ MS-272.
228
+
229
+ ## Direct Loading
230
+
231
+ ```python
232
+ from motius import Pipeline
233
+
234
+ pipeline = Pipeline.from_pretrained("ZeyuLing/Motius-MotionMillion-3B-HumanML272")
235
+ ```
model_index.json CHANGED
@@ -1,19 +1,40 @@
1
  {
2
  "_class_name": "MotionMillionPipeline",
3
- "_diffusers_version": "hftrainer",
4
- "pipeline": {
5
- "library": "hftrainer",
6
- "class_name": "MotionMillionPipeline",
7
- "module": "hftrainer.pipelines.motionmillion"
8
- },
9
- "bundle": {
10
- "library": "hftrainer",
11
- "class_name": "MotionMillionBundle",
12
- "module": "hftrainer.models.motion.motionmillion"
13
- },
14
  "weights": {
15
  "fsq": "fsq.safetensors",
16
  "ar": "ar.safetensors",
17
  "text_encoder": "text_encoder"
18
- }
19
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  {
2
  "_class_name": "MotionMillionPipeline",
 
 
 
 
 
 
 
 
 
 
 
3
  "weights": {
4
  "fsq": "fsq.safetensors",
5
  "ar": "ar.safetensors",
6
  "text_encoder": "text_encoder"
7
+ },
8
+ "_library_name": "motius",
9
+ "format_version": 1,
10
+ "pipeline_class": "motius.pipelines.motionmillion.MotionMillionPipeline",
11
+ "bundle_class": "motius.models.motionmillion.MotionMillionBundle",
12
+ "tasks": [
13
+ "text_to_motion"
14
+ ],
15
+ "required_files": [
16
+ "ar.safetensors",
17
+ "fsq.safetensors",
18
+ "mean.npy",
19
+ "mm_config.json",
20
+ "std.npy",
21
+ "text_encoder/.gitattributes",
22
+ "text_encoder/README.md",
23
+ "text_encoder/config.json",
24
+ "text_encoder/generation_config.json",
25
+ "text_encoder/model-00001-of-00002.safetensors",
26
+ "text_encoder/model-00002-of-00002.safetensors",
27
+ "text_encoder/model.safetensors.index.json",
28
+ "text_encoder/special_tokens_map.json",
29
+ "text_encoder/spiece.model",
30
+ "text_encoder/tokenizer.json",
31
+ "text_encoder/tokenizer_config.json"
32
+ ],
33
+ "api": {
34
+ "loader": "motius.Pipeline.from_pretrained",
35
+ "task_methods": [
36
+ "infer_text_to_motion"
37
+ ]
38
+ },
39
+ "artifact_format": "motius-motionmillion-v1"
40
+ }