AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)
Paper • 2608.13492 • Published
Image-to-Video
Diffusers
AlayaWorldPipeline
alayaworld
video-generation
world-model
distillation
lora
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("fill-in-base-model", dtype=torch.bfloat16, device_map="cuda")
pipe.load_lora_weights("AlayaLab/AlayaWorld-v1.1-stage3")
prompt = "A man with short gray hair plays a red electric guitar."
input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png")
image = pipe(image=input_image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")AlayaWorld v1.1 — Stage3 Few-Step Student (DMD LoRA)
4-step DMD-distilled student of AlayaWorld (real-time inference). LoRA on top of the stage2b AR teacher; LTX-2 Community License.
| File | Size | Content |
|---|---|---|
lora.safetensors |
2.5 GB | generator LoRA (rank 256) |
history_encoder.pt |
33 MB | compressed frame-history encoder |
Usage — with AlayaLab/AlayaWorld:
paths:
resume_checkpoint: <stage2b download dir> # teacher base
dmd_resume: <this repo download dir> # lora.safetensors
history_encoder: <this repo download dir>/history_encoder.pt
Run: VALIDATE_ONLY=1 CONFIG_PATH=configs/infer_i2v_camera.yaml bash scripts/finetune/train.sh
- Downloads last month
- 306