MODUS

MODUS for any-to-any generation (15 aligned modalities) as presented in MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities.

Project page: https://modus-multimodal.epfl.ch/ Code: https://github.com/EPFL-VILAB/Modus

Inference config (important):

• modality config: conf/modalities/instruction_16mod_stage2.yaml
• weights are bf16.

Files

• model.safetensors — trained weights (bf16)
• ae.safetensors — VAE (image decode)
• config.json / llm_config.json / vit_config.json — architecture config
• vocab.json / merges.txt / tokenizer_config.json — tokenizer