Any-to-Any
Transformers
Safetensors
English
Chinese
neo_chat
feature-extraction
native multimodal
image-generation
image-editing
custom_code
Instructions to use sensenova/SenseNova-U1.5-8B-MoT-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use sensenova/SenseNova-U1.5-8B-MoT-SFT with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("sensenova/SenseNova-U1.5-8B-MoT-SFT", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README_CN.md with huggingface_hub
Browse files- README_CN.md +136 -0
README_CN.md
ADDED
|
@@ -0,0 +1,136 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# SenseNova-U1.5-8B-MoT
|
| 2 |
+
|
| 3 |
+
<p align="center">
|
| 4 |
+
<a href="./README.md">English</a> | <strong>简体中文</strong>
|
| 5 |
+
</p>
|
| 6 |
+
|
| 7 |
+
<p align="center">
|
| 8 |
+
<a href="https://github.com/OpenSenseNova/SenseNova-U1"><img src="https://img.shields.io/badge/GitHub-SenseNova--U1-181717?logo=github" alt="GitHub"></a>
|
| 9 |
+
<a href="https://huggingface.co/collections/sensenova/sensenova-u15"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-U1.5-yellow" alt="Hugging Face 上的 SenseNova-U1.5"></a>
|
| 10 |
+
<a href="https://huggingface.co/blog/sensenova/neo-unify"><img src="https://img.shields.io/badge/Architecture-NEO--unify-2459B8" alt="NEO-unify"></a>
|
| 11 |
+
<a href="https://github.com/OpenSenseNova/SenseNova-U1/blob/refs/heads/feat/u1.5/LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"></a>
|
| 12 |
+
</p>
|
| 13 |
+
|
| 14 |
+
<p align="center">
|
| 15 |
+
<img src="https://raw.githubusercontent.com/OpenSenseNova/SenseNova-U1/refs/heads/feat/u1.5/docs/assets/teaserU1.5.png" alt="SenseNova-U1.5 原生统一多模态架构" width="100%">
|
| 16 |
+
</p>
|
| 17 |
+
|
| 18 |
+
## 概述
|
| 19 |
+
|
| 20 |
+
**SenseNova-U1.5-8B-MoT** 是我们最新的原生统一多模态权重,面向更准确、更一致、更可靠且更具审美表现力的视觉创作。基于 [NEO-unify](https://huggingface.co/blog/sensenova/neo-unify),我们进一步强化了 patchify 层、数据质量与分布、任务定义、Prompt 增强和后训练流程。
|
| 21 |
+
|
| 22 |
+
正式版重点提升了六项用户可感知的能力:
|
| 23 |
+
|
| 24 |
+
- **更高质量的图像生成:** 提升构图、色彩和谐度、材质渲染、光照、真实感与细粒度细节。
|
| 25 |
+
- **更好的文字渲染与信息图生成:** 提升中英文文字可读性,并在海报、信息图、品牌素材及其他文字密集设计中呈现更清晰的信息层级。
|
| 26 |
+
- **更高效的原生 4K 生成:** 提升全局结构连贯性、高分辨率输出稳定性与生成效率。
|
| 27 |
+
- **更可靠的原生图像编辑:** 在局部编辑、文字编辑、多参考图编辑、插入与替换等任务中,更好地保持主体身份与非编辑内容。
|
| 28 |
+
- **更强的复杂指令遵循:** 更稳定地执行对象数量、空间关系、版式、风格及单个请求中的多项约束。
|
| 29 |
+
- **更精确的视觉控制:** 通过边界框、视觉标记以及单图或多图参考,实现更准确的区域级与对象级控制。
|
| 30 |
+
|
| 31 |
+
## 效果展示
|
| 32 |
+
|
| 33 |
+
<p align="center">
|
| 34 |
+
<img src="https://raw.githubusercontent.com/OpenSenseNova/SenseNova-U1/refs/heads/feat/u1.5/docs/assets/u1.5_teaser2.webp" alt="SenseNova-U1.5 文生图与图像编辑效果" width="100%">
|
| 35 |
+
</p>
|
| 36 |
+
|
| 37 |
+
## 核心评测
|
| 38 |
+
|
| 39 |
+
<p align="center">
|
| 40 |
+
<img src="https://raw.githubusercontent.com/OpenSenseNova/SenseNova-U1/refs/heads/feat/u1.5/docs/assets/benchmarks/u1.5_radial.webp" alt="SenseNova-U1.5 基准评测概览" width="100%">
|
| 41 |
+
</p>
|
| 42 |
+
|
| 43 |
+
<details>
|
| 44 |
+
<summary>查看详细评测结果</summary>
|
| 45 |
+
|
| 46 |
+
<p align="center">
|
| 47 |
+
<img src="https://raw.githubusercontent.com/OpenSenseNova/SenseNova-U1/refs/heads/feat/u1.5/docs/assets/benchmarks/u1.5_combined.webp" alt="SenseNova-U1.5 详细基准评测结果" width="100%">
|
| 48 |
+
</p>
|
| 49 |
+
|
| 50 |
+
</details>
|
| 51 |
+
|
| 52 |
+
## 快速开始
|
| 53 |
+
|
| 54 |
+
参考推理代码位于 [SenseNova-U1 GitHub 仓库](https://github.com/OpenSenseNova/SenseNova-U1/tree/refs/heads/feat/u1.5)。
|
| 55 |
+
|
| 56 |
+
### 安装
|
| 57 |
+
|
| 58 |
+
```bash
|
| 59 |
+
git clone https://github.com/OpenSenseNova/SenseNova-U1.git
|
| 60 |
+
cd SenseNova-U1
|
| 61 |
+
uv sync
|
| 62 |
+
source .venv/bin/activate
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
上游环境使用 Python 3.11、PyTorch 2.8 和 CUDA 12.8。其他 CUDA 版本及可选的 FlashAttention 配置请参阅[安装指南](https://github.com/OpenSenseNova/SenseNova-U1/blob/refs/heads/feat/u1.5/docs/installation_CN.md)。
|
| 66 |
+
|
| 67 |
+
### 文生图
|
| 68 |
+
|
| 69 |
+
```bash
|
| 70 |
+
python examples/t2i/inference.py \
|
| 71 |
+
--model_path sensenova/SenseNova-U1.5-8B-MoT \
|
| 72 |
+
--prompt "A cinematic mountain lake at sunrise, realistic photography." \
|
| 73 |
+
--width 2048 --height 2048 \
|
| 74 |
+
--device_map auto \
|
| 75 |
+
--output output.png
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
### 图像编辑
|
| 79 |
+
|
| 80 |
+
```bash
|
| 81 |
+
python examples/editing/inference.py \
|
| 82 |
+
--model_path sensenova/SenseNova-U1.5-8B-MoT \
|
| 83 |
+
--image input.png \
|
| 84 |
+
--prompt "Change the jacket to cobalt blue. Preserve the face, pose, background, lighting, and framing." \
|
| 85 |
+
--output edited.png
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
更多推理选项、支持的分辨率及批量处理方法请参阅[推理示例](https://github.com/OpenSenseNova/SenseNova-U1/blob/refs/heads/feat/u1.5/examples/README_CN.md)。
|
| 89 |
+
|
| 90 |
+
## 最佳实践
|
| 91 |
+
|
| 92 |
+
对于主体明确、约束较少的任务,直接使用自然语言 Prompt 通常即可。复杂生成或编辑任务可在需要额外规划时使用 Prompt 增强,并明确写出必须保持不变的内容。
|
| 93 |
+
|
| 94 |
+
安装说明以及可选的 Image PE、Caption-to-Prompt 和 Editing PE Recipe 请参阅 **[SenseNova-U1.5 Cookbook](https://github.com/OpenSenseNova/SenseNova-U1/blob/refs/heads/feat/u1.5/docs/u1.5_best_practices_CN.md)**。
|
| 95 |
+
|
| 96 |
+
## 进行中的改进
|
| 97 |
+
|
| 98 |
+
正式版在预览版基础上有所改进,但以下方面仍存在挑战:
|
| 99 |
+
|
| 100 |
+
- **细节或色彩过强:** 部分 Prompt 可能产生过多高频细节或过饱和色彩,通常可通过降低 `cfg_scale` 缓解。
|
| 101 |
+
- **密集文字错误:** 密集、较长、小字号或中英文混排文字仍可能出现错误。
|
| 102 |
+
- **高约束版式偏差:** 高约束版式中的精确计数、对齐或层级可能不完全准确。
|
| 103 |
+
- **细粒度细节不稳定:** 小尺寸人脸、手部、肢体与细粒度物体结构仍可能不稳定。
|
| 104 |
+
- **复杂编辑偏移:** 大范围、多轮或多参考图编辑仍可能发生偏移,尤其是同时需要保留多个区域时。
|
| 105 |
+
|
| 106 |
+
## 模型
|
| 107 |
+
|
| 108 |
+
| 模型 | 阶段 | HF 权重 |
|
| 109 |
+
| :--- | :--- | :------ |
|
| 110 |
+
| **SenseNova-U1.5-8B-MoT** | 最终模型 | [🤗 模型](https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT) |
|
| 111 |
+
| SenseNova-U1.5-8B-MoT-SFT | 监督微调 | [🤗 模型](https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT-SFT) |
|
| 112 |
+
|
| 113 |
+
## 引用
|
| 114 |
+
|
| 115 |
+
如果这个项目对您的研究有帮助,请考虑为仓库点 Star 并引用:
|
| 116 |
+
|
| 117 |
+
```bibtex
|
| 118 |
+
@misc{sensenova2026neounify,
|
| 119 |
+
title = {NEO-unify: Building Native Multimodal Unified Models End to End},
|
| 120 |
+
author = {SenseNova},
|
| 121 |
+
journal = {Hugging Face blog},
|
| 122 |
+
url = {https://huggingface.co/blog/sensenova/neo-unify},
|
| 123 |
+
year = {2026}
|
| 124 |
+
}
|
| 125 |
+
|
| 126 |
+
@article{sensenova2026sensenovau1,
|
| 127 |
+
title = {SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture},
|
| 128 |
+
author = {Diao, Haiwen and Wu, Penghao and Deng, Hanming and Wang, Jiahao and Bai, Shihao and Wu, Silei and Fan, Weichen and Ye, Wenjie and Tong, Wenwen and Fan, Xiangyu and others},
|
| 129 |
+
journal = {arXiv preprint arXiv:2605.12500},
|
| 130 |
+
year = {2026}
|
| 131 |
+
}
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
## 许可证
|
| 135 |
+
|
| 136 |
+
本模型基于 [Apache 2.0 License](https://github.com/OpenSenseNova/SenseNova-U1/blob/refs/heads/feat/u1.5/LICENSE) 开源发布。
|