feat: 上传完整实现代码(切片/教师打标/标签/训练/评测/ONNX 导出)与 FP16 版本
Browse files- README.md +43 -1
- code/append_sources.py +81 -0
- code/siai_all.py +810 -0
- model_fp16.onnx +3 -0
- requirements.txt +9 -2
- train_log.csv +9 -0
- 训练素材清单.txt +0 -0
README.md
CHANGED
|
@@ -20,6 +20,16 @@ datasets:
|
|
| 20 |
(像素差、dHash、QR 分解、Farnebäck 光流)的判定行为,可在 NVIDIA / AMD / Intel 显卡上
|
| 21 |
通过 DirectML 统一推理。
|
| 22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
## 模型详情
|
| 24 |
|
| 25 |
| 项目 | 说明 |
|
|
@@ -31,7 +41,8 @@ datasets:
|
|
| 31 |
| 输入 | `frame_a`、`frame_b`:灰度 128×128,float32,范围 [0, 1],NCHW |
|
| 32 |
| 输出 | `score`:形状 [batch, 1],Sigmoid 后位于 [0, 1],越接近 1 越可能重复 |
|
| 33 |
| 导出格式 | ONNX,opset 17,动态 batch |
|
| 34 |
-
|
|
|
|
|
| 35 |
| 运行后端 | onnxruntime:`DmlExecutionProvider`(DirectML)/ `CPUExecutionProvider` |
|
| 36 |
| 建议阈值 | 0.5;需要更低误判时可用 0.6~0.7 |
|
| 37 |
| 许可证 | CC BY-NC 4.0(仅限非商业用途) |
|
|
@@ -112,6 +123,37 @@ print(float(score)) # 1.0 表示两帧重复
|
|
| 112 |
| Epoch | 8 |
|
| 113 |
| 硬件 | NVIDIA P106-100(6 GB),CUDA 12.4,PyTorch 2.6.0 |
|
| 114 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
## 评测结果
|
| 116 |
|
| 117 |
### 与教师的一致性(验证集 78147 样本,按视频隔离)
|
|
|
|
| 20 |
(像素差、dHash、QR 分解、Farnebäck 光流)的判定行为,可在 NVIDIA / AMD / Intel 显卡上
|
| 21 |
通过 DirectML 统一推理。
|
| 22 |
|
| 23 |
+
## 训练数据声明
|
| 24 |
+
|
| 25 |
+
本模型在训练过程中**未分发、未再发布任何原始视频、音频或视频帧**。训练仅使用从公开网络获取的视频中抽取的灰度切片,且全部监督信号由 Easy4K 自研的 SFDP 四阶判重判决器(像素差 / dHash / QR 码 / Farnebäck 光流)自动生成,不含人工标注。
|
| 26 |
+
|
| 27 |
+
视频著作权归原作者所有。所用素材清单见 `training-materials.txt`(含标题、BV 号、UP 主、原始链接),仅用于说明数据来源,不构成对原作品的授权或再分发。
|
| 28 |
+
|
| 29 |
+
若权利人认为其作品不应被用于本研究,请提交 issue,我们将在 72 小时内从清单中移除,并在后续版本训练中排除该来源。
|
| 30 |
+
|
| 31 |
+
模型权重以 **CC BY-NC 4.0** 发布,仅限研究与非商业用途。模型输出为 0~1 的相似度分数,不具备生成、重建或复现原视频内容的能力。
|
| 32 |
+
|
| 33 |
## 模型详情
|
| 34 |
|
| 35 |
| 项目 | 说明 |
|
|
|
|
| 41 |
| 输入 | `frame_a`、`frame_b`:灰度 128×128,float32,范围 [0, 1],NCHW |
|
| 42 |
| 输出 | `score`:形状 [batch, 1],Sigmoid 后位于 [0, 1],越接近 1 越可能重复 |
|
| 43 |
| 导出格式 | ONNX,opset 17,动态 batch |
|
| 44 |
+
| FP32 版本 | `model.onnx`,约 9.95 MB,对应 Easy4K「完美模式」 |
|
| 45 |
+
| FP16 版本 | `model_fp16.onnx`,约 5 MB(内部半精度、输入输出保持 float32),对应 Easy4K「性能模式」 |
|
| 46 |
| 运行后端 | onnxruntime:`DmlExecutionProvider`(DirectML)/ `CPUExecutionProvider` |
|
| 47 |
| 建议阈值 | 0.5;需要更低误判时可用 0.6~0.7 |
|
| 48 |
| 许可证 | CC BY-NC 4.0(仅限非商业用途) |
|
|
|
|
| 123 |
| Epoch | 8 |
|
| 124 |
| 硬件 | NVIDIA P106-100(6 GB),CUDA 12.4,PyTorch 2.6.0 |
|
| 125 |
|
| 126 |
+
## 完整实现代码
|
| 127 |
+
|
| 128 |
+
模型训练、导出与数据来源记录的全部实现代码随仓库提供,位于 [`code/`](./code):
|
| 129 |
+
|
| 130 |
+
| 文件 | 作用 |
|
| 131 |
+
|---|---|
|
| 132 |
+
| `code/siai_all.py` | 一体化流水线:视频切片(`tiles`)、教师打标(`teacher`)、软标签生成(`label`)、训练(`train`)、评测(`eval`)、ONNX 导出(`pack`) |
|
| 133 |
+
| `code/append_sources.py` | 训练素材清单的生成与去重追加,产出可溯源的 `training-materials.txt` |
|
| 134 |
+
|
| 135 |
+
主要命令:
|
| 136 |
+
|
| 137 |
+
```bash
|
| 138 |
+
# 1) 从视频抽取灰度切片
|
| 139 |
+
python code/siai_all.py tiles --raw <视频目录> --segments 4 --out <切片目录>
|
| 140 |
+
|
| 141 |
+
# 2) 用四阶判决器教师生成逐片段标签 CSV
|
| 142 |
+
python code/siai_all.py teacher --tiles <切片目录> --labels <标签目录>
|
| 143 |
+
|
| 144 |
+
# 3) 生成软标签训练集与验证集
|
| 145 |
+
python code/siai_all.py label
|
| 146 |
+
|
| 147 |
+
# 4) 训练(结束后自动导出 FP32 与 FP16 两份 ONNX)
|
| 148 |
+
python code/siai_all.py train --out <输出目录> --epochs 1 --workers 8 --lr 1e-4
|
| 149 |
+
|
| 150 |
+
# 5) 评测 / 打包
|
| 151 |
+
python code/siai_all.py eval --out <输出目录>
|
| 152 |
+
python code/siai_all.py pack --onnx <输出目录>/easy4k_sfdp_siamese_fp32.onnx --out <发布目录>
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
依赖见 `requirements.txt`(PyTorch、OpenCV、onnx、onnxruntime、onnxconverter-common)。教师端二进制 `SfdpTeacher` 属于 Easy4K 应用程序的一部分,不随本仓库分发。
|
| 156 |
+
|
| 157 |
## 评测结果
|
| 158 |
|
| 159 |
### 与教师的一致性(验证集 78147 样本,按视频隔离)
|
code/append_sources.py
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# -*- coding: utf-8 -*-
|
| 2 |
+
"""把新下载的 B 站素材出处追加到 D:\\SFDP-Viedo\\训练素材清单.txt(去重,可反复运行)。"""
|
| 3 |
+
import json
|
| 4 |
+
import os
|
| 5 |
+
|
| 6 |
+
ROOT = r"D:\SFDP-Viedo"
|
| 7 |
+
META = os.path.join(ROOT, "音乐MV", "sources_bili.json")
|
| 8 |
+
TXT = os.path.join(ROOT, "训练素材清单.txt")
|
| 9 |
+
MARK = os.path.join(ROOT, ".listed_bvids.json")
|
| 10 |
+
EXTRA_DIRS = [os.path.join(ROOT, "二次元MV"), os.path.join(ROOT, "真人MV")]
|
| 11 |
+
VIDEO_EXT = (".mp4", ".mkv", ".webm", ".mov", ".flv")
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def load(p, d):
|
| 15 |
+
try:
|
| 16 |
+
with open(p, "r", encoding="utf-8") as f:
|
| 17 |
+
return json.load(f)
|
| 18 |
+
except Exception:
|
| 19 |
+
return d
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
HEADER = """训练数据声明
|
| 23 |
+
----------------
|
| 24 |
+
本模型在训练过程中未分发、未再发布任何原始视频、音频或视频帧。训练仅使用从公开网络
|
| 25 |
+
获取的视频中抽取的灰度切片,且全部监督信号由 Easy4K 自研的 SFDP 四阶判重判决器
|
| 26 |
+
(像素差 / dHash / QR 码 / Farneback 光流)自动生成,不含人工标注。
|
| 27 |
+
视频著作权归原作者所有。本清单仅用于说明数据来源,不构成对原作品的授权或再分发。
|
| 28 |
+
若权利人认为其作品不应被用于本研究,请提交 issue,我们将在 72 小时内从清单中移除,
|
| 29 |
+
并在后续版本训练中排除该来源。
|
| 30 |
+
模型权重以 CC BY-NC 4.0 发布,仅限研究与非商业用途。
|
| 31 |
+
|
| 32 |
+
"""
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def ensure_header():
|
| 36 |
+
old = ""
|
| 37 |
+
if os.path.exists(TXT):
|
| 38 |
+
with open(TXT, "r", encoding="utf-8") as f:
|
| 39 |
+
old = f.read()
|
| 40 |
+
if "训练数据声明" in old:
|
| 41 |
+
return
|
| 42 |
+
with open(TXT, "w", encoding="utf-8") as f:
|
| 43 |
+
f.write(HEADER + old)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
ensure_header()
|
| 47 |
+
|
| 48 |
+
listed = set(load(MARK, []))
|
| 49 |
+
meta = load(META, [])
|
| 50 |
+
new = [m for m in meta if m.get("bvid") and m["bvid"] not in listed]
|
| 51 |
+
if new:
|
| 52 |
+
with open(TXT, "a", encoding="utf-8") as f:
|
| 53 |
+
f.write("\n== 新增 B 站素材(自动采集,出处可溯) ==\n")
|
| 54 |
+
for i, m in enumerate(new, 1):
|
| 55 |
+
f.write("%3d. %s\n" % (i, m.get("file") or m["bvid"]))
|
| 56 |
+
f.write(" 标题:%s\n" % m.get("title", ""))
|
| 57 |
+
f.write(" 作者:%s | 时长:%ss | 清晰度:%s | 切片:待统计\n" % (
|
| 58 |
+
m.get("author", ""), m.get("duration", ""), m.get("qn", "")))
|
| 59 |
+
f.write(" 来源:https://www.bilibili.com/video/%s\n" % m["bvid"])
|
| 60 |
+
listed |= {m["bvid"] for m in new}
|
| 61 |
+
with open(MARK, "w", encoding="utf-8") as f:
|
| 62 |
+
json.dump(sorted(listed), f, ensure_ascii=False)
|
| 63 |
+
print("APPENDED", len(new), "entries", flush=True)
|
| 64 |
+
|
| 65 |
+
for d in EXTRA_DIRS:
|
| 66 |
+
if not os.path.isdir(d):
|
| 67 |
+
continue
|
| 68 |
+
changed = False
|
| 69 |
+
for n in sorted(os.listdir(d)):
|
| 70 |
+
if os.path.splitext(n)[1].lower() not in VIDEO_EXT:
|
| 71 |
+
continue
|
| 72 |
+
key = "manual:" + os.path.basename(d) + "/" + n
|
| 73 |
+
if key in listed:
|
| 74 |
+
continue
|
| 75 |
+
with open(TXT, "a", encoding="utf-8") as f:
|
| 76 |
+
f.write("\n[手动素材] %s\\%s\n 来源:用户投放(待补充出处)\n" % (os.path.basename(d), n))
|
| 77 |
+
listed.add(key)
|
| 78 |
+
changed = True
|
| 79 |
+
if changed:
|
| 80 |
+
with open(MARK, "w", encoding="utf-8") as f:
|
| 81 |
+
json.dump(sorted(listed), f, ensure_ascii=False)
|
code/siai_all.py
ADDED
|
@@ -0,0 +1,810 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python
|
| 2 |
+
"""Easy4K SFDP Siamese NET AI pipeline (compact)."""
|
| 3 |
+
import argparse, csv, hashlib, http.cookiejar, json, math, os, random, re, subprocess, sys, time
|
| 4 |
+
import urllib.error, urllib.parse, urllib.request
|
| 5 |
+
from functools import lru_cache
|
| 6 |
+
|
| 7 |
+
import cv2, numpy as np, torch, torch.nn as nn
|
| 8 |
+
from torch.utils.data import DataLoader, Dataset, Sampler
|
| 9 |
+
|
| 10 |
+
DATA = r"D:\sfdp-ai\siamese\data"
|
| 11 |
+
TILES = os.environ.get("SFDP_TILES") or os.path.join(DATA, "tiles")
|
| 12 |
+
LABELS = os.path.join(DATA, "labels")
|
| 13 |
+
TEACHER = r"D:\sfdp-ai\bin\SfdpTeacher.exe"
|
| 14 |
+
RAW = os.path.join(DATA, "raw")
|
| 15 |
+
BILI_RAW = os.path.join(DATA, "raw_bili")
|
| 16 |
+
TEST_TILES = os.path.join(DATA, "tiles_test")
|
| 17 |
+
TEST_LABELS = os.path.join(DATA, "labels_test")
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
class DSConv(nn.Module):
|
| 21 |
+
def __init__(self, cin, cout, stride=1):
|
| 22 |
+
super().__init__()
|
| 23 |
+
self.dw = nn.Conv2d(cin, cin, 3, stride, 1, groups=cin, bias=False)
|
| 24 |
+
self.bn1 = nn.BatchNorm2d(cin)
|
| 25 |
+
self.pw = nn.Conv2d(cin, cout, 1, bias=False)
|
| 26 |
+
self.bn2 = nn.BatchNorm2d(cout)
|
| 27 |
+
self.act = nn.ReLU6(inplace=True)
|
| 28 |
+
|
| 29 |
+
def forward(self, x):
|
| 30 |
+
x = self.act(self.bn1(self.dw(x)))
|
| 31 |
+
return self.act(self.bn2(self.pw(x)))
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
class Encoder(nn.Module):
|
| 35 |
+
def __init__(self, emb=512):
|
| 36 |
+
super().__init__()
|
| 37 |
+
self.stem = nn.Sequential(nn.Conv2d(1, 32, 3, 2, 1, bias=False),
|
| 38 |
+
nn.BatchNorm2d(32), nn.ReLU6(inplace=True))
|
| 39 |
+
self.blocks = nn.Sequential(
|
| 40 |
+
DSConv(32, 48, 2), DSConv(48, 48), DSConv(48, 96, 2), DSConv(96, 96),
|
| 41 |
+
DSConv(96, 160, 2), DSConv(160, 160), DSConv(160, 224, 2), DSConv(224, 224),
|
| 42 |
+
DSConv(224, 320, 2), DSConv(320, 320))
|
| 43 |
+
self.proj = nn.Conv2d(320, emb, 1, bias=False)
|
| 44 |
+
self.bn = nn.BatchNorm2d(emb)
|
| 45 |
+
self.act = nn.ReLU6(inplace=True)
|
| 46 |
+
|
| 47 |
+
def forward(self, x):
|
| 48 |
+
x = self.act(self.bn(self.proj(self.blocks(self.stem(x)))))
|
| 49 |
+
return x.mean(dim=(2, 3))
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
class SiameseSFDP(nn.Module):
|
| 53 |
+
def __init__(self, emb=512, hidden=1024):
|
| 54 |
+
super().__init__()
|
| 55 |
+
self.encoder = Encoder(emb)
|
| 56 |
+
self.head = nn.Sequential(nn.Linear(emb * 4, hidden), nn.ReLU(inplace=True),
|
| 57 |
+
nn.Dropout(0.1), nn.Linear(hidden, 1), nn.Sigmoid())
|
| 58 |
+
|
| 59 |
+
def forward(self, a, b):
|
| 60 |
+
ea = self.encoder(a); eb = self.encoder(b)
|
| 61 |
+
return self.head(torch.cat([ea, eb, (ea - eb).abs(), ea * eb], dim=1))
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def count_params(m):
|
| 65 |
+
return sum(p.numel() for p in m.parameters())
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
@lru_cache(maxsize=4096)
|
| 69 |
+
def load_gray(p):
|
| 70 |
+
img = cv2.imread(p, cv2.IMREAD_GRAYSCALE)
|
| 71 |
+
if img is None:
|
| 72 |
+
raise FileNotFoundError(p)
|
| 73 |
+
return img
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
_GRAY_CACHE = {}
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def load_gray_cached(path):
|
| 80 |
+
"""同一切片会被同片段的多个样本反复读取:按 worker 进程做一个小 LRU,
|
| 81 |
+
省掉重复 PNG 解码(320x320 约 100KB,512 条 ≈ 50MB/worker)。"""
|
| 82 |
+
a = _GRAY_CACHE.get(path)
|
| 83 |
+
if a is not None:
|
| 84 |
+
return a
|
| 85 |
+
try:
|
| 86 |
+
a = load_gray(path)
|
| 87 |
+
except FileNotFoundError:
|
| 88 |
+
# 切片可能正被搬运/重写:短暂重试,实在没有就返回黑图,绝不让整轮训练崩掉
|
| 89 |
+
a = None
|
| 90 |
+
for _ in range(3):
|
| 91 |
+
time.sleep(0.2)
|
| 92 |
+
try:
|
| 93 |
+
a = load_gray(path)
|
| 94 |
+
break
|
| 95 |
+
except FileNotFoundError:
|
| 96 |
+
continue
|
| 97 |
+
if a is None:
|
| 98 |
+
a = np.zeros((320, 320), np.uint8)
|
| 99 |
+
if len(_GRAY_CACHE) >= 512:
|
| 100 |
+
_GRAY_CACHE.clear()
|
| 101 |
+
_GRAY_CACHE[path] = a
|
| 102 |
+
return a
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
FALLBACK_TILES = os.path.join(DATA, "tiles")
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
def _resolve_tile(root, rel):
|
| 109 |
+
"""Hot tiles (SSD) may still be copying: fall back to the original tiles dir."""
|
| 110 |
+
p = os.path.join(root, rel)
|
| 111 |
+
if os.path.exists(p):
|
| 112 |
+
return p
|
| 113 |
+
fb = os.path.join(FALLBACK_TILES, rel)
|
| 114 |
+
return fb if os.path.exists(fb) else p
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
class PairDataset(Dataset):
|
| 118 |
+
def __init__(self, manifest, tiles_root, crop=128, augment=False):
|
| 119 |
+
self.items = [json.loads(l) for l in open(manifest, "r", encoding="utf-8") if l.strip()]
|
| 120 |
+
self.tiles_root = tiles_root; self.crop = crop; self.augment = augment
|
| 121 |
+
|
| 122 |
+
def __len__(self):
|
| 123 |
+
return len(self.items)
|
| 124 |
+
|
| 125 |
+
def __getitem__(self, i):
|
| 126 |
+
it = self.items[i]
|
| 127 |
+
a = load_gray_cached(_resolve_tile(self.tiles_root, it["a"]))
|
| 128 |
+
b = load_gray_cached(_resolve_tile(self.tiles_root, it["b"]))
|
| 129 |
+
c = self.crop; h, w = a.shape
|
| 130 |
+
x = min(max(0, it["x"]), w - c); y = min(max(0, it["y"]), h - c)
|
| 131 |
+
pa, pb = a[y:y + c, x:x + c], b[y:y + c, x:x + c]
|
| 132 |
+
if self.augment:
|
| 133 |
+
if np.random.rand() < 0.5:
|
| 134 |
+
pa, pb = pa[:, ::-1], pb[:, ::-1]
|
| 135 |
+
if np.random.rand() < 0.3:
|
| 136 |
+
g = 0.8 + np.random.rand() * 0.45
|
| 137 |
+
pa = np.clip((pa / 255.0) ** g * 255.0, 0, 255).astype(np.uint8)
|
| 138 |
+
pb = np.clip((pb / 255.0) ** g * 255.0, 0, 255).astype(np.uint8)
|
| 139 |
+
ta = torch.from_numpy(np.ascontiguousarray(pa)).float().div_(255.0).unsqueeze(0)
|
| 140 |
+
tb = torch.from_numpy(np.ascontiguousarray(pb)).float().div_(255.0).unsqueeze(0)
|
| 141 |
+
return (ta, tb, torch.tensor([it["soft"]], dtype=torch.float32),
|
| 142 |
+
torch.tensor([it["hard"]], dtype=torch.float32))
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
# ---------------- bilibili ----------------
|
| 146 |
+
BUA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
|
| 147 |
+
"(KHTML, like Gecko) Chrome/124.0 Safari/537.36")
|
| 148 |
+
BILI_META = os.path.join(BILI_RAW, "sources_bili.json")
|
| 149 |
+
_jar = http.cookiejar.CookieJar()
|
| 150 |
+
_op = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(_jar))
|
| 151 |
+
_op.addheaders = [("User-Agent", BUA), ("Referer", "https://www.bilibili.com/")]
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
def bjson(url, params=None):
|
| 155 |
+
full = url + ("?" + urllib.parse.urlencode(params) if params else "")
|
| 156 |
+
last = None
|
| 157 |
+
for i in range(3):
|
| 158 |
+
try:
|
| 159 |
+
with _op.open(full, timeout=25) as r:
|
| 160 |
+
return json.loads(r.read().decode("utf-8"))
|
| 161 |
+
except urllib.error.HTTPError as e:
|
| 162 |
+
if e.code == 412:
|
| 163 |
+
raise
|
| 164 |
+
last = e; time.sleep(2 + i * 2)
|
| 165 |
+
except Exception as e:
|
| 166 |
+
last = e; time.sleep(2 + i * 2)
|
| 167 |
+
raise last
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
def bview(bvid):
|
| 171 |
+
d = bjson("https://api.bilibili.com/x/web-interface/view", {"bvid": bvid})
|
| 172 |
+
return (d.get("data") or None) if d.get("code") == 0 else None
|
| 173 |
+
|
| 174 |
+
|
| 175 |
+
def bplay(bvid, cid, qn):
|
| 176 |
+
d = bjson("https://api.bilibili.com/x/player/playurl",
|
| 177 |
+
{"bvid": bvid, "cid": cid, "qn": qn, "fnval": 0, "fourk": 0})
|
| 178 |
+
if d.get("code") != 0:
|
| 179 |
+
return None
|
| 180 |
+
dd = d.get("data") or {}; durl = dd.get("durl") or []
|
| 181 |
+
return {"quality": dd.get("quality"), "url": durl[0].get("url")} if durl else None
|
| 182 |
+
|
| 183 |
+
|
| 184 |
+
BILI_MAX_BPS = 6_000_000 # 下载限速 ~48 Mbps(千兆宽带留给家里其他人)
|
| 185 |
+
|
| 186 |
+
|
| 187 |
+
def bdownload(url, dst):
|
| 188 |
+
tmp = dst + ".part"
|
| 189 |
+
req = urllib.request.Request(url, headers={"User-Agent": BUA, "Referer": "https://www.bilibili.com/"})
|
| 190 |
+
for _ in range(3):
|
| 191 |
+
try:
|
| 192 |
+
with urllib.request.urlopen(req, timeout=30) as r, open(tmp, "wb") as f:
|
| 193 |
+
t0 = time.time()
|
| 194 |
+
got = 0
|
| 195 |
+
while True:
|
| 196 |
+
ch = r.read(262144)
|
| 197 |
+
if not ch:
|
| 198 |
+
break
|
| 199 |
+
f.write(ch)
|
| 200 |
+
got += len(ch)
|
| 201 |
+
# 令牌桶式限速:按已下载字节应花的时间补 sleep
|
| 202 |
+
expected = got / BILI_MAX_BPS
|
| 203 |
+
elapsed = time.time() - t0
|
| 204 |
+
if expected > elapsed:
|
| 205 |
+
time.sleep(expected - elapsed)
|
| 206 |
+
os.replace(tmp, dst); return True
|
| 207 |
+
except Exception:
|
| 208 |
+
if os.path.exists(tmp):
|
| 209 |
+
os.remove(tmp)
|
| 210 |
+
time.sleep(3)
|
| 211 |
+
return False
|
| 212 |
+
|
| 213 |
+
|
| 214 |
+
def cmd_rank(args):
|
| 215 |
+
root = getattr(args, "outdir", None) or BILI_RAW
|
| 216 |
+
meta_path = os.path.join(root, "sources_bili.json")
|
| 217 |
+
os.makedirs(root, exist_ok=True)
|
| 218 |
+
try:
|
| 219 |
+
_op.open("https://www.bilibili.com/", timeout=20).read(2048)
|
| 220 |
+
except Exception:
|
| 221 |
+
pass
|
| 222 |
+
meta = json.load(open(meta_path, "r", encoding="utf-8")) if os.path.exists(meta_path) else []
|
| 223 |
+
done = {m["bvid"] for m in meta}
|
| 224 |
+
seen, cands = set(), []
|
| 225 |
+
|
| 226 |
+
def add(items):
|
| 227 |
+
for it in items:
|
| 228 |
+
b = it.get("bvid"); dur = int(it.get("duration") or 0); title = it.get("title", "")
|
| 229 |
+
if not b or b in seen or b in done or not (args.min_sec <= dur <= args.max_sec):
|
| 230 |
+
continue
|
| 231 |
+
if any(x in title for x in ("直播", "回放", "合集", "教程", "reaction", "解说")):
|
| 232 |
+
continue
|
| 233 |
+
seen.add(b)
|
| 234 |
+
cands.append({"bvid": b, "title": title, "author": (it.get("owner") or {}).get("name", ""),
|
| 235 |
+
"duration": dur})
|
| 236 |
+
|
| 237 |
+
for rid in (3, 129, 1, 181, 71, 160, 4):
|
| 238 |
+
try:
|
| 239 |
+
d = bjson("https://api.bilibili.com/x/web-interface/ranking/v2", {"rid": rid})
|
| 240 |
+
add((d.get("data") or {}).get("list") or [])
|
| 241 |
+
except Exception as e:
|
| 242 |
+
print("RANK_FAIL", rid, e, flush=True)
|
| 243 |
+
time.sleep(1.2)
|
| 244 |
+
for pn in range(1, args.pages + 1):
|
| 245 |
+
try:
|
| 246 |
+
d = bjson("https://api.bilibili.com/x/web-interface/popular", {"ps": 20, "pn": pn})
|
| 247 |
+
add((d.get("data") or {}).get("list") or [])
|
| 248 |
+
except Exception as e:
|
| 249 |
+
print("POPULAR_FAIL", pn, e, flush=True)
|
| 250 |
+
time.sleep(1.2)
|
| 251 |
+
for rid in (3, 129, 1, 181):
|
| 252 |
+
for pn in range(1, 6):
|
| 253 |
+
try:
|
| 254 |
+
d = bjson("https://api.bilibili.com/x/web-interface/dynamic/region",
|
| 255 |
+
{"rid": rid, "ps": 20, "pn": pn})
|
| 256 |
+
add((d.get("data") or {}).get("archives") or [])
|
| 257 |
+
except Exception as e:
|
| 258 |
+
print("REGION_FAIL", rid, pn, e, flush=True); break
|
| 259 |
+
time.sleep(1.0)
|
| 260 |
+
existing = list(done); random.shuffle(existing)
|
| 261 |
+
seeds = ([c["bvid"] for c in cands] + existing[:args.seed_existing])[:args.seeds + args.seed_existing]
|
| 262 |
+
for b in seeds:
|
| 263 |
+
try:
|
| 264 |
+
d = bjson("https://api.bilibili.com/x/web-interface/archive/related", {"bvid": b})
|
| 265 |
+
add(d.get("data") or [])
|
| 266 |
+
except Exception as e:
|
| 267 |
+
print("RELATED_FAIL", b, e, flush=True)
|
| 268 |
+
time.sleep(0.5)
|
| 269 |
+
print("CANDIDATES", len(cands), flush=True)
|
| 270 |
+
saved = 0
|
| 271 |
+
for it in cands:
|
| 272 |
+
if saved >= args.target:
|
| 273 |
+
break
|
| 274 |
+
v = bview(it["bvid"])
|
| 275 |
+
if not v:
|
| 276 |
+
continue
|
| 277 |
+
pu = None
|
| 278 |
+
for qn in (80, 64, 32):
|
| 279 |
+
pu = bplay(it["bvid"], v["cid"], qn)
|
| 280 |
+
if pu:
|
| 281 |
+
break
|
| 282 |
+
if not pu or not pu.get("url"):
|
| 283 |
+
continue
|
| 284 |
+
name = re.sub(r"[^0-9A-Za-z._-]+", "_", it["title"])[:60] or it["bvid"]
|
| 285 |
+
ext = ".flv" if ".flv" in pu["url"] else ".mp4"
|
| 286 |
+
fname = f"{it['bvid']}_{name}{ext}"
|
| 287 |
+
dst = os.path.join(root, fname)
|
| 288 |
+
if os.path.exists(dst):
|
| 289 |
+
continue
|
| 290 |
+
if not bdownload(pu["url"], dst):
|
| 291 |
+
print("DL_FAIL", it["bvid"], flush=True); continue
|
| 292 |
+
meta.append({**it, "file": fname, "cid": v["cid"], "qn": pu["quality"],
|
| 293 |
+
"license": "research use only, not redistributed"})
|
| 294 |
+
saved += 1
|
| 295 |
+
print("OK", fname, flush=True)
|
| 296 |
+
json.dump(meta, open(meta_path, "w", encoding="utf-8"), ensure_ascii=False, indent=2)
|
| 297 |
+
print("TOTAL_FILES", len(meta), flush=True)
|
| 298 |
+
|
| 299 |
+
|
| 300 |
+
# ---------------- tiles ----------------
|
| 301 |
+
PATTERNS = {"1-1": [1], "2-2": [2], "3-3": [3], "mix": [2, 3, 2, 1, 3, 2]}
|
| 302 |
+
VEXT = (".webm", ".ogg", ".ogv", ".mp4", ".mov", ".mkv", ".flv")
|
| 303 |
+
|
| 304 |
+
|
| 305 |
+
def clip_id(stem, seg, pat):
|
| 306 |
+
s = re.sub(r"[^0-9A-Za-z._-]+", "_", stem).strip("_")[:40]
|
| 307 |
+
if any(ord(c) > 127 for c in stem) or len(s) < 4:
|
| 308 |
+
s = (s + "_" if s else "mv_") + hashlib.sha1(stem.encode("utf-8")).hexdigest()[:8]
|
| 309 |
+
return f"{s}_s{seg:02d}_{pat}"
|
| 310 |
+
|
| 311 |
+
|
| 312 |
+
def pulldown(frames, rounds, n):
|
| 313 |
+
out, i = [], 0
|
| 314 |
+
while len(out) < n:
|
| 315 |
+
out.extend([frames[i % len(frames)]] * rounds[i % len(rounds)]); i += 1
|
| 316 |
+
return out[:n]
|
| 317 |
+
|
| 318 |
+
|
| 319 |
+
def cmd_tiles(args):
|
| 320 |
+
rng = random.Random(args.seed); os.makedirs(args.out, exist_ok=True)
|
| 321 |
+
idx_path = os.path.join(args.out, "index.jsonl")
|
| 322 |
+
done = set()
|
| 323 |
+
if os.path.exists(idx_path):
|
| 324 |
+
for line in open(idx_path, "r", encoding="utf-8"):
|
| 325 |
+
try:
|
| 326 |
+
done.add(json.loads(line)["clip"])
|
| 327 |
+
except Exception:
|
| 328 |
+
pass
|
| 329 |
+
videos = [n for n in sorted(os.listdir(args.raw)) if os.path.splitext(n)[1].lower() in VEXT]
|
| 330 |
+
print("VIDEOS", len(videos), "ALREADY", len(done), flush=True)
|
| 331 |
+
total = 0
|
| 332 |
+
with open(idx_path, "a", encoding="utf-8") as idx:
|
| 333 |
+
for vi, name in enumerate(videos):
|
| 334 |
+
cap = cv2.VideoCapture(os.path.join(args.raw, name))
|
| 335 |
+
if not cap.isOpened():
|
| 336 |
+
print("OPEN_FAIL", name, flush=True); continue
|
| 337 |
+
nf = int(cap.get(cv2.CAP_PROP_FRAME_COUNT) or 0)
|
| 338 |
+
fps = float(cap.get(cv2.CAP_PROP_FPS) or 0)
|
| 339 |
+
w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH) or 0)
|
| 340 |
+
h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT) or 0)
|
| 341 |
+
if nf < args.frames * 3 or w < max(args.tile, args.min_width) or h < args.tile:
|
| 342 |
+
print("SKIP_DIM", name, w, h, nf, flush=True); cap.release(); continue
|
| 343 |
+
max_start = max(0, nf - args.frames * 4)
|
| 344 |
+
for si in range(args.segments):
|
| 345 |
+
start = rng.randint(0, max_start)
|
| 346 |
+
x = rng.randint(0, w - args.tile); y = rng.randint(0, h - args.tile)
|
| 347 |
+
cap.set(cv2.CAP_PROP_POS_FRAMES, start)
|
| 348 |
+
frames = []
|
| 349 |
+
for _ in range(max(4, args.frames // 2)):
|
| 350 |
+
ok, fr = cap.read()
|
| 351 |
+
if not ok:
|
| 352 |
+
break
|
| 353 |
+
frames.append(cv2.cvtColor(fr[y:y + args.tile, x:x + args.tile], cv2.COLOR_BGR2GRAY))
|
| 354 |
+
if len(frames) < 4:
|
| 355 |
+
continue
|
| 356 |
+
for pat, rounds in PATTERNS.items():
|
| 357 |
+
clip = clip_id(os.path.splitext(name)[0], si, pat)
|
| 358 |
+
if clip in done:
|
| 359 |
+
continue
|
| 360 |
+
cdir = os.path.join(args.out, clip); os.makedirs(cdir, exist_ok=True)
|
| 361 |
+
for k, img in enumerate(pulldown(frames, rounds, args.frames)):
|
| 362 |
+
cv2.imwrite(os.path.join(cdir, f"f{k:04d}.png"), img)
|
| 363 |
+
idx.write(json.dumps({"clip": clip, "src": name, "pattern": pat,
|
| 364 |
+
"start_frame": start, "x": x, "y": y,
|
| 365 |
+
"tile": args.tile, "frames": args.frames, "fps": fps},
|
| 366 |
+
ensure_ascii=False) + "\n")
|
| 367 |
+
idx.flush(); total += 1
|
| 368 |
+
cap.release()
|
| 369 |
+
if (vi + 1) % 10 == 0:
|
| 370 |
+
print("PROGRESS", vi + 1, "/", len(videos), "clips", total, flush=True)
|
| 371 |
+
print("TOTAL_CLIPS", total, flush=True)
|
| 372 |
+
|
| 373 |
+
|
| 374 |
+
def cmd_teacher(args):
|
| 375 |
+
os.makedirs(args.labels, exist_ok=True)
|
| 376 |
+
print("SCAN_START", args.tiles, flush=True)
|
| 377 |
+
dirs = [d for d in sorted(os.listdir(args.tiles)) if os.path.isdir(os.path.join(args.tiles, d))]
|
| 378 |
+
print("SCAN_DIRS", len(dirs), flush=True)
|
| 379 |
+
todo = []
|
| 380 |
+
for i, d in enumerate(dirs):
|
| 381 |
+
if not os.path.exists(os.path.join(args.labels, d + ".csv")):
|
| 382 |
+
todo.append(d)
|
| 383 |
+
if (i + 1) % 2000 == 0:
|
| 384 |
+
print("SCAN", i + 1, "/", len(dirs), flush=True)
|
| 385 |
+
print("DIRS", len(dirs), "TODO", len(todo), flush=True)
|
| 386 |
+
for i, d in enumerate(todo):
|
| 387 |
+
subprocess.run([TEACHER, os.path.join(args.tiles, d), os.path.join(args.labels, d + ".csv"),
|
| 388 |
+
"--mode", "uhd", "--crop", "128", "--per-pair", "4", "--limit", "0"],
|
| 389 |
+
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
| 390 |
+
if (i + 1) % 10 == 0:
|
| 391 |
+
print("PROGRESS", i + 1, "/", len(todo), flush=True)
|
| 392 |
+
print("DONE", flush=True)
|
| 393 |
+
|
| 394 |
+
|
| 395 |
+
# ---------------- label / train / eval / pack ----------------
|
| 396 |
+
def soft_from_flow(flow):
|
| 397 |
+
return max(0.0, min(1.0, 1.0 - flow / 2.0))
|
| 398 |
+
|
| 399 |
+
|
| 400 |
+
def cmd_label(args):
|
| 401 |
+
rng = random.Random(args.seed)
|
| 402 |
+
clip_src = {}
|
| 403 |
+
p = os.path.join(TILES, "index.jsonl")
|
| 404 |
+
if os.path.exists(p):
|
| 405 |
+
for line in open(p, "r", encoding="utf-8"):
|
| 406 |
+
try:
|
| 407 |
+
r = json.loads(line); clip_src[r["clip"]] = r["src"]
|
| 408 |
+
except Exception:
|
| 409 |
+
pass
|
| 410 |
+
rows = []
|
| 411 |
+
for name in sorted(os.listdir(LABELS)):
|
| 412 |
+
if not name.endswith(".csv"):
|
| 413 |
+
continue
|
| 414 |
+
clip = name[:-4]; src = clip_src.get(clip, "unknown")
|
| 415 |
+
with open(os.path.join(LABELS, name), "r", encoding="utf-8", newline="") as f:
|
| 416 |
+
for r in csv.DictReader(f):
|
| 417 |
+
try:
|
| 418 |
+
mad, flow, var = float(r["mad"]), float(r["flow"]), float(r["var"])
|
| 419 |
+
stage = r.get("stage", "") or ""
|
| 420 |
+
dup = int(float(r.get("dup", 0) or 0))
|
| 421 |
+
x, y = int(float(r["x"])), int(float(r["y"]))
|
| 422 |
+
except (KeyError, ValueError, TypeError):
|
| 423 |
+
continue # 被中断写坏的 CSV 行(缺列/None)直接跳过
|
| 424 |
+
if dup == 1:
|
| 425 |
+
soft = 1.0 if (flow == 0.0 and var == 0.0) else soft_from_flow(flow)
|
| 426 |
+
elif stage == "Flow":
|
| 427 |
+
if 0.1 <= flow <= 0.4:
|
| 428 |
+
continue
|
| 429 |
+
soft = soft_from_flow(flow)
|
| 430 |
+
else:
|
| 431 |
+
soft = 0.0
|
| 432 |
+
rows.append({"a": os.path.join(clip, r["frame_a"]), "b": os.path.join(clip, r["frame_b"]),
|
| 433 |
+
"x": x, "y": y, "crop": 128,
|
| 434 |
+
"soft": round(soft, 4), "hard": 1 if (stage == "Flow" and mad < 1.0) else 0,
|
| 435 |
+
"stage": stage, "src": src, "clip": clip})
|
| 436 |
+
pos = [r for r in rows if r["soft"] >= 0.9]
|
| 437 |
+
neg = [r for r in rows if r["soft"] <= 0.1]
|
| 438 |
+
mid = [r for r in rows if 0.1 < r["soft"] < 0.9]
|
| 439 |
+
rng.shuffle(pos); rng.shuffle(neg); rng.shuffle(mid)
|
| 440 |
+
neg.sort(key=lambda r: -r["hard"])
|
| 441 |
+
m = int(len(pos) * args.max_neg_ratio)
|
| 442 |
+
neg_keep = neg[:m] if len(neg) > m else neg
|
| 443 |
+
if neg_keep and len(pos) > len(neg_keep):
|
| 444 |
+
pos = pos[:len(neg_keep)]
|
| 445 |
+
mid_keep = mid[:int((len(pos) + len(neg_keep)) * args.mid_ratio)]
|
| 446 |
+
data = pos + neg_keep + mid_keep; rng.shuffle(data)
|
| 447 |
+
val_f = lambda r: sum(ord(c) for c in r["src"]) % 7 == 0
|
| 448 |
+
train = [r for r in data if not val_f(r)]; val = [r for r in data if val_f(r)]
|
| 449 |
+
for path, items in ((os.path.join(DATA, "train.jsonl"), train), (os.path.join(DATA, "val.jsonl"), val)):
|
| 450 |
+
with open(path, "w", encoding="utf-8") as f:
|
| 451 |
+
for it in items:
|
| 452 |
+
f.write(json.dumps(it, ensure_ascii=False) + "\n")
|
| 453 |
+
st = {"raw": len(rows), "kept_pos": len(pos), "kept_neg": len(neg_keep),
|
| 454 |
+
"kept_mid": len(mid_keep), "hard_mined": sum(1 for r in data if r["hard"]),
|
| 455 |
+
"train": len(train), "val": len(val)}
|
| 456 |
+
json.dump(st, open(os.path.join(DATA, "label_stats.json"), "w", encoding="utf-8"),
|
| 457 |
+
ensure_ascii=False, indent=2)
|
| 458 |
+
print(json.dumps(st, ensure_ascii=False), flush=True)
|
| 459 |
+
|
| 460 |
+
|
| 461 |
+
def eval_model(model, loader, device):
|
| 462 |
+
model.eval(); se = n = correct = pe = pn = ne = nn_ = 0
|
| 463 |
+
with torch.no_grad():
|
| 464 |
+
for a, b, y, _ in loader:
|
| 465 |
+
a, b, y = a.to(device), b.to(device), y.to(device)
|
| 466 |
+
p = model(a, b)
|
| 467 |
+
se += torch.abs(p - y).sum().item(); n += y.numel()
|
| 468 |
+
correct += ((p >= 0.5) == (y >= 0.5)).sum().item()
|
| 469 |
+
mm = y >= 0.9
|
| 470 |
+
if mm.any():
|
| 471 |
+
pe += torch.abs(p[mm] - y[mm]).sum().item(); pn += int(mm.sum())
|
| 472 |
+
mm = y <= 0.1
|
| 473 |
+
if mm.any():
|
| 474 |
+
ne += torch.abs(p[mm] - y[mm]).sum().item(); nn_ += int(mm.sum())
|
| 475 |
+
return {"mae": se / max(1, n), "acc": correct / max(1, n),
|
| 476 |
+
"pos_mae": pe / max(1, pn), "neg_mae": ne / max(1, nn_), "n": n}
|
| 477 |
+
|
| 478 |
+
|
| 479 |
+
class ClipGroupSampler(Sampler):
|
| 480 |
+
"""按片段分组取样:同一 clip 的���片连续读,机械盘上把随机读变成顺序读。
|
| 481 |
+
片段之间每个 epoch 重新打乱、片段内部也打乱,样本分布不变。"""
|
| 482 |
+
|
| 483 |
+
def __init__(self, dataset, shuffle=True, seed=1234):
|
| 484 |
+
self.groups = {}
|
| 485 |
+
for i, it in enumerate(dataset.items):
|
| 486 |
+
self.groups.setdefault(it["clip"], []).append(i)
|
| 487 |
+
self.clips = sorted(self.groups.keys())
|
| 488 |
+
self.shuffle = shuffle
|
| 489 |
+
self.seed = seed
|
| 490 |
+
self.epoch = 0
|
| 491 |
+
self.n = len(dataset)
|
| 492 |
+
|
| 493 |
+
def set_epoch(self, e):
|
| 494 |
+
self.epoch = e
|
| 495 |
+
|
| 496 |
+
def __len__(self):
|
| 497 |
+
return self.n
|
| 498 |
+
|
| 499 |
+
def __iter__(self):
|
| 500 |
+
rng = random.Random(self.seed + self.epoch)
|
| 501 |
+
clips = self.clips[:]
|
| 502 |
+
if self.shuffle:
|
| 503 |
+
rng.shuffle(clips)
|
| 504 |
+
for c in clips:
|
| 505 |
+
idx = self.groups[c]
|
| 506 |
+
if self.shuffle:
|
| 507 |
+
idx = idx[:]
|
| 508 |
+
rng.shuffle(idx)
|
| 509 |
+
for i in idx:
|
| 510 |
+
yield i
|
| 511 |
+
|
| 512 |
+
|
| 513 |
+
def _worker_init(_):
|
| 514 |
+
# 每个 DataLoader worker 只用 1 个 torch 线程:默认多线程在 8 个 worker 里会互相抢核,
|
| 515 |
+
# 实测反而拖慢取数(GPU 在等 batch)。解码/切片本身是单线程活。
|
| 516 |
+
try:
|
| 517 |
+
torch.set_num_threads(1)
|
| 518 |
+
except Exception:
|
| 519 |
+
pass
|
| 520 |
+
|
| 521 |
+
|
| 522 |
+
def cmd_train(args):
|
| 523 |
+
random.seed(args.seed); np.random.seed(args.seed)
|
| 524 |
+
torch.manual_seed(args.seed); torch.cuda.manual_seed_all(args.seed)
|
| 525 |
+
torch.backends.cudnn.benchmark = True
|
| 526 |
+
os.makedirs(args.out, exist_ok=True)
|
| 527 |
+
dev = "cuda" if torch.cuda.is_available() else "cpu"
|
| 528 |
+
print("DEVICE", dev, torch.cuda.get_device_name(0) if dev == "cuda" else "", flush=True)
|
| 529 |
+
tr = PairDataset(os.path.join(DATA, "train.jsonl"), TILES, augment=bool(args.augment))
|
| 530 |
+
va = PairDataset(os.path.join(DATA, "val.jsonl"), TILES)
|
| 531 |
+
print("SAMPLES", len(tr), len(va), flush=True)
|
| 532 |
+
tl = DataLoader(tr, batch_size=args.batch, sampler=ClipGroupSampler(tr, shuffle=True),
|
| 533 |
+
num_workers=args.workers, worker_init_fn=_worker_init, prefetch_factor=2,
|
| 534 |
+
pin_memory=(dev == "cuda"), drop_last=True, persistent_workers=args.workers > 0)
|
| 535 |
+
vl = DataLoader(va, batch_size=args.batch, sampler=ClipGroupSampler(va, shuffle=False),
|
| 536 |
+
num_workers=max(2, args.workers // 2), worker_init_fn=_worker_init,
|
| 537 |
+
pin_memory=(dev == "cuda"))
|
| 538 |
+
model = SiameseSFDP().to(dev); print("PARAMS", count_params(model), flush=True)
|
| 539 |
+
opt = torch.optim.AdamW(model.parameters(), lr=args.lr, weight_decay=1e-4)
|
| 540 |
+
steps = max(1, len(tl)); total = args.epochs * steps
|
| 541 |
+
|
| 542 |
+
def lr_at(step):
|
| 543 |
+
warm = args.warmup * steps
|
| 544 |
+
if step < warm:
|
| 545 |
+
return (step + 1) / max(1, warm)
|
| 546 |
+
return 0.5 * (1 + math.cos(math.pi * (step - warm) / max(1, total - warm)))
|
| 547 |
+
|
| 548 |
+
sched = torch.optim.lr_scheduler.LambdaLR(opt, lr_at)
|
| 549 |
+
best = 1e9
|
| 550 |
+
if args.resume and os.path.exists(args.resume):
|
| 551 |
+
ck = torch.load(args.resume, map_location=dev, weights_only=True)
|
| 552 |
+
model.load_state_dict(ck["model"]); opt.load_state_dict(ck["opt"])
|
| 553 |
+
best = ck.get("best", best)
|
| 554 |
+
print("RESUME", args.resume, "prev", ck.get("epoch"), flush=True)
|
| 555 |
+
log = os.path.join(args.out, "train_log.csv")
|
| 556 |
+
if not os.path.exists(log):
|
| 557 |
+
with open(log, "w", newline="", encoding="utf-8") as f:
|
| 558 |
+
csv.writer(f).writerow(["epoch", "lr", "loss", "val_mae", "val_acc", "pos_mae", "neg_mae", "sec"])
|
| 559 |
+
for ep in range(args.epochs):
|
| 560 |
+
if hasattr(tl.sampler, "set_epoch"):
|
| 561 |
+
tl.sampler.set_epoch(ep)
|
| 562 |
+
model.train(); t0 = time.time(); run = cnt = 0
|
| 563 |
+
se = 0
|
| 564 |
+
step_i = 0
|
| 565 |
+
for a, b, y, hard in tl:
|
| 566 |
+
a, b, y, hard = a.to(dev), b.to(dev), y.to(dev), hard.to(dev)
|
| 567 |
+
p = model(a, b); w = 1.0 + (args.hard_weight - 1.0) * hard
|
| 568 |
+
loss = (w * (p - y) ** 2).mean()
|
| 569 |
+
opt.zero_grad(set_to_none=True); loss.backward()
|
| 570 |
+
nn.utils.clip_grad_norm_(model.parameters(), 5.0)
|
| 571 |
+
opt.step(); sched.step()
|
| 572 |
+
run += loss.item() * a.size(0); cnt += a.size(0)
|
| 573 |
+
se += torch.abs(p - y).sum().item()
|
| 574 |
+
step_i += 1
|
| 575 |
+
if step_i % 50 == 0:
|
| 576 |
+
print(json.dumps({"step": step_i, "steps": len(tl), "ep": ep,
|
| 577 |
+
"loss": round(run / max(1, cnt), 5),
|
| 578 |
+
"pmae": round(se / max(1, cnt), 5),
|
| 579 |
+
"elapsed": round(time.time() - t0, 1)}), flush=True)
|
| 580 |
+
m = eval_model(model, vl, dev); sec = time.time() - t0; lr = opt.param_groups[0]["lr"]
|
| 581 |
+
print(json.dumps({"epoch": ep, "loss": round(run / max(1, cnt), 5), "lr": round(lr, 6),
|
| 582 |
+
**{k: round(v, 5) for k, v in m.items()}, "sec": round(sec, 1)}), flush=True)
|
| 583 |
+
with open(log, "a", newline="", encoding="utf-8") as f:
|
| 584 |
+
csv.writer(f).writerow([ep, round(lr, 6), round(run / max(1, cnt), 5), round(m["mae"], 5),
|
| 585 |
+
round(m["acc"], 5), round(m["pos_mae"], 5), round(m["neg_mae"], 5),
|
| 586 |
+
round(sec, 1)])
|
| 587 |
+
ck = {"model": model.state_dict(), "opt": opt.state_dict(), "epoch": ep, "best": best}
|
| 588 |
+
torch.save(ck, os.path.join(args.out, "last.pt"))
|
| 589 |
+
if m["mae"] < best:
|
| 590 |
+
best = m["mae"]; ck["best"] = best
|
| 591 |
+
torch.save(ck, os.path.join(args.out, "best.pt"))
|
| 592 |
+
print("BEST", round(best, 5), "epoch", ep, flush=True)
|
| 593 |
+
print("TRAIN_DONE best_mae", round(best, 5), flush=True)
|
| 594 |
+
ck_best = os.path.join(args.out, "best.pt")
|
| 595 |
+
if not os.path.exists(ck_best):
|
| 596 |
+
ck_best = os.path.join(args.out, "last.pt") # 本轮没刷新最优时用 last.pt 导出,避免导出报 FileNotFoundError
|
| 597 |
+
print("EXPORT_SRC", ck_best, flush=True)
|
| 598 |
+
export_onnx(ck_best, os.path.join(args.out, "easy4k_sfdp_siamese_fp32.onnx"))
|
| 599 |
+
|
| 600 |
+
|
| 601 |
+
def export_onnx(ckpt, out, opset=17):
|
| 602 |
+
import onnx
|
| 603 |
+
ck = torch.load(ckpt, map_location="cpu", weights_only=True)
|
| 604 |
+
m = SiameseSFDP(); m.load_state_dict(ck["model"]); m.eval()
|
| 605 |
+
a = torch.rand(2, 1, 128, 128); b = torch.rand(2, 1, 128, 128)
|
| 606 |
+
torch.onnx.export(m, (a, b), out, input_names=["frame_a", "frame_b"], output_names=["score"],
|
| 607 |
+
dynamic_axes={"frame_a": {0: "batch"}, "frame_b": {0: "batch"}, "score": {0: "batch"}},
|
| 608 |
+
opset_version=opset, do_constant_folding=True)
|
| 609 |
+
o = onnx.load(out); onnx.checker.check_model(o)
|
| 610 |
+
meta = {"model_name": "Easy4K SFDP - Siamese NET AI", "model_version": "0.6.0",
|
| 611 |
+
"task": "frame-duplicate similarity regression",
|
| 612 |
+
"input": "two grayscale 128x128 patches, float32, [0,1]",
|
| 613 |
+
"output": "score in [0,1]; 1 = duplicate/same picture",
|
| 614 |
+
"training": "knowledge distillation from Easy4K SFDP four-stage judge",
|
| 615 |
+
"runtime": "onnxruntime DirectML (NVIDIA / AMD / Intel) or CPU",
|
| 616 |
+
"license": "cc-by-nc-4.0", "params": str(count_params(m))}
|
| 617 |
+
have = {p.key for p in o.metadata_props}
|
| 618 |
+
for k, v in meta.items():
|
| 619 |
+
if k not in have:
|
| 620 |
+
p = o.metadata_props.add(); p.key, p.value = k, v
|
| 621 |
+
onnx.save(o, out)
|
| 622 |
+
print("EXPORTED", out, "size_mb", round(os.path.getsize(out) / 1048576, 2), flush=True)
|
| 623 |
+
|
| 624 |
+
# FP16 版:内部权重/算子转半精度,输入输出仍保持 float32,调用方不用改代码
|
| 625 |
+
try:
|
| 626 |
+
from onnxconverter_common import float16 as _f16
|
| 627 |
+
o16 = _f16.convert_float_to_float16(onnx.load(out), keep_io_types=True)
|
| 628 |
+
p16 = out.replace("_fp32.onnx", "_fp16.onnx") if out.endswith("_fp32.onnx") else (out[:-5] + "_fp16.onnx")
|
| 629 |
+
onnx.save(o16, p16)
|
| 630 |
+
print("EXPORTED_FP16", p16, "size_mb", round(os.path.getsize(p16) / 1048576, 2), flush=True)
|
| 631 |
+
except Exception as e:
|
| 632 |
+
print("EXPORTED_FP16_SKIP", repr(e), flush=True)
|
| 633 |
+
|
| 634 |
+
|
| 635 |
+
def cmd_eval(args):
|
| 636 |
+
ds = PairDataset(os.path.join(DATA, "val.jsonl"), TILES)
|
| 637 |
+
dl = DataLoader(ds, batch_size=512, shuffle=False, num_workers=6)
|
| 638 |
+
ck = torch.load(args.ckpt, map_location="cpu", weights_only=True)
|
| 639 |
+
net = SiameseSFDP(); net.load_state_dict(ck["model"]); net.eval()
|
| 640 |
+
S, Y, ST = [], [], []
|
| 641 |
+
with torch.no_grad():
|
| 642 |
+
off = 0
|
| 643 |
+
for a, b, y, _ in dl:
|
| 644 |
+
p = net(a, b).squeeze(1).numpy(); S.append(p); Y.append(y.squeeze(1).numpy())
|
| 645 |
+
ST.extend([ds.items[off + i]["stage"] for i in range(len(p))]); off += len(p)
|
| 646 |
+
s = np.concatenate(S); y = np.concatenate(Y); t = (y >= 0.5).astype(int); p = (s >= 0.5).astype(int)
|
| 647 |
+
tp = int(((p == 1) & (t == 1)).sum()); fp = int(((p == 1) & (t == 0)).sum())
|
| 648 |
+
fn = int(((p == 0) & (t == 1)).sum()); tn = int(((p == 0) & (t == 0)).sum())
|
| 649 |
+
print(json.dumps({"n": len(s), "acc": round((tp + tn) / len(s), 5),
|
| 650 |
+
"precision": round(tp / max(1, tp + fp), 5),
|
| 651 |
+
"recall": round(tp / max(1, tp + fn), 5)}), flush=True)
|
| 652 |
+
by = {}
|
| 653 |
+
for st in sorted(set(ST)):
|
| 654 |
+
mm = np.array([x == st for x in ST])
|
| 655 |
+
by[st] = {"n": int(mm.sum()), "mean_score": round(float(s[mm].mean()), 4),
|
| 656 |
+
"mean_target": round(float(y[mm].mean()), 4)}
|
| 657 |
+
print("BY_STAGE", json.dumps(by, ensure_ascii=False), flush=True)
|
| 658 |
+
for th in (0.4, 0.5, 0.6, 0.7, 0.8):
|
| 659 |
+
pp = (s >= th).astype(int)
|
| 660 |
+
tp_ = int(((pp == 1) & (t == 1)).sum()); fp_ = int(((pp == 1) & (t == 0)).sum())
|
| 661 |
+
fn_ = int(((pp == 0) & (t == 1)).sum())
|
| 662 |
+
pr = tp_ / max(1, tp_ + fp_); rc = tp_ / max(1, tp_ + fn_)
|
| 663 |
+
print(json.dumps({"t": th, "acc": round(float((pp == t).mean()), 5), "precision": round(pr, 5),
|
| 664 |
+
"recall": round(rc, 5), "f1": round(2 * pr * rc / max(1e-9, pr + rc), 5)}), flush=True)
|
| 665 |
+
|
| 666 |
+
|
| 667 |
+
def cmd_compare(args):
|
| 668 |
+
items = []
|
| 669 |
+
for name in sorted(os.listdir(TEST_LABELS)):
|
| 670 |
+
if not name.endswith(".csv"):
|
| 671 |
+
continue
|
| 672 |
+
clip = name[:-4]
|
| 673 |
+
for r in csv.DictReader(open(os.path.join(TEST_LABELS, name), "r", encoding="utf-8", newline="")):
|
| 674 |
+
flow, var = float(r["flow"]), float(r["var"]); dup = int(float(r.get("dup", 0) or 0))
|
| 675 |
+
stage = r.get("stage", "")
|
| 676 |
+
soft = (1.0 if (flow == 0.0 and var == 0.0) else soft_from_flow(flow)) if dup == 1 else \
|
| 677 |
+
(soft_from_flow(flow) if stage == "Flow" else 0.0)
|
| 678 |
+
items.append((os.path.join(clip, r["frame_a"]), os.path.join(clip, r["frame_b"]),
|
| 679 |
+
int(float(r["x"])), int(float(r["y"])), soft))
|
| 680 |
+
print("TEST_ITEMS", len(items), flush=True)
|
| 681 |
+
A, B = [], []
|
| 682 |
+
for pa, pb, x, y, _ in items:
|
| 683 |
+
a = load_gray(os.path.join(TEST_TILES, pa)); b = load_gray(os.path.join(TEST_TILES, pb))
|
| 684 |
+
h, w = a.shape; x = min(max(0, x), w - 128); y = min(max(0, y), h - 128)
|
| 685 |
+
A.append(torch.from_numpy(np.ascontiguousarray(a[y:y + 128, x:x + 128])).float().div_(255.0).unsqueeze(0))
|
| 686 |
+
B.append(torch.from_numpy(np.ascontiguousarray(b[y:y + 128, x:x + 128])).float().div_(255.0).unsqueeze(0))
|
| 687 |
+
A = torch.stack(A); B = torch.stack(B)
|
| 688 |
+
y = torch.tensor([it[4] for it in items], dtype=torch.float32).unsqueeze(1)
|
| 689 |
+
t = (y >= 0.5).int().numpy()
|
| 690 |
+
for path in args.ckpts.split(","):
|
| 691 |
+
path = path.strip()
|
| 692 |
+
if not os.path.exists(path):
|
| 693 |
+
print("MISSING", path, flush=True); continue
|
| 694 |
+
ck = torch.load(path, map_location="cpu", weights_only=True)
|
| 695 |
+
net = SiameseSFDP(); net.load_state_dict(ck["model"]); net.eval()
|
| 696 |
+
outs = []
|
| 697 |
+
with torch.no_grad():
|
| 698 |
+
for i in range(0, len(A), 512):
|
| 699 |
+
outs.append(net(A[i:i + 512], B[i:i + 512]).numpy())
|
| 700 |
+
s = np.concatenate(outs)
|
| 701 |
+
rec = {"ckpt": os.path.basename(os.path.dirname(path)), "epoch": ck.get("epoch"),
|
| 702 |
+
"mae": round(float(np.abs(s - y.numpy()).mean()), 5)}
|
| 703 |
+
for th in (0.5, 0.6, 0.7):
|
| 704 |
+
pp = (s >= th).astype(int)
|
| 705 |
+
tp = int(((pp == 1) & (t == 1)).sum()); fp = int(((pp == 1) & (t == 0)).sum())
|
| 706 |
+
fn = int(((pp == 0) & (t == 1)).sum()); tn = int(((pp == 0) & (t == 0)).sum())
|
| 707 |
+
rec[f"acc@{th}"] = round((tp + tn) / len(s), 4)
|
| 708 |
+
print(rec, flush=True)
|
| 709 |
+
|
| 710 |
+
|
| 711 |
+
def cmd_pack(args):
|
| 712 |
+
import shutil
|
| 713 |
+
card = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), "card.md"), "r", encoding="utf-8").read()
|
| 714 |
+
os.makedirs(args.out, exist_ok=True)
|
| 715 |
+
shutil.copy2(args.onnx, os.path.join(args.out, "model.onnx"))
|
| 716 |
+
_p16 = args.onnx.replace("_fp32.onnx", "_fp16.onnx")
|
| 717 |
+
if os.path.exists(_p16):
|
| 718 |
+
shutil.copy2(_p16, os.path.join(args.out, "model_fp16.onnx"))
|
| 719 |
+
size = round(os.path.getsize(args.onnx) / 1048576, 2)
|
| 720 |
+
best = {}
|
| 721 |
+
if os.path.exists(args.log):
|
| 722 |
+
for row in csv.DictReader(open(args.log, "r", encoding="utf-8")):
|
| 723 |
+
if not best or float(row["val_mae"]) < float(best["val_mae"]):
|
| 724 |
+
best = row
|
| 725 |
+
ev = {}
|
| 726 |
+
if os.path.exists(args.eval):
|
| 727 |
+
for enc in ("utf-8-sig", "utf-16", "utf-16-le"):
|
| 728 |
+
try:
|
| 729 |
+
for line in open(args.eval, "r", encoding=enc, errors="ignore"):
|
| 730 |
+
line = line.strip()
|
| 731 |
+
if line.startswith("{") and '"precision"' in line:
|
| 732 |
+
ev = json.loads(line); break
|
| 733 |
+
except Exception:
|
| 734 |
+
pass
|
| 735 |
+
if ev:
|
| 736 |
+
break
|
| 737 |
+
st = json.load(open(os.path.join(DATA, "label_stats.json"), "r", encoding="utf-8"))
|
| 738 |
+
clips = len([d for d in os.listdir(TILES) if os.path.isdir(os.path.join(TILES, d))])
|
| 739 |
+
pct = lambda x: ("%.2f%%" % (float(x) * 100)) if str(x) not in ("", "n/a") else "n/a"
|
| 740 |
+
vals = {"params": args.params, "opset": 17, "size_mb": size, "epochs": args.epochs, "clips": clips,
|
| 741 |
+
"samples": st.get("train", 0) + st.get("val", 0), "train": st.get("train"), "val": st.get("val"),
|
| 742 |
+
"ratio": "1 : %.2f" % (st.get("kept_neg", 0) / max(1, st.get("kept_pos", 1))),
|
| 743 |
+
"hard": st.get("hard_mined"), "mae": best.get("val_mae", "n/a"),
|
| 744 |
+
"acc": pct(best.get("val_acc", "n/a")), "precision": pct(ev.get("precision", "n/a")),
|
| 745 |
+
"recall": pct(ev.get("recall", "n/a")), "pos_mae": best.get("pos_mae", "n/a"),
|
| 746 |
+
"neg_mae": best.get("neg_mae", "n/a"), "amd_ms": args.amd_ms, "cpu_ms": args.cpu_ms}
|
| 747 |
+
for k, v in vals.items():
|
| 748 |
+
card = card.replace("{" + k + "}", str(v))
|
| 749 |
+
cfg = {"model_name": "Easy4K SFDP - Siamese NET AI", "model_version": "0.6.0",
|
| 750 |
+
"architecture": "siamese depthwise-separable CNN + difference/product fusion head",
|
| 751 |
+
"input": {"names": ["frame_a", "frame_b"], "shape": ["batch", 1, 128, 128],
|
| 752 |
+
"dtype": "float32", "range": [0, 1], "color": "grayscale"},
|
| 753 |
+
"output": {"name": "score", "shape": ["batch", 1], "meaning": "1 = duplicate"},
|
| 754 |
+
"opset": 17, "providers": ["DmlExecutionProvider", "CPUExecutionProvider"],
|
| 755 |
+
"recommended_threshold": 0.5,
|
| 756 |
+
"metrics": {"val_mae": float(best.get("val_mae", 0)), "val_acc": float(best.get("val_acc", 0)),
|
| 757 |
+
"precision": float(ev.get("precision", 0)), "recall": float(ev.get("recall", 0))},
|
| 758 |
+
"license": "cc-by-nc-4.0"}
|
| 759 |
+
json.dump(cfg, open(os.path.join(args.out, "config.json"), "w", encoding="utf-8"),
|
| 760 |
+
ensure_ascii=False, indent=2)
|
| 761 |
+
open(os.path.join(args.out, "README.md"), "w", encoding="utf-8").write(card)
|
| 762 |
+
open(os.path.join(args.out, "LICENSE"), "w", encoding="utf-8").write(
|
| 763 |
+
"Easy4K SFDP - Siamese NET AI\nCopyright (c) 2026 Easy4K\nCC BY-NC 4.0\n"
|
| 764 |
+
"https://creativecommons.org/licenses/by-nc/4.0/legalcode\n")
|
| 765 |
+
open(os.path.join(args.out, "requirements.txt"), "w", encoding="utf-8").write(
|
| 766 |
+
"onnxruntime-directml>=1.18\nnumpy>=1.26\n")
|
| 767 |
+
src = os.path.join(RAW, "sources.json")
|
| 768 |
+
if os.path.exists(src):
|
| 769 |
+
shutil.copy2(src, os.path.join(args.out, "sources.json"))
|
| 770 |
+
print("PACKED", args.out, flush=True)
|
| 771 |
+
|
| 772 |
+
|
| 773 |
+
def main():
|
| 774 |
+
ap = argparse.ArgumentParser(); sub = ap.add_subparsers(dest="cmd", required=True)
|
| 775 |
+
p = sub.add_parser("rank")
|
| 776 |
+
p.add_argument("--target", type=int, default=40); p.add_argument("--pages", type=int, default=6)
|
| 777 |
+
p.add_argument("--min-sec", type=int, default=60); p.add_argument("--max-sec", type=int, default=420)
|
| 778 |
+
p.add_argument("--seeds", type=int, default=60); p.add_argument("--seed-existing", type=int, default=300)
|
| 779 |
+
p.add_argument("--outdir", default=BILI_RAW); p.set_defaults(func=cmd_rank)
|
| 780 |
+
p = sub.add_parser("tiles")
|
| 781 |
+
p.add_argument("--raw", required=True); p.add_argument("--out", default=TILES)
|
| 782 |
+
p.add_argument("--tile", type=int, default=320); p.add_argument("--frames", type=int, default=32)
|
| 783 |
+
p.add_argument("--segments", type=int, default=4); p.add_argument("--min-width", type=int, default=640)
|
| 784 |
+
p.add_argument("--seed", type=int, default=1234); p.set_defaults(func=cmd_tiles)
|
| 785 |
+
p = sub.add_parser("teacher")
|
| 786 |
+
p.add_argument("--tiles", default=TILES); p.add_argument("--labels", default=LABELS)
|
| 787 |
+
p.set_defaults(func=cmd_teacher)
|
| 788 |
+
p = sub.add_parser("label")
|
| 789 |
+
p.add_argument("--max-neg-ratio", type=float, default=2.0); p.add_argument("--mid-ratio", type=float, default=0.25)
|
| 790 |
+
p.add_argument("--seed", type=int, default=2026); p.set_defaults(func=cmd_label)
|
| 791 |
+
p = sub.add_parser("train")
|
| 792 |
+
p.add_argument("--out", required=True); p.add_argument("--epochs", type=int, default=8)
|
| 793 |
+
p.add_argument("--batch", type=int, default=256); p.add_argument("--lr", type=float, default=2e-3)
|
| 794 |
+
p.add_argument("--workers", type=int, default=8); p.add_argument("--hard-weight", type=float, default=2.0)
|
| 795 |
+
p.add_argument("--warmup", type=int, default=2); p.add_argument("--augment", type=int, default=0)
|
| 796 |
+
p.add_argument("--seed", type=int, default=2026); p.add_argument("--resume", default="")
|
| 797 |
+
p.set_defaults(func=cmd_train)
|
| 798 |
+
p = sub.add_parser("eval"); p.add_argument("--ckpt", required=True); p.set_defaults(func=cmd_eval)
|
| 799 |
+
p = sub.add_parser("compare"); p.add_argument("--ckpts", required=True); p.set_defaults(func=cmd_compare)
|
| 800 |
+
p = sub.add_parser("pack")
|
| 801 |
+
p.add_argument("--onnx", required=True); p.add_argument("--out", required=True)
|
| 802 |
+
p.add_argument("--log", required=True); p.add_argument("--eval", required=True)
|
| 803 |
+
p.add_argument("--params", default="2602017"); p.add_argument("--epochs", default="8")
|
| 804 |
+
p.add_argument("--amd-ms", default="0.56"); p.add_argument("--cpu-ms", default="0.317")
|
| 805 |
+
p.set_defaults(func=cmd_pack)
|
| 806 |
+
args = ap.parse_args(); args.func(args)
|
| 807 |
+
|
| 808 |
+
|
| 809 |
+
if __name__ == "__main__":
|
| 810 |
+
main()
|
model_fp16.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:666011d5fdee4a9b3e03d58e1c1f04fecc9c8dd25d76a7125c769f9bd1808597
|
| 3 |
+
size 5248967
|
requirements.txt
CHANGED
|
@@ -1,2 +1,9 @@
|
|
| 1 |
-
|
| 2 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# 推理
|
| 2 |
+
onnxruntime-directml>=1.18
|
| 3 |
+
numpy>=1.26
|
| 4 |
+
|
| 5 |
+
# 训练 / 导出
|
| 6 |
+
torch>=2.6.0
|
| 7 |
+
opencv-python>=4.9
|
| 8 |
+
onnx>=1.16
|
| 9 |
+
onnxconverter-common>=1.16
|
train_log.csv
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
epoch,lr,loss,val_mae,val_acc,pos_mae,neg_mae,sec
|
| 2 |
+
0,0.001001,0.0362,0.03359,0.96952,0.01559,0.04536,529.4
|
| 3 |
+
1,0.002,0.02679,0.03907,0.96521,0.0104,0.06385,478.4
|
| 4 |
+
2,0.001866,0.02528,0.03599,0.96902,0.00891,0.05877,477.7
|
| 5 |
+
3,0.0015,0.02081,0.03631,0.96857,0.00749,0.06233,478.5
|
| 6 |
+
4,0.001,0.01861,0.03052,0.97374,0.00938,0.04752,477.6
|
| 7 |
+
5,0.0005,0.01506,0.03056,0.97374,0.00911,0.04821,478.1
|
| 8 |
+
6,0.000134,0.01389,0.02959,0.97451,0.00888,0.04681,476.9
|
| 9 |
+
7,0.0,0.01253,0.02788,0.97562,0.01063,0.04085,477.7
|
训练素材清单.txt
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|