yuukicammy commited on
Commit
62db7d4
·
0 Parent(s):
.dockerignore ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ .git
2
+ .venv
3
+ __pycache__
4
+ *.pyc
5
+ .ruff_cache
6
+ .pytest_cache
7
+ archive
8
+ tests
9
+ docs
10
+ *.md
11
+ !README.md
.gitignore ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+ *.egg-info/
6
+ *.egg
7
+ src/*.egg-info/
8
+ dist/
9
+ build/
10
+ *.whl
11
+
12
+ # Virtual environment
13
+ .venv/
14
+
15
+ # Build output
16
+ node_modules/
17
+
18
+ # Test
19
+ test-results/
20
+ playwright-report/
21
+ .coverage
22
+ htmlcov/
23
+ coverage/
24
+
25
+ # Logs
26
+ logs/
27
+ *.log
28
+ archive/
29
+
30
+ # OS
31
+ .DS_Store
32
+ Thumbs.db
33
+
34
+ # IDE
35
+ .idea/
36
+ *.swp
37
+ *.swo
38
+
39
+ # Linter / Type checker cache
40
+ .mypy_cache/
41
+ .ruff_cache/
42
+
43
+ # Environment variables
44
+ .env
45
+ .env.*
46
+
47
+ # uv
48
+ .python-version
DEVELOPMENT.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 開発ガイド
2
+
3
+ ## セットアップ
4
+
5
+ ### Python
6
+
7
+ ```bash
8
+ uv sync --dev
9
+ ```
10
+
11
+ ## ローカルでの起動
12
+
13
+ コンテナや `Dockerfile` は不要です。リポジトリルートで依存関係を入れたうえで、`uvicorn` を直接起動します。
14
+
15
+ 1. **依存関係の同期**(未実施の場合)
16
+
17
+ ```bash
18
+ uv sync --dev
19
+ ```
20
+
21
+ 2. **Web サーバー起動**
22
+
23
+ ```bash
24
+ uv run uvicorn kuzushiji_ocr.web:app --reload --host 127.0.0.1 --port 8000
25
+ ```
26
+
27
+ 3. ブラウザで `http://127.0.0.1:8000` を開く。
28
+
29
+ OCR 本体は Modal 上の Qwen2.5-VL(`vlm_app.py`)を呼び出すため、**初回から推論させるには**ローカルで `modal` にログイン済みであること、および `modal deploy src/kuzushiji_ocr/vlm_app.py` でアプリがデプロイ済みであることが必要です(詳細はルートの `README.md`)。
30
+
31
+ Hugging Face Space 用の `Dockerfile` は、ローカル開発では使いません(Space 上での実行専用)。
32
+
33
+ ## リント・フォーマット
34
+
35
+ [Ruff](https://docs.astral.sh/ruff/) を使用します(設定は `pyproject.toml` の `[tool.ruff]`)。
36
+
37
+ ```bash
38
+ uv run ruff check .
39
+ uv run ruff format .
40
+ ```
41
+
42
+ ## テスト
43
+
44
+ ```bash
45
+ uv run pytest
46
+ ```
47
+
48
+ ## ディレクトリ構成
49
+
50
+ | パス | 内容 |
51
+ |------|------|
52
+ | `src/kuzushiji_ocr/` | Qwen2.5-VL ベースの本流(FastAPI `web.py`、`vlm_app.py`、テンプレート) |
53
+ | `comparison/` | 別手法との比較用(PaddleOCR / GLM / NDL 古典籍OCR / Gemini などの Modal またはローカルスクリプト) |
54
+ | `archive/pre-restructure-2026-04/` | 再編前のフラット構成のスナップショット |
55
+ | `docs/` | 利用・設計メモ |
56
+ | `tests/` | pytest |
57
+
58
+ ## Hugging Face Space
59
+
60
+ ルートの `Dockerfile` で `kuzushiji_ocr.web:app` を `uvicorn` 起動します。Space の README に YAML フロントマターを含めているため、リポジトリをそのまま Docker Space に接続できます。
61
+
62
+ **注意:** Space 上から Modal に接続するには、実行環境に `modal` の認証情報(トークン等)を安全に渡す必要があります。公開 Space ではシークレットの取り扱いに十分注意してください。
Dockerfile ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Hugging Face Spaces(Docker SDK)用。ローカル開発は `uv run uvicorn ...` を参照。
2
+ FROM python:3.12-slim-bookworm
3
+
4
+ WORKDIR /app
5
+
6
+ RUN pip install --no-cache-dir uv
7
+
8
+ COPY pyproject.toml uv.lock ./
9
+ COPY src ./src
10
+
11
+ RUN uv sync --frozen --no-dev
12
+
13
+ ENV PORT=7860
14
+ EXPOSE 7860
15
+
16
+ CMD ["uv", "run", "uvicorn", "kuzushiji_ocr.web:app", "--host", "0.0.0.0", "--port", "7860"]
LICENSE ADDED
@@ -0,0 +1,176 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: くずし字 OCR
3
+ emoji: 📜
4
+ colorFrom: gray
5
+ colorTo: green
6
+ sdk: docker
7
+ pinned: false
8
+ license: apache-2.0
9
+ ---
10
+
11
+ # くずし字 OCR
12
+
13
+ くずし字が含まれる画像をアップロードすると、Modal 上の Qwen2.5-VL により **bounding box 付き重畳画像** と **多段推論による解説テキスト** を返す Web アプリケーションです。
14
+
15
+ ## デモ(限定公開)
16
+
17
+ Hugging Face Space でホストする場合は、このリポジトリを Space に接続し、**Docker** SDK でビルドしてください。ローカルと同様に `uvicorn` がポート **7860** で待ち受けます。
18
+
19
+ ## 要件
20
+
21
+ - Python 3.10 以上
22
+ - [uv](https://docs.astral.sh/uv/)(推奨)
23
+ - Modal 利用時: アカウントと `modal` CLI、デプロイ済みの `src/kuzushiji_ocr/vlm_app.py`
24
+
25
+ ## ローカルでの起動
26
+
27
+ 依存関係のインストール:
28
+
29
+ ```bash
30
+ uv sync --dev
31
+ ```
32
+
33
+ Web サーバー:
34
+
35
+ ```bash
36
+ uv run uvicorn kuzushiji_ocr.web:app --reload --port 8000
37
+ ```
38
+
39
+ ブラウザで `http://127.0.0.1:8000` を開きます。
40
+
41
+ ## Modal(VLM 推論)
42
+
43
+ 事前に VLM アプリをデプロイしておきます。
44
+
45
+ ```bash
46
+ modal deploy src/kuzushiji_ocr/vlm_app.py
47
+ ```
48
+
49
+ 詳細は `src/kuzushiji_ocr/vlm_app.py` のドキュメント文字列を参照してください。
50
+
51
+ 別モデル(PaddleOCR、GLM、NDL 古典籍OCR、Gemini など)で比較実験するスクリプトは `comparison/` にあります。
52
+
53
+ ## ドキュメント
54
+
55
+ - [はじめに](docs/getting-started.md)
56
+ - [開発者向け](DEVELOPMENT.md)
57
+
58
+ ## ライセンス
59
+
60
+ Apache License 2.0 — [LICENSE](LICENSE)
comparison/Dockerfile.ndlkotenocr ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # NDL古典籍OCR (ndlkotenocr_cli) — Modal 用ビルド
2
+ # 元: https://github.com/ndl-lab/ndlkotenocr_cli/blob/master/docker/Dockerfile
3
+ # COPY の代わりにリポジトリを clone し、ビルドコンテキスト不要で再現可能にする
4
+
5
+ FROM nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04
6
+
7
+ ENV PROJECT_DIR=/root/kotenocr_cli
8
+ ENV FORCE_CUDA="1"
9
+ ENV TORCH_CUDA_ARCH_LIST="7.5+PTX"
10
+ ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
11
+ ENV DEBIAN_FRONTEND=noninteractive
12
+
13
+ RUN set -x \
14
+ && apt update \
15
+ && apt upgrade -y
16
+
17
+ RUN set -x \
18
+ && apt update \
19
+ && apt -y install locales \
20
+ && locale-gen ja_JP.UTF-8
21
+ ENV LANG ja_JP.UTF-8
22
+ ENV LANGUAGE ja_JP:ja
23
+ ENV LC_ALL ja_JP.UTF-8
24
+ RUN localedef -f UTF-8 -i ja_JP ja_JP.utf8
25
+
26
+ # Modal の関数ランタイムは PATH 上の `python` と `pip` を検出する(python3 / pip3 のみだと失敗する)
27
+ RUN set -x && apt -y install libgl1-mesa-dev libglib2.0-0 zip git python3-pip wget python-is-python3 \
28
+ && ln -sf /usr/bin/pip3 /usr/bin/pip
29
+
30
+ ARG NDLKOTEN_BRANCH=master
31
+ RUN set -x && git clone --depth 1 --branch "${NDLKOTEN_BRANCH}" \
32
+ https://github.com/ndl-lab/ndlkotenocr_cli.git "${PROJECT_DIR}"
33
+
34
+ RUN set -x && pip3 install --no-cache-dir torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 torchtext==0.16.1 --index-url https://download.pytorch.org/whl/cu121
35
+ RUN set -x \
36
+ && pip3 install --no-cache-dir -r ${PROJECT_DIR}/requirements.txt
37
+
38
+ RUN set -x && wget -nc https://lab.ndl.go.jp/dataset/ndlkotensekiocr/trocr/model-ver2.zip -P ${PROJECT_DIR}/src/text_kotenseki_recognition/
39
+ RUN set -x && wget -nc https://lab.ndl.go.jp/dataset/ndlkotensekiocr/layoutmodel/ndl_kotenseki_layout_ver3.pth -P ${PROJECT_DIR}/src/ndl_kotenseki_layout/models/
40
+ RUN set -x && cd ${PROJECT_DIR}/src/text_kotenseki_recognition/ && unzip -o model-ver2.zip
41
+ RUN set -x && pip3 install --no-cache-dir mmcv==2.1.0 -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1/index.html
42
+
43
+ WORKDIR ${PROJECT_DIR}
comparison/glm_ocr_app.py ADDED
@@ -0,0 +1,272 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ GLM-OCR (0.9B) によるくずし字 OCR(Modal 上で実行)。
3
+
4
+ 特徴:
5
+ - わずか 0.9B パラメータで OmniDocBench v1.5 総合1位(94.62)
6
+ - T4 GPU(16GB VRAM)で動作可能(低コスト)
7
+
8
+ 前提:
9
+ - Modal アカウントと認証済み
10
+ - 初回はイメージビルドに時間がかかる(〜10 分)
11
+
12
+ 単一画像:
13
+ modal run comparison/glm_ocr_app.py --image-path ./sample.jpg
14
+
15
+ ディレクトリ一括処理:
16
+ modal run comparison/glm_ocr_app.py --input-dir ./dataset/
17
+
18
+ 座標付き構造化出力(NDL OCR 互換 JSON):
19
+ modal run comparison/glm_ocr_app.py --image-path ./sample.jpg --structured
20
+ """
21
+
22
+ from __future__ import annotations
23
+
24
+ import io
25
+ import json
26
+ import re
27
+ from pathlib import Path
28
+ from typing import Any
29
+
30
+ import modal
31
+
32
+ MODEL_ID = "zai-org/GLM-OCR"
33
+
34
+ app = modal.App("kuzushiji-glm-ocr")
35
+
36
+ hf_secret = modal.Secret.from_name("huggingface-secret")
37
+
38
+
39
+ def _download_model():
40
+ from huggingface_hub import snapshot_download
41
+
42
+ snapshot_download(MODEL_ID)
43
+
44
+
45
+ glm_image = (
46
+ modal.Image
47
+ .debian_slim(python_version="3.11")
48
+ .pip_install(
49
+ "torch>=2.5.0",
50
+ "torchvision>=0.20.0",
51
+ "transformers>=5.3.0",
52
+ "accelerate>=0.34.0",
53
+ "huggingface_hub",
54
+ "Pillow",
55
+ )
56
+ .run_function(_download_model, secrets=[hf_secret])
57
+ )
58
+
59
+ IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".bmp", ".jp2"}
60
+
61
+ DEFAULT_PROMPT = (
62
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。\n"
63
+ "縦書きの場合は右から左の順で、行ごとに改行して出力してください。\n"
64
+ "読めない文字は□で示してください。翻刻テキストのみを出力し、説明は不要です。"
65
+ )
66
+
67
+ STRUCTURED_PROMPT = (
68
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。\n"
69
+ "各テキスト行について、画像上のバウンディングボックス座標(ピクセル)と"
70
+ "翻刻テキストを以下の JSON 配列で出力してください。\n"
71
+ '形式: [[x1, y1, x2, y2, "翻刻テキスト"], ...]\n'
72
+ "座標は画像の左上を原点としたピクセル値です。\n"
73
+ "縦書きの場合は右の列から順に並べてください。読めない文字は□で示してください。\n"
74
+ "JSON のみを出力し、説明文は付けないでください。"
75
+ )
76
+
77
+
78
+ def _parse_structured_output(raw: str) -> list | None:
79
+ """VLM の生テキストから JSON を抽出し、NDL OCR 互換の
80
+ [[x1, y1, x2, y2, "text"], ...] 形式に正規化する。"""
81
+ match = re.search(r"\[.*\]", raw, re.DOTALL)
82
+ if not match:
83
+ return None
84
+ try:
85
+ data = json.loads(match.group())
86
+ except (json.JSONDecodeError, TypeError):
87
+ return None
88
+
89
+ result: list[list] = []
90
+ for item in data:
91
+ if isinstance(item, list) and len(item) >= 5:
92
+ result.append(item[:5])
93
+ elif isinstance(item, dict):
94
+ bbox = item.get("bbox", item.get("box", []))
95
+ text = item.get("text", item.get("content", ""))
96
+ if len(bbox) == 4:
97
+ result.append([*bbox, text])
98
+ return [result] if result else None
99
+
100
+
101
+ MAX_SIDE = 1536
102
+
103
+
104
+ def _fit_image(image, max_side: int = MAX_SIDE):
105
+ """長辺が max_side を超える場合にアスペクト比を維持してリサイズする。"""
106
+ w, h = image.size
107
+ if max(w, h) <= max_side:
108
+ return image
109
+ scale = max_side / max(w, h)
110
+ from PIL import Image as PILImage
111
+
112
+ return image.resize((int(w * scale), int(h * scale)), PILImage.LANCZOS)
113
+
114
+
115
+ @app.cls(image=glm_image, gpu="A10G", timeout=60 * 60, memory=32768, secrets=[hf_secret])
116
+ class GLMOCRInference:
117
+ @modal.enter()
118
+ def setup(self):
119
+ import torch
120
+ from transformers import AutoModelForImageTextToText, AutoProcessor
121
+
122
+ self.processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
123
+ self.model = AutoModelForImageTextToText.from_pretrained(
124
+ MODEL_ID,
125
+ dtype=torch.bfloat16,
126
+ device_map="auto",
127
+ trust_remote_code=True,
128
+ )
129
+
130
+ @modal.method()
131
+ def ocr_pages(
132
+ self,
133
+ pages: list[tuple[str, bytes]],
134
+ prompt: str = DEFAULT_PROMPT,
135
+ structured: bool = False,
136
+ ) -> dict[str, Any]:
137
+ import torch
138
+ from PIL import Image
139
+
140
+ effective_prompt = STRUCTURED_PROMPT if structured else prompt
141
+ max_tokens = 4096 if structured else 2048
142
+
143
+ texts: dict[str, str] = {}
144
+ jsons: dict[str, Any] = {}
145
+
146
+ for name, data in pages:
147
+ image = _fit_image(Image.open(io.BytesIO(data)).convert("RGB"))
148
+
149
+ messages = [
150
+ {
151
+ "role": "user",
152
+ "content": [
153
+ {"type": "image"},
154
+ {"type": "text", "text": effective_prompt},
155
+ ],
156
+ }
157
+ ]
158
+
159
+ text_input = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
160
+ inputs = self.processor(
161
+ text=[text_input],
162
+ images=[image],
163
+ return_tensors="pt",
164
+ padding=True,
165
+ ).to(self.model.device)
166
+
167
+ with torch.inference_mode():
168
+ output_ids = self.model.generate(**inputs, max_new_tokens=max_tokens)
169
+
170
+ trimmed = output_ids[0][inputs["input_ids"].shape[-1] :]
171
+ output = self.processor.decode(
172
+ trimmed,
173
+ skip_special_tokens=True,
174
+ clean_up_tokenization_spaces=False,
175
+ )
176
+
177
+ stem = Path(name).stem
178
+ texts[f"{stem}_main.txt"] = output
179
+
180
+ if structured:
181
+ parsed = _parse_structured_output(output)
182
+ if parsed is not None:
183
+ jsons[f"{stem}.json"] = parsed
184
+
185
+ return {"txt": texts, "json": jsons}
186
+
187
+
188
+ # ---------------------------------------------------------------------------
189
+ # Local entrypoint
190
+ # ---------------------------------------------------------------------------
191
+
192
+
193
+ @app.local_entrypoint()
194
+ def main(
195
+ image_path: str | None = None,
196
+ input_dir: str | None = None,
197
+ structured: bool = False,
198
+ prompt: str | None = None,
199
+ ) -> None:
200
+ """
201
+ 例:
202
+ modal run comparison/glm_ocr_app.py --image-path ./page.jpg
203
+ modal run comparison/glm_ocr_app.py --input-dir ./dataset/
204
+ modal run comparison/glm_ocr_app.py --image-path ./page.jpg --structured
205
+ """
206
+ if input_dir:
207
+ _run_batch(input_dir, structured, prompt)
208
+ elif image_path:
209
+ _run_single(image_path, structured, prompt)
210
+ else:
211
+ print("用法:")
212
+ print(" modal run comparison/glm_ocr_app.py --image-path ./your.jpg")
213
+ print(" modal run comparison/glm_ocr_app.py --input-dir ./images/")
214
+ print(" modal run comparison/glm_ocr_app.py --image-path ./your.jpg --structured")
215
+
216
+
217
+ def _run_single(image_path: str, structured: bool, prompt: str | None) -> None:
218
+ p = Path(image_path)
219
+ data = p.read_bytes()
220
+ print(f"推論中: {p.name} ({len(data)} bytes) [model={MODEL_ID}]")
221
+
222
+ inference = GLMOCRInference()
223
+ kwargs: dict[str, Any] = {"structured": structured}
224
+ if prompt:
225
+ kwargs["prompt"] = prompt
226
+ out = inference.ocr_pages.remote([(p.name, data)], **kwargs)
227
+ _print_results(out)
228
+
229
+
230
+ def _run_batch(input_dir: str, structured: bool, prompt: str | None) -> None:
231
+ src = Path(input_dir)
232
+ if not src.is_dir():
233
+ print(f"ディレクトリが見つかりません: {input_dir}")
234
+ return
235
+
236
+ pages: list[tuple[str, bytes]] = []
237
+ for p in sorted(src.rglob("*")):
238
+ if p.is_file() and p.suffix.lower() in IMAGE_EXTS:
239
+ pages.append((p.name, p.read_bytes()))
240
+
241
+ if not pages:
242
+ print(f"画像が見つかりません: {input_dir}")
243
+ return
244
+
245
+ total_mb = sum(len(d) for _, d in pages) / 1024 / 1024
246
+ print(f"{len(pages)} 枚の画像を処理中 ({total_mb:.1f} MB) [model={MODEL_ID}] ...")
247
+
248
+ inference = GLMOCRInference()
249
+ kwargs: dict[str, Any] = {"structured": structured}
250
+ if prompt:
251
+ kwargs["prompt"] = prompt
252
+ out = inference.ocr_pages.remote(pages, **kwargs)
253
+ _print_results(out)
254
+
255
+
256
+ def _print_results(out: dict[str, Any]) -> None:
257
+ texts = out.get("txt", {})
258
+ jsons = out.get("json", {})
259
+
260
+ if not texts and not jsons:
261
+ print("出力がありません。")
262
+ return
263
+
264
+ for name, text in texts.items():
265
+ print(f"--- {name} ---")
266
+ print(text[:2000] + ("..." if len(text) > 2000 else ""))
267
+
268
+ if jsons:
269
+ print("\n========== JSON 出力 ==========")
270
+ for name, obj in jsons.items():
271
+ print(f"--- {name} ---")
272
+ print(json.dumps(obj, ensure_ascii=False, indent=2)[:4000])
comparison/ndlkotenocr_app.py ADDED
@@ -0,0 +1,287 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ NDL古典籍OCR (ndlkotenocr_cli) を Modal 上で実行するアプリ。
3
+
4
+ 前提:
5
+ - Modal アカウントと `modal token new` などで認証済み
6
+
7
+ デプロイ:
8
+ modal deploy comparison/ndlkotenocr_app.py
9
+
10
+ ローカルから 1 枚試す:
11
+ modal run comparison/ndlkotenocr_app.py --image-path ./sample.jpg
12
+
13
+ ディレクトリ一括処理(Volume 経由):
14
+ modal run comparison/ndlkotenocr_app.py --input-dir ./dataset/
15
+ """
16
+
17
+ from __future__ import annotations
18
+
19
+ import json
20
+ import os
21
+ import shutil
22
+ import subprocess
23
+ from pathlib import Path
24
+ from typing import Any
25
+
26
+ import modal
27
+
28
+ DOCKERFILE = "Dockerfile.ndlkotenocr"
29
+
30
+ app = modal.App("ndl-koten-ocr")
31
+
32
+ # Dockerfile にローカル COPY が無いため、context は Dockerfile 所在ディレクトリで足りる
33
+ koten_image = modal.Image.from_dockerfile(str(DOCKERFILE))
34
+
35
+ vol = modal.Volume.from_name("ndl-koten-ocr-data", create_if_missing=True)
36
+ VOL_PATH = "/data"
37
+ IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".bmp", ".jp2"}
38
+
39
+
40
+ def _collect_outputs(
41
+ output_root: str,
42
+ input_basename: str,
43
+ layout_b: bool,
44
+ ) -> dict[str, Any]:
45
+ """output_root/<input_root名>/ 以下の txt / json を読み取る(-s b のときは各 bookid 配下)。"""
46
+ base = Path(output_root) / input_basename
47
+ texts: dict[str, str] = {}
48
+ jsons: dict[str, Any] = {}
49
+
50
+ def read_pair(txt_dir: Path, json_dir: Path, prefix: str) -> None:
51
+ if txt_dir.is_dir():
52
+ for p in sorted(txt_dir.glob("*.txt")):
53
+ key = f"{prefix}{p.name}" if prefix else p.name
54
+ texts[key] = p.read_text(encoding="utf-8", errors="replace")
55
+ if json_dir.is_dir():
56
+ for p in sorted(json_dir.glob("*.json")):
57
+ key = f"{prefix}{p.name}" if prefix else p.name
58
+ try:
59
+ jsons[key] = json.loads(p.read_text(encoding="utf-8"))
60
+ except json.JSONDecodeError:
61
+ jsons[key] = None
62
+
63
+ if layout_b:
64
+ for book_dir in sorted(base.iterdir()):
65
+ if not book_dir.is_dir():
66
+ continue
67
+ prefix = f"{book_dir.name}/"
68
+ read_pair(book_dir / "txt", book_dir / "json", prefix)
69
+ else:
70
+ read_pair(base / "txt", base / "json", "")
71
+
72
+ return {"txt": texts, "json": jsons}
73
+
74
+
75
+ @app.function(volumes={VOL_PATH: vol})
76
+ def upload_images(files: list[tuple[str, bytes]], input_subdir: str = "input") -> int:
77
+ """ローカルの画像を Volume にアップロードする。相対パスを維持する。"""
78
+ img_dir = Path(VOL_PATH) / input_subdir / "img"
79
+ img_dir.mkdir(parents=True, exist_ok=True)
80
+ count = 0
81
+ for relpath, data in files:
82
+ dest = img_dir / relpath
83
+ dest.parent.mkdir(parents=True, exist_ok=True)
84
+ dest.write_bytes(data)
85
+ count += 1
86
+ vol.commit()
87
+ return count
88
+
89
+
90
+ @app.function(
91
+ image=koten_image,
92
+ gpu="A10G",
93
+ timeout=60 * 60,
94
+ memory=32768,
95
+ volumes={VOL_PATH: vol},
96
+ )
97
+ def infer_from_volume(
98
+ input_subdir: str = "input",
99
+ output_subdir: str = "output",
100
+ input_structure: str = "s",
101
+ add_image_info: bool = False,
102
+ ) -> dict[str, Any]:
103
+ """Volume 上の画像に対して OCR 推論を実行し、結果も Volume に保存する。"""
104
+ vol.reload()
105
+ input_root = Path(VOL_PATH) / input_subdir
106
+ output_root = Path(VOL_PATH) / output_subdir
107
+
108
+ if output_root.exists():
109
+ shutil.rmtree(output_root)
110
+
111
+ cmd = [
112
+ "python3",
113
+ "main.py",
114
+ "infer",
115
+ str(input_root),
116
+ str(output_root),
117
+ "-s",
118
+ input_structure,
119
+ ]
120
+ if add_image_info:
121
+ cmd.append("-a")
122
+
123
+ subprocess.run(
124
+ cmd,
125
+ cwd="/root/kotenocr_cli",
126
+ env={**os.environ, "PYTHONUNBUFFERED": "1"},
127
+ check=True,
128
+ )
129
+
130
+ vol.commit()
131
+ return _collect_outputs(str(output_root), input_root.name, input_structure == "b")
132
+
133
+
134
+ @app.function(
135
+ image=koten_image,
136
+ gpu="A10G",
137
+ timeout=60 * 60,
138
+ memory=32768,
139
+ )
140
+ def infer_pages(
141
+ pages: list[tuple[str, bytes]],
142
+ input_structure: str = "s",
143
+ add_image_info: bool = False,
144
+ ) -> dict[str, Any]:
145
+ """
146
+ 複数ページを一度に推論する。
147
+
148
+ Args:
149
+ pages: (ファイル名, JPEG/PNG 等のバイト列) のリスト。構造 s のとき img 直下に並ぶ。
150
+ input_structure: ndlkotenocr_cli の -s と同じ(s=単一資料, b=複数 bookid)。
151
+ add_image_info: True のとき main.py に -a を付与(JSON 形式が変わる)。
152
+
153
+ Returns:
154
+ {"txt": {filename: text}, "json": {filename: object}}
155
+ """
156
+ if not pages:
157
+ return {"txt": {}, "json": {}}
158
+
159
+ work = Path("/tmp/ndl_koten_work")
160
+ input_root = work / "in"
161
+ output_root = work / "out"
162
+ if work.exists():
163
+ shutil.rmtree(work)
164
+ img_dir = input_root / "img"
165
+ img_dir.mkdir(parents=True)
166
+
167
+ for name, data in pages:
168
+ safe = Path(name).name
169
+ if not safe:
170
+ continue
171
+ (img_dir / safe).write_bytes(data)
172
+
173
+ layout_b = False
174
+ if input_structure == "b":
175
+ # 1 冊だけ: bookid フォルダを挟む(README の -s b 形式)
176
+ book = "book1"
177
+ nested = img_dir / book
178
+ nested.mkdir(parents=True)
179
+ for p in list(img_dir.iterdir()):
180
+ if p.is_file():
181
+ p.rename(nested / p.name)
182
+ input_structure_cli = "b"
183
+ layout_b = True
184
+ else:
185
+ input_structure_cli = input_structure
186
+
187
+ cmd = [
188
+ "python3",
189
+ "main.py",
190
+ "infer",
191
+ str(input_root),
192
+ str(output_root),
193
+ "-s",
194
+ input_structure_cli,
195
+ ]
196
+ if add_image_info:
197
+ cmd.append("-a")
198
+
199
+ subprocess.run(
200
+ cmd,
201
+ cwd="/root/kotenocr_cli",
202
+ env={**os.environ, "PYTHONUNBUFFERED": "1"},
203
+ check=True,
204
+ )
205
+
206
+ return _collect_outputs(str(output_root), input_root.name, layout_b)
207
+
208
+
209
+ @app.local_entrypoint()
210
+ def main(
211
+ image_path: str | None = None,
212
+ input_dir: str | None = None,
213
+ output_dir: str = "output",
214
+ structure: str = "s",
215
+ ) -> None:
216
+ """
217
+ 例:
218
+ modal run comparison/ndlkotenocr_app.py --image-path ./page.jpg
219
+ modal run comparison/ndlkotenocr_app.py --input-dir ./dataset/
220
+ """
221
+ if input_dir:
222
+ _run_batch(input_dir, output_dir, structure)
223
+ elif image_path:
224
+ _run_single(image_path, structure)
225
+ else:
226
+ print("用法:")
227
+ print(" modal run comparison/ndlkotenocr_app.py --image-path ./your.jpg")
228
+ print(" modal run comparison/ndlkotenocr_app.py --input-dir ./images/")
229
+
230
+
231
+ def _run_single(image_path: str, structure: str) -> None:
232
+ p = Path(image_path)
233
+ data = p.read_bytes()
234
+ print(f"推論中: {p.name} ({len(data)} bytes) ...")
235
+ out = infer_pages.remote([(p.name, data)], structure, add_image_info=False)
236
+ _print_results(out)
237
+
238
+
239
+ def _run_batch(input_dir: str, output_dir: str, structure: str) -> None:
240
+ src = Path(input_dir)
241
+ if not src.is_dir():
242
+ print(f"ディレクトリが見つかりません: {input_dir}")
243
+ return
244
+
245
+ files: list[tuple[str, bytes]] = []
246
+ for p in sorted(src.rglob("*")):
247
+ if p.is_file() and p.suffix.lower() in IMAGE_EXTS:
248
+ relpath = str(p.relative_to(src))
249
+ files.append((relpath, p.read_bytes()))
250
+
251
+ if not files:
252
+ print(f"画像が見つかりません: {input_dir}")
253
+ return
254
+
255
+ total_mb = sum(len(d) for _, d in files) / 1024 / 1024
256
+ print(f"{len(files)} 枚の画像を Volume にアップロード中 ({total_mb:.1f} MB) ...")
257
+ count = upload_images.remote(files, input_subdir="input")
258
+ print(f"{count} 枚アップロード完了")
259
+
260
+ print("推論中 ...")
261
+ out = infer_from_volume.remote(
262
+ input_subdir="input",
263
+ output_subdir=output_dir,
264
+ input_structure=structure,
265
+ )
266
+ _print_results(out)
267
+ print(f"\n結果は Volume 'ndl-koten-ocr-data' の /{output_dir}/ にも保存済み")
268
+ print(" modal volume get ndl-koten-ocr-data <remote-path> <local-path>")
269
+
270
+
271
+ def _print_results(out: dict[str, Any]) -> None:
272
+ texts = out.get("txt", {})
273
+ jsons = out.get("json", {})
274
+
275
+ if not texts and not jsons:
276
+ print("出力がありません。")
277
+ return
278
+
279
+ for name, text in texts.items():
280
+ print(f"--- {name} ---")
281
+ print(text[:2000] + ("..." if len(text) > 2000 else ""))
282
+
283
+ if jsons:
284
+ print("\n========== JSON 出力 ==========")
285
+ for name, obj in jsons.items():
286
+ print(f"--- {name} ---")
287
+ print(json.dumps(obj, ensure_ascii=False, indent=2)[:4000])
comparison/paddle_ocr_app.py ADDED
@@ -0,0 +1,277 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ PaddleOCR-VL-1.5 (0.9B) によるくずし字 OCR(Modal 上で実行)。
3
+
4
+ 特徴:
5
+ - 0.9B パラメータで OmniDocBench v1.5 スコア 94.5
6
+ - 109 言語対応(日本語含む)
7
+ - スキャン歪み・傾き・照明変動に強い(Real5-OmniDocBench 全5シナリオ1位)
8
+ - T4 GPU(16GB VRAM)で動作可能(低コスト)
9
+
10
+ 前提:
11
+ - Modal アカウントと認証済み
12
+ - 初回はイメージビルドに時間がかかる(〜10 分)
13
+
14
+ 単一画像:
15
+ modal run comparison/paddle_ocr_app.py --image-path ./sample.jpg
16
+
17
+ ディレクトリ一括処理:
18
+ modal run comparison/paddle_ocr_app.py --input-dir ./dataset/
19
+
20
+ 座標付き構造化出力(NDL OCR 互換 JSON):
21
+ modal run comparison/paddle_ocr_app.py --image-path ./sample.jpg --structured
22
+ """
23
+
24
+ from __future__ import annotations
25
+
26
+ import base64
27
+ import json
28
+ import re
29
+ from pathlib import Path
30
+ from typing import Any
31
+
32
+ import modal
33
+
34
+ MODEL_ID = "PaddlePaddle/PaddleOCR-VL-1.5"
35
+
36
+ app = modal.App("kuzushiji-paddle-ocr")
37
+
38
+ hf_secret = modal.Secret.from_name("huggingface-secret")
39
+
40
+
41
+ def _download_model():
42
+ from huggingface_hub import snapshot_download
43
+
44
+ snapshot_download(MODEL_ID)
45
+
46
+
47
+ paddle_image = (
48
+ modal.Image
49
+ .debian_slim(python_version="3.11")
50
+ .pip_install(
51
+ "torch>=2.5.0",
52
+ "torchvision>=0.20.0",
53
+ "transformers>=4.49.0",
54
+ "accelerate>=0.34.0",
55
+ "qwen-vl-utils>=0.0.8",
56
+ "Pillow",
57
+ "huggingface_hub",
58
+ )
59
+ .run_function(_download_model, secrets=[hf_secret])
60
+ )
61
+
62
+ IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".bmp", ".jp2"}
63
+
64
+ _MIME_MAP = {
65
+ ".png": "image/png",
66
+ ".jpg": "image/jpeg",
67
+ ".jpeg": "image/jpeg",
68
+ ".tif": "image/tiff",
69
+ ".tiff": "image/tiff",
70
+ ".bmp": "image/bmp",
71
+ }
72
+
73
+ DEFAULT_PROMPT = (
74
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。\n"
75
+ "縦書きの場合は右から左の順で、行ごとに改行して出力してください。\n"
76
+ "読めない文字は□で示してください。翻刻テキストのみを出力し、説明は不要です。"
77
+ )
78
+
79
+ STRUCTURED_PROMPT = (
80
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。\n"
81
+ "各テキスト行について、画像上のバウンディングボックス座標(ピクセル)と"
82
+ "翻刻テキストを以下の JSON 配列で出力してください。\n"
83
+ '形式: [[x1, y1, x2, y2, "翻刻テキスト"], ...]\n'
84
+ "座標は画像の左上を原点としたピクセル値です。\n"
85
+ "縦書きの場合は右の列から順に並べてください。読めない文字は□で示してください。\n"
86
+ "JSON のみを出力し、説明文は付けないでください。"
87
+ )
88
+
89
+
90
+ def _parse_structured_output(raw: str) -> list | None:
91
+ """VLM の生テキストから JSON を抽出し、NDL OCR 互換の
92
+ [[x1, y1, x2, y2, "text"], ...] 形式に正規化する。"""
93
+ match = re.search(r"\[.*\]", raw, re.DOTALL)
94
+ if not match:
95
+ return None
96
+ try:
97
+ data = json.loads(match.group())
98
+ except (json.JSONDecodeError, TypeError):
99
+ return None
100
+
101
+ result: list[list] = []
102
+ for item in data:
103
+ if isinstance(item, list) and len(item) >= 5:
104
+ result.append(item[:5])
105
+ elif isinstance(item, dict):
106
+ bbox = item.get("bbox", item.get("box", []))
107
+ text = item.get("text", item.get("content", ""))
108
+ if len(bbox) == 4:
109
+ result.append([*bbox, text])
110
+ return [result] if result else None
111
+
112
+
113
+ @app.cls(image=paddle_image, gpu="T4", timeout=60 * 60, memory=16384, secrets=[hf_secret])
114
+ class PaddleOCRInference:
115
+ @modal.enter()
116
+ def setup(self):
117
+ import torch
118
+ from transformers import AutoModelForImageTextToText, AutoProcessor
119
+
120
+ self.processor = AutoProcessor.from_pretrained(
121
+ MODEL_ID,
122
+ min_pixels=256 * 28 * 28,
123
+ max_pixels=1280 * 28 * 28,
124
+ )
125
+ self.model = AutoModelForImageTextToText.from_pretrained(
126
+ MODEL_ID,
127
+ dtype=torch.bfloat16,
128
+ device_map="auto",
129
+ )
130
+
131
+ @modal.method()
132
+ def ocr_pages(
133
+ self,
134
+ pages: list[tuple[str, bytes]],
135
+ prompt: str = DEFAULT_PROMPT,
136
+ structured: bool = False,
137
+ ) -> dict[str, Any]:
138
+ import torch
139
+ from qwen_vl_utils import process_vision_info
140
+
141
+ effective_prompt = STRUCTURED_PROMPT if structured else prompt
142
+ max_tokens = 4096 if structured else 2048
143
+
144
+ texts: dict[str, str] = {}
145
+ jsons: dict[str, Any] = {}
146
+
147
+ for name, data in pages:
148
+ b64 = base64.b64encode(data).decode()
149
+ suffix = Path(name).suffix.lower()
150
+ mime = _MIME_MAP.get(suffix, "image/jpeg")
151
+
152
+ messages = [
153
+ {
154
+ "role": "user",
155
+ "content": [
156
+ {"type": "image", "image": f"data:{mime};base64,{b64}"},
157
+ {"type": "text", "text": effective_prompt},
158
+ ],
159
+ }
160
+ ]
161
+
162
+ text_input = self.processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
163
+ image_inputs, video_inputs = process_vision_info(messages)
164
+ inputs = self.processor(
165
+ text=[text_input],
166
+ images=image_inputs,
167
+ videos=video_inputs,
168
+ padding=True,
169
+ return_tensors="pt",
170
+ ).to(self.model.device)
171
+
172
+ with torch.inference_mode():
173
+ output_ids = self.model.generate(**inputs, max_new_tokens=max_tokens)
174
+
175
+ trimmed = output_ids[0][inputs["input_ids"].shape[-1] :]
176
+ output = self.processor.decode(
177
+ trimmed,
178
+ skip_special_tokens=True,
179
+ clean_up_tokenization_spaces=False,
180
+ )
181
+
182
+ stem = Path(name).stem
183
+ texts[f"{stem}_main.txt"] = output
184
+
185
+ if structured:
186
+ parsed = _parse_structured_output(output)
187
+ if parsed is not None:
188
+ jsons[f"{stem}.json"] = parsed
189
+
190
+ return {"txt": texts, "json": jsons}
191
+
192
+
193
+ # ---------------------------------------------------------------------------
194
+ # Local entrypoint
195
+ # ---------------------------------------------------------------------------
196
+
197
+
198
+ @app.local_entrypoint()
199
+ def main(
200
+ image_path: str | None = None,
201
+ input_dir: str | None = None,
202
+ structured: bool = False,
203
+ prompt: str | None = None,
204
+ ) -> None:
205
+ """
206
+ 例:
207
+ modal run comparison/paddle_ocr_app.py --image-path ./page.jpg
208
+ modal run comparison/paddle_ocr_app.py --input-dir ./dataset/
209
+ modal run comparison/paddle_ocr_app.py --image-path ./page.jpg --structured
210
+ """
211
+ if input_dir:
212
+ _run_batch(input_dir, structured, prompt)
213
+ elif image_path:
214
+ _run_single(image_path, structured, prompt)
215
+ else:
216
+ print("用法:")
217
+ print(" modal run comparison/paddle_ocr_app.py --image-path ./your.jpg")
218
+ print(" modal run comparison/paddle_ocr_app.py --input-dir ./images/")
219
+ print(" modal run comparison/paddle_ocr_app.py --image-path ./your.jpg --structured")
220
+
221
+
222
+ def _run_single(image_path: str, structured: bool, prompt: str | None) -> None:
223
+ p = Path(image_path)
224
+ data = p.read_bytes()
225
+ print(f"推論中: {p.name} ({len(data)} bytes) [model={MODEL_ID}]")
226
+
227
+ inference = PaddleOCRInference()
228
+ kwargs: dict[str, Any] = {"structured": structured}
229
+ if prompt:
230
+ kwargs["prompt"] = prompt
231
+ out = inference.ocr_pages.remote([(p.name, data)], **kwargs)
232
+ _print_results(out)
233
+
234
+
235
+ def _run_batch(input_dir: str, structured: bool, prompt: str | None) -> None:
236
+ src = Path(input_dir)
237
+ if not src.is_dir():
238
+ print(f"ディレクトリが見つかりません: {input_dir}")
239
+ return
240
+
241
+ pages: list[tuple[str, bytes]] = []
242
+ for p in sorted(src.rglob("*")):
243
+ if p.is_file() and p.suffix.lower() in IMAGE_EXTS:
244
+ pages.append((p.name, p.read_bytes()))
245
+
246
+ if not pages:
247
+ print(f"画像が見つかりません: {input_dir}")
248
+ return
249
+
250
+ total_mb = sum(len(d) for _, d in pages) / 1024 / 1024
251
+ print(f"{len(pages)} 枚の画像を処理中 ({total_mb:.1f} MB) [model={MODEL_ID}] ...")
252
+
253
+ inference = PaddleOCRInference()
254
+ kwargs: dict[str, Any] = {"structured": structured}
255
+ if prompt:
256
+ kwargs["prompt"] = prompt
257
+ out = inference.ocr_pages.remote(pages, **kwargs)
258
+ _print_results(out)
259
+
260
+
261
+ def _print_results(out: dict[str, Any]) -> None:
262
+ texts = out.get("txt", {})
263
+ jsons = out.get("json", {})
264
+
265
+ if not texts and not jsons:
266
+ print("出力がありません。")
267
+ return
268
+
269
+ for name, text in texts.items():
270
+ print(f"--- {name} ---")
271
+ print(text[:2000] + ("..." if len(text) > 2000 else ""))
272
+
273
+ if jsons:
274
+ print("\n========== JSON 出力 ==========")
275
+ for name, obj in jsons.items():
276
+ print(f"--- {name} ---")
277
+ print(json.dumps(obj, ensure_ascii=False, indent=2)[:4000])
comparison/vision_app.py ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Gemini API を使った古典籍 OCR(ローカル実行)。
3
+
4
+ 前提:
5
+ - Gemini API キーを取得済み(https://aistudio.google.com/apikey)
6
+ - 環境変数 GEMINI_API_KEY に設定:
7
+ export GEMINI_API_KEY='your-api-key'
8
+
9
+ 単一画像:
10
+ uv run python comparison/vision_app.py --image-path ./sample.jpg
11
+
12
+ ディレクトリ一括処理:
13
+ uv run python comparison/vision_app.py --input-dir ./dataset/ --output-dir ./output/
14
+
15
+ プロンプトのカスタマイズ:
16
+ uv run python comparison/vision_app.py --image-path ./sample.jpg --prompt "この画像の文字を読んでください"
17
+ """
18
+
19
+ from __future__ import annotations
20
+
21
+ import argparse
22
+ import mimetypes
23
+ import shutil
24
+ from pathlib import Path
25
+
26
+ from google import genai
27
+ from google.genai import types
28
+
29
+ IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".bmp", ".jp2"}
30
+
31
+ DEFAULT_PROMPT = (
32
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。"
33
+ "縦書きの場合は右から左の順で、行ごとに改行して出力してください。"
34
+ "読めない文字は□で示してください。"
35
+ )
36
+
37
+ DEFAULT_MODEL = "gemini-3-flash-preview"
38
+
39
+
40
+ def _get_client() -> genai.Client:
41
+ return genai.Client()
42
+
43
+
44
+ def _get_mime_type(path: Path) -> str:
45
+ mt, _ = mimetypes.guess_type(str(path))
46
+ return mt or "image/jpeg"
47
+
48
+
49
+ def ocr_image(
50
+ client: genai.Client,
51
+ image_bytes: bytes,
52
+ mime_type: str,
53
+ prompt: str,
54
+ model: str,
55
+ ) -> str:
56
+ """1 枚の画像を Gemini で OCR する。"""
57
+ response = client.models.generate_content(
58
+ model=model,
59
+ contents=[
60
+ prompt,
61
+ types.Part.from_bytes(data=image_bytes, mime_type=mime_type),
62
+ ],
63
+ )
64
+ return response.text or ""
65
+
66
+
67
+ def run_single(
68
+ image_path: Path,
69
+ prompt: str,
70
+ model: str,
71
+ ) -> str:
72
+ """1 枚の画像を OCR してテキストを返す。"""
73
+ client = _get_client()
74
+ data = image_path.read_bytes()
75
+ mime = _get_mime_type(image_path)
76
+ print(f"OCR 中: {image_path.name} ({len(data)} bytes) [model={model}]")
77
+ return ocr_image(client, data, mime, prompt, model)
78
+
79
+
80
+ def run_directory(
81
+ input_dir: Path,
82
+ output_dir: Path,
83
+ prompt: str,
84
+ model: str,
85
+ ) -> None:
86
+ """ディレクトリ内の画像を一括 OCR し、結果をファイルに保存する。"""
87
+ image_files = sorted(p for p in input_dir.rglob("*") if p.is_file() and p.suffix.lower() in IMAGE_EXTS)
88
+
89
+ if not image_files:
90
+ print(f"画像が見つかりません: {input_dir}")
91
+ return
92
+
93
+ if output_dir.exists():
94
+ shutil.rmtree(output_dir)
95
+ output_dir.mkdir(parents=True)
96
+
97
+ client = _get_client()
98
+ print(f"{len(image_files)} 枚の画像を処理中 [model={model}] ...")
99
+
100
+ for p in image_files:
101
+ relpath = p.relative_to(input_dir)
102
+ print(f" OCR: {relpath}")
103
+
104
+ text = ocr_image(
105
+ client,
106
+ p.read_bytes(),
107
+ _get_mime_type(p),
108
+ prompt,
109
+ model,
110
+ )
111
+ (output_dir / f"{p.stem}.txt").write_text(text, encoding="utf-8")
112
+
113
+ print(f"\n完了: {len(image_files)} 枚処理済み")
114
+ print(f"出力: {output_dir}")
115
+
116
+
117
+ def main() -> None:
118
+ parser = argparse.ArgumentParser(
119
+ description="Gemini API で古典籍画像を OCR する",
120
+ )
121
+ parser.add_argument("--image-path", type=str, help="単一画像のパス")
122
+ parser.add_argument("--input-dir", type=str, help="画像ディレクトリのパス(一括処理)")
123
+ parser.add_argument("--output-dir", type=str, default="output_vision", help="出力ディレクトリ(デフォルト: output_vision)")
124
+ parser.add_argument("--prompt", type=str, default=DEFAULT_PROMPT, help="OCR プロンプト")
125
+ parser.add_argument("--model", type=str, default=DEFAULT_MODEL, help=f"Gemini モデル名(デフォルト: {DEFAULT_MODEL})")
126
+ args = parser.parse_args()
127
+
128
+ if args.input_dir:
129
+ run_directory(Path(args.input_dir), Path(args.output_dir), args.prompt, args.model)
130
+ elif args.image_path:
131
+ text = run_single(Path(args.image_path), args.prompt, args.model)
132
+ print("--- 翻刻結果 ---")
133
+ print(text)
134
+ else:
135
+ parser.print_help()
136
+
137
+
138
+ if __name__ == "__main__":
139
+ main()
pyproject.toml ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [build-system]
2
+ requires = ["setuptools>=61"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "kuzushiji-ocr"
7
+ version = "0.1.0"
8
+ description = "くずし字 OCR — Modal 上の VLM 推論と FastAPI Web UI"
9
+ readme = "README.md"
10
+ requires-python = ">=3.10"
11
+ license = { text = "Apache-2.0" }
12
+ dependencies = [
13
+ "google-genai>=1.0",
14
+ "modal>=0.62",
15
+ "fastapi>=0.115",
16
+ "uvicorn>=0.34",
17
+ "python-multipart>=0.0.20",
18
+ "Pillow>=11.0",
19
+ "jinja2>=3.1",
20
+ ]
21
+
22
+ [dependency-groups]
23
+ dev = [
24
+ "pytest>=8.3.4",
25
+ "pytest-asyncio>=1.0.0",
26
+ "ruff>=0.14.10",
27
+ ]
28
+
29
+ [tool.setuptools.packages.find]
30
+ where = ["src"]
31
+
32
+ [tool.ruff]
33
+ line-length = 160
34
+ indent-width = 4
35
+ target-version = "py310"
36
+ extend-exclude = [".venv", "build", "dist", "archive"]
37
+ src = ["src", "comparison"]
38
+
39
+ [tool.ruff.lint]
40
+ select = ["E", "F", "W", "B", "I"]
41
+ extend-select = [
42
+ "C4",
43
+ "SIM",
44
+ "ERA",
45
+ "UP",
46
+ ]
47
+ extend-ignore = ["SIM108"]
48
+
49
+ [tool.ruff.format]
50
+ quote-style = "double"
51
+ indent-style = "space"
52
+ preview = true
53
+ line-ending = "auto"
54
+ docstring-code-format = true
55
+
56
+ [tool.ruff.lint.isort]
57
+ known-first-party = ["kuzushiji_ocr"]
58
+ section-order = ["future", "standard-library", "third-party", "first-party", "local-folder"]
src/kuzushiji_ocr/__init__.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ """くずし字 OCR — Modal 上の VLM 推論と FastAPI Web UI。"""
2
+
3
+ __version__ = "0.1.0"
src/kuzushiji_ocr/templates/index.html ADDED
@@ -0,0 +1,838 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="ja">
3
+ <head>
4
+ <meta charset="UTF-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
6
+ <title>くずし字 OCR</title>
7
+ <link rel="preconnect" href="https://fonts.googleapis.com">
8
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
9
+ <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=Noto+Sans+JP:wght@400;500;600;700&display=swap" rel="stylesheet">
10
+ <style>
11
+ /* ===== Aspara Design Tokens ===== */
12
+ :root {
13
+ --color-action: #2C2520;
14
+ --color-action-hover: #1a1512;
15
+ --color-accent: #CC785C;
16
+ --color-accent-hover: #B5654A;
17
+ --color-accent-light: #E8A892;
18
+ --color-base-bg: #F5F3F0;
19
+ --color-base-border: #E6E3E0;
20
+ --color-base-surface: #FDFCFB;
21
+ --color-text-primary: #2C2520;
22
+ --color-text-secondary: #6B5F55;
23
+ --color-text-muted: #9B8F85;
24
+ --color-status-error: #C84C3C;
25
+ --color-status-success: #5A8B6F;
26
+ --color-status-warning: #D4864E;
27
+ --font-sans: "Inter", "Noto Sans JP", -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
28
+ --font-mono: "JetBrains Mono", Consolas, Monaco, monospace;
29
+ --radius: 0.5rem;
30
+ }
31
+
32
+ /* ===== Reset & Base ===== */
33
+ *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
34
+
35
+ body {
36
+ font-family: var(--font-sans);
37
+ background: var(--color-base-bg);
38
+ color: var(--color-text-primary);
39
+ line-height: 1.6;
40
+ letter-spacing: -0.01em;
41
+ min-height: 100vh;
42
+ }
43
+
44
+ /* ===== Layout ===== */
45
+ .app-header {
46
+ background: var(--color-base-surface);
47
+ border-bottom: 1px solid var(--color-base-border);
48
+ padding: 1rem 2rem;
49
+ display: flex;
50
+ align-items: center;
51
+ gap: 0.75rem;
52
+ }
53
+ .app-header h1 {
54
+ font-size: 1.25rem;
55
+ font-weight: 700;
56
+ color: var(--color-text-primary);
57
+ }
58
+ .app-header .subtitle {
59
+ font-size: 0.8rem;
60
+ color: var(--color-text-muted);
61
+ font-weight: 400;
62
+ }
63
+
64
+ .container {
65
+ max-width: 1400px;
66
+ margin: 0 auto;
67
+ padding: 2rem;
68
+ }
69
+
70
+ /* ===== Card ===== */
71
+ .card {
72
+ background: var(--color-base-surface);
73
+ border: 1px solid var(--color-base-border);
74
+ border-radius: var(--radius);
75
+ padding: 1.5rem;
76
+ transition: border-color 150ms;
77
+ }
78
+ .card:hover { border-color: var(--color-accent); }
79
+
80
+ /* ===== Upload Zone ===== */
81
+ .upload-section { margin-bottom: 1.5rem; }
82
+
83
+ .drop-zone {
84
+ border: 2px dashed var(--color-base-border);
85
+ border-radius: var(--radius);
86
+ padding: 3rem 2rem;
87
+ text-align: center;
88
+ cursor: pointer;
89
+ transition: border-color 150ms, background 150ms;
90
+ background: var(--color-base-surface);
91
+ position: relative;
92
+ }
93
+ .drop-zone:hover,
94
+ .drop-zone.dragover {
95
+ border-color: var(--color-accent);
96
+ background: rgba(204, 120, 92, 0.04);
97
+ }
98
+ .drop-zone input[type="file"] {
99
+ position: absolute;
100
+ inset: 0;
101
+ opacity: 0;
102
+ cursor: pointer;
103
+ }
104
+ .drop-zone .icon {
105
+ width: 48px;
106
+ height: 48px;
107
+ margin: 0 auto 1rem;
108
+ color: var(--color-text-muted);
109
+ }
110
+ .drop-zone .label {
111
+ font-size: 0.95rem;
112
+ color: var(--color-text-secondary);
113
+ font-weight: 500;
114
+ }
115
+ .drop-zone .hint {
116
+ font-size: 0.8rem;
117
+ color: var(--color-text-muted);
118
+ margin-top: 0.25rem;
119
+ }
120
+
121
+ .preview-area {
122
+ display: none;
123
+ margin-top: 1rem;
124
+ position: relative;
125
+ }
126
+ .preview-area.visible { display: block; }
127
+ .preview-area img {
128
+ max-height: 200px;
129
+ max-width: 100%;
130
+ border-radius: var(--radius);
131
+ border: 1px solid var(--color-base-border);
132
+ object-fit: contain;
133
+ }
134
+ .preview-info {
135
+ margin-top: 0.5rem;
136
+ font-size: 0.8rem;
137
+ color: var(--color-text-muted);
138
+ }
139
+ .preview-clear {
140
+ position: absolute;
141
+ top: 0.5rem;
142
+ right: 0.5rem;
143
+ background: var(--color-action);
144
+ color: white;
145
+ border: none;
146
+ border-radius: 50%;
147
+ width: 28px;
148
+ height: 28px;
149
+ font-size: 1rem;
150
+ cursor: pointer;
151
+ display: flex;
152
+ align-items: center;
153
+ justify-content: center;
154
+ opacity: 0.8;
155
+ transition: opacity 150ms;
156
+ }
157
+ .preview-clear:hover { opacity: 1; }
158
+
159
+ /* ===== Controls ===== */
160
+ .controls {
161
+ display: flex;
162
+ align-items: center;
163
+ gap: 2rem;
164
+ flex-wrap: wrap;
165
+ margin-bottom: 2rem;
166
+ }
167
+
168
+ .model-selector {
169
+ display: flex;
170
+ align-items: center;
171
+ gap: 1.25rem;
172
+ }
173
+ .model-selector .group-label {
174
+ font-size: 0.85rem;
175
+ font-weight: 600;
176
+ color: var(--color-text-secondary);
177
+ }
178
+ .radio-option {
179
+ display: flex;
180
+ align-items: center;
181
+ gap: 0.4rem;
182
+ cursor: pointer;
183
+ font-size: 0.85rem;
184
+ color: var(--color-text-primary);
185
+ }
186
+ .radio-option input[type="radio"] {
187
+ accent-color: var(--color-accent);
188
+ width: 16px;
189
+ height: 16px;
190
+ cursor: pointer;
191
+ }
192
+ .radio-option .tag {
193
+ font-size: 0.7rem;
194
+ padding: 0.1rem 0.4rem;
195
+ border-radius: 9999px;
196
+ font-weight: 500;
197
+ }
198
+ .tag-fast {
199
+ background: rgba(90, 139, 111, 0.12);
200
+ color: var(--color-status-success);
201
+ }
202
+ .tag-quality {
203
+ background: rgba(204, 120, 92, 0.12);
204
+ color: var(--color-accent);
205
+ }
206
+
207
+ .btn-run {
208
+ background: var(--color-accent);
209
+ color: white;
210
+ border: none;
211
+ padding: 0.6rem 1.5rem;
212
+ border-radius: var(--radius);
213
+ font-size: 0.9rem;
214
+ font-weight: 600;
215
+ cursor: pointer;
216
+ transition: background 150ms;
217
+ display: flex;
218
+ align-items: center;
219
+ gap: 0.5rem;
220
+ font-family: var(--font-sans);
221
+ }
222
+ .btn-run:hover:not(:disabled) { background: var(--color-accent-hover); }
223
+ .btn-run:disabled {
224
+ opacity: 0.5;
225
+ cursor: not-allowed;
226
+ }
227
+
228
+ /* ===== Results ===== */
229
+ .results {
230
+ display: none;
231
+ gap: 1.5rem;
232
+ }
233
+ .results.visible {
234
+ display: grid;
235
+ grid-template-columns: 1fr 1fr;
236
+ }
237
+ @media (max-width: 900px) {
238
+ .results.visible { grid-template-columns: 1fr; }
239
+ }
240
+
241
+ .result-image-panel .card { padding: 1rem; }
242
+ .result-image-panel img {
243
+ width: 100%;
244
+ border-radius: calc(var(--radius) - 4px);
245
+ cursor: zoom-in;
246
+ }
247
+
248
+ .result-text-panel { display: flex; flex-direction: column; gap: 0.75rem; }
249
+
250
+ /* ===== Accordion ===== */
251
+ .accordion {
252
+ background: var(--color-base-surface);
253
+ border: 1px solid var(--color-base-border);
254
+ border-radius: var(--radius);
255
+ overflow: hidden;
256
+ transition: border-color 150ms;
257
+ }
258
+ .accordion:hover { border-color: var(--color-accent); }
259
+
260
+ .accordion-header {
261
+ display: flex;
262
+ align-items: center;
263
+ gap: 0.5rem;
264
+ padding: 0.75rem 1rem;
265
+ cursor: pointer;
266
+ user-select: none;
267
+ background: none;
268
+ border: none;
269
+ width: 100%;
270
+ font-family: var(--font-sans);
271
+ font-size: 0.9rem;
272
+ font-weight: 600;
273
+ color: var(--color-text-primary);
274
+ text-align: left;
275
+ }
276
+ .accordion-header:hover { background: rgba(0, 0, 0, 0.02); }
277
+
278
+ .accordion-chevron {
279
+ width: 16px;
280
+ height: 16px;
281
+ transition: transform 200ms;
282
+ color: var(--color-text-muted);
283
+ flex-shrink: 0;
284
+ }
285
+ .accordion.open .accordion-chevron { transform: rotate(90deg); }
286
+
287
+ .accordion-badge {
288
+ font-size: 0.7rem;
289
+ padding: 0.1rem 0.5rem;
290
+ border-radius: 9999px;
291
+ font-weight: 500;
292
+ margin-left: auto;
293
+ }
294
+ .badge-observation { background: rgba(90, 139, 111, 0.12); color: var(--color-status-success); }
295
+ .badge-transcription { background: rgba(204, 120, 92, 0.12); color: var(--color-accent); }
296
+ .badge-review { background: rgba(44, 37, 32, 0.1); color: var(--color-text-secondary); }
297
+ .badge-bbox { background: rgba(212, 134, 78, 0.12); color: var(--color-status-warning); }
298
+
299
+ .accordion-body {
300
+ max-height: 0;
301
+ overflow: hidden;
302
+ transition: max-height 280ms ease;
303
+ }
304
+ /* 多段推論の長文(校訂・解説など)が 2000px 超で切れないよう、ビューポート基準で拡張し内部スクロール */
305
+ .accordion.open .accordion-body {
306
+ max-height: min(88vh, 32000px);
307
+ overflow-y: auto;
308
+ overflow-x: hidden;
309
+ -webkit-overflow-scrolling: touch;
310
+ }
311
+
312
+ .accordion-content {
313
+ padding: 0 1rem 1rem;
314
+ font-size: 0.85rem;
315
+ line-height: 1.8;
316
+ color: var(--color-text-primary);
317
+ white-space: pre-wrap;
318
+ word-break: break-word;
319
+ }
320
+ .accordion-content.mono {
321
+ font-family: var(--font-mono);
322
+ font-size: 0.75rem;
323
+ line-height: 1.5;
324
+ background: var(--color-base-bg);
325
+ margin: 0 1rem 1rem;
326
+ padding: 1rem;
327
+ border-radius: calc(var(--radius) - 2px);
328
+ max-height: 400px;
329
+ overflow-y: auto;
330
+ }
331
+
332
+ /* Markdown 本文(観察 / 翻刻 / 校訂・解説 など) */
333
+ .accordion-content.markdown-body {
334
+ white-space: normal;
335
+ overflow-x: auto;
336
+ }
337
+ .accordion-content.markdown-body > :first-child { margin-top: 0; }
338
+ .accordion-content.markdown-body > :last-child { margin-bottom: 0; }
339
+ .accordion-content.markdown-body h1,
340
+ .accordion-content.markdown-body h2,
341
+ .accordion-content.markdown-body h3,
342
+ .accordion-content.markdown-body h4 {
343
+ font-weight: 600;
344
+ color: var(--color-text-primary);
345
+ margin: 0.85rem 0 0.4rem;
346
+ line-height: 1.35;
347
+ }
348
+ .accordion-content.markdown-body h1 { font-size: 1.05rem; }
349
+ .accordion-content.markdown-body h2 { font-size: 1rem; }
350
+ .accordion-content.markdown-body h3 { font-size: 0.95rem; }
351
+ .accordion-content.markdown-body h4 { font-size: 0.9rem; }
352
+ .accordion-content.markdown-body p {
353
+ margin: 0.5rem 0;
354
+ }
355
+ .accordion-content.markdown-body ul,
356
+ .accordion-content.markdown-body ol {
357
+ margin: 0.45rem 0;
358
+ padding-left: 1.35rem;
359
+ }
360
+ .accordion-content.markdown-body li { margin: 0.2rem 0; }
361
+ .accordion-content.markdown-body li > p { margin: 0.25rem 0; }
362
+ .accordion-content.markdown-body blockquote {
363
+ margin: 0.6rem 0;
364
+ padding: 0.35rem 0 0.35rem 0.85rem;
365
+ border-left: 3px solid var(--color-base-border);
366
+ color: var(--color-text-secondary);
367
+ }
368
+ .accordion-content.markdown-body hr {
369
+ border: none;
370
+ border-top: 1px solid var(--color-base-border);
371
+ margin: 1rem 0;
372
+ }
373
+ .accordion-content.markdown-body a {
374
+ color: var(--color-accent);
375
+ text-decoration: underline;
376
+ text-underline-offset: 2px;
377
+ }
378
+ .accordion-content.markdown-body a:hover {
379
+ color: var(--color-accent-hover);
380
+ }
381
+ .accordion-content.markdown-body pre {
382
+ font-family: var(--font-mono);
383
+ font-size: 0.78rem;
384
+ line-height: 1.45;
385
+ background: var(--color-base-bg);
386
+ padding: 0.75rem 1rem;
387
+ border-radius: calc(var(--radius) - 2px);
388
+ overflow-x: auto;
389
+ margin: 0.55rem 0;
390
+ }
391
+ .accordion-content.markdown-body :not(pre) > code {
392
+ font-family: var(--font-mono);
393
+ font-size: 0.82em;
394
+ background: rgba(44, 37, 32, 0.07);
395
+ padding: 0.12em 0.4em;
396
+ border-radius: 4px;
397
+ }
398
+ .accordion-content.markdown-body table {
399
+ border-collapse: collapse;
400
+ width: 100%;
401
+ font-size: 0.82rem;
402
+ margin: 0.65rem 0;
403
+ }
404
+ .accordion-content.markdown-body th,
405
+ .accordion-content.markdown-body td {
406
+ border: 1px solid var(--color-base-border);
407
+ padding: 0.35rem 0.55rem;
408
+ text-align: left;
409
+ }
410
+ .accordion-content.markdown-body th {
411
+ background: rgba(44, 37, 32, 0.04);
412
+ font-weight: 600;
413
+ }
414
+
415
+ /* ===== Loading ===== */
416
+ .loading-overlay {
417
+ display: none;
418
+ position: fixed;
419
+ inset: 0;
420
+ background: rgba(245, 243, 240, 0.85);
421
+ z-index: 1000;
422
+ align-items: center;
423
+ justify-content: center;
424
+ backdrop-filter: blur(4px);
425
+ }
426
+ .loading-overlay.visible {
427
+ display: flex;
428
+ }
429
+ .loading-card {
430
+ background: var(--color-base-surface);
431
+ border: 1px solid var(--color-base-border);
432
+ border-radius: var(--radius);
433
+ padding: 2.5rem 3rem;
434
+ text-align: center;
435
+ max-width: 400px;
436
+ }
437
+ .spinner {
438
+ width: 40px;
439
+ height: 40px;
440
+ border: 3px solid var(--color-base-border);
441
+ border-top-color: var(--color-accent);
442
+ border-radius: 50%;
443
+ animation: spin 0.8s linear infinite;
444
+ margin: 0 auto 1.5rem;
445
+ }
446
+ @keyframes spin { to { transform: rotate(360deg); } }
447
+
448
+ .loading-title {
449
+ font-size: 1rem;
450
+ font-weight: 600;
451
+ color: var(--color-text-primary);
452
+ margin-bottom: 0.5rem;
453
+ }
454
+ .loading-stage {
455
+ font-size: 0.85rem;
456
+ color: var(--color-text-secondary);
457
+ margin-bottom: 0.25rem;
458
+ }
459
+ .loading-hint {
460
+ font-size: 0.75rem;
461
+ color: var(--color-text-muted);
462
+ margin-top: 0.75rem;
463
+ }
464
+
465
+ .progress-bar {
466
+ width: 100%;
467
+ height: 4px;
468
+ background: var(--color-base-border);
469
+ border-radius: 2px;
470
+ margin-top: 1rem;
471
+ overflow: hidden;
472
+ }
473
+ .progress-fill {
474
+ height: 100%;
475
+ background: var(--color-accent-light);
476
+ border-radius: 2px;
477
+ width: 0%;
478
+ animation: progress-pulse 2s ease-in-out infinite;
479
+ }
480
+ @keyframes progress-pulse {
481
+ 0%, 100% { width: 20%; margin-left: 0; }
482
+ 50% { width: 50%; margin-left: 25%; }
483
+ }
484
+
485
+ /* ===== Errors ===== */
486
+ .error-banner {
487
+ display: none;
488
+ background: rgba(200, 76, 60, 0.08);
489
+ border: 1px solid rgba(200, 76, 60, 0.25);
490
+ border-radius: var(--radius);
491
+ padding: 0.75rem 1rem;
492
+ margin-bottom: 1.5rem;
493
+ font-size: 0.85rem;
494
+ color: var(--color-status-error);
495
+ }
496
+ .error-banner.visible { display: block; }
497
+
498
+ /* ===== Model Info ===== */
499
+ .model-info {
500
+ font-size: 0.8rem;
501
+ color: var(--color-text-muted);
502
+ margin-bottom: 1rem;
503
+ }
504
+
505
+ /* ===== Lightbox ===== */
506
+ .lightbox {
507
+ display: none;
508
+ position: fixed;
509
+ inset: 0;
510
+ background: rgba(0, 0, 0, 0.8);
511
+ z-index: 2000;
512
+ align-items: center;
513
+ justify-content: center;
514
+ cursor: zoom-out;
515
+ }
516
+ .lightbox.visible { display: flex; }
517
+ .lightbox img {
518
+ max-width: 95vw;
519
+ max-height: 95vh;
520
+ object-fit: contain;
521
+ border-radius: var(--radius);
522
+ }
523
+
524
+ /* ===== Footer ===== */
525
+ .app-footer {
526
+ text-align: center;
527
+ padding: 2rem;
528
+ font-size: 0.75rem;
529
+ color: var(--color-text-muted);
530
+ }
531
+
532
+ /* ===== Accessibility ===== */
533
+ @media (prefers-reduced-motion: reduce) {
534
+ *, *::before, *::after {
535
+ animation-duration: 0.01ms !important;
536
+ transition-duration: 0.01ms !important;
537
+ }
538
+ }
539
+ </style>
540
+ </head>
541
+ <body>
542
+
543
+ <!-- Header -->
544
+ <header class="app-header">
545
+ <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/><polyline points="14 2 14 8 20 8"/><line x1="16" y1="13" x2="8" y2="13"/><line x1="16" y1="17" x2="8" y2="17"/><polyline points="10 9 9 9 8 9"/></svg>
546
+ <h1>くずし字 OCR</h1>
547
+ <span class="subtitle">Qwen2.5-VL — 多段生成パイプライン</span>
548
+ </header>
549
+
550
+ <main class="container">
551
+ <!-- Upload -->
552
+ <section class="upload-section">
553
+ <div class="card">
554
+ <div class="drop-zone" id="dropZone">
555
+ <svg class="icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="17 8 12 3 7 8"/><line x1="12" y1="3" x2="12" y2="15"/></svg>
556
+ <div class="label">画像をドラッグ&ドロップ、またはクリックして選択</div>
557
+ <div class="hint">JPG, PNG, TIFF, BMP 対応</div>
558
+ <input type="file" id="fileInput" accept="image/*">
559
+ </div>
560
+ <div class="preview-area" id="previewArea">
561
+ <img id="previewImg" alt="プレビュー">
562
+ <button class="preview-clear" id="clearBtn" title="クリア">&times;</button>
563
+ <div class="preview-info" id="previewInfo"></div>
564
+ </div>
565
+ </div>
566
+ </section>
567
+
568
+ <!-- Controls -->
569
+ <section class="controls">
570
+ <div class="model-selector">
571
+ <span class="group-label">モデル:</span>
572
+ <label class="radio-option">
573
+ <input type="radio" name="model" value="7b" checked>
574
+ Qwen2.5-VL-7B
575
+ <span class="tag tag-fast">高速</span>
576
+ </label>
577
+ <label class="radio-option">
578
+ <input type="radio" name="model" value="32b">
579
+ Qwen2.5-VL-32B
580
+ <span class="tag tag-quality">高精度</span>
581
+ </label>
582
+ </div>
583
+ <button class="btn-run" id="runBtn" disabled>
584
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><polygon points="5 3 19 12 5 21 5 3"/></svg>
585
+ OCR 実行
586
+ </button>
587
+ </section>
588
+
589
+ <!-- Error -->
590
+ <div class="error-banner" id="errorBanner"></div>
591
+
592
+ <!-- Results -->
593
+ <div class="model-info" id="modelInfo"></div>
594
+ <section class="results" id="results">
595
+ <!-- Left: annotated image -->
596
+ <div class="result-image-panel">
597
+ <div class="card">
598
+ <img id="annotatedImg" alt="重畳画像">
599
+ </div>
600
+ </div>
601
+
602
+ <!-- Right: text panels -->
603
+ <div class="result-text-panel" id="textPanels"></div>
604
+ </section>
605
+ </main>
606
+
607
+ <!-- Loading -->
608
+ <div class="loading-overlay" id="loading">
609
+ <div class="loading-card">
610
+ <div class="spinner"></div>
611
+ <div class="loading-title">OCR 処理中...</div>
612
+ <div class="loading-stage" id="loadingStage">Modal GPU 上で推論を実行しています</div>
613
+ <div class="loading-hint">構造化 OCR と多段推論を並列実行中</div>
614
+ <div class="progress-bar"><div class="progress-fill"></div></div>
615
+ </div>
616
+ </div>
617
+
618
+ <!-- Lightbox -->
619
+ <div class="lightbox" id="lightbox">
620
+ <img id="lightboxImg" alt="拡大画像">
621
+ </div>
622
+
623
+ <footer class="app-footer">
624
+ Qwen2.5-VL on Modal — くずし字 OCR Viewer
625
+ </footer>
626
+
627
+ <script src="https://cdn.jsdelivr.net/npm/marked@12.0.2/marked.min.js" crossorigin="anonymous"></script>
628
+ <script src="https://cdn.jsdelivr.net/npm/dompurify@3.1.7/dist/purify.min.js" crossorigin="anonymous"></script>
629
+ <script>
630
+ const $ = (sel) => document.querySelector(sel);
631
+ const dropZone = $('#dropZone');
632
+ const fileInput = $('#fileInput');
633
+ const previewArea = $('#previewArea');
634
+ const previewImg = $('#previewImg');
635
+ const previewInfo = $('#previewInfo');
636
+ const clearBtn = $('#clearBtn');
637
+ const runBtn = $('#runBtn');
638
+ const errorBanner = $('#errorBanner');
639
+ const results = $('#results');
640
+ const annotatedImg = $('#annotatedImg');
641
+ const textPanels = $('#textPanels');
642
+ const loading = $('#loading');
643
+ const loadingStage = $('#loadingStage');
644
+ const modelInfo = $('#modelInfo');
645
+ const lightbox = $('#lightbox');
646
+ const lightboxImg = $('#lightboxImg');
647
+
648
+ if (typeof marked !== 'undefined' && typeof marked.use === 'function') {
649
+ marked.use({ breaks: true, gfm: true });
650
+ }
651
+
652
+ let selectedFile = null;
653
+
654
+ /* --- Upload --- */
655
+
656
+ function handleFile(file) {
657
+ if (!file || !file.type.startsWith('image/')) return;
658
+ selectedFile = file;
659
+ const reader = new FileReader();
660
+ reader.onload = (e) => {
661
+ previewImg.src = e.target.result;
662
+ previewArea.classList.add('visible');
663
+ const sizeMB = (file.size / 1024 / 1024).toFixed(2);
664
+ previewInfo.textContent = `${file.name} (${sizeMB} MB)`;
665
+ };
666
+ reader.readAsDataURL(file);
667
+ runBtn.disabled = false;
668
+ dropZone.style.display = 'none';
669
+ }
670
+
671
+ function clearFile() {
672
+ selectedFile = null;
673
+ previewImg.src = '';
674
+ previewArea.classList.remove('visible');
675
+ previewInfo.textContent = '';
676
+ runBtn.disabled = true;
677
+ dropZone.style.display = '';
678
+ fileInput.value = '';
679
+ results.classList.remove('visible');
680
+ errorBanner.classList.remove('visible');
681
+ modelInfo.textContent = '';
682
+ }
683
+
684
+ fileInput.addEventListener('change', (e) => {
685
+ if (e.target.files.length) handleFile(e.target.files[0]);
686
+ });
687
+
688
+ dropZone.addEventListener('dragover', (e) => {
689
+ e.preventDefault();
690
+ dropZone.classList.add('dragover');
691
+ });
692
+ dropZone.addEventListener('dragleave', () => dropZone.classList.remove('dragover'));
693
+ dropZone.addEventListener('drop', (e) => {
694
+ e.preventDefault();
695
+ dropZone.classList.remove('dragover');
696
+ if (e.dataTransfer.files.length) handleFile(e.dataTransfer.files[0]);
697
+ });
698
+
699
+ clearBtn.addEventListener('click', clearFile);
700
+
701
+ /* --- Lightbox --- */
702
+
703
+ annotatedImg.addEventListener('click', () => {
704
+ lightboxImg.src = annotatedImg.src;
705
+ lightbox.classList.add('visible');
706
+ });
707
+ lightbox.addEventListener('click', () => lightbox.classList.remove('visible'));
708
+
709
+ /* --- Accordion builder --- */
710
+
711
+ const STAGE_META = {
712
+ '観察': { badge: 'badge-observation', label: 'Stage 1' },
713
+ '翻刻': { badge: 'badge-transcription', label: 'Stage 2' },
714
+ '校訂・解説': { badge: 'badge-review', label: 'Stage 3' },
715
+ 'bbox': { badge: 'badge-bbox', label: 'Data' },
716
+ '出力': { badge: 'badge-observation', label: 'Output' },
717
+ };
718
+
719
+ function createAccordion(title, content, open, isMono) {
720
+ const meta = STAGE_META[title] || { badge: 'badge-observation', label: '' };
721
+ const div = document.createElement('div');
722
+ div.className = 'accordion' + (open ? ' open' : '');
723
+ const bodyHtml = isMono ? escapeHtml(content) : renderMarkdown(content);
724
+ const contentClass = isMono ? 'accordion-content mono' : 'accordion-content markdown-body';
725
+ div.innerHTML = `
726
+ <button class="accordion-header">
727
+ <svg class="accordion-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="9 18 15 12 9 6"/></svg>
728
+ ${escapeHtml(title)}
729
+ <span class="accordion-badge ${meta.badge}">${escapeHtml(meta.label)}</span>
730
+ </button>
731
+ <div class="accordion-body">
732
+ <div class="${contentClass}">${bodyHtml}</div>
733
+ </div>
734
+ `;
735
+ div.querySelector('.accordion-header').addEventListener('click', () => {
736
+ div.classList.toggle('open');
737
+ });
738
+ return div;
739
+ }
740
+
741
+ function escapeHtml(str) {
742
+ const d = document.createElement('div');
743
+ d.textContent = str == null ? '' : String(str);
744
+ return d.innerHTML;
745
+ }
746
+
747
+ function renderMarkdown(str) {
748
+ const raw = str == null ? '' : String(str);
749
+ if (typeof marked === 'undefined' || typeof DOMPurify === 'undefined') {
750
+ return escapeHtml(raw);
751
+ }
752
+ try {
753
+ const html = marked.parse(raw);
754
+ return DOMPurify.sanitize(html);
755
+ } catch (e) {
756
+ return escapeHtml(raw);
757
+ }
758
+ }
759
+
760
+ /* --- Run OCR --- */
761
+
762
+ runBtn.addEventListener('click', async () => {
763
+ if (!selectedFile) return;
764
+ const model = document.querySelector('input[name="model"]:checked').value;
765
+
766
+ errorBanner.classList.remove('visible');
767
+ results.classList.remove('visible');
768
+ modelInfo.textContent = '';
769
+ loading.classList.add('visible');
770
+ runBtn.disabled = true;
771
+
772
+ const stages = ['構造化 OCR + 多段推論を並列実行中...', 'Modal GPU 上で推論を実行しています'];
773
+ let si = 0;
774
+ const stageInterval = setInterval(() => {
775
+ si = (si + 1) % stages.length;
776
+ loadingStage.textContent = stages[si];
777
+ }, 3000);
778
+
779
+ try {
780
+ const formData = new FormData();
781
+ formData.append('image', selectedFile);
782
+ formData.append('model', model);
783
+
784
+ const resp = await fetch('/api/ocr', { method: 'POST', body: formData });
785
+ if (!resp.ok) {
786
+ const err = await resp.json().catch(() => ({ detail: resp.statusText }));
787
+ throw new Error(err.detail || `HTTP ${resp.status}`);
788
+ }
789
+
790
+ const data = await resp.json();
791
+
792
+ /* Errors */
793
+ if (data.errors && data.errors.length) {
794
+ errorBanner.textContent = data.errors.join('\n');
795
+ errorBanner.classList.add('visible');
796
+ }
797
+
798
+ /* Model info */
799
+ modelInfo.textContent = `使用モデル: ${data.model_used}`;
800
+
801
+ /* Annotated image */
802
+ annotatedImg.src = `data:image/png;base64,${data.annotated_image}`;
803
+
804
+ /* Text panels */
805
+ textPanels.innerHTML = '';
806
+ const stageOrder = ['観察', '翻刻', '校訂・解説'];
807
+ const renderedStages = new Set();
808
+
809
+ for (const name of stageOrder) {
810
+ if (data.stages[name]) {
811
+ textPanels.appendChild(createAccordion(name, data.stages[name], true, false));
812
+ renderedStages.add(name);
813
+ }
814
+ }
815
+ for (const [name, text] of Object.entries(data.stages)) {
816
+ if (!renderedStages.has(name)) {
817
+ textPanels.appendChild(createAccordion(name, text, true, false));
818
+ }
819
+ }
820
+
821
+ if (data.bboxes && data.bboxes.length) {
822
+ const bboxJson = JSON.stringify(data.bboxes, null, 2);
823
+ textPanels.appendChild(createAccordion('bbox', bboxJson, false, true));
824
+ }
825
+
826
+ results.classList.add('visible');
827
+ } catch (e) {
828
+ errorBanner.textContent = `エラー: ${e.message}`;
829
+ errorBanner.classList.add('visible');
830
+ } finally {
831
+ clearInterval(stageInterval);
832
+ loading.classList.remove('visible');
833
+ runBtn.disabled = !selectedFile;
834
+ }
835
+ });
836
+ </script>
837
+ </body>
838
+ </html>
src/kuzushiji_ocr/vlm_app.py ADDED
@@ -0,0 +1,638 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Qwen2.5-VL によるくずし字 OCR(Modal 上で実行)。
3
+ --model 7b / 32b でモデルサイズを選択可能。
4
+
5
+ 前提:
6
+ - Modal アカウントと `modal token new` などで認証済み
7
+ - 初回はイメージビルドにモデルダウンロード含め時間がかかる
8
+
9
+ 単一画像 (7B, 高速・低コスト):
10
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./sample.jpg --model 7b
11
+
12
+ 単一画像 (32B, 高精度):
13
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./sample.jpg --model 32b
14
+
15
+ 多段推論(専門家の思考プロセスで翻刻+解説):
16
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./sample.jpg --deep-think
17
+
18
+ ディレクトリ一括処理:
19
+ modal run src/kuzushiji_ocr/vlm_app.py --input-dir ./dataset/
20
+
21
+ 座標付き構造化出力(NDL OCR 互換 JSON):
22
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./sample.jpg --structured
23
+ """
24
+
25
+ from __future__ import annotations
26
+
27
+ import base64
28
+ import json
29
+ import re
30
+ from pathlib import Path
31
+ from typing import Any
32
+
33
+ import modal
34
+
35
+ # ---------------------------------------------------------------------------
36
+ # モデル構成: 7B / 32B
37
+ # ---------------------------------------------------------------------------
38
+
39
+ MODELS: dict[str, dict[str, Any]] = {
40
+ "7b": {
41
+ "id": "Qwen/Qwen2.5-VL-7B-Instruct",
42
+ "gpu": "A10G",
43
+ "memory": 32768,
44
+ "min_pixels": 256 * 28 * 28,
45
+ "max_pixels": 1280 * 28 * 28,
46
+ },
47
+ "32b": {
48
+ "id": "Qwen/Qwen2.5-VL-32B-Instruct",
49
+ "gpu": "A100-80GB:1",
50
+ "memory": 65536,
51
+ "min_pixels": 256 * 28 * 28,
52
+ "max_pixels": 768 * 28 * 28,
53
+ },
54
+ }
55
+
56
+ app = modal.App("kuzushiji-vlm-ocr")
57
+
58
+ hf_secret = modal.Secret.from_name("huggingface-secret")
59
+
60
+ model_cache = modal.Volume.from_name("kuzushiji-vlm-models", create_if_missing=True)
61
+ CACHE_DIR = "/model-cache"
62
+
63
+ vlm_image = modal.Image.debian_slim(python_version="3.11").pip_install(
64
+ "torch>=2.5.0",
65
+ "torchvision>=0.20.0",
66
+ "transformers>=4.49.0",
67
+ "accelerate>=0.34.0",
68
+ "qwen-vl-utils>=0.0.8",
69
+ "Pillow",
70
+ "huggingface_hub",
71
+ )
72
+
73
+ IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".bmp", ".jp2"}
74
+
75
+ _MIME_MAP = {
76
+ ".png": "image/png",
77
+ ".jpg": "image/jpeg",
78
+ ".jpeg": "image/jpeg",
79
+ ".tif": "image/tiff",
80
+ ".tiff": "image/tiff",
81
+ ".bmp": "image/bmp",
82
+ }
83
+
84
+ DEFAULT_PROMPT = (
85
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。\n"
86
+ "縦書きの場合は右から左の順で、行ごとに改行して出力してください。\n"
87
+ "読めない文字は□で示してください。翻刻テキストと説明を出力してください。"
88
+ "画像全体として何が写っているかも簡潔に説明してください。"
89
+ )
90
+
91
+ STRUCTURED_PROMPT = (
92
+ "この古典籍(くずし字)の画像に書かれている文字をすべて翻刻してください。\n"
93
+ "あわせて、文字以外の目立つ物体(挿絵・図・表・欄外の絵、蔵書印・印章、欄枠・見出し枠、"
94
+ "写真・貼り付け、表紙装飾など)も検出してください。\n"
95
+ "各要素について、画像上のバウンディングボックス座標(ピクセル)とラベルを"
96
+ "次の JSON 配列で出力してください。\n"
97
+ '- テキスト行: [x1, y1, x2, y2, "翻刻テキスト"]\n'
98
+ '- 文字以外の物体: [x1, y1, x2, y2, "簡潔な物体名・説明", "object"]\n'
99
+ "(6 番目の要素は必ず英語小文字の object とし、テキスト行には付けないこと)\n"
100
+ "座標は画像の左上を原点としたピクセル値です(VLM が入力に用いる解像度上の値でよい)。\n"
101
+ "縦書きテキストは右の列から順に並べてください。読めない文字は□で示してください。\n"
102
+ "JSON のみを出力し、説明文は付けないでください。"
103
+ )
104
+
105
+ # ---------------------------------------------------------------------------
106
+ # 多段推論 — 多段生成パイプラインで専門家の思考プロセスを再現
107
+ #
108
+ # Stage 1(観察)— 画像を見て文書の種類・時代・書体・レイアウトを分析
109
+ # Stage 2(翻刻)— Stage 1 の分析を踏まえて丁寧に翻刻
110
+ # Stage 3(校訂・解説)— Stage 1+2 を踏まえて校訂し、内容を解説
111
+ #
112
+ # 各ステージの出力は会話履歴に追加され、次のステージのコンテキストになる。
113
+ # ---------------------------------------------------------------------------
114
+
115
+ DEEP_THINK_SYSTEM = """\
116
+ あなたは日本の古典籍・くずし字を専門とする文献学者です。\
117
+ 数十年にわたり和古書・写本・版本を読み解いてきた経験を持ち、\
118
+ 書体の変遷、異体字、変体仮名、合字、踊り字に精通しています。"""
119
+
120
+ DEEP_THINK_STAGES: list[tuple[str, str, int]] = [
121
+ # 各要素は ステージ名・プロンプト・max_new_tokens のタプル
122
+ (
123
+ "観察",
124
+ """\
125
+ この画像を、くずし字の専門家として丁寧に観察・分析してください。
126
+ 以下の点を報告してください。
127
+
128
+ 1. 物体と文書の種類: 文字が書かれている物体は何か(紙・木箱・器・掛軸・石碑・屏風・扇・布など)を特定し、\
129
+ 文書としての種類(写本/版本/書状/日記/物語/公文書/商品ラベル/銘・落款など)を推定する
130
+ 2. 素材と保存状態: 素材の特徴(料紙・木材・陶磁器・漆器など)、装丁、損傷・虫食いの有無
131
+ 3. 時代の推定: 書風・紙質・印章・内容から推定される年代
132
+ 4. 挿絵・図・印: 挿絵、蔵書印、書き入れなどの有無と内容
133
+ 5. 書体の特徴: 楷書寄り/草書寄り/行書、筆の運び方の癖
134
+ 6. 変体仮名・異体字: 使われている変体仮名や異体字の傾向
135
+ 7. レイアウト: 縦書き行数、行間、丁付け、柱、振り仮名の有無
136
+
137
+ まだ翻刻はしないでください。観察と分析のみを行ってください。""",
138
+ 2048,
139
+ ),
140
+ (
141
+ "翻刻",
142
+ """\
143
+ 先ほどの観察結果を踏まえて、画像中のくずし字を翻刻してください。
144
+
145
+ - 右から左、上から下の順で一行ずつ翻刻する
146
+ - 読めない文字は□で示す
147
+ - 推定できる読みは( )で補う
148
+ - 変体仮名は現行のひらがなに直し、原字が特定できる場合は注記する
149
+ (例: 「は」←「者」)
150
+ - 踊り字(ゝ、〻、く等)は展開して示す
151
+ - 行番号を付けて出力する
152
+
153
+ 翻刻テキストのみを出力してください。解説はまだ不要です。""",
154
+ 4096,
155
+ ),
156
+ (
157
+ "校訂・解説",
158
+ """\
159
+ 観察結果と翻刻テキストを踏まえて、以下を行ってください。
160
+
161
+ ## 校訂
162
+ - 文脈・文法から翻刻の読みが正しいか検証する
163
+ - □の箇所を前後の文脈から埋められないか再検討する
164
+ - 明らかな誤写・衍字・脱字があれば指摘する
165
+ - 修正がある場合は、修正後の翻刻テキストを改めて出力する
166
+
167
+ ## 内容の解説
168
+ - 翻刻した文章全体の現代語訳または要約
169
+ - 文書の歴史的・文学的位置づけ(わかる範囲で)
170
+ - 特筆すべき語彙・表現・文化的背景の注釈
171
+
172
+ 分量は読みやすい長さに抑え、解説は要点を箇条書きで簡潔に。冗長な前置きや繰り返しは避けてください。""",
173
+ 4096,
174
+ ),
175
+ ]
176
+
177
+
178
+ def _strip_markdown_code_fence(raw: str) -> str:
179
+ """先頭の ``` または ```json …(改行なしで `[` が続く場合を含む)を取り除き、閉じ ``` があればそこまで。"""
180
+ t = raw.strip()
181
+ if not t.startswith("```"):
182
+ return t
183
+ i = 3
184
+ while i < len(t) and t[i] in " \t":
185
+ i += 1
186
+ while i < len(t) and (t[i].isalpha() or t[i] == "-"):
187
+ i += 1
188
+ while i < len(t) and t[i] in " \t":
189
+ i += 1
190
+ if i < len(t) and t[i] == "\n":
191
+ i += 1
192
+ rest = t[i:]
193
+ end = rest.rfind("```")
194
+ if end != -1:
195
+ rest = rest[:end]
196
+ return rest.strip()
197
+
198
+
199
+ def _extract_json_array_source(raw: str) -> str | None:
200
+ """```json ... ``` 内または先頭の JSON 配列 `[...]` を取り出す。
201
+
202
+ `` ```json [`` のようにフェンスと `[` が同一行の場合も扱う。
203
+ """
204
+ t = raw.strip()
205
+ if "```" in t:
206
+ t = _strip_markdown_code_fence(t[t.find("```") :])
207
+ t = t.strip()
208
+ if t.startswith("["):
209
+ return t
210
+ m = re.search(r"\[\s*\[", t)
211
+ if m:
212
+ return t[m.start() :]
213
+ m = re.search(r"\[", t)
214
+ return t[m.start() :] if m else None
215
+
216
+
217
+ # 完成した 1 行 bbox のみマッチ(JSON が途中で切れても、閉じ `]` まで取れた行は拾える)
218
+ _BBOX_LINE_RE = re.compile(
219
+ r"\[\s*(-?\d+(?:\.\d+)?)\s*,\s*(-?\d+(?:\.\d+)?)\s*,\s*"
220
+ r"(-?\d+(?:\.\d+)?)\s*,\s*(-?\d+(?:\.\d+)?)\s*,\s*"
221
+ r'"((?:\\.|[^"\\])*)"\s*'
222
+ r'(?:,\s*"((?:\\.|[^"\\])*)"\s*)?'
223
+ r"\]",
224
+ )
225
+
226
+
227
+ def parse_bbox_rows_from_text(raw: str) -> list[list]:
228
+ """json.loads に失敗した VLM 生テキストから、完成した ``[x1,y1,x2,y2,"label"]`` 行だけを抽出する。"""
229
+ out: list[list] = []
230
+ for m in _BBOX_LINE_RE.finditer(raw or ""):
231
+ row: list = [
232
+ float(m.group(1)),
233
+ float(m.group(2)),
234
+ float(m.group(3)),
235
+ float(m.group(4)),
236
+ m.group(5),
237
+ ]
238
+ extra = m.group(6)
239
+ if extra is not None and extra.lower() in ("object", "obj"):
240
+ row.append("object")
241
+ out.append(row)
242
+ return out
243
+
244
+
245
+ def _looks_like_flat_bbox_row(item: Any) -> bool:
246
+ """先頭4要素が数値座標の bbox 1 行か(行のリストと区別する)。"""
247
+ if not isinstance(item, list) or len(item) < 5:
248
+ return False
249
+ for i in range(4):
250
+ x = item[i]
251
+ if isinstance(x, bool) or not isinstance(x, (int, float)):
252
+ return False
253
+ return True
254
+
255
+
256
+ def _normalize_structured_item(item: Any) -> list | None:
257
+ """VLM の検出1件を ``[x1,y1,x2,y2,label]`` または 6 要素(末尾 ``object``)に統一。"""
258
+ if isinstance(item, list) and len(item) >= 5 and _looks_like_flat_bbox_row(item):
259
+ row = list(item[:6])
260
+ if len(row) == 5:
261
+ return row
262
+ if len(row) >= 6:
263
+ tag = str(row[5]).lower()
264
+ if tag in ("object", "obj"):
265
+ return [*row[:5], "object"]
266
+ return row[:5]
267
+ if isinstance(item, dict):
268
+ inner = item.get("bbox_2d") or item.get("bbox") or item.get("box")
269
+ if isinstance(inner, (list, tuple)) and len(inner) >= 5:
270
+ x1, y1, x2, y2 = inner[0], inner[1], inner[2], inner[3]
271
+ label = str(inner[4])
272
+ obj_hint = str(item.get("object", item.get("type", item.get("category", "")))).lower()
273
+ if len(inner) >= 6:
274
+ six = str(inner[5]).lower()
275
+ if six in ("object", "obj"):
276
+ return [x1, y1, x2, y2, label, "object"]
277
+ if obj_hint in ("object", "obj", "物体"):
278
+ return [x1, y1, x2, y2, label, "object"]
279
+ return [x1, y1, x2, y2, label]
280
+ if isinstance(inner, (list, tuple)) and len(inner) == 4:
281
+ bbox = list(inner)
282
+ text = item.get("text", item.get("content", item.get("label", "")))
283
+ typ = str(item.get("type", item.get("category", item.get("object", "text")))).lower()
284
+ if typ in ("object", "obj", "物体"):
285
+ return [*bbox, text, "object"]
286
+ return [*bbox, text]
287
+ return None
288
+
289
+
290
+ def _parse_structured_output(raw: str) -> list | None:
291
+ """VLM の生テキストから JSON を抽出し、
292
+ [[x1, y1, x2, y2, "text"] または [x1, y1, x2, y2, "label", "object"], ...]
293
+ 形式に正規化する。
294
+
295
+ Qwen 系の ``{"bbox_2d": [x1,y1,x2,y2,"ラベル"], "object": "テキスト行"}`` 形式にも対応する。
296
+
297
+ json.loads が失敗する(コードフェンス付き JSON・出力途中切れなど)場合は、完成行のみ正規表現で抽出する。
298
+ """
299
+ src = _extract_json_array_source(raw)
300
+ result: list[list] = []
301
+
302
+ if src:
303
+ try:
304
+ data = json.loads(src)
305
+ except (json.JSONDecodeError, TypeError):
306
+ data = None
307
+ else:
308
+ if isinstance(data, list):
309
+ for item in data:
310
+ row = _normalize_structured_item(item)
311
+ if row:
312
+ result.append(row)
313
+ continue
314
+ if isinstance(item, list):
315
+ for sub in item:
316
+ if not isinstance(sub, list):
317
+ continue
318
+ row = _normalize_structured_item(sub)
319
+ if row:
320
+ result.append(row)
321
+
322
+ if result:
323
+ return [result]
324
+
325
+ rows = parse_bbox_rows_from_text(raw)
326
+ return [rows] if rows else None
327
+
328
+
329
+ # ---------------------------------------------------------------------------
330
+ # 共通推論ロジック(モジュールレベル関数)
331
+ # ---------------------------------------------------------------------------
332
+
333
+
334
+ def _generate(processor: Any, model: Any, messages: list[dict[str, Any]], max_new_tokens: int) -> str:
335
+ """メッセージから1回の生成を行い、テキストを返す。"""
336
+ import torch
337
+ from qwen_vl_utils import process_vision_info
338
+
339
+ text_input = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
340
+ image_inputs, video_inputs = process_vision_info(messages)
341
+ inputs = processor(
342
+ text=[text_input],
343
+ images=image_inputs,
344
+ videos=video_inputs,
345
+ padding=True,
346
+ return_tensors="pt",
347
+ ).to(model.device)
348
+
349
+ with torch.inference_mode():
350
+ output_ids = model.generate(**inputs, max_new_tokens=max_new_tokens)
351
+
352
+ trimmed = output_ids[0][inputs.input_ids.shape[1] :]
353
+ return processor.decode(trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)
354
+
355
+
356
+ def _deep_think(processor: Any, model: Any, image_uri: str) -> dict[str, str]:
357
+ """多段生成パイプライン。各ステージを独立した1ターン生成で実行。"""
358
+ import time
359
+
360
+ stage_outputs: dict[str, str] = {}
361
+ context_so_far = ""
362
+
363
+ for i, (stage_name, prompt, max_tokens) in enumerate(DEEP_THINK_STAGES):
364
+ n = len(DEEP_THINK_STAGES)
365
+ print(f"[多段推論] Stage {i + 1}/{n}: {stage_name} ...")
366
+ t0 = time.time()
367
+
368
+ user_text = ""
369
+ if context_so_far:
370
+ user_text += f"これまでの分析結果:\n---\n{context_so_far}---\n\n"
371
+ user_text += prompt
372
+
373
+ messages: list[dict[str, Any]] = [
374
+ {"role": "system", "content": DEEP_THINK_SYSTEM},
375
+ {
376
+ "role": "user",
377
+ "content": [
378
+ {"type": "image", "image": image_uri},
379
+ {"type": "text", "text": user_text},
380
+ ],
381
+ },
382
+ ]
383
+
384
+ output = _generate(processor, model, messages, max_tokens)
385
+ stage_outputs[stage_name] = output
386
+
387
+ elapsed = time.time() - t0
388
+ print(f"[多段推論] Stage {i + 1}/{n}: {stage_name} 完了 ({elapsed:.1f}s)")
389
+
390
+ context_so_far += f"## {stage_name}\n{output}\n\n"
391
+
392
+ return stage_outputs
393
+
394
+
395
+ def _run_ocr(
396
+ processor: Any,
397
+ model: Any,
398
+ pages: list[tuple[str, bytes]],
399
+ prompt: str,
400
+ structured: bool,
401
+ deep_think: bool,
402
+ ) -> dict[str, Any]:
403
+ """OCR 推論の本体。7B / 32B 共通。"""
404
+ texts: dict[str, str] = {}
405
+ jsons: dict[str, Any] = {}
406
+
407
+ for name, data in pages:
408
+ b64 = base64.b64encode(data).decode()
409
+ suffix = Path(name).suffix.lower()
410
+ mime = _MIME_MAP.get(suffix, "image/jpeg")
411
+ image_uri = f"data:{mime};base64,{b64}"
412
+
413
+ if deep_think:
414
+ stem = Path(name).stem
415
+ stage_outputs = _deep_think(processor, model, image_uri)
416
+ parts = []
417
+ for stage_name, output in stage_outputs.items():
418
+ parts.append(f"{'=' * 20} {stage_name} {'=' * 20}")
419
+ parts.append(output)
420
+ parts.append("")
421
+ texts[f"{stem}_main.txt"] = "\n".join(parts)
422
+ continue
423
+
424
+ effective_prompt = STRUCTURED_PROMPT if structured else prompt
425
+ max_tokens = 8192 if structured else 2048
426
+
427
+ messages: list[dict[str, Any]] = [
428
+ {
429
+ "role": "user",
430
+ "content": [
431
+ {"type": "image", "image": image_uri},
432
+ {"type": "text", "text": effective_prompt},
433
+ ],
434
+ }
435
+ ]
436
+
437
+ output = _generate(processor, model, messages, max_tokens)
438
+ stem = Path(name).stem
439
+ texts[f"{stem}_main.txt"] = output
440
+
441
+ if structured:
442
+ parsed = _parse_structured_output(output)
443
+ if parsed is not None:
444
+ jsons[f"{stem}.json"] = parsed
445
+
446
+ return {"txt": texts, "json": jsons}
447
+
448
+
449
+ # ---------------------------------------------------------------------------
450
+ # Modal クラス: 7B / 32B
451
+ # ---------------------------------------------------------------------------
452
+
453
+
454
+ def _setup_model(model_key: str) -> tuple[Any, Any]:
455
+ """Volume 上のキャッシュからモデルを読み込む。未キャッシュなら自動DL。"""
456
+ import os
457
+ import time
458
+
459
+ import torch
460
+ from huggingface_hub import snapshot_download
461
+ from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
462
+
463
+ os.environ["HF_HOME"] = CACHE_DIR
464
+ cfg = MODELS[model_key]
465
+
466
+ print(f"モデル準備中: {cfg['id']} ...")
467
+ t0 = time.time()
468
+ snapshot_download(cfg["id"])
469
+ model_cache.commit()
470
+ dl = time.time() - t0
471
+ print(f"モデル準備完了 ({dl:.1f}s)")
472
+
473
+ processor = AutoProcessor.from_pretrained(
474
+ cfg["id"],
475
+ min_pixels=cfg["min_pixels"],
476
+ max_pixels=cfg["max_pixels"],
477
+ )
478
+ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
479
+ cfg["id"],
480
+ torch_dtype=torch.bfloat16,
481
+ device_map="auto",
482
+ )
483
+ return processor, model
484
+
485
+
486
+ @app.cls(
487
+ image=vlm_image,
488
+ gpu="A10G",
489
+ timeout=60 * 60,
490
+ memory=32768,
491
+ secrets=[hf_secret],
492
+ volumes={CACHE_DIR: model_cache},
493
+ )
494
+ class VLMInference7B:
495
+ @modal.enter()
496
+ def setup(self):
497
+ self.processor, self.model = _setup_model("7b")
498
+
499
+ @modal.method()
500
+ def ocr_pages(
501
+ self,
502
+ pages: list[tuple[str, bytes]],
503
+ prompt: str = DEFAULT_PROMPT,
504
+ structured: bool = False,
505
+ deep_think: bool = False,
506
+ ) -> dict[str, Any]:
507
+ return _run_ocr(self.processor, self.model, pages, prompt, structured, deep_think)
508
+
509
+
510
+ @app.cls(
511
+ image=vlm_image,
512
+ gpu="A100-80GB:1",
513
+ timeout=60 * 60,
514
+ memory=65536,
515
+ secrets=[hf_secret],
516
+ volumes={CACHE_DIR: model_cache},
517
+ )
518
+ class VLMInference32B:
519
+ @modal.enter()
520
+ def setup(self):
521
+ self.processor, self.model = _setup_model("32b")
522
+
523
+ @modal.method()
524
+ def ocr_pages(
525
+ self,
526
+ pages: list[tuple[str, bytes]],
527
+ prompt: str = DEFAULT_PROMPT,
528
+ structured: bool = False,
529
+ deep_think: bool = False,
530
+ ) -> dict[str, Any]:
531
+ return _run_ocr(self.processor, self.model, pages, prompt, structured, deep_think)
532
+
533
+
534
+ # ---------------------------------------------------------------------------
535
+ # Local entrypoint
536
+ # ---------------------------------------------------------------------------
537
+
538
+
539
+ def _get_inference(model: str) -> VLMInference7B | VLMInference32B:
540
+ if model == "32b":
541
+ return VLMInference32B()
542
+ return VLMInference7B()
543
+
544
+
545
+ @app.local_entrypoint()
546
+ def main(
547
+ image_path: str | None = None,
548
+ input_dir: str | None = None,
549
+ model: str = "7b",
550
+ structured: bool = False,
551
+ deep_think: bool = False,
552
+ prompt: str | None = None,
553
+ ) -> None:
554
+ """
555
+ 例:
556
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./page.jpg
557
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./page.jpg --model 32b
558
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./page.jpg --deep-think
559
+ modal run src/kuzushiji_ocr/vlm_app.py --input-dir ./dataset/
560
+ modal run src/kuzushiji_ocr/vlm_app.py --image-path ./page.jpg --structured
561
+ """
562
+ if model not in MODELS:
563
+ print(f"不明なモデル: {model} (選択肢: {', '.join(MODELS)})")
564
+ return
565
+
566
+ if input_dir:
567
+ _run_batch(input_dir, model, structured, deep_think, prompt)
568
+ elif image_path:
569
+ _run_single(image_path, model, structured, deep_think, prompt)
570
+ else:
571
+ print("用法:")
572
+ print(" modal run src/kuzushiji_ocr/vlm_app.py --image-path ./your.jpg")
573
+ print(" modal run src/kuzushiji_ocr/vlm_app.py --image-path ./your.jpg --model 32b")
574
+ print(" modal run src/kuzushiji_ocr/vlm_app.py --image-path ./your.jpg --deep-think")
575
+ print(" modal run src/kuzushiji_ocr/vlm_app.py --input-dir ./images/")
576
+ print(" modal run src/kuzushiji_ocr/vlm_app.py --image-path ./your.jpg --structured")
577
+
578
+
579
+ def _run_single(image_path: str, model: str, structured: bool, deep_think: bool, prompt: str | None) -> None:
580
+ p = Path(image_path)
581
+ data = p.read_bytes()
582
+ model_id = MODELS[model]["id"]
583
+ mode = "多段推論" if deep_think else "structured" if structured else "standard"
584
+ print(f"推論中: {p.name} ({len(data)} bytes) [model={model_id}] [mode={mode}]")
585
+
586
+ inference = _get_inference(model)
587
+ kwargs: dict[str, Any] = {"structured": structured, "deep_think": deep_think}
588
+ if prompt:
589
+ kwargs["prompt"] = prompt
590
+ out = inference.ocr_pages.remote([(p.name, data)], **kwargs)
591
+ _print_results(out)
592
+
593
+
594
+ def _run_batch(input_dir: str, model: str, structured: bool, deep_think: bool, prompt: str | None) -> None:
595
+ src = Path(input_dir)
596
+ if not src.is_dir():
597
+ print(f"ディレクトリが見つかりません: {input_dir}")
598
+ return
599
+
600
+ pages: list[tuple[str, bytes]] = []
601
+ for p in sorted(src.rglob("*")):
602
+ if p.is_file() and p.suffix.lower() in IMAGE_EXTS:
603
+ pages.append((p.name, p.read_bytes()))
604
+
605
+ if not pages:
606
+ print(f"画像が見つかりません: {input_dir}")
607
+ return
608
+
609
+ total_mb = sum(len(d) for _, d in pages) / 1024 / 1024
610
+ model_id = MODELS[model]["id"]
611
+ mode = "多段推論" if deep_think else "structured" if structured else "standard"
612
+ print(f"{len(pages)} 枚の画像を処理中 ({total_mb:.1f} MB) [model={model_id}] [mode={mode}] ...")
613
+
614
+ inference = _get_inference(model)
615
+ kwargs: dict[str, Any] = {"structured": structured, "deep_think": deep_think}
616
+ if prompt:
617
+ kwargs["prompt"] = prompt
618
+ out = inference.ocr_pages.remote(pages, **kwargs)
619
+ _print_results(out)
620
+
621
+
622
+ def _print_results(out: dict[str, Any]) -> None:
623
+ texts = out.get("txt", {})
624
+ jsons = out.get("json", {})
625
+
626
+ if not texts and not jsons:
627
+ print("出力がありません。")
628
+ return
629
+
630
+ for name, text in texts.items():
631
+ print(f"--- {name} ---")
632
+ print(text)
633
+
634
+ if jsons:
635
+ print("\n========== JSON 出力 ==========")
636
+ for name, obj in jsons.items():
637
+ print(f"--- {name} ---")
638
+ print(json.dumps(obj, ensure_ascii=False, indent=2)[:4000])
src/kuzushiji_ocr/web.py ADDED
@@ -0,0 +1,629 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ くずし字 OCR Web アプリケーション。
3
+
4
+ Modal 上の Qwen2.5-VL (7B/32B) で OCR を実行し、
5
+ bounding box 重畳画像と多段生成テキストを返す。
6
+
7
+ 起動:
8
+ uv run uvicorn kuzushiji_ocr.web:app --reload --port 8000
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import ast
14
+ import asyncio
15
+ import base64
16
+ import io
17
+ import json
18
+ import logging
19
+ import math
20
+ import re
21
+ from concurrent.futures import ThreadPoolExecutor
22
+ from pathlib import Path
23
+ from typing import Any
24
+
25
+ import modal
26
+ from fastapi import FastAPI, File, Form, HTTPException, UploadFile
27
+ from fastapi.responses import HTMLResponse
28
+ from PIL import Image, ImageDraw, ImageFont, ImageOps
29
+
30
+ from kuzushiji_ocr.vlm_app import (
31
+ MODELS,
32
+ _looks_like_flat_bbox_row,
33
+ _normalize_structured_item,
34
+ _parse_structured_output,
35
+ parse_bbox_rows_from_text,
36
+ )
37
+
38
+ logging.basicConfig(level=logging.INFO, format="%(levelname)s [%(name)s] %(message)s")
39
+ logger = logging.getLogger("kuzushiji-ocr")
40
+
41
+ APP_DIR = Path(__file__).resolve().parent
42
+ TEMPLATES_DIR = APP_DIR / "templates"
43
+
44
+ app = FastAPI(title="くずし字 OCR")
45
+ _executor = ThreadPoolExecutor(max_workers=4)
46
+
47
+ # ---------------------------------------------------------------------------
48
+ # フォント解決
49
+ # ---------------------------------------------------------------------------
50
+
51
+ _FONT_SEARCH_PATHS = [
52
+ "/System/Library/Fonts/ヒラギノ角ゴシック W3.ttc",
53
+ "/System/Library/Fonts/Supplemental/Arial Unicode.ttf",
54
+ "/Library/Fonts/Arial Unicode.ttf",
55
+ "/usr/share/fonts/truetype/noto/NotoSansCJK-Regular.ttc",
56
+ "/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc",
57
+ "/usr/share/fonts/truetype/fonts-japanese-gothic.ttf",
58
+ ]
59
+
60
+
61
+ def _resolve_font(size: int = 14) -> ImageFont.FreeTypeFont | ImageFont.ImageFont:
62
+ for p in _FONT_SEARCH_PATHS:
63
+ if Path(p).exists():
64
+ try:
65
+ return ImageFont.truetype(p, size)
66
+ except OSError:
67
+ continue
68
+ return ImageFont.load_default()
69
+
70
+
71
+ # ---------------------------------------------------------------------------
72
+ # 多段推論テキストパーサー
73
+ # ---------------------------------------------------------------------------
74
+
75
+ _STAGE_RE = re.compile(r"={10,}\s+(.+?)\s+={10,}")
76
+
77
+
78
+ def _parse_stages(text: str) -> dict[str, str]:
79
+ """多段推論の出力を ``{"観察": "...", "翻刻": "...", ...}`` に分解する。"""
80
+ headers = list(_STAGE_RE.finditer(text))
81
+ if not headers:
82
+ return {"出力": text.strip()}
83
+
84
+ stages: dict[str, str] = {}
85
+ for i, m in enumerate(headers):
86
+ name = m.group(1).strip()
87
+ start = m.end()
88
+ end = headers[i + 1].start() if i + 1 < len(headers) else len(text)
89
+ stages[name] = text[start:end].strip()
90
+ return stages
91
+
92
+
93
+ # ---------------------------------------------------------------------------
94
+ # 画像読み込み(EXIF Orientation)
95
+ # ---------------------------------------------------------------------------
96
+
97
+
98
+ def _image_bytes_exif_aligned(data: bytes) -> bytes:
99
+ """EXIF Orientation を適用した画像を PNG バイト列にする。
100
+
101
+ ブラウザ表示と同じ向き・寸法のラスタにそろえ、VLM 入力と bbox 重畳で
102
+ 同一のピクセル格子を使う(推論はこのバイト列を送る)。
103
+ """
104
+ img = Image.open(io.BytesIO(data))
105
+ img = ImageOps.exif_transpose(img)
106
+ buf = io.BytesIO()
107
+ img.convert("RGB").save(buf, format="PNG", optimize=True)
108
+ return buf.getvalue()
109
+
110
+
111
+ # ---------------------------------------------------------------------------
112
+ # Qwen VL 入力解像度(qwen_vl_utils.smart_resize と同一式)
113
+ #
114
+ # AutoProcessor の min_pixels / max_pixels に合わせて画像が縮小される。
115
+ # プロンプトの「ピクセル」はこの解像度上の値であることが多く、
116
+ # そのまま元画像幅で 0-1000 扱いするとずれる。
117
+ # ---------------------------------------------------------------------------
118
+
119
+ _QWEN_PATCH_FACTOR = 28 # image_patch_size(14) * SPATIAL_MERGE_SIZE(2)
120
+
121
+
122
+ def _round_by_factor(number: float, factor: int) -> int:
123
+ return int(round(number / factor) * factor)
124
+
125
+
126
+ def _floor_by_factor(number: float, factor: int) -> int:
127
+ return int(math.floor(number / factor) * factor)
128
+
129
+
130
+ def _ceil_by_factor(number: float, factor: int) -> int:
131
+ return int(math.ceil(number / factor) * factor)
132
+
133
+
134
+ def _smart_resize_like_qwen(
135
+ height: int,
136
+ width: int,
137
+ min_pixels: int,
138
+ max_pixels: int,
139
+ factor: int = _QWEN_PATCH_FACTOR,
140
+ ) -> tuple[int, int]:
141
+ """qwen_vl_utils.vision_process.smart_resize と同じ (h_bar, w_bar) を返す。"""
142
+ assert max_pixels >= min_pixels, "max_pixels must be >= min_pixels"
143
+ h_bar = max(factor, _round_by_factor(height, factor))
144
+ w_bar = max(factor, _round_by_factor(width, factor))
145
+ if h_bar * w_bar > max_pixels:
146
+ beta = math.sqrt((height * width) / max_pixels)
147
+ h_bar = _floor_by_factor(height / beta, factor)
148
+ w_bar = _floor_by_factor(width / beta, factor)
149
+ elif h_bar * w_bar < min_pixels:
150
+ beta = math.sqrt(min_pixels / (height * width))
151
+ h_bar = _ceil_by_factor(height * beta, factor)
152
+ w_bar = _ceil_by_factor(width * beta, factor)
153
+ return h_bar, w_bar
154
+
155
+
156
+ def _vision_input_size_px(img_w: int, img_h: int, model_key: str) -> tuple[int, int]:
157
+ """VLM が実際にリサイズして見る (proc_h, proc_w)。vlm_app MODELS と一致させる。"""
158
+ cfg = MODELS[model_key]
159
+ return _smart_resize_like_qwen(
160
+ img_h,
161
+ img_w,
162
+ cfg["min_pixels"],
163
+ cfg["max_pixels"],
164
+ )
165
+
166
+
167
+ def _bboxes_within_proc_pixels(bboxes: list[list], proc_w: int, proc_h: int, tol: float = 1.5) -> bool:
168
+ """全角が VLM 入力リサイズ後の幅・高さに収まるなら proc 空間のピクセルとみなせる。"""
169
+ for b in bboxes:
170
+ if len(b) < 4:
171
+ continue
172
+ x1, y1, x2, y2 = b[0], b[1], b[2], b[3]
173
+ if min(x1, x2) < -tol or min(y1, y2) < -tol:
174
+ return False
175
+ if max(x1, x2) > proc_w + tol or max(y1, y2) > proc_h + tol:
176
+ return False
177
+ return True
178
+
179
+
180
+ def _map_proc_pixel_bboxes_to_original(bboxes: list[list], proc_w: int, proc_h: int, img_w: int, img_h: int) -> list[list]:
181
+ sx = img_w / proc_w
182
+ sy = img_h / proc_h
183
+ out: list[list] = []
184
+ for b in bboxes:
185
+ if len(b) < 4:
186
+ out.append(b)
187
+ continue
188
+ out.append([b[0] * sx, b[1] * sy, b[2] * sx, b[3] * sy, *b[4:]])
189
+ return out
190
+
191
+
192
+ # ---------------------------------------------------------------------------
193
+ # Bounding Box 正規化
194
+ # ---------------------------------------------------------------------------
195
+
196
+
197
+ def _normalize_bboxes(bboxes: list[list], img_w: int, img_h: int, model_key: str = "7b") -> list[list]:
198
+ """Qwen2.5-VL が返す座標を元画像のピクセル座標に変換する。
199
+
200
+ 優先:
201
+ 1. 値が VLM 入力リサイズ後の解像度に収まる → proc 上のピクセルとみなし線形拡大
202
+ 2. それ以外 → 従来の 0-1000 / フル解像度ピクセル推定
203
+ """
204
+ if not bboxes:
205
+ return bboxes
206
+
207
+ proc_h, proc_w = _vision_input_size_px(img_w, img_h, model_key)
208
+ if _bboxes_within_proc_pixels(bboxes, proc_w, proc_h):
209
+ scaled = _map_proc_pixel_bboxes_to_original(bboxes, proc_w, proc_h, img_w, img_h)
210
+ logger.info(
211
+ "bbox 座標変換 (VLM 入力リサイズ空間 → 元画像):\n"
212
+ " 元画像 : %d x %d\n"
213
+ " 推論時解像度: %d x %d (smart_resize, model=%s)\n"
214
+ " 先頭 (変換前): %s\n"
215
+ " 先頭 (変換後): %s",
216
+ img_w,
217
+ img_h,
218
+ proc_w,
219
+ proc_h,
220
+ model_key,
221
+ bboxes[0][:5] if bboxes else "[]",
222
+ [round(v, 1) if isinstance(v, float) else v for v in scaled[0][:5]] if scaled else "[]",
223
+ )
224
+ return scaled
225
+
226
+ return _normalize_bboxes_heuristic(bboxes, img_w, img_h)
227
+
228
+
229
+ def _normalize_bboxes_heuristic(bboxes: list[list], img_w: int, img_h: int) -> list[list]:
230
+ """proc 空間に収まらない場合の座標系推定(0-1000 / フル解像度ピクセル)。"""
231
+ coords = []
232
+ for b in bboxes:
233
+ if len(b) >= 4:
234
+ coords.extend([b[0], b[1], b[2], b[3]])
235
+ if not coords:
236
+ return bboxes
237
+
238
+ max_coord = max(coords)
239
+ img_max = max(img_w, img_h)
240
+
241
+ if max_coord <= 1.0:
242
+ mode = "0-1"
243
+ scale_x, scale_y = img_w, img_h
244
+ elif max_coord > 1100:
245
+ # Qwen の 0-1000 正規化では通常 1100 を超えない → 実ピクセルとみなす
246
+ mode = "pixel (>1100)"
247
+ logger.info(
248
+ "bbox 座標変換なし (max_coord>1100 → ピクセルと判定):\n 画像サイズ : %d x %d\n max_coord : %.1f\n 先頭 : %s",
249
+ img_w,
250
+ img_h,
251
+ max_coord,
252
+ bboxes[0][:5] if bboxes else "[]",
253
+ )
254
+ return bboxes
255
+ elif max_coord <= 1100 and img_max > 1100:
256
+ # 座標が ~1000 範囲で画像がそれより大きい → 0-1000 正規化
257
+ mode = "0-1000"
258
+ scale_x, scale_y = img_w / 1000, img_h / 1000
259
+ elif 0.8 < max_coord / img_max < 1.2:
260
+ # 座標範囲が画像サイズと近い → ピクセル座標
261
+ mode = "pixel"
262
+ logger.info(
263
+ "bbox 座標変換なし (ピクセル座標):\n 画像サイズ : %d x %d\n max_coord : %.1f\n 先頭 : %s",
264
+ img_w,
265
+ img_h,
266
+ max_coord,
267
+ bboxes[0][:5] if bboxes else "[]",
268
+ )
269
+ return bboxes
270
+ else:
271
+ # 判断がつかない場合は 0-1000 と仮定(Qwen2.5-VL のデフォルト)
272
+ mode = "0-1000 (fallback)"
273
+ scale_x, scale_y = img_w / 1000, img_h / 1000
274
+
275
+ scaled = [[b[0] * scale_x, b[1] * scale_y, b[2] * scale_x, b[3] * scale_y, *b[4:]] for b in bboxes]
276
+ logger.info(
277
+ "bbox 座標変換 (ヒューリスティック):\n"
278
+ " 画像サイズ : %d x %d\n"
279
+ " 座標モード : %s (max_coord=%.1f)\n"
280
+ " スケール : x=%.4f, y=%.4f\n"
281
+ " 変換前 (先頭): %s\n"
282
+ " 変換後 (先頭): %s",
283
+ img_w,
284
+ img_h,
285
+ mode,
286
+ max_coord,
287
+ scale_x,
288
+ scale_y,
289
+ bboxes[0][:5] if bboxes else "[]",
290
+ [round(v, 1) if isinstance(v, float) else v for v in scaled[0][:5]] if scaled else "[]",
291
+ )
292
+ return scaled
293
+
294
+
295
+ # ---------------------------------------------------------------------------
296
+ # 画像重畳
297
+ # ---------------------------------------------------------------------------
298
+
299
+ _COLOR_TEXT = (204, 120, 92) # --color-accent
300
+ _FILL_TEXT = (204, 120, 92, 35)
301
+ _COLOR_OBJ = (90, 139, 111) # --color-status-success
302
+ _FILL_OBJ = (90, 139, 111, 35)
303
+
304
+
305
+ def _rect_intersects(a: tuple[int, int, int, int], b: tuple[int, int, int, int]) -> bool:
306
+ """軸平行矩形 a, b = (x1, y1, x2, y2) が重なるか。"""
307
+ return not (a[2] <= b[0] or a[0] >= b[2] or a[3] <= b[1] or a[1] >= b[3])
308
+
309
+
310
+ def _place_label_background(
311
+ x1: int,
312
+ y1: int,
313
+ lw: int,
314
+ lh: int,
315
+ img_w: int,
316
+ img_h: int,
317
+ placed: list[tuple[int, int, int, int]],
318
+ ) -> tuple[int, int]:
319
+ """ラベル背景矩形の左上 (lx, ly)。既に配置したラベル矩形と重ならない位置を探す。"""
320
+ if lw <= 0 or lh <= 0:
321
+ return 0, 0
322
+ lw = min(lw, img_w)
323
+ lh = min(lh, img_h)
324
+
325
+ def free(lx: int, ly: int) -> bool:
326
+ r = (lx, ly, lx + lw, ly + lh)
327
+ if r[0] < 0 or r[1] < 0 or r[2] > img_w or r[3] > img_h:
328
+ return False
329
+ return not any(_rect_intersects(r, p) for p in placed)
330
+
331
+ base_lx = max(0, min(int(x1), img_w - lw))
332
+ y_start = max(0, y1 - lh)
333
+ for ly in range(y_start, img_h - lh + 1, 2):
334
+ if free(base_lx, ly):
335
+ return base_lx, ly
336
+ for ly in range(0, min(y_start, img_h - lh + 1), 2):
337
+ if free(base_lx, ly):
338
+ return base_lx, ly
339
+ for lx in range(0, img_w - lw + 1, 6):
340
+ for ly in range(0, img_h - lh + 1, 2):
341
+ if free(lx, ly):
342
+ return lx, ly
343
+ return base_lx, max(0, min(y1 - lh, img_h - lh))
344
+
345
+
346
+ def _draw_overlays(image_bytes: bytes, bboxes: list[list], model_key: str = "7b") -> bytes:
347
+ """元画像に bounding box + ラベルを重畳して PNG bytes を返す。"""
348
+ img = Image.open(io.BytesIO(image_bytes)).convert("RGBA")
349
+ bboxes = _normalize_bboxes(bboxes, img.width, img.height, model_key)
350
+
351
+ overlay = Image.new("RGBA", img.size, (0, 0, 0, 0))
352
+ draw = ImageDraw.Draw(overlay)
353
+ font_size = max(14, min(img.width, img.height) // 50)
354
+ font = _resolve_font(font_size)
355
+ pad = 4
356
+ img_w, img_h = img.width, img.height
357
+
358
+ # 物体 bbox を先に描画し、テキストラベルは後から重ならない位置へ
359
+ def _is_object_box(box: list) -> bool:
360
+ return len(box) >= 6 and str(box[5]).lower() in ("object", "obj")
361
+
362
+ sorted_boxes = sorted(bboxes, key=lambda b: (0 if _is_object_box(b) else 1,))
363
+
364
+ for box in sorted_boxes:
365
+ if len(box) < 5:
366
+ continue
367
+ x1, y1, x2, y2 = int(box[0]), int(box[1]), int(box[2]), int(box[3])
368
+ is_obj = _is_object_box(box)
369
+ color = _COLOR_OBJ if is_obj else _COLOR_TEXT
370
+ fill = _FILL_OBJ if is_obj else _FILL_TEXT
371
+ draw.rectangle([x1, y1, x2, y2], outline=color, width=2, fill=fill)
372
+
373
+ placed_labels: list[tuple[int, int, int, int]] = []
374
+
375
+ for box in sorted_boxes:
376
+ if len(box) < 5:
377
+ continue
378
+ x1, y1, x2, y2 = int(box[0]), int(box[1]), int(box[2]), int(box[3])
379
+ label = str(box[4])
380
+ if not label:
381
+ continue
382
+ is_obj = _is_object_box(box)
383
+ color = _COLOR_OBJ if is_obj else _COLOR_TEXT
384
+ tb = draw.textbbox((0, 0), label, font=font)
385
+ tw, th = tb[2] - tb[0], tb[3] - tb[1]
386
+ lw, lh = tw + pad * 2, th + pad * 2
387
+ lx, ly = _place_label_background(x1, y1, lw, lh, img_w, img_h, placed_labels)
388
+ placed_labels.append((lx, ly, lx + lw, ly + lh))
389
+ draw.rectangle(
390
+ [lx, ly, lx + tw + pad * 2, ly + th + pad * 2],
391
+ fill=(*color, 230),
392
+ )
393
+ draw.text((lx + pad, ly + pad), label, fill=(255, 255, 255, 255), font=font)
394
+
395
+ result = Image.alpha_composite(img, overlay).convert("RGB")
396
+ buf = io.BytesIO()
397
+ result.save(buf, format="PNG", optimize=True)
398
+ return buf.getvalue()
399
+
400
+
401
+ # ---------------------------------------------------------------------------
402
+ # Modal 呼び出しヘルパー
403
+ #
404
+ # デプロイ済みアプリに対して modal.Cls.lookup() で接続する。
405
+ # 事前に `modal deploy src/kuzushiji_ocr/vlm_app.py` が必要。
406
+ # ---------------------------------------------------------------------------
407
+
408
+ _MODAL_APP_NAME = "kuzushiji-vlm-ocr"
409
+
410
+
411
+ def _get_inference(model_key: str) -> Any:
412
+ cls_name = "VLMInference32B" if model_key == "32b" else "VLMInference7B"
413
+ cls = modal.Cls.from_name(_MODAL_APP_NAME, cls_name)
414
+ return cls()
415
+
416
+
417
+ def _call_structured(inference: Any, pages: list[tuple[str, bytes]]) -> dict[str, Any]:
418
+ return inference.ocr_pages.remote(pages, structured=True)
419
+
420
+
421
+ def _call_deep_think(inference: Any, pages: list[tuple[str, bytes]]) -> dict[str, Any]:
422
+ return inference.ocr_pages.remote(pages, deep_think=True)
423
+
424
+
425
+ def _collect_bboxes_from_modal_json(json_data: Any) -> list[list]:
426
+ """Modal / VLM の JSON 配列から bbox 行を集める。
427
+
428
+ 次のどちらにも対応する:
429
+ - ``[[x1,y1,x2,y2,label], ...]`` — bbox のフラットな配列
430
+ - ``[[[x1,...], ...]]`` — 1 段ラップ(ページ相当)
431
+
432
+ 誤って ``extend`` して座標をばらすことはしない。
433
+ """
434
+ out: list[list] = []
435
+ if not isinstance(json_data, list):
436
+ return out
437
+ for item in json_data:
438
+ if not isinstance(item, list) or not item:
439
+ continue
440
+ if _looks_like_flat_bbox_row(item):
441
+ row = _normalize_structured_item(item)
442
+ if row:
443
+ out.append(row)
444
+ continue
445
+ for sub in item:
446
+ if not isinstance(sub, list):
447
+ continue
448
+ row = _normalize_structured_item(sub)
449
+ if row:
450
+ out.append(row)
451
+ return out
452
+
453
+
454
+ def _extract_bboxes(structured_result: dict[str, Any]) -> tuple[list[list], str]:
455
+ """structured 出力から bbox リストをフラットに取り出す。
456
+
457
+ Returns:
458
+ (bboxes, raw_text) — raw_text はデバッグ用の生テキスト。
459
+ """
460
+ raw_text = ""
461
+ for txt in structured_result.get("txt", {}).values():
462
+ raw_text = txt
463
+ break
464
+
465
+ # 1) Modal 側でパース済みの JSON を使う
466
+ bboxes: list[list] = []
467
+ for json_data in structured_result.get("json", {}).values():
468
+ bboxes.extend(_collect_bboxes_from_modal_json(json_data))
469
+
470
+ if bboxes:
471
+ logger.info(
472
+ "bbox %d 件取得(Modal パース済み JSON):\n%s",
473
+ len(bboxes),
474
+ "\n".join(f" [{i}] {b}" for i, b in enumerate(bboxes)),
475
+ )
476
+ return bboxes, raw_text
477
+
478
+ # 2) JSON が空の場合、生テキストを Web サーバー側で再パースする
479
+ if raw_text:
480
+ logger.info("Modal 側 JSON が空 — 生テキストを再パース:\n %s", raw_text[:300])
481
+
482
+ # 2a) vlm_app の標準パーサー(json.loads — ダブルクォートのみ対応)
483
+ parsed = _parse_structured_output(raw_text)
484
+ if parsed:
485
+ for chunk in parsed:
486
+ if isinstance(chunk, list):
487
+ bboxes.extend(_collect_bboxes_from_modal_json(chunk))
488
+
489
+ # 2b) VLM がシングルクォートで出力した場合のフォールバック
490
+ if not bboxes:
491
+ bboxes = _parse_bbox_lenient(raw_text)
492
+
493
+ if bboxes:
494
+ logger.info(
495
+ "bbox %d 件取得(Web サーバー側再パース):\n%s",
496
+ len(bboxes),
497
+ "\n".join(f" [{i}] {b}" for i, b in enumerate(bboxes)),
498
+ )
499
+ return bboxes, raw_text
500
+
501
+ logger.warning("bbox 取得失敗 — VLM 生テキスト:\n %s", raw_text[:500] if raw_text else "(空)")
502
+ return [], raw_text
503
+
504
+
505
+ def _parse_bbox_lenient(raw: str) -> list[list]:
506
+ """VLM がシングルクォートや Python リテラルで出力した場合に対応する。"""
507
+ row_scan = parse_bbox_rows_from_text(raw)
508
+ if row_scan:
509
+ return row_scan
510
+
511
+ match = re.search(r"\[.*\]", raw, re.DOTALL)
512
+ if not match:
513
+ return []
514
+
515
+ text = match.group()
516
+
517
+ # シングルクォートをダブルクォートに置換して json.loads
518
+ try:
519
+ data = json.loads(text.replace("'", '"'))
520
+ except (json.JSONDecodeError, TypeError):
521
+ pass
522
+ else:
523
+ return _flatten_bbox_list(data)
524
+
525
+ # ast.literal_eval で Python リテラルとしてパース
526
+ try:
527
+ data = ast.literal_eval(text)
528
+ except (ValueError, SyntaxError):
529
+ return []
530
+
531
+ return _flatten_bbox_list(data)
532
+
533
+
534
+ def _flatten_bbox_list(data: Any) -> list[list]:
535
+ """パース済み JSON 配列を ``[[x1,y1,x2,y2,label], ...]`` にフラット化する。
536
+
537
+ ``bbox_2d`` 付きオブジェクト形式も ``vlm_app._normalize_structured_item`` で扱う。
538
+ """
539
+ if not isinstance(data, list):
540
+ return []
541
+
542
+ result: list[list] = []
543
+ for item in data:
544
+ row = _normalize_structured_item(item)
545
+ if row:
546
+ result.append(row)
547
+ return result
548
+
549
+
550
+ # ---------------------------------------------------------------------------
551
+ # ルーティング
552
+ # ---------------------------------------------------------------------------
553
+
554
+
555
+ @app.get("/", response_class=HTMLResponse)
556
+ async def index() -> HTMLResponse:
557
+ html = (TEMPLATES_DIR / "index.html").read_text(encoding="utf-8")
558
+ return HTMLResponse(html)
559
+
560
+
561
+ @app.post("/api/ocr")
562
+ async def ocr(
563
+ image: UploadFile = File(...), # noqa: B008
564
+ model: str = Form("7b"),
565
+ ) -> dict[str, Any]:
566
+ if model not in MODELS:
567
+ raise HTTPException(400, f"不明なモデル: {model} (選択肢: {', '.join(MODELS)})")
568
+
569
+ data = await image.read()
570
+ filename = image.filename or "upload.jpg"
571
+
572
+ try:
573
+ img = Image.open(io.BytesIO(data))
574
+ img.verify()
575
+ except Exception:
576
+ raise HTTPException(400, "無効な画像ファイルです") from None
577
+
578
+ try:
579
+ aligned = _image_bytes_exif_aligned(data)
580
+ except Exception:
581
+ raise HTTPException(400, "無効な画像ファイルです") from None
582
+
583
+ # Modal の MIME は拡張子で決まるため、実体が PNG なら .png に合わせる
584
+ aligned_name = f"{Path(filename).stem}.png"
585
+ pages: list[tuple[str, bytes]] = [(aligned_name, aligned)]
586
+ inference = _get_inference(model)
587
+ loop = asyncio.get_event_loop()
588
+
589
+ structured_fut = loop.run_in_executor(_executor, _call_structured, inference, pages)
590
+ deep_think_fut = loop.run_in_executor(_executor, _call_deep_think, inference, pages)
591
+
592
+ structured_result, deep_think_result = await asyncio.gather(structured_fut, deep_think_fut, return_exceptions=True)
593
+
594
+ # --- bbox ---
595
+ bboxes: list[list] = []
596
+ structured_raw = ""
597
+ if isinstance(structured_result, dict):
598
+ bboxes, structured_raw = _extract_bboxes(structured_result)
599
+
600
+ # --- 重畳画像(VLM に渡したのと同一の aligned ラスタに描画) ---
601
+ if bboxes:
602
+ annotated_bytes = _draw_overlays(aligned, bboxes, model)
603
+ else:
604
+ annotated_bytes = aligned
605
+ annotated_b64 = base64.b64encode(annotated_bytes).decode()
606
+
607
+ # --- ステージテキスト ---
608
+ stages: dict[str, str] = {}
609
+ if isinstance(deep_think_result, dict):
610
+ for text in deep_think_result.get("txt", {}).values():
611
+ stages = _parse_stages(text)
612
+ break
613
+
614
+ # --- エラー情報 ---
615
+ errors: list[str] = []
616
+ if isinstance(structured_result, Exception):
617
+ errors.append(f"構造化 OCR エラー: {structured_result}")
618
+ if isinstance(deep_think_result, Exception):
619
+ errors.append(f"多段推論エラー: {deep_think_result}")
620
+ if isinstance(structured_result, dict) and not bboxes:
621
+ errors.append(f"構造化 OCR は完了しましたが、bounding box を抽出できませんでした。 VLM 生テキスト: {structured_raw[:300]}")
622
+
623
+ return {
624
+ "annotated_image": annotated_b64,
625
+ "stages": stages,
626
+ "bboxes": bboxes,
627
+ "model_used": MODELS[model]["id"],
628
+ "errors": errors,
629
+ }
tests/test_imports.py ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ from kuzushiji_ocr import __version__
2
+ from kuzushiji_ocr.web import app
3
+
4
+
5
+ def test_app_title() -> None:
6
+ assert app.title == "くずし字 OCR"
7
+
8
+
9
+ def test_version() -> None:
10
+ assert __version__ == "0.1.0"
uv.lock ADDED
The diff for this file is too large to render. See raw diff