caipeng328 commited on
Commit
73759a2
Β·
verified Β·
1 Parent(s): 710ea2e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -22
README.md CHANGED
@@ -16,21 +16,21 @@ tags:
16
  <div align="center">
17
 
18
  <h1 align="center">
19
- NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
20
 
21
  </h1>
22
 
23
  [![License](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE)
24
- [![GitHub](https://img.shields.io/badge/GitHub-Repository-black?logo=github)](https://github.com/caipeng328/NaviDC-OCR)
25
  [![Papers with Code: SOTA on OmniDocBench v1.6](https://paperswithcode.co/api/v1/papers/2608.12898/leaderboard-badge.svg?eval=25856&live=1)](https://paperswithcode.co/api/v1/papers/2608.12898/leaderboard-badge-link?eval=25856)
26
  </div>
27
 
28
  <div align="center">
29
- <img src="https://raw.githubusercontent.com/caipeng328/NaviDC-OCR/refs/heads/main/assets/score.png" width="800">
30
  </div>
31
 
32
  ## πŸ”₯ News
33
-
34
  - **2026/09/01** - We noticed that EMNLP 2026 is hosting the [Dr.DocBench Challenge](https://eval.ai/web/challenges/challenge-page/2717/overview), a document parsing competition. We evaluated NaviDC-OCR with its native weights, achieving better results than MinerU 2.5 Pro and PaddleOCR-VL 1.6. Detailed results are shown below **dr.docbench-challenge**. We welcome the use of NaviDC‑OCR for competitions. Going forward, we will continue to deliver competitive parsing models for the community.
35
  - **2026/08/29** β€” Thanks to Nandraj for the GGUF conversion and llama.cpp support! [πŸ”— NaviDC-OCR-GGUF](https://huggingface.co/nandraj/NaviDC-OCR-GGUF)
36
  - **2026/08/17** β€” NaviDC-OCR [model weights](https://huggingface.co/StarDoc-AI/NaviDC-OCR) and [technical report](https://arxiv.org/pdf/2608.12898) have been released.
@@ -38,11 +38,11 @@ NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Docum
38
 
39
  ## πŸ“– Introduction
40
 
41
- **NaviDC-OCR** is a lightweight (~1.2B parameters), open-source Vision-Language Model designed specifically for document parsing.
42
 
43
- Unlike existing methods that mainly target either digital documents or camera-captured documents, NaviDC-OCR unifies both scenarios within a single framework.
44
 
45
- Compared with previous document parsing models, NaviDC-OCR introduces
46
 
47
  - Multi-node Consensus Voting (MCV) for automatic pseudo-label generation
48
  - Geometry-aware document modeling for camera-captured documents
@@ -51,18 +51,18 @@ Compared with previous document parsing models, NaviDC-OCR introduces
51
  - Progressive four-stage training pipeline
52
  - Content-Structure Decoupled Learning for tables and formulas
53
 
54
- These techniques enable NaviDC-OCR to achieve state-of-the-art performance on both digital and camera-captured document benchmarks while remaining lightweight enough for practical deployment.
55
 
56
  ## πŸ“Š Experimental Results
57
 
58
- NaviDC-OCR achieves state-of-the-art performance on multiple public document parsing benchmarks.
59
 
60
  ### Layout Visualization of Distorted Documents
61
- To evaluate the model's ability to understand complex document deformations, we conduct a visual evaluation on the public dewarping datasets DocUNet and DIR300, with representative results shown in Figure. NaviDC-OCR directly performs layout and content parsing on distorted documents without dewarping preprocessing or a dedicated rectification model, demonstrating robust parsing under complex geometric deformations.
62
  <div align="center">
63
 
64
- <img src="https://raw.githubusercontent.com/caipeng328/NaviDC-OCR/refs/heads/main/assets/dir300.png" width="800" alt="Parsing evaluation on the DIR300 dataset.">
65
- <img src="https://raw.githubusercontent.com/caipeng328/NaviDC-OCR/refs/heads/main/assets/docunet.png" width="800" alt="Parsing evaluation on the DocUNet dataset.">
66
  </div>
67
 
68
  ---
@@ -72,7 +72,7 @@ To evaluate the model's ability to understand complex document deformations, we
72
  |ζ¨‘εž‹|overall ↑|Text edit ↓|formula cdm ↑|Table teds ↑|order edit ↓|
73
  |---|---|---|---|---|---|
74
  |**Specialized VLMs**| | | | | |
75
- |Navidc‑OCR|67.96|0.1903|0.02|64.97|0.398|
76
  |Mineru 2.5 pro|62.26|0.3402|0.04|67.75|0.356|
77
  |OvisOCR2|59.25|0.3883|0.00|61.59|0.3791|
78
  |PaddleOCRvl 1.6|55.11 | 0.4364 | 0.21 | 51.34 |0.412 |
@@ -84,7 +84,7 @@ To evaluate the model's ability to understand complex document deformations, we
84
 
85
  | Model Type | Methods | Param | Overall ↑ | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | Read Order Edit ↓ |
86
  |---|---|---:|---:|---:|---:|---:|---:|---:|
87
- | **Specialized VLMs** | **NaviDC-OCR** | **1.2B** | **96.87** | <u>0.027</u> | 96.36 | **97.05** | **98.52** | <u>0.122</u> |
88
  | | OvisOCR2 | 0.8B | <u>96.58</u> | **0.025** | **97.53** | <u>94.76</u> | <u>97.16</u> | **0.111** |
89
  | | PaddleOCR-VL-1.6 | 0.9B | 96.33 | 0.033 | <u>97.49</u> | <u>94.76</u> | 97.11 | 0.127 |
90
  | | MinerU2.5-Pro | 1.2B | 95.75 | 0.036 | 97.45 | 93.42 | 95.92 | 0.120 |
@@ -116,7 +116,7 @@ To evaluate the model's ability to understand complex document deformations, we
116
  ### [Wild_OmniDocBench](https://github.com/VirtualLUOUCAS/Wild_OmniDocBench)
117
  | Model Type | Methods | Param | Overall ↑ | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | Read Order Edit ↓ |
118
  | ------------------- | ----------------- | ----: | -----------: | ----------: | ------------: | -----------: | -------------: | ----------------: |
119
- | **Decoupled VLMs** | **NaviDC-OCR** | 1.2B | **88.53** | **0.1173** | 88.26 | **89.05** | **92.14** | **0.2011** |
120
  | | PaddleOCR-VL-1.6 | 0.9B | 87.36 | 0.1369 | 88.42 | <u>85.76</u> | <u>90.14</u> | 0.2057 |
121
  | | MinerU2.5-Pro | 1.2B | 87.33 | 0.1362 | <u>90.15</u> | 85.46 | 90.12 | <u>0.2013</u> |
122
  | | GLM-OCR | 0.9B | 85.08 | 0.1514 | 89.09 | 81.31 | 85.90 | 0.2228 |
@@ -132,7 +132,7 @@ To evaluate the model's ability to understand complex document deformations, we
132
  ### [PureDocBench](https://github.com/zhihengli-casia/puredocbench/)
133
  | Model Type | Model | Clean Overall ↑ | Clean Text ↓ | Clean Formula ↑ | Clean Table ↑ | Digital Degraded Overall ↑ | Digital Degraded Text ↓ | Digital Degraded Formula ↑ | Digital Degraded Table ↑ | Real Degraded Overall ↑ | Real Degraded Text ↓ | Real Degraded Formula ↑ | Real Degraded Table ↑ |
134
  | ------------------ | ----------------- | --------------: | -----------: | --------------: | ------------: | -------------------------: | ----------------------: | -------------------------: | -----------------------: | ----------------------: | -------------------: | ----------------------: | --------------------: |
135
- | **Decoupled VLM** | **NaviDC-OCR** | **86.90** | **0.111** | **81.01** | **91.09** | <u>77.47</u> | 0.206 | **72.59** | 80.45 | **70.85** | <u>0.302</u> | **65.11** | **77.66** |
136
  | | DotsMOCR | 76.27 | 0.151 | 66.23 | 77.65 | 73.16 | <u>0.198</u> | 64.32 | 74.95 | 61.73 | 0.312 | 54.39 | 61.97 |
137
  | | MinerU2.5-Pro | 75.87 | 0.222 | 65.14 | 84.68 | 71.77 | 0.272 | 61.79 | 80.73 | 62.56 | 0.375 | 52.70 | 72.47 |
138
  | | YouTu-Parsing | 75.02 | 0.230 | 67.34 | 80.74 | 69.66 | 0.270 | 61.44 | 74.49 | 60.29 | 0.360 | 52.20 | 64.69 |
@@ -156,7 +156,7 @@ To evaluate the model's ability to understand complex document deformations, we
156
  ### [ICDAR2026 Sci-ImageMiner](https://sites.google.com/view/sci-imageminer/)
157
  | # | Team | RMS | TEDS | Weighted |
158
  | ----: | --------------- | --------: | --------: | --------: |
159
- | **1** | **NaviDC-OCR** | **17.23** | **66.39** | **41.81** |
160
  | 2 | VLMinators | 17.29 | 64.31 | 40.80 |
161
  | 3 | Ricoh_SRCB | 16.23 | 61.12 | 38.67 |
162
  | 4 | Vassilis Sioros | 14.94 | 55.20 | 35.07 |
@@ -353,9 +353,9 @@ def infer(image: Image.Image, prompt: str) -> str:
353
 
354
 
355
 
356
- processor = AutoProcessor.from_pretrained("StarDoc-AI/NaviDC-OCR", trust_remote_code=True, use_fast=True)
357
  model = AutoModel.from_pretrained(
358
- "StarDoc-AI/NaviDC-OCR",
359
  trust_remote_code=True,
360
  torch_dtype=torch.bfloat16,
361
  ).cuda().eval()
@@ -405,8 +405,8 @@ If you would like to perform complete document parsing, please refer to our GitH
405
  ## Citation
406
 
407
  ```bibtex
408
- @article{navidc_ocr,
409
- title={NaviDC-OCR: Navigating Document Parsing Across Digital and Camera-Captured Documents},
410
  author={Cai, Peng and Zou, Zhaofan and Liu, Shifa and Wang, Yikun and Tang, Jiawei and Yang, Kaicheng and Tong, Meng and He, Zhongjiang and Sun, Hao},
411
  journal={arXiv preprint arXiv:2608.12898},
412
  year={2026}
@@ -418,7 +418,7 @@ Thanks to Nandraj for the GGUF conversion and llama.cpp support! [πŸ”— NaviDC-O
418
 
419
  ## Acknowledgements
420
 
421
- NaviDC-OCR is built upon
422
 
423
  - MinerU
424
  - Qwen2.5-VL
 
16
  <div align="center">
17
 
18
  <h1 align="center">
19
+ TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
20
 
21
  </h1>
22
 
23
  [![License](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE)
24
+ [![GitHub](https://img.shields.io/badge/GitHub-Repository-black?logo=github)](https://github.com/caipeng328/TeleOCR)
25
  [![Papers with Code: SOTA on OmniDocBench v1.6](https://paperswithcode.co/api/v1/papers/2608.12898/leaderboard-badge.svg?eval=25856&live=1)](https://paperswithcode.co/api/v1/papers/2608.12898/leaderboard-badge-link?eval=25856)
26
  </div>
27
 
28
  <div align="center">
29
+ <img src="https://raw.githubusercontent.com/caipeng328/TeleOCR/refs/heads/main/assets/score.png" width="800">
30
  </div>
31
 
32
  ## πŸ”₯ News
33
+ - **2026/09/10** - We have renamed NaviDC-OCR to TeleOCR, and all subsequent model iterations will be developed and released under the TeleOCR version.
34
  - **2026/09/01** - We noticed that EMNLP 2026 is hosting the [Dr.DocBench Challenge](https://eval.ai/web/challenges/challenge-page/2717/overview), a document parsing competition. We evaluated NaviDC-OCR with its native weights, achieving better results than MinerU 2.5 Pro and PaddleOCR-VL 1.6. Detailed results are shown below **dr.docbench-challenge**. We welcome the use of NaviDC‑OCR for competitions. Going forward, we will continue to deliver competitive parsing models for the community.
35
  - **2026/08/29** β€” Thanks to Nandraj for the GGUF conversion and llama.cpp support! [πŸ”— NaviDC-OCR-GGUF](https://huggingface.co/nandraj/NaviDC-OCR-GGUF)
36
  - **2026/08/17** β€” NaviDC-OCR [model weights](https://huggingface.co/StarDoc-AI/NaviDC-OCR) and [technical report](https://arxiv.org/pdf/2608.12898) have been released.
 
38
 
39
  ## πŸ“– Introduction
40
 
41
+ **TeleOCR** is a lightweight (~1.2B parameters), open-source Vision-Language Model designed specifically for document parsing.
42
 
43
+ Unlike existing methods that mainly target either digital documents or camera-captured documents, TeleOCR unifies both scenarios within a single framework.
44
 
45
+ Compared with previous document parsing models, TeleOCR introduces
46
 
47
  - Multi-node Consensus Voting (MCV) for automatic pseudo-label generation
48
  - Geometry-aware document modeling for camera-captured documents
 
51
  - Progressive four-stage training pipeline
52
  - Content-Structure Decoupled Learning for tables and formulas
53
 
54
+ These techniques enable TeleOCR to achieve state-of-the-art performance on both digital and camera-captured document benchmarks while remaining lightweight enough for practical deployment.
55
 
56
  ## πŸ“Š Experimental Results
57
 
58
+ TeleOCR achieves state-of-the-art performance on multiple public document parsing benchmarks.
59
 
60
  ### Layout Visualization of Distorted Documents
61
+ To evaluate the model's ability to understand complex document deformations, we conduct a visual evaluation on the public dewarping datasets DocUNet and DIR300, with representative results shown in Figure. TeleOCR directly performs layout and content parsing on distorted documents without dewarping preprocessing or a dedicated rectification model, demonstrating robust parsing under complex geometric deformations.
62
  <div align="center">
63
 
64
+ <img src="https://raw.githubusercontent.com/caipeng328/TeleOCR/refs/heads/main/assets/dir300.png" width="800" alt="Parsing evaluation on the DIR300 dataset.">
65
+ <img src="https://raw.githubusercontent.com/caipeng328/TeleOCR/refs/heads/main/assets/docunet.png" width="800" alt="Parsing evaluation on the DocUNet dataset.">
66
  </div>
67
 
68
  ---
 
72
  |ζ¨‘εž‹|overall ↑|Text edit ↓|formula cdm ↑|Table teds ↑|order edit ↓|
73
  |---|---|---|---|---|---|
74
  |**Specialized VLMs**| | | | | |
75
+ |TeleOCR|67.96|0.1903|0.02|64.97|0.398|
76
  |Mineru 2.5 pro|62.26|0.3402|0.04|67.75|0.356|
77
  |OvisOCR2|59.25|0.3883|0.00|61.59|0.3791|
78
  |PaddleOCRvl 1.6|55.11 | 0.4364 | 0.21 | 51.34 |0.412 |
 
84
 
85
  | Model Type | Methods | Param | Overall ↑ | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | Read Order Edit ↓ |
86
  |---|---|---:|---:|---:|---:|---:|---:|---:|
87
+ | **Specialized VLMs** | **TeleOCR** | **1.2B** | **96.87** | <u>0.027</u> | 96.36 | **97.05** | **98.52** | <u>0.122</u> |
88
  | | OvisOCR2 | 0.8B | <u>96.58</u> | **0.025** | **97.53** | <u>94.76</u> | <u>97.16</u> | **0.111** |
89
  | | PaddleOCR-VL-1.6 | 0.9B | 96.33 | 0.033 | <u>97.49</u> | <u>94.76</u> | 97.11 | 0.127 |
90
  | | MinerU2.5-Pro | 1.2B | 95.75 | 0.036 | 97.45 | 93.42 | 95.92 | 0.120 |
 
116
  ### [Wild_OmniDocBench](https://github.com/VirtualLUOUCAS/Wild_OmniDocBench)
117
  | Model Type | Methods | Param | Overall ↑ | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | Read Order Edit ↓ |
118
  | ------------------- | ----------------- | ----: | -----------: | ----------: | ------------: | -----------: | -------------: | ----------------: |
119
+ | **Decoupled VLMs** | **TeleOCR** | 1.2B | **88.53** | **0.1173** | 88.26 | **89.05** | **92.14** | **0.2011** |
120
  | | PaddleOCR-VL-1.6 | 0.9B | 87.36 | 0.1369 | 88.42 | <u>85.76</u> | <u>90.14</u> | 0.2057 |
121
  | | MinerU2.5-Pro | 1.2B | 87.33 | 0.1362 | <u>90.15</u> | 85.46 | 90.12 | <u>0.2013</u> |
122
  | | GLM-OCR | 0.9B | 85.08 | 0.1514 | 89.09 | 81.31 | 85.90 | 0.2228 |
 
132
  ### [PureDocBench](https://github.com/zhihengli-casia/puredocbench/)
133
  | Model Type | Model | Clean Overall ↑ | Clean Text ↓ | Clean Formula ↑ | Clean Table ↑ | Digital Degraded Overall ↑ | Digital Degraded Text ↓ | Digital Degraded Formula ↑ | Digital Degraded Table ↑ | Real Degraded Overall ↑ | Real Degraded Text ↓ | Real Degraded Formula ↑ | Real Degraded Table ↑ |
134
  | ------------------ | ----------------- | --------------: | -----------: | --------------: | ------------: | -------------------------: | ----------------------: | -------------------------: | -----------------------: | ----------------------: | -------------------: | ----------------------: | --------------------: |
135
+ | **Decoupled VLM** | **TeleOCR** | **86.90** | **0.111** | **81.01** | **91.09** | <u>77.47</u> | 0.206 | **72.59** | 80.45 | **70.85** | <u>0.302</u> | **65.11** | **77.66** |
136
  | | DotsMOCR | 76.27 | 0.151 | 66.23 | 77.65 | 73.16 | <u>0.198</u> | 64.32 | 74.95 | 61.73 | 0.312 | 54.39 | 61.97 |
137
  | | MinerU2.5-Pro | 75.87 | 0.222 | 65.14 | 84.68 | 71.77 | 0.272 | 61.79 | 80.73 | 62.56 | 0.375 | 52.70 | 72.47 |
138
  | | YouTu-Parsing | 75.02 | 0.230 | 67.34 | 80.74 | 69.66 | 0.270 | 61.44 | 74.49 | 60.29 | 0.360 | 52.20 | 64.69 |
 
156
  ### [ICDAR2026 Sci-ImageMiner](https://sites.google.com/view/sci-imageminer/)
157
  | # | Team | RMS | TEDS | Weighted |
158
  | ----: | --------------- | --------: | --------: | --------: |
159
+ | **1** | **TeleOCR** | **17.23** | **66.39** | **41.81** |
160
  | 2 | VLMinators | 17.29 | 64.31 | 40.80 |
161
  | 3 | Ricoh_SRCB | 16.23 | 61.12 | 38.67 |
162
  | 4 | Vassilis Sioros | 14.94 | 55.20 | 35.07 |
 
353
 
354
 
355
 
356
+ processor = AutoProcessor.from_pretrained("StarDoc-AI/TeleOCR", trust_remote_code=True, use_fast=True)
357
  model = AutoModel.from_pretrained(
358
+ "StarDoc-AI/TeleOCR",
359
  trust_remote_code=True,
360
  torch_dtype=torch.bfloat16,
361
  ).cuda().eval()
 
405
  ## Citation
406
 
407
  ```bibtex
408
+ @article{teleocr,
409
+ title={TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents},
410
  author={Cai, Peng and Zou, Zhaofan and Liu, Shifa and Wang, Yikun and Tang, Jiawei and Yang, Kaicheng and Tong, Meng and He, Zhongjiang and Sun, Hao},
411
  journal={arXiv preprint arXiv:2608.12898},
412
  year={2026}
 
418
 
419
  ## Acknowledgements
420
 
421
+ TeleOCR is built upon
422
 
423
  - MinerU
424
  - Qwen2.5-VL