README.md

---
license: mit
language:
- en
base_model:
- PP-OCRv6_mobile_det
- PP-OCRv6_mobile_rec
- PP-LCNet_x0_25_textline_ori
pipeline_tag: text-classification
tags:
- OCR
- paddle
- PPOCRv6
- axera
---

# PPOCR_v6
> English | [中文](./README-zh.md)

This version of PPOCR_v6 has been converted to run on AXERA NPU with **w8a16** quantization.

## Conversion Tool Links

If you are interested in model conversion, you can export axmodel through the following links:

- [ax-samples-github](https://github.com/AXERA-TECH/ax-samples), other interesting samples

- [Pulsar2 Documentation, ONNX to axmodel conversion](https://pulsar2-docs.readthedocs.io/en/latest/pulsar2/introduction.html)

## Supported Platforms

- AX650
  - [M4N-Dock (AXERA Pi Pro)](https://wiki.sipeed.com/hardware/zh/maixIV/m4ndock/m4ndock.html)
  - [M.2 Accelerator Card](https://axcl-docs.readthedocs.io/zh-cn/latest/doc_guide_hardware.html)
- AX630C
  - [AXERA Pi 2](https://axera-pi-2-docs-cn.readthedocs.io/zh-cn/latest/index.html)
  - [Module-LLM](https://docs.m5stack.com/zh_CN/module/Module-LLM)
  - [LLM630 Compute Kit](https://docs.m5stack.com/zh_CN/core/LLM630%20Compute%20Kit)
- AX615
  - [AX615 IPC SoC](https://www.axera-tech.com/zh-hans/product/2956.html)

### Performance Benchmarks

| Chip  | Model                           | npu_mode | Latency (ms) |
| ----- | ------------------------------- | -------- | ------------ |
|       | PP-OCRv6_small_det              | NPU1     | 42.158       |
|       | PP-OCRv6_small_det              | NPU3     | 23.837       |
| AX650 | PP-LCNet_x0_25_textline_ori     | NPU1     | 0.294        |
|       | PP-LCNet_x0_25_textline_ori     | NPU3     | 0.172        |
|       | PP-OCRv6_small_rec              | NPU1     | 2.473        |
|       | PP-OCRv6_small_rec              | NPU3     | 1.073        |
|       | -                               | -        | -            |
|       | PP-OCRv6_small_det              | NPU1     | \            |
|       | PP-OCRv6_small_det              | NPU2     | 186.738      |
| AX630c| PP-LCNet_x0_25_textline_ori     | NPU1     | 0.428        |
|       | PP-LCNet_x0_25_textline_ori     | NPU2     | 0.381        |
|       | PP-OCRv6_small_rec              | NPU1     | 25.251       |
|       | PP-OCRv6_small_rec              | NPU2     | 9.535        |
|       | -                               | -        | -            |
|       | PP-OCRv6_small_det              | NPU1     | \            |
|       | PP-OCRv6_small_det              | NPU2     | 198.401      |
| AX615 | PP-LCNet_x0_25_textline_ori     | NPU1     | 0.851        |
|       | PP-LCNet_x0_25_textline_ori     | NPU2     | 0.728        |
|       | PP-OCRv6_small_rec              | NPU1     | 36.178       |
|       | PP-OCRv6_small_rec              | NPU2     | 8.036        |

Benchmarking command:

``` bash
ax_run_model -w 10 -r 100 -m xx.axmodel
```

Recognition and detection ONNX model sources: [small-det-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_det_onnx) and [small-rec-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_rec_onnx) from [PaddlePaddle/PP-OCRv6](https://huggingface.co/collections/PaddlePaddle/pp-ocrv6)

Text direction classifier: [AXERA-TECH/PPOCR_v5](https://huggingface.co/AXERA-TECH/PPOCR_v5/tree/main/onnx)

## Usage

Download all files from this repository to your device.

```
PPOCR_v6# tree -L 1
.
|-- 11.jpg          # Test image
|-- README-zh.md
|-- README.md
|-- axmodel         # Axmodel files for each version
|-- cls.json        # Text direction classifier axmodel conversion config
|-- dataset         # Quantization dataset & test dataset
|-- det.json        # Detection model axmodel conversion config
|-- fonts           # Rendering fonts
|-- onnx            # Original ONNX files for each model
|-- ppocrv6_ax.py   # Axmodel inference pipeline
|-- ppocrv6_onnx.py # ONNX inference pipeline
|-- rec.json        # Recognition model axmodel conversion config
|-- res-ax.jpg      # Axmodel inference result
|-- res-onnx.jpg    # ONNX inference result
|-- run_det_ax.py   # Detection axmodel accuracy test script
|-- run_det_onnx.py # Detection ONNX accuracy test script
|-- run_rec_ax.py   # Recognition axmodel accuracy test script
`-- run_rec_onnx.py # Recognition ONNX accuracy test script
```

### Conversion
```
cd dataset 
sh download_quant_dataset.sh
sh download_val_dataset.sh
cd ..
pulsar2 build --config det.json 
pulsar2 build --config cls.json 
pulsar2 build --config rec.json 

```

### Testing

#### Detection

``` python
python3 run_det_onnx.py --resize_mode letterbox   # resize_mode options: letterbox (default), stretch (official)
"""
  Images:       50
  GT boxes:     201
  DET boxes:    151
  Matched:      77
  Precision:    0.5099 (50.99%)
  Recall:       0.3831 (38.31%)
  Hmean (F1):   0.4375
"""
python3 run_det_ax.py --resize_mode letterbox   # resize_mode options: letterbox (default), stretch (official)
"""
  Images:       50
  GT boxes:     201
  DET boxes:    150
  Matched:      75
  Precision:    0.5000 (50.00%)
  Recall:       0.3731 (37.31%)
  Hmean (F1):   0.4274
"""
```

Note: When `resize_mode` is `stretch`, it follows the official approach of directly resizing to the model input size. When `resize_mode` is `letterbox`, it pads the bottom-right corner, which has a smaller gap from the dynamic-input ONNX model and achieves better metrics in this test. You can download the `inference.onnx` with dynamic input `shape` from [small-det-onnx](https://huggingface.co/PaddlePaddle/PP-OCRv6_small_det_onnx) for comparison.

#### Recognition

``` python
python3 run_rec_onnx.py
"""
  Total samples:         2077
  Correct (exact match): 1563
  Accuracy:              0.7525 (75.25%)
  Norm Edit Distance:    0.8947
"""
python3 run_rec_ax.py
"""
  Total samples:         2077
  Correct (exact match): 1518
  Accuracy:              0.7309 (73.09%)
  Norm Edit Distance:    0.8781
"""
```

### Inference

Run inference on AX650 host, such as M4N-Dock (AXERA Pi Pro).

Input image:
![input](./11.jpg)


``` python
python3 ppocrv6_onnx.py --use_angle_cls --visualize --image 11.jpg
python3 ppocrv6_ax.py --use_angle_cls --visualize --image 11.jpg
```

Output image:
![output](./res-ax.jpg)

### Others
The `tiny` recognition model has a large quantization error. Metrics are as follows:
```
onnx-preds:
  Total samples:         2077
  Correct (exact match): 1271
  Accuracy:              0.6119 (61.19%)
  Norm Edit Distance:    0.8263

ax-w8a16-preds:
  Total samples:         2077
  Correct (exact match): 1178
  Accuracy:              0.5672 (56.72%)
  Norm Edit Distance:    0.7941
```

#### TODO

- [x] ax630c performance benchmark
- [x] ax615 performance benchmark