README.md

---
pipeline_tag: object-detection
license: apache-2.0
base_model: microsoft/table-transformer-structure-recognition-v1.1-pub
library_name: zeromodels
tags:
- keras
- zeromodels
- table-transformer
- detr
- object-detection
- arxiv:2110.00061
- pytorch
- jax
- tf
---

## ***See [our collection](https://huggingface.co/collections/zeromodels/table-transformer-6a928f000a88c73a41f9411e) for all versions of Table Transformer.***

# Run Table Transformer with Keras 3: JAX, PyTorch, or TensorFlow

[![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-Table%20Transformer-blue)](https://imvision12.github.io/ZeroModels/table_transformer/) [![Collection](https://img.shields.io/badge/HF-Table%20Transformer%20collection-yellow)](https://huggingface.co/collections/zeromodels/table-transformer-6a928f000a88c73a41f9411e)

# zeromodels/table-transformer-structure-recognition-v1.1-pub

Paper: [PubTables-1M: Towards comprehensive table extraction from unstructured documents (arXiv:2110.00061)](https://arxiv.org/abs/2110.00061) · [HF Papers](https://huggingface.co/papers/2110.00061)

Table Transformer (TATR) applies the DETR detection recipe to tables. A ResNet-18 backbone produces a feature map, a transformer encoder-decoder attends over it with a fixed set of learned object queries, and each query emits one class and one box, with no anchors and no NMS. The v1.1 structure model trained on PubTables-1M (scientific tables).

For more details on the model, please go to Microsoft's original [model card](https://huggingface.co/microsoft/table-transformer-structure-recognition-v1.1-pub).

Pure-**Keras 3** conversion of [`microsoft/table-transformer-structure-recognition-v1.1-pub`](https://huggingface.co/microsoft/table-transformer-structure-recognition-v1.1-pub) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.

This is a **table-structure recognition** checkpoint (`TableTransformerDetect`): each query predicts a table element (classes: table, column, row, column header, projected row header, spanning cell).

## ✨ Quick start

```python
import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.table_transformer import (
    TableTransformerDetect,
    TableTransformerImageProcessor,
)
from zeromodels.models.table_transformer.table_transformer_image_processor import (
    TABLE_STRUCTURE_LABELS,
)

model = TableTransformerDetect.from_weights("zeromodels/table-transformer-structure-recognition-v1.1-pub")
processor = TableTransformerImageProcessor.from_weights("zeromodels/table-transformer-structure-recognition-v1.1-pub")

image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
    output,
    threshold=0.6,
    target_sizes=[(image.height, image.width)],
    label_names=TABLE_STRUCTURE_LABELS,
)[0]
for score, name, box in zip(
    results["scores"], results["label_names"], results["boxes"]
):
    print(f"{name}: {float(score):.3f} {box}")
```

Load any Table Transformer variant the same way with `from_weights("zeromodels/<variant>")`:

| Variant | Hub | Task |
|---|---|---|
| `table-transformer-detection` | [`zeromodels/table-transformer-detection`](https://huggingface.co/zeromodels/table-transformer-detection) | table detection |
| `table-transformer-structure-recognition` | [`zeromodels/table-transformer-structure-recognition`](https://huggingface.co/zeromodels/table-transformer-structure-recognition) | table-structure recognition |
| `table-transformer-structure-recognition-v1.1-all` | [`zeromodels/table-transformer-structure-recognition-v1.1-all`](https://huggingface.co/zeromodels/table-transformer-structure-recognition-v1.1-all) | table-structure recognition |
| `table-transformer-structure-recognition-v1.1-fin` | [`zeromodels/table-transformer-structure-recognition-v1.1-fin`](https://huggingface.co/zeromodels/table-transformer-structure-recognition-v1.1-fin) | table-structure recognition |
| `table-transformer-structure-recognition-v1.1-pub` | [`zeromodels/table-transformer-structure-recognition-v1.1-pub`](https://huggingface.co/zeromodels/table-transformer-structure-recognition-v1.1-pub) | table-structure recognition |

## Tips

- Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
- Detection finds tables in a page image; structure recognition decomposes a cropped table into rows, columns, and header / spanning cells.
- Both tasks use `TableTransformerDetect` + `post_process_object_detection`; only the checkpoint and the class set differ.
- Pass `label_names=TABLE_DETECTION_LABELS` for the detection checkpoint and `label_names=TABLE_STRUCTURE_LABELS` for structure recognition (both live in `zeromodels.models.table_transformer.table_transformer_image_processor`).
- See [Table Transformer docs](https://imvision12.github.io/ZeroModels/table_transformer/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
- Community / upstream safetensors still work via the `hf:` prefix, e.g. `TableTransformerDetect.from_weights("hf:microsoft/table-transformer-structure-recognition-v1.1-pub")`.

## Special Thanks

A huge thank you to the Microsoft Table Transformer authors for creating and releasing these models.

License: Apache 2.0.