zeromodels/table-transformer-structure-recognition-v1.1-fin

See our collection for all versions of Table Transformer.

Run Table Transformer with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/table-transformer-structure-recognition-v1.1-fin

Paper: PubTables-1M: Towards comprehensive table extraction from unstructured documents (arXiv:2110.00061) · HF Papers

Table Transformer (TATR) applies the DETR detection recipe to tables. A ResNet-18 backbone produces a feature map, a transformer encoder-decoder attends over it with a fixed set of learned object queries, and each query emits one class and one box, with no anchors and no NMS. The v1.1 structure model fine-tuned on FinTabNet (financial tables).

For more details on the model, please go to Microsoft's original model card.

Pure-Keras 3 conversion of microsoft/table-transformer-structure-recognition-v1.1-fin for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a table-structure recognition checkpoint (TableTransformerDetect): each query predicts a table element (classes: table, column, row, column header, projected row header, spanning cell).

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from PIL import Image
from zeromodels.models.table_transformer import (
    TableTransformerDetect,
    TableTransformerImageProcessor,
)
from zeromodels.models.table_transformer.table_transformer_image_processor import (
    TABLE_STRUCTURE_LABELS,
)

model = TableTransformerDetect.from_weights("zeromodels/table-transformer-structure-recognition-v1.1-fin")
processor = TableTransformerImageProcessor.from_weights("zeromodels/table-transformer-structure-recognition-v1.1-fin")

image = Image.open("your_image.jpg").convert("RGB")
inputs = processor(image)
output = model(inputs["pixel_values"], training=False)
results = processor.post_process_object_detection(
    output,
    threshold=0.6,
    target_sizes=[(image.height, image.width)],
    label_names=TABLE_STRUCTURE_LABELS,
)[0]
for score, name, box in zip(
    results["scores"], results["label_names"], results["boxes"]
):
    print(f"{name}: {float(score):.3f} {box}")

Load any Table Transformer variant the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Task
table-transformer-detection	zeromodels/table-transformer-detection	table detection
table-transformer-structure-recognition	zeromodels/table-transformer-structure-recognition	table-structure recognition
table-transformer-structure-recognition-v1.1-all	zeromodels/table-transformer-structure-recognition-v1.1-all	table-structure recognition
table-transformer-structure-recognition-v1.1-fin	zeromodels/table-transformer-structure-recognition-v1.1-fin	table-structure recognition
table-transformer-structure-recognition-v1.1-pub	zeromodels/table-transformer-structure-recognition-v1.1-pub	table-structure recognition

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Detection finds tables in a page image; structure recognition decomposes a cropped table into rows, columns, and header / spanning cells.
• Both tasks use TableTransformerDetect + post_process_object_detection; only the checkpoint and the class set differ.
• Pass label_names=TABLE_DETECTION_LABELS for the detection checkpoint and label_names=TABLE_STRUCTURE_LABELS for structure recognition (both live in zeromodels.models.table_transformer.table_transformer_image_processor).
• See Table Transformer docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. TableTransformerDetect.from_weights("hf:microsoft/table-transformer-structure-recognition-v1.1-fin").

Special Thanks

A huge thank you to the Microsoft Table Transformer authors for creating and releasing these models.

License: Apache 2.0.