spybyscript/gliner25-base-litert

GLiNER2.5 base (gliner2.5-base-v1) — LiteRT (Android)

A LiteRT conversion of fastino/gliner2.5-base-v1 (revision 1a8bc24e00dc7300b9017c81d63e3dcdabb26596, Apache-2.0, by Fastino), converted by spybyscript. It carries every trained head: entities, classification, structures, relations and records. gliner2's own code decodes the outputs. For the model itself (training, labels, benchmarks, intended use), see the original model card, copied here as model-card.md. The split into graphs follows FluidInference's Core ML port (gliner2-5-base-coreml, Apache-2.0).

Files

File	Size	Contents
gliner2_base_encoder_wfp16.tflite	189 MB	encoder for the CPU and the GPU at FP32 (inputs_embeds, attention_mask → hidden_states), float16-stored weights
gliner2_base_encoder_fp16hw.tflite	189 MB	the same encoder rewritten for FP16 hardware (NPU): LayerNorm power-of-two pre-scale, first feed-forward layer split in two
npu/qualcomm/gliner2_base_encoder_fp16hw_<chip>.tflite	182 MB each	fp16hw precompiled for one Snapdragon NPU (below)
gliner2_base_head_fp32.tflite	8.5 MB	boundary head: routed states, boundary marginals, pool projections, null and count heads, classification logits
gliner2_base_scorer_fp32.tflite	2.1 MB	candidate scorer and record candidate states
gliner2_base_explicit_fp32.tflite	2.1 MB	explicit-span scorer (entity attributes, choice fields)
gliner2_base_relation_fp32.tflite	23.7 MB	relation scorer over head/tail pairs
gliner2_base_record_assignment_fp32.tflite	1.2 MB	record decoder: instance-to-field assignment, object and latent-seed scores
gliner2_base_record_anchorless_fp32.tflite	2.9 MB	record decoder for anchorless records
gliner2_base_embeddings_f16.bin	197 MB	token table, float16 [128011, 768], little-endian, row-major
runtime/		litert_boundary_runtime.py (reference runtime), verify_parity.py, parity_corpus.json

config.json lists every graph's signatures and shapes.

Use

# pip install gliner2==2.0.0 torch numpy ai-edge-litert
import sys; sys.path.insert(0, "runtime")
from litert_boundary_runtime import LiteRTBoundaryExtractor
from gliner2 import Schema

model = LiteRTBoundaryExtractor(".")          # this repository's folder
model.extract_entities("Alice founded Acme in Toronto in 2020.", ["person", "organization", "city"])
model.extract_relations("Alice founded Acme in Toronto.", ["founded", "located_in"])
schema = Schema().entities(["person", "organization"]).relations(["works_for"])
model.extract("Ravi Shah is an engineer at Tidewater Systems.", schema, include_confidence=True)

Every public gliner2 call works: extract, extract_entities, classify_text, extract_json and extract_relations, plus their batch_* and *_long forms. The runtime runs gliner2's BoundaryExtractor with every trained module on LiteRT. Schema handling, the candidate pool and the decoders are the library's own code.

On a phone, run the same graphs with LiteRT's CompiledModel; the runtime shows the host steps.

• The encoder is the only graph worth an accelerator. Use the NPU with your chip's precompiled file from npu/ (or fp16hw, which the phone compiles in about 6 s), the GPU with wfp16 at FP32 (or fp16hw at FP16 where it stays finite), or the CPU.
• The six head graphs run on the CPU in 1–16 ms each.

Fixed sizes. The original runs with dynamic shapes; these are FluidInference's bucket sizes. One request holds at most:

• 128 subword tokens (schema and text) and 64 text words;
• 8 queries (entity labels, structure fields, relation roles) and 8 classification choices;
• a candidate pool of 192 (the checkpoint's own);
• 4 relation types with 256 head/tail pairs;
• 64 explicit spans per query;
• records of up to 8 fields with 192 candidates each.

A larger request is refused with a ValueError, never truncated. Run long text through the *_long calls, for example extract_entities_long(text, labels, chunk_size=40, chunk_overlap=8). The weights change only in storage and compute precision.

Validation

runtime/verify_parity.py runs each request through gliner2 with the original weights and through the reference runtime (LiteRT CPU), then compares the structured results:

• The corpus (runtime/parity_corpus.json): 296 requests over 16 call types (entities, relations, structures, the three record modes, attributes, single- and multi-label classification, mixed schemas). All 296 are identical, with span F1 1.0 and confidences within 0.0014.
• FluidInference's eleven schema-path fixtures: 11 of 11.

validation-full.json and validation.json hold the numbers.

Phones

LiteRT 2.3.2 with qnn-runtime 2.50.0. Each graph ran on a request recorded from the corpus: 10 calls after a rest, at thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as whole calls. The NPU compiled the graphs on the phone; for the encoder that takes about 6 s, once.

Graph	Galaxy Z Fold 8 (SM8850): CPU / GPU / NPU	Galaxy S23 Ultra (SM8550): CPU / GPU / NPU
encoder, 128 tokens	38 / 37 (FP32), 20 (FP16) / 9.9 ms	143 / 67 (FP32) / 13 ms
head	0.7 / — / 0.6 ms	1.2 / — / 1.2 ms
scorer	2.6 / — / does not run	2.9 / — / 3.4 ms
explicit spans	5.4 / — / does not run	5.6 / — / does not run
relations	4.9 / — / 1.3 ms	16 / — / 2.8 ms
record assignment	6.0 / not measured / 4.3 ms	8.9 / 21.5 ms (LOW priority) / 4.0 ms
anchorless records	2.9 / — / 0.6 ms	10.8 / — / 1.3 ms

— means LiteRT's GPU does not support the graph; run it on the CPU.

• The precompiled encoder (npu/qualcomm/gliner2_base_encoder_fp16hw_SM8850.tflite) on the Fold: 8.1 ms a call, loaded in 0.2 s, with 31 of 32 recorded requests unchanged (below), as with the encoder the phone compiled itself.
• A request with the encoder on the NPU and the heads on the CPU takes about 15 ms on the Fold and 20 ms on the S23. On the CPU alone it takes about 45 ms on the Fold and 150 ms on the S23.
• Answers with the phones' encoder outputs. 32 recorded requests, two per call type, ran through the rest of the pipeline:
  • GPU FP32: unchanged on both phones;
  • NPU: 31 of 32 on the Fold and 30 of 32 on the S23 (only latent-record requests changed);
  • fp16hw at GPU FP16: 31 of 32 on the Fold. On the S23's Adreno 740 it gives non-finite outputs, with or without FP32 accumulation, so use FP32 there.
• Dirty memory, encoder: about 0.45 GB on the CPU, 0.6 GB on the GPU at FP16 and 0.85 GB at FP32. The NPU holds about 0.5 GB after compiling on the phone.
• GPU priority depends on the GPU. LOW priority is meant to let other GPU work go first. On the Fold's Adreno 840 it keeps an app's rendering smooth; on the S23's Adreno 740 it does not, and only slows each call. The measurements are with GLiNER2.5-Decide's encoder: see spybyscript/gliner25-decide-litert.

Qualcomm NPU (npu/)

Runtime:

• LiteRT 2.3.2 with litert-npu-runtime-qualcomm 2.3.2 and Qualcomm's qnn-runtime 2.50.0 (Qualcomm's AI Stack License);
• files precompiled on x86-64 Linux with QAIRT 2.50.0, each in one NPU partition (930 of 930 operators);
• a file runs only on its chip (Build.SOC_MODEL) and with these runtime versions.

Each file is gliner2_base_encoder_fp16hw.tflite (signature t128) compiled ahead of time, so the phone loads it without compiling. npu/manifest.json lists every file's chip, size and SHA-256. Keep the head graphs on the CPU.

Chip	Snapdragon	Hexagon	File	Run on a phone
SM8850	8 Elite Gen 5	V81	npu/qualcomm/gliner2_base_encoder_fp16hw_SM8850.tflite	Galaxy Z Fold 8
SM8750	8 Elite	V79	npu/qualcomm/gliner2_base_encoder_fp16hw_SM8750.tflite	not yet
SM8650	8 Gen 3	V75	npu/qualcomm/gliner2_base_encoder_fp16hw_SM8650.tflite	not yet
SM8550	8 Gen 2	V73	npu/qualcomm/gliner2_base_encoder_fp16hw_SM8550.tflite	not yet
SM8450	8 Gen 1	V69	npu/qualcomm/gliner2_base_encoder_fp16hw_SM8450.tflite	not yet

Files

• LICENSE (Apache-2.0) and NOTICE.txt
• model-card.md, the original card
• config.json, the contract
• validation.json and validation-full.json
• SHA256SUMS
• tokenizer/
• source/ (the checkpoint's configs)
• runtime/
• npu/ (the precompiled encoder per chip and npu/manifest.json)