FatimahEmadEldin/KnowledgeGraphEval-2026-AdaptNER-AraBERTv02-Flat

NAMAA Community — AdaptNER (AraBERTv02, flat baseline)

KnowledgeGraphEval 2026 · Subtask 1 (AdaptNER) · ArabicNLP 2026

Single-label (flat) Arabic Named Entity Recognition. This checkpoint fine-tunes aubmindlab/bert-base-arabertv02 on the flat variant of the Wojood corpus with a standard softmax BIO tagger over 21 entity types.

It is published as the team's flat control baseline — the reference point that quantifies what the nested formulation buys. It is deliberately not a competitive system: a flat tagger assigns exactly one label per token and therefore cannot represent overlapping mentions, which is the defining property of the Wojood/Konooz annotation scheme.

How this checkpoint relates to the paper. This is a development-phase checkpoint from the NAMAA submission to KnowledgeGraphEval 2026. It is not the system described in the camera-ready paper. The submitted AdaptNER system, which ranked 1st with a test micro-F1 of 75.96, is an exact-span majority vote over three models: ARBERTv2 + CRF, XLM-R-large + CRF, and ARBERTv2 + GlobalPointer. See §6. We release this checkpoint because the team's in-domain Wojood experiments were built on it.

Which checkpoint should I use? For real work, use KnowledgeGraphEval-2026-AdaptNER-AraBERTv02-Nested. It shares this model's encoder and recipe, but scores 0.73 vs 0.69 on the official Konooz development set and can express nested entities. Use this checkpoint when you specifically want a flat tagger, or to reproduce the flat-vs-nested ablation.

---

1. Model details

Developed by	NAMAA Community (see Citation)
Team / submission	NAMAA at KnowledgeGraphEval 2026 (development-phase checkpoint)
Experiment ID	EXP-001 (flat baseline)
Model type	BERT encoder + single-label token classification head (softmax)
Base encoder	aubmindlab/bert-base-arabertv02
Language	Arabic (MSA + Palestinian/Lebanese dialect)
Parameters	134,635,050 (F32)
Architecture	12 layers, hidden 768, 12 heads, vocab 64,000, max positions 512
Label space	42 classes: 21 B- + 20 I- + O (see caveat)
Decision rule	argmax over 42 classes
Max sequence length	256 sub-words
License	Apache-2.0
Finetuned from	AraBERTv02 (Antoun et al., 2020)

---

2. Intended use

In scope

• Flat Arabic NER where one label per token is sufficient.
• The team's development-phase flat-vs-nested ablation.
• A drop-in token-classification pipeline model (works with aggregation_strategy="simple", unlike the nested siblings).

Out of scope

• Nested / overlapping entities. Structurally impossible — this is the point of the baseline.
• Multi-token UNIT mentions — the label space has no I-UNIT class (see below).
• Languages other than Arabic.
• High-stakes decisions about people or organisations without human review.

---

3. Task and data

KnowledgeGraphEval 2026 (Aljabari et al., 2026) is the first ArabicNLP shared task on Arabic knowledge-graph construction. It splits the problem into Subtask 1 (AdaptNER) — cross-domain nested NER — and Subtask 2 (RE) — relation extraction. Both are scored with strict entity-level micro-F1.

Role	Dataset	Size	Notes
Training / model selection	Wojood 1.1, flat variant (Jarrar et al., 2022)	23,125 train / 3,304 dev sentences	~448K tokens, ~72K mentions, 21 entity types
Official cross-domain evaluation	Konooz (Hamad et al., 2025)	~50K tokens, 10 unseen MSA domains	Dev release: 100 sentences (10 per domain), 3,166 tokens, unlabelled

The model never sees Konooz during training — that domain shift is the challenge the task targets.

Konooz domains: Agriculture, Art, Economics, Finance, Health, History, Law, Politics, Science, Sport.

Note that the flat variant is a lossy projection of the nested annotation: where the nested data records a token as belonging to several mentions, the flat data keeps one. Metrics on this split are therefore not directly comparable to nested-split metrics.

Label space

21 entity types, in the official submission column order:

CARDINAL, CURR, DATE, EVENT, FAC, GPE, LANGUAGE, LAW, LOC, MONEY, NORP,
OCC, ORDINAL, ORG, PERCENT, PERS, PRODUCT, QUANTITY, TIME, UNIT, WEBSITE

Caveat — I-UNIT is absent. The head has 42 classes, not the 43 (21 B- + 21 I- + O) that a complete flat BIO scheme requires: I-UNIT never appeared in the flat training split, so it was never allocated a class. Consequently multi-token UNIT mentions cannot be predicted — only single-token ones. The nested siblings do not have this gap (they carry all 21 B-/I- pairs).

---

4. Training configuration

Recorded verbatim in repro_bundle.json.

Hyperparameter	Value
Epochs	8 (best checkpoint at epoch 8.0)
Learning rate	3e-5
Batch size (train / eval)	32 / 64
Gradient accumulation	1
Optimizer	AdamW
Weight decay	0.01
Warmup ratio	0.1
Max length	256
Precision	bf16 (+ TF32)
Early stopping	patience 3, monitor dev micro-F1
Seed	42
group_by_length	true
label_all_subtokens	false (label on first sub-word; continuations masked)

Environment — Python 3.12.13, PyTorch 2.11.0+cu128, Transformers 5.10.2, Datasets 4.0.0, CUDA 12.8, Tesla T4. Arabic normalisation via CAMeL Tools (Obeid et al., 2020).

---

5. Evaluation

5.1 Methodology

Entity-level strict micro-F1: a prediction counts as correct only if the span boundaries and the entity type match the gold mention exactly.

micro-F1 = 2TP / (2TP + FP + FN)

5.2 Overall results

Split	Loss	Precision	Recall	micro-F1
Wojood validation (in-domain)	0.1356	0.8908	0.9101	0.9003
Konooz dev (cross-domain, official)	—	—	—	0.69

Scope of reporting. This run recorded validation metrics only. Unlike the nested siblings, no test_metrics.json and no test-set prediction file were produced for this checkpoint, so no Wojood test-split score and no per-entity-type breakdown are available. The gap is a reporting gap, not a hidden result — the numbers were never computed. The per-type breakdown in the nested cards should be consulted for type-level behaviour.

Note that precision (0.8908) trails recall (0.9101) here, the opposite balance from the nested AraBERTv02 sibling — the flat tagger over-predicts slightly relative to what it recovers.

5.3 Per-domain breakdown (Wojood validation)

Sub-corpus	Sentences	F1
Awraq_G3	714	0.9703
T4: Elections	161	0.9389
Awraq_G1	466	0.9380
Awraq_G2	490	0.9379
T5: Migration	65	0.8862
T1: Terrorism	138	0.8525
PCMA	105	0.8419
T6: Health policy	108	0.8399
Curras	523	0.8204
T2: Foreign policy	144	0.8058
T3: Security	83	0.8053
QUORA	265	0.6833
Lebanese	42	0.5882

The spread is wide — 0.59 to 0.97, a 0.38 F1 range across sub-corpora of the same corpus. Dense formulaic archival text (Awraq_G3) is near-solved; dialectal and user-generated slices (Lebanese 0.5882, QUORA 0.6833) are the floor. Lebanese has only 42 validation sentences, so that figure is noisy.

5.4 Flat vs nested: what the nested head buys

Same encoder, same seed, same hyperparameters — only the task head and data variant differ:

Sub-corpus	Flat F1 (this model)	Nested F1 (AraBERTv02 nested)	Δ
Awraq_G1	0.9380	0.9520	+0.0140
Awraq_G2	0.9379	0.9623	+0.0244
Awraq_G3	0.9703	0.9833	+0.0130
Curras	0.8204	0.8205	+0.0001
Lebanese	0.5882	0.5600	−0.0282
PCMA	0.8419	0.8802	+0.0383
QUORA	0.6833	0.6833	0.0000
T1: Terrorism	0.8525	0.9020	+0.0495
T2: Foreign policy	0.8058	0.8026	−0.0032
T3: Security	0.8053	0.8571	+0.0518
T4: Elections	0.9389	0.9418	+0.0029
T5: Migration	0.8862	0.8991	+0.0129
T6: Health policy	0.8399	0.8590	+0.0191
Overall (validation)	0.9003	0.9245	+0.0242
Konooz dev (development phase)	0.69	0.73	+0.04

Both columns are validation-split figures on their respective data variants, so the comparison is indicative rather than strict. The nested head wins on 10 of 13 sub-corpora and by +0.04 on the official cross-domain metric. A flat tagger cannot emit mentions that overlap across entity types, which limits its recall on nested annotation.

---

6. Official shared-task results

These are the official test results for team NAMAA, as reported in the camera-ready paper. All scores are entity-level micro-averages under exact span-and-type matching, in percent. This checkpoint is not one of the submitted systems.

Subtask 1 — AdaptNER (Konooz test, 10 unseen domains). Ranked 1st.

System	Recall	Precision	Micro-F1
Shared-task baseline	63.87	62.30	63.00
ARBERTv2 + CRF	70.08	80.69	75.01
XLM-R-large + CRF	71.12	79.41	75.04
ARBERTv2 + GlobalPointer	66.17	84.02	74.03
Exact-span ensemble (≥2 of 3 votes)	69.44	83.83	75.96

Subtask 2 — Relation Extraction (WojoodRelations, 41 labels). Ranked 2nd.

System	Recall	Precision	Micro-F1
Shared-task baseline	–	–	93.09
Qwen3-8B (LoRA)	89.35	95.59	92.36
Qwen3.5-4B (LoRA)	89.58	96.20	92.77
Llama-3.1-8B-Instruct (LoRA)	89.03	96.21	92.48
Majority-vote ensemble	90.38	96.80	93.48

Development-phase Konooz results

During development the team scored its candidate encoders on the Konooz development set (100 sentences, entity-level micro-F1). These scores are not in the camera-ready paper. We include them here because they are the only cross-domain numbers available for this checkpoint.

System	Konooz dev F1	System	Konooz dev F1
ARBERTv2 (unweighted)	0.74	MARBERT	0.68
Top-3 weighted-vote ensemble	0.74	mDeBERTa-v3 base	0.67
AraBERTv02, nested	0.73	MARBERTv2 ‡	0.67
ARBERTv2 (class-weighted)	0.72	ARBERTv2 + Konooz-MLM (last-3)	0.65
XLM-R large	0.71	AraBERTv02 organiser baseline	0.61
AraBERTv02, flat	0.69	Qwen3-235B (zero-shot)	0.26

‡ This MARBERTv2 entry may be a different run from the released MARBERTv2-Nested checkpoint. That entry's in-domain validation F1 was 0.873, while the released checkpoint's is 0.9071.

---

7. Sibling checkpoints

Checkpoint	Formulation	Wojood dev F1	Wojood test F1	Macro-F1 (test)	Konooz dev F1
AraBERTv02 nested	nested multi-label	0.9245	0.9284	0.8411	0.73
MARBERTv2 nested	nested multi-label	0.9071	0.9092	0.5877	0.67
AraBERTv02 flat (this model)	flat single-label	0.9003	not recorded	not recorded	0.69

Dev F1 for the flat row is on the flat data variant; the nested rows are on the nested variant.

---

8. Limitations and known issues

• Cannot represent nested entities. One label per token, by construction. On a corpus where 22.5% of mentions are nested, this is a hard ceiling, and it is the reason the checkpoint exists as a control rather than a candidate.
• No I-UNIT class. Multi-token UNIT mentions are unpredictable (see §3).
• No test-split or per-type metrics. Never computed for this run (see §5.2).
• Cross-domain degradation. 0.9003 in-domain validation to 0.69 on Konooz.
• Dialect and user-generated text. Lebanese (0.5882) and QUORA (0.6833) trail the archival sub-corpora by roughly 0.30 F1.
• Small-sample domains. Lebanese (42 sentences), T5: Migration (65) and T3: Security (83) give noisy per-domain estimates.
• Column order matters. Submissions are token + 21 space-separated B/I/O columns in the exact order listed in §3. A mis-ordered file scores against the wrong channels.

---

9. Usage

Unlike the nested siblings, this is an ordinary single-label tagger, so the standard pipeline works directly:

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="NAMAA-Space/KnowledgeGraphEval-2026-AdaptNER-AraBERTv02-Flat",
    aggregation_strategy="simple",
)

for ent in ner("مرحبا بك في جامعة بيرزيت في فلسطين"):
    print(f"{ent['entity_group']:<10} {ent['word']}  ({ent['score']:.3f})")

For explicit control over sub-word alignment:

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

REPO = "NAMAA-Space/KnowledgeGraphEval-2026-AdaptNER-AraBERTv02-Flat"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForTokenClassification.from_pretrained(REPO).eval()

def predict(words):
    enc = tok(words, is_split_into_words=True, truncation=True,
              max_length=256, return_tensors="pt")
    with torch.no_grad():
        pred = model(**enc).logits.argmax(-1)[0]

    out, seen = [], set()
    for pos, wid in enumerate(enc.word_ids(0)):   # first sub-word of each word only
        if wid is None or wid in seen:
            continue
        seen.add(wid)
        out.append((words[wid], model.config.id2label[pred[pos].item()]))
    return out

for word, tag in predict("وقع الاتفاق في القدس عام 1993".split()):
    print(f"{word}\t{tag}")

---

10. Reproducibility

File	Contents
repro_bundle.json	Full config, resolved data paths, label set, seed, library versions
final_metrics.json	Wojood validation overall + per-domain F1
training_args.bin	Serialised TrainingArguments

No test-split metrics or prediction file exist for this run (§5.2).

Training notebooks, per-run configs, development-phase Konooz predictions and the evaluation scripts are in KnowledgeGraphEval-2026-AdaptNER-Experiments.

---

11. Citation

This system

@inproceedings{namaa-knowledgegrapheval-2026,
  title     = {{NAMAA} at {KnowledgeGraphEval} 2026: Entity-Span Voting and Constrained
               {LLM} Ensembles for {A}rabic Knowledge Graph Construction},
  author    = {Fathallah, Mahmoud and Zaytoon, Mohamed and Emad Eldin, Fatimah and
               A.azim, Mona and Nacar, Omer},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  address   = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

Affiliations: Alexandria University; Trouve Labs; Ain Shams University; Tuwaiq Academy; NAMAA Community.

Shared task

@inproceedings{aljabari-etal-2026-knowledgegrapheval,
  title     = {{KnowledgeGraphEval} 2026: The First Shared Task on {A}rabic Knowledge Graphs},
  author    = {Aljabari, Alaa and Hamad, Nagham and El Mekki, Abdellah and
               Abdul-Mageed, Muhammad and Zitouni, Imed and Chawla, Sanjay and Jarrar, Mustafa},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  publisher = {Association for Computational Linguistics}
}

Training corpus — Wojood

@inproceedings{jarrar-etal-2022-wojood,
  title     = {{W}ojood: Nested {A}rabic Named Entity Corpus and Recognition using {BERT}},
  author    = {Jarrar, Mustafa and Khalilia, Mohammed and Ghanem, Sana},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference (LREC)},
  year      = {2022},
  address   = {Marseille, France},
  pages     = {3626--3636},
  publisher = {European Language Resources Association},
  url       = {https://aclanthology.org/2022.lrec-1.387/}
}

Evaluation corpus — Konooz

@inproceedings{hamad-etal-2025-konooz,
  title     = {{K}onooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition},
  author    = {Hamad, Nagham and Khalilia, Mohammed and Jarrar, Mustafa},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  year      = {2025},
  pages     = {7316--7331},
  publisher = {Association for Computational Linguistics},
  url       = {https://aclanthology.org/2025.findings-acl.382/}
}

Base encoder — AraBERT

@inproceedings{antoun-etal-2020-arabert,
  title     = {{A}ra{BERT}: Transformer-based Model for {A}rabic Language Understanding},
  author    = {Antoun, Wissam and Baly, Fady and Hajj, Hazem},
  booktitle = {Proceedings of the 4th Workshop on Open-Source Arabic Corpora and
               Processing Tools (OSACT), with a Shared Task on Offensive Language Detection},
  year      = {2020},
  address   = {Marseille, France},
  pages     = {9--15},
  publisher = {European Language Resources Association},
  url       = {https://aclanthology.org/2020.osact-1.2/}
}

Also referenced

@inproceedings{obeid-etal-2020-camel,
  title     = {{CAM}e{L} Tools: An Open Source Python Toolkit for {A}rabic
               Natural Language Processing},
  author    = {Obeid, Ossama and Zalmout, Nasser and Khalifa, Salam and Taji, Dima and
               Oudah, Mai and Alhafni, Bashar and Inoue, Go and Eryani, Fadhl and
               Erdmann, Alexander and Habash, Nizar},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC)},
  year      = {2020},
  pages     = {7022--7032},
  url       = {https://aclanthology.org/2020.lrec-1.868/}
}

---

12. Acknowledgements

We thank Tuwaiq Academy for its continued support of this work, and the organisers of KnowledgeGraphEval 2026 for providing the Wojood, Konooz and WojoodRelations resources and the evaluation framework.

Wojood and Konooz are distributed by SinaLab, Birzeit University under their own terms. The Apache-2.0 license here covers the fine-tuned weights and code, not the underlying corpora — users must obtain those under the original licenses.