Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,139 @@
+---
+license: apache-2.0
+base_model: mistralai/Ministral-3-8B-Instruct-2512
+tags:
+  - microsoft-365
+  - intune
+  - entra
+  - defender
+  - sysadmin
+  - local-first
+language:
+  - en
+---
+
+# OpenAdmin 8B
+
+An open-weight model for Microsoft 365 administration — Intune, Entra and
+Defender — fine-tuned from Ministral 3 8B by the
+[OpenAdminOS](https://openadminos.com) community.
+
+**4.9 GB at Q4_K_M.** Runs on an 8 GB machine, CPU-only at roughly 14-16
+tokens/second on a mini-PC. No GPU required.
+
+## What it is for
+
+Grounded, read-only administrative work: answering questions about your tenant
+from documentation and Graph data, planning API calls with least-privilege
+scopes, and refusing to act on destructive requests. It is the local tier of a
+two-model setup; complex multi-step agentic work escalates to OpenAdmin 20B or
+a hosted provider.
+
+## What we optimised for, and why
+
+Most of the effort went into three behaviours that matter more than raw scores
+when a model advises someone with production tenant access:
+
+- **It does not invent Microsoft features.** Asked about a setting that does
+  not exist, it says so instead of producing a plausible default value,
+  licensing requirement and portal path. An earlier checkpoint scored our
+  highest number and did exactly that; it was not released.
+- **It refuses to make changes, for the right reason.** Destructive requests
+  get a safety refusal naming the blast radius and pointing at the
+  confirmation flow — not "I couldn't find that", which an earlier checkpoint
+  produced and which is safe but incoherent.
+- **It answers ordinary questions directly.** Definitions and comparisons are
+  answered from knowledge; version numbers and limits it is unsure of are
+  deferred to documentation.
+
+## Evaluation
+
+Scored mechanically — schema validation, exact match, regex constraints, tool
+call verification. No LLM judges. Full harness, task set and results are
+public in the [pipeline repository](https://github.com/OpenAdminOS/OpenAdminOS/tree/main/model).
+
+### Behaviour suite (162 tasks, never used for training decisions)
+
+| category | tasks | score |
+|---|---|---|
+| Abstention — refuses to invent | 24 | 23 / 24 |
+| Abstention — honours exact-reply contracts | 12 | 12 / 12 |
+| Write-safety — with context | 17 | 17 / 17 |
+| Write-safety — bare chat | 17 | 17 / 17 |
+| Identity | 30 | 27 / 30 |
+| Graph call planning | 32 | 27 / 32 |
+| Answer quality — comparisons | 20 | 19 / 20 |
+| Answer quality — data handling | 10 | 8 / 10 |
+| **Total** | **162** | **150 / 162** |
+
+Every category has at least 30 tasks except where noted, and the naming pools
+are disjoint from the training data. This suite exists because our original
+one had categories of 4 and 5 tasks, where a single item was worth 25
+percentage points and told us almost nothing.
+
+The abstention miss is a correct refusal our regex did not match, not a
+fabrication: zero of the 24 invented-feature probes produced an invented
+answer. The identity and data-handling misses occur without a system prompt;
+the shipped prompt (below) states the base model and the local-execution
+facts, and with it all sixteen items of our release smoke script pass.
+
+A higher-scoring checkpoint (154/162) was rejected for this release because
+three of its abstention answers invented default values and licence
+requirements. We would rather ship the model that scores four points lower and
+does not make things up.
+
+### Known limitations
+
+- **Multi-step agentic trajectories are the weak spot.** On long tool-use
+  chains it is measurably behind OpenAdmin 20B. Escalate that work.
+- **It is not a documentation database.** Facts come from retrieval at query
+  time. Without a documentation index in the prompt it answers from training
+  memory, which is frozen at the base model's cutoff. Retrieval is worth
+  roughly 30 tasks on our suite; use it.
+- **Graph planning covers the common Intune, Entra and Defender surface**, not
+  the whole API.
+
+## Usage
+
+```
+ollama run openadminos/openadmin-8b
+```
+
+Or with llama.cpp:
+
+```
+llama-server --model openadmin-8b-Q4_K_M.gguf --ctx-size 16384 --jinja
+```
+
+A default system prompt ships with the Ollama image and is included as
+`system-prompt.txt`. It matters: it tells the model to answer concept questions
+directly and to defer on uncertain specifics.
+
+## Training data
+
+Synthetic and machine-validated before admission. Agent manifests are checked
+against the product's JSON schema, Graph plans against a curated endpoint
+table, and fleet-reasoning arithmetic is computed by the generator so it is
+correct by construction.
+
+**No tenant data. No scraped conversations. No distillation from proprietary
+model APIs.** The dataset is published at
+[OpenAdminOS/openadmin-sft](https://huggingface.co/datasets/OpenAdminOS/openadmin-sft).
+
+## Honest notes on how this was built
+
+Sixteen training runs. Several scored well and were rejected: one fabricated
+documentation, one narrated its reasoning at the user, one answered a request
+to wipe 200 devices with "I can't find that number in the supplied data". Each
+was caught by reading the model's actual output, not by a benchmark.
+
+The run log — including every failure and what it taught us — is published
+alongside the pipeline. We think that record is more useful than a headline
+number, and we have invalidated one of our own headline numbers already when a
+broader task set showed it was a tie.
+
+## Licence
+
+Apache 2.0, matching the base model. Microsoft, Intune, Entra and Defender are
+trademarks of Microsoft Corporation; this project is not affiliated with or
+endorsed by Microsoft or Mistral AI.