File size: 2,238 Bytes
9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 46ff4df 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 1dad23e 9904168 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-0.8B
pipeline_tag: text-classification
library_name: transformers
language:
- en
- pl
tags:
- nvfp4
- fp4
- compressed-tensors
- vllm
- quantized
- tentaguard
- guard
- security
- prompt-injection
- tentaflow
---
# TentaGuard — NVFP4 (W4A4, vLLM)
**TentaGuard** is a lightweight security classifier (guard) — a fine-tune of
[`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B). It is used **mainly inside the
[TentaFlow](https://github.com/Slyb00ts/TentaFlow) application** to scan external content — messages, documents,
web-search results, etc. — for **hidden attacks** (prompt injection / jailbreak) before it
reaches the main LLM.
The model does NOT generate user-facing replies — it returns a single digit:
| Label | Meaning |
|-------|---------|
| `0` | benign (safe content) |
| `1` | prompt injection / tool abuse (technical attack) |
| `2` | jailbreak (behavioural manipulation) |
If the text contains BOTH injection and jailbreak → `1`.
## Input format
A classifier system prompt + a user message `<|guard|>\n{text}`. **Build the prompt with the
model tokenizer (`apply_chat_template`)** — do not rely on a generic chat template.
## Accuracy (guard test set)
- Exact (0/1/2): **~96.6%** (full precision) / **~94.8%** (Q5_K_M)
- Safe / Unsafe: **~98.3%**
## Authors
Trained by: **Katarzyna Nowak**, **Piotr Jarocki**, **Damian Pala**, **Jakub Rurański**.
## License & attribution
Apache-2.0, inherited from the base model [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B).
This checkpoint is a fine-tune for attack detection, built for the [TentaFlow](https://github.com/Slyb00ts/TentaFlow) application.
## Usage (vLLM)
`compressed-tensors` format (`nvfp4-pack-quantized`): 4-bit weights (FP4 E2M1, groups of 16,
FP8 E4M3 block scales + a global FP32 scale), 4-bit activations (W4A4), `lm_head` kept in full
precision. PTQ calibration via [`llm-compressor`](https://github.com/vllm-project/llm-compressor)
on real guard prompts.
NVFP4 is hardware-accelerated on **Blackwell (sm_100+)**; on older GPUs vLLM loads it as
**weight-only** (smaller VRAM, no FP4 acceleration).
```bash
vllm serve TentaFlow/TentaGuard-NVFP4
```
|