Text Classification
Transformers
Safetensors
English
Polish
qwen3_5_text
text-generation
nvfp4
fp4
compressed-tensors
vllm
quantized
tentaguard
guard
security
prompt-injection
tentaflow
Instructions to use TentaFlow/TentaGuard-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TentaFlow/TentaGuard-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="TentaFlow/TentaGuard-NVFP4")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("TentaFlow/TentaGuard-NVFP4") model = AutoModelForCausalLM.from_pretrained("TentaFlow/TentaGuard-NVFP4", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -22,49 +22,50 @@ tags:
|
|
| 22 |
|
| 23 |
# TentaGuard — NVFP4 (W4A4, vLLM)
|
| 24 |
|
| 25 |
-
**TentaGuard**
|
| 26 |
-
[`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B).
|
| 27 |
-
[TentaFlow](https://github.com/Slyb00ts/TentaFlow)**
|
| 28 |
-
|
| 29 |
-
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
-
|
|
| 34 |
-
|-------
|
| 35 |
-
| `0` |
|
| 36 |
-
| `1` | prompt injection /
|
| 37 |
-
| `2` | jailbreak (
|
| 38 |
|
| 39 |
-
|
| 40 |
|
| 41 |
-
##
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
|
| 46 |
-
##
|
| 47 |
|
| 48 |
-
- Exact (0/1/2): **~96.6%** (
|
| 49 |
-
- Safe/Unsafe: **~98.3%**
|
| 50 |
|
| 51 |
-
##
|
| 52 |
|
| 53 |
-
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
Apache-2.0,
|
| 58 |
-
|
| 59 |
|
| 60 |
-
##
|
| 61 |
|
| 62 |
-
|
| 63 |
-
FP8 E4M3 +
|
| 64 |
-
[`llm-compressor`](https://github.com/vllm-project/llm-compressor)
|
|
|
|
| 65 |
|
| 66 |
-
NVFP4
|
| 67 |
-
|
| 68 |
|
| 69 |
```bash
|
| 70 |
vllm serve TentaFlow/TentaGuard-NVFP4
|
|
|
|
| 22 |
|
| 23 |
# TentaGuard — NVFP4 (W4A4, vLLM)
|
| 24 |
|
| 25 |
+
**TentaGuard** is a lightweight security classifier (guard) — a fine-tune of
|
| 26 |
+
[`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B). It is used **mainly inside the
|
| 27 |
+
[TentaFlow](https://github.com/Slyb00ts/TentaFlow) application** to scan external content — messages, documents,
|
| 28 |
+
web-search results, etc. — for **hidden attacks** (prompt injection / jailbreak) before it
|
| 29 |
+
reaches the main LLM.
|
| 30 |
|
| 31 |
+
The model does NOT generate user-facing replies — it returns a single digit:
|
| 32 |
|
| 33 |
+
| Label | Meaning |
|
| 34 |
+
|-------|---------|
|
| 35 |
+
| `0` | benign (safe content) |
|
| 36 |
+
| `1` | prompt injection / tool abuse (technical attack) |
|
| 37 |
+
| `2` | jailbreak (behavioural manipulation) |
|
| 38 |
|
| 39 |
+
If the text contains BOTH injection and jailbreak → `1`.
|
| 40 |
|
| 41 |
+
## Input format
|
| 42 |
|
| 43 |
+
A classifier system prompt + a user message `<|guard|>\n{text}`. **Build the prompt with the
|
| 44 |
+
model tokenizer (`apply_chat_template`)** — do not rely on a generic chat template.
|
| 45 |
|
| 46 |
+
## Accuracy (guard test set, 58 examples)
|
| 47 |
|
| 48 |
+
- Exact (0/1/2): **~96.6%** (full precision) / **~94.8%** (Q5_K_M)
|
| 49 |
+
- Safe / Unsafe: **~98.3%**
|
| 50 |
|
| 51 |
+
## Authors
|
| 52 |
|
| 53 |
+
Trained by: **Katarzyna Nowak**, **Piotr Jarocki**, **Damian Pala**, **Jakub Rurański**.
|
| 54 |
|
| 55 |
+
## License & attribution
|
| 56 |
|
| 57 |
+
Apache-2.0, inherited from the base model [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B).
|
| 58 |
+
This checkpoint is a fine-tune for attack detection, built for the [TentaFlow](https://github.com/Slyb00ts/TentaFlow) application.
|
| 59 |
|
| 60 |
+
## Usage (vLLM)
|
| 61 |
|
| 62 |
+
`compressed-tensors` format (`nvfp4-pack-quantized`): 4-bit weights (FP4 E2M1, groups of 16,
|
| 63 |
+
FP8 E4M3 block scales + a global FP32 scale), 4-bit activations (W4A4), `lm_head` kept in full
|
| 64 |
+
precision. PTQ calibration via [`llm-compressor`](https://github.com/vllm-project/llm-compressor)
|
| 65 |
+
on real guard prompts.
|
| 66 |
|
| 67 |
+
NVFP4 is hardware-accelerated on **Blackwell (sm_100+)**; on older GPUs vLLM loads it as
|
| 68 |
+
**weight-only** (smaller VRAM, no FP4 acceleration).
|
| 69 |
|
| 70 |
```bash
|
| 71 |
vllm serve TentaFlow/TentaGuard-NVFP4
|