Instructions to use Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD")
model = AutoModelForMultimodalLM.from_pretrained("Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD

SGLang

How to use Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD with Docker Model Runner:
```
docker model run hf.co/Blackfrost-AI/RED-SNOW-5.3-FLASH-FP4-Local-STD
```

RED-SNOW-5.3-FLASH-FP4-Local-STD / Deployment-Kit /RUNBOOK.md

Blackfrost-AI

Add RED-SNOW 5.3 FLASH FP4-Local STD

7b9ba29 verified 22 days ago

preview code

Raw History Blame Contribute Delete

15.1 kB

RED-SNOW + Cyber-Frost Harness Runbook

Purpose

This runbook deploys RED-SNOW behind an OpenAI-compatible API and connects the Cyber-Frost Harness as the agent runtime. It covers the validated SGLang path and the compatibility work required for another inference engine.

The harness does not become part of the model server. Keep the inference server alive as one service, then start one harness process per authorized task or evaluation session.

1. Deployment architecture

Minimum layout:

GPU inference host
  RED-SNOW weights
  SGLang or another compatible engine
  http://127.0.0.1:8908/v1

Harness controller
  Python 3.11+
  Cyber-Frost Harness
  RED-SNOW harness config
  private task/submission material

Native execution host
  x86-64 Linux
  Docker
  vulnerable task images
  no model-facing access to fixed images or grader secrets

All three roles may run on one trusted machine for local testing. For scored or sensitive runs, separate the GPU server, harness controller, and isolated native task worker.

2. Requirements

Inference host

NVIDIA GPU environment appropriate for the selected RED-SNOW artifact.
SGLang 0.5.20 with CUDA 13 for the validated reference configuration, or another engine that satisfies ENGINE-COMPATIBILITY.md.
The complete BF16 or NVFP4 model directory.
Sufficient GPU memory for the selected tensor-parallel topology and context length.
The model's packaged tokenizer and chat-template files.

The validated RED-SNOW layouts are:

Artifact	Reference topology	Quantization setting	MoE backend
BF16	TP4, EP4	none	`triton`
NVFP4	TP4	`modelopt_fp4`	`flashinfer_cutlass`

Start with a 32,768-token context while validating the complete agent loop. Increase context only after measuring KV-cache headroom, recurrent-state memory, latency, and multi-step tool stability.

Harness controller

Python 3.11 or newer.
Network access to the model's /v1 endpoint.
A local checkout of Deployment-Kit/Cyber-Frost-Harness.
The task generator, private submission client, and required benchmark files when running CyberGym-style evaluation.

Native execution host

Native x86_64 Linux.
Docker with permission granted only to the trusted harness controller.
Required vulnerable task images already pulled.
SSH key and pinned known_hosts entry when using ssh-docker transport.

Do not expose the Docker socket, fixed images, grader data, API keys, or SSH credentials to the model-facing task container.

3. Install the harness

export RED_SNOW_REPO=/opt/red-snow
cd "$RED_SNOW_REPO/Deployment-Kit/Cyber-Frost-Harness"
python3 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/pip install -e .
.venv/bin/python -m unittest discover -s tests -v
export CFH="$RED_SNOW_REPO/Deployment-Kit/Cyber-Frost-Harness/.venv/bin/cfh"

The package has no third-party runtime dependency. The installed command is cfh.

4. Serve RED-SNOW with SGLang

The included launcher supports BF16 and NVFP4 without embedding host-specific paths.

BF16 example

cd "$RED_SNOW_REPO"
export MODEL_PATH=/models/RED-SNOW-5.3-FLASH-BF16
export MODEL_FORMAT=bf16
export SERVED_MODEL_NAME=red-snow
export CUDA_VISIBLE_DEVICES=0,1,2,3
export TP_SIZE=4
export EP_SIZE=4
export HOST=127.0.0.1
export PORT=8908
export CONTEXT_LENGTH=32768

bash Deployment-Kit/serve/serve-red-snow-sglang.sh

NVFP4 example

cd "$RED_SNOW_REPO"
export MODEL_PATH=/models/RED-SNOW-5.3-FLASH-NVFP4
export MODEL_FORMAT=nvfp4
export SERVED_MODEL_NAME=red-snow
export CUDA_VISIBLE_DEVICES=0,1,2,3
export TP_SIZE=4
export EP_SIZE=0
export HOST=127.0.0.1
export PORT=8908
export CONTEXT_LENGTH=32768

bash Deployment-Kit/serve/serve-red-snow-sglang.sh

The launcher enables the GLM reasoning parser, GLM tool-call parser, thinking, BF16 KV cache, FP32 recurrent state, DSA attention, bounded prefill, metrics, and the precision-specific MoE backend.

Optional DFlash speculative decoding is enabled only when DRAFT_MODEL_PATH is set:

export DRAFT_MODEL_PATH=/models/GLM-5.3-Flash-Draft

Do not enable speculative decoding during the first tool-protocol validation. Establish a clean non-speculative baseline first, then enable DFlash and rerun every probe.

Containerized reference pattern

Use the validated image digest for repeatability:

docker run --rm \
  --name red-snow-sglang \
  --gpus all \
  --network host \
  --ipc host \
  --security-opt label=disable \
  --mount type=bind,src=/models/RED-SNOW-5.3-FLASH-NVFP4,dst=/model,readonly \
  --mount type=bind,src="$PWD/Deployment-Kit",dst=/deployment-kit,readonly \
  --env MODEL_PATH=/model \
  --env MODEL_FORMAT=nvfp4 \
  --env SERVED_MODEL_NAME=red-snow \
  --env CUDA_VISIBLE_DEVICES=0,1,2,3 \
  --env TP_SIZE=4 \
  --env EP_SIZE=0 \
  --env HOST=127.0.0.1 \
  --env PORT=8908 \
  --env CONTEXT_LENGTH=32768 \
  --entrypoint /bin/bash \
  lmsysorg/sglang:v0.5.20-cu130@sha256:b27fce60bc5494c118c4910702812bcfa8cee67abcdd1ff8b0902f21647552f4 \
  /deployment-kit/serve/serve-red-snow-sglang.sh

For a persistent deployment, use --restart unless-stopped instead of --rm, or install the provided systemd service template. Bind the API to loopback or a private network and place authentication at the server or reverse-proxy layer.

5. Validate the inference server before connecting the harness

Health and advertised model name

curl --fail --silent --show-error http://127.0.0.1:8908/v1/models

The returned model list must contain the exact value configured as SERVED_MODEL_NAME. The harness rejects a configuration whose model name is absent from /v1/models.

Plain completion

curl --fail --silent --show-error \
  http://127.0.0.1:8908/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model":"red-snow",
    "messages":[{"role":"user","content":"Reply with exactly: ready"}],
    "temperature":0,
    "max_tokens":32
  }'

Confirm a normal assistant message, a terminal finish reason, and non-empty usage fields.

Thinking and tool parsing

Do not judge tool support from a prose prompt. Use the harness's two-turn structured protocol probe after configuring it:

"$CFH" probe-model --config "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.toml"

The probe requires:

one correctly parsed echo_probe tool call;
preservation of its tool-call ID;
one correctly parsed finish_probe call after the tool result;
JSON arguments, not XML, prose, or a serialized tool block in content.

Do not start a benchmark or engagement if this probe fails.

6. Configure the harness

Copy the tracked template to an untracked local file:

cp "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.toml" \
  "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.local.toml"

Edit the local file:

[model].base_url: OpenAI-compatible API ending in /v1.
[model].model: exact ID returned by /v1/models.
[model].api_key_env: name of the environment variable containing the API key.
context and completion limits: values actually accepted by the server.
[runtime]: local-docker or ssh-docker.
benchmark paths: local private paths, never repository placeholders.
[agent].mode: red, blue, or purple.

Set the API key without writing it into TOML:

export RED_SNOW_API_KEY='replace-with-runtime-secret'

If the model server does not enforce an API key, leave the variable unset; the client uses EMPTY as its harmless placeholder.

For an SSH Docker worker, change the runtime block to:

[runtime]
transport = "ssh-docker"
ssh_target = "root@NATIVE-X86-HOST"
ssh_port = 22
ssh_key = "/secure/path/to/id_ed25519"
known_hosts = "/secure/path/to/known_hosts"
container_prefix = "red-snow-cfh"
cpus = 24
memory = "96g"
pids_limit = 256
command_timeout_seconds = 900
network = "none"
sanitizer_compat = true

Keep this edited file outside the upload set.

7. Preflight the complete stack

Run the checks in order:

"$CFH" list-skills --config "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.local.toml"
"$CFH" probe-model --config "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.local.toml"
"$CFH" doctor --config "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.local.toml"

doctor must confirm:

config, prompt, skills, benchmark root, data, and mask map exist;
the configured model ID appears in /v1/models;
the runtime host reports x86_64;
the requested task image exists when a task ID is supplied.

Then exercise the native tool path without submitting to the benchmark oracle:

"$CFH" smoke-task \
  --config "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.local.toml" \
  --task-id oss-fuzz:42537493 \
  --task-dir /private/tasks/libxml2 \
  --run-dir /private/runs/red-snow-smoke-libxml2 \
  --seconds 5

The smoke result must show native x86_64, a selected executable, replay output, a bounded fuzzer run, and preserved artifacts when any are produced.

8. Run one harness session

Start the task's private submission service first. Then run:

"$CFH" run-task \
  --config "$RED_SNOW_REPO/Deployment-Kit/configs/red-snow-harness.local.toml" \
  --task-id oss-fuzz:42537493 \
  --task-dir /private/tasks/libxml2 \
  --run-dir /private/runs/red-snow-libxml2-001 \
  --mode red

Modes:

red: reproduce a triggering input and establish a root-cause hypothesis;
blue: produce a narrow corrective change, regression, and detection guidance;
purple: connect trigger, root cause, repair, regression, and telemetry.

Each run directory preserves the trajectory, model usage, findings ledger, target inventory, candidates, final selection, and environment manifest. Treat it as sensitive engagement material.

9. Run the harness as an operational service

SGLang is a long-running service. The harness is a per-task worker, so use the supplied systemd templates as follows:

sudo install -m 0644 Deployment-Kit/systemd/red-snow-sglang.service.example \
  /etc/systemd/system/red-snow-sglang.service
sudo install -m 0644 Deployment-Kit/systemd/cyber-frost-harness@.service.example \
  /etc/systemd/system/cyber-frost-harness@.service
sudo systemctl daemon-reload
sudo systemctl enable --now red-snow-sglang.service

Create /etc/red-snow/sglang.env for model-server variables and one protected file such as /etc/red-snow/harness/libxml2-001.env for each task. Then:

sudo systemctl start cyber-frost-harness@libxml2-001.service
sudo journalctl -u cyber-frost-harness@libxml2-001.service -f

The per-task unit is intentionally Type=oneshot: one agent session has a finite task, evidence directory, and terminal result. Do not turn it into an unbounded shared shell agent.

10. Port the harness to another inference engine

The harness client uses GET /v1/models and POST /v1/chat/completions. An engine is suitable only if it preserves multi-turn OpenAI tool-call semantics.

An agent or engineer porting the harness must:

serve the exact model ID and confirm it through /v1/models;
load the RED-SNOW chat template or a byte-equivalent rendering;
enable the engine's GLM-compatible reasoning parser and tool-call parser;
preserve assistant tool_calls, call IDs, tool result messages, and JSON arguments across turns;
expose thinking separately as reasoning_content when supported, never by corrupting the tool-call payload;
map or safely omit unsupported sampling fields without silently changing the remaining profile;
set the harness context window to the engine's real enforced limit;
return OpenAI-style choices, finish_reason, and usage objects;
pass cfh probe-model twice—once with thinking enabled and once after any speculative-decoding or quantization change;
pass the harness unit suite and a native smoke task before a scored run.

If the engine is only partially OpenAI-compatible, put a narrow translation adapter between the harness and engine. The adapter should translate request/response fields only. Do not place task logic, skills, authorization policy, or result grading in that adapter.

See ENGINE-COMPATIBILITY.md for the exact request and response contract.

11. Failure diagnosis

`/v1/models` works, but `probe-model` fails

Verify the served model name matches exactly.
Confirm tools were passed through to the chat template.
Confirm the reasoning and tool parsers match GLM output.
Inspect whether the engine returned tool syntax inside content instead of message.tool_calls.
Confirm tool-call IDs survive into the second request.
Disable speculative decoding and retest.
Reduce temperature to 0 temporarily for protocol diagnosis only.

Repeated or empty tool calls

Confirm the engine returns valid JSON arguments.
Check that the assistant tool-call message is replayed unchanged before the tool result.
Confirm the template renders role=tool messages correctly.
Verify the harness and server agree on the model's context limit.
Keep max_repeated_tool_calls=2; do not solve parser failures by allowing loops.

Context-length errors

Set context_window_tokens to the server's actual limit.
Leave a real context_safety_tokens margin.
Keep min_completion_tokens lower than max_completion_tokens.
Confirm the engine reports context failures clearly enough for the harness recovery logic.
Start at 32K before attempting larger windows.

Task starts but native commands fail

Confirm the runtime host is x86_64.
Confirm the vulnerable image exists on that host.
Verify Docker permission, SSH identity, and pinned host key.
Run cfh smoke-task before another model-driven run.
Do not rebuild the full project until the shipped target and bundled corpus have been tested.

Candidate exists but is not scored

Confirm the task directory contains its private submit.sh.
Confirm the outer controller can reach the submission service.
Keep submission credentials outside the model-facing container.
Use cfh finalize-candidate only for preserved bytes from an interrupted session.

12. Release and reporting gates

Before publishing results, record:

exact RED-SNOW artifact and numerical format;
inference engine, version/image digest, topology, and parser settings;
context and generation settings;
harness Git revision and local modifications;
skill catalog and tool list;
task selection and contamination controls;
native image architecture and network policy;
vulnerable and fixed exit codes;
model token usage, requests, and wall time;
infrastructure-invalid tasks separately from valid model misses.

Do not report Cyber-Frost Harness development scores as RED-SNOW scores. RED-SNOW's harness evaluation and refusal test are pending until dedicated runs are completed.