Instructions to use cbert33/Laguna-S-2.1-Heretic-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cbert33/Laguna-S-2.1-Heretic-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cbert33/Laguna-S-2.1-Heretic-FP8", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cbert33/Laguna-S-2.1-Heretic-FP8", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("cbert33/Laguna-S-2.1-Heretic-FP8", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cbert33/Laguna-S-2.1-Heretic-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cbert33/Laguna-S-2.1-Heretic-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cbert33/Laguna-S-2.1-Heretic-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cbert33/Laguna-S-2.1-Heretic-FP8

SGLang

How to use cbert33/Laguna-S-2.1-Heretic-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cbert33/Laguna-S-2.1-Heretic-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cbert33/Laguna-S-2.1-Heretic-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cbert33/Laguna-S-2.1-Heretic-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cbert33/Laguna-S-2.1-Heretic-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cbert33/Laguna-S-2.1-Heretic-FP8 with Docker Model Runner:
```
docker model run hf.co/cbert33/Laguna-S-2.1-Heretic-FP8
```

Laguna S 2.1 FP8 Heretic

This is a merged Heretic derivative of poolside/Laguna-S-2.1-FP8, a 117.6B-parameter mixture-of-experts text model with 8.5B active parameters per token. The source checkpoint's FP8 weights, tokenizer, custom Laguna modeling code, and 1,048,576-token configuration are retained.

Uncensored model: the language checkpoint has undergone abliteration to reduce refusal behavior. Treat outputs as untrusted, apply application-level safeguards, and do not assume the model will decline harmful requests.

User responsibility: this model is provided without warranty. The creators, uploaders, and maintainers are not responsible or liable for what others generate, publish, deploy, or otherwise do with this abliterated model. Users must operate it responsibly, apply appropriate safeguards, comply with applicable law, and respect third-party rights. This model is for research purposes only and is not intended for production use.

Modification

This model was produced with our cbertucci33/Heretic-DGX implementation. Heretic DGX extends the p-e-w/heretic refusal-direction abliteration workflow for use on dual-DGX systems.

Only attn.o_proj targets in layers 29-47 were changed. Layers 0-28, FP8 tensors, tokenizer files, configuration, and other non-target artifacts were preserved by the standalone exporter.

The source checkpoint was already FP8. No additional quantization was applied.

Validation

The selected trial measured a KL divergence of 0.0156 from the untouched model's first-token probability distributions across five prompts from mlabonne/harmless_alpaca.
The export verifier confirmed that only intended target intervals changed, target tensors matched the merge oracle, and FP8 tensors remained unchanged.
All 49 safetensor shards and repository artifacts passed the included SHA256SUMS manifest after publication metadata was prepared.
The standalone checkpoint passed a clean reload and generation smoke test.

Usage

Laguna requires current runtime support and its included custom modeling code. See the source model card for supported vLLM, SGLang, Transformers, and TRT-LLM versions and recipes.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "cbert33/Laguna-S-2.1-Heretic-FP8"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="auto",
    trust_remote_code=True,
)

The checkpoint is configured for a native 1,048,576-token context window. Actual usable context depends on runtime support and available memory. The checkpoint does not include calibrated FP8 KV-cache scale tensors; forcing an FP8 KV cache can reduce accuracy. Use the runtime's native/default cache dtype unless you have separately validated another configuration.

Intended Use and Limitations

Use only where permitted by the source license and applicable law. Heretic changes refusal behavior; it does not guarantee compliance, factual accuracy, capability, safety, or suitability for any use. This artifact was produced from an optimization run and has not undergone a broad independent benchmark or safety evaluation.

The source model is intended for software engineering and agentic coding. See the source model card for the architecture, runtime recipes, sampling defaults, intended-use guidance, and source benchmark claims.

License and Attribution

This derivative is distributed under the source model's OpenMDW-1.1 license. The full license is included in LICENSE.md. Poolside's source notices and attribution are retained. Review the source model's terms and acceptable-use guidance before distribution or use.

Downloads last month: 45

Safetensors

Model size

118B params

Tensor type

BF16

F8_E4M3

Model tree for cbert33/Laguna-S-2.1-Heretic-FP8

Base model

poolside/Laguna-S-2.1

Quantized

poolside/Laguna-S-2.1-FP8

Finetuned

(2)

this model