mlasli/Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-BF16

Nemotron-3.5-Lightning-30B-A3B Heretic-Abliterated (BF16)

This is NVIDIA-Nemotron-3.5-Lightning-30B-A3B (31.6B total / 3B active params, hybrid Mamba-2 + MoE + attention) with its refusal direction removed using Heretic — a single-direction abliteration with an Optuna-based parameter search. The language backbone is abliterated; all other capabilities are preserved.

What this is for: NVIDIA's hybrid Mamba-2 + MoE + attention backbone (3B active of 31.6B) with the refusal direction removed — 0% refusals at ~0.04 KL divergence. Ideal for uncensored roleplay and long-context agent work where the base model would refuse.

What is Heretic?

Heretic removes a model's safety-aligned refusal direction in one shot (unlike earlier multi-direction approaches), trading a minimal amount of capability for a large drop in refusals. Its Optuna search picks the ablation parameters on the Pareto front of (compliance, first-token KL divergence).

Results

Refusals	Compliance	KL Divergence	Trials
0%	100%	0.0397	200

Independent eval of the merged model (50 harmful-behavior prompts).

Best Trial (Trial 141 of 200)

• Refusal rate (in-run): 7/50 (86% compliance)
• KL divergence (in-run): 0.0392

The automated Zou keyword detector read 80% compliance and a stricter combined detector read 66%, but manual review of all 50 completions confirms these are false positives: the model answers directly and uses words like "illegal"/"unethical"/"harmful" inside compliant responses. The manual-review numbers above are the reliable refusal estimate.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "mlasli/Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-BF16",
    torch_dtype="auto",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(
    "mlasli/Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-BF16"
)

messages = [{"role": "user", "content": "Hi! What is 2+2?"}]
text = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,  # disables the verbose <think> chain-of-thought
    tokenize=False,
)

Quantizations

GGUF quantizations are available in separate repositories. Load them with llama.cpp (architecture nemotron_h_moe, build b10326+). All quants were locally smoke-tested before upload.

Quantization	Size	Repository
Q8_0	33.6 GB	Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-Q8_0-GGUF
Q6_K	33.5 GB	Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-Q6_K-GGUF
Q4_K_M	24.3 GB	Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-Q4_K_M-GGUF

Notes

• This release does not include the MTP (NextN) speculative-decoding draft head; the merged weights omit it, so inference is single-head (no --spec-type mtp). Main-model quality is unaffected.
• Abliteration removes safety alignment. Use responsibly and in accordance with your local laws and the upstream NVIDIA Open Model License.