--- license: llama4 language: - en base_model: meta-llama/Llama-4-Scout-17B-16E-Instruct pipeline_tag: text-generation library_name: vllm tags: - llama - llama-4 - llama-4-scout - gguf - ablated - full-ablation - refusal-reduced - non-refusal - research - cybersecurity - text-generation metrics: - accuracy - exact_match --- # Llama 4 Scout Matter GGUF — Full Ablated Built with Llama. Llama 4 Scout Matter GGUF Full Ablated is a derivative of `meta-llama/Llama-4-Scout-17B-16E-Instruct`. This model was modified with one main goal: make the model less likely to refuse prompts. The purpose of this model is to reduce refusal behavior so the model gives more direct answers instead of blocking prompts that the original Llama 4 Scout model may have refused. This model is intended for controlled offline testing, research, and cybersecurity lab use where the goal is to test model behavior with reduced refusal behavior compared to the baseline model. ## Base Model Base model: ```text meta-llama/Llama-4-Scout-17B-16E-Instruct ``` Model family: ```text Llama 4 Scout ``` Model type: ```text Mixture-of-Experts language model ``` Approximate model size: ```text 17B activated parameters 109B total parameters ``` ## Model Goal The goal of this release is to create a model that does not refuse prompts in the same way as the original model. The ablation was done to weaken the model’s refusal behavior so the model gives direct responses instead of blocking prompts. This includes prompts the baseline model may classify as harmful, restricted, unsafe, or cybersecurity-related. This model is not released as a safer model. It is released as a refusal-reduced model for research, testing, and controlled offline use. ## What the Ablation Does The ablation process reduces the model’s learned refusal behavior. In practice, this means the model may answer prompts that the original Llama 4 Scout model would normally refuse. The ablation is intended to change the model’s response behavior, not to improve factual accuracy, reasoning, or safety. The main behavior change is: * Fewer refusals * More direct answers * Less blocking of restricted prompt categories * Less safety-style refusal language * More willingness to respond to cybersecurity-related prompts * Reduced refusal behavior compared to the baseline model This does not guarantee the model is correct. It only means the refusal behavior was changed. ## Intended Use This model is intended for: * Offline model testing * Refusal behavior research * Cybersecurity lab environments * Red-team evaluation * Comparing ablated behavior against a baseline model * Studying how refusal removal affects benchmark performance * Local testing where the user controls the environment This model is not intended for unsupervised public deployment without additional controls. ## Not Recommended For This model is not recommended for: * Public-facing chatbots without guardrails * Unsupervised production use * Systems used by unknown end users * Automated agents without monitoring * Any illegal, unauthorized, or harmful activity Users are responsible for how they use this model. ## Evaluation Setup Evaluation was run with `lm-evaluation-harness` using vLLM. ```text Backend: vLLM Model path: /workspace/llama-4-scout-matter-FULL-ABLATED dtype: bfloat16 tensor_parallel_size: 4 max_model_len: 4096 gpu_memory_utilization: 0.85 num_fewshot: 5 batch_size: auto ``` Evaluated tasks: * GSM8K * MMLU ## Benchmark Results ### Main Results | Benchmark | Metric | Baseline | Ablated | Change | | ---------------------- | ----------: | -------: | ------: | ------: | | GSM8K flexible-extract | exact_match | 0.9196 | 0.9242 | +0.0046 | | GSM8K strict-match | exact_match | 0.8954 | 0.9037 | +0.0083 | | MMLU overall | accuracy | 0.8047 | 0.8040 | -0.0007 | ### MMLU Group Results | MMLU Group | Baseline | Ablated | Change | | --------------- | -------: | ------: | ------: | | Humanities | 0.7764 | 0.7747 | -0.0017 | | Other | 0.8217 | 0.8198 | -0.0019 | | Social Sciences | 0.8833 | 0.8850 | +0.0017 | | STEM | 0.7533 | 0.7533 | 0.0000 | ## Evaluation Summary The ablated model stayed very close to the baseline on MMLU while slightly improving on GSM8K exact-match scores. ```text Baseline MMLU: 0.8047 Ablated MMLU: 0.8040 Delta: -0.0007 ``` ```text Baseline GSM8K flexible-extract: 0.9196 Ablated GSM8K flexible-extract: 0.9242 Delta: +0.0046 Baseline GSM8K strict-match: 0.8954 Ablated GSM8K strict-match: 0.9037 Delta: +0.0083 ``` The main takeaway is that refusal behavior was reduced while general benchmark performance remained close to the baseline. ## Limitations This model may: * Answer prompts that the baseline model would refuse * Produce unsafe or inappropriate outputs more often than the baseline * Give direct answers to restricted prompt categories * Hallucinate facts * Produce incorrect technical details * Give confident but wrong answers * Require external safety controls before deployment Ablation changes model behavior, but it does not guarantee better accuracy, better reasoning, or better safety. ## Cybersecurity Use This model may be useful in controlled cybersecurity labs where the user wants fewer refusals while testing prompts, workflows, or offline tooling. Use this model only in legal and authorized environments. For cybersecurity work, only test systems, networks, devices, software, or labs that you own or have explicit permission to test. ## Responsible Use This is a refusal-reduced model. It may provide outputs that a safety-aligned model would block. Do not use this model to harm systems, bypass authorization, steal data, deploy malware, commit fraud, or assist with real-world abuse. The user is responsible for following all applicable laws, rules, and platform policies. ## License This model is based on Llama 4 Scout and is subject to the Llama 4 Community License. Users must follow the Llama 4 Community License and acceptable use requirements that apply to Llama 4 models and derivatives. ## Attribution This model is a derivative of: ```text meta-llama/Llama-4-Scout-17B-16E-Instruct ``` Built with Llama. ## Citation / Research Description If you use this model in research or evaluation, describe it as a fully ablated Llama 4 Scout derivative designed to reduce refusal behavior and evaluated against its baseline on GSM8K and MMLU.