Instructions to use QuantFactory/SecurityLLM-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QuantFactory/SecurityLLM-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("QuantFactory/SecurityLLM-GGUF", dtype="auto")

llama-cpp-python

How to use QuantFactory/SecurityLLM-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="QuantFactory/SecurityLLM-GGUF",
	filename="SecurityLLM.Q2_K.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use QuantFactory/SecurityLLM-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf QuantFactory/SecurityLLM-GGUF:Q4_K_M

Use Docker

docker model run hf.co/QuantFactory/SecurityLLM-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use QuantFactory/SecurityLLM-GGUF with Ollama:
```
ollama run hf.co/QuantFactory/SecurityLLM-GGUF:Q4_K_M
```

Unsloth Studio new

How to use QuantFactory/SecurityLLM-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantFactory/SecurityLLM-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantFactory/SecurityLLM-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for QuantFactory/SecurityLLM-GGUF to start chatting

Docker Model Runner
How to use QuantFactory/SecurityLLM-GGUF with Docker Model Runner:
```
docker model run hf.co/QuantFactory/SecurityLLM-GGUF:Q4_K_M
```

Lemonade

How to use QuantFactory/SecurityLLM-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull QuantFactory/SecurityLLM-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.SecurityLLM-GGUF-Q4_K_M

List all available models

lemonade list

QuantFactory/SecurityLLM-GGUF

This is quantized version of ZySec-AI/SecurityLLM created using llama.cpp

Original Model Card

ZySec-7B

ZySec-7B, stands as a pivotal innovation for security professionals, leveraging the advanced capabilities of HuggingFace's Zephyr language model series. This AI model is crafted to be an omnipresent cybersecurity ally, offering on-demand, expert guidance in cybersecurity issues. Picture ZySec-7B as an ever-present digital teammate, adept at navigating the complexities of security challenges.

The efficacy of ZySec-7B lies in its comprehensive training across numerous cybersecurity fields, providing a deep and wide-ranging understanding of the sector. ZySec is developed using the DPO technique, utilizing a varied dataset encompassing critical topics such as:

Sophisticated areas like Attack Surface Threats, Cloud Security, and the Cyber Kill Chain.
Key compliance and regulatory frameworks, including CIS Controls, FedRAMP, PCI DSS, and ISO/IEC 27001.
Practical aspects like Cloud Secure Migration, Data Exfiltration Techniques, and Security Incident Handling.
Crucial strategic fields such as Security Governance, Risk Management, and Security Architecture Review.

ZySec-7B's training spans over 30 unique domains, each enriched with thousands of data points, delivering unparalleled expertise.

As the first of its kind in an open-source, AI-driven cybersecurity series, ZySec-7B transcends the conventional role of a support tool, redefining organizational security approaches. Its open-source nature not only invites community contributions but also enhances its flexibility and transparency in managing vast cybersecurity data. ZySec-7B is instrumental in providing vital, actionable insights for strategic decision-making and advanced risk management. More than a mere software, ZySec-7B is a community-enhanced strategic tool, equipping your team to proactively confront and stay ahead of the dynamic landscape of cyber threats and regulatory demands.

For suggestions please use Road Map

Details of dataset distribution here - Dataset Distribution

Fully compatible with LM Studio. Search for “Zysec” and here is what you get. Here is a sample output of ZySec writing email to John about database security using LM Studio:

The training is funded by ZySec AI, the mobile app for Cyber Security professionals.

Official GGUF version is hosted here - ZySec-7B-v1-GGUF on HuggingFace

ZySec AI: Unleashing the Potential of the ZySec Series Model

Project ZySec, an integral part of ZySec AI, stands at the forefront of integrating Artificial Intelligence into Cybersecurity. Centered around the innovative ZySec 7B model, it's designed to revolutionize the cybersecurity landscape with AI-driven solutions. ZySec AI isn't just a tool, it's a transformative approach, blending AI's cutting-edge capabilities with the unique intricacies of cybersecurity, while ensuring privacy and security.

Discover the Key Features of Project ZySec

AI-Driven Cybersecurity: Tap into the power of the ZySec 7B model, a bespoke AI solution fine-tuned for cybersecurity.
24/7 Expert Assistance: Benefit from round-the-clock support and expert advice, guaranteeing smooth operations during any SOC shift.
Efficient Playbook Access: Streamline your workflow with quick and easy access to playbooks and documents, enhancing information retrieval.
Standards Explorer: Navigate various standards with ease, akin to a seasoned expert's proficiency.
Ongoing Internet Research: Leverage AI-enabled, thorough internet research for exhaustive insights. (Note: Internet use is optional and specific to this feature).

About Project ZySec by ZySec AI

ZySec AI an opensource project with a vision towards fusioning of Cybersecurity with Artificial Intelligence. Our goal is to transform the way security professionals engage with technology. More than a mere tool, ZySec AI symbolizes a comprehensive strategy to augment security operations, merging the innovative essence of AI with cybersecurity's distinctive challenges, always ensuring privacy and security.

https://github.com/ZySec-AI/ZySec

The ZySec Roadmap

https://github.com/ZySec-AI/.github/blob/main/roadmap.md

Downloads last month: 766

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 1 Ask for provider support

QuantFactory
/

SecurityLLM-GGUF