Instructions to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="lewiswatson/Frame2KG-Gemma4-e4b-JSON",
	filename="Frame2KG-Gemma4-e4b-JSON.IQ4_XS.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Use Docker

docker model run hf.co/lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

LM Studio
Jan

vLLM

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lewiswatson/Frame2KG-Gemma4-e4b-JSON"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lewiswatson/Frame2KG-Gemma4-e4b-JSON",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Ollama
How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with Ollama:
```
ollama run hf.co/lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M
```

Unsloth Studio

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lewiswatson/Frame2KG-Gemma4-e4b-JSON to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for lewiswatson/Frame2KG-Gemma4-e4b-JSON to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for lewiswatson/Frame2KG-Gemma4-e4b-JSON to start chatting

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with Docker Model Runner:
```
docker model run hf.co/lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M
```

Lemonade

How to use lewiswatson/Frame2KG-Gemma4-e4b-JSON with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lewiswatson/Frame2KG-Gemma4-e4b-JSON:Q4_K_M

Run and chat with the model

lemonade run user.Frame2KG-Gemma4-e4b-JSON-Q4_K_M

List all available models

lemonade list

Frame2KG-Gemma4-e4b-JSON

This repository contains GGUF quantised files for the Frame2KG fine-tuned Gemma4 e4b model.

These GGUF releases are optimized deployment variants of Frame2KG models. They are provided for practical inference use and may not exactly match the original weights, checkpoints, or evaluation configuration reported in the Frame2KG paper.

Model Details

Family: Gemma4
Size: e4b
Output format: JSON Frame2KG graph output
Base model: google/gemma-4-E4B-it
Model type: gemma4
Architecture: Gemma4ForConditionalGeneration
Fine-tuning method: PEFT LoRA
LoRA rank: 8
LoRA alpha: 16
Trainable added token count: 0

Files

File	Size	Notes
`Frame2KG-Gemma4-e4b-JSON.f16.gguf`	13.92 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.IQ4_XS.gguf`	4.71 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q2_K.gguf`	4.08 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q3_K_L.gguf`	4.65 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q3_K_M.gguf`	4.49 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q3_K_S.gguf`	4.31 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q4_K_M.gguf`	4.94 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q4_K_S.gguf`	4.82 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q5_K_M.gguf`	5.33 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q5_K_S.gguf`	5.26 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q6_K.gguf`	5.75 GB	model weights
`Frame2KG-Gemma4-e4b-JSON.Q8_0.gguf`	7.43 GB	model weights
`mmproj-Frame2KG-Gemma4-e4b-JSON.f16.gguf`	944.49 MB	multimodal projector
`mmproj-Frame2KG-Gemma4-e4b-JSON.Q8_0.gguf`	533.94 MB	multimodal projector

Usage Notes

These files are intended for llama.cpp-compatible GGUF runtimes.

Scope

Frame2KG models are intended to convert image or frame content into structured graph-style outputs. The exact output format depends on the variant:

JSON variants target JSON-formatted Frame2KG output.
CT variants target compressed Frame2KG graph tokens.

Citation

If you use this model in your work, please cite the paper:

@inproceedings{watson2026frame2kg,
  title = {Frame2KG: A Benchmark and Evaluation Toolkit for Interpretable Frame-to-Graph Generation},
  author = {Watson, Lewis N. and Strathearn, Carl and Mitchell, Kenny and Yu, Yanchao},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
  month = {May},
  year = {2026},
  pages = {10912--10926},
  address = {Palma, Mallorca, Spain},
  publisher = {European Language Resources Association (ELRA)},
  editor = {Piperidis, Stelios and Bel, Núria and van den Heuvel, Henk and Ide, Nancy and Krek, Simon and Toral, Antonio},
  doi = {10.63317/4ys6kofrzoc5},
  url = {https://doi.org/10.63317/4ys6kofrzoc5}
}

Disclaimer

This model is provided as is, without warranties or guarantees of any kind, either express or implied. The authors make no representations regarding the accuracy, reliability, safety, suitability, or performance of the model or its outputs.

The model may generate incorrect, incomplete, or misleading results and should not be relied upon for critical, safety-sensitive, legal, medical, financial, or other high-stakes decisions. Use of this model is entirely at your own risk.

The authors accept no liability for damages, losses, or consequences arising from use, misuse, or inability to use the model.

Downloads last month: -

GGUF

Model size

7B params

Architecture

gemma4

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for lewiswatson/Frame2KG-Gemma4-e4b-JSON

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Quantized

(232)

this model

lewiswatson
/

Frame2KG-Gemma4-e4b-JSON