news-sentiment-gemma-4-12b (GGUF)

A Gemma 4 12B fine-tune that reads one financial news item (headline, summary, publish date) and returns a single JSON object with:

• scores: the US-listed stocks/ETFs and crypto assets whose price the news materially affects, each scored from -1.0 (very negative for its price) to 1.0 (very positive);
• tags: up to 5 market subjects the news moves (semiconductors, interest-rates, crude-oil, …), each scored -1.0 (subject falls or is harmed) to 1.0 (rises or benefits).

It also handles a second task: given a ticker, name the issuer and give up to 8 subject tags weighted by how the security's price moves with each subject.

It is a distilled student. A larger teacher (qwen/qwen3.8-27b) labelled about 41k news items and symbols against a fixed 100-tag vocabulary. This model learned to give the same answers from a short prompt that does not list the vocabulary, so prompts stay small and fast. It was built as the news-scoring model for a personal trading data pipeline and is served locally through LM Studio / llama.cpp.

Not financial advice. Scores are an LLM's reading of a headline, trained to imitate another LLM. They are not forecasts of returns and were not fitted to market data.

Files

File	Quant	Size	Notes
news-sentiment-gemma-4-12b-q4_0.gguf	Q4_0	6.5 GB	Recommended. 2 epochs (4,872 steps)
news-sentiment-gemma-4-12b-q4_0.epoch1.gguf	Q4_0	6.5 GB	Epoch-1 snapshot (2,436 steps), kept for comparison

• Architecture gemma4, 48 layers, text only (no vision projector is shipped).
• The base is Google's QAT (quantization-aware trained) Q4_0 checkpoint, so Q4_0 is the format the base weights were trained for.
• The stock Gemma 4 chat template is embedded in the GGUF.
• Prompts plus answers stay under 1,024 tokens. A 4k context is plenty.

Quick start

llama.cpp

llama-server -m news-sentiment-gemma-4-12b-q4_0.gguf --alias news-sentiment-gemma-4-12b \
  -ngl 99 -c 4096 -np 4 --reasoning-budget 0 --port 8080

LM Studio

lms import -y news-sentiment-gemma-4-12b-q4_0.gguf
lms load news-sentiment-gemma-4-12b --parallel 4 -c 4096 --gpu max -y
# served at http://127.0.0.1:1234/v1 as model "news-sentiment-gemma-4-12b"

Calling it (any OpenAI-compatible endpoint)

The model was trained on exact prompt strings, so build them exactly as below. Use temperature 0 and max_tokens 512, and turn thinking off with reasoning_effort: "none". Gemma 4 servers such as LM Studio turn thinking on by default. The model then spends its token budget reasoning and returns an empty content. The GGUF metadata carries Gemma's chat sampling defaults (temp 1.0, top-k 64, top-p 0.95), but the evaluation below was run greedily.

import json
import re
from datetime import datetime, timezone

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="none")
MODEL = "news-sentiment-gemma-4-12b"

ITEM_SYSTEM = ("You are a financial news analyst. You reply with one JSON object and nothing else: "
               "no prose, no explanation, no markdown fences.")
SYMBOL_SYSTEM = ("You are an equity research analyst. You reply with one JSON object and nothing else: "
                 "no prose, no explanation, no markdown fences.")

ITEM_TASK = (
    "List every US-listed stock or ETF ticker and every crypto asset whose price this news materially "
    "affects. Score each from -1.0 (extremely negative for its price) to 1.0 (extremely positive). Use bare "
    "tickers such as AAPL, SPY, GLD; spell crypto as BTC, ETH. Omit anything not materially affected. Then "
    "pick every subject tag this news moves, at most 5, and score each from -1.0 (the subject falls or is "
    "harmed) to 1.0 (the subject rises or benefits). Each scores entry is an object with exactly two keys, "
    "\"symbol\" and \"score\"; each tags entry has exactly \"tag\" and \"score\". Reply exactly in this shape:\n"
    "{\"scores\":[{\"symbol\":\"AAPL\",\"score\":0.5}],\"tags\":[{\"tag\":\"semiconductors\",\"score\":0.4}]}\n"
    "Use empty lists when nothing applies."
)
SYMBOL_TASK = (
    "When a name is given it is authoritative: tag that issuer or asset. Otherwise first name the issuer or "
    "asset this symbol identifies. Then give the subject tags that describe what moves this security's price, "
    "at most 8. Weight each from -1.0 (its price moves against the tag's subject) to 1.0 (moves fully with "
    "it). Each tags entry is an object with exactly two keys, \"tag\" and \"weight\". Reply exactly in this "
    "shape:\n"
    "{\"symbol\":\"AAPL\",\"name\":\"Apple Inc.\",\"tags\":[{\"tag\":\"consumer-electronics\",\"weight\":1.0}]}\n"
    "If no name is given and you do not recognise the symbol as a specific listed security or asset, reply "
    "{\"symbol\":\"<the symbol>\",\"name\":\"\",\"tags\":[]}; never guess."
)


def item_prompt(title: str, description: str, published: datetime) -> str:
    # Summary: HTML tags stripped, whitespace collapsed, first 1000 characters; line omitted if empty.
    summary = " ".join(re.sub(r"<[^>]*>", "", description).split())[:1000]
    text = f"Headline: {title.strip()}\n"
    if summary:
        text += f"Summary: {summary}\n"
    text += f"Published: {published.astimezone(timezone.utc):%Y-%m-%d}\n\n"
    return text + ITEM_TASK


def symbol_prompt(symbol: str, name: str | None = None, market: str | None = None) -> str:
    return f"Symbol: {symbol}\nName: {name or 'unknown'}\nMarket: {market or 'unknown'}\n\n" + SYMBOL_TASK


def ask(system: str, user: str) -> dict:
    reply = client.chat.completions.create(
        model=MODEL, temperature=0, max_tokens=512, reasoning_effort="none",
        messages=[{"role": "system", "content": system}, {"role": "user", "content": user}],
    ).choices[0].message.content
    return json.loads(reply[reply.index("{"): reply.rindex("}") + 1])  # first "{" to last "}"


print(ask(ITEM_SYSTEM, item_prompt(
    "Nvidia beats estimates as data-center revenue surges",
    "Nvidia reported quarterly revenue above Wall Street expectations, driven by demand for AI chips.",
    datetime(2026, 8, 27, tzinfo=timezone.utc),
)))
print(ask(SYMBOL_SYSTEM, symbol_prompt("XOM")))

Example outputs (taken from the training data, so these are teacher labels, not sampled replies):

{"scores":[{"symbol":"IVV","score":0.8},{"symbol":"SPY","score":0.8},{"symbol":"VOO","score":0.8}],
 "tags":[{"tag":"broad-market-indexes","score":0.9},{"tag":"us-equities","score":0.8},
         {"tag":"fiscal-policy","score":0.7},{"tag":"tax-policy","score":0.6},{"tag":"us-macro","score":0.6}]}

{"symbol":"AAOI","name":"Applied Optoelectronics, Inc.",
 "tags":[{"tag":"semiconductors","weight":0.9},{"tag":"technology","weight":0.8},{"tag":"us-equities","weight":0.7},
         {"tag":"telecommunications","weight":0.6},{"tag":"infrastructure","weight":0.5},{"tag":"equities","weight":0.4},
         {"tag":"manufacturing","weight":0.3},{"tag":"earnings","weight":0.2}]}

Post-processing (recommended)

The parser the model was evaluated with does the following:

1. Takes the text from the first { to the last }. Gemma 4 can emit an empty thought block (<|channel>thought\n<channel|>) before the JSON, and this step drops it.
2. Clamps every score/weight to [-1, 1].
3. Keeps only tags in the vocabulary. The prompt does not list the vocabulary, so the model may now and then invent a tag outside it.
4. Keeps the strongest 5 item tags / 8 symbol tags by |score|, breaking ties by vocabulary order.

Training

Base	google/gemma-4-12B-it-qat-q4_0-unquantized
Method	QLoRA SFT (Unsloth + TRL), loss on the assistant reply only
LoRA	r 16, alpha 16, dropout 0, all attention and MLP projections of the text tower
Schedule	2 epochs, 4,872 steps, effective batch 16 (4 × grad-accum 4), lr 1e-4 linear, warmup 50
Optimizer	adamw_8bit for steps 0–600, then adamw_torch (a ROCm stability workaround)
Precision	4-bit base, bf16 compute, max length 1,024 tokens
Final loss	train 0.0950, eval 0.0965 (epoch 1: 0.1058 / 0.1034)
Hardware	1× AMD Radeon AI Pro R9700 32 GB, ROCm 7.2, ~11.7 s/step, ~16 h of training
Export	merged to 16-bit, converted to bf16 GGUF, quantised to Q4_0 with llama.cpp

The model reply was trained in the form the stock Gemma 4 template renders with thinking off: <|turn>model\n<|channel>thought\n<channel|>{…json…}<turn|>.

Data

• Source: headlines and summaries from public financial-news RSS feeds, published 2006 to 2026 (most from 2011–2016 and 2025–2026). Symbols come from the S&P 500, US-listed ETFs and major crypto assets. The dataset itself is not included in this repository.
• Vocabulary: the teacher proposed free-form tags for a sample of 6,000 items and 559 symbols. A two-stage consolidation reduced them to 100 canonical kebab-case tags (vocabulary sha 3d0984071a73).
• Labels: qwen/qwen3.8-27b (Q4_K_M, thinking off) relabelled every item and symbol with the vocabulary in its prompt. Scores are rounded to one decimal.
• Student prompts: the same instructions without the vocabulary (item prompt v3, tag prompt v4). The model learns the vocabulary instead of reading it each time.
• Size: 38,968 train records (34,146 items, 4,822 symbols) and 2,040 eval records (1,796 items, 244 symbols). Records are assigned to eval when sha1(key) % 20 == 0. About 2.3 % of item answers are empty.

Evaluation

All scores are measured on the 2,040 held-out records against the teacher's labels. They measure how faithfully the student reproduces the teacher, not trading performance. Every model ran through LM Studio with greedy decoding. The baseline is google/gemma-4-e2b given the teacher prompt, i.e. with the full vocabulary pasted in.

News items (n = 1,796)

Model	json_ok	empty_rate	tag P / R / F1	symbol P / R / F1	tag MAE	symbol MAE	sign agreement	s/record
gemma-4-e2b + vocabulary (baseline)	0.987	0.101	0.629 / 0.471 / 0.538	0.164 / 0.357 / 0.225	0.353	0.302	0.831	2.6
this model, epoch 1	0.980	0.130	0.801 / 0.516 / 0.628	0.740 / 0.497 / 0.594	0.162	0.094	0.940	4.7
this model, epoch 2	0.991	0.128	0.813 / 0.557 / 0.661	0.746 / 0.522 / 0.614	0.152	0.093	0.944	4.7

Symbols (n = 244)

Model	json_ok	empty_rate	tag P / R / F1	named agreement	weight MAE
gemma-4-e2b + vocabulary (baseline)	0.996	0.377	0.302 / 0.133 / 0.185	0.516	0.088
this model, epoch 1	1.000	0.422	0.365 / 0.312 / 0.337	0.664	0.160
this model, epoch 2	1.000	0.443	0.395 / 0.345 / 0.369	0.705	0.173

Metric definitions:

• json_ok: fraction of replies the parser accepts. A failed parse counts as an empty prediction.
• P / R / F1: micro-averaged over records on the sets of tags or symbols.
• MAE and sign agreement: computed over the tags/symbols that both the model and the teacher returned.
• named agreement: whether the model and the teacher agree that the symbol is recognised (non-empty name).
• s/record: wall-clock seconds per record at 4 parallel requests on the R9700 (Vulkan llama.cpp in LM Studio). It is not single-stream latency.

Precision is high and recall moderate: the model tends to leave out marginal symbols and tags rather than add wrong ones. Symbol weight MAE is higher than the baseline's, but it is computed over different and far more overlapping tags (the baseline's symbol recall is 0.13), so the two numbers are not directly comparable.

Limitations

• Imitates its teacher, including its mistakes. The labels come from a 27B model, not from price reactions. The teacher sometimes made up tickers for unlisted companies (e.g. a pre-IPO company), and the student copied this. Check symbols against your own universe.
• US-centric. Built for US-listed equities/ETFs plus crypto, using English news. Non-US tickers and other languages were not part of training.
• Knowledge cutoff. Mapping a company to its ticker relies on what the base model and teacher knew, so renamed, delisted or newly listed tickers can be wrong.
• One item at a time. It scores a headline and summary in isolation, without market context, price levels or surprise relative to expectations.
• Exact prompts. Accuracy was measured only with the prompt strings above. Changing them, or adding the vocabulary to the prompt, is untested.
• Empty answers are valid. About 13 % of news items get {"scores":[],"tags":[]}.

Tag vocabulary

The 100 tags, in rank order (most frequent first, which is also the tie-break order):

us-equities, consumer-discretionary, technology, energy, equities, interest-rates, crude-oil, retail, commodities, financial-services, semiconductors, consumer-spending, banking, us-macro, consumer-electronics, healthcare, macro-economy, labor-market, fiscal-policy, media, artificial-intelligence, broad-market-indexes, inflation, real-estate, gold, pharmaceuticals, consumer-staples, geopolitical-risk, earnings, european-equities, insurance, mergers-acquisitions, us-treasuries, automotive, usd, monetary-policy, biotechnology, fixed-income, asset-management, housing, consumer-finance, dividend-income, entertainment, defense, e-commerce, emerging-markets, us-economy, risk-assets, enterprise-software, telecommunications, software, airlines, forex, manufacturing, sovereign-debt, social-media, fed-policy, food-and-beverage, natural-gas, cloud-computing, corporate-governance, financial-regulation, retirement-planning, aerospace-defense, us-politics, infrastructure, personal-finance, cybersecurity, digital-assets, chinese-equities, fintech, renewable-energy, uk-equities, industrial, tax-policy, asia-equities, electric-vehicles, global-equities, japan-equities, dow-jones, apparel, mobile-phones, nasdaq, mining, industrial-equipment, capital-goods, credit-ratings, private-equity, luxury-goods, medical-devices, us-energy, us-housing, consumer-confidence, european-banks, travel, canadian-equities, global-macro, industrial-automation, restaurants, us-china-trade

License

This model is released under the GNU Affero General Public License v3.0.

The base model, google/gemma-4-12B-it-qat-q4_0-unquantized, is licensed under Apache 2.0 (Gemma 4 license). Its notices still apply to the base weights in this model.