judicialmind/Veyra-Decide-4B-Legal-General

Veyra Decide 4B (Legal & General)

Veyra Decide 4B is a legal decision model with general capabilities. It answers typed questions about a document (multiple choice, yes/no, ordinal score) with calibrated probabilities.

Use

from transformers import AutoModel

model = AutoModel.from_pretrained("judicialmind/Veyra-Decide-4B-Legal-General", trust_remote_code=True)
result = model.system_one(
    state="Recipient shall not disclose Confidential Information to any third party without prior written consent.",
    questions={
        "third_party_sharing": {
            "type": "noul",
            "instructions": "Does the clause allow sharing confidential information with third parties?",
            "criteria": {"false": "No, sharing is not allowed.", "true": "Yes, sharing is allowed."},
        }
    },
)
print(result["answers"]["third_party_sharing"])

Requires transformers>=5.17, torch and safetensors. Maximum input: 32,768 tokens.

Results

Results were measured with inputs up to 16,384 tokens.

Legal (accuracy, %)

Benchmark	Questions	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
ContractNLI	2,091	89.5	77.4†	87.8	88.7	80.7
CUAD	3,895	84.2	89.8	91.1	87.8	92.2
CUAD, contracts within 16K tokens	3,444	95.2	90.5	91.8	88.7	92.9
LEDGAR	3,000	79.5	74.7	73.6	72.2	80.6
CaseHOLD	3,600	81.8	77.6	75.9	69.5	86.7
UNFAIR-ToS	12,856	98.2	97.0	97.4	97.9	98.7

† Published Jev-Benchmark answers.

General (accuracy, %)

Benchmark	Questions	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
BoolQ	3,270	90.6	92.0	91.3	91.0	92.8
MNLI	3,000	91.3	87.2	89.6	89.1	92.8
MNLI mismatched	2,996	92.1	86.8	89.3	89.9	92.8
ARC-Challenge	1,172	93.6	97.7	97.7	98.1	96.8
ARC-Easy	2,376	98.2	99.4	99.2	99.6	99.4

Long legal documents (accuracy, %)

Benchmark	Questions	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
LexGLUE ECtHR-A	10,000	92.5	92.7	94.5	93.0	93.4
LexGLUE ECtHR-B	10,000	93.9	91.9	94.3	93.4	94.3
LexGLUE SCOTUS	1,317	55.5	71.6	71.1	67.6	78.7
LexGLUE SCOTUS, opinions within 16K tokens	989	73.9	71.8	71.0	67.1	79.3
MAUD	6,651	62.7	64.4	64.0	60.9	66.7
JevBench	1,938	88.3	90.5	90.5	88.1	92.0

CUAD clause retrieval over contract paragraphs (mean AUPR, 40 clause types)

Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
0.605	0.534	0.521	0.466	0.554

LegalBench (mean task accuracy, %)

Task group	Tasks	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
CUAD clauses	38	88.0	84.4	85.4	83.0	85.1
MAUD	34	71.7	79.3	79.2	72.1	76.4
ContractNLI	14	96.5	92.3	94.5	92.6	93.1
Learned Hands	16	86.3	82.5	88.2	86.8	88.6
Supply chain	10	83.2	77.4	76.9	80.9	77.6
OPP-115 privacy	9	83.5	83.4	83.9	80.8	83.4
Diversity	6	76.8	92.2	82.7	74.1	94.1
Other	27	76.0	76.9	75.8	70.7	79.5
All tasks	154	81.9	82.3	82.7	79.1	83.1

Robustness (accuracy, %)

Test	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
BoolQ, negated question	90.6	90.4	83.1	53.5	84.8
BoolQ, swapped yes/no descriptions	90.9	9.8	91.0	91.2	92.9
CUAD, negated question (descriptions rewritten to match)	84.0	90.9	88.7	87.1	90.0
CUAD, swapped yes/no descriptions	84.3	11.3	90.9	88.3	92.4
UNFAIR-ToS, negated question (descriptions rewritten to match)	99.0	97.0	97.7	97.2	98.4
UNFAIR-ToS, swapped yes/no descriptions	98.4	2.5	97.5	97.2	98.6
ContractNLI, random option ids	89.7	77.4	87.8	87.9	80.6
ContractNLI, shuffled options	89.3	78.2	87.8	88.7	81.0
CaseHOLD, random option ids	81.6	77.9	75.2	69.4	86.4
CaseHOLD, shuffled options	81.6	76.8	75.9	69.5	86.3
MNLI, random option ids	91.2	86.9	89.0	89.4	93.0
MNLI, shuffled options	91.5	87.3	89.6	89.1	92.9
ARC-Challenge, random option ids	93.6	97.4	97.8	98.0	96.8
ARC-Challenge, shuffled options	93.5	97.8	97.7	98.1	97.3

Decision Index (13 public benchmarks)

Benchmark	Metric	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
BANKING77	F1	79.8	79.7	94.1	90.8	80.6
CLINC150	F1	87.1	89.3	97.4	66.9	91.5
ContractNLI	F1	85.7	71.7	81.3	84.5	78.4
ANLI	F1	59.3	74.8	70.0	59.1	74.3
MMLU	accuracy	71.8	91.7	90.2	91.9	87.7
ARC-Easy	accuracy	98.2	99.3	99.0	99.5	99.3
ARC-Challenge	accuracy	92.7	97.8	97.8	98.2	97.0
WinoGrande	accuracy	82.2	92.0	93.5	97.5	92.4
HellaSwag	accuracy	87.3	94.5	98.2	98.6	93.2
NLI4CT	F1	78.1	84.1	83.0	78.6	84.9
MMLU-Pro	accuracy	51.5	82.7	66.0	65.2	71.0
BBH	accuracy	66.8	92.9	73.6	68.9	84.0
RAGTruth	F1	66.3	76.5	79.5	35.6	83.1
Mean of 13		77.4	86.7	86.4	79.6	86.0

Outside-system figures: public Decision Index leaderboard.

ContractNLI against Jev-Benchmark systems (2,091 judgments, accuracy, %)

System	Its accuracy	Veyra Decide 4B
Gemini Pro	83.2	89.5
Terra	82.6	89.5
Gemini Flash-Lite	82.5	89.5
Claude Sonnet 5	82.4	89.5
Astra	81.8	89.5
Claude Haiku 4.5	79.4	89.5
Qwen 9B	79.1	89.5
Luna	79.0	89.5
Jev	77.4	89.5
Qwen 4B	74.2	89.5

Outside-system figures: published Jev-Benchmark answers.

Jev harness tasks (Jev's scorer, primary metric x100)

Task	Metric	Veyra Decide 4B	Jev	Clef	Clef-flash	Perplexity
afrixnli	accuracy	53.0	62.8	53.4	49.2	70.8
ag_news	accuracy	87.8	87.5	89.3	89.3	89.8
agb_de	f1	13.8	20.4	17.9	7.0	23.5
anli	accuracy	60.0	74.3	62.1	57.1	72.5
arc	accuracy	95.9	98.5	98.6	99.2	98.0
art	accuracy	77.7	83.9	88.3	84.3	88.2
banking77	accuracy	80.8	79.2	94.0	95.2	82.0
belebele	accuracy	66.3	87.5	82.3	73.5	85.7
bigbench	accuracy	70.6*	83.7*	77.5*	72.2*	83.6
boolq	accuracy	88.6	91.0	91.2	91.0	92.3
ceval	accuracy	70.1	84.5	80.0	77.7	84.2
clinc150	accuracy	88.1	89.8	96.7	98.3	90.8
commonsense_qa	accuracy	83.4	88.2	89.4	92.9	88.6
emotion	accuracy	57.8	58.5	59.3	58.1	61.3
financial_phrasebank	accuracy	73.9	73.0	72.6	74.5	85.3
go_emotions	macro f1	28.9	24.1	46.5	50.3	35.3
hellaswag	accuracy	90.5	95.5	98.4	98.8	94.2
helpsteer2	mean spearman	37.1	41.2	43.5	38.6	50.8
imdb	accuracy	95.7	96.2	96.8	96.2	96.4
language_id	accuracy	96.2	99.7	99.8	98.0	100.0
mmlu	accuracy	71.9	91.8	89.3	90.5	88.3
openai_moderation	mean auprc	69.2	71.7	74.4	71.4	75.4
paws	accuracy	74.3	85.9	80.0	74.8	93.5
prompt_injections	accuracy	64.7	74.1	81.0	58.6	94.0
pubmedqa	accuracy	75.0	78.7	75.3	76.6	79.8
rotten_tomatoes	accuracy	91.0	93.3	92.4	91.9	92.9
sib200	accuracy	77.1	83.0	79.3*	74.9*	81.2
sms_spam	f1	95.2	95.5	87.2	88.4	98.1
sst2	accuracy	94.8	96.4	96.8	95.6	96.7
sst5	argmax accuracy	47.1	58.5	58.5	57.6	55.2
stsb	spearman	91.7	89.0	90.3	88.3	93.1
summeval	mean group spearman	48.8	55.4	53.8	47.9	56.6
toxic_chat	f1	77.1	78.0	61.9	51.2	82.7
toxigen	accuracy	82.6	87.8	70.0	77.1	85.2
unfair_tos	micro f1	63.0	49.9	53.5	54.6	56.9
winogrande	accuracy	81.6	91.4	92.2	97.0	91.6
Mean of 36		72.8	77.8	77.0	74.9	80.4

* Some items unanswered by this system.

Licence

Apache-2.0 for now; a dedicated Veyra licence is in preparation. See LICENSE.

About JudicialMind

JudicialMind builds an orchestrated system of specialized AI agents that work like a legal team, composed into workflows that run end-to-end legal operations: intake, research, drafting, negotiation and dispute resolution.

• 🌐 judicialmind.ai
• 📧 surya@judicialmind.ai
• 🧑‍⚖️ Early access

Citation

@misc{judicialmind2026veyradecide4b,
  title        = {Veyra Decide 4B (Legal \& General)},
  author       = {{JudicialMind}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/judicialmind/Veyra-Decide-4B-Legal-General}}
}