judicialmind/Veyra-Decide-27B-Legal-General

Veyra Decide 27B (Legal & General)

Veyra Decide 27B is a legal decision model with general capabilities. It answers typed questions about a document (multiple choice, yes/no, ordinal score) with calibrated probabilities.

Use

from transformers import AutoModel

model = AutoModel.from_pretrained("judicialmind/Veyra-Decide-27B-Legal-General", trust_remote_code=True)
result = model.system_one(
    state="Recipient shall not disclose Confidential Information to any third party without prior written consent.",
    questions={
        "third_party_sharing": {
            "type": "noul",
            "instructions": "Does the clause allow sharing confidential information with third parties?",
            "criteria": {"false": "No, sharing is not allowed.", "true": "Yes, sharing is allowed."},
        }
    },
)
print(result["answers"]["third_party_sharing"])

Requires transformers>=5.17, torch and safetensors. Maximum input: 131,072 tokens.

Results

Results were measured with inputs up to 16,384 tokens.

Legal (accuracy, %)

Benchmark	Questions	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
ContractNLI	2,091	90.3	77.4†	87.8	88.7	80.7
CUAD	3,895	84.4	89.8	91.1	87.8	92.2
CUAD, contracts within 16K tokens	3,444	95.5	90.5	91.8	88.7	92.9
LEDGAR	3,000	81.3	74.7	73.6	72.2	80.6
CaseHOLD	3,600	84.9	77.6	75.9	69.5	86.7
UNFAIR-ToS	12,856	98.5	97.0	97.4	97.9	98.7

† Published Jev-Benchmark answers.

General (accuracy, %)

Benchmark	Questions	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
BoolQ	3,270	92.8	92.0	91.3	91.0	92.8
MNLI	3,000	92.3	87.2	89.6	89.1	92.8
MNLI mismatched	2,996	92.9	86.8	89.3	89.9	92.8
ARC-Challenge	1,172	96.2	97.7	97.7	98.1	96.8
ARC-Easy	2,376	99.1	99.4	99.2	99.6	99.4

Long legal documents (accuracy, %)

Benchmark	Questions	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
LexGLUE ECtHR-A	10,000	93.9	92.7	94.5	93.0	93.4
LexGLUE ECtHR-B	10,000	94.9	91.9	94.3	93.4	94.3
LexGLUE SCOTUS	1,317	58.8	71.6	71.1	67.6	78.7
LexGLUE SCOTUS, opinions within 16K tokens	989	78.3	71.8	71.0	67.1	79.3
MAUD	6,651	68.0	64.4	64.0	60.9	66.7
JevBench	1,938	90.0	90.5	90.5	88.1	92.0

CUAD clause retrieval over contract paragraphs (mean AUPR, 40 clause types)

Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
0.651	0.534	0.521	0.466	0.554

LegalBench (mean task accuracy, %)

Task group	Tasks	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
CUAD clauses	38	88.9	84.4	85.4	83.0	85.1
MAUD	34	77.6	79.3	79.2	72.1	76.4
ContractNLI	14	96.3	92.3	94.5	92.6	93.1
Learned Hands	16	87.6	82.5	88.2	86.8	88.6
Supply chain	10	82.4	77.4	76.9	80.9	77.6
OPP-115 privacy	9	83.8	83.4	83.9	80.8	83.4
Diversity	6	94.4	92.2	82.7	74.1	94.1
Other	27	79.8	76.9	75.8	70.7	79.5
All tasks	154	84.9	82.3	82.7	79.1	83.1

Robustness (accuracy, %)

Test	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
BoolQ, negated question	92.1	90.4	83.1	53.5	84.8
BoolQ, swapped yes/no descriptions	92.7	9.8	91.0	91.2	92.9
CUAD, negated question (descriptions rewritten to match)	84.5	90.9	88.7	87.1	90.0
CUAD, swapped yes/no descriptions	84.4	11.3	90.9	88.3	92.4
UNFAIR-ToS, negated question (descriptions rewritten to match)	99.2	97.0	97.7	97.2	98.4
UNFAIR-ToS, swapped yes/no descriptions	98.7	2.5	97.5	97.2	98.6
ContractNLI, random option ids	90.1	77.4	87.8	87.9	80.6
ContractNLI, shuffled options	90.6	78.2	87.8	88.7	81.0
CaseHOLD, random option ids	85.0	77.9	75.2	69.4	86.4
CaseHOLD, shuffled options	85.7	76.8	75.9	69.5	86.3
MNLI, random option ids	92.3	86.9	89.0	89.4	93.0
MNLI, shuffled options	92.6	87.3	89.6	89.1	92.9
ARC-Challenge, random option ids	97.1	97.4	97.8	98.0	96.8
ARC-Challenge, shuffled options	96.8	97.8	97.7	98.1	97.3

Decision Index (13 public benchmarks)

Benchmark	Metric	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
BANKING77	F1	88.5	79.7	94.1	90.8	80.6
CLINC150	F1	93.7	89.3	97.4	66.9	91.5
ContractNLI	F1	86.1	71.7	81.3	84.5	78.4
ANLI	F1	73.2	74.8	70.0	59.1	74.3
MMLU	accuracy	83.5	91.7	90.2	91.9	87.7
ARC-Easy	accuracy	99.0	99.3	99.0	99.5	99.3
ARC-Challenge	accuracy	96.6	97.8	97.8	98.2	97.0
WinoGrande	accuracy	86.5	92.0	93.5	97.5	92.4
HellaSwag	accuracy	95.7	94.5	98.2	98.6	93.2
NLI4CT	F1	83.7	84.1	83.0	78.6	84.9
MMLU-Pro	accuracy	65.9	82.7	66.0	65.2	71.0
BBH	accuracy	80.7	92.9	73.6	68.9	84.0
RAGTruth	F1	81.5	76.5	79.5	35.6	83.1
Mean of 13		85.7	86.7	86.4	79.6	86.0

Outside-system figures: public Decision Index leaderboard.

ContractNLI against Jev-Benchmark systems (2,091 judgments, accuracy, %)

System	Its accuracy	Veyra Decide 27B
Gemini Pro	83.2	90.3
Terra	82.6	90.3
Gemini Flash-Lite	82.5	90.3
Claude Sonnet 5	82.4	90.3
Astra	81.8	90.3
Claude Haiku 4.5	79.4	90.3
Qwen 9B	79.1	90.3
Luna	79.0	90.3
Jev	77.4	90.3
Qwen 4B	74.2	90.3

Outside-system figures: published Jev-Benchmark answers.

Jev harness tasks (Jev's scorer, primary metric x100)

Task	Metric	Veyra Decide 27B	Jev	Clef	Clef-flash	Perplexity
afrixnli	accuracy	58.6	62.8	53.4	49.2	70.8
ag_news	accuracy	88.9	87.5	89.3	89.3	89.8
agb_de	f1	28.0	20.4	17.9	7.0	23.5
anli	accuracy	74.6	74.3	62.1	57.1	72.5
arc	accuracy	97.5	98.5	98.6	99.2	98.0
art	accuracy	86.2	83.9	88.3	84.3	88.2
banking77	accuracy	87.7	79.2	94.0	95.2	82.0
belebele	accuracy	79.5	87.5	82.3	73.5	85.7
bigbench	accuracy	81.3*	83.7*	77.5*	72.2*	83.6
boolq	accuracy	92.6	91.0	91.2	91.0	92.3
ceval	accuracy	82.2	84.5	80.0	77.7	84.2
clinc150	accuracy	93.3	89.8	96.7	98.3	90.8
commonsense_qa	accuracy	86.4	88.2	89.4	92.9	88.6
emotion	accuracy	60.0	58.5	59.3	58.1	61.3
financial_phrasebank	accuracy	70.9	73.0	72.6	74.5	85.3
go_emotions	macro f1	31.9	24.1	46.5	50.3	35.3
hellaswag	accuracy	96.7	95.5	98.4	98.8	94.2
helpsteer2	mean spearman	43.4	41.2	43.5	38.6	50.8
imdb	accuracy	96.2	96.2	96.8	96.2	96.4
language_id	accuracy	99.9	99.7	99.8	98.0	100.0
mmlu	accuracy	84.5	91.8	89.3	90.5	88.3
openai_moderation	mean auprc	75.7	71.7	74.4	71.4	75.4
paws	accuracy	81.5	85.9	80.0	74.8	93.5
prompt_injections	accuracy	78.4	74.1	81.0	58.6	94.0
pubmedqa	accuracy	81.2	78.7	75.3	76.6	79.8
rotten_tomatoes	accuracy	92.4	93.3	92.4	91.9	92.9
sib200	accuracy	80.8	83.0	79.3*	74.9*	81.2
sms_spam	f1	97.6	95.5	87.2	88.4	98.1
sst2	accuracy	96.9	96.4	96.8	95.6	96.7
sst5	argmax accuracy	51.7	58.5	58.5	57.6	55.2
stsb	spearman	92.4	89.0	90.3	88.3	93.1
summeval	mean group spearman	54.6	55.4	53.8	47.9	56.6
toxic_chat	f1	81.5	78.0	61.9	51.2	82.7
toxigen	accuracy	84.4	87.8	70.0	77.1	85.2
unfair_tos	micro f1	64.9	49.9	53.5	54.6	56.9
winogrande	accuracy	86.7	91.4	92.2	97.0	91.6
Mean of 36		78.4	77.8	77.0	74.9	80.4

* Some items unanswered by this system.

Licence

Apache-2.0 for now; a dedicated Veyra licence is in preparation. See LICENSE.

About JudicialMind

JudicialMind builds an orchestrated system of specialized AI agents that work like a legal team, composed into workflows that run end-to-end legal operations: intake, research, drafting, negotiation and dispute resolution.

• 🌐 judicialmind.ai
• 📧 surya@judicialmind.ai
• 🧑‍⚖️ Early access

Citation

@misc{judicialmind2026veyradecide27b,
  title        = {Veyra Decide 27B (Legal \& General)},
  author       = {{JudicialMind}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/judicialmind/Veyra-Decide-27B-Legal-General}}
}