CNPE-7B

CNPE-7B is the model implementation of CNPE (Comparison-Native framework for Paper Evaluation), introduced in From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation. It is built on Qwen2.5-7B-Instruct and evaluates scientific papers by comparing them against each other rather than assigning absolute scores in isolation.

Training Data

The model was trained on ICLR 2025 submissions collected from OpenReview, following the train-test split of DeepReview to ensure fair comparison, with the average human reviewer score used as ground truth. Qwen2.5-7B-Instruct was chosen as the base model.

Results

Performance on the ICLR 2025 test set, covering acceptance-decision metrics (Accuracy, F1, AUC, Cohen's κ) and ranking metrics (Spearman's ρ, pairwise accuracy, MAP@20, NDCG@20):

Method Acc. F1 AUC κ Spearman ρ Pair. Acc. MAP@20 NDCG@20
DeepReview-14B 0.6845 0.6254 0.6624 0.2510 0.4014 0.6419 0.1478 0.7204
CNPE-7B 0.7192 0.6732 0.7408 0.3464 0.4091 0.6448 0.7076 0.8153

CNPE-7B achieves the best result on every metric. With only 7B parameters, it obtains an average relative improvement of 21.8% over the strong baseline DeepReview-14B.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "zpjbtdjm/CNPE-7B", torch_dtype="bfloat16", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("zpjbtdjm/CNPE-7B")

For the full pipeline (pair sampling, training, and preference aggregation), see the official code repository: ECNU-Text-Computing/ComparisonReview.

Responsible Use

CNPE-7B is a research system for studying automated paper evaluation. Its judgments are advisory evidence and should not replace human peer review.

Citation

@inproceedings{zheng2026isolated,
  title={From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation},
  author={Zheng, Pujun and Yao, Jiacheng and Zheng, Jinquan and Gu, Chenyang and He, Guoxiu and Liu, Jiawei and Huang, Yong and Guo, Tianrui and Lu, Wei},
  booktitle={Findings of the Association for Computational Linguistics: ACL 2026},
  pages={23863--23883},
  year={2026}
}
Downloads last month
21
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for zpjbtdjm/CNPE-7B

Base model

Qwen/Qwen2.5-7B
Finetuned
(3219)
this model