DeepMiRT: miRNA Target Prediction with RNA Foundation Models

Model Description

DeepMiRT predicts miRNA-target interactions using RNA-FM embeddings and cross-attention. It ranks #1 on eCLIP benchmarks among 12 methods (AUROC 0.7511 / 0.7524) and achieves AUROC 0.96 on a held-out test set of 813K samples.

Architecture

Encoder: RNA-FM T12 (12-layer Transformer, pre-trained on 23M ncRNAs) — shared for both miRNA and target
Interaction: Cross-attention (2 layers, 8 heads) — target queries miRNA
Classifier: MLP (640 → 256 → 64 → 1)

The RNA-FM backbone is frozen. Only the cross-attention module and classifier head are trained (~4M of 103M parameters).

Usage

pip install deepmirt

from deepmirt import predict

probs = predict(
    mirna_seqs=["UGAGGUAGUAGGUUGUAUAGUU"],
    target_seqs=["ACUGCAGCAUAUCUACUAUUUGCUACUGUAACCAUUGAUCU"],
)
print(f"Interaction probability: {probs[0]:.4f}")

Training

Data: 8.1 million miRNA-target pairs from miRTarBase, TarBase v9, miRBench, CLASH, and related sources
Released model: frozen RNA-FM backbone; only cross-attention and the MLP head were optimized
Checkpoint: epoch=27-val_auroc=0.9612.ckpt (this file)
Hardware: 2x NVIDIA L20 GPUs, mixed precision (fp16)
Progressive unfreezing of the top 3 RNA-FM layers was tried and did not improve over the frozen-backbone result (best unfrozen val AUROC 0.9608 vs frozen 0.9612), so the frozen checkpoint is the released model.

Performance

Numbers below are from the paper/mirbench_aps evaluation snapshot used in the paper and GitHub README.

Benchmark	AUROC	APS	Rank
miRBench eCLIP (Klimentova 2022)	0.7511	0.7850	#1/12
miRBench eCLIP (Manakov 2022)	0.7524	0.7947	#1/12
miRBench CLASH (Hejret 2023)	0.6952	0.7260	#5/12
Internal test set (813K samples)	0.9606	0.9669	#1/16

The 0.96 AUROC is the internal 813K held-out test set, not the eCLIP benchmarks.

Files

epoch=27-val_auroc=0.9612.ckpt — Best frozen-backbone checkpoint (495 MB)
config.yaml — Training configuration used for the released model (freeze_backbone: true, unfreezing.enabled: false)

License

MIT

Downloads last month: 16

liuliu2333
/

deepmirt