Visual Document Retrieval
Safetensors
ColPali
sentence-transformers
colpali_engine
qwen3_5
multimodal-retrieval
late-interaction
colqwen
ColQwen3_5
vidore
mteb
qwen3.5
model-merge
per-block-merge
MaxSim
multi-vector
Instructions to use vultr/VultronRetrieverPrime-Qwen3.5-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- ColPali
How to use vultr/VultronRetrieverPrime-Qwen3.5-8B with ColPali:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- sentence-transformers
How to use vultr/VultronRetrieverPrime-Qwen3.5-8B with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("vultr/VultronRetrieverPrime-Qwen3.5-8B") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Commit ·
c209084
1
Parent(s): 2dfb251
Declare causal retrieval attention contract
Browse files- README.md +10 -9
- config.json +1 -0
README.md
CHANGED
|
@@ -170,8 +170,9 @@ pip install causal-conv1d flash-linear-attention
|
|
| 170 |
```
|
| 171 |
|
| 172 |
- `causal-conv1d` + `flash-linear-attention` are required (the hybrid layers import them at runtime).
|
| 173 |
-
-
|
| 174 |
-
`
|
|
|
|
| 175 |
|
| 176 |
## Usage
|
| 177 |
|
|
@@ -206,20 +207,20 @@ print(scores.shape) # torch.Size([2, 2])
|
|
| 206 |
```
|
| 207 |
|
| 208 |
`config.json` carries `dim=320`, so `custom_text_proj` is sized correctly at load, with no manual
|
| 209 |
-
config edits needed.
|
|
|
|
| 210 |
|
| 211 |
## Serving with vLLM
|
| 212 |
|
| 213 |
vLLM serves this model natively through its pooling runner (the `ColQwen3_5` architecture), returning
|
| 214 |
the per-token multi-vectors for late-interaction scoring. It requires a vLLM build that includes the
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
prompt contract is applied **client-side**: wrap each page image in the instruction template, append
|
| 220 |
the query-augmentation tokens to each query, and set the visual-token budget through
|
| 221 |
-
`mm-processor-kwargs`. Prefix caching and chunked prefill
|
| 222 |
-
the GatedDeltaNet hybrid both break the causal-prefix invariant).
|
| 223 |
|
| 224 |
```python
|
| 225 |
import torch
|
|
|
|
| 170 |
```
|
| 171 |
|
| 172 |
- `causal-conv1d` + `flash-linear-attention` are required (the hybrid layers import them at runtime).
|
| 173 |
+
- This merged checkpoint was selected and evaluated with **causal** full-attention layers. Load its
|
| 174 |
+
`retrieval_attention_contract="causal"` metadata without overriding it.
|
| 175 |
+
- Use **SDPA**, matching the evaluation runtime of record.
|
| 176 |
|
| 177 |
## Usage
|
| 178 |
|
|
|
|
| 207 |
```
|
| 208 |
|
| 209 |
`config.json` carries `dim=320`, so `custom_text_proj` is sized correctly at load, with no manual
|
| 210 |
+
config edits needed. It also carries `retrieval_attention_contract="causal"`; a conflicting runtime
|
| 211 |
+
override is an error.
|
| 212 |
|
| 213 |
## Serving with vLLM
|
| 214 |
|
| 215 |
vLLM serves this model natively through its pooling runner (the `ColQwen3_5` architecture), returning
|
| 216 |
the per-token multi-vectors for late-interaction scoring. It requires a vLLM build that includes the
|
| 217 |
+
projection-bias fix and explicit retrieval-attention metadata support. The handler reads
|
| 218 |
+
`retrieval_attention_contract="causal"` from this checkpoint and constructs all eight full-attention
|
| 219 |
+
layers as `AttentionType.DECODER`; it must reject missing, conflicting, or unsupported contract
|
| 220 |
+
metadata. The server uses the stock chat/image processor, so the ColQwen3.5
|
| 221 |
prompt contract is applied **client-side**: wrap each page image in the instruction template, append
|
| 222 |
the query-augmentation tokens to each query, and set the visual-token budget through
|
| 223 |
+
`mm-processor-kwargs`. Prefix caching and chunked prefill remain off for the GatedDeltaNet hybrid.
|
|
|
|
| 224 |
|
| 225 |
```python
|
| 226 |
import torch
|
config.json
CHANGED
|
@@ -4,6 +4,7 @@
|
|
| 4 |
],
|
| 5 |
"image_token_id": 248056,
|
| 6 |
"model_type": "qwen3_5",
|
|
|
|
| 7 |
"text_config": {
|
| 8 |
"attention_bias": false,
|
| 9 |
"attention_dropout": 0.0,
|
|
|
|
| 4 |
],
|
| 5 |
"image_token_id": 248056,
|
| 6 |
"model_type": "qwen3_5",
|
| 7 |
+
"retrieval_attention_contract": "causal",
|
| 8 |
"text_config": {
|
| 9 |
"attention_bias": false,
|
| 10 |
"attention_dropout": 0.0,
|