Instructions to use deepseek-ai/DeepSeek-V4-Flash-Vision-Exp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4-Flash-Vision-Exp with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="deepseek-ai/DeepSeek-V4-Flash-Vision-Exp")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-Vision-Exp")
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4-Flash-Vision-Exp with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp

SGLang

How to use deepseek-ai/DeepSeek-V4-Flash-Vision-Exp with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4-Flash-Vision-Exp with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
```

Waiting for 2x NVIDIA DGX Sparks supported

#10

by reswewr - opened about 21 hours ago

Discussion

reswewr

about 21 hours ago

That's better if 2x NVIDIA DGX Sparks can run it with nvfp4 kv cache 😀

calvarado2004

about 21 hours ago

I'm testing MIA's recipe https://github.com/MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark/

sandw

about 11 hours ago

•

edited about 11 hours ago

Man, you guys are all big shots.

alaistair

about 6 hours ago

•

edited about 6 hours ago

I'm testing MIA's recipe https://github.com/MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark/

Am having issues on the vision part. It's looking a flakey at the moment. Have got Claude taking a look:

"The model isn't one brain — it's 256 small specialists. For every piece of text it reads, it picks the best 6 of those 256 to do the work.

The model ships with two different rulebooks for picking specialists: one for text, and a separate one for images. That image rulebook is a real, trained part of the model — it's in the downloaded weights, and it picks noticeably different specialists than the text one.

The bug: the image rulebook gets loaded into memory and then never opened. Nothing in the code ever reads it. So every piece of every image gets its specialists chosen by the text rulebook instead.

I measured it on a real image request: on average 5 of the 6 chosen specialists are the wrong ones, and this happens in all 43 layers of the model. In the first 3 layers it's worse — every piece of every image goes to the exact same 6 specialists, while text spreads across all 256.

Why it happened is subtle and slightly unlucky. The original code marks image pieces with special ID numbers that sit outside the normal vocabulary, so the code can spot them by asking "is this number too big to be a word?" Our version — correctly, for how the serving engine works — switched to using a real in-vocabulary ID number instead. That was the right call, but it silently broke the "is this an image?" test that the specialist-picker depended on. And because the new ID is a valid word ID, the lookup still succeeds; it just quietly returns the wrong answer instead of erroring.

Why nobody spotted it: ask it "what colour is the top-left corner?" and it still answers "red", correctly. Naming four bright colours survives bad routing easily. You'd only see the damage on harder work — reading small text, counting objects, detailed descriptions. It answered a test perfectly during the very run that proved it was misrouting.

There's a second, smaller issue: with a large image the model can only "see" about a third of that image at a time, where it was trained to see the whole thing at once.

Two things worth knowing:

Text-only work is completely unaffected. With no image in the prompt, the code path is identical to the original. So agentic coding on this model is fine.
The picture-handling itself is correct. Cutting up, measuring, converting to numbers, slotting into the text — all verified as exactly right, down to identical values. The bug is purely in what happens to those image pieces afterwards, inside the model.

Practical upshot: don't trust any vision benchmark until this is fixed — you'd be measuring the bug, not the model."

EDIT: This was based on my knowledge last night (31st August 2026) - looks like some updates since then. Checking now (1st September 2026)

BPAM

about 3 hours ago

•

edited about 3 hours ago

Working config on 2x DGX Spark (GB10, 128 GB each, TP=2 over 200G RoCE), official FP8 weights @ rev 86f746b:

vLLM 0.21.1rc1.dev339+g1967a5627bc3 (the tonyd2wild DSpark/NVFP4-KV recipe lineage, base image pinned by digest) + the vision-exp-default branch port files generated at commit d39f94a
--hf-overrides '{"architectures":["DeepseekV4VForConditionalGeneration"]}' + the branch's registry alias; clear $VLLM_CACHE_ROOT/modelinfos on BOTH nodes before first boot
--kv-cache-dtype nvfp4_ds_mla --block-size 256, DSpark speculative decoding k=5 — the branch's k=3 recommendation was measured with the Patch-4 mount missing; with it restored, k=5 wins count by ~33% and is neutral elsewhere (details: https://github.com/tonyd2wild/DeepSeek-v4-Flash-Vision-Exp-DSpark-1M-NVFP4-KV-2x-DGX-Spark/issues/48)
gpu_memory_utilization 0.78, max-cudagraph-capture-size = max_num_seqs x (k+1)
Measured warm single-stream: count 85.5 / code 49.9 / prose 29.5 tok/s; KV pool at gmu 0.78 / 500K ctx / 2 seqs: 1.065–1.124M tokens across three boots (2.13–2.25x)
Vision smoke (color/count/position + simple rendered-text OCR) all exact via standard OpenAI image_url content parts

Re the routing analysis above: our smoke tests were simple-tier only and passed on this stack (vLLM port, ds4v_* @ d39f94a) — happy to run a harder counting/small-text repro to compare stacks.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment