IMvision12 commited on
Commit
2334c00
·
verified ·
1 Parent(s): 295cac7

Add zeromodels PVT weights + zm_config + model card

Browse files
Files changed (3) hide show
  1. README.md +82 -0
  2. model.weights.h5 +3 -0
  3. zm_config.json +44 -0
README.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-classification
3
+ license: apache-2.0
4
+ base_model: Zetatech/pvt-medium-224
5
+ library_name: zeromodels
6
+ tags:
7
+ - keras
8
+ - zeromodels
9
+ - image-classification
10
+ - pvt
11
+ - backbone
12
+ - arxiv:2102.12122
13
+ - pytorch
14
+ - jax
15
+ - tf
16
+ ---
17
+
18
+ ## ***See [our collection](https://huggingface.co/collections/zeromodels/pvt-and-pvtv2-6a90e9dd0a2b03a982d0b876) for all PVT and PVTv2 versions.***
19
+
20
+ # Run PVT with Keras 3: JAX, PyTorch, or TensorFlow
21
+
22
+ [![GitHub](https://img.shields.io/badge/GitHub-ZeroModels-black?logo=github)](https://github.com/IMvision12/ZeroModels) [![Docs](https://img.shields.io/badge/Docs-PVT-blue)](https://imvision12.github.io/ZeroModels/pvt/) [![Collection](https://img.shields.io/badge/HF-PVT%20collection-yellow)](https://huggingface.co/collections/zeromodels/pvt-and-pvtv2-6a90e9dd0a2b03a982d0b876)
23
+
24
+ # zeromodels/pvt-medium-224
25
+
26
+ Paper: [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions (arXiv:2102.12122)](https://arxiv.org/abs/2102.12122) · [HF Papers](https://huggingface.co/papers/2102.12122)
27
+
28
+ PVT is a hierarchical vision transformer: four pyramid stages with spatial-reduction attention over non-overlapping patches and learned position embeddings. Use `PvtImageClassify` for logits or `PvtModel` for tokens / per-stage features via `as_backbone=True`.
29
+
30
+ - Parameters: ~44.2M
31
+ - ImageNet-1k top-1: **81.2%**
32
+
33
+ For more details on the model, see the upstream [model card](https://huggingface.co/Zetatech/pvt-medium-224).
34
+
35
+ Pure-**Keras 3** conversion of [`Zetatech/pvt-medium-224`](https://huggingface.co/Zetatech/pvt-medium-224) for [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on **TensorFlow / Torch / JAX**.
36
+
37
+ This is an **image-classification / backbone** checkpoint (`PvtImageClassify` / `PvtModel`).
38
+
39
+ ## ✨ Quick start
40
+
41
+ ```python
42
+ import os
43
+ os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
44
+
45
+ from PIL import Image
46
+ import numpy as np
47
+ from zeromodels.models.pvt import PvtImageClassify, PvtModel
48
+
49
+ model = PvtImageClassify.from_weights("zeromodels/pvt-medium-224")
50
+ backbone = PvtModel.from_weights("zeromodels/pvt-medium-224", as_backbone=True)
51
+
52
+ image = Image.open("your_image.jpg").convert("RGB").resize((224, 224))
53
+ x = np.asarray(image, dtype="float32")[None] # (1, H, W, 3), raw [0, 255]
54
+ print(model(x).shape) # (1, num_classes)
55
+ feats = backbone(x)
56
+ print(len(feats), [tuple(f.shape) for f in feats]) # 4-stage feature pyramid
57
+ ```
58
+
59
+ Normalization is baked into the graph, so pass raw `[0, 255]` pixels. Load any PVT
60
+ variant the same way with `from_weights("zeromodels/<variant>")`:
61
+
62
+ | Variant | ImageNet-1k top-1 | Hub |
63
+ |---|---|---|
64
+ | `pvt-tiny-224` | 75.1% | [`zeromodels/pvt-tiny-224`](https://huggingface.co/zeromodels/pvt-tiny-224) |
65
+ | `pvt-small-224` | 79.8% | [`zeromodels/pvt-small-224`](https://huggingface.co/zeromodels/pvt-small-224) |
66
+ | `pvt-medium-224` | 81.2% | [`zeromodels/pvt-medium-224`](https://huggingface.co/zeromodels/pvt-medium-224) |
67
+ | `pvt-large-224` | 81.7% | [`zeromodels/pvt-large-224`](https://huggingface.co/zeromodels/pvt-large-224) |
68
+
69
+ ## Tips
70
+
71
+ - Set `KERAS_BACKEND` **before** importing Keras / zeromodels.
72
+ - `PvtImageClassify` returns class logits; `PvtModel` returns features (`as_backbone=True` for the four-stage pyramid).
73
+ - Both the model and its data format (`channels_last` / `channels_first`) are supported and bit-exact.
74
+ - See the [docs](https://imvision12.github.io/ZeroModels/pvt/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/).
75
+ - Upstream checkpoints load directly: `PvtImageClassify.from_weights("hf:Zetatech/pvt-medium-224")`.
76
+
77
+ ## Special Thanks
78
+
79
+ A huge thank you to the PVT authors ([whai362/PVT](https://github.com/whai362/PVT)) and the Hugging Face
80
+ community for creating and releasing these models.
81
+
82
+ License: see the YAML `license` above (matches the upstream checkpoint).
model.weights.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4edb71a433aee109701f22629dccbb4017a117b05b0649f5b4227585eb5af535
3
+ size 178076912
zm_config.json ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "library_name": "zeromodels",
3
+ "zeromodels_version": "1.2.6",
4
+ "model_module": "zeromodels.models.pvt",
5
+ "model_class": "PvtImageClassify",
6
+ "variant": "pvt-medium-224",
7
+ "weights": "model.weights.h5",
8
+ "schema_version": 2,
9
+ "model_type": "pvt",
10
+ "vision_config": {
11
+ "hidden_sizes": [
12
+ 64,
13
+ 128,
14
+ 320,
15
+ 512
16
+ ],
17
+ "depths": [
18
+ 3,
19
+ 4,
20
+ 18,
21
+ 3
22
+ ],
23
+ "num_attention_heads": [
24
+ 1,
25
+ 2,
26
+ 5,
27
+ 8
28
+ ],
29
+ "sr_ratios": [
30
+ 8,
31
+ 4,
32
+ 2,
33
+ 1
34
+ ],
35
+ "mlp_ratios": [
36
+ 8,
37
+ 8,
38
+ 4,
39
+ 4
40
+ ],
41
+ "image_size": 224,
42
+ "num_classes": 1000
43
+ }
44
+ }