Instructions to use txmedai/vector-surgical-sam2-endovis18 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use txmedai/vector-surgical-sam2-endovis18 with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained(txmedai/vector-surgical-sam2-endovis18) with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained(txmedai/vector-surgical-sam2-endovis18) with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>): # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
Surgical SAM 2 โ Vector package
Pinned Vector runtime package for the Surgical SAM 2 small checkpoint fine-tuned on EndoVis 2018 instrument masks. Its declared capability is prompted image segmentation: an RGB image and a bounding-box prompt produce an instrument mask and the model's predicted IoU. It is not an autonomous detector and not an RL policy.
Provenance:
- Upstream code and checkpoint:
jinlab-imvr/Surgical-SAM-2(Apache-2.0) - Seldinger runtime fork:
SeldingerMed/Surgical-SAM-2@fd424ac18126728d92125140f87374fdaf8ae78a - Checkpoint SHA-256:
c6202ee710badf747b4bf93fa3bfd8d82555e89d8ca3ff1e827493052d282955 - Training domain: EndoVis 2018
- Intended Vector evaluation domain: held-out EndoVis 2017 validation frames with box prompts derived from reference masks
The cross-dataset result is technical segmentation evidence only. It does not establish autonomous instrument detection, procedural safety, or clinical safety.
- Downloads last month
- -