许可协议：apache-2.0
标签：

• 视觉
  推理：false
  管道标签：零样本目标检测

Grounding DINO模型（基础变体）

Grounding DINO模型由刘世龙、曾昭阳、任天鹤、李峰、张浩、杨杰、李春元、杨建伟、苏航、朱军和张磊在论文《Grounding DINO：将DINO与接地预训练结合用于开放集目标检测》中提出。Grounding DINO通过添加文本编码器扩展了闭集目标检测模型，实现了开放集目标检测。该模型取得了显著成果，例如在COCO零样本检测上达到了52.5 AP。

Grounding DINO概览。图片取自原论文。

预期用途与限制

您可以使用原始模型进行零样本目标检测（即无需标注数据即可检测图像中的物体）。

使用方法

以下是使用该模型进行零样本目标检测的示例：

import requests  

import torch  
from PIL import Image  
from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection   

model_id = "IDEA-Research/grounding-dino-base"  
device = "cuda" if torch.cuda.is_available() else "cpu"  

processor = AutoProcessor.from_pretrained(model_id)  
model = AutoModelForZeroShotObjectDetection.from_pretrained(model_id).to(device)  

image_url = "http://images.cocodataset.org/val2017/000000039769.jpg"  
image = Image.open(requests.get(image_url, stream=True).raw)  
# 检测猫和遥控器  
# 非常重要：文本查询需小写并以句点结尾  
text = "a cat. a remote control."  

inputs = processor(images=image, text=text, return_tensors="pt").to(device)  
with torch.no_grad():  
    outputs = model(**inputs)  

results = processor.post_process_grounded_object_detection(  
    outputs,  
    inputs.input_ids,  
    box_threshold=0.4,  
    text_threshold=0.3,  
    target_sizes=[image.size[::-1]]  
)  

BibTeX条目及引用信息

@misc{liu2023grounding,  
      title={Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection},   
      author={Shilong Liu and Zhaoyang Zeng and Tianhe Ren and Feng Li and Hao Zhang and Jie Yang and Chunyuan Li and Jianwei Yang and Hang Su and Jun Zhu and Lei Zhang},  
      year={2023},  
      eprint={2303.05499},  
      archivePrefix={arXiv},  
      primaryClass={cs.CV}  
}