license: apache-2.0
tags:

• 视觉
• 深度估计
• dinov2
  inference: false

模型卡：基于DINOv2骨干网络的DPT模型

模型详情

基于DINOv2骨干网络的DPT（密集预测变换器）模型，由Oquab等人在论文《DINOv2: 无监督学习鲁棒视觉特征》中提出。

DPT架构图。摘自原论文。

资源

• DINOv2论文
• DPT论文

与Transformers库配合使用

from transformers import AutoImageProcessor, DPTForDepthEstimation  
import torch  
import numpy as np  
from PIL import Image  
import requests  

url = "http://images.cocodataset.org/val2017/000000039769.jpg"  
image = Image.open(requests.get(url, stream=True).raw)  

image_processor = AutoImageProcessor.from_pretrained("facebook/dpt-dinov2-base-kitti")  
model = DPTForDepthEstimation.from_pretrained("facebook/dpt-dinov2-base-kitti")  

# 为模型准备图像  
inputs = image_processor(images=image, return_tensors="pt")  

with torch.no_grad():  
    outputs = model(**inputs)  
    predicted_depth = outputs.predicted_depth  

# 插值至原始尺寸  
prediction = torch.nn.functional.interpolate(  
    predicted_depth.unsqueeze(1),  
    size=image.size[::-1],  
    mode="bicubic",  
    align_corners=False,  
)  

# 可视化预测结果  
output = prediction.squeeze().cpu().numpy()  
formatted = (output * 255 / np.max(output)).astype("uint8")  
depth = Image.fromarray(formatted)  

模型用途

预期用途

该模型旨在展示使用DINOv2作为骨干网络的DPT框架能够实现强大的深度估计能力。

BibTeX条目及引用信息

@misc{oquab2023dinov2,  
      title={DINOv2: 无监督学习鲁棒视觉特征},  
      author={Maxime Oquab and Timothée Darcet and Théo Moutakanni and Huy Vo and Marc Szafraniec and Vasil Khalidov and Pierre Fernandez and Daniel Haziza and Francisco Massa and Alaaeldin El-Nouby and Mahmoud Assran and Nicolas Ballas and Wojciech Galuba and Russell Howes and Po-Yao Huang and Shang-Wen Li and Ishan Misra and Michael Rabbat and Vasu Sharma and Gabriel Synnaeve and Hu Xu and Hervé Jegou and Julien Mairal and Patrick Labatut and Armand Joulin and Piotr Bojanowski},  
      year={2023},  
      eprint={2304.07193},  
      archivePrefix={arXiv},  
      primaryClass={cs.CV}  
}