license: apache-2.0
tags:

• 视觉
• 深度估计
• dinov2
  inference: false

模型卡片：基于DINOv2骨干网络的DPT模型

模型详情

DPT（密集预测变换器）模型采用DINOv2骨干网络，如Oquab等人在论文DINOv2: 无监督学习鲁棒视觉特征中提出的架构。

DPT架构图。引自原论文。

资源

• DINOv2论文
• DPT论文

与Transformers库配合使用

from transformers import AutoImageProcessor, DPTForDepthEstimation  
import torch  
import numpy as np  
from PIL import Image  
import requests  

url = "http://images.cocodataset.org/val2017/000000039769.jpg"  
image = Image.open(requests.get(url, stream=True).raw)  

image_processor = AutoImageProcessor.from_pretrained("facebook/dpt-dinov2-small-nyu")  
model = DPTForDepthEstimation.from_pretrained("facebook/dpt-dinov2-small-nyu")  

# 为模型准备图像  
inputs = image_processor(images=image, return_tensors="pt")  

with torch.no_grad():  
    outputs = model(**inputs)  
    predicted_depth = outputs.predicted_depth  

# 插值至原始尺寸  
prediction = torch.nn.functional.interpolate(  
    predicted_depth.unsqueeze(1),  
    size=image.size[::-1],  
    mode="bicubic",  
    align_corners=False,  
)  

# 可视化预测结果  
output = prediction.squeeze().cpu().numpy()  
formatted = (output * 255 / np.max(output)).astype("uint8")  
depth = Image.fromarray(formatted)  

模型用途

预期用途

该模型旨在证明：采用DINOv2作为骨干网络的DPT框架能够实现强大的深度估计能力。

BibTeX引用信息

@misc{oquab2023dinov2,  
      title={DINOv2: 无监督学习鲁棒视觉特征},  
      author={Maxime Oquab and Timothée Darcet and Théo Moutakanni and Huy Vo and Marc Szafraniec and Vasil Khalidov and Pierre Fernandez and Daniel Haziza and Francisco Massa and Alaaeldin El-Nouby and Mahmoud Assran and Nicolas Ballas and Wojciech Galuba and Russell Howes and Po-Yao Huang and Shang-Wen Li and Ishan Misra and Michael Rabbat and Vasu Sharma and Gabriel Synnaeve and Hu Xu and Hervé Jegou and Julien Mairal and Patrick Labatut and Armand Joulin and Piotr Bojanowski},  
      year={2023},  
      eprint={2304.07193},  
      archivePrefix={arXiv},  
      primaryClass={cs.CV}  
}