license: apache-2.0
tags:

• 视觉
• 深度估计
• dinov2
  inference: false

模型卡片：基于DINOv2骨干网络的DPT模型

模型详情

本模型采用DINOv2作为骨干网络的DPT（密集预测变换器）模型，源自Oquab等人提出的论文《DINOv2: 无监督学习鲁棒视觉特征》。

DPT架构图。引自原论文。

资源

• DINOv2论文
• DPT论文

与Transformers库配合使用

from transformers import AutoImageProcessor, DPTForDepthEstimation  
import torch  
import numpy as np  
from PIL import Image  
import requests  

url = "http://images.cocodataset.org/val2017/000000039769.jpg"  
image = Image.open(requests.get(url, stream=True).raw)  

image_processor = AutoImageProcessor.from_pretrained("facebook/dpt-dinov2-base-nyu")  
model = DPTForDepthEstimation.from_pretrained("facebook/dpt-dinov2-base-nyu")  

# 为模型准备图像  
inputs = image_processor(images=image, return_tensors="pt")  

with torch.no_grad():  
    outputs = model(**inputs)  
    predicted_depth = outputs.predicted_depth  

# 插值至原始尺寸  
prediction = torch.nn.functional.interpolate(  
    predicted_depth.unsqueeze(1),  
    size=image.size[::-1],  
    mode="bicubic",  
    align_corners=False,  
)  

# 可视化预测结果  
output = prediction.squeeze().cpu().numpy()  
formatted = (output * 255 / np.max(output)).astype("uint8")  
depth = Image.fromarray(formatted)  

模型用途

预期用途

该模型旨在证明：采用DINOv2作为骨干网络的DPT框架能实现强大的深度估计功能。

BibTeX引用信息

@misc{oquab2023dinov2,  
      title={DINOv2: 无监督学习鲁棒视觉特征},  
      author={Maxime Oquab and Timothée Darcet and Théo Moutakanni and Huy Vo and Marc Szafraniec and Vasil Khalidov and Pierre Fernandez and Daniel Haziza and Francisco Massa and Alaaeldin El-Nouby and Mahmoud Assran and Nicolas Ballas and Wojciech Galuba and Russell Howes and Po-Yao Huang and Shang-Wen Li and Ishan Misra and Michael Rabbat and Vasu Sharma and Gabriel Synnaeve and Hu Xu and Hervé Jegou and Julien Mairal and Patrick Labatut and Armand Joulin and Piotr Bojanowski},  
      year={2023},  
      eprint={2304.07193},  
      archivePrefix={arXiv},  
      primaryClass={cs.CV}  
}