license: apache-2.0
tags:

• 视觉
• dinov2
• 深度估计
  inference: false

模型卡片：基于DINOv2骨干的DPT模型

模型详情

基于DINOv2骨干的DPT（密集预测变换器）模型，由Oquab等人在论文《DINOv2：无监督学习鲁棒视觉特征》中提出。论文链接：DINOv2: Learning Robust Visual Features without Supervision。

DPT架构图。摘自原论文。

资源

• DINOv2论文
• DPT论文

与Transformers库结合使用

from transformers import AutoImageProcessor, DPTForDepthEstimation  
import torch  
import numpy as np  
from PIL import Image  
import requests  

url = "http://images.cocodataset.org/val2017/000000039769.jpg"  
image = Image.open(requests.get(url, stream=True).raw)  

image_processor = AutoImageProcessor.from_pretrained("facebook/dpt-dinov2-giant-kitti")  
model = DPTForDepthEstimation.from_pretrained("facebook/dpt-dinov2-giant-kitti")  

# 为模型准备图像  
inputs = image_processor(images=image, return_tensors="pt")  

with torch.no_grad():  
    outputs = model(**inputs)  
    predicted_depth = outputs.predicted_depth  

# 插值至原始尺寸  
prediction = torch.nn.functional.interpolate(  
    predicted_depth.unsqueeze(1),  
    size=image.size[::-1],  
    mode="bicubic",  
    align_corners=False,  
)  

# 可视化预测结果  
output = prediction.squeeze().cpu().numpy()  
formatted = (output * 255 / np.max(output)).astype("uint8")  
depth = Image.fromarray(formatted)  

模型用途

预期用途

该模型旨在证明：使用DINOv2作为骨干网络的DPT框架能够实现强大的深度估计能力。

BibTeX引用信息

@misc{oquab2023dinov2,  
      title={DINOv2: Learning Robust Visual Features without Supervision},   
      author={Maxime Oquab and Timothée Darcet and Théo Moutakanni and Huy Vo and Marc Szafraniec and Vasil Khalidov and Pierre Fernandez and Daniel Haziza and Francisco Massa and Alaaeldin El-Nouby and Mahmoud Assran and Nicolas Ballas and Wojciech Galuba and Russell Howes and Po-Yao Huang and Shang-Wen Li and Ishan Misra and Michael Rabbat and Vasu Sharma and Gabriel Synnaeve and Hu Xu and Hervé Jegou and Julien Mairal and Patrick Labatut and Armand Joulin and Piotr Bojanowski},  
      year={2023},  
      eprint={2304.07193},  
      archivePrefix={arXiv},  
      primaryClass={cs.CV}  
}