库名称: py-feat 管道标签: 图像特征提取 标签:

• 模型中心混合
• PyTorch模型中心混合 许可证: CC-BY-NC-4.0

img2pose

模型描述

img2pose采用Faster R-CNN技术预测照片中所有人脸的六自由度姿态（6DoF）。该模型的一个有趣特性是能够将3D人脸投影到2D平面，同时识别每张人脸的边界框，无需依赖其他人脸检测模型。

模型详情

• 模型类型: 卷积神经网络（CNN）
• 架构: Faster R-CNN
• 框架: PyTorch

模型来源

• 代码库: GitHub仓库
• 论文: img2pose: 通过6DoF人脸姿态估计实现人脸对齐与检测

引用

若在研究中应用本模型，请引用以下论文：

Vítor Albiero, Xingyu Chen, Xi Yin, Guan Pang, Tal Hassner, "img2pose: 通过6DoF人脸姿态估计实现人脸对齐与检测"，CVPR，2021，arXiv:2012.07791

@inproceedings{albiero2021img2pose,
  title={img2pose: 通过6DoF人脸姿态估计实现人脸对齐与检测},
  author={Albiero, Vítor and Chen, Xingyu and Yin, Xi and Pang, Guan and Hassner, Tal},
  booktitle={CVPR},
  year={2021},
  url={https://arxiv.org/abs/2012.07791},
}

致谢

感谢Albiero Vítor以宽松许可协议分享代码与训练权重。

使用示例

import numpy as np
import os
import json
import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from feat.facepose_detectors.img2pose.deps.models import FasterDoFRCNN, postprocess_img2pose
from feat.utils.io import get_resource_path
from torchvision.models.detection.backbone_utils import resnet_fpn_backbone


# 加载模型配置
facepose_config_file = hf_hub_download(repo_id= "py-feat/img2pose", filename="config.json", cache_dir=get_resource_path())
with open(facepose_config_file, "r") as f:
    facepose_config = json.load(f)
           
# 初始化img2pose
device = 'cpu'
backbone = resnet_fpn_backbone(backbone_name="resnet18", weights=None)
backbone.eval()
backbone.to(device)
facepose_detector = FasterDoFRCNN(backbone=backbone,
                            num_classes=2,
                            min_size=facepose_config['min_size'],
                            max_size=facepose_config['max_size'],
                            pose_mean=torch.tensor(facepose_config['pose_mean']),
                            pose_stddev=torch.tensor(facepose_config['pose_stddev']),
                            threed_68_points=torch.tensor(facepose_config['threed_points']),
                            rpn_pre_nms_top_n_test=facepose_config['rpn_pre_nms_top_n_test'],
                            rpn_post_nms_top_n_test=facepose_config['rpn_post_nms_top_n_test'],
                            bbox_x_factor=facepose_config['bbox_x_factor'],
                            bbox_y_factor=facepose_config['bbox_y_factor'],
                            expand_forehead=facepose_config['expand_forehead'])
facepose_model_file = hf_hub_download(repo_id= "py-feat/img2pose", filename="model.safetensors", cache_dir=get_resource_path())
facepose_checkpoint = load_file(facepose_model_file)
facepose_detector.load_state_dict(facepose_checkpoint)
facepose_detector.eval()
facepose_detector.to(device)

# 测试模型
face_image = "测试图片路径.jpg"  # 替换为实际图片路径

img2pose_output = facepose_detector(face_image)

# 后处理
img2pose_output = postprocess_img2pose(img2pose_output[0])
bbox = img2pose_output['boxes']
poses = img2pose_output['dofs']
facescores = img2pose_output['scores']