license: apache-2.0 library_name: timm tags:

• 图像特征提取
• timm
• transformers datasets:
• imagenet-21k

vit_huge_patch14_224.orig_in21k 模型卡

一个基于Vision Transformer（ViT）的图像分类模型。由论文作者在ImageNet-21k上使用JAX预训练，后由Ross Wightman移植到PyTorch。该模型不含分类头，适用于特征提取和微调场景。

模型详情

• 模型类型: 图像分类/特征骨干网络
• 模型统计:
  • 参数量（百万）: 630.8
  • 计算量（GMACs）: 162.0
  • 激活值（百万）: 95.1
  • 图像尺寸: 224 x 224
• 相关论文:
  • 《An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale》: https://arxiv.org/abs/2010.11929v2
• 训练数据集: ImageNet-21k
• 原始实现: https://github.com/google-research/vision_transformer

使用示例

图像分类

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model('vit_huge_patch14_224.orig_in21k', pretrained=True)
model = model.eval()

# 获取模型专用预处理（标准化/尺寸调整）
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # 将单张图像扩展为batch_size=1

top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

图像嵌入

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model(
    'vit_huge_patch14_224.orig_in21k',
    pretrained=True,
    num_classes=0,  # 移除分类器nn.Linear
)
model = model.eval()

# 获取模型专用预处理
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # 输出为(batch_size, num_features)张量

# 等效方案（无需设置num_classes=0）
output = model.forward_features(transforms(img).unsqueeze(0))
# 输出为未池化的(1, 257, 1280)张量

output = model.forward_head(output, pre_logits=True)
# 输出为(1, num_features)张量

模型对比

查看该模型在timm中的测试结果，了解数据集和运行时指标。

引用文献

@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}

@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}