标签:

• 图像转文本
• 图像描述生成 许可证: apache-2.0 微件:
• 示例图片: src: >- https://huggingface.co/datasets/mishig/sample_images/resolve/main/savanna.jpg 标题示例: 热带草原
• 示例图片: src: >- https://huggingface.co/datasets/mishig/sample_images/resolve/main/football-match.jpg 标题示例: 足球比赛
• 示例图片: src: >- https://huggingface.co/datasets/mishig/sample_images/resolve/main/airport.jpg 标题示例: 机场 复制来源: nlpconnect/vit-gpt2-image-captioning

nlpconnect/vit-gpt2-image-captioning

这是一个由@ydshieh在flax中训练的图片描述生成模型，这是此模型的PyTorch版本。

使用transformers的图解图像描述生成

• 详情请见: https://ankur3107.github.io/blogs/the-illustrated-image-captioning-using-transformers/

示例运行代码

from transformers import VisionEncoderDecoderModel, ViTImageProcessor, AutoTokenizer
import torch
from PIL import Image

model = VisionEncoderDecoderModel.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
feature_extractor = ViTImageProcessor.from_pretrained("nlpconnect/vit-gpt2-image-captioning")
tokenizer = AutoTokenizer.from_pretrained("nlpconnect/vit-gpt2-image-captioning")

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)