许可证: mit 语言:

• 拉丁语 数据集:
• CATMuS/medieval 标签:
• trocr
• 图像转文本 演示样例:
• 图片: https://huggingface.co/medieval-data/trocr-medieval-castilian-hybrida/resolve/main/images/hybrida-1.png 示例标题: Hybrida 1
• 图片: https://huggingface.co/medieval-data/trocr-medieval-castilian-hybrida/resolve/main/images/hybrida-2.png 示例标题: Hybrida 2
• 图片: https://huggingface.co/medieval-data/trocr-medieval-castilian-hybrida/resolve/main/images/hybrida-3.png 示例标题: Hybrida 3

简介

这是一个针对中世纪卡斯蒂利亚语（特别是Hybrida手写体）的TrOCR模型。基础模型为microsoft/trocr-base-handwritten。该模型首先微调至Caroline字体：medieval-data/trocr-medieval-latin-caroline，然后基于保存的检查点进一步微调至卡斯蒂利亚Hybrida字体。

训练使用的数据集是CATMuS。

该模型尚未经过正式测试。初步检查表明需要进一步微调。

微调过程使用了本代码库中的finetune.py脚本。

使用方法

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
import requests

# 从IAM数据库加载图像
url = 'https://huggingface.co/medieval-data/trocr-medieval-castilian-hybrida/resolve/main/images/hybrida-1.png'
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")

processor = TrOCRProcessor.from_pretrained('medieval-data/trocr-medieval-castilian-hybrida')
model = VisionEncoderDecoderModel.from_pretrained('medieval-data/trocr-medieval-castilian-hybrida')
pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

参考文献与引用信息

TrOCR论文

@misc{li2021trocr,
      title={TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models}, 
      author={Minghao Li and Tengchao Lv and Lei Cui and Yijuan Lu and Dinei Florencio and Cha Zhang and Zhoujun Li and Furu Wei},
      year={2021},
      eprint={2109.10282},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

CATMuS论文

@unpublished{clerice:hal-04453952,
  TITLE = {{CATMuS Medieval: A multilingual large-scale cross-century dataset in Latin script for handwritten text recognition and beyond}},
  AUTHOR = {Cl{\'e}rice, Thibault and Pinche, Ariane and Vlachou-Efstathiou, Malamatenia and Chagu{\'e}, Alix and Camps, Jean-Baptiste and Gille-Levenson, Matthias and Brisville-Fertin, Olivier and Fischer, Franz and Gervers, Michaels and Boutreux, Agn{\`e}s and Manton, Avery and Gabay, Simon and O'Connor, Patricia and Haverals, Wouter and Kestemont, Mike and Vandyck, Caroline and Kiessling, Benjamin},
  URL = {https://inria.hal.science/hal-04453952},
  NOTE = {working paper or preprint},
  YEAR = {2024},
  MONTH = Feb,
  KEYWORDS = {Historical sources ; medieval manuscripts ; Latin scripts ; benchmarking dataset ; multilingual ; handwritten text recognition},
  PDF = {https://inria.hal.science/hal-04453952/file/ICDAR24___CATMUS_Medieval-1.pdf},
  HAL_ID = {hal-04453952},
  HAL_VERSION = {v1},
}