语言：英语
许可证：MIT
库名称：transformers
标签：

• 图像转文本

小部件示例：

• 图片1：积液双基底示例图
• 图片2：入院第二天拍摄的胸片示例
• 图片3：重复积液示例图

Chexpert-plus评估报告

使用方法：

import torch
from PIL import Image
from transformers import BertTokenizer, ViTImageProcessor, VisionEncoderDecoderModel, GenerationConfig
import requests

mode = "impression"
# 加载模型
model = VisionEncoderDecoderModel.from_pretrained(f"IAMJB/chexpert-mimic-cxr-{mode}-baseline").eval()
tokenizer = BertTokenizer.from_pretrained(f"IAMJB/chexpert-mimic-cxr-{mode}-baseline")
image_processor = ViTImageProcessor.from_pretrained(f"IAMJB/chexpert-mimic-cxr-{mode}-baseline")
#
# 生成参数配置
generation_args = {
   "bos_token_id": model.config.bos_token_id,
   "eos_token_id": model.config.eos_token_id,
   "pad_token_id": model.config.pad_token_id,
   "num_return_sequences": 1,
   "max_length": 128,
   "use_cache": True,
   "beam_width": 2,
}
#
# 推理过程
refs = []
hyps = []
with torch.no_grad():
   url = "https://huggingface.co/IAMJB/interpret-cxr-impression-baseline/resolve/main/effusions-bibasal.jpg"
   image = Image.open(requests.get(url, stream=True).raw)
   pixel_values = image_processor(image, return_tensors="pt").pixel_values
   # 生成预测文本
   generated_ids = model.generate(
       pixel_values,
       generation_config=GenerationConfig(
           **{**generation_args, "decoder_start_token_id": tokenizer.cls_token_id})
   )
   generated_texts = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
   print(generated_texts)

若使用本模型，请引用以下文献：

@misc{chambon2024chexpertplusaugmentinglarge,
      title={CheXpert Plus：增强版大型胸部X光数据集——整合放射学文本报告、患者人口统计及多模态影像}, 
      author={皮埃尔·尚邦、让-贝努瓦·德尔布鲁克、托马斯·苏纳克、黄士诚、陈志宏、玛雅·瓦尔玛、张清华、朱楚雄、柯蒂斯·P·朗洛茨},
      year={2024},
      eprint={2405.19538},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2405.19538}, 
}