语言:

• 英语 标签:
• 问答系统 许可证: Apache-2.0 数据集:
• NewsQA 评估指标:
• F1值
• 精确匹配率

基于NewsQA微调的BERT Base Uncased模型

该BERT（Base）模型使用Transformers库中修改版的run_squad.py传统脚本在NewsQA数据集上进行微调。训练过程中排除了包含noAnswer和badQuestion的样本。

$ cd ~/projects/transformers/examples/legacy/question-answering
$ mkdir bert_base_uncased_finetuned_newsqa
$ python run_newsqa.py \
    --model_type bert \
    --model_name_or_path "bert-base-uncased" \
    --do_train \
    --do_eval \
    --do_lower_case \
    --num_train_epochs 2 \
    --per_gpu_train_batch_size 8 \
    --per_gpu_eval_batch_size 32 \
    --max_seq_length 384 \
    --max_grad_norm inf \
    --doc_stride 128 \
    --train_file "~/projects/data/newsqa/combined-newsqa-data-v1.json" \
    --predict_file "~/projects/data/newsqa/combined-newsqa-data-v1.json" \
    --output_dir "./bert_base_uncased_finetuned_newsqa" \
    --save_steps 20000

评估结果:

{'精确匹配率': 60.19350380096752, 'F1值': 73.29371985128037, '总样本数': 4341, '有答案精确匹配': 60.19350380096752, '有答案F1值': 73.29371985128037, '有答案样本数': 4341, '最佳精确匹配': 60.19350380096752, '最佳精确阈值': 0.0, '最佳F1值': 73.29371985128037, '最佳F1阈值': 0.0}

数据准备请参照NewsQA仓库说明。

评估微调后的模型:

python run_newsqa.py \
  --model_type bert \
  --model_name_or_path "./bert_large_uncased_finetuned_newsqa/checkpoint-700000" \
  --do_eval \
  --do_lower_case \
  --per_gpu_eval_batch_size 32 \
  --max_seq_length 384 \
  --max_grad_norm inf \
  --doc_stride 128 \
  --predict_file "~/projects/data/newsqa/combined-newsqa-data-v1.json" \
  --output_dir "./bert_large_uncased_finetuned_newsqa_eval"