语言: 德语
数据集:

• common_voice
  评估指标:
• 词错误率(WER)
• 字符错误率(CER)
  标签:
• 音频
• 自动语音识别
• 语音
• XLSR微调周
• Hugging Face ASR排行榜
  许可证: Apache-2.0
  模型索引:
• 名称: Florian Zimmermeister @A\Ware 的XLSR Wav2Vec2德语带语言模型
  结果:
  • 任务:
    名称: 语音识别
    类型: 自动语音识别
    数据集:
    名称: Common Voice 德语
    类型: common_voice
    参数: de
    指标:
    • 名称: 测试WER
      类型: wer
      值: 5.7467896819046755
    • 名称: 测试CER
      类型: cer
      值: 1.8980142607670552

测试结果

评估

该模型可按以下方式在Common Voice的德语测试数据上进行评估。

import torchaudio.functional as F  
import torch  
from transformers import AutoModelForCTC, AutoProcessor  
import re  
from datasets import load_dataset, load_metric  

CHARS_TO_IGNORE = [",", "?", "¿", ".", "!", "¡", ";", "；", ":", '""', "%", '"', "�", "ʿ", "·", "჻", "~", "՞",  
                   "؟", "،", "।", "॥", "«", "»", "„", "“", "”", "「", "」", "‘", "’", "《", "》", "(", ")", "[", "]",  
                   "{", "}", "=", "`", "_", "+", "<", ">", "…", "–", "°", "´", "ʾ", "‹", "›", "©", "®", "—", "→", "。",  
                   "、", "﹂", "﹁", "‧", "～", "﹏", "，", "｛", "｝", "（", "）", "［", "］", "【", "】", "‥", "〽",  
                   "『", "』", "〝", "〟", "⟨", "⟩", "〜", "：", "！", "？", "♪", "؛", "/", "\\", "º", "−", "^", "ʻ", "ˆ"]  

chars_to_ignore_regex = f"[{re.escape(''.join(CHARS_TO_IGNORE))}]"  

counter = 0  
wer_counter = 0  
cer_counter = 0  

def main():  
    model = AutoModelForCTC.from_pretrained("flozi00/wav2vec2-large-xlsr-53-german-with-lm")  
    processor = AutoProcessor.from_pretrained("flozi00/wav2vec2-large-xlsr-53-german-with-lm")  

    wer = load_metric("wer")  
    cer = load_metric("cer")  

    ds = load_dataset("common_voice", "de", split="test")  
    #ds = ds.select(range(100))  

    def calculate_metrics(batch):  
        global counter, wer_counter, cer_counter  
        resampled_audio = F.resample(torch.tensor(batch["audio"]["array"]), 48_000, 16_000).numpy()  

        input_values = processor(resampled_audio, return_tensors="pt", sampling_rate=16_000).input_values  

        with torch.no_grad():  
            logits = model(input_values).logits.numpy()[0]  

        decoded = processor.decode(logits)  
        pred = decoded.text  

        ref = re.sub(chars_to_ignore_regex, "", batch["sentence"]).upper()  

        wer_result = wer.compute(predictions=[pred], references=[ref])  
        cer_result = cer.compute(predictions=[pred], references=[ref])  

        counter += 1  
        wer_counter += wer_result  
        cer_counter += cer_result  

        print(f"WER: {(wer_counter/counter)*100} | CER: {(cer_counter/counter)*100}")  

        return batch  

    ds.map(calculate_metrics, remove_columns=ds.column_names)  

main()  

致谢:

声学模型是jonatasgrosman的模型的副本，我用于训练匹配的kenlm语言模型。