language: zh datasets:

• SberDevices/Golos
• common_voice
• bond005/rulibrispeech
• bond005/sova_rudevices metrics:
• 词错误率(WER)
• 字错误率(CER) tags:
• 音频
• 自动语音识别
• 语音
• common_voice
• SberDevices/Golos
• bond005/rulibrispeech
• bond005/sova_rudevices
• dangrebenkin/voxforge-ru-dataset license: apache-2.0 widget:
• example_title: 俄语测试音频"нейросети это хорошо"（意为"神经网络很棒"） src: https://huggingface.co/bond005/wav2vec2-large-ru-golos-with-lm/resolve/main/test_sound_ru.flac model-index:
• name: Ivan Bondarenko开发的带语言模型的XLSR Wav2Vec2俄语模型 results:
  • task: name: 语音识别 type: 自动语音识别 dataset: name: Sberdevices Golos (众包) type: SberDevices/Golos args: ru metrics:
    • name: 测试WER type: wer value: 6.883
    • name: 测试CER type: cer value: 1.637
  • task: name: 语音识别 type: 自动语音识别 dataset: name: Sberdevices Golos (远场) type: SberDevices/Golos args: ru metrics:
    • name: 测试WER type: wer value: 15.044
    • name: 测试CER type: cer value: 5.128
  • task: name: 自动语音识别 type: 自动语音识别 dataset: name: Common Voice俄语 type: common_voice args: ru metrics:
    • name: 测试WER type: wer value: 12.115
    • name: 测试CER type: cer value: 2.980
  • task: name: 自动语音识别 type: 自动语音识别 dataset: name: 俄语Librispeech type: bond005/rulibrispeech args: ru metrics:
    • name: 测试WER type: wer value: 15.736
    • name: 测试CER type: cer value: 3.573
  • task: name: 自动语音识别 type: 自动语音识别 dataset: name: Sova俄语设备 type: bond005/sova_rudevices args: ru metrics:
    • name: 测试WER type: wer value: 20.652
    • name: 测试CER type: cer value: 7.287
  • task: name: 自动语音识别 type: 自动语音识别 dataset: name: Voxforge俄语 type: dangrebenkin/voxforge-ru-dataset args: ru metrics:
    • name: 测试WER type: wer value: 19.079
    • name: 测试CER type: cer value: 5.864

Wav2Vec2大型俄语Golos带语言模型

该Wav2Vec2模型基于facebook/wav2vec2-large-xlsr-53，使用Sberdevices Golos俄语数据集进行微调，并应用了音高变换、声音加速/减速、混响等音频增强技术。

该2-gram语言模型基于从三个公开来源获取的俄语文本语料库构建：

• Taiga随机10%子集
• 俄语维基百科
• 俄语维基新闻

使用说明

使用本模型时，请确保语音输入采样率为16kHz。

您可以通过编写自定义推理脚本来使用该模型：

import os
import warnings

import librosa
import nltk
import numpy as np

import torch
from datasets import load_dataset
from transformers import Wav2Vec2ForCTC, Wav2Vec2ProcessorWithLM

MODEL_ID = "bond005/wav2vec2-large-ru-golos-with-lm"
DATASET_ID = "bond005/sberdevices_golos_10h_crowd"
SAMPLES = 30

nltk.download('punkt')
num_processes = max(1, os.cpu_count())

test_dataset = load_dataset(DATASET_ID, split=f"test[:{SAMPLES}]")
processor = Wav2Vec2ProcessorWithLM.from_pretrained(MODEL_ID)
model = Wav2Vec2ForCTC.from_pretrained(MODEL_ID)

# 数据集预处理
# 需要将音频文件读取为数组
def speech_file_to_array_fn(batch):
    speech_array = batch["audio"]["array"]
    batch["speech"] = np.asarray(speech_array, dtype=np.float32)
    return batch

removed_columns = set(test_dataset.column_names)
removed_columns -= {'transcription', 'speech'}
removed_columns = sorted(list(removed_columns))
with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    test_dataset = test_dataset.map(
        speech_file_to_array_fn,
        num_proc=num_processes,
        remove_columns=removed_columns
    )

inputs = processor(test_dataset["speech"], sampling_rate=16_000,
                   return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(inputs.input_values,
                   attention_mask=inputs.attention_mask).logits
predicted_sentences = processor.batch_decode(
    logits=logits.numpy(),
    num_processes=num_processes
).text

with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    for i, predicted_sentence in enumerate(predicted_sentences):
        print("-" * 100)
        print("参考文本:", test_dataset[i]["transcription"])
        print("预测结果:", predicted_sentence)

也可通过此Colab脚本使用Google Colab版本。

评估

本模型在SberDevices Golos、Common Voice 6.0（俄语部分）和俄语Librispeech的测试子集上进行了评估，但仅使用SberDevices Golos的训练子集进行训练。您可以在我的Kaggle页面上查看其他数据集（包括俄语Librispeech和SOVA俄语设备）的评估脚本：https://www.kaggle.com/code/bond005/wav2vec2-ru-lm-eval

引用

如需引用本模型，请使用：

@misc{bondarenko2022wav2vec2-large-ru-golos,
  title={Ivan Bondarenko开发的带2-gram语言模型的XLSR Wav2Vec2俄语模型},
  author={Bondarenko, Ivan},
  publisher={Hugging Face},
  journal={Hugging Face Hub},
  howpublished={\url{https://huggingface.co/bond005/wav2vec2-large-ru-golos-with-lm}},
  year={2022}
}