标签:

• pyannote
• pyannote音频
• pyannote音频模型
• 音频
• 语音
• 说话人
• 说话人分割
• 语音活动检测
• 重叠语音检测
• 重分割 数据集:
• ami
• dihard
• voxconverse 许可证: mit 推理: false

🎹 说话人分割

该模型来自论文《端到端说话人分割用于重叠感知的重分割》，
作者为Hervé Bredin和Antoine Laurent。

在线演示可在Hugging Face Space上体验。

支持

商业咨询和科研合作，请联系我。
技术问题和错误报告，请查看pyannote.audio的GitHub仓库。

使用说明

需依赖正在开发中的pyannote.audio 2.0版本：参见安装指南。

语音活动检测

from pyannote.audio.pipelines import VoiceActivityDetection
pipeline = VoiceActivityDetection(segmentation="pyannote/segmentation")
HYPER_PARAMETERS = {
  # 开始/结束激活阈值
  "onset": 0.5, "offset": 0.5,
  # 移除短于此值的语音区域（秒）。
  "min_duration_on": 0.0,
  # 填补短于此值的非语音区域（秒）。
  "min_duration_off": 0.0
}
pipeline.instantiate(HYPER_PARAMETERS)
vad = pipeline("audio.wav")
# `vad`是一个pyannote.core.Annotation实例，包含语音区域

重叠语音检测

from pyannote.audio.pipelines import OverlappedSpeechDetection
pipeline = OverlappedSpeechDetection(segmentation="pyannote/segmentation")
pipeline.instantiate(HYPER_PARAMETERS)
osd = pipeline("audio.wav")
# `osd`是一个pyannote.core.Annotation实例，包含重叠语音区域

重分割

from pyannote.audio.pipelines import Resegmentation
pipeline = Resegmentation(segmentation="pyannote/segmentation", 
                          diarization="baseline")
pipeline.instantiate(HYPER_PARAMETERS)
resegmented_baseline = pipeline({"audio": "audio.wav", "baseline": baseline})
# 其中`baseline`应作为pyannote.core.Annotation实例提供

原始分数

from pyannote.audio import Inference
inference = Inference("pyannote/segmentation")
segmentation = inference("audio.wav")
# `segmentation`是一个pyannote.core.SlidingWindowFeature
# 实例，包含原始分割分数，如上图所示（输出）

可重复研究

为复现论文《端到端说话人分割用于重叠感知的重分割》的结果，请使用pyannote/segmentation@Interspeech2021及以下超参数：

预期输出（及VBx基线）也提供在/reproducible_research子目录中。

引用

@inproceedings{Bredin2021,
  Title = {{End-to-end speaker segmentation for overlap-aware resegmentation}},
  Author = {{Bredin}, Herv{\'e} and {Laurent}, Antoine},
  Booktitle = {Proc. Interspeech 2021},
  Address = {Brno, Czech Republic},
  Month = {August},
  Year = {2021},

@inproceedings{Bredin2020,
  Title = {{pyannote.audio: neural building blocks for speaker diarization}},
  Author = {{Bredin}, Herv{\'e} and {Yin}, Ruiqing and {Coria}, Juan Manuel and {Gelly}, Gregory and {Korshunov}, Pavel and {Lavechin}, Marvin and {Fustes}, Diego and {Titeux}, Hadrien and {Bouaziz}, Wassim and {Gill}, Marie-Philippe},
  Booktitle = {ICASSP 2020, IEEE International Conference on Acoustics, Speech, and Signal Processing},
  Address = {Barcelona, Spain},
  Month = {May},
  Year = {2020},
}