许可协议：Apache-2.0
数据集：

• librispeech_asr
  评估指标：
• wer（词错误率）
  管道标签：自动语音识别
  标签：
• 自动语音识别
• ONNX
• 英特尔®神经压缩器
• neural-compressor
  库名称：transformers

INT4量化版Whisper大型ONNX模型

Whisper是一种预训练模型，用于自动语音识别（ASR）和语音翻译。经过68万小时标注数据训练后，Whisper模型展现出强大的泛化能力，无需微调即可适应多种数据集和领域。本仓库提供ONNX格式的Whisper大型模型INT4权重量化版本，由英特尔®神经压缩器和英特尔®Transformers扩展驱动。

该INT4 ONNX模型通过英特尔®神经压缩器的仅权重量化方法生成。

导出ONNX模型

FP32模型通过openai/whisper-large导出：

optimum-cli export onnx --model openai/whisper-large whisper-large-with-past/ --task automatic-speech-recognition-with-past --opset 13  

安装ONNX Runtime

需安装onnxruntime>=1.16.0以支持MatMulFpQ4算子。

执行量化

从主分支构建英特尔®神经压缩器并运行INT4仅权重量化。

权重量化配置如下：

关键代码如下，完整脚本参见Whisper示例：

from neural_compressor import quantization, PostTrainingQuantConfig  
from neural_compressor.utils.constant import FP32  

model_list = ['encoder_model.onnx', 'decoder_model.onnx', 'decoder_with_past_model.onnx']  
for model in model_list:  
    config = PostTrainingQuantConfig(  
        approach="weight_only",  
        calibration_sampling_size=[8],  
        op_type_dict={".*": {"weight": {"bits": 4,   
                                        "algorithm": ["RTN"],   
                                        "scheme": ["sym"],   
                                        "group_size": 32}}},)  
    q_model = quantization.fit(  
        os.path.join("/path/to/whisper-large-with-past", model), # FP32模型路径  
        config,  
        calib_dataloader=dataloader)  
    q_model.save(os.path.join("/path/to/whisper-large-onnx-int4", model)) # INT4模型路径  

评估

算子统计
INT4 ONNX模型的算子统计如下：

词错误率评估
在librispeech_asr数据集上的评估代码如下：

import os  
from evaluate import load  
from datasets import load_dataset  
from transformers import WhisperForConditionalGeneration, WhisperProcessor, AutoConfig  
model_name = 'openai/whisper-large'  
model_path = 'whisper-large-onnx-int4'  
processor = WhisperProcessor.from_pretrained(model_name)  
model = WhisperForConditionalGeneration.from_pretrained(model_name)  
config = AutoConfig.from_pretrained(model_name)  
wer = load("wer")  
librispeech_test_clean = load_dataset("librispeech_asr", "clean", split="test")  

from optimum.onnxruntime import ORTModelForSpeechSeq2Seq  
from transformers import PretrainedConfig  
model_config = PretrainedConfig.from_pretrained(model_name)  
predictions = []  
references = []  
sessions = ORTModelForSpeechSeq2Seq.load_model(  
            os.path.join(model_path, 'encoder_model.onnx'),  
            os.path.join(model_path, 'decoder_model.onnx'),  
            os.path.join(model_path, 'decoder_with_past_model.onnx'))  
model = ORTModelForSpeechSeq2Seq(sessions[0], sessions[1], model_config, model_path, sessions[2])  
for idx, batch in enumerate(librispeech_test_clean):  
    audio = batch["audio"]  
    input_features = processor(audio["array"], sampling_rate=audio["sampling_rate"], return_tensors="pt").input_features  
    reference = processor.tokenizer._normalize(batch['text'])  
    references.append(reference)  
    predicted_ids = model.generate(input_features)[0]  
    transcription = processor.decode(predicted_ids)  
    prediction = processor.tokenizer._normalize(transcription)  
    predictions.append(prediction)  
wer_result = wer.compute(references=references, predictions=predictions)  
print(f"词错误率结果: {wer_result * 100}")  

性能指标：