pipeline_tag: 句子相似度
language: 英文
license: apache-2.0
tags:

• 句子转换器
• 特征提取
• 句子相似度
• 转换器

hku-nlp/instructor-large

这是一个通用嵌入模型：它能在测试阶段无需额外训练的情况下，将任意文本片段（如标题、句子、文档等）映射为固定长度的向量。通过指令控制，生成的嵌入向量具有领域特异性（例如专为科学、金融等领域优化）和任务感知性（例如针对分类、信息检索等任务定制）。

该模型可轻松通过sentence-transformer库调用。

安装

git clone https://github.com/HKUNLP/instructor-embedding  
cd sentence-transformers  
pip install -e .  

计算定制化嵌入向量

随后可按以下方式生成领域特定且任务感知的嵌入向量：

from sentence_transformers import SentenceTransformer  
sentence = "3D ActionSLAM: wearable person tracking in multi-floor environments"  
instruction = "Represent the Science title; Input:"  
model = SentenceTransformer('hku-nlp/instructor-large')  
embeddings = model.encode([[instruction,sentence,0]])  
print(embeddings)  

计算句子相似度

还可进一步使用该模型计算两组句子间的相似度，并支持嵌入向量定制化。

from sklearn.metrics.pairwise import cosine_similarity  
sentences_a = [['Represent the Science sentence; Input: ','Parton energy loss in QCD matter',0],   
               ['Represent the Financial statement; Input: ','The Federal Reserve on Wednesday raised its benchmark interest rate.',0]]  
sentences_b = [['Represent the Science sentence; Input: ','The Chiral Phase Transition in Dissipative Dynamics', 0],  
               ['Represent the Financial statement; Input: ','The funds rose less than 0.5 per cent on Friday',0]]  
embeddings_a = model.encode(sentences_a)  
embeddings_b = model.encode(sentences_b)  
similarities = cosine_similarity(embeddings_a,embeddings_b)  
print(similarities)