license: mit
pipeline_tag: image-feature-extraction
library_name: transformers

简介

MoonViT 是一款原生分辨率的视觉编码器，其初始化和持续预训练均基于 SigLIP-SO-400M。
为便于单独使用 MoonViT，我们已将其实现和权重从 moonshotai/Kimi-VL-A3B-Instruct 中分离出来。

若您对 MoonViT 的训练过程感兴趣，欢迎阅读论文 Kimi-VL 技术报告。

使用示例

from PIL import Image  
from transformers import AutoModel, AutoImageProcessor  

model_path = "moonshotai/MoonViT-SO-400M"  
model = AutoModel.from_pretrained(  
    model_path,  
    torch_dtype="auto",  
    device_map="auto",  
    trust_remote_code=True,  
)  
processor = AutoImageProcessor.from_pretrained(model_path, trust_remote_code=True)  

image_path = "./figures/demo.png"  
image = Image.open(image_path)  

images_processed = processor(image, return_tensors="pt").to(dtype=model.dtype, device=model.device)  
image_features: list = model(images_processed.pixel_values, images_processed.image_grid_hws)  

print(f"dtype: {image_features[0].dtype}, shape: {image_features[0].shape}")  
# 输出: dtype: torch.bfloat16, shape: torch.Size([1092, 4, 1152])