基础模型:

• black-forest-labs/FLUX.1-dev 库名称: diffusers 许可证名称: flux-1-dev-non-commercial-license 许可证链接: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md 管道标签: 图像到图像 推理: 否 标签:
• ControlNet 许可证: 其他

⚡ Flux.1-dev: 表面法线ControlNet ⚡

这是由Jasper研究团队开发的Flux.1-dev表面法线地图ControlNet。

使用方法

该模型可直接与diffusers库一起使用

import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline

# 加载管道
controlnet = FluxControlNetModel.from_pretrained(
  "jasperai/Flux.1-dev-Controlnet-Surface-Normals",
  torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
  "black-forest-labs/FLUX.1-dev",
  controlnet=controlnet,
  torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# 加载控制图像
control_image = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Surface-Normals/resolve/main/examples/surface.jpg"
)

prompt = "窗前举着停止标志的男人"

image = pipe(
    prompt, 
    control_image=control_image,
    controlnet_conditioning_scale=0.6,
    num_inference_steps=28, 
    guidance_scale=3.5,
    height=control_image.size[1],
    width=control_image.size[0]
).images[0]
image

💡 注意: 您可以使用controlnet_aux库中的NormalBaeDetector计算条件映射

from controlnet_aux import NormalBaeDetector
from diffusers.utils import load_image 

normal_bae = NormalBaeDetector.from_pretrained("lllyasviel/Annotators")

normal_bae.to("cuda")

# 加载图像
im = load_image(
  "https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Surface-Normals/resolve/main/examples/output.jpg"
)

surface = normal_bae(im)

训练

该模型使用Clipdrop的表面法线估计模型以及开源表面法线估计模型如Boundary Aware Encoder (BAE)计算的法线地图进行训练。

许可证

本模型遵循Flux.1-dev模型许可证。