Run FluxKontextPipeline with multi-images input, But An error occurred

[veridone]

Describe the bug

I Use this examples:

conditioning_images = [
Image.open("image1.jpg").convert("RGB").resize((512, 512)),
Image.open("image2.jpg").convert("RGB").resize((512, 512)),
]
output = pipe(
prompt="A surreal fusion of the input images in a fantasy environment",
image=conditioning_images,
guidance_scale=4.0
).images[0]

And error occurred:

File "/home/work/test.py", line 30, in
output = pipe(
File "/home/work/anaconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/home/work/anaconda3/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux_kontext.py", line 972, in call
latents, image_latents, latent_ids, image_ids = self.prepare_latents(
File "/home/work/anaconda3/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux_kontext.py", line 687, in prepare_latents
image_latents = self._pack_latents(
File "/home/work/anaconda3/lib/python3.10/site-packages/diffusers/pipelines/flux/pipeline_flux_kontext.py", line 574, in _pack_latents
latents = latents.view(batch_size, num_channels_latents, height // 2, 2, width // 2, 2)
RuntimeError: shape '[1, 16, 64, 2, 64, 2]' is invalid for input of size 524288

Reproduction

My Code is:

from diffusers import FluxKontextPipeline
from PIL import Image
pipe = FluxKontextPipeline.from_pretrained(
"stabilityai/flux-kontext",
torch_dtype=torch.float16,
).to("cuda")
conditioning_images = [
Image.open("image1.jpg").convert("RGB").resize((512, 512)),
Image.open("image2.jpg").convert("RGB").resize((512, 512)),
]
prompt = "A surreal fusion of the input images in a fantasy environment"
output = pipe(
prompt=prompt,
image=conditioning_images,
guidance_scale=4.0
).images[0]

Logs

System Info

python: 3.10
diffusers: 0.35.0.dev0

Who can help?

No response

[asomoza]

Hi, that's not how it works, Kontext doesn't accept multiple images, not here or in any other implementation. What people are doing is to just stitch them together either horizontally or vertically, and passing that as a unique source image, you have to take into consideration the inference size too.

For example if you want to stitch two 512x512 images horizontally, you will have to change the inference size to a height of 512 and a width of 1024.

[veridone]

I see the 'call' function accept image list input, will it be supported in the future?

[asomoza]

the image list input is so it can work with multiple prompts, same as other pipelines. Is not a question of it will be supported or not, the model doesn't accept multiple input images.

We could refactor the pipeline to do the image stitching inside but personally I think that's something the user or the app has to do, for example sometimes is better to make a bigger image for the subject an a smaller one with the object you want to insert or vice versa.