[Kandinsky 5] Fix I2V conditioning: don't inject the image latent into visual_cond channels#14282
[Kandinsky 5] Fix I2V conditioning: don't inject the image latent into visual_cond channels#14282sashakunitsyn wants to merge 2 commits into
Conversation
…annels Kandinsky5I2VPipeline.prepare_latents injected the conditioning image both as the first latent frame and into the visual_cond channels; the reference implementation (kandinskylab/kandinsky-5) only does the former. The duplicate over-conditions the first frame and produces mesh/visual artifacts. Remove the redundant injection to match the reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@sayakpaul @yiyixuxu |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
Hi @sashakunitsyn, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. |
What does this PR do?
Kandinsky5I2VPipeline.prepare_latentsinjects the conditioning image twice: once as the first latent frame (latents[:, 0:1] = image_latents, correct) and again into the DiT'svisual_condconditioning channels (visual_cond[:, 0:1] = image_latents). This duplicate signal over-conditions the first frame, producing mesh/visual artifacts on checkpoints trained with the reference convention. The reference implementation (kandinskylab/kandinsky-5→kandinsky/generation_utils.py::generate) does not do the second injection —visual_condis created as zeros. This PR removes that extra injection to match the reference.Before fix
base_i2v_stock_gh.mp4
After fix
base_i2v_condfix_gh.mp4