Skip to content

[Kandinsky 5] Fix I2V conditioning: don't inject the image latent into visual_cond channels#14282

Open
sashakunitsyn wants to merge 2 commits into
huggingface:mainfrom
sashakunitsyn:fix/kandinsky5-i2v-visual-cond
Open

[Kandinsky 5] Fix I2V conditioning: don't inject the image latent into visual_cond channels#14282
sashakunitsyn wants to merge 2 commits into
huggingface:mainfrom
sashakunitsyn:fix/kandinsky5-i2v-visual-cond

Conversation

@sashakunitsyn

@sashakunitsyn sashakunitsyn commented Jul 24, 2026

Copy link
Copy Markdown

What does this PR do?

Kandinsky5I2VPipeline.prepare_latents injects the conditioning image twice: once as the first latent frame (latents[:, 0:1] = image_latents, correct) and again into the DiT's visual_cond conditioning channels (visual_cond[:, 0:1] = image_latents). This duplicate signal over-conditions the first frame, producing mesh/visual artifacts on checkpoints trained with the reference convention. The reference implementation (kandinskylab/kandinsky-5kandinsky/generation_utils.py::generate) does not do the second injection — visual_cond is created as zeros. This PR removes that extra injection to match the reference.

Before fix

base_i2v_stock_gh.mp4

After fix

base_i2v_condfix_gh.mp4

…annels

Kandinsky5I2VPipeline.prepare_latents injected the conditioning image both as the
first latent frame and into the visual_cond channels; the reference implementation
(kandinskylab/kandinsky-5) only does the former. The duplicate over-conditions the
first frame and produces mesh/visual artifacts. Remove the redundant injection to
match the reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added size/S PR with diff < 50 LOC pipelines and removed size/S PR with diff < 50 LOC labels Jul 24, 2026
@leffff

leffff commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@sayakpaul @yiyixuxu
We are preparing for a release of something interesting on top of our Kandinsky5. We found a bug in our I2V implementation. We want to merge this one-line fix ASAP, so we can proceed to our release!
We will be grateful for your review!

@sashakunitsyn
sashakunitsyn marked this pull request as ready for review July 24, 2026 11:54
@github-actions github-actions Bot added the size/S PR with diff < 50 LOC label Jul 24, 2026
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sayakpaul
sayakpaul requested a review from yiyixuxu July 24, 2026 18:34
@github-actions

Copy link
Copy Markdown
Contributor

Hi @sashakunitsyn, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pipelines size/S PR with diff < 50 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants