VLM-Steered Velocity Control with Instruction-Level Similarity for Flow-Based Image Editing
For flow-based, inversion-free text-guided image editing, introduce a VLM-guided control over the model’s velocity field. Create multiple coupling-controlled velocity candidates, then use a VLM-derived Instruction-Level Similarity (ILS) score to choose the velocity that best matches the editing instruction at each step. Add adaptive spatial masking to preserve regions not intended for change.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
Source-anchoring (coupling) in flow-based editing can be too restrictive, leading to near-copy outputs where the requested semantic change does not appear. A timestep-adaptive, instruction-aware velocity choice aims to better balance preservation versus edit strength across the denoising process.
What changed
Instead of using a fixed source-coupling schedule for all timesteps, the method (1) generates complementary coupling-controlled velocities, (2) dynamically selects among them using ILS computed from instruction consistency, and (3) applies adaptive spatial masking to protect unrelated regions.
Where it helps
- Text-guided edits that must preserve most of the original image
- Flow-based image editing systems that suffer from overly restrictive source anchoring
- Editing pipelines needing dynamic, timestep-specific edit strength
- Interactive image retouching where instruction adherence is critical
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The approach depends on the quality and stability of the VLM’s instruction-following signal (ILS) and on having a reliable spatial masking strategy to avoid unintended changes in non-edited areas. Reported effectiveness is based on the cited experimental settings only.
Related World Models papers
Sources
- VLM as an Editing Pilot: VLM-Steered Velocity Control for Flow-Based Image Editing ↗
arXiv · primary · Checked 2026-11-10
