← AI Techniques Radar
Inference efficiency

VLM-Steered Velocity Control with Instruction-Level Similarity for Flow-Based Image Editing

For flow-based, inversion-free text-guided image editing, introduce a VLM-guided control over the model’s velocity field. Create multiple coupling-controlled velocity candidates, then use a VLM-derived Instruction-Level Similarity (ILS) score to choose the velocity that best matches the editing instruction at each step. Add adaptive spatial masking to preserve regions not intended for change.

Discovered 2026-11-10 · Sources checked 2026-11-10

This summary was generated with AI assistance.

Why it matters

Source-anchoring (coupling) in flow-based editing can be too restrictive, leading to near-copy outputs where the requested semantic change does not appear. A timestep-adaptive, instruction-aware velocity choice aims to better balance preservation versus edit strength across the denoising process.

What changed

Instead of using a fixed source-coupling schedule for all timesteps, the method (1) generates complementary coupling-controlled velocities, (2) dynamically selects among them using ILS computed from instruction consistency, and (3) applies adaptive spatial masking to protect unrelated regions.

Where it helps

  • Text-guided edits that must preserve most of the original image
  • Flow-based image editing systems that suffer from overly restrictive source anchoring
  • Editing pipelines needing dynamic, timestep-specific edit strength
  • Interactive image retouching where instruction adherence is critical

Evidence & reproduction

Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.

Limitations & open questions

The approach depends on the quality and stability of the VLM’s instruction-following signal (ILS) and on having a reliable spatial masking strategy to avoid unintended changes in non-edited areas. Reported effectiveness is based on the cited experimental settings only.

Related World Models papers

Sources