Image morphing via diffusion models

Abstract

Diffusion-based generative models have emerged as a cornerstone in modern image synthesis, showcasing remarkable capabilities in producing semantically rich and photorealistic outputs. Yet, existing morphing pipelines relying on these models often struggle with two core issues: maintaining semantic coherence across intermediate frames and managing high computational overhead. In this work, we propose a new, two-phase morphing framework that addresses both challenges through the synergistic use of Latent Consistency Models (LCMs), Low-Rank Adaptation (LoRA), and the FILM (Frame Interpolation for Large Motion) algorithm.

First, we refine the DiffMorpher pipeline by incorporating LCM-LoRA “acceleration vectors,” which significantly reduce the sampling time needed to generate keyframes while being almost lossless. Second, we repurpose FILM for frame interpolation, allowing us to fill temporal gaps with high-quality subframes that further enhance the smoothness of the final morphing sequence. Notably, we integrate an optimized version of the pipeline into a user-friendly web application, lowering barriers to entry for both novices and experienced video editors.

Our report details the mathematical and algorithmic underpinnings—from Markov chain formulations in forward and reverse diffusion to the self-consistency property exploited by LCMs—and illustrates how each subsystem coalesces into a cohesive, end-to-end platform for generating fluid, high-resolution morphs. Empirical evaluations reveal significant improvements in both speed and frame quality when compared to baseline diffusion-based approaches.