Stitchable Neural Networks
Zizheng Pan
Description: T-Stitch: Accelerating Sampling in Pre-trained Diffusion Models with Trajectory Stitching
sd (761) transformer (367) diffusion (242) dit (114) stable diffusion (107) model stitching (1)
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model. In this paper, we introduce sampling Trajectory Stitching ( T-Stitch ), a simple yet efficient technique to improve the sampling efficiency with little or no generation degradation.
Instead of solely using a large DPM for the entire sampling trajectory, T-Stitch first leverages a smaller DPM in the initial steps as a cheap drop-in replacement of the larger DPM and switches to the larger DPM at a later stage. Our key insight is that different diffusion models learn similar encodings under the same training data distribution and smaller models are capable of generating good global structures in the early steps. Extensive experiments demonstrate that T-Stitch is training-free, generally a
T-Stitch of two model combinations: DiT-XL/S, DiT-XL/B and DiT-B/S. We adopt DDIM 100 timesteps with a classifier-free guidance scale of 1.5.
Stitchable Neural Networks
Zizheng Pan