Watch and track your favorite playlist.
Curated by: USENIX (112 videos)
NSDI '24 - Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models Shubham Agarwal and Subrata Mitra, Adobe Research; Sarthak Chakraborty, UIUC; Srikrishna Karanam, Koyel Mukherjee, and Shiv Kumar Saini, Adobe Research Text-to-image generation using diffusion models has seen explosive popularity owing to their ability in producing high quality images adhering to text prompts. However, diffusion-models go through a large number of iterative denoising steps, and are resource-intensive, requiring expensive GPUs and incurring considerable latency. In this paper, we introduce a novel approximate-caching technique that can reduce such iterative denoising steps by reusing intermediate noise states created during a prior image generation. Based on this idea, we present an end-to-end text-to-image generation system, NIRVANA, that uses approximate-caching with a novel cache management policy to provide 21% GPU compute savings, 19.8% end-to-end latency reduction, and 19% dollar savings on two real production workloads. We further present an extensive characterization of real production text-to-image prompts from the perspective of caching, popularity and reuse of intermediate states in a large production environment. View the full NSDI '24 program at https://www.usenix.org/conference/nsdi24/technical-sessions