TryOnDiffusion

TryOnDiffusion

TryOnDiffusion generates a photorealistic image of how a garment would look on a target person from two inputs: a photo of the person and a photo of someone wearing the garment. The CVPR 2023 research project from University of Washington and Google Research uses a diffusion model with two parallel UNets to preserve fabric detail while warping the clothing across different body poses and shapes. Output runs through 128x128 and 256x256 stages before super-resolution to 1024x1024.

Earlier virtual try-on systems split garment warping and blending into separate steps, which often traded detail for pose flexibility. TryOnDiffusion warps garments implicitly through cross-attention in a Parallel-UNet and fuses warp and blend in one pass. On its test set it reports FID 13.447 and KID 6.964, beating TryOnGAN, SDAFN, and HR-VITON, with 92.72% user preference on random inputs and 95.80% on challenging pose cases.

Fashion researchers, computer vision teams, and e-commerce developers use the project page demo to preview upper-body try-ons interactively. The authors note limitations: upper-body clothing only, reliance on segmentation and pose preprocessing, mostly clean backgrounds in training data, and visualization without fit guarantees.

Top Features:
  1. Parallel-UNet fuses garment warping and person blending in one diffusion network

  2. Pipeline outputs 128x128, 256x256, then super-resolves to 1024x1024 images

  3. Reports FID 13.447 and KID 6.964 on the project test set, below HR-VITON at 18.705 FID

  4. Cross-attention warps segmented garment features onto the clothing-agnostic person image

  5. Interactive demo on the project page for person-garment try-on previews

  6. User study shows 92.72% preference on random inputs versus three prior methods

  7. Pose embeddings modulate both UNets via FiLM across all scales

Pros:
  1. Preserves garment detail while handling large pose and body shape changes in one network

  2. State-of-the-art FID and KID scores versus TryOnGAN, SDAFN, and HR-VITON on published benchmarks

  3. Free interactive demo on the project page with no signup

  4. Outputs up to 1024x1024 resolution through a multi-stage diffusion pipeline

Cons:
  1. Limited to upper-body clothing; full-body try-on is listed as future work

  2. Quality depends on segmentation map and pose estimation preprocessing

  3. Trained mostly on clean uniform backgrounds; complex scenes are untested

  4. Visualization only; the authors do not promise accurate garment fit

FAQs:

What does TryOnDiffusion do?

TryOnDiffusion takes a person image and a garment image, then generates a visualization of how the garment might look on that person. It targets photorealistic detail while handling pose and body shape changes between the two subjects.

What is Parallel-UNet in TryOnDiffusion?

Parallel-UNet is TryOnDiffusion's core architecture with two UNets running in parallel. The person-UNet processes clothing-agnostic RGB and a noisy image, while the garment-UNet feeds segmented garment features through cross-attention to warp and blend in one step.

Is TryOnDiffusion free to use?

TryOnDiffusion is a free research project with an interactive demo on tryondiffusion.github.io. There is no paid product or signup on the project page. It was published at CVPR 2023 by University of Washington and Google Research authors.

How does TryOnDiffusion compare to prior methods?

TryOnDiffusion reports lower FID and KID than TryOnGAN, SDAFN, and HR-VITON on its test sets. A user study with 15 non-experts per input gave TryOnDiffusion 92.72% preference on random cases and 95.80% on challenging pose differences.

What resolution does TryOnDiffusion output?

TryOnDiffusion generates try-on images through 128x128 and 256x256 Parallel-UNet stages, then applies standard super-resolution diffusion to reach 1024x1024 output resolution on the project page.

What are TryOnDiffusion limitations?

TryOnDiffusion focuses on upper-body clothing only, depends on segmentation and pose preprocessing accuracy, and was trained mostly on clean backgrounds. The authors state it visualizes try-on results without promising garment fit.

Category:

Pricing:

Free

Tags:

Virtual Try-On
Parallel-UNet
Diffusion Models
Clothing Transfer
CVPR Research
Garment Warping
Image Synthesis
Diffusion-Based Architecture

Tech used:

Bootstrap
jQuery

Reviews:

Give your opinion on TryOnDiffusion :-

Overall rating

Join thousands of AI enthusiasts in the World of AI!

Best Free TryOnDiffusion Alternatives (and Paid)

By Rishit