MagicTryOn: AI Video Virtual Try-on - Real-time Dynamic Garment Swapping

Core Advantages

Universal People & Apparel

Supports fitting any garment image onto any person in a video, with no need for specific models or pose libraries, offering extreme versatility.

High Stability in Dynamic Videos

Maintains stable and seamless garment fit even during intense movements like dancing or turning, without jitter or drifting.

High-Fidelity Detail Precision

Perfectly restores garment materials, contours, and structures, such as lace and prints, for a result that looks like a real recording.

Core Technical Highlights

Diffusion Transformer (DiT) as a U-Net Replacement

We use DiT as the backbone network. Its powerful long-range dependency modeling capabilities can handle spatiotemporal information simultaneously, ensuring continuity and consistency between video frames. Compared to traditional U-Net, DiT can better capture complex details to generate higher-definition, stylistically unified videos.

Coarse-to-Fine Garment Preservation Strategy

Coarse-grained Guidance: In the embedding stage, we convert the garment image into a "garment token" to provide the model with strong initial direction.
Fine-grained Conditioning: During the denoising stage, we inject multi-dimensional information like garment semantics, textures, and contour lines, allowing the model to progressively refine details for a realistic try-on effect.

Garment-Aware Loss Function

During training, we designed a mask-aware loss that focuses the model on optimizing the generation quality of the "garment area", avoiding interference from irrelevant regions like the background, thereby significantly improving the fidelity of garment details.

Get Started

1. Install Environment

We recommend using Conda to create a separate Python environment.

# Requires python==3.12.9 cuda==12.3 torch==2.2
conda create -n magictryon python==3.12.9
conda activate magictryon
pip install -r requirements.txt

2. Download Pretrained Weights

cd Magic-TryOn
HF_ENDPOINT=https://hf-mirror.com huggingface-cli download LuckyLiGY/MagicTryOn --local-dir ./weights/MagicTryOn_14B_V1

3. Run Demo

Image Try-On

# Upper Body
CUDA_VISIBLE_DEVICES=0 python inference/image_tryon/predict_image_tryon_up.py

# Lower Body
CUDA_VISIBLE_DEVICES=1 python inference/image_tryon/predict_image_tryon_low.py

Video Try-On

# Upper Body
CUDA_VISIBLE_DEVICES=0 python inference/video_tryon/predict_video_tryon_up.py

# Lower Body
CUDA_VISIBLE_DEVICES=1 python inference/video_tryon/predict_video_tryon_low.py

For more detailed steps on custom try-on, please refer to our GitHub repository.

News & Roadmap

2025/06/09: 🎉 MagicTryOn source code and pretrained weights have been released! You can download them from 🤗HuggingFace.

2025/05/27: Our paper is available on ArXiv 🥳!

Development Roadmap (To-Do List)

Release the source code
Release the inference demo and pretrained weights
Release the customized try-on utilities
Release the testing scripts
Release the training scripts
Release the second version of the pretrained model weights
Update Gradio App