Skip to content

[scheduler] Add SDE Samplers: Curved Euler and Overshoot #12192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

Abhinay1997
Copy link
Contributor

@Abhinay1997 Abhinay1997 commented Aug 19, 2025

What does this PR do?

Before submitting

Hello @yiyixuxu, this PR adds two SDE schedulers as described in Improving Rectified Flow with Boundary Conditions and AMO Sampler

I have added the docs and the scheduler files. Let me know if I missed something.

Test code:

import torch
from diffusers import FluxPipeline
from diffusers.schedulers import FlowMatchStochasticOvershootDiscreteScheduler, FlowMatchStochasticCurvedEulerScheduler

pipe = FluxPipeline.from_pretrained('black-forest-labs/FLUX.1-Krea-dev', torch_dtype=torch.bfloat16)
pipe.to('cuda')

euler_scheduler = pipe.scheduler
overshoot_scheduler = FlowMatchStochasticOvershootDiscreteScheduler.from_pretrained('black-forest-labs/FLUX.1-Krea-dev', subfolder='scheduler')
cec_scheduler = FlowMatchStochasticCurvedEulerScheduler.from_pretrained('black-forest-labs/FLUX.1-Krea-dev', subfolder='scheduler')

prompt = "A kangaroo holding a sign that says 'Sting like a bee, float like a butterfly' wearing black sunglasses and boxing gloves, the golden gate bridge in the background."

pipe.scheduler = euler_scheduler
out = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=28,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]
out.save('euler.jpg')

## CIRCULAR EULER
pipe.scheduler = cec_scheduler
out = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=28,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]
out.save('circular_euler.jpg')

## OVERSHOOT C = 1.0 default
pipe.scheduler = overshoot_scheduler
pipe.scheduler.set_c(2.0)
out = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=28,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]
out.save('overshoot.jpg')

Attached comparisons with FlowMatchEulerDiscreteScheduler with stochastic_sampling set to False & True.

grid

grid

Write up on the theory here

@Abhinay1997 Abhinay1997 changed the title Add SDE Samplers: Curved Euler and Overshoot [scheduler] Add SDE Samplers: Curved Euler and Overshoot Aug 20, 2025
@Abhinay1997
Copy link
Contributor Author

Would like to hear your thoughts as well, @a-r-r-o-w

@a-r-r-o-w
Copy link
Member

Thanks for the amazing work @Abhinay1997! I can help do a full code review soon, but it looks good from a quick glance. Good writeup as well!

Always great to have more flexibility with new techniques if there's a principled study on it. The final decision for supporting new schedulers is @yiyixuxu's so let's wait for her review :) I'll try to understand the papers and check for correctness in the mean time

The generated images look good overall. Not a blocker on my end, but I can notice some block-ness and artifacts in the images. It's probably due to downscaling of the images when creating the grid, but I'll ping @asomoza for further exploration as his eyes are much better than mine in analyzing image quality.

@Abhinay1997
Copy link
Contributor Author

Thank you for the kind words! Of course, I'll wait for her review and thoughts on this. As for the blockiness and artifacts, that does seem to be an export artifact as I rendered the grid in tldraw and exported the whole thing as as single low res image.

But the smoothing does come from the SDE process itself which is worth the tradeoff of better text rendering and randomness maybe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants