Skip to content

support cuda 13.0 and trtllm kernel by Aug 21 2025 #9495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

support cuda 13.0 and trtllm kernel by Aug 21 2025 #9495

wants to merge 1 commit into from
+57 −59

Conversation

rainj-me
Copy link
Collaborator

Motivation

#9490

Support cuda130 with custom flashinfer and trtllm kernel Aug 21 2025

TODO

  • fix the merlin moe kernels

Modifications

  • use custom flashinfer to support cuda130 and load trtllm kernel Aug 21 2025
  • fix cub::Sum cub::Max issue and let it support both cuda12x and 130
  • use torch 2.8.x and cuda 130

Accuracy Tests

Benchmarking and Profiling

Checklist

@rainj-me rainj-me mentioned this pull request Aug 22, 2025
Open
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhyncs zhyncs Awaiting requested review from zhyncs zhyncs will be requested when the pull request is marked ready for review zhyncs is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock will be requested when the pull request is marked ready for review ispobock is a code owner

@HandH1998 HandH1998 Awaiting requested review from HandH1998 HandH1998 will be requested when the pull request is marked ready for review HandH1998 is a code owner

@BBuf BBuf Awaiting requested review from BBuf BBuf will be requested when the pull request is marked ready for review BBuf is a code owner

@yizhang2077 yizhang2077 Awaiting requested review from yizhang2077 yizhang2077 will be requested when the pull request is marked ready for review yizhang2077 is a code owner

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy will be requested when the pull request is marked ready for review merrymercy is a code owner

@FlamingoPg FlamingoPg Awaiting requested review from FlamingoPg FlamingoPg will be requested when the pull request is marked ready for review FlamingoPg is a code owner

@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw will be requested when the pull request is marked ready for review HaiShaw is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@rainj-me