Skip to content

Navigation Menu

Explore
By company size
By use case
By industry
View all solutions
Topics
- AI
- DevOps
- Security
- Software Development
- View all
Explore
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

triton-inference-server / python_backend Public

Notifications You must be signed in to change notification settings
Fork 155
Star 582

Code
Pull requests 12
Actions
Security
Insights

Additional navigation options

Code
Pull requests
Actions
Security
Insights

Optimize GPU tensor support for Python backend#293

Merged

krishung5 merged 16 commits intomaintriton-inference-server/python_backend:mainfrom krish-python-gputriton-inference-server/python_backend:krish-python-gpuCopy head branch name to clipboard

Oct 25, 2023

+778-207

Conversation Commits16 (16)Checks Files changed

Commits

Commits on Oct 25, 2023

Use CUDA shared memory pool to optimize tensor transfer between processes
krishung5
committed
Fix up: use the data ptr to get the tensor
krishung5
committed
Remove extra data copy. Use cudaMemcpyAsync for GPU output to improve performance.
krishung5
committed
Fix error handling. Fix bls tensor lifetime
krishung5
committed
Move CUDAMemPoolMessage inside TRITON_ENABLE_GPU directive
krishung5
committed
Fix CPU build
krishung5
committed
Address comments
krishung5
committed
Fix GPU tensor lifecycle
krishung5
committed
Make it be able to share cuda pool on different devices to the stub when needed
krishung5
committed
Remove data copy from PbMemory class
krishung5
committed
Fix up syntax, remove unused comments
krishung5
committed
Simplify PbMemory functionality. Let different io cases handle the memory allocation
krishung5
committed
Remove duplicated logic
krishung5
committed
Address comments
krishung5
committed
Address comment
krishung5
committed
Fix CPU only build
krishung5
committed

Footer

© 2025 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.