Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Buffer Management to SnappyInput Stream #274

Open
chietti opened this issue Jan 14, 2021 · 1 comment
Open

Add Buffer Management to SnappyInput Stream #274

chietti opened this issue Jan 14, 2021 · 1 comment

Comments

@chietti
Copy link

chietti commented Jan 14, 2021

SnappyOutputStream uses a buffer management system to reduce memory pressure and GC overhead. The same mechanism would be helpful in SnappyInputStream as well.

We want to use Snappy for protocol compression in a high volume system. Currently we can see a significant increase in GC overhead due to the massive amount of buffers being allocated in SnappyInputStream. Adding the same BufferAllocator as in SnappyOutputStream would help to significantly reduce the amount of allocated memory.

Btw: we use both variants, the chunked one generated by SnappyOutputStream and the raw one used by a native Snappy implementation.

@SHaaD94
Copy link

SHaaD94 commented Jul 29, 2022

Hi, @xerial.
I would be happy to raise a PR for this.
Would you approve such change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants