Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ArrayPool for poolable byte buffers #392

Merged
merged 2 commits into from
Feb 1, 2024
Merged

Use ArrayPool for poolable byte buffers #392

merged 2 commits into from
Feb 1, 2024

Conversation

drewnoakes
Copy link
Owner

No description provided.

}

ArrayPool<byte>.Shared.Return(buffer);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case where there's the possibility of throwing (e.g. calling an underlying stream we don't control), we should consider wrapping that logic try + finally.

try
{ 
   // logic that may throw
}
finally
{
  ArrayPool<byte>.Shared.Return(buffer);
}

To ensure we don't leak a buffer and degrade the performance of the ArrayPool for other callers.

This should be unnecessary when we're confident we won't throw.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added try/finally in a few places but not in others. Leaking a buffer isn't the end of the world. There can be a perf hit for having try/catch, even when nothing actually throws at run-time.

Comment on lines 335 to 348
var buffer = new byte[257];
var buffer = ArrayPool<byte>.Shared.Rent(257);

while (true)
{
var b = reader.GetByte();
if (b == 0)
return bytes.ToArray();
buffer[0] = b;
reader.GetBytes(buffer, 1, b);
bytes.Write(buffer, 0, b + 1);
var len = reader.GetByte();
if (len == 0)
break;
buffer[0] = len;
reader.GetBytes(buffer, offset: 1, count: len);
bytes.Write(buffer, 0, len + 1);
}

ArrayPool<byte>.Shared.Return(buffer);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also kill thevar bytes = new MemoryStream(); allocation, and write directly into the buffer (keeping track of the written bytes)?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is we don't know how large the stream will be in the end. Each iteration of the loop will add another chunk of bytes, and the result needs to be trimmed to size (with the current API). It's a pretty bizarre encoding format (XMP in GIF — see https://dl.photoprism.app/pdf/20120101-Adobe_XMP_Specification_Part_3.pdf section 1.1.2 on page 11).

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed some more improvements in GifReader.

@drewnoakes
Copy link
Owner Author

@iamcarbon I'm going to merge but if you have any further thoughts, I'll file a follow up PR.

@drewnoakes drewnoakes merged commit af0aa35 into main Feb 1, 2024
2 checks passed
@drewnoakes drewnoakes deleted the pool-buffers branch February 1, 2024 02:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants