Skip to content

Commit 7b4240a

Browse files
authored
fix(s3storage): release Bytebuf allocated by WAL earlier to prevent memory fragmentation (#2341)
Signed-off-by: Ning Yu <[email protected]>
1 parent c4e2bcf commit 7b4240a

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

s3stream/src/main/java/com/automq/stream/s3/S3Storage.java

+8
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,14 @@ private static long recoverContinuousRecords0(Iterator<RecoverResult> it,
290290
continue;
291291
}
292292

293+
// During the WAL recovery, when restored data is split into StreamObjects, we will upload each StreamObject
294+
// individually. This implementation causes the ByteBuf allocated by WriteAheadLog#recover to be released in
295+
// a fragmented manner, potentially generating significant memory fragmentation.
296+
// To avoid this issue, we optimize memory management by proactively releasing the ByteBuf through the
297+
// StreamRecordBatch#encoded method invocation, thereby centralizing memory release operations and
298+
// preventing memory fragmentation.
299+
streamRecordBatch.encoded();
300+
293301
Long expectNextOffset = streamNextOffsets.get(streamId);
294302
Queue<StreamRecordBatch> discontinuousRecords = streamDiscontinuousRecords.get(streamId);
295303
if (expectNextOffset == null || expectNextOffset == streamRecordBatch.getBaseOffset()) {

0 commit comments

Comments
 (0)