feat(fs): introduce chunk fs cache cleanup worker

[dtfiedler]

Chunks can quickly fill up a disk, even if contiguous data and header filesystem caches are cleaned up. This PR introduces a chunk filesystem cleanup worker that is disabled by default and responsible for deleting old(er) chunks.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.78%. Comparing base (ebde843) to head (0b094db).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #226      +/-   ##
===========================================
+ Coverage    68.74%   68.78%   +0.03%     
===========================================
  Files           32       32              
  Lines         7983     7992       +9     
  Branches       434      434              
===========================================
+ Hits          5488     5497       +9     
  Misses        2494     2494              
  Partials         1        1              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[thiagocarvalhodev]

This block is almost identical to the contiguousDataFsCacheCleanupWorker.

Consider abstracting the shared logic into a reusable function, for example:

function createFsCacheCleanupWorker({ basePath, thresholdInSeconds, isEnabled = true }) {
  if (!isEnabled || isNaN(thresholdInSeconds)) {
    return undefined;
  }

  return new FsCleanupWorker({
    log,
    basePath,
    shouldDelete: async (path) => {
      try {
        const stats = await fs.promises.stat(path);
        const mostRecentTime =
          stats.atime > stats.mtime ? stats.atime : stats.mtime;
        const currentTimestamp = Date.now();
        const thresholdDate = new Date(
          currentTimestamp - thresholdInSeconds * 1000,
        );
        return mostRecentTime <= thresholdDate;
      } catch (err) {
        log.error(`Error getting file stats for ${path}`, err);
        return false;
      }
    },
  });
}

export const contiguousDataFsCacheCleanupWorker = createFsCacheCleanupWorker({
  basePath: 'data/contiguous',
  thresholdInSeconds: contiguousDataCacheCleanupThresholdInSeconds,
});

export const chunkDataFsCacheCleanupWorker = createFsCacheCleanupWorker({
  basePath: 'data/chunks',
  thresholdInSeconds: config.CHUNK_DATA_CACHE_CLEANUP_THRESHOLD,
  isEnabled: config.ENABLE_FS_CHUNK_CACHE_CLEANUP,
});

[dtfiedler]

indeed it is - good rec! will pull in