Use a worker pool to add concurrency limit on gif renders #35
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This uses https://github.com/alitto/pond to add a middleware that wraps all gif rendering endpoints with a worker pool that limits concurrent executions to the number of CPUs, and allows up to 40 requests to sit in a queue until a worker frees up. After that limit is reached, requests with get a HTTP 429 instead to avoid hitting the overall Cloud Run active request limit.
This isn't exactly heavily tested, but I have poked at it locally a fair bit with lower numbers for the pool, and it seems to work as expected...? The worst case I'd expect is that this causes a memory leak if I got something wrong. I don't think it'll crash exporter, so it'll probably be a net improvement over intermittently denying all requests.
I also got a warning from running
go mod tidy
after adding the new dependency. It seems to happen on main as well, so I don't think I introduced it, but if I chose the wrong thing it might be making unnecessary changes to the deps: