Add an option to retry when GlotPress download fails on "429 Too Many Requests" errors #402
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AliSoftware
force-pushed
the
l10n/429-retry
branch
from
91a75de
to
840740b
Compare
AliSoftware
changed the title
spencertransier
approved these changes
Aug 18, 2022
mokagio
approved these changes
Aug 19, 2022
There was a problem hiding this comment.
I'd normally make a snarky remark to suggest adding tests for the retry logic. But I don't think it's needed in this instance because:
- Requires non-trivial test setup to cover a niche feature
- We already have a new implementation of the whole workflow in the works, so any extra time spent here has low ROI
mokagio
added a commit
to wordpress-mobile/WordPress-iOS
that referenced
this pull request
Aug 19, 2022
The automation hit an HTTP error, likely the known HTTP 429 GlotPress error (e.g. see wordpress-mobile/release-toolkit#402), and those files were deleted. The tooling is right in deleting the files when it can't find a translation for them. But in this instance the translations were there, we just had an issue in downloading them. Notice that does not apply to the `release_notes.txt` file. There is no translation available at this point in time.
AliSoftware
added a commit
to wordpress-mobile/WordPress-Android
that referenced
this pull request
Aug 19, 2022
So I can get the latest non-breaking version of the release-toolkit for when I will be doing the final release later today Especially as that latest minor version includes wordpress-mobile/release-toolkit#402 which will help me mitigate any quota limit error I'll very likely get while downloading latest translations from GlotPress
6 tasks
mokagio
added a commit
to wordpress-mobile/WordPress-iOS
that referenced
this pull request
Sep 2, 2022
These were deleted during the last `gp_downloadmetadata` likely because of a 429 error from GlotPress. Various GlotPress actions were updated to retry when receiving a 429 in wordpress-mobile/release-toolkit#402, but `gp_downloadmetadata` was not one of them.
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why?
We still keep having
429 - Too Many Requestserrors from time to time when running the actions which download the app translations from GlotPress.For example, on average I encounter those on at least 1-3 locales every time I'm doing a new beta or final release, forcing me to interrupt the lane, discard the unstaged changes in git, restart the lane from scratch again, and pray it will pass on those subsequent tries, otherwise I'll have to do all that again. And since the more you retry, the closer you are to the quota limit… things tend to actually get worse on subsequent retries.
This has become quite painful while doing betas and releases, especially for WordPress (which has more than 40 locales to download, so the chances of hitting the
429error on at least one of them are higher)How?
This simply asks the user if they want to retry a the download of a locale if one fails.
UI.confirmprompt before retrying, which allows to leave some time before the first and next retry of the request, giving us even more chances for the quota limit to cool down before the next try.Related work
In parallel to that, other work is being done to help getting rid of this
429error and quota limit. Especially, we've asked for thegp-import-exportplugin to be installed on both of our GlotPress instances so we could do bulk-downloads of all the locales at once with a single request, and we plan to implement the code to in therelease-toolkitto be able to use it hopefully soon (we already have a Draft here, but it's still very early stage so far).Supporting bulk-download in the
release-toolkitwill require a bit more involved work though, for which I don't have the bandwidth for just yet. So this workaround of still using one-at-a-time locales download but allowing retry would hopefully help mitigate the issue in the meantime.