Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup downloads on timescaledb #4979

Draft
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

jonatas
Copy link

@jonatas jonatas commented Aug 23, 2024

It's second part of #4642.

Add the downloads and log_downloads table to the timescaledb database.

  1. The downloads table represents the gem_downloads but with flat results, every download is a row in the database.
  2. The LogDownloads mimics LogTicket and tracks what logs are already processed by timescaledb database.
  3. There's a maintenance task that can keep picking pending Logs to be processed.

The actual code makes a duplication of LogTicket and also FastlyLogDownloadProcessor mimics the previous processor. I think we can live with both until we build the trust and guarantee all statistics are working as expected.

TODO

  • Add task to migrate data from LogTicket to LogDownloads
  • Test this PR with timescaledb disabled and no timescaledb database running
  • Make sure when timescaledb is disabled no background jobs or maintenance tasks are scheduled

@jonatas jonatas changed the title setup downloads on timescaledb Setup downloads on timescaledb Aug 23, 2024
@simi simi self-requested a review August 28, 2024 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

1 participant