Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: document snapshots and historical data providers #1809

Open
jcstein opened this issue Nov 27, 2024 · 4 comments
Open

docs: document snapshots and historical data providers #1809

jcstein opened this issue Nov 27, 2024 · 4 comments
Assignees
Labels
good first issue Good for newcomers help wanted Extra attention is needed P1 High Priority

Comments

@jcstein
Copy link
Member

jcstein commented Nov 27, 2024

Kyve from #1755

Summary

Kyve has bigquery dataset and RPC for historical access that should be documented.

Details

"I wanted to share the final integration of the Celestia blob and app data with our KYVE Trustless API Endpoint. It supports the blob.Get and the blob.GetAll query as well as block and block_results. For example, you can now query This blob through the Trustless API like this https://staging-613b8d89-data.services.kyve.network/celestia/Get?height=799489&namespace=AAAAAAAAAAAAAAAAAAAAAAAAAMod4SqXFNgavI8=&commitment=FlQmmPb3LYF0hLNd9YQfi9sRb42TKMfXh4KBtBa4BmA=

We have also published all of the Celestia Data on BigQuery and already have users subscribing to them."

Proposed solution

Create a dedicated page for historical data access. Pull out the "production" endpoints from network pages and include them in this page, too.

Please use the Trustless API production endpoint instead: https://data.services.kyve.network/#/celestia

polkachu snapshot from #1598

similar to #1465, add https://polkachu.com/archive_snapshots/celestia consensus archive snapshot to docs

also others from Slack:

https://celestia-tools.stavr.tech/bridge/
https://docs.services-ernventures.com/testnets/celestia/api-snapshot-addrbook#for-bridge-node-v0.18.2-mocha
https://itrocket.net/services/testnet/celestia/#snap-bridge

kingsuper #1465

https://kingsuper.org/snapshots/celestia-bridge-mainnet - note that it is a community snapshot, and not archival

also add kingsuper to consensus nodes if it's not already there, at least a link to the site

@jcstein jcstein added good first issue Good for newcomers help wanted Extra attention is needed P1 High Priority labels Nov 27, 2024
@hexmarkd
Copy link
Contributor

hexmarkd commented Feb 8, 2025

Hey @jcstein, I would like to work on this issue (and #1598 too). We developed celestia-snapshot-finder and maintain celestia-snapshots repository with fresh, verified snapshots. We plan to add more. Would love to include these along with the other providers - do you have any preferences for how this should be structured in the docs?

@jcstein
Copy link
Member Author

jcstein commented Feb 17, 2025

gm @hexmarkd - I'd like to consider working this into the docs, but not entirely sure how. does this help with just consensus nodes or also DA?

@hexmarkd
Copy link
Contributor

gm @jcstein,
Yeah, this helps with both consensus and DA layer nodes. Snapshot providers typically offer pruned and archive snapshots for consensus nodes and archive snapshots for DA nodes — we make sure these snapshots are getting updated and working before including them in celestia-snapshots. To make snapshot access easier for node runners, I was thinking we could add a dedicated page under Run a Node -> Resources in the docs that:

  • Explains what snapshots are (pruned vs archive).
  • Lists multiple snapshot providers for different chains (mainnet, mocha, arabica).
  • Introduces celestia-snapshot-finder as a way to automate snapshot selection and downloading.

We could also update the current quick sync section in the install guide to mention that there are multiple snapshot providers available, with a link to new page. Let me know what you think — happy to discuss!

@jcstein
Copy link
Member Author

jcstein commented Feb 24, 2025

gm @hexmarkd I think this all sounds good, I like the plan. want to open a PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed P1 High Priority
Projects
None yet
Development

No branches or pull requests

2 participants