You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We can't currently easily query for all of the records in dynamodb that are associated with a granule. If a published granule were to be removed from Cumulus, there is not an easy way for us to remove the associated features from Hydrocron.
We can delete individual records manually, but as some granules contain ~20,000+ features, this isn't a good solution.
We could add the granuleUR as a field during ingest and then create a secondary index to allow querying on the granuleUR, or could get creative with a mapping between features and the cycle/pass, but we would still need a way to query on a field other than feature id.
The text was updated successfully, but these errors were encountered:
I like the idea of adding the granuleUR to the database and creating a secondary index as I think that would allow the efficient retrieval of the data that can then be removed. It looks like you can batch delete items like you can batch write items.
Adding the granuleUR may also support the work in issue #71 Track granule ingest.
#150 adds granuleUR as a field in the databases and sets up the secondary index to query on them. This ticket can be for implementing the batch writer with the delete option described in the link above? We could potentially set it up as another lambda that performs the query and delete given the granuleUR as input?
I like the idea of creating a separate "delete" Lambda that can either be used by the track ingest architecture if we want to automate things or can be manually invoked via the AWS CLI or python script for cases where we want to delete data from the database. It would be good to prioritize some of this work during our tag up.
We can't currently easily query for all of the records in dynamodb that are associated with a granule. If a published granule were to be removed from Cumulus, there is not an easy way for us to remove the associated features from Hydrocron.
We can delete individual records manually, but as some granules contain ~20,000+ features, this isn't a good solution.
We could add the granuleUR as a field during ingest and then create a secondary index to allow querying on the granuleUR, or could get creative with a mapping between features and the cycle/pass, but we would still need a way to query on a field other than feature id.
The text was updated successfully, but these errors were encountered: