Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update unity_sds_client with Data Service Delete #52

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/unity-py-python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,9 @@ jobs:
if: steps.changes.outputs.src == 'true'
run: |
unity_py_proposed_version=`poetry version -s`
echo "curl -s -o /dev/null -w \"%{http_code}\" https://pypi.org/project/unity-sds-client/$unity_py_proposed_version/"
status_code=`curl -s -o /dev/null -w "%{http_code}" https://pypi.org/project/unity-sds-client/$unity_py_proposed_version/`
# https://pypi.org/pypi/unity-sds-client/0.8.0/json
echo "curl -s -o /dev/null -w \"%{http_code}\" https://pypi.org/pypi/unity-sds-client/$unity_py_proposed_version/json"
status_code=`curl -s -o /dev/null -w \"%{http_code}\" https://pypi.org/pypi/unity-sds-client/$unity_py_proposed_version/json`
echo "Received status code of $status_code"
if ((status_code == 200)); then
echo "Version already exists."
Expand Down
16 changes: 16 additions & 0 deletions libs/unity-py/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.8.0] - 2025-1-14

### Added

### Fixed

### Changed

Updated the unity-py to include the Data Service Delete.

### Removed

### Security

### Deprecated

## [0.7.0] - 2024-11-21

### Added
Expand Down
2 changes: 1 addition & 1 deletion libs/unity-py/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "unity-sds-client"
version = "0.7.0"
version = "0.8.0"
description = "Unity-Py is a Python client to simplify interactions with NASA's Unity Platform."
authors = ["Anil Natha, Mike Gangl"]
readme = "README.md"
Expand Down
18 changes: 17 additions & 1 deletion libs/unity-py/tests/test_unity_data_service.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from unity_sds_client.unity import Unity
from unity_sds_client.unity_exception import UnityException
from unity_sds_client.unity_session import UnitySession
from unity_sds_client.unity import UnityEnvironments
from unity_sds_client.resources.collection import Collection
from unity_sds_client.unity_services import UnityServices as Services

Expand Down Expand Up @@ -47,4 +48,19 @@ def test_collection_creation(cleanup_update_test):
s.set_project("unity")
s.set_venue("test")
ds = s.client(Services.DATA_SERVICE)
ds.create_collection(Collection("urn:nasa:unity:unity:test:my_collection_id"), True)
ds.create_collection(Collection("urn:nasa:unity:unity:test:my_collection_id"), True)

def test_collection_deletion():

unity = Unity(UnityEnvironments.TEST)
token = unity._session.get_auth().get_token()
data_service = unity.client(Services.DATA_SERVICE)

collections = data_service.get_collections(100)

granule_id = 'urn:nasa:unity:emit:dev:unity-tutorial___1:summary_table.txt'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not query for the specific collection we're going to delete from? getting a listing and then choosing the first (with no idea what it is?) is hardly a test. Does an error get thrown if we request a delete and the file doesn't exist?

For a test like this, i'd expect us to:

find/create a collection
add a file to it
check to see if that file exists
delete the file
ensure its deleted

delete the collection on cleanup. this is certainly an integration test, and i'm not sure we can do all of that right now (maybe talk to Nga) but how you're running this test is not going to be sufficient.

for i in range(len(collections)):
if granule_id in collections[i].collection_id:
data_service.delete_collection_data(collections[i], granule_id)
break
17 changes: 17 additions & 0 deletions libs/unity-py/unity_sds_client/services/data_service.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the delete you have is fine (but very manual), but the name of it is not great. When i saw it i thought it would delete all the data in the collection.

  1. call this "delete_collection_item" and keep the method as a colleciton_id + granule id
  2. Create another method that's called "delete_dataset" which takes as input a "dataset" object. The dataset objects we get back from a query to the system include the collection object and the granuleid string. This is more in keeping with the domain objects we've created.

Any methods created need to include the docstring that describes them. See https://github.com/unity-sds/unity-monorepo/blob/feature/unity-sds-client-update-ds-delete/libs/unity-py/unity_sds_client/resources/collection.py#L90-L99 as an example. without this peope won't really know how to use this method other than digging into the source code. Documentation ends up at https://unity-sds.github.io/unity-monorepo/unity-sds-client/

Both

Original file line number Diff line number Diff line change
Expand Up @@ -190,3 +190,20 @@ def define_custom_metadata(self, metadata: dict):
params={"venue": self._session._venue}, json=metadata)
if response.status_code != 200:
raise UnityException("Error adding custom metadata: " + response.message)

#
#Delete single granule from a given Collection
#
def delete_collection_data(self, collection: type = Collection, granule_id: str = None):
#Must provide granule_id
if (granule_id == "") or (granule_id is None):
raise Exception("Error deleting collection data: Please provide granule ID")

else:
url = f'{self.endpoint}am-uds-dapa/collections/{collection.collection_id}/items/{granule_id}/'
token = self._session.get_auth().get_token()
header = {"Authorization": f"Bearer {token}"}
response = requests.delete(url=url, headers=header)
print(response.status_code)
if response.status_code != 200:
raise UnityException("Error deleting collection: " + response.reason)
Loading