Skip to content

[New Feature]: Improve the OGC registration process #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LucaCinquini opened this issue May 10, 2025 · 6 comments
Open

[New Feature]: Improve the OGC registration process #408

LucaCinquini opened this issue May 10, 2025 · 6 comments
Assignees
Labels
enhancement New feature or request U-SPS

Comments

@LucaCinquini
Copy link
Collaborator

Currently, once a DAG is registered via the OGC registration method, if the developer wants to update it, they need to first unregister it, then register it again. We want to change the implementation such that once a DAG is registered, the registration method can be executed again to push updates.

As part of this ticket, make sure that the OGC registration method respects the "enabled/disabled" status of a DAG (see the "db_cleanup_dag" DAG which is disabled by default, which then can enabled automatically at registration).

@nikki-t
Copy link
Collaborator

nikki-t commented May 21, 2025

The OGC API - Processes - Part 2: Deploy, Replace, Undeploy indicates the following:

A server that implements the DRU Requirements Class provides the ability to dynamically deploy, replace and undeploy processes.

The HTTP POST method is used to deploy a new process to the API.

The HTTP PUT method is used to replace the definition of a previously deployed processes that are accessible via the Processes API endpoint.

Finally, the HTTP DELETE method is used to undeploy a previously deployed process that is accessible via the Processes API endpoint.

I think the goal of the specification is for the user to use the HTTP PUT method to replace (or update) existing processes and I am wondering that if we change this behavior will the API no longer be OGC compliant?

I would still like to investigate how to make this change and it looks like this is the function that raises an exception if the process already exists and this file contains the deploy, replace, and undeploy functions.

I think it might make sense to remove the lines that raise an HTTPException and returns a 409.

@LucaCinquini
Copy link
Collaborator Author

LucaCinquini commented May 22, 2025 via email

@nikki-t
Copy link
Collaborator

nikki-t commented May 22, 2025

@LucaCinquini - It looks like the PUT method may already be implemented: https://github.com/unity-sds/unity-sps-ogc-processes-api/blob/a1d8b707fe4f713f4ee7d8a592d36221cccad49d/app/src/openapi_server/impl/dru_api.py#L139-L200

I tested it to make sure it works and I get the following error message back:

{
    "detail": "Method Not Allowed"
}

Which makes me think the method may not be defined for the API Gateway but I will look further and see if I can figure out the origin of the error message.

@nikki-t
Copy link
Collaborator

nikki-t commented May 22, 2025

I had the wrong API call which produced the above error message. Instead this is what I get when I try a PUT request to replace a DAG:

{
    "detail": "400 Client Error: BAD REQUEST for url: http://airflow-webserver.sps.svc.cluster.local:8080/api/v1/dags/cwl_dag/dagRuns"
}

Which is similar to what I see when I run the API locally and make queries against it so I think I can debug locally and then apply the changes to my deployment.

@nikki-t
Copy link
Collaborator

nikki-t commented May 23, 2025

@nikki-t
Copy link
Collaborator

nikki-t commented Jun 3, 2025

I updated the GitBook docs: https://app.gitbook.com/o/xZRqGQeQXJ0RP4VMj7Lq/s/UMIRhLdbRQTvMWop8Il9/developer-docs[…]velopers-guide/tutorial-using-the-ogc-processes-api-with-curl

And staged updates for Cognito tokens and the update process in the Python notebook: https://github.com/unity-sds/unity-monorepo/blob/408-ogc-registration-replace/libs/unity-py/notebooks/ogc_notebook.ipynb

But am holding off until we finalize the use of tokens in the unity-py client before creating a PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request U-SPS
Projects
Status: Todo
Development

No branches or pull requests

2 participants