Skip to content

Latest commit

 

History

History
61 lines (44 loc) · 2.24 KB

README.md

File metadata and controls

61 lines (44 loc) · 2.24 KB
Raw

Metadata Schema Publisher

Automatically publishes metadata schemas to schema.humacellatlas.org when changes are made to the GitHub repo.

Architecture

Architecture of schema.humancellatlas.org metadata publisher

Setup

Install NodeJS dependencies

Run:

npm install

To install required node libraries. Even though the Lambdas are written in Python the framework is in NodeJS and it needs extra plugins.

Install Python dependencies

Run:

pip install -r requirements.txt

Run Locally

Run a function locally

Run with a simulated GitHub event:

serverless invoke local --function onGithubPush --path ./tests/files/mock-event.json

Invoke on AWS

serverless invoke --function onGithubPush --path ./tests/files/github-event-push.json

Deploy

serverless deploy -v

AWS Permissions

If you encounter any permissions in running serverless cli commands manually, you could setup a new user with the same permissions as metadata-schema-publisher-serverless-cli user in AWS. which has adequate permissions to deploy and invoke the publisher lambda function. Alternatively, create a new access token for that user and set it up in your credentials and aws profile config.

Current Deployment

metadata-schema-publisher-prod-onGithubPush `

###Logic

  • On GitHub push event from WebHook
  • Go through all files ending with .json in the json_schema folder in the rep and update content to reflect latest versions in version.json
  • Check the id field in each schema and use that as the key of S3
  • Check if the S3 key exists and if not upload the file
  • Send notifications for when the process starts and a summary when it finishes

Setting up new schema environments

See notes: https://docs.google.com/document/d/1FQo9ofrFTSqLKx2a6eCHqTIpJxhAtShlCVkx6j6yA6s/edit