Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨🔧 452 - [Feature branch] - Add ega permissions integration #457

Merged
merged 39 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
49b0512
Update docker-compose, .env.example files. add axios dep
anncatton Sep 17, 2024
4227b2b
Update config, secrets values. Create egaClient with token fetch
anncatton Sep 17, 2024
0e06d57
initial refresh token logic
anncatton Sep 19, 2024
cfa6d35
add ega endpoint constants
anncatton Sep 23, 2024
ce6d8a7
add section for env vars in readme
anncatton Sep 23, 2024
912e16a
remove logs
anncatton Sep 23, 2024
83be068
remove public key reference
anncatton Sep 23, 2024
aa42318
Add zod, update ts. Add expiry to approved app list data
anncatton Sep 24, 2024
870ba74
Add api response types, permissions and users get funcs
anncatton Sep 24, 2024
6162104
Add safe parsing for api list results. Add dacId to config
anncatton Sep 25, 2024
3570e4a
add tsdocs
anncatton Sep 25, 2024
76a740e
Add safeParse checks to ega client calls.
anncatton Sep 25, 2024
f62d34b
Add failure returns to ega client calls, restructure types
anncatton Sep 25, 2024
14d35f4
remove empty file
anncatton Sep 25, 2024
12bb890
move failure types to separate file, add appId to approved users data
anncatton Sep 26, 2024
e995c79
modify list response types
anncatton Sep 26, 2024
965f0bb
Create ega permissions main job function, add happy path flow
anncatton Sep 30, 2024
e866b2b
move safeParseArray to own file, update imports
anncatton Sep 30, 2024
ab2e9bc
Merge pull request #458 from icgc-argo/453-ega-api-funcs
anncatton Sep 30, 2024
cacf7cb
Merge branch '452-ega-auth-client' into 454-implement-job-flow
anncatton Sep 30, 2024
ef48443
WIP - fix for pagination when checking permissions
anncatton Oct 3, 2024
04879b4
Expand tsdocs
anncatton Oct 3, 2024
951bebc
add comment for api bug, reorg ega services
anncatton Oct 3, 2024
f4be3e8
move custom errors to types dir
anncatton Oct 3, 2024
816df19
➕ Add pThrottle dependency as source code, rate limit ega client func…
anncatton Oct 4, 2024
9476767
rename types + util functions for clarity
anncatton Oct 11, 2024
81d23b6
remove unneeded fields from EgaPermissionRequest response schema
anncatton Oct 11, 2024
5bbda86
Add refresh token logic for 401 errors. Fetch all user permissions in…
anncatton Oct 15, 2024
db89e47
Merge pull request #460 from icgc-argo/454-implement-job-flow
anncatton Oct 15, 2024
e92dab7
Add ega public key fetch to secrets
anncatton Oct 20, 2024
b25cca9
Split clients into separate files. Fix error handling order
anncatton Oct 20, 2024
9cb3949
Adds ega job report data. Adds job to main batch job, with feature fl…
anncatton Oct 21, 2024
041962a
add newline, cleanup
anncatton Oct 21, 2024
d7e93d1
add axios-retry to egaClient
anncatton Nov 26, 2024
4d3a9d1
improve ega report structure, add example report to tsdoc
anncatton Nov 26, 2024
7880d28
remove unnecessary comments
anncatton Nov 26, 2024
5a1d79b
reject promise when max retry limit exceeded, add retry limits to con…
anncatton Nov 26, 2024
afbb2c0
Merge pull request #463 from icgc-argo/455-implement-error-retry
anncatton Nov 26, 2024
a227b1e
Merge pull request #462 from icgc-argo/add-refresh-token-flow
anncatton Nov 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 32 additions & 22 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ JWT_TOKEN_PUBLIC_KEY=
############
# true or false
VAULT_ENABLED=false
VAULT_SECRETS_PATH=/service/secrets_v1
VAULT_SECRETS_PATH=
VAULT_URL=
VAULT_ROLE=
# for local development/testing
Expand All @@ -48,34 +48,34 @@ DACO_REVIEW_POLICY_NAME=DACO-REVIEW
############
# Storage #
############
OBJECT_STORAGE_ENDPOINT=https://object.cancercollaboratory.org:9080
OBJECT_STORAGE_REGION=
OBJECT_STORAGE_BUCKET=
OBJECT_STORAGE_KEY=
OBJECT_STORAGE_SECRET=
OBJECT_STORAGE_ENDPOINT=http://localhost:8085
OBJECT_STORAGE_REGION=nova
OBJECT_STORAGE_BUCKET=daco
OBJECT_STORAGE_KEY=minio
OBJECT_STORAGE_SECRET=minio123
OBJECT_STORAGE_TIMEOUT_MILLIS=5000


############
# EMAIL #
############
EMAIL_HOST=smtp.gmail.com
EMAIL_PORT=587
EMAIL_HOST=localhost
EMAIL_PORT=1025
EMAIL_USER=
EMAIL_PASSWORD=
EMAIL_FROM_ADDRESS=
EMAIL_FROM_NAME=
EMAIL_DACO_ADDRESS=
EMAIL_FROM_ADDRESS=[email protected]
EMAIL_FROM_NAME=DacoAdmin
EMAIL_DACO_ADDRESS=[email protected]
# for emails directed to daco reviewers
EMAIL_REVIEWER_FIRSTNAME=
EMAIL_REVIEWER_LASTNAME=
EMAIL_REVIEWER_FIRSTNAME=DACO
EMAIL_REVIEWER_LASTNAME=ADMIN
DCC_MAILING_LIST=
DACO_SURVEY_URL=

##############
# UI #
##############
DACO_UI_BASE_URL=https://dac.dev.argo.cancercollaboratory.org
DACO_UI_BASE_URL=http://localhost:3000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for removing specific system references.

DACO_UI_APPLICATION_SECTION_PATH=/applications/{id}?section={section}

##############
Expand All @@ -88,16 +88,16 @@ FILE_UPLOAD_LIMIT=#in bytes x * 1024 * 1024
##############

# ATTESTATION
ATTESTATION_UNIT_COUNT=
ATTESTATION_UNIT_OF_TIME=
DAYS_TO_ATTESTATION=
ATTESTATION_UNIT_COUNT=1
ATTESTATION_UNIT_OF_TIME=years
DAYS_TO_ATTESTATION=45

# EXPIRY
DAYS_TO_EXPIRY_1=
DAYS_TO_EXPIRY_2=
DAYS_POST_EXPIRY=
EXPIRY_UNIT_COUNT=
EXPIRY_UNIT_OF_TIME=
DAYS_TO_EXPIRY_1=90
DAYS_TO_EXPIRY_2=45
DAYS_POST_EXPIRY=90
EXPIRY_UNIT_COUNT=2
EXPIRY_UNIT_OF_TIME=years

#############
# Daco Encryption
Expand All @@ -109,3 +109,13 @@ DACO_ENCRYPTION_KEY=
#############
FEATURE_RENEWAL_ENABLED=false
FEATURE_ADMIN_PAUSE_ENABLED=false

#############
# EGA
#############
EGA_CLIENT_ID=
EGA_AUTH_HOST=
EGA_AUTH_REALM_NAME=
EGA_API_URL=
EGA_USERNAME=
EGA_PASSWORD=
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,5 @@ typings/
dist/

# misc
.DS_Store
.DS_Store
.vscode/settings.json
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ debug: dcompose

#run the docker compose file
dcompose:
docker-compose -f dac-api-compose/docker-compose.yaml up -d
docker compose -f dac-api-compose/docker-compose.yaml up -d

# run all tests
verify:
npm run test

stop:
docker-compose -f dac-api-compose/docker-compose.yaml down --remove-orphans
docker compose -f dac-api-compose/docker-compose.yaml down --remove-orphans

# delete. everything.
nuke:
docker-compose -f dac-api-compose/docker-compose.yaml down --volumes --remove-orphans
docker compose -f dac-api-compose/docker-compose.yaml down --volumes --remove-orphans
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,17 @@ Development of the Data Access Control API
| ---------- | ------------- | ----------------------------------------------------------------------------------------- | --------------------------------- | ------- |
| `NODE_ENV` | isDevelopment | Enables `'/applications/:id'` DELETE endpoint. Enables `debug.log` file in Logger options | set `NODE_ENV` to `"development"` | `false` |

## Environment Variables

| Name | Description | Type | Required | Default |
| ------------------- | ----------------------------------------------------------------------------- | -------- | -------- | ------- |
| EGA_CLIENT_ID | Client ID for EGA API | `string` | true | |
| EGA_AUTH_HOST | Root URL for EGA authentication server | `string` | true | |
| EGA_AUTH_REALM_NAME | Realm name for EGA authentication server | `string` | true | |
| EGA_API_URL | Root URL for EGA API | `string` | true | |
| EGA_USERNAME | Username for account used to gain access token from EGA authentication server | `string` | true | |
| EGA_PASSWORD | Password for account used to gain access token from EGA authentication server | `string` | true | |

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

## Feature Flags

| Name | Config Path | Description | Trigger | Default |
Expand Down
8 changes: 3 additions & 5 deletions dac-api-compose/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
version: '3.8'

services:
vault:
image: vault
image: vault:1.13.3
volumes:
- $PWD/logs/:/tmp/logs
- ./vault:/scripts
Expand Down Expand Up @@ -47,8 +45,8 @@ services:

# for email services
mailhog:
image: mailhog/mailhog
container_name: 'mailhog'
image: jcalonso/mailhog:latest
container_name: mailhog
ports:
- '1025:1025'
- '8025:8025'
Expand Down
126 changes: 73 additions & 53 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
"@types/errorhandler": "^1.5.0",
"@types/express": "^4.17.11",
"@types/express-fileupload": "^1.1.6",
"@types/jsonwebtoken": "^9.0.7",
"@types/lodash": "^4.14.168",
"@types/memoizee": "^0.4.5",
"@types/migrate-mongo": "^8.1.0",
Expand Down Expand Up @@ -84,6 +85,7 @@
"@overture-stack/ego-token-middleware": "^2.4.1",
"archiver": "^5.3.0",
"aws-sdk": "^2.923.0",
"axios": "^1.7.7",
"body-parser": "^1.19.0",
"cd": "^0.3.3",
"connect-mongo": "^4.4.1",
Expand Down
Loading