Update base fields to have org scope #1314
Conversation
|
@slifty As I pushed this I thought maybe you will want this change to be done via a GET followed by a PUT in the API. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1314 +/- ##
=======================================
Coverage 87.34% 87.34%
=======================================
Files 185 185
Lines 2387 2387
Branches 321 314 -7
=======================================
Hits 2085 2085
- Misses 276 302 +26
+ Partials 26 0 -26 ☔ View full report in Codecov by Sentry. |
|
To your point above @bickelj I don't think that a migration is the right way to adjust seed data or production data. I think we should update the seed sql in this PR instead and then for any instances just run some scripted curls to make these changes via the existing API instance (any script to do that will be simple and IMHO doesn't need to be included in the repository, but could be part of the PR description instead) |
|
@slifty I wouldn't call it simple, but it seems to do the trick: API_DOMAIN='api-test.philanthropydatacommons.org'; TOKEN='yada.yada.yada'; curl -4 https://${API_DOMAIN}/baseFields \
| jq -c '.[] | select(.shortCode | startswith("organization_")) | select(.scope == "proposal") | {id: .id, label: .label, description: .description, scope: "organization", dataType: .dataType, shortCode: .shortCode}' \
| tr -d '{' \
| sed 's/"id":\([0-9]*\),\(.*\)/--next -X '\''PUT'\'' -H '\''content-type: application\/json'\'' -H '\''Authorization: Bearer '${TOKEN}''\'' -d '\''{\2'\'' https:\/\/'${API_DOMAIN}'\/\1 /g' \
| xargs -0 | tr '\n' ' 'Except that a real token is too long: Then after pasting and running all that, run this to verify, this should come up empty because it filters on starting with "organization" and scope is "proposal" (all should be "organization" now): API_DOMAIN='api-test.philanthropydatacommons.org'; curl -4 https://${API_DOMAIN}/baseFields \
| jq -c '.[] | select(.shortCode | startswith("organization_")) | select(.scope == "proposal") | {id: .id, label: .label, description: .description, scope: "organization", dataType: .dataType, shortCode: .shortCode}' \
| tr -d '{'So that was an interesting exercise but I think running a SQL command would have been the better way. |
bickelj
force-pushed
the
org-scope-base-fields
branch
from
0bc58bf
to
9b11114
Compare
|
I just ran the proposed seed on a fresh database from main at 2af1e4b and it succeeded. |
Without this change the "gold" data feature does not return gold data.
This is because the base fields need 'organization' scope. This changes
the relevant base fields in one fell swoop.
If you have no organization-scoped base fields in your database, run
this on your database:
```sql
UPDATE base_fields SET scope = 'organization'
WHERE short_code LIKE 'organization_%' AND scope = 'proposal';
```
Alternatively, if you wish to do the change as it happened in
production on 2024-11-22, you may use this script:
```bash
API_DOMAIN='api-test.philanthropydatacommons.org'; TOKEN='yada.yada.yada'; curl -4 https://${API_DOMAIN}/baseFields \
| jq -c '.[] | select(.shortCode | startswith("organization_")) | select(.scope == "proposal") | {id: .id, label: .label, description: .description, scope: "organization", dataType: .dataType, shortCode: .shortCode}' \
| tr -d '{' \
| sed 's/"id":\([0-9]*\),\(.*\)/--next -X '\''PUT'\'' -H '\''content-type: application\/json'\'' -H '\''Authorization: Bearer '${TOKEN}''\'' -d '\''{\2'\'' https:\/\/'${API_DOMAIN}'\/\1 /g' \
| xargs -0 | tr '\n' ' '
```
With a long JWT, you cannot use it to set `TOKEN` because its too long
for args to be sent to `curl`, so keep it as `yada.yada.yada` and do a
find and replace separately in a text file. Then use multiple `curl`
commands with pastes of args from the output of the above, updating
your JWT as needed when it expires. To check that the updates happened,
you can look at the base fields manually and/or use the following:
```bash
API_DOMAIN='api-test.philanthropydatacommons.org'; curl -4 https://${API_DOMAIN}/baseFields \
| jq -c '.[] | select(.shortCode | startswith("organization_")) | select(.scope == "proposal") | {id: .id, label: .label, description: .description, scope: "organization", dataType: .dataType, shortCode: .shortCode}' \
| tr -d '{'
```
This should come up blank if all your organization base fields were
successfully updated. Of course, set API_DOMAIN to your local domain
in the above commands.
Issue #1138
bickelj
force-pushed
the
org-scope-base-fields
branch
from
9b11114
to
713e24a
Compare
Without this change the "gold" data feature does not return gold data. This is because the base fields need 'organization' scope. This changes the relevant base fields in one fell swoop.
Issue #1138