Skip to content

Commit

Permalink
🔨 remove grapher DB code
Browse files Browse the repository at this point in the history
  • Loading branch information
Marigold committed Jul 19, 2023
1 parent 6bf5793 commit ffdb9e8
Show file tree
Hide file tree
Showing 53 changed files with 281 additions and 2,089 deletions.
13 changes: 2 additions & 11 deletions scripts/docs/data-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ To produce [our dataset](../dataset) we are constantly developing our dedicated
command tool [`cowid`](../cowidev/cowid-api) which eases:

1. Running several _sub-processes_ (or pipelines) that generate _intermediate datasets_.
2. Jointly processing and merging all these intermediate datasets into the final and complete dataset.
2. Jointly processing and merging all these intermediate datasets into the final and complete dataset.

Consequently, the dataset is updated multiple times a day (_at least_ at 06:00 and 18:00 UTC), using the latest generated intermediate datasets.

Expand Down Expand Up @@ -87,9 +87,6 @@ cowid hosp generate
# Update Grapher-ready files
cowid hosp grapher-io
# Update Grapher database
cowid hosp grapher-db
```

```{seealso}
Expand All @@ -105,9 +102,6 @@ re-publish the dataset.
```
# Generate dataset
cowid casedeath generate
# Update Grapher database
cowid casedeath grapher-db
```


Expand All @@ -133,7 +127,7 @@ cowid xm generate
```

## Variants pipeline
We run this pipeline daily.
We run this pipeline daily.
### Execution steps

```
Expand Down Expand Up @@ -164,9 +158,6 @@ cowid oxcgrt get
# Update Grapher files
cowid oxcgrt grapher-io
# Upload data to database
cowid oxcgrt grapher-db
```


Expand Down
18 changes: 1 addition & 17 deletions scripts/scripts/grapherupdate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ SCRIPTS_DIR=$ROOT_DIR/scripts
export OWID_COVID_PROJECT_DIR=${ROOT_DIR}
export OWID_COVID_CONFIG=${OWID_COVID_PROJECT_DIR}/scripts/config.yaml
export OWID_COVID_SECRETS=${OWID_COVID_PROJECT_DIR}/scripts/secrets.yaml
export PATH=$PATH:/usr/local/bin/
export PATH=$PATH:/usr/local/bin/


# FUNCTIONS
Expand Down Expand Up @@ -47,19 +47,3 @@ source $SCRIPTS_DIR/venv/bin/activate

# Make sure we have the latest commit.
git checkout $BRANCH && git pull


# Run Grapher updates
cowidev-grapher-db

## Additional individual grapher updates
minute=$(date +%M)
if [ $minute == 40 ] ; then
cowid --server casedeath grapher-db
cowid --server decoupling grapher-db
cowid --server hosp grapher-db
# cowid --server gmobility grapher-db
cowid --server sweden grapher-db
cowid --server uk-nations grapher-db
cowid --server oxcgrt grapher-db
fi
Loading

0 comments on commit ffdb9e8

Please sign in to comment.