Skip to content

Home

Jump to bottom
anoronh4 edited this page Jul 18, 2022 · 3 revisions

Overview

This App was created to show metadata of samples from DMP Recapture Repository in various stages of processing. There are 3 user roles:

  1. Admin: View everything
  2. Clinical: View most relevant columns, including some restricted values, with certain samples (failed, not-processed, etc) filtered out.
  3. Non-clinical: View most relevant columns, excluding some restricted values, with certain samples (failed, not-processed, etc) filtered out. This is the default view.

There are 3 environments in which to run the tracker:

  1. Production: Used by all. Shows saved information in the Prod database.
  2. Development: Used for testing. Shows saved information in the Dev database.
  3. Local: Also used for testing, but only viewable by the tester. Shows saved information in the Dev database.

The environment can be changed by editing the config file sample-tracking-api/app/lims_user_config. Always test local before dev, and dev before prod.

Local testing

After cloning your branch to a fresh directory, you will need to perform some setup before running for the first time.

Setup

Create a virtual miniconda environment and install packages in sample-tracking-api/requirements.txt. Python 3.9.7 works as of 4.18.22. This will run the API component of the app.

Copy a configuration file from the dev codebase on delphi, for example: cp /data/www/flask/sample-tracking-dev/sample-tracking-api/app/lims_user_config /path/to/mydir/sample-tracking-api/app/ Then edit the file so that env parameter indicates local.

Execution

You will need two sessions:

  1. One session where the conda environment is activated and running:
cd /path/to/mydir/sample-tracking-api/
python run.py
  1. One session where the frontend is running:
cd /path/to/mydir/sample-tracking-frontend/
npm run start-local

If you get an error related to react-scripts, you may need to install the package using: npm i react-scripts before retrying.

One session (can be one of the two above) must be opened using ssh tunneling. For example:

ssh -L 8080:localhost:3000 -L 5000:127.0.0.1:5000 delphi.mskcc.org

And then you will be able to access from your local computer at http://localhost:8080 . The other port, 5000, is necessary if you want to test API calls from your local environment. With the example above, you would use the api endpoint "http://localhost:5000"

Testing different roles

User roles are generally managed using EZGroups. A quicker way to test roles, especially for local testing, is to change ADMIN_GROUPS or CLINICAL_GROUPS in sample-tracking-api/app/__init__.py. For example, a tester who has the default "non-clinical" role but has membership in the LDAP group "zzPDL_Group2", can switch to admin view with the following line-change:

#previous
ADMIN_GROUPS = ['zzPDL_Group1']
#testing
ADMIN_GROUPS = ['zzPDL_Group1','zzPDL_Group2']

Maintaining Production and Development Trackers

Configuration

At the time of writing, the production (prod) and development (dev) trackers are configured to work with uwsgi and nginx on delphi. The deployed code are located, respectively, at:

/data/www/flask/sample-tracking-prod/
/data/www/flask/sample-tracking-dev/

The uwsgi services are defined with the following files:

/usr/lib/systemd/system/uwsgi-backend-dev.service
/usr/lib/systemd/system/uwsgi-frontend-dev.service
/usr/lib/systemd/system/uwsgi-backend-prod.service
/usr/lib/systemd/system/uwsgi-frontend-prod.service

Each service can be restarted with systemctl, for example:

# as uwsgi user
sudo /usr/bin/systemctl restart uwsgi-frontend-dev.service

Each of these service files uses an initialization file located at /data/www/uwsgi/vassals. You can see the ini file used in the text of the service file.

Currently, each uwsgi service is started with the --touch-reload parameter pointing to the respective ini file, which means we can touch ini files to reload the service.

While there are 4 uwsgi services, only 1 nginx service is necessary to maintain. The service file is at /usr/lib/systemd/system/nginx.service. Generally it does not need to be restarted when changes occur to the deployment code. Status of each service can be viewed with systemctl status <service file>.

Changes to Frontend

When making changes to the frontend code in dev or prod environments, it is recommended to re-build the javascript packages, otherwise the changes may not take effect. Start with a fresh sample-tracking-frontend directory from the code repo, enter the directory and run the following:

npm i react-scripts
# for dev only
npm run build-dev
# for prod only
npm run build-prod

Then replace the old directory with the new one so that uwsgi is pointing to it, and reload it by touching the ini file. It may be helpful to temporarily stash the old directory until confidence in the new directory is achieved.

Making changes to the database

Flask gives you the ability to manage the structure of the database in a versioned manner. If you need to add or remove columns from the database, you should do so through flask. You can do so in the conda environment with these commands as a rough guide:

cd /path/to/mydir/sample-tracking-api/
python manager.py db migrate
# a successful migrate command creates an executable script that should first be revised
python manager.py db edit 
python manager.py db upgrade

Sometimes you may not be able to migrate automatically until you reconcile the versioning held in sample-tracking-api/migrations/versions/. Run python manager.py db to see all available commands.

When migrating the database, make sure to edit the env parameter in lims_user_config to set the correct database URI.

Clone this wiki locally