init commit #116
Open
init commit #116
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* gn3/auth/authentication.py: new function `credentials_in_database` * gn3/auth/authentication/__init__.py: replace package with module * gn3/settings.py: new `AUTH_MIGRATIONS` configuration variable * migrations/auth/20221103_02_sGrIs-create-user-credentials-table.py: new migration * tests/unit/auth/test_credentials.py: test the `credentials_in_database` function * tests/unit/conftest.py: more test fixtures
* tests/unit/auth/conftest.py: add fixtures specific to auth * tests/unit/auth/test_migration_create_users_table.py: import from new fixtures module * tests/unit/conftest.py: remove auth-specific fixtures from *ALL* unit tests fixture module.
* tests/unit/auth/test_create_user_credentials_table.py: new tests
* gn3/auth/authentication.py: Fix issues caught by tests * tests/unit/auth/test_credentials.py: Add fixtures and tests for credentials checking
* main.py: Provide the `apply-migrations` CLI command to run the migrations
against the auth database.
The command can be invoked with:
$ flask apply-migrations
* gn3/settings.py: Omit trailing slash * tests/unit/auth/test_create_table_migrations.py: Generalise testing migrations that create tables. * tests/unit/auth/test_create_user_credentials_table.py: delete * tests/unit/auth/test_migration_create_users_table.py: delete
* migrations/auth/20221108_02_wxTr9-create-privileges-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration
* migrations/auth/20221108_03_Pbhb1-create-resource-categories-table.py: new migration. * tests/unit/auth/test_create_table_migrations.py: test new migration.
* migrations/auth/20221108_04_CKcSL-init-data-in-resource-categories-table.py: new migration. * tests/unit/auth/test_migration_init_data_in_resource_categories_table.py: test new migration.
* tests/unit/auth/test_migration_init_data_in_resource_categories_table.py: Test that the data is initialised properly. Test that rollback works as expected.
selected This is because base_samples was set to all_samples_ordered, which only includes primary samples + parents/f1s. Setting this to an empty list fixed the issue and caused it to use all samples again.
* gn3/migrations.py: Minor change * migrations/auth/20221110_01_WtZ1I-create-resources-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration
* migrations/auth/20221110_02_z1dWf-create-mrna-resources-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test for new migration
* migrations/auth/20221110_03_ka3W0-create-phenotype-resources-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test for new migration
* migrations/auth/20221110_04_6PRFQ-create-genotype-resources-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration
* migrations/auth/20221110_05_BaNtL-create-roles-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration * tests/unit/auth/test_migrations_indexes.py: test new migration
* migrations/auth/20221110_06_Pq2kT-create-generic-roles-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration
* tests/unit/auth/test_create_table_migrations.py: new migration * tests/unit/auth/test_migrations_indexes.py: test new migration
* migrations/auth/20221110_07_7WGa1-create-role-privileges-table.py: new migration Commit ee72678 only contains tests for this migration.
* migrations/auth/20221110_08_23psB-add-privilege-category-and-privilege-description-columns-to-privileges-table.py: new migration * tests/unit/auth/test_migrations_add_remove_columns.py: test new migration
* .gitignore: ignore all yoyo configuration files * README.md: Update documentation * yoyo.auth.ini: stop tracking the yoyo configuration file.
* migrations/auth/20221113_01_7M0hv-enumerate-initial-privileges.py: new migration. * tests/unit/auth/test_migrations_insert_data_into_empty_table.py: test new migration.
Add table `generic_role_privileges` table to link the generic roles to the privileges they provide. * migrations/auth/20221114_01_n8gsF-create-generic-role-privileges-table.py: new migration * tests/unit/auth/test_create_table_migrations.py: test new migration * tests/unit/auth/test_migrations_indexes.py: test new migration
Due to the fact that the data migration requires higher privileges than the user details, separate the user details migration endpoint from the data migrations endpoint.
The functions that fetch the unlinked data have checks to ensure they are called by users with the appropriate privileges. This commit enables a temporary escalation of privileges to enable fetching the data, so as to allow successful data migration.
Rework the migration trigger endpoint such that it does a bulk import of all the resources in the redis store. It also expects that the user that triggers the migration have the appropriate privileges.
For Phenotypes, list the Phenotype traits rather than the dataset(s).
Enable the 'OFFSET' clause in the queries to allow for progressive fetching of data from the database using the 'Previous' and 'Next' buttons.
* gn3/db/sample_data.py: Remove 're' import. (get_trait_csv_sample_data): Remove fetching sample data. Signed-off-by: Munyoki Kilyungi <[email protected]>
The way data is linked to the resources needs to be reworked. This commit removes all the existing migration scripts that created the tables formerly used for linking data in preparation for reworking the system.
This is an initial attempt: it does not allow a search to be carried out across the data available in the database. I will need to rework this, probably start from the UI and work backward.
The configuration in gn3.settings can (and does) get overwritten by values in the environment variable `GN3_CONF` and any configurations passed in the call to the `gn3.app.create_app` function; as such, this commit changes the configuration used in the code to user the final configuration values that are in the running application's `flask.current_app.config` object.
Fix the bug where the system was trying to load a user from a non-existing OAuth2 client, leading to an exception.
There is need to run external scripts using the same configurations as the application but without the need to couple the script to the application. In this case, we provide the needed configuration directly in the CLI, and modify the existing `gn3.db_utils.database_connection` function to allow it to work coupled to the app or otherwise.
Remove the deprecated function and fix a myriad of bugs that arise from removing the function. Issue: https://issues.genenetwork.org/issues/bugfix_coupling_current_app_and_db_utils
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.