3 21 Deployment Test Plan

·         #1957 Remove 'nucmd work run' ·         Opened by anurse 2 hours ago	Pull Request #4: Removed 'nucmd work run' command	·         "nucmd work run":  verify error message on execution ·         "nucmd work invoke": verify this command's behavior is unaffected by change ·         Invoke each job with JobHost.exe, validate successful execution and logging. ·         Run JobHost.exe with "dbg" switch, verify successful execution and logging.
·         #1955 Add nucmds for querying the search index ·         Opened by anurse 3 days ago	Pull Request #5: Added Search Client Commands	·         nucmd search query ·         Validate running without password ·         Run without query, verify exception handling ·         Run with every combination of two argument flags, verify results ·         Perform Search syntax and search relevance tests (link:  Automated Search Tests) ·         Verify E2E experience of building a client app that consumes this data ·         nucmd search checksums ·         Validate password security for the command ·         Test user experience of searching for a single checksum using tool. ·         Test the experience of getting the checksums for multiple versions of a package ·         Test the experience of getting the checksums for multiple packages from a single uploader ·         nucmd search diag ·         Validate password security for the command ·         Verify output data formatting ·         nucmd search fields ·         Validate password security for the command ·         Verify E2E experience of using listed fields in search queries.
·         #1940 Add work service jobs to maintain search service index ·         Opened by anurse 6 days ago   ·         #1922 Build a work job to generate search relevance rankings ·         Opened by anurse 14 days ago	Pull Request #8: Fix NuGet/NuGetGallery#1940 by adding search maintenance tasks Pull Request #5: Fix NuGet/NuGetGallery#1922 by creating a job to generate search relevance rankings	GenerateSearchRankingsJob ·         Verify successful messages get logged from this job. ·         Verify blob content post-execution o    Confirm new package downloads are reflected in rankings. o    Sanity-check the package rankings data per project type.   RebuildSearchIndexJob ·         Verify successful messages get logged from this job. ·         Edit a package, rebuild, verify updated search index   UpdateSearchIndexJob ·         Verify successful messages get logged from this job. ·         Upload a new package, verify presence in search index after 2 mins. ·         Upload a new package version, verify presence in search index after 2 mins. ·         Edit a package, verify presence in search index after 2 mins.   Andrew's Test Notes: The search service has some tasks that maintain the index (FullBuild, UpdateIndex, etc.). Those need to be plugged in to the Work Service ·         Rebuild the entire index (this job will not be scheduled, but will be available to be invoked, and should be run before deployment of the search service to new environments) ·         Update the index incrementally ·         Check Index Consistency (verify that the number of documents is near the number of packages in the DB)	(Note:  big)
·         #1923 CreateOnlineDatabaseBackupJob silently fails if source database does not exist ·         Opened by anurse 14 days ago	Pull Request #9: Fix NuGet/NuGetGallery#1923 by failing with an exception if source database not found	·         Run without target db, verify exception details ·         Define target in AppModel.xml, verify backup target in live site backend ·         Define target in payload, verify backup target in live site backend ·         E2E:  Confirm experience of using JobHost to run job and discovering requirement. ·         E2E:  Confirm experience of scheduling jobs without a payload and discovering requirement.
·         #1905 Create initial search service ·         Opened by anurse 19 days ago ·         1 comment   https://github.com/NuGet/NuGetApi/issues/11	-	·         Verify tokenization (e.g. AbAcAd can be found by searching for "Ab Ac", "AbAc", "Ab Ad", etc.) ·         E2E for search relevance (link:  Automated Search Tests) ·         E2E for debugging into poor search results (e.g. in response to a package owner complaint) ·         E2E for user scenarios around complaining about poor search results (how would someone get in touch with us?) ·         E2E for consuming the search service in a custom client app ·         Validate scenarios from Andrew's test notes below. ·         Bug Bash (time permitting; I might wait until the search service is better integrated)   Andrew's Test Notes Using the Admin Key in the CSCFG file, you can access  https://<searchservicehost>/search/console  in order to test various queries. Since the service is not yet integrated into the Gallery, this provides a simple way to bash on the data and ensure a few things: ·         When a new package is uploaded, it appears in the index within 2 minutes ·         When a new version of an existing package is uploaded, it appears within 2 minutes ·         When a package is listed/unlisted, the index is updated within 2 minutes (ensure "Ignore Filter" is not checked as that will ignore the listed/unlisted filter) ·         When a package is edited, the index is updated about 2 minutes AFTER the edit is finalized by HandlePackageEdits.
·         #1901 CreateOnlineDatabaseBackupJob throws if TargetDatabase is null Issue #1941: Unhandled NullReferenceException in CreateOnlineDatabaseBackup if TargetDatabaseName is not defined	Pull Request #10: Fix NuGet/NuGetGallery#1901 by adding ye olde null-check	·         Re-run repro scenario, verify exception handling. ·         Define target in AppModel.xml, verify backup target in live site backend ·         Define target in payload, verify backup target in live site backend ·         E2E:  Confirm experience of using JobHost to run job and discovering requirement. ·         E2E:  Confirm experience of scheduling jobs without a payload and discovering requirement.