Performance Guard
The performance guard is designed to prevent negative performance impacts from unknowingly being merged into the master branch. When a developer submits a PR the CI pipeline is run. During the CI pipeline, several performance tests are run. When the CI pipeline finishes the performance guard compares the performance test results against tests with the same parameters that were most recently run against the master branch. The performance guard will generate a status for the PR based on the comparison of the results.
- Performance Storage Service
- Django
- Github API
- NoisePage Checks Github App
- Time-Series Database
- TimescaleDB
When a developer creates a PR it generates an event that the NoisePage Performance Guard Github app is subscribed to. When the create event occurs it is sent to the performance storage service which initializes a Github check run for the PR. When the CI pipeline completes this creates a Github status. This status creation generates an event that the NoisePage Performance Guard Github app is subscribed to. This event is sent to the performance storage service which will then query the time-series database for the most recent performance results associated with that PR branch. The performance storage service then extracts the test configuration from each of the performance results and queries the time-series database for the most recent performance results from the master branch with those test configurations. The performance storage service then compares the percentage difference between the two results. Finally, the performance storage service updates the check run as completed and assigns a status based on the worst performance difference as follows:
| Min % Change | Status |
|---|---|
| >= (0%) | Success |
| (0%) to (-5%) | Neutral |
| < 5% | Failure |
The check run additionally displays all the information for each performance test in its details.
The following is a chart show the code dependencies in the performance storage service, relating to the OLTPBench tests.
