diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 0000000..6138baf --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,83 @@ +# Development guidelines + +These guidelines should be followed for code to be used in production. PoCs can decide not to follow all of the guidelines, however, if they are intended to be used in production, extra work needs to be done to meet production quality. +The following topics are covered: + +- [1. Version Control of Sources](#1-version-control-of-sources) + - [git-flow](#git-flow) +- [2. Coding style](#2-coding-style) + - [Fail-Fast](#fail-fast) + - [Environments](#environments) +- [Logs](#logs) +- [3. Testing](#3-testing) + - [Unit Tests](#unit-tests) + - [Integration Tests](#integration-tests) + - [Manual Tests](#manual-tests) + +## 1. Version Control of Sources + +`git` is used as version control system of source code. Git repositories are hosted on [github.com](github.com). All sorts of code, scripts (bash, SQL, ...) and documentation should be tracked. + +**IMPORTANT:** read [Git Flow](GIT_FLOW.md) to learn about the `git` and `PR` workflow. + +### git-flow + +[Git Flow](GIT_FLOW.md) + +## 2. Coding style + +Some guidelines that are generic to all languages are noted here. Language specific things can be found here: + +[Coding Guidelines](README.md#coding-guidelines) + +Keep in mind that most of the time is spend **reading** code (not writing), yours or from someone else. Hence try to make the live easier of the one **reading** and trying to understand your code. + +### Fail-Fast + +**The goals is not to make no mistakes. The goal is to find and fix them quickly!** + +- When there is missing environment variable or start-up parameters, instead of still starting up the system normally or using fall-back strategy (fall-back to default environments/parameters), the system should fail and stop so that we can be notified and fix the problem right away. + +- When a client sends a request with invalid parameters, instead of silently correct the parameters and continue handling the request, the server should let the request fail so that client can be notified and fix the problem as soon as possible. Sometimes it can make sense to be permissive on what you accept and restrictive on what you send (i.e. sanitize wrong input if possible). + +- Exceptions should never be silently swallowed. Exceptions should only be caught when the catcher know how to handle it; otherwise, let the exception be thrown outside. And let the app crash if no part of the app knows how to handle it (An exception caused by unexpected bugs). And then fix it. + +### Environments + +Three environments are available + +- *dev*: the development environment is mainly used to test and develop things that either need access to hardware other than the notebook provides or for interface development, where the component under development must be accessible by other systems. Whenever development needs other resources, it should be *dev* resources. +- *int*: the integration or staging environment is used to verify that production-ready changes to a component work as expected together with other resources (in terms of functionality, performance, ...) +- *prod*: everything that other people and systems rely on for doing their work. This can be other internal systems, external services, internal or external users of apps and services. + +Development should happen as much as possible locally on the notebook. There are still some restrictions that we're trying to mitigate (e.g. currently it's not possible to build a docker container locally when in the Bundesnetz or `gov-public` Wifi). + +## Logs + +Each service must write logs using different log levels. The logs should be by default written in `stdout`, but at best configurable as well as the output format should be configurable. Logs will be centralized in a logging infrastructure. + +## 3. Testing + +Testing is an integral part of development and mandatory for production targeted code. This applies to apps as well as scripts. We distinguish three categories of tests: + +- unit tests +- integration test +- manual tests + +The first two are automated tests, the third one obviously manual. Language specific details about testing can be found here: + +- [python](PYTHON.md#9-unit-testing-frameworks) +- [bash](BASH.md#4-unit-tests--shellspec) +- [javascript](JAVASCRIPT.md#testing) + +### Unit Tests + +Unit Tests can be performed before a docker image is built using a dedicated test runner for Unit tests. Unit Tests furthermore don't require external resources. If useful, external resources can be mocked in Unit Tests. Target code coverage for Unit Tests is above 70%. + +### Integration Tests + +Integration Tests are performed with the built docker image and have access to external resources. Integration test should make sure that the newly built version works well together with the existing data and other services. Integration tests should use staging (or integration) environment and not production resources. + +### Manual Tests + +Manual tests should be performed after major deploys directly on prod to verify that deployment was successful (Note: this might change once deployment is automated and e2e frontend tests exist). diff --git a/GITHUB.md b/GITHUB.md new file mode 100644 index 0000000..14cac33 --- /dev/null +++ b/GITHUB.md @@ -0,0 +1,10 @@ +# GITHUB + +All github repository of the geoadmin organization are managed by terraform and all settings as well as user management MUST BE done only using terraform ! + +## Table of content + +1. [Repository naming convention](GITHUB_NAMES.md) +2. [Git cheat sheet](GIT_CHEAT_SHEET.md) +3. [Git Flow](GIT_FLOW.md) +4. [Github Workflows](GITHUB_WORKFLOW.md) diff --git a/GITHUB_WORKFLOW.md b/GITHUB_WORKFLOW.md new file mode 100644 index 0000000..7203199 --- /dev/null +++ b/GITHUB_WORKFLOW.md @@ -0,0 +1,30 @@ +# Github Workflow for Geoadmin repositories + +Most of the Geoadmin services repositories should be configured to use github workflow to automate the services release process. + +We use github reusable workflow that are centralized in [geoadmin/.github](https://github.com/geoadmin/.github) repository. In the service repository we only need the github workflow to call the reusable workflow. + +At best the workflows are automatically set during creation of the repository using terraform. + +- [Setting up SemVer Workflow](#setting-up-semver-workflow) +- [Setting up Milestone Workflow](#setting-up-milestone-workflow) + +## Setting up SemVer Workflow + +For more information on this workflow see [SemVer 2.0 Version](VERSIONING_RELEASE.md#semver-20). + +You can find the template that setup the workflow used by terraform here: [geoadmin/template-service-semver-public](https://github.com/geoadmin/template-service-semver-public) + +:warning: Once the repository has been created and the workflows set, you need to create the initial tag `v0.0.0` due to a bug in the semver release workflow + +```bash +git checkout master +git tag v0.0.0 +git push origin v0.0.0 +``` + +## Setting up Milestone Workflow + +For more information on this workflow see [Milestone Version](VERSIONING_RELEASE.md#milestone-version). + +You can find the template that setup the workflow used by terraform here: [geoadmin/template-service-milestone-public](https://github.com/geoadmin/template-service-milestone-public) diff --git a/GIT_FLOW.md b/GIT_FLOW.md new file mode 100644 index 0000000..8c407b9 --- /dev/null +++ b/GIT_FLOW.md @@ -0,0 +1,394 @@ +# Git Workflow + +Definition of the swisstopo `git` workflow used in github repositories in order to keep the history as clean as possible. + +- [Goal](#goal) +- [Branch Model](#branch-model) + - [Personal branches naming convention](#personal-branches-naming-convention) +- [Long term development: `develop-DESCRIPTION`](#long-term-development-develop-description) +- [Pull Request Rules](#pull-request-rules) + - [PR for Request For Comment (RFC)](#pr-for-request-for-comment-rfc) + - [PR Work in Progress](#pr-work-in-progress) +- [Git Workflows](#git-workflows) + - [Create Personal Branch](#create-personal-branch) + - [Create PR](#create-pr) + - [Modify a PR](#modify-a-pr) + - [Close a PR](#close-a-pr) +- [New release](#new-release) +- [Merge Conflict](#merge-conflict) + - [1. Conflict in a personal head branch](#1-conflict-in-a-personal-head-branch) + - [2. Conflict in a shared head branch](#2-conflict-in-a-shared-head-branch) + - [Merging `develop` or `develop-YYYY-MM-DD` into `master`](#merging-develop-or-develop-yyyy-mm-dd-into-master) + - [Merging `master` into `develop` or `develop-YYYY-MM-DD`](#merging-master-into-develop-or-develop-yyyy-mm-dd) + - [Merging `develop` into `develop-DESCRIPTION`$](#merging-develop-into-develop-description) +- [GIT Configuration](#git-configuration) +- [Github repository Configuration](#github-repository-configuration) +- [References](#references) + +## Goal + +The goal of the following rules and workflow is to avoid such git history + +![git-flow-merge.svg](./assets/images/git-flow-merge.svg) + +Which can end up quite messy + +![git-flow-mess.png](./assets/images/git-flow-mess.png) + +But to have an history like this + +![git-flow-rebase.svg](./assets/images/git-flow-rebase.svg) + +Which is, when we avoid `hotfix`, much easier to read and understand. + +## Branch Model + +We follow the git flow as in the picture below: + +![git-flow.svg](./assets/images/git-flow.svg) + +This flow is an adaptation of the original [Git Flow](https://nvie.com/posts/a-successful-git-branching-model/) with some ideas from [Github Flow](http://scottchacon.com/2011/08/31/github-flow.html) and [Gitlab Flow](https://docs.gitlab.com/ee/topics/gitlab_flow.html). + +The main branch is `master` and the branches for development are `develop` and `develop-YYYY-MM-DD` (see [Milestone Versioning](VERSIONING_RELEASE.md#milestone-version). + +- Everything in `master`, `develop` and `develop-YYYY-MM-DD` branch has production quality +- Directly pushing to `master`, `develop` or `develop-YYYY-MM-DD` is forbidden and should be protected by github branch rule +- `master`, `develop` and `develop-YYYY-MM-DD` are **SHARED** branches +- `push --force` on shared branches is **FORBIDDEN** +- Code on `master` is deployed on PROD and INT +- Code on `develop` or `develop-YYYY-MM-DD` is deployed on DEV +- New features and bug fixes are done on **PERSONAL** branches (see below for [naming conventions](#personal-branches-naming-convention)) and merged in `develop` or in `develop-YYYY-MM-DD` +- Personal branches are owned by the creator and only the owner is allowed to push to it ! This allow the uses of `push --force` on those branches. +- Hot fixe should be avoided, if not avoidable it must be small changes and directly merged into `master`, `develop` or `develop-YYYY-MM-DD` + +### Personal branches naming convention + +The naming convention below is important for the github actions to work properly ! + +| Branch | Description | +|--------|-------------| +| `data-JIRA-*` | Personal branch to be merged by PR into `develop-YYYY-MM-DD`. This branch SHOULD have a `JIRA` ticket number and optionally a very short title at the end. This branch contains data integration related work. | +| `feat-JIRA-*`, `feature-JIRA-*` | Personal branch to be merged by PR into `develop` or `develop-YYYY-MM-DD`. This branch SHOULD have a `JIRA` ticket number and optionally a very short title at the end. This branch contains non data integration related work that should be done for the next release. | +| `bug-JIRA-*`, `bugfix-JIRA-*` | Personal branch to be merged by PR into `develop` or `develop-YYYY-MM-DD`. This branch SHOULD have a `JIRA` ticket number and optionally a very short title at the end. This branch contains bugfix related work that should be done for the next release. | +| `hotfix-JIRA-*` | Personal branch to be merged by PR directly into `master` and will be deployed as hot fix. Note here the JIRA ticket might be optional if there is no JIRA ticket related to the bug fix. | + +**NOTES:** + +- **Prefix in branch names are important in order to set the correct PR label for release note categorization!** +- **JIRA ticket number in branch is important in order to link the branch to the JIRA ticket !** +- **Git commit should also have the JIRA ticket number in the title for the linking !** + +## Long term development: `develop-DESCRIPTION` + +In case of a long term development, we should avoid working weeks or even worse months long on a single PR +creating at the end a huge PR to review. Review of huge PR is not efficient and often of bad review quality +(missing important points). To avoid this and still keep the branch rules defined above (`master` and +`develop` branches have production and working quality codes), we can create a third shared branch; + +```text +develop-DESCRIPTION +``` + +where `DESCRIPTION` is a meaningful short description that can be easily understood (for example `develop-refactor-drawing`, `develop-migration-frankfurt`). + +This branch must then follow these rules: + +- Must be created from `develop` + - for milestone workflow repository (e.g. `mf-chsdi3`) then we have two use cases + 1. The development is done within a few to several weeks and is to be completed before the end of the current milestone. The work is data integration related. + - `develop-DESCRIPTION` is created from `develop-YYYY-MM-DD` branch + - `develop-DESCRIPTION` is to be merged into `develop-YYYY-MM-DD` branch at the end. + 2. The development spanned accross several milestones and is not really data integration related + - `develop-DESCRIPTION` is created from `master` + - `develop-DESCRIPTION` is merged into `master` at the end +- `develop-DESCRIPTION` is a shared branch where several developer can work on it and **MUST** have github branch rule set accordingly. + - Directly pushing to `develop-DESCRIPTION` is forbidden and should be protected by github branch rule + - All PRs to `develop-DESCRIPTION` must be peer reviewed before merging (protected by github rule) +- New features/bug fixes are done on **PERSONAL** branches (see below for [naming conventions](#personal-branches-naming-convention)) and merged into `develop-DESCRIPTION` via reviewed PRs. +- Personal branches are owned by the creator and only the owner is allowed to push to it ! This allow the uses of `push --force` on those branches. +- PRs to `develop-DESCRIPTION` code might not be working and might make the unit test failing, but still needs to have productive/clean code. +- TODO in `develop-DESCRIPTION` PRs are allowed. +- All other PR rules defined below in [Pull Request Rules](#pull-request-rules) are still valid (except of the two points mentioned just above) +- `develop-DESCRIPTION` branch needs to be regularly updated from its upstream branch; create a PR to merge `develop` into `develop-DESCRIPTION` +- Once `develop-DESCRIPTION` has reached a fully working productive state, it can be merged back into its upstream branch; create a PR to merge `develop-DESCRIPTION` into `develop` and once merged delete `develop-DESCRIPTION` + +## Pull Request Rules + +- Pull requests should only be created if the changes in the `feature|bugfix|hotfix` branch is **ready** to be discussed/reviewed/merged. Exception to the rules are PR for RFC (see [PR for Request For Comment (RFC)](#pr-for-request-for-comment-rfc)). +- A PR should be of **production** quality +- A PR has a proper title. The title MUST contain the JIRA related ticket number if any (`PB-9999: ...`). +- A PR Title should reflect the changes and be written in past tense. Title is used to generate the Release Note ! +- A PR is only assigned for review when it's **finished** and not WIP anymore, all commits must be **ready** for review (see below) +- A PR contains atomic and well named commits: + - Each commit should have a comment in the following forms + + ```text + : Title (should not be more than 100 chars) + + Description of the issue being fixed by the commit. Eventually how the issue could be reproduced. + + Description of the solution to the issue that is being fixed. + ``` + + - if you have too many commits in your PR, **squash** them before assigning the PR for review +- A PR addresses a well described and well documented issue +- A PR describes the changes that are introduced and mentions possible side-effects for testing +- A PR describes the necessary steps to undertake before merging (infrastructure, data, varnish, etc.) +- A PR provides a test link - and if that's not possible describes how it can be tested +- PR must be **peer-reviewed** (unless changes are trivial). This applies to everything under version control (code, scripts, config, SQL,...). Try to review quickly if your review is requested. +- EVERY Developer is responsible to periodically checks his review requests (using email notification and/or the github [Pull Request page](https://github.com/pulls/review-requested)) and to answer them ASAP, in order to not slow down development. +- PR must be merged with merge commits. This keeps the original commit hashes, adds a dedicated merge-commit and makes it easier to figure out which commits have been merged in which branches (no `fast forward` merge). +- **Avoid** changing commits after review (avoid _squash_, _split_ or _rewrite_), do new commit to apply the requested changes by the reviewer. +- PR must be **up to date** before merge, do a `git rebase` before merging. + + ```bash + git fetch origin + git rebase origin/develop # or origin/master + git push --force origin PR_BRANCH + ``` + +- **Don't use** the `Update Branch` Github button, use git CLI instead (see above). + ![github-update-branch-button](./assets/images/github-update-branch-button.png) + Alternatively you can use the `Update with rebase` Github button ![github-update-branch-rebase-button.png](./assets/images/github-update-branch-rebase-button.png) +- Work is considered done, if a PR is **merged** (not when it's created) + +### PR for Request For Comment (RFC) + +PR can be used for starting a discussion as RFC. In this case the first rule can be ignored, but you need then to follow these ones: + +- PRs for RFC must have `RFC` label and must be marked as `Draft`. + + ![github-pr-rfc-draft-screenshot.png](./assets/images/github-pr-rfc-draft-screenshot.png) + +- Git commit don't requires propers message as the PR is in draft mode +- Once the RFC has ended, there is the following two possible flow: + 1. The PR is closed with a comment. Nothing gets merged! + 1. The PR is ready for review. + - This imply to first remove `RFC` label + - Then make sure that the PR follows the [Pull Request Rules](#pull-request-rules) above + - Finally remove it from `Draft` mode by clicking on `Ready for Review` button + + ![github-pr-draft-ready-for-review.png](./assets/images/github-pr-draft-ready-for-review.png) + +### PR Work in Progress + +PR could be used to test the code on CodeBuild (usually only needed when setting up CodeBuild): + +- PRs for WIP must have `WIP` label and must be marked as `Draft`. + + ![github-pr-wip-draft-screenshot.png](./assets/images/github-pr-wip-draft-screenshot.png) + +- Git commit don't requires propers message as the PR is in draft mode +- Once the WIP has ended, there is the following two possible flow: + 1. The PR is closed with a comment. Nothing gets merged! + 1. The PR is ready for review. + - This imply to first remove `WIP` label + - Then make sure that the PR follows the [Pull Request Rules](#pull-request-rules) above + - Finally remove it from `Draft` mode by clicking on `Ready for Review` button + + ![github-pr-draft-ready-for-review.png](./assets/images/github-pr-draft-ready-for-review.png) + +## Git Workflows + +This workflow refers to SemVer versioning, for milestone simply replace `develop` by `develop-YYYY-MM-DD`. + +### Create Personal Branch + +1. Update the `develop` branch + + ```bash + git checkout develop + git pull # git pull origin develop + ``` + +1. Create and checkout your personal branch named according to the [convention](#personal-branches-naming-convention). + + ```bash + git checkout -b PRIVATE_BRANCH # git branch PRIVATE_BRANCH; git checkout PRIVATE_BRANCH + ``` + +1. Do your work and push it to the server + + ```bash + git add . + git commit -m "did something" + git push origin PRIVATE_BRANCH # NEVER EVER DO `git push` always specify the destination `origin` and source + ``` + +### Create PR + +1. Update first your personal branch + + ```bash + git checkout PRIVATE_BRANCH + git fetch origin develop:develop # git fetch origin master:master + git rebase origin/develop + # or for master PR + git rebase origin/master + ``` + +1. Eventually re-organize, squash or rewrites your commits to have them production ready + + ```bash + git rebase -i origin/develop + # or + git rebase -i origin/master + + # then push the new commits + git push --force origin PERSONAL_BRANCH # force push on personal branch is allowed + ``` + +1. Login into Github and navigate to the repo in order to create the PR (follow the [Pull Request Rules](#pull-request-rules) above) +1. Assign one or more reviewer + +### Modify a PR + +Once a PR has been created, some modification might be requested by peers (reviewer). + +1. Create new commits and push them back to your personal branch (PR branch) +1. Eventually update the PR branch with `git rebase origin/develop|origin/master` +1. Re-request a review of your changes + +### Close a PR + +Once a PR has been approved do as follow + +1. Make sure that the PR is up to date + + ```bash + git fetch origin develop:develop + git rebase -i origin/develop + # or + git rebase -i origin/master + + # then push the new commits + git push --force origin PRIVATE_BRANCH + ``` + +1. Then merge the PR to `develop` or `master`. This should be done from Github and not from the CLI + +1. Finally do some clean up actions: delete your local PR branch as well as the remote PR branch (if not automatically done) + + ```bash + git branch -d PRIVATE_BRANCH + ``` + +## New release + +See [Versioning and Release](./VERSIONING_RELEASE.md) + +## Merge Conflict + +When trying to merge a PR, conflict might occur. There is 2 type of conflicts that needs to be dealt differently. + +### 1. Conflict in a personal head branch + +When merging a personal branch into a shared branch (e.g. `feat-my-feature` into `develop`), we should +never have conflict as the head branch should be always rebased on top of the base branch. Due to this +conflict always happens during the rebase and needs to be fixed during the rebase step on the personal branch. + +### 2. Conflict in a shared head branch + +When merging two shared branches, conflict might occurs and we can't directly fix them on a shared +branch has we have a PR policy. + +#### Merging `develop` or `develop-YYYY-MM-DD` into `master` + +In this case we should never have any conflict because should always keep `develop*` up to date by +merging `master` into `develop*` (this is only needed when hotfixes have been merged into `master`). + +#### Merging `master` into `develop` or `develop-YYYY-MM-DD` + +When hotfixes have been merged into `master` then need to be added as well on the `develop*` branch. +For this we create a PR to merge `master` into `develop*`. In this case conflict might occur ! To +solve them procede as follow: + +1. First open a PR to merge `master` into `develop` or `develop-YYYY-MM-DD` +2. Then if the PR can't be merge do to conflict, checkout both branch locally (make sure that both branch are up to date with their origin) + + ```bash + git checkout master + git pull + git checkout develop + git pull + ``` + +3. Then manually merge `master` into `develop` + + ```bash + git merge master + ``` + +4. Solve the merge conflict using the merge tool (at best with Beyond Compare) + + ```bash + git mergetool + ``` + +5. Once all conflict have been resolved, commit the mergeconflict changes, it is a best practice to put +the files that had conflict in the merge commit message. For this simply uncomment the conflicts line in the +automatic commit message. + + ```bash + git commit + ``` + +6. Finally, push the merge. Before this make sure that the PR open in step 1. has been approved, otherwise you won't be able to merge ! + + ```bash + git push origin develop + ``` + +#### Merging `develop` into `develop-DESCRIPTION`$ + +Follow the same step as in [Merging `master` into `develop` or `develop-YYYY-MM-DD`](#merging-master-into-develop-or-develop-yyyy-mm-dd) + +## GIT Configuration + +In order to have a proper git history, each developer MUST HAVE the following git configuration + +```bash +$ git config --global --list +... +user.name= +user.email= +merge.ff=false +pull.rebase=true +... +``` + +To edit the config do: + +```bash +git config --global --add user.name +git config --global --add user.email +git config --global --add merge.ff false +git config --global --add pull.rebase true +# or +$ git config --global --edit +... +[user] + name = + email = +[merge] + ff = false +... +``` + +This ensure that a commit has a proper author and that the default behavior for merge is a non fast forward. + +## Github repository Configuration + +- Each github repo MUST BE configured to enforce PR review. +- Each github repo MUST DISALLOW the PR Merge if the PR is **not up to date**. +- Each github repo configuration MUST BE managed by Terraform, see [geoadmin/infra-terraform-github-bgdi](https://github.com/geoadmin/infra-terraform-github-bgdi) + +## References + +Here below are the reference list for the flow: + +- [“A successful Git branching model” by Vincent Driessen](https://nvie.com/posts/a-successful-git-branching-model/) +- [“GitHub Flow” by Scott Chacon](http://scottchacon.com/2011/08/31/github-flow.html) +- [“Introduction to GitLab Flow” by GitLab](https://docs.gitlab.com/ee/topics/gitlab_flow.html) +- [“GitFlow considered harmful” by Adam Ruka](http://endoflineblog.com/gitflow-considered-harmful) +- [“OneFlow — a Git branching model and workflow” by Adam Ruka](http://endoflineblog.com/oneflow-a-git-branching-model-and-workflow) diff --git a/JAVASCRIPT.md b/JAVASCRIPT.md index 282499d..8042ff8 100644 --- a/JAVASCRIPT.md +++ b/JAVASCRIPT.md @@ -15,6 +15,8 @@ There are a number of style guides out there. Here a few important things that w - [Component method](#component-method) - [Definition](#definition) - [Method called from template](#method-called-from-template) +- [Testing](#testing) + - [E2E tests of web application](#e2e-tests-of-web-application) - [Debugging on Mobile Phone](#debugging-on-mobile-phone) **The foremost goal is that reading and understanding your javascript code is easy for someone else (or yourself in a few months time).** @@ -241,6 +243,7 @@ export default { }, } ``` + ```javascript import { ref } from 'vue' import { Map } from 'ol' @@ -266,6 +269,7 @@ export default { }, } ``` + ```javascript import { ref } from 'vue' import { Map } from 'ol' @@ -323,6 +327,20 @@ Also prefer the short syntax `increment() {}` to `increment: function() {}` Methods called from a template (except for event listeners) should not have any side effects, such as changing data or triggering asynchronous processes. If you find yourself tempted to do that you should probably use a [lifecycle hook](https://v3.vuejs.org/guide/instance.html#lifecycle-hooks) instead. For more infos about this see [Vue 3 - Methods](https://v3.vuejs.org/guide/data-methods.html#methods). +## Testing + +### E2E tests of web application + +In general for kind of end to end test for web application we use [Cypress.io](https://www.cypress.io/). + +For cypress best practice see [Cypress Best practice](https://docs.cypress.io/guides/references/best-practices) + +We also the following best practices: + +1. Don't create too much test case (`it()`) + 1. Each test case cost on cypress cloud + 2. Each test case takes time to initialized + ## Debugging on Mobile Phone Debugging issue that only touch Mobile Phones, especially IPhones, can be quite difficult without a Mac. diff --git a/README.md b/README.md index 290c648..349cc80 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,17 @@ Geoadmin PP BGDI general and coding guidelines. ## Table of Content -1. Coding guidelines - 1. [Ansible](ANSIBLE.md) - 2. [Bash](BASH.md) - 3. [Docker](DOCKER.md) - 4. [Javascript](JAVASCRIPT.md) - 5. [Python](PYTHON.md) - 6. [Python packaging](PYTHON_PACKAGE.md) -2. Github guidelines +1. Coding guidelines (#coding-guidelines) + 1. [General development](DEVELOPMENT.md) + 2. [Ansible](ANSIBLE.md) + 3. [Bash](BASH.md) + 4. [Docker](DOCKER.md) + 5. [Javascript](JAVASCRIPT.md) + 6. [Python](PYTHON.md) + 7. [Python packaging](PYTHON_PACKAGE.md) +2. [Github guidelines](GITHUB.md) 1. [Repository naming convention](GITHUB_NAMES.md) 2. [Git cheat sheet](GIT_CHEAT_SHEET.md) + 3. [Git Flow](GIT_FLOW.md) + 4. [Github Workflows](GITHUB_WORKFLOW.md) +3. [Versioning and release](VERSIONING_RELEASE.md) diff --git a/VERSIONING_RELEASE.md b/VERSIONING_RELEASE.md new file mode 100644 index 0000000..b96d9de --- /dev/null +++ b/VERSIONING_RELEASE.md @@ -0,0 +1,235 @@ +# Versioning and Release + +- [Introduction](#introduction) +- [SemVer 2.0](#semver-20) + - [New SemVer Beta Version](#new-semver-beta-version) + - [New SemVer Release](#new-semver-release) + - [Hotfix Release](#hotfix-release) +- [Milestone version](#milestone-version) + - [Goal](#goal) + - [Definition](#definition) + - [Milestone Workflow](#milestone-workflow) + - [Milestone release](#milestone-release) + - [Milestone hotfix](#milestone-hotfix) +- [Release Notes](#release-notes) +- [Deployment](#deployment) + +## Introduction + +Each piece of software must have a version that can be found as a TAG in git repository and as Docker Image TAG (if the software is dockerized). + +There are two categories of softwares: + +1. Pure services; services without data integration dependencies (e.g. service-stac, service-wmts, service-qrcode, ...) +2. Data services; services tightly coupled with data integration (e.g. config-wms-mapfile, service-search-sphinx, mf-chsdi3, ...) + +These two categories use different versioning scheme + +1. Pure services => [SemVer 2.0 version](#semver-20) +2. Data services => [Milestone version](#milestone-version) + +## SemVer 2.0 + +Specification: [SemVer 2.0](https://semver.org/) + +Code that is not tightly coupled with data integration process uses SemVer 2.0 versioning. + +SemVer was initially developed for API versioning but proved to be useful for applications as well. Quoting the summary of SemVer: + +> Given a version number `MAJOR.MINOR.PATCH`, increment the: +> +> - `MAJOR` version when you make incompatible API (or software) changes, +> - `MINOR` version when you add functionality in a backwards compatible manner, and +> - `PATCH` version when you make backwards compatible bug fixes. +> +> Additional labels for pre-release and build metadata are available as extensions to the `MAJOR.MINOR.PATCH` format. + +We use the SemVer with the `v` prefix; `vMAJOR.MINOR.PATCH` + +The git repository tagging and release is automated via github actions (see [.github/workflows/semver-release.yml](https://github.com/geoadmin/.github/blob/master/.github/workflows/semver-release.yml)). +It should be noted explicitly that this action doesn't change the git history, i.e. it doesn't modify files (e.g. a `version` file) or add annotated tags. + +See also [Setting up SemVer Workflow](GITHUB_WORKFLOW.md#setting-up-semver-workflow). + +### New SemVer Beta Version + +Beta versions are automatically done on every push to `develop`. A beta version as the following form; `NEXT_VERSION-beta.X` where `NEXT_VERSION` is the next semVer version (last version with `MINOR` incremented by one, e.g. `v1.2.0`) and `X` an incremental number. + +### New SemVer Release + +New version are done on demand by creating a PR from `develop` into `master`. The PR title should be +set to one of the following bump types: + +- `#major` to increment the `MAJOR` version (for breaking changes version) +- `#minor` to increment the `MINOR` version (default behavior) +- `#patch` to increment the `PATCH` version (for hotfix) + +Once the PR is opened, a github action automatically set the title to `New Release - `. + +***NOTE: If the bump type is not set in the original title, then by default it will be set to `minor`.*** +***If any commit(s) since last release contains a bump type, then the higher bump type has precedence.*** +***For example if a previous commit contained the `#major` bump type and the `#minor` bump type is set*** +***to the New Release PR, then `#major` takes precedence.*** + +The PR should be reviewed by at least one person before merging. + +When the PR is merged, a github action generate a new git TAG version (based on the bump type). + +After tagging a new Github Release with Release Note is generated by the action. See [Release Notes](#release-notes) + +### Hotfix Release + +Hotfix Release are created by creating a PR to merge `hotfix-*` branch directly into `master`. The +PR title should reflect the changes. Optionally the PR title can contain one of the following bump type pragma: + +- `#patch` to increment the `PATCH` version (hotfix default behavior) +- `#minor` to increment the `MINOR` version (if the hotfix requires more a minor changes as a simple patch) +- `#major` to increment the `MAJOR` version **! SHOULD NEVER BE DONE IN HOTFIX !** + +Once the PR has been opened, a github action automatically updates the PR title to ` - ` where _bump-type_ is automatically set based on the branch prefix: + +- #patch -> for `hotfix-*` or `bug-*` +- #minor -> for `feat-*` or `feature-*` + +The title can then still be manually updated if needed and the manual _bump type_ pragma takes +precedence to auto pragma from branch name. + +Then the PR needs to be reviewed and merged. + +When the PR is merged, a github action generates a new git TAG version (based on the _bump type_). + +After tagging a new Github Release with Release Note is also generated by the action. See [Release Notes](#release-notes) + +Once all hotfixes have been done, we need to merge back `master` into `develop` in order to back port the changes. See [Merging `master` into `develop` or `develop-YYYY-MM-DD`](./GIT_FLOW.md#merging-master-into-develop-or-develop-yyyy-mm-dd) for solving conflicts. + +*NOTE: alternatively we considered using `git cherry-pick` instead of merging, which keeps a cleaner git history. However this often results in merge conflicts during the next release. To avoid such conflicts or to have the conflicts as soon as possible (which is easier to resolve sooner than later) we use the merge strategy over cherry-pick* + +## Milestone version + +For code that is tightly coupled with data integration, a versioning scheme based on a scheduled milestone makes more sense as the code is released with a defined cycle (currently every two months). + +See also [Setting up Milestone Workflow](GITHUB_WORKFLOW.md#setting-up-milestone-workflow). + +### Goal + +The goal is to have accross all repositories and docker images the same tagging for a milestone that we can easily identify. The processes and tagging should be as much as possible automated via github actions triggered by PR. + +### Definition + + A Milestone is defined by a date (scheduled deploy date) in the following format: + +```text +YYYY-MM-DD +``` + +Each repository that follows this scheme **MUST** have a milestone defined as above and all PR related to this milestone should be attached to it. + +Each milestone release (git repository state that is used for deployment) must have a git tag on `master` branch as follow + +```text +YYYY-MM-DD-rcX +``` + +with `X` being an incremental number starting from `1` and where `YYYY-MM-DD` refers to the corresponding milestone (could defer from the date of the tag creation). This git tag is automated via github action when the a PR is merged into `master` (see [.github/workflows/milestone-release.yml](https://github.com/geoadmin/.github/blob/master/.github/workflows/milestone-release.yml)). + +For services that uses docker images, all milestone images are tagged with the same git tag; `YYYY-MM-DD-rcX`. For docker image +built during the milestone (PR merged in the `develop-YYYY-MM-DD` branch, they are tagged as `.latest` (e.g. `develop-2022-09-10.latest`). + +For each git tag on `master` branch a github Release is automatically generated via github action ([.github/workflows/milestone-release.yml](https://github.com/geoadmin/.github/blob/master/.github/workflows/milestone-release.yml)). This Release contains [Release Notes](#release-notes) automatically generated from the milestone PRs. + +### Milestone Workflow + +1. In the beginning of the milestone, a developer creates the milestone branch `develop-YYYY-MM-DD` on the repository based on top of `master` (can be done directly within github web interface or from CLI). The branch creation trigger the following github workflow: + + - `create-milestone.yml` this workflow does then the following actions + - Get the milestone name from the branch name; `develop-YYYY-MM-DD` => `YYYY-MM-DD` + - Create or re-open github milestone `YYYY-MM-DD` + - Set the github default branch to the next milestone branch `develop-YYYY-MM-DD`. (NOTE: this really set to the next milestone branch, so if you create the overnext milestone branch, the default branch stays on the next milestone branch and not the overnext) + - Set milestone github branch protection. This branch protection is as follow: + - Require at least one approval review + - Require CI status check to pass and branch to be up to date + - Don't allow force push + - Don't allow branch deletion + - Enforce all configured restrictions for administrators too + + ```bash + # Create the milestone branch from CLI + git checkout master + git pull # get latest state of master + git branch develop-YYYY-MM-DD + git push origin develop-YYYY-MM-DD + ``` + +2. The developer creates a branch using the correct [naming convention](GIT_FLOW.md#personal-branches-naming-convention) based on top of the milestone branch `develop-YYYY-MM-DD`. + + ```bash + git checkout develop-YYYY-MM-DD + git pull # get the latest state of milestone branch + git checkout -b data-PB-9999-layer-A + ``` + +3. Once the work is done and ready for review, the developer opens a PR to merge his changes into `develop-YYYY-MM-DD`. Opening a PR trigger the following workflows: + + - `pr-auto-milestone.yml`. This workflow does the following actions: + - Add the PR to the correct github milestone (based on the base branch name `develop-YYYY-MM-DD`) + - Set PR label based on head branch prefix; `data-` => `data-integration`, `feat-` => `feature` and `bug-` => `bug` (see [geoadmin/.github/.github/release-drafter-labeler-config.yml](https://github.com/geoadmin/.github/blob/master/.github/release-drafter-labeler-config.yml) for more info on auto labels) + + The PR label is used later on to categorize the [Release Notes](#release-notes). + + The PR title must be set accordingly to the rules in [Pull Request Rules](GIT_FLOW.md#pull-request-rules) as it will be used in the [Release Notes](#release-notes). + +4. If the PR don't require any Release Note (e.g. code cleanup, typo fix, ...), then set the following label to the PR; `skip-release-note` +5. Once the PR has been reviewed it is merged and closed. + +#### Milestone release + +Once all work for a milestone has been done (this can be checked using the github milestone view, e.g. [geoadmin/mf-chsdi3/milestones](https://github.com/geoadmin/mf-chsdi3/milestones)), a new Release can be created like follow: + +1. Create a PR to merge `develop-YYYY-MM-DD` into `master` (note the PR title will be automatically set, so you can give any dummy title during creation). The PR creation/edit/update trigger the following workflow: + + - `pr-auto-milestone.yml`, which does the following actions + - Add the PR to the correct github milestone (based on the head branch name `develop-YYYY-MM-DD`) + - Set the PR label to `new-release` + - Check that all milestone PRs (except this one) are closed. This will prevent to merge this PR as long as there are open milestone PRs. + - Set PR title to the new release, e.g. : `New Release 2022-06-29-rc1` + +2. Review the changes (quick review to make sure that everything is included). +3. Merge the PR. This trigger the following workflow + - `milestone-version.yml` with the following actions + - Tag the repository with `YYYY-MM-DD-rcX`. With `X` being `1` or incremented if the tag already exists. + - Generate github Release with Release Note based on the git tag. + - Close the github Milestone + - Delete the milestone branch protection + - Delete the milestone branch + - Set the default github branch either to the next milestone branch if it already exists otherwise to `master` +4. The [Release Notes](#release-notes) of the release must be reviewed and edited if needed. + +#### Milestone hotfix + +1. Create a personal branch on top of `master` following the [naming convention](GIT_FLOW.md#personal-branches-naming-convention). +2. Do the work +3. Open a PR to merge the work into `master`. This trigger github action to add the correct label to the PR. +4. Once the PR has been reviewed, and merged a github action is triggered to tag the repo with `YYYY-MM-DD-rcX` with `X` being incremented based on the last milestone tag. A github release is then also generated with [Release Notes](#release-notes). +5. When all milestone hotfix have been done, merge back `master` into `develop-YYYY-MM-DD` in order to update this latest branch. See [Merging `master` into `develop` or `develop-YYYY-MM-DD`](./GIT_FLOW.md#merging-master-into-develop-or-develop-yyyy-mm-dd) for solving conflicts. + +## Release Notes + +Release notes are automated by github action and written in the description of the github Release. They are taken from the PR title. If the PR title contains a Jira issue number, this number is transformed into a hyperlink to the Jira issue. + +EACH DEVELOPER IS RESPONSIBLE to write meaningful concise PR title that can be understood in the Release Notes context. PR REVIEWER(s) SHOULD also review the PR title. + +The release notes are categorized based on the PR label(s). + +| Label | Release Note Category | +|-------|-----------------------| +| feature | Features | +| bug | Bug Fixes | +| data | Data Integration | +| new-release | Ignored, not part of the release notes | +| skip-release-note | Ignored, not part of the release notes | + +## Deployment + +The latest status of `develop`/`develop-YYYY-MM-DD` (i.e. the most recent `974517877189.dkr.ecr.eu-central-1.amazonaws.com/:latest` image) can and should be deployed frequently to DEV staging. It's acceptable to either deploy `develop:latest`/`develop-YYYY-MM-DD:latest` on DEV staging or a tagged pre-release. + +For INT staging we deploy only tagged pre-releases or releases, for PROD we deploy only releases. diff --git a/assets/images/git-flow-merge.svg b/assets/images/git-flow-merge.svg new file mode 100644 index 0000000..e399f09 --- /dev/null +++ b/assets/images/git-flow-merge.svg @@ -0,0 +1,3 @@ + + +
master
master
hotfix
hotfix
develop
develop
feature-A
feature-A
feature-B
feature-B
time
time
master
master
develop
develop
GITK view
GITK view
GIT Flow
GIT FlowViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/assets/images/git-flow-mess.png b/assets/images/git-flow-mess.png new file mode 100644 index 0000000..944ddff Binary files /dev/null and b/assets/images/git-flow-mess.png differ diff --git a/assets/images/git-flow-rebase.svg b/assets/images/git-flow-rebase.svg new file mode 100644 index 0000000..3b94a21 --- /dev/null +++ b/assets/images/git-flow-rebase.svg @@ -0,0 +1,3 @@ + + +
rebase
rebase
master
master
hotfix
hotfix
develop
develop
feature-A
feature-A
feature-B
feature-B
time
time
B1
B1
B2
B2
B1'
B1'
B2'
B2'
master
master
develop
develop
GITK view
GITK view
GIT Flow
GIT FlowViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/assets/images/git-flow.svg b/assets/images/git-flow.svg new file mode 100644 index 0000000..828b1ed --- /dev/null +++ b/assets/images/git-flow.svg @@ -0,0 +1,3 @@ + + +
rebase
rebase
master
master
hotfix
hotfix
develop
develop
feature-A
feature-A
feature-B
feature-B
time
time
B1
B1
B2
B2
B1'
B1'
B2'
B2'
GIT Flow
GIT FlowViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/assets/images/github-pr-draft-ready-for-review.png b/assets/images/github-pr-draft-ready-for-review.png new file mode 100644 index 0000000..c0a28b1 Binary files /dev/null and b/assets/images/github-pr-draft-ready-for-review.png differ diff --git a/assets/images/github-pr-rfc-draft-screenshot.png b/assets/images/github-pr-rfc-draft-screenshot.png new file mode 100644 index 0000000..718bc1f Binary files /dev/null and b/assets/images/github-pr-rfc-draft-screenshot.png differ diff --git a/assets/images/github-update-branch-button.png b/assets/images/github-update-branch-button.png new file mode 100644 index 0000000..5b17776 Binary files /dev/null and b/assets/images/github-update-branch-button.png differ diff --git a/assets/images/github-update-branch-rebase-button.png b/assets/images/github-update-branch-rebase-button.png new file mode 100644 index 0000000..73936f3 Binary files /dev/null and b/assets/images/github-update-branch-rebase-button.png differ