From 11cf8a7afb3646d804e8969731893d10abd4f54b Mon Sep 17 00:00:00 2001 From: Jerry Ng Date: Wed, 10 May 2023 08:49:02 +0800 Subject: [PATCH 1/6] docs: fix fork me on github ribbon --- docs/layouts/partials/custom-footer.html | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/layouts/partials/custom-footer.html b/docs/layouts/partials/custom-footer.html index 1db3a39f5..e22f918fc 100644 --- a/docs/layouts/partials/custom-footer.html +++ b/docs/layouts/partials/custom-footer.html @@ -4,5 +4,14 @@ --> - Fork me on GitHub + Fork me on GitHub From d9e9b9a1fda7a3eb995dd4505f433f3938909d25 Mon Sep 17 00:00:00 2001 From: Jerry Ng Date: Wed, 10 May 2023 08:54:00 +0800 Subject: [PATCH 2/6] docs: fix missing leading slash in url --- docs/content/install/using-docker.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/install/using-docker.md b/docs/content/install/using-docker.md index efb4658f7..af574c3c6 100644 --- a/docs/content/install/using-docker.md +++ b/docs/content/install/using-docker.md @@ -4,7 +4,7 @@ description: Information about Athens' Docker images weight: 1 --- -Whether setting Athens up using [Kubernetes](install/install-on-kubernetes/) or using the [Walkthrough](/Walkthrough), you'll most likely be using one of the images that the Athens project produces. This document details what images are available, and has a recap from the Walkthrough of how to use them on their own. +Whether setting Athens up using [Kubernetes](/install/install-on-kubernetes/) or using the [Walkthrough](/Walkthrough), you'll most likely be using one of the images that the Athens project produces. This document details what images are available, and has a recap from the Walkthrough of how to use them on their own. --- From f33874ad814edc5228225b7e25e0bef879c795f3 Mon Sep 17 00:00:00 2001 From: Jerry Ng Date: Wed, 10 May 2023 09:12:17 +0800 Subject: [PATCH 3/6] docs: update helm cli installation istruction --- docs/content/install/install-on-kubernetes.md | 75 +++++++------------ 1 file changed, 28 insertions(+), 47 deletions(-) diff --git a/docs/content/install/install-on-kubernetes.md b/docs/content/install/install-on-kubernetes.md index 992265af3..8cf712643 100644 --- a/docs/content/install/install-on-kubernetes.md +++ b/docs/content/install/install-on-kubernetes.md @@ -6,9 +6,9 @@ weight: 1 When you follow the instructions in the [Walkthrough](/walkthrough), you end up with an Athens Proxy that uses in-memory storage. This is only suitable for trying out the Athens proxy for a short period of time, as you will quickly run out of memory and Athens won't persist modules between restarts. In order to run a more production-like proxy, you may want to run Athens on a [Kubernetes](https://kubernetes.io/) cluster. To aid in deployment of the Athens proxy on Kubernetes, a [Helm](https://www.helm.sh/) chart has been provided. This guide will walk you through installing Athens on a Kubernetes cluster using Helm. -* [Prerequisites](#prerequisites) -* [Configure Helm](#configure-helm) -* [Deploy Athens](#deploy-athens) +- [Prerequisites](#prerequisites) +- [Configure Helm](#configure-helm) +- [Deploy Athens](#deploy-athens) --- @@ -16,35 +16,16 @@ When you follow the instructions in the [Walkthrough](/walkthrough), you end up In order to install Athens on your Kubernetes cluster, there are a few prerequisites that you must satisfy. If you already have completed the following steps, please continue to [configuring helm](#configure-helm). This guide assumes you have already created a Kubernetes cluster. -* Install the [Kubernetes CLI](#install-the-kubernetes-cli). -* Install the [Helm CLI](#install-the-helm-cli). +- Install the [Kubernetes CLI](#install-the-kubernetes-cli). +- Install the [Helm CLI](#install-the-helm-cli). ### Install the Kubernetes CLI -In order to interact with your Kubernetes Cluster, you will need to [install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/). +To work with your Kubernetes Cluster, it's necessary to have the Kubernetes CLI, also known as `kubectl`, installed on your computer. You can download and install it by following the instructions provided in the [official Kubernetes documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/). -### Install The Helm CLI +### Install the Helm CLI -[Helm](https://github.com/kubernetes/helm) is a tool for installing pre-configured applications on Kubernetes. -Install `helm` by running the following command: - -#### MacOS - -```console -brew install kubernetes-helm -``` - -#### Windows - -1. Download the latest [Helm release](https://storage.googleapis.com/kubernetes-helm/helm-v2.7.2-windows-amd64.tar.gz). -1. Decompress the tar file. -1. Copy **helm.exe** to a directory on your PATH. - -#### Linux - -```console -curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash -``` +[Helm](https://github.com/helm/helm) is a tool for installing pre-configured applications on Kubernetes. You can download and install `helm` by following the instructions available on the [official Helm documentation](https://helm.sh/docs/intro/install/). ## Configure Helm @@ -88,7 +69,7 @@ $ helm repo update Next, install the chart with default values to `athens` namespace: -``` +```console $ helm install gomods/athens-proxy -n athens --namespace athens ``` @@ -133,10 +114,10 @@ When using the `disk` storage provider, you can configure a number of options re ```yaml persistence: - enabled: false - accessMode: ReadWriteOnce - size: 4Gi - storageClass: + enabled: false + accessMode: ReadWriteOnce + size: 4Gi + storageClass: ``` Add it to `override-values.yaml` file and run: @@ -151,7 +132,7 @@ helm install gomods/athens-proxy -n athens --namespace athens -f override-values To use the Mongo DB storage provider, you will first need a MongoDB instance. Once you have deployed MongoDB, you can configure Athens using the connection string via `storage.mongo.url`. You will also need to set `storage.type` to "mongo". -``` +```console helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=mongo --set storage.mongo.url= ``` @@ -162,7 +143,7 @@ S3 bucket name, respectively. By default, Athens will attempt to load AWS creden variables, shared credentials files, and EC2 instance credentials. To manually specify AWS credentials, set `storage.s3.access_key_id`, `storage.s3.secret_access_key`, and change `storage.s3.useDefaultConfiguration` to `false`. -``` +```console helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=s3 --set storage.s3.region= --set storage.s3.bucket= ``` @@ -173,21 +154,21 @@ This URL can also be an kubernetes-internal one (e.g. something like `minio-serv You need to create a bucket inside your minio-installation or use an existing one. The bucket needs to be referenced in `storage.minio.bucket`. Last athens need authentication credentials for your minio in `storage.minio.accessKey` and `storage.minio.secretKey`. -``` +```console helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=minio --set storage.minio.endpoint= --set storage.minio.bucket= --set storage.minio.accessKey= --set storage.minio.secretKey= ``` #### Google Cloud Storage - To use Google Cloud Storage storage with Athens, set `storage.type` to `gcp`. You need to set `storage.gcp.projectID` and `storage.gcp.bucket` to the - desired GCP project and bucket name, respectively. +To use Google Cloud Storage storage with Athens, set `storage.type` to `gcp`. You need to set `storage.gcp.projectID` and `storage.gcp.bucket` to the +desired GCP project and bucket name, respectively. - Depending on your deployment environment you will also need to set `storage.gcp.serviceAccount` to a key which has read/write access to - the GCS bucket. If you are running Athens inside GCP, you will most likely not need this as GCP figures out internal authentication between products for you. +Depending on your deployment environment you will also need to set `storage.gcp.serviceAccount` to a key which has read/write access to +the GCS bucket. If you are running Athens inside GCP, you will most likely not need this as GCP figures out internal authentication between products for you. - ``` - helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=gcp --set storage.gcp.projectID= --set storage.gcp.bucket= - ``` +```console +helm install gomods/athens-proxy -n athens --namespace athens --set storage.type=gcp --set storage.gcp.projectID= --set storage.gcp.bucket= +``` ### Kubernetes Service @@ -237,16 +218,16 @@ You can set the `URL` for the [upstream module repository](https://docs.gomods.i You have a few good options for what you can set as an upstream: -- `https://gocenter.io` to use JFrog's GoCenter -- `https://proxy.golang.org` to use the Go Module Mirror -- The URL to any other Athens server +- `https://gocenter.io` to use JFrog's GoCenter +- `https://proxy.golang.org` to use the Go Module Mirror +- The URL to any other Athens server The example below shows you how to set GoCenter up as upstream module repository: ```yaml upstreamProxy: - enabled: true - url: "https://gocenter.io" + enabled: true + url: "https://gocenter.io" ``` Add it to `override-values.yaml` file and run: From 20d3b753286c991c5eb69276a80a97079bd79e08 Mon Sep 17 00:00:00 2001 From: Jerry Ng Date: Wed, 10 May 2023 09:44:34 +0800 Subject: [PATCH 4/6] docs: add docs about network mode --- docs/content/configuration/download.md | 2 +- docs/content/configuration/filter.md | 2 +- docs/content/configuration/network.md | 19 +++++++++++++++++++ 3 files changed, 21 insertions(+), 2 deletions(-) create mode 100644 docs/content/configuration/network.md diff --git a/docs/content/configuration/download.md b/docs/content/configuration/download.md index 8f4fb17de..3a8355bbc 100644 --- a/docs/content/configuration/download.md +++ b/docs/content/configuration/download.md @@ -59,7 +59,7 @@ download "github.com/pkg/*" { The first two lines describe the _default_ behavior for all modules. This behavior is overridden for select module groups below. In this case, the default behavior is: -- Immediatley redirect all requests to `https://proxy.golang.org` +- Immediately redirect all requests to `https://proxy.golang.org` - In the background, download the module from the version control system (VCS) and store it The rest of the file contains `download` blocks. These override the default behavior for specific groups of modules. diff --git a/docs/content/configuration/filter.md b/docs/content/configuration/filter.md index 456784487..b50515153 100644 --- a/docs/content/configuration/filter.md +++ b/docs/content/configuration/filter.md @@ -1,7 +1,7 @@ --- title: Filtering modules (deprecated) description: Configuring modules that are stored on the proxy -weight: 6 +weight: 7 --- >Note: the filter file that this page documents is deprecated. Please instead see ["Filtering with the download mode file"](/configuration/download) for updated instructions on how to filter modules in Athens. diff --git a/docs/content/configuration/network.md b/docs/content/configuration/network.md new file mode 100644 index 000000000..a703bf531 --- /dev/null +++ b/docs/content/configuration/network.md @@ -0,0 +1,19 @@ +--- +title: Network mode +description: Configures how Athens will return the results of the /list endpoint +weight: 6 +--- + +The NetworkMode configuration in Athens determines how results are returned by the `/list` endpoint. This endpoint can retrieve information from both Athens' own [storage](/configuration/storage) and the upstream version control system (VCS). + +> Note: The NetworkMode configuration can also affect the behavior of other endpoints by improving error messaging + +## Network mode keywords + +There are 3 modes available for the NetworkMode configuration. To configure the `NetworkMode` settings at `config.dev.toml`, set the `NetworkMode` to 1 of the 3 available modes: + +1. `strict`: In this mode, Athens will merge versions from the VCS and storage, but will fail if either of them fails. This mode provides the most consistent results. + +2. `offline`: This mode only retrieves versions from Athens' storage and never reaches out to the VCS. + +3. `fallback`: This mode retrieves versions from Athens' storage only if the VCS fails. Note that using this mode may result in inconsistent results since fallback mode does its best to provide the available versions at the time of the request. From 187b98905fb6b2460cfd940b40722234cb98c91b Mon Sep 17 00:00:00 2001 From: Jerry Ng Date: Wed, 17 May 2023 19:36:14 +0800 Subject: [PATCH 5/6] docs: add use case for offline network mode --- docs/content/configuration/network.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/content/configuration/network.md b/docs/content/configuration/network.md index a703bf531..c61ebde14 100644 --- a/docs/content/configuration/network.md +++ b/docs/content/configuration/network.md @@ -17,3 +17,9 @@ There are 3 modes available for the NetworkMode configuration. To configure the 2. `offline`: This mode only retrieves versions from Athens' storage and never reaches out to the VCS. 3. `fallback`: This mode retrieves versions from Athens' storage only if the VCS fails. Note that using this mode may result in inconsistent results since fallback mode does its best to provide the available versions at the time of the request. + +## Use cases + +### Fetching modules for offline environments + +When working in offline environments with private networks lacking direct internet access, Athens' offline mode can be useful. For example, you can [pre-download](/configuration/prefill-disk-cache/) modules using Athens from a machine with internet access. Subsequently, the pre-downloaded modules are accessible within the offline network through Athens, facilitating development and build without requiring an active internet connection. From e0a7805b4484c0607cf91389882ee203b815b956 Mon Sep 17 00:00:00 2001 From: Jerry Ng Date: Wed, 17 May 2023 19:52:34 +0800 Subject: [PATCH 6/6] docs: add use cases for strict and fallback network modes --- docs/content/configuration/network.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/content/configuration/network.md b/docs/content/configuration/network.md index c61ebde14..f1294549f 100644 --- a/docs/content/configuration/network.md +++ b/docs/content/configuration/network.md @@ -12,7 +12,7 @@ The NetworkMode configuration in Athens determines how results are returned by t There are 3 modes available for the NetworkMode configuration. To configure the `NetworkMode` settings at `config.dev.toml`, set the `NetworkMode` to 1 of the 3 available modes: -1. `strict`: In this mode, Athens will merge versions from the VCS and storage, but will fail if either of them fails. This mode provides the most consistent results. +1. `strict`: In this mode, Athens will merge versions from the VCS and storage but will fail if either of them fails. This mode provides the most consistent results. 2. `offline`: This mode only retrieves versions from Athens' storage and never reaches out to the VCS. @@ -20,6 +20,22 @@ There are 3 modes available for the NetworkMode configuration. To configure the ## Use cases +### Ensuring consistency in module version retrieval + +When working in environments that prioritize consistency and reliability, configuring Athens with the `strict` mode guarantees a dependable and predictable module version resolution mechanism. + +Using the `strict` mode, Athens merges module versions from its storage and the VCS, while ensuring that any failure in either source results in a failure response. + +Choose Athens' `strict` network mode for a reliable and consistent approach to module version retrieval. + ### Fetching modules for offline environments -When working in offline environments with private networks lacking direct internet access, Athens' offline mode can be useful. For example, you can [pre-download](/configuration/prefill-disk-cache/) modules using Athens from a machine with internet access. Subsequently, the pre-downloaded modules are accessible within the offline network through Athens, facilitating development and build without requiring an active internet connection. +When working in offline environments with private networks lacking direct internet access, Athens' `offline` mode can be useful. + +For example, you can [pre-download](/configuration/prefill-disk-cache/) modules using Athens from a machine with internet access. Subsequently, the pre-downloaded modules are accessible within the offline network through Athens, facilitating development and builds without requiring an active internet connection. + +### Ensuring availability of modules versions + +In certain situations, the stability or availability of the VCS may vary. To mitigate potential disruptions, Athens offers the `fallback` mode. If the VCS encounters issues or fails to respond, Athens falls back to serving modules from its storage to ensure continued availability. + +However, using the `fallback` mode may result in inconsistent results. Athens will provide the available versions at the time of the request, which may differ from the latest versions in the VCS.