doc: update readme and docs to fit the current skipper project (#2729)

Signed-off-by: Sandor Szücs <[email protected]>

docs/data-clients/kubernetes.md

@@ -4,7 +4,6 @@ Skipper's Kubernetes dataclient can be used, if you want to run Skipper as
 [kubernetes-ingress-controller](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-controllers).
 It will get its route information from provisioned
 [Ingress Objects](https://kubernetes.io/docs/concepts/services-networking/ingress).
-Detailed information you find in our [godoc for dataclient kubernetes](https://godoc.org/github.com/zalando/skipper/dataclients/kubernetes).
 
 ## Kubernetes Ingress Controller deployment
 
@@ -22,32 +21,35 @@ proxy made to apply updates very often. Skipper is used in
 production with more than 200.000 routing table entries.
 Skipper has Filters to change http data and Predicates to change the
 matching rules, both can combined and chained. You can set these in
-ingress.yaml files to build resiliency patterns like ratelimit or
+ingress.yaml files to build resiliency patterns like load shedding, ratelimit or
 circuitbreaker. You can also use them to build more highlevel
 deployment patterns, for example feature toggles, shadow traffic or
 blue-green deployments.
 
 Skipper's main features:
 
 - Filters - create, update, delete all kind of HTTP data
-  - [collection of base http manipulations](https://godoc.org/github.com/zalando/skipper/filters/builtin): for example manipulating Path, Querystring, ResponseHeader, RequestHeader and redirect handling
-  - [cookie handling](https://godoc.org/github.com/zalando/skipper/filters/cookie)
-  - [circuitbreakers](https://godoc.org/github.com/zalando/skipper/filters/circuit): consecutiveBreaker or rateBreaker
-  - [ratelimit](https://godoc.org/github.com/zalando/skipper/filters/ratelimit): based on client or backend data
-  - Shadow traffic: [tee()](https://godoc.org/github.com/zalando/skipper/filters/tee)
-- Predicates - advanced matching capability
-  - URL Path match: `Path("/foo")`
-  - Host header match: `Host("^www.example.org$")`
-  - [Querystring](https://godoc.org/github.com/zalando/skipper/predicates/query): `QueryParam("featureX")`
-  - [Cookie based](https://godoc.org/github.com/zalando/skipper/predicates/cookie): `Cookie("alpha", /^enabled$/)`
-  - [source whitelist](https://godoc.org/github.com/zalando/skipper/predicates/source): `Source("1.2.3.4/24")`
-  - [time based interval](https://godoc.org/github.com/zalando/skipper/predicates/interval)
-  - [traffic by percentage](https://godoc.org/github.com/zalando/skipper/predicates/traffic) supports also sticky sessions
+   - [collection of base http manipulations](../reference/filters.md):
+     for example [manipulating Path](../reference/filters.md#http-path), [Querystring](../reference/filters.md#http-query), [HTTP Headers](../reference/filters.md#http-headers) and [redirect](../reference/filters.md#http-redirect) handling
+   - [cookie handling](../reference/filters.md#cookie-handling)
+   - [circuitbreakers](../reference/filters.md#circuit-breakers)
+   - [ratelimit](../reference/filters.md#rate-limit): based on client or backend data
+   - [Shadow traffic filters](../reference/filters.md#shadow-traffic)
+- [Predicates](../reference/predicates.md) - advanced matching capability
+   - URL [Path](../reference/predicates.md#the-path-tree) match: `Path("/foo")`
+   - [Host header](../reference/predicates.md#host) match: `Host("^www.example.org$")`
+   - [Querystring](../reference/predicates.md#queryparam): `QueryParam("featureX")`
+   - [Cookie based](../reference/predicates.md#cookie): `Cookie("alpha", /^enabled$/)`
+   - [source IP allowlist](../reference/predicates.md#source): `Source("1.2.3.4/24")` or `ClientIP("1.2.3.4/24")`
+   - [time based interval](../reference/predicates.md#interval)
+   - [traffic by percentage](../reference/predicates.md#trafficsegment) supports also sticky sessions
 - Kubernetes integration
-  - All Filters and Predicates can be used with 2 annotations
-    - Predicates: `zalando.org/skipper-predicate`
-    - Filters: `zalando.org/skipper-filter`
-  - Custom routes can be defined with the annotation `zalando.org/skipper-routes`
-  - [metrics](https://godoc.org/github.com/zalando/skipper/metrics)
-  - access logs
-  - Blue-Green deployments, with another Ingress annotation `zalando.org/backend-weights`
+   - All Filters and Predicates can be used with 2 [annotations](../kubernetes/ingress-usage.md#skipper-ingress-annotations)
+      - Predicates: `zalando.org/skipper-predicate`
+      - Filters: `zalando.org/skipper-filter`
+   - Custom routes can be defined with the annotation `zalando.org/skipper-routes`
+   - [RouteGroup CRD](../kubernetes/routegroups.md) to support all skipper features without limitation
+   - [monitoring](../operation/operation.md#monitoring)
+   - [opentracing](../operation/operation.md#opentracing)
+   - access logs with fine granular control of logs by status codes
+   - Blue-Green deployments, with another Ingress annotation `zalando.org/backend-weights`

docs/kubernetes/ingress-controller.md

@@ -323,13 +323,10 @@ run:
 kubectl create -f docs/kubernetes/deploy/deployment
 ```
 
-!!! note
-    The `extensions/v1beta1` and `networking.k8s.io/v1beta1` API versions of Ingress is no longer served as of v1.22.
-    In order to use `networking.k8s.io/v1` extension, you need to add `-kubernetes-ingress-v1` to the `skipper-ingress` container's arguments. This feature is added from skipper version v0.13.149
-
 Now, let's see what we have just deployed.
 This will create serviceaccount, PodSecurityPolicy and RBAC rules such that
-skipper-ingress is allowed to listen on the hostnetwork.
+skipper-ingress is allowed to listen on the hostnetwork and poll
+ingress resources.
 
 ```yaml
 # cat docs/kubernetes/deploy/deployment/rbac.yaml
@@ -495,7 +492,7 @@ The source code is available at [GitHub](https://github.com/baez90/skipper-helm)
 The chart includes resource definitions for the following use cases:
 
 - RBAC
-- CoreOS [Prometheus-Operator](https://github.com/coreos/prometheus-operator)
+- [Prometheus-Operator](https://github.com/prometheus-operator/prometheus-operator)
 
 As this chart is not maintained by the Skipper developers and is still under development only the basic deployment workflow is covered here.
 Check the GitHub repository for all details.
@@ -641,7 +638,7 @@ line option `-enable-swarm` and `-enable-ratelimits`.
 The rest depends on the implementation, that can be:
 
 - [Redis](https://redis.io)
-- [SWIM](https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf)
+- alpha version: [SWIM](https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf)
 
 ### Redis based
 
@@ -755,6 +752,13 @@ the communication work with TCP and UDP to the specified `swarm-port`:
 
 ## Upgrades
 
+Please always read the announcements of the vX.Y.**0**
+[release page](https://github.com/zalando/skipper/releases/tag/v0.18.0),
+because these will document in case we break something in a backwards non
+compatible way. Most of the time it will be safe to deploy minor
+version updates, but better to know in advance if something could
+break.
+
 ### <v0.14.0 to >=v0.14.0
 
 Kubernetes dataclient removes support for ingress v1beta1.

readme.md

@@ -31,19 +31,26 @@ Skipper
 - simultaneously streams incoming requests and backend responses
 - optionally acts as a final endpoint (shunt), e.g. as a static file server or a mock backend for diagnostics
 - updates routing rules without downtime, while supporting multiple types of data sources — including
-  [etcd](https://github.com/coreos/etcd), [Kubernetes Ingress](https://opensource.zalando.com/skipper/data-clients/kubernetes/), [Innkeeper (deprecated)](https://github.com/zalando/innkeeper), [static files](https://opensource.zalando.com/skipper/data-clients/eskip-file/), [route string](https://opensource.zalando.com/skipper/data-clients/route-string/) and
-  [custom configuration sources](https://godoc.org/github.com/zalando/skipper/predicates/#source)
+  [etcd](https://github.com/coreos/etcd), [Kubernetes Ingress](https://opensource.zalando.com/skipper/data-clients/kubernetes/), [static files](https://opensource.zalando.com/skipper/data-clients/eskip-file/), [route string](https://opensource.zalando.com/skipper/data-clients/route-string/) and
+  [custom configuration sources](https://opensource.zalando.com/skipper/tutorials/development/#dataclients)
 - can serve as a
   [Kubernetes Ingress controller](https://zalando.github.io/skipper/data-clients/kubernetes/)
   without reloads. You can use it in combination with a controller that will route public traffic to
   your skipper fleet; [see AWS example](https://github.com/zalando-incubator/kube-ingress-aws-controller)
-- shipped with eskip: a descriptive configuration language designed for routing rules
+- shipped with
+   - eskip: a descriptive configuration language designed for routing
+     rules
+   - routesrv: proxy to omit kube-apiserver overload leveraging Etag
+     header to reduce amount of CPU used in your skipper data plane
+   - webhook: Kubernetes validation webhook to make sure your
+     manifests are deployed safely
 
 Skipper provides a default executable command with a few built-in filters. However, its primary use case is to
 be extended with custom filters, predicates or data sources. [Go here for additional documentation](https://pkg.go.dev/github.com/zalando/skipper).
 
 A few examples for extending Skipper:
 
+- Example proxy with custom filter https://github.com/szuecs/skipper-example-proxy
 - Image server https://github.com/zalando-stups/skrop
 - Plugins repository https://github.com/skipper-plugins/, [plugin docs](https://opensource.zalando.com/skipper/reference/plugins/)
 
@@ -125,8 +132,8 @@ Skipper will then be available on http://localhost:9090
 #### Authentication Proxy
 
 Skipper can be used as an authentication proxy, to check incoming
-requests with Basic auth or an OAuth2 provider including audit
-logging. See the documentation at:
+requests with Basic auth or an OAuth2 provider or an OpenID Connect
+provider including audit logging. See the documentation at:
 [https://pkg.go.dev/github.com/zalando/skipper/filters/auth](https://pkg.go.dev/github.com/zalando/skipper/filters/auth).
 
 
@@ -141,6 +148,7 @@ Build and test all packages:
 
     make deps
     make install
+    make lint
     make shortcheck
 
 > On Mac the tests may fail because of low max open file limit. Please make sure you have correct limits setup
@@ -167,8 +175,9 @@ you will can be found in our [ingress-controller deployment docs](https://openso
 
 For AWS integration, we provide an ingress controller
 https://github.com/zalando-incubator/kube-ingress-aws-controller, that
-manage ALBs in front of your skipper deployment.
-A [production example](https://github.com/zalando-incubator/kubernetes-on-aws/blob/stable/cluster/manifests/skipper/),
+manage ALBs or NLBs in front of your skipper deployment.
+A [production example for skipper](https://github.com/zalando-incubator/kubernetes-on-aws/blob/stable/cluster/manifests/skipper/)
+and a [production example for kube-ingress-aws-controller](https://github.com/zalando-incubator/kubernetes-on-aws/tree/dev/cluster/manifests/ingress-controller/),
 can be found in our Kubernetes configuration https://github.com/zalando-incubator/kubernetes-on-aws.
 
 - [Comparison with other Ingress controllers](https://opensource.zalando.com/skipper/kubernetes/ingress-controller/#comparison-with-other-ingress-controllers)
@@ -182,20 +191,25 @@ includes information about [deployment use cases](https://opensource.zalando.com
 and detailed information on these topics:
 
 - The [Routing](https://pkg.go.dev/github.com/zalando/skipper/routing) Mechanism
-- Matching Requests
+- [Matching Requests](https://opensource.zalando.com/skipper/tutorials/basics/#route-matching)
 - [Filters](https://opensource.zalando.com/skipper/reference/filters/) - Augmenting Requests and Responses
-- Service Backends
-- Route Definitions
-- Data Sources: [eskip file](https://pkg.go.dev/github.com/zalando/skipper/eskipfile), [etcd](https://pkg.go.dev/github.com/zalando/skipper/etcd), [Kubernetes](https://pkg.go.dev/github.com/zalando/skipper/dataclients/kubernetes), [Route string](https://pkg.go.dev/github.com/zalando/skipper/dataclients/routestring)
-- [Circuit Breakers](https://pkg.go.dev/github.com/zalando/skipper/filters/circuit)
-- Extending It with Customized [Predicates](https://opensource.zalando.com/skipper/reference/predicates/), [Filters](https://opensource.zalando.com/skipper/reference/filters/), can be done by [Plugins](https://opensource.zalando.com/skipper/reference/plugins/) or [Lua Scripts](https://opensource.zalando.com/skipper/reference/scripts/)
 - [Predicates](https://opensource.zalando.com/skipper/reference/predicates/) - additional predicates to match a route
-- [Proxy Packages](https://pkg.go.dev/github.com/zalando/skipper/proxy)
+- Service [Backends](https://opensource.zalando.com/skipper/reference/backends/)
+- Route Definitions fetched by dataclients:
+   - [route string](https://opensource.zalando.com/skipper/data-clients/route-string/)
+   - [eskip file](https://opensource.zalando.com/skipper/data-clients/eskip-file/)
+   - [remote eskip](https://opensource.zalando.com/skipper/data-clients/eskip-remote/)
+   - [etcd](https://opensource.zalando.com/skipper/data-clients/etcd/)
+   - [kubernetes](https://opensource.zalando.com/skipper/data-clients/kubernetes/)
+- [Circuit Breakers](https://pkg.go.dev/github.com/zalando/skipper/filters/circuit)
+- Extending It with Custom [Predicates](https://opensource.zalando.com/skipper/tutorials/development/#predicates), [Filters](https://opensource.zalando.com/skipper/tutorials/development/#filters), can be done by [building your own proxy](https://opensource.zalando.com/skipper/tutorials/built-your-own/), [Plugins](https://opensource.zalando.com/skipper/reference/plugins/) or [Lua Scripts](https://opensource.zalando.com/skipper/reference/scripts/)
+- [Proxy Package](https://pkg.go.dev/github.com/zalando/skipper/proxy)
 - [Logging](https://pkg.go.dev/github.com/zalando/skipper/logging) and [Metrics](https://pkg.go.dev/github.com/zalando/skipper/metrics)
-- Performance Considerations
+- [Operations guide](https://opensource.zalando.com/skipper/operation/operation/)
+- [Authentication and Authorization](https://opensource.zalando.com/skipper/reference/filters/#authentication-and-authorization)
+- [Load Shedders](https://opensource.zalando.com/skipper/reference/filters/#load-shedding)
 - [Rate Limiters](https://pkg.go.dev/github.com/zalando/skipper/filters/ratelimit)
-- [Opentracing plugin](https://github.com/skipper-plugins/opentracing/) or extend [create your own](https://opensource.zalando.com/skipper/reference/plugins/#opentracing-plugins)
-- [WAF plugin](https://github.com/jptosso/coraza-waf) - Web Application Firewall filter for Skipper routes
+- [Opentracing tracers](https://pkg.go.dev/github.com/zalando/skipper/tracing/tracers) or extend [create your own](https://opensource.zalando.com/skipper/reference/plugins/#opentracing-plugins)
 
 #### 1 Minute Skipper introduction
 
@@ -222,9 +236,9 @@ The following example shows a skipper routes file in eskip format, that has 3 na
 
 Matching the route:
 
-- baidu is using Path() matching to differentiate the HTTP requests to select the route.
-- google is the default matching with wildcard '*'
-- yandex is the default matching with wildcard '*' if you have a cookie "yandex=true"
+- baidu is using `Path()` matching to differentiate the HTTP requests to select the route.
+- google is the default matching with wildcard `*`
+- yandex is the default matching with wildcard `*` if you have a cookie `yandex=true`
 
 Request Filters:
 
@@ -242,7 +256,7 @@ To test each route you can use curl:
     % curl -v localhost:9090/
     % curl -v --cookie "yandex=true" localhost:9090/
 
-To see the request that is made by the tee() filter you can use nc:
+To see the shadow traffic request that is made by the `tee()` filter you can use nc:
 
     [terminal1]% nc -l 12345
     [terminal2]% curl -v --cookie "yandex=true" localhost:9090/
@@ -253,33 +267,34 @@ This introduction was [moved to ingress controller documentation](https://openso
 
 For More details, please check out our [Kubernetes ingress controller docs](https://opensource.zalando.com/skipper/kubernetes/ingress-controller/), our [ingress usage](https://opensource.zalando.com/skipper/kubernetes/ingress-usage/) and how to handle [common backend problems in Kubernetes](https://opensource.zalando.com/skipper/kubernetes/ingress-backends/).
 
-You should have a base understanding of [Kubernetes](https://kubernetes.io) and
-[Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/).
-
 ### Packaging support
 
 See https://github.com/zalando/skipper/blob/master/packaging/readme.md
 
-Skipper uses Go modules, so you might need to add `GO111MODULE=on` in
-your custom build process.
-
 In case you want to implement and link your own modules into your
-skipper, there is https://github.com/skipper-plugins organization to enable you to do
-so. In order to explain you the build process with custom Go modules
-there is https://github.com/skipper-plugins/skipper-tracing-build,
-that was used to build skipper's [opentracing
-package](https://github.com/skipper-plugins/opentracing).
-We moved the opentracing plugin source into the `tracing` package.
-
+skipper, there is https://github.com/skipper-plugins organization to
+enable you to do so. In order to explain you the build process with
+custom Go modules there is
+https://github.com/skipper-plugins/skipper-tracing-build, that was
+used to build skipper's [opentracing package](https://github.com/skipper-plugins/opentracing).
+We moved the opentracing plugin source into the `tracing` package, so
+there is no need to use plugins for this case.
+
+Because Go plugins are not very well supported by Go itself we do not
+recommend to use plugins, but you can extend skipper and
+[build your own proxy](https://opensource.zalando.com/skipper/tutorials/built-your-own/).
 
 ## Community
 
 User or developer questions can be asked in our [public Google Group](https://groups.google.com/forum/#!forum/skipper-router)
 
-We also have a slack channel #skipper in gophers.slack.com. Get an [invite](https://invite.slack.golangbridge.org).
+We have a slack channel #skipper in gophers.slack.com. Get an [invite](https://invite.slack.golangbridge.org).
 If for some reason this link doesn't work, you can find more information about
 the gophers communities [here](https://github.com/gobridge/about-us/blob/master/README.md#onlineoffline-communities).
 
+The preferred communication channel is the slack channel, because the google group is a manual process to add members.
+Feel also free to [create an issue](https://github.com/zalando/skipper/issues/new/choose), if you dislike chat and post your questions there.
+
 ### Proposals
 
 We do our proposals open in [Skipper's Google drive](https://drive.google.com/drive/folders/0B9LwJMF9koB-ZEk4bEhZal9uOWM).
@@ -289,11 +304,11 @@ bigger change we will invite you to a document, such that we can work together.
 
 ### Users
 
-Zalando uses this project as shop frontend http router with 350000 routes, as
-Kubernetes ingress controller and runs several custom skipper
-instances that use skipper as library.
+Zalando used this project as shop frontend http router with 350000 routes.
+We use it as Kubernetes ingress controller in more than 100 production clusters. With every day traffic between 500k and 7M RPS serving 15000 ingress and 3750 RouteGroups at less than ¢5/1M requests.
+We also run several custom skipper instances that use skipper as library.
 
-Sergio Ballesteros from [spotahome](https://www.spotahome.com/)
+Sergio Ballesteros from [spotahome](https://www.spotahome.com/) said 2018:
 > We also ran tests with several ingress controllers and skipper gave us the more reliable results. Currently we are running skipper since almost 2 years with like 20K Ingress rules.
 > The fact that skipper is written in go let us understand the code, add features and fix bugs since all of our infra stack is golang.