-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ambient: add traffic routing docs #12781
Changes from 3 commits
54f3d9c
ea5ae65
69f91af
f0b67e4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -983,6 +983,7 @@ UIDs | |
uint32 | ||
ulimit | ||
un-injecting | ||
uncaptured | ||
uncomment | ||
uncommented | ||
unconfigured | ||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -13,4 +13,77 @@ This page is under construction. | |||||
|
||||||
## Traffic routing | ||||||
|
||||||
In ambient mode, workloads can fall into 3 categories: | ||||||
* Uncaptured: this is a standard pod without any mesh features enabled. | ||||||
* Captured: this is a pod that has traffic intercepted by `ztunnel`. Pods can be captured by setting the `istio.io/dataplane-mode=ambient` label on a namespace. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we also mention the redirection annotation on the pod here @howardjohn ? |
||||||
* Waypoint enabled: this is a pod that is "Captured" *and* has a waypoint proxy deployed. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Noting the next line just uses waypoint, would it be better to have the single word in the glossary? |
||||||
A waypoint will, by default, apply to all pods in the same namespace. | ||||||
It can optionally be set to apply to only a specific service account with the `istio.io/for-service-account` annotation on the `Gateway`. | ||||||
If there is both a namespace waypoint and service account waypoint, the service account waypoint takes precedence. | ||||||
|
||||||
Depending on which category a workload is in, the request path will be different. | ||||||
|
||||||
### Ztunnel routing | ||||||
|
||||||
#### Outbound | ||||||
|
||||||
When a captured pod makes an outbound request, it will be transparently redirected to `ztunnel` which will determine where and how to forward the request. | ||||||
In general, the traffic routing behaves just like Kubernetes default traffic routing; | ||||||
requests to a `Service` will be sent to an endpoint within the `Service` while requests directly to a `Pod` IP will go directly to that IP. | ||||||
|
||||||
However, depending on the destination's capabilities, different behavior will occur. | ||||||
If the destination is also captured, or otherwise has Istio proxy capabilities (such as a sidecar), the request will be upgraded to an encrypted HBONE tunnel. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
If the destination has a waypoint proxy, in addition to being upgraded to HBONE, the request will instead be forwarded to that waypoint. | ||||||
|
||||||
Note that in the case of a request to a `Service`, a specific endpoint will be selected to determine if it has a waypoint. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible to explain how the specific endpoint would be selected? Is it randomly? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That is an implementation detail - currently its random but we will optimize it in the future. I don't mind putting it but it will change There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see, ok, SGTM not documenting it for now. |
||||||
However, if it *has* a waypoint, the request will be sent with a target destination of the `Service`, not the selected endpoint. | ||||||
This allows the waypoint to apply service-oriented policies to the traffic. | ||||||
In the rare case that a `Service` has a mix of waypoint enabled and non-enabled endpoints, some requests would be sent to a waypoint while other requests to the same service would not. | ||||||
|
||||||
#### Inbound | ||||||
|
||||||
When a captured pod receives an inbound request, it will be transparently redirected to `ztunnel`. | ||||||
When `ztunnel` receives the request, it will apply Authorization Policies and forward the request only if the request meets the policies. | ||||||
|
||||||
A pod can receive HBONE traffic or plaintext traffic. | ||||||
By default, both will be accepted by `ztunnel`. | ||||||
Because plaintext request will have no peer identity when Authorization Policies are evaluated, | ||||||
howardjohn marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
a user can set a policy requiring an identity (either *any* identity, or a specific one) to block all plaintext traffic. | ||||||
|
||||||
When the destination is waypoint enabled, all requests *must* go through the waypoint where policy is enforced. | ||||||
The `ztunnel` will make sure this occurs. | ||||||
However, there is an edge case: a well behaving HBONE client (such as another `ztunnel` or Istio sidecar) would know to send to the waypoint, but other clients | ||||||
(such as a workload outside of the mesh) likely would not know anything about waypoint proxies and send requests directly. | ||||||
When these direct calls are made, the ztunnel will "hairpin" the request to its own waypoint to ensure policies are properly enforced. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What exactly does "hairpin" mean? "Capture and redirect"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Basically client -> ztunnel -> waypoint -> ztunnel |
||||||
|
||||||
### Waypoint routing | ||||||
|
||||||
A waypoint exclusively receives HBONE requests. | ||||||
Upon receiving a request, the waypoint will ensure it is targeting either a `Pod` that it manages or a `Service` that contains a `Pod` it manages. | ||||||
|
||||||
For either type of request, the waypoint will enforce policies (such as `AuthorizationPolicy`, `WasmPlugin`, `Telemetry`, etc) before forwarding. | ||||||
|
||||||
For direct requests to `Pod`s, these are simply forwarded directly after policy is applied. | ||||||
|
||||||
For requests to `Service`s, the waypoint will also apply routing and load balancing. | ||||||
By default, a `Service` will simply route to itself, load balancing across its endpoints. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does waypoint only route to waypoint enabled endpoints? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks, makes sense. |
||||||
This can be overridden with Routes for that `Service`. | ||||||
|
||||||
For example, the below policy will ensure that requests to the `echo` service are forwarded to `echo-v1`: | ||||||
|
||||||
{{< text yaml >}} | ||||||
apiVersion: gateway.networking.k8s.io/v1beta1 | ||||||
kind: HTTPRoute | ||||||
metadata: | ||||||
name: echo | ||||||
spec: | ||||||
parentRefs: | ||||||
- kind: Service | ||||||
name: echo | ||||||
rules: | ||||||
- backendRefs: | ||||||
- name: echo-v1 | ||||||
port: 80 | ||||||
{{< /text >}} | ||||||
|
||||||
## Security |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume there will be some intro material that describes the architectural components, but I wonder if it might be worth adding some glossary entries and reference them in the first use in this doc, to make it more easy to read alone?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes this is intended to be the deep dive. Writing the docs in reverse order, but users will see this last. Agree on glossary
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ill make another PR with glossary