Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ambient: add traffic routing docs #12781

Merged
merged 4 commits into from
Feb 27, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .spelling
Original file line number Diff line number Diff line change
Expand Up @@ -983,6 +983,7 @@ UIDs
uint32
ulimit
un-injecting
uncaptured
uncomment
uncommented
unconfigured
Expand Down
73 changes: 73 additions & 0 deletions content/en/docs/ops/ambient/architecture/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,77 @@ This page is under construction.

## Traffic routing

In ambient mode, workloads can fall into 3 categories:
* Uncaptured: this is a standard pod without any mesh features enabled.
* Captured: this is a pod that has traffic intercepted by `ztunnel`. Pods can be captured by setting the `istio.io/dataplane-mode=ambient` label on a namespace.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume there will be some intro material that describes the architectural components, but I wonder if it might be worth adding some glossary entries and reference them in the first use in this doc, to make it more easy to read alone?

Suggested change
* Captured: this is a pod that has traffic intercepted by `ztunnel`. Pods can be captured by setting the `istio.io/dataplane-mode=ambient` label on a namespace.
* Captured: this is a pod that has traffic intercepted by {{< gloss >}}ztunnel{{< /gloss >}}. Pods can be captured by setting the `istio.io/dataplane-mode=ambient` label on a namespace.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is intended to be the deep dive. Writing the docs in reverse order, but users will see this last. Agree on glossary

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ill make another PR with glossary

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also mention the redirection annotation on the pod here @howardjohn ?

* Waypoint enabled: this is a pod that is "Captured" *and* has a waypoint proxy deployed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Waypoint enabled: this is a pod that is "Captured" *and* has a waypoint proxy deployed.
* Waypoint enabled: this is a pod that is "Captured" *and* has a {{< gloss >}}waypoint proxy{{< /gloss >}} deployed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting the next line just uses waypoint, would it be better to have the single word in the glossary?

A waypoint will, by default, apply to all pods in the same namespace.
It can optionally be set to apply to only a specific service account with the `istio.io/for-service-account` annotation on the `Gateway`.
If there is both a namespace waypoint and service account waypoint, the service account waypoint takes precedence.

Depending on which category a workload is in, the request path will be different.

### Ztunnel routing

#### Outbound

When a captured pod makes an outbound request, it will be transparently redirected to `ztunnel` which will determine where and how to forward the request.
In general, the traffic routing behaves just like Kubernetes default traffic routing;
requests to a `Service` will be sent to an endpoint within the `Service` while requests directly to a `Pod` IP will go directly to that IP.

However, depending on the destination's capabilities, different behavior will occur.
If the destination is also captured, or otherwise has Istio proxy capabilities (such as a sidecar), the request will be upgraded to an encrypted HBONE tunnel.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If the destination is also captured, or otherwise has Istio proxy capabilities (such as a sidecar), the request will be upgraded to an encrypted HBONE tunnel.
If the destination is also captured, or otherwise has Istio proxy capabilities (such as a sidecar), the request will be upgraded to an encrypted {{< gloss >}}HBONE{{< /gloss >}} tunnel.

If the destination has a waypoint proxy, in addition to being upgraded to HBONE, the request will instead be forwarded to that waypoint.

Note that in the case of a request to a `Service`, a specific endpoint will be selected to determine if it has a waypoint.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to explain how the specific endpoint would be selected? Is it randomly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is an implementation detail - currently its random but we will optimize it in the future. I don't mind putting it but it will change

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, ok, SGTM not documenting it for now.

However, if it *has* a waypoint, the request will be sent with a target destination of the `Service`, not the selected endpoint.
This allows the waypoint to apply service-oriented policies to the traffic.
In the rare case that a `Service` has a mix of waypoint enabled and non-enabled endpoints, some requests would be sent to a waypoint while other requests to the same service would not.

#### Inbound

When a captured pod receives an inbound request, it will be transparently redirected to `ztunnel`.
When `ztunnel` receives the request, it will apply Authorization Policies and forward the request only if the request meets the policies.

A pod can receive HBONE traffic or plaintext traffic.
By default, both will be accepted by `ztunnel`.
Because plaintext request will have no peer identity when Authorization Policies are evaluated,
a user can set a policy requiring an identity (either *any* identity, or a specific one) to block all plaintext traffic.

When the destination is waypoint enabled, all requests *must* go through the waypoint where policy is enforced.
The `ztunnel` will make sure this occurs.
However, there is an edge case: a well behaving HBONE client (such as another `ztunnel` or Istio sidecar) would know to send to the waypoint, but other clients
(such as a workload outside of the mesh) likely would not know anything about waypoint proxies and send requests directly.
When these direct calls are made, the ztunnel will "hairpin" the request to its own waypoint to ensure policies are properly enforced.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly does "hairpin" mean? "Capture and redirect"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically

client -> ztunnel -> waypoint -> ztunnel


### Waypoint routing

A waypoint exclusively receives HBONE requests.
Upon receiving a request, the waypoint will ensure it is targeting either a `Pod` that it manages or a `Service` that contains a `Pod` it manages.

For either type of request, the waypoint will enforce policies (such as `AuthorizationPolicy`, `WasmPlugin`, `Telemetry`, etc) before forwarding.

For direct requests to `Pod`s, these are simply forwarded directly after policy is applied.

For requests to `Service`s, the waypoint will also apply routing and load balancing.
By default, a `Service` will simply route to itself, load balancing across its endpoints.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does waypoint only route to waypoint enabled endpoints?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, makes sense.

This can be overridden with Routes for that `Service`.

For example, the below policy will ensure that requests to the `echo` service are forwarded to `echo-v1`:

{{< text yaml >}}
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: echo
spec:
parentRefs:
- kind: Service
name: echo
rules:
- backendRefs:
- name: echo-v1
port: 80
{{< /text >}}

## Security