ServiceEntry support

[stevenctl]

Description

• Generate Backends and Endpoints (CDS and EDS)
• Waypoint capture
• BackendRef to ServiceEntry kind
• BackendRef to Hostname kind
• HTTPRoute ParentRef to ServiceEntry kind
• HTTPRoute ParentRef to Hostname kind

API changes

• backendRef, for both Gateway and Waypoint, can point at networking.istio.io/Hostname using the ServiceEntry hostname to point at a ServiceEntry in any namespace as the backend.
• parentRef, only for Waypoint, can use networking.istio.io/Hostname to attach policy a Service or ServiceEntry

Code changes

• Add ServiceEntry to common collections as it's used across serviceentry CDS/EDS code and Waypoint translation code for attachment purposes.
• Add a serviceentry plugin that provides Backends, Endpoints and a backendRef resolver
• Make a few functions used by the k8s plugin re-usable

Context

#10841

Testing steps

1. Deploy kgateway
2. Install Istio
3. Deploy a gateway
4. Create a ServiceEntry and a backing pod (or do DNS egress)
5. Create an HTTPRoute that points to the SE (either via name/ns or hostname)
6. Send traffic

Unit/in-memory tests:

• Unit tests at the setup_test level that ensure EDS/CDS looks right for various configs (DNS, EDS, STATIC inline)
• Unit tests at the waypoint_translator level that ensures policy attachment and waypoint attachment works (hostname, vs name/ns references, namespace vs direct attachment)

E2E tests:

• Send traffic via ingres to SE
• Send traffic via Waypoint to SE (this should be a matrix of policies x destination types)
• Test DNS SE (need to figure out how to do so safely, probably just send to a kube svc hostname using DNS resolution) ; do this with both ingress and Waypoint

Notes for reviewers

• Improving the handling of multiple hostnames, with regards to HTTPRoute attachment, and DR attachment, should be handled in a follow up.
• Allowing DR to apply to inline endpoints for non-EDS ServiceEntry should be done in a follow up.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

[stevenctl]

Closing connection 0 curl: (28) Resolving timed out after 3000 milliseconds command terminated with exit code 28

the issue was I didn't put addresses in the spec, the VIPs. 1.25 has the feature to write auto-allocated addresses to status turned on by default. This doesn't matter for Istio impls, because Istio impls have these auto-allocated addresses in-memory in istiod. It matters for us, because we need to get that info from istiod somehow.

We must document somewhere that the status.addresses functionality must be enabled OR users must manually write addresses in their ServiceEntry spec.

https://istio.io/latest/news/releases/1.23.x/announcing-1.23/#dns-auto-allocation-improvements

[lgadban]

mostly LGTM, just a few comments/questions

[lgadban]

not suggesting to change this but more for my own understanding, do we actually need ServiceEntries here as part of the CommonCollections? i.e. will anything outside of the SE plugin need this?

[stevenctl]

The waypoint is already reading them. The alternative is to create a "logical service" that merges over Service and ServiceEntry for waypoint to utilize.

If we don't put this here, we still end up sharing informers but run the collection logic twice for waypoint (attachment to SE) and service entry (CDS/EDS)

[PetrMc]

do i need to add authorizationpolicies in collection here. My proto code works without it, but i might be missing something? Please let me know!

[stevenctl]

Authz can live in the waypointqueries initialization until we make it apply to non-waypoints. No time to get that in for 2.0.0 release though.

[shashankram]

Approved, but will defer to @lgadban /others for final approval

[stevenctl]

spoke with @lgadban offline, can address anything else in follow ups.

[lgadban]

LGTM!