Allow customers to request CA cert (e.g. for external clients)

[sbernauer]

There is a similar Issue for Pods: #320

As a SDP user I need to get the current ca.crt so that I can put it in external clients or e.g. OpenShift routes.

### Things to watch out
- [ ] The mechanism must work wit CA cert rotation. We e.g. need to return a list of certs that are not expired yet
- [ ] The mechanism is aligned with the Discovery 2.0. The reason is that Discovery 2.0 might include the ca cert for the stacklet as well. But even *if* so, this API might give all certs (see rotation above) and the discovery only the current one. However, this is speculation as Discovery 2.0 is not there yet

Follow-ups

#597

Workaround

Until this is implemented you can use one of the following workarounds:

1. Read the ca.crt from the referenced Secret in the SecretClass. Usually it is called secret-provisioner-tls-ca and is located either in the default or stackable-operators namespace.
2. Use a Pod similar to the following

apiVersion: v1
kind: Pod
metadata:
  name: extract-ca-cert
spec:
  volumes:
    - name: tls
      ephemeral:
        volumeClaimTemplate:
          metadata:
            annotations:
              secrets.stackable.tech/class: tls
              secrets.stackable.tech/scope: pod
          spec:
            storageClassName: secrets.stackable.tech
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: "1"
  containers:
    - name: extract-ca-cert
      image: docker.stackable.tech/stackable/testing-tools:0.2.0-stackable24.3.0
      command: [bash, -c]
      args:
        - |
          cat /tls/ca.crt
          sleep infinity
      volumeMounts:
        - name: tls
          mountPath: /tls
  securityContext:
    fsGroup: 1000

[fhennig]

This needs to be documented as well (once implemented)

[nightkr]

So we have a few options to go on here...

1. Have sec-op expose it as a REST API
   • Could be either completely standalone or use k8s aggregation
2. Have sec-op write it as a ConfigMap
3. 2 but with a custom CRD instead of a ConfigMap

I'm inclined to say that a CM is the way to go here. It should be easier for outsiders to interact with, and is already supported by things like k8s mounts. It also means we don't have to interact with webhooks or worry about what happens when said webhook goes down.

If we go with the CM approach then we also have to decide who is responsible for deciding where it should go.

• 2a. Configured in the SecretClass backend
  • Transparently handles k8sSearch as well (just provide the CM yourself!)
  • Users can't assume anything about how to get a particular SecretClass's trust info
  • Simpler lifecycle
• 2b. New CRD (preliminarily named TruststoreClaim)
  • Automatically replicates to the user's namespace
    • Do we actually care about accessing this from inside k8s? Should users just use regular secret volumes for this?
  • Lets users decide formatting
  • Need to decide how to handle k8sSearch (one option is to let admins provide a fixed configmap to serve)

So far, I'm leaning towards 2b, but that is far from a final call.

[sbernauer]

I just wanted to mention that in the future we also need a mechanism to generate cert-pairs for the users.
E.g. users of mTLS secured Kafka need some way of obtaining a cert issued on "sebastian.bernauer", valid for 1 year. Or something like that. Just to keep in mind when designing this

[nightkr]

I don't think end-user authn is in scope for the secret operator. If you want to share the same PKI we should consider ways to delegate that to a different authority.

Either including that CA's root cert(s), cross-signing them, or issuing sub-CA certificates directly in some fashion.

[siegfriedweber]

I have a conceptual question regarding the k8sSearch solution.

In pods, volume mounts are used to request contents referenced by a SecretClass. If you want to "mount" the contents into a ConfigMap instead of a Pod, volume mounts cannot be used. A TrustStore object is a replacement for a mount request into a ConfigMap. The annotation secrets.stackable.tech/class in the volume claim equals the property secretClassName in the TrustStore. However, there is no equivalent for the annotation secrets.stackable.tech/scope.

Now to the big difference when using k8sSearch: In the "Pod-case", the object containing the contents references the SecretClass with the label secrets.stackable.tech/class, whereas in the "ConfigMap-case", the object containing the contents is referenced by the SecretClass with the property trustStoreConfigMapName.

Would it be possible to align these two approaches somehow?

---
apiVersion: secrets.stackable.tech/v1alpha1
kind: TrustStore
metadata:
  name: truststore-k8ssearch
spec:
  secretClassName: trust-roots
---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: trust-roots
spec:
  backend:
    k8sSearch:
      searchNamespace:
        name: default
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: root-ca
  namespace: default
  labels:
    secrets.stackable.tech/class: trust-roots
data:
  ca.crt: ...

[nightkr]

Yeah.. scope is really the core of the difference. Secret contents are specific to some the particular target (where scope decides how narrow that targeting is), but the trust information concerns the whole trust domain (≃ one SecretClass).

Every secret is (ideally) unique, but (the contents of) every TrustStore targeting the same SecretClass is identical.

[siegfriedweber]

Documentation

https://docs.stackable.tech/home/nightly/secret-operator/truststore/

[nightkr]

Release note:

Users (and services) can now use the new TrustStore resource to request the trust root certificates associated with a SecretClass.

[lfrancke]

Thank you both @siegfriedweber @nightkr for adding docs and release notes without me having to ask 💓