How can I import my own certificates?

[eyanez111]

Hello Community,

    I am trying to add to heartbeat (all our infra is running in kubernetes) all our pods that have certs so we can monitor those via https and alert when is time to renew them. To access those via http I need to import certs to the hearbet yaml config. I saw that on the elastic documentation:

https://www.elastic.co/guide/en/beats/heartbeat/7.16/monitor-http-options.html#monitor-http-tls-ssl

• type: http
  id: my-http-service
  name: My HTTP Service
  hosts: ["https://myhost:443"]
  schedule: '@every 5s'
  ssl:
  certificate_authorities: ['/etc/ca.crt']
  supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

What I do not find is how to add those certs to the pod? I have the secrets of the certs but I am not sure if I should create a secret for heatbeat? or if I should modify the existing one that I see in the kube-system:

heartbeat-token-lk8sl kubernetes.io/service-account-token 3 2d1h

if I have to create one do you have a process on how by creation it will be called by hearbeat? do I have to use any apiVersion? and if it is done by modifying the heartbeat token do I need to replace the ca.crt that I see there?

Thanks
eyanez111

[DanRoscigno]

Hi @eyanez111

In your example config above you are saying that the cert is at /etc/ca.crt, so you would mount the secret as a volume at /etc/ca.crt. Take a look at this example, the person who needed the cert was using it to connect to Elasticsearch, but it does not matter. Just create the secret, create the volume mount, and create the volume. I have not added a cert to Heartbeat this way, but I have done this for other software.

https://stackoverflow.com/questions/48757833/how-to-add-self-signed-certificate-in-kubernetes-elastic-search-deployment-using

[eyanez111]

Thanks for the answer. I really appreciate it.

   So this is the example provided by Elastic:

ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

I assume then that based on the link you shared if I already have the cert on an existing secret that the website uses all what I have to do is:

ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

volumeMounts:
- name: cert
  mountPath: /etc/ssl/certs
...

volumes:
- name: cert
  secret:
  secretName: **cert-from-the-website-I-want-to-monitor**
  defaultMode: 0400

or I do not need the part:
ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

Thanks again

[DanRoscigno]

I think you have it mostly correct, just change this piece:

volumeMounts:
- name: cert
  mountPath: /etc/ssl/certs

to

volumeMounts:
- name: cert
  mountPath: /etc/ca.crt
  subPath: config.json
  readOnly: true

If it does not work, then have a look at the logs and shell into the container and have a look at the file mounted at /etc/ca.crt. It has been a couple years since I worked on Beats in k8s.

[DanRoscigno]

There is also an example that I wrote years ago that shows how to create a configmap that contains the cert and mount it. The cert in the example is the cert to connect to Elasticsearch, so that is not what you want (I think in your use case you want to give Heartbeat a cert to connect to a service that you are managing, correct?), but the technique is the same.

[eyanez111]

Hello @DanRoscigno,

     Sorry it did not work I did:

ssl:
certificate_authorities: ['/etc/ca.crt']
supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

volumeMounts:
- name: cert
  mountPath: /etc/ca.crt
  subPath: config.json
  readOnly: true
...

volumes:
- name: cert
  secret:
  secretName: **cert-from-the-website-I-want-to-monitor**
  defaultMode: 0400

 What goes on "subPath: config.json" do I need to leave it like that or should I actually put a path?

[DanRoscigno]

the name of the file that you want is subPath (ca.crt). I find it confusing too.

[eyanez111]

thanks for answering so fast. So it should be like:

volumeMounts:

• name: cert
  mountPath: /etc/ca.crt
  subPath: config.json ---> name of the path for the ca.crt?
  readOnly: true

  but how do I point at any path if the cert I have it on a secret?

  thanks

[DanRoscigno]

three steps:

1. Create secret

2. Make a volume named cert

3. Mount the volume certin avolumeMountalso namedcert. This third step is where you specify the filename, by specifying both the mountPathandsubPath. The mountPathincludes the dir name and filename, and thesubPath` is just the filename.

[eyanez111]

One question. I do have the secret but is in a different namespace do I have to copy that secret and put it on the same namespace that I am running it? I think that is the main thing why it is failing.

as per what you said then I think is like this right

    ssl:
    certificate_authorities: ['/etc/**SECRETNAME**']
    supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

    volumeMounts:
    - name: cert
      mountPath: /etc/**SECRETNAME**
      subPath: **SECRETNAME**
      readOnly: true

    volumes:
    - name: cert
      secret:
      secretName: **SECRETNAME**
      defaultMode: 0400

Thanks a lot for all the help

[DanRoscigno]

My pleasure!

The secret definitely needs to be in the same namespace unless your pod is running with admin level.

kubectl describe pod and kubectl exec -it demo-pod -- /bin/sh will help you debug it. Open a shell into the pod and look at /etc/ca.crt to see if it contains the right thing, and describe will tell you about the secret and volume.

[eyanez111]

ah sorry I just talked to DEV and looks like we do not have a hold of the cert. We only have the tls.cert. So there is not a CA involved. On the secret I just have the tls.cert and the tls.key. Is it possible to do it like that:

ssl:
    certificate_authorities: ['/etc/tls.crt']
    supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]

    volumeMounts:
    - name: cert
      mountPath: /etc/tls.crt
      subPath: tls.crt
      readOnly: true

    volumes:
    - name: cert
      secret: kubernetes.io/tls
      secretName: CERT-NAME
      defaultMode: 0400

again deeply thank you as not even Elastic support has been able to point me the right direction.

[DanRoscigno]

I would get it working without k8s first, then work on k8s.

[eyanez111]

sorry but I just have a k8s environment.

[eyanez111]

ah, I was able to get the ca after talking to devs and IT. So I was ready to proceed but after:

1. Moving the secret (I added the CA)
2. Going create the volumes:

        volumeMounts:
        - name: cert
          mountPath: /etc/tls.cert
          readOnly: true
          subPath: tls.crt
      volumes:
      - name: cert
        secret: kubernetes.io/tls
        secretName: NAME
        defaultMode: 0400

This is what I get:
error: error validating "heartbeat-kubernetes-prod-rancher.yaml": error validating data: [ValidationError(Deployment.spec.template.spec.volumes[0]): unknown field "defaultMode" in io.k8s.api.core.v1.Volume, ValidationError(Deployment.spec.template.spec.volumes[0].secret): invalid type for io.k8s.api.core.v1.SecretVolumeSource: got "string", expected "map", ValidationError(Deployment.spec.template.spec.volumes[0]): unknown field "secretName" in io.k8s.api.core.v1.Volume]; if you choose to ignore these errors, turn validation off with --validate=false

so not sure why is not recognizing the secretName... any idea? I think I am very very close!

[DanRoscigno]

I don't think secretName belongs in the cert volume. Look at this example, and setup the cert like that. If you need more details on the volumes, etc., have a look at the k8s docs.

[eyanez111]

I found the error. I was adding what type of secret and secret is just to define that it is a secret what is trying to mount. so I just left it as Secret that worked. I see now the cert in the pod. I think I am quite close.... thanks again for all the help without your assistance I would not have been able to do this.

[DanRoscigno]

my pleasure, once you get it all working can you pop the manifest in here for the next person?

[eyanez111]

for sure, I am still working on it but at least I see the mount and the cert there. So definitely heading the right direction

[eyanez111]

I have not been able to make it work with the tls but I think this means it will only work with the ca.crt and not with the tls.crt. Still the process what you described @DanRoscigno it was pretty much this:

1. Create secret

 apiVersion: v1
 kind: Secret
 metadata:
   name: self-signed-certificate-secret
 data:
   ca-certificates.crt: <base 64 encoded string of certificate>

2. Make a volume named cert:

   volumes: 
        -  
          secret: 
            secretName: NAME FOR YOUR SECRET
            defaultMode: 0400
          name: cert

3. Mount the volume certin avolumeMountalso namedcert. This third step is where you specify the filename, by specifying both the mountPathandsubPath. The mountPathincludes the dir name and filename, and thesubPath` is just the filename.

      volumeMounts: 
        - 
          mountPath: /etc/ca.crt
          name: cert
          readOnly: true
          subPath: ca.crt
   

4. Last step for heatbeat you need to add this on the hearbeat.yaml part:

    - type: http
      id: service
      name:  NAME
      hosts: ["https://YOUR-URL"]
      schedule: '@every 5s'
      ssl:
        certificate_authorities: ['/etc/ca.crt']
        supported_protocols: ["TLSv1.0", "TLSv1.1", "TLSv1.2"]
        
       

That is how you add a secret based on a CA to heartbeat thanks again @DanRoscigno