Automatic credential rotation

[reidsunderland]

We have a sender to AWS S3 where the access key ID and secret key expire every 3 months. NASA provided a script that will allow us to auto-rotate the keys: https://github.com/jjmcnelis/data-publication/blob/main/s3-upload-info/rotate_upload_access_keys.py

i.e. we use the existing keys to get new keys before the old keys expire, then we switch to using the new keys.

This is fairly easy to build into a plugin. But the tricky part is how to update and keep track of the credentials. The keys are in credentials.conf, but once we use the old keys to generate new keys, we need to update credentials.conf.

Having an automated process update credentials.conf is risky, because if something goes wrong it could cause lots of problems. And committing the changes to Git would be another challenge.

[petersilva]

so it's like the credentials for requesting the keys should be what's in credentials.conf, and the keys themselves should be stored in a statefile. the expiry could be in the statefile also.
if the statefile doesn't exist, or has expired, use info from credentials.conf to generate a new one.

[petersilva]

um... it's worse than that.. because the statefile needs to be shared by all the nodes participating... harrumph.

[reidsunderland]

yeah, and in this case, the credentials used to request the keys are the old keys. So unless the credentials.conf is updated, once the original keys in there expire, if the statefile is ever removed we're stuck.

"The access keys are generated automatically every 90 days, and they expire after 120 days. You can obtain the new access keys using your old access keys during the 30-day period while both sets are active "

[reidsunderland]

One solution I've been thinking of is a plugin to integrate with Passbolt's API.

Store the keys in Passbolt and have a plugin that can retrieve and update it. credentials.conf would store credentials used to connect to Passbolt.

[petersilva]

brain storming... uhh.. so something that:

• something that polls passbolt and writes the credentials down in some file.
• something that posts credentials to passbolt.

another option... redis. have a credentials store in redis,read and write it...
no local copies needed.

maybe refactor credentials to support multiple of these...

[reidsunderland]

what I was thinking is a plugin that pulls a credential from passbolt and adds it to the credentials object that's in memory. It would query passbolt when the process starts

Kind of the same as we have for BearerTokens here: https://github.com/MetPX/sarracenia/blob/development/sarracenia/flowcb/authenticate/__init__.py

But would work with any kind of credential, not just bearer tokens.

A more generic credentials interface that could work with redis, passbolt, credentials.conf on disk or anything else we want to implement would be great, but a lot more work.

[petersilva]

I worry any time real-time constraints arrive... like if everything is down... do we need passbolt up before we can start any data pumps? would want some kind of local fallback in case it isn't up when we start up... like the statefiles.