suggestions for sharing addons across multiple copilot apps

[codekitchen]

Hi, I'm looking for any suggestions anybody has on a good way to share addon templates between multiple copilot apps, while making it easy for those apps to adopt new versions of the template.

To make the question more concrete, we've created our own RDS addon that uses regular (not serverless v1) Aurora and adds quite a bit of alerting and other goodies. Right now we are just copying that addon template to each copilot service that is using it, but it is already getting very difficult to propagate changes to that template across the different apps, especially given that realistically you'll have to make modifications to at least the parameter defaults to enable different instance sizes, etc. So just copying in a new version of the template isn't enough, you have to do a sort of manual merging process to copy it in while preserving the parameter changes specific to your service.

I took a go at solving this by having the copilot addon be nothing but a CF substack resource (AWS::CloudFormation::Stack) pointing to a shared template in S3. But in the end I realized this won't work currently, because copilot parses the addons template looking for resource names in order to do its magic to find secrets, security groups, etc in the outputs. Since those resources are defined in the substack, copilot can no longer parse them out.

[codekitchen]

One solution I've just arrived at is to separate all the tweakable parameters into a separate .yml file, so you have something like this:

database-cluster.yml
database-cluster-config.yml

So database-cluster-config.yml is the only file that individual apps should have to modify, and we can copy over database-cluster.yml with a new version when we want to update it.

This definitely feels a bit clunky and will take some training but it is better than my baseline of manually merging CF templates to update. I'm still definitely interested in a better solution.

[huanjani]

Hi @codekitchen!
Does the svc package command help at all, for what you're trying to do?
https://aws.github.io/copilot-cli/docs/commands/svc-package/

[huanjani]

Ahhh, gotcha gotcha. Thanks for the clarification. Yeah, svc package won't help you with that at all 😂 .
Configuring the addons template via the manifest is an interesting idea for sure. I'm curious if it's a popular ask.
I can't think of a workaround beyond your solution above, but the Copilot team will noodle on this!

[codekitchen]

thanks so much!

[huanjani]

My teammate may have the solution for you!
https://aws.github.io/copilot-cli/docs/developing/additional-aws-resources/#customizing-the-parameters-section

[codekitchen]

Ah I hadn't noticed that in the documentation. It sounds like that is the same idea I arrived at in my first response above, but it's good to see it officially documented. I do worry about it being confusing and error-prone for team members who are less familiar with Copilot -- right now we are tackling that as best we can with lots of comments in the yml files to guide people.

[codekitchen]

Oh sorry, I misunderstood this documentation until I actually tried it out -- addons.parameters.yml is a file with special meaning in Copilot, not just another template name that will be concatenated with the others. It defines actual parameter values that will be passed to CF as parameter values, rather than concatenated into the addon template itself. That is a little bit of an improvement, thank you for pointing me to that.