Skip to content

Commit

Permalink
Improved README (#27)
Browse files Browse the repository at this point in the history
  • Loading branch information
ryan-blunden authored May 25, 2022
1 parent 274424a commit a6353a0
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 35 deletions.
65 changes: 32 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
# GitOps Secrets

Hello, GitOps Secrets! Goodbye environment variable storage limits.
A SecretOps workflow for bundling encrypted secrets into your deployments.

![GitOps SecretsDiagram](https://user-images.githubusercontent.com/133014/158977309-ce9efc17-ba94-4cb7-a7a4-bdb101a67e6d.jpg)

It's been a long-standing frustration that AWS Lambda deployments have a 4KB environment variable limit. This limit also impacts other environments such as [Vercel](https://vercel.com/support/articles/how-do-i-workaround-vercel-s-4-kb-environment-variables-limit) and the [Serverless framework](https://www.serverless.com/framework/docs/providers/aws/guide/variables) who use AWS Lambda as their infrastructure provider.
## Usage

A GitOps Secrets workflow eliminates environment variable limits without insecure hacks such as storing unencrypted .env files in your builds and only takes three steps:

1. Install the GitOps Secrets package (currently in developer preview):
1. Install the `gitops-secretes` package:

```sh
npm install gitops-secrets
Expand Down Expand Up @@ -47,25 +45,27 @@ const secrets = loadSecrets();
## Background and Motivation
As creators of the [Doppler Universal Secrets Platform](https://www.doppler.com/) who provide secrets sync integrations for [Vercel](https://vercel.com/integrations/doppler) and [Serverless](https://docs.doppler.com/docs/enclave-installation-serverless), we've helped customers individually to work around this limitation.
Exceeding AWS Lambda's 4KB environment variable limit is a common problem that also impacts platforms such as [Vercel](https://vercel.com/support/articles/how-do-i-workaround-vercel-s-4-kb-environment-variables-limit) and the [Serverless framework](https://www.serverless.com/framework/docs/providers/aws/guide/variables) which deploy on top of AWS Lambda.
A SecretOps workflow that bundles encrypted secrets into a deployment eliminates such environment variable limits without insecure hacks such as storing unencrypted .env files in your builds.
But long-term, we wanted a generic, flexible and open source solution that both our customers and other teams experiencing the same issue could use.
As creators of the [Doppler SecretOps Platform](https://www.doppler.com/) which provide secrets sync integrations for [Vercel](https://vercel.com/integrations/doppler) and [Serverless](https://docs.doppler.com/docs/enclave-installation-serverless), we built this to provide a secure solution for our customers and the open source community.
Our goal was to design a new way of accessing secrets in production that:
- [x] Allowed for a secrets payload of any size
- [x] Could be up and running in minutes
- [x] Scaled to work in any environment, including local development
- [x] Could support the most restrictive serverless platforms
- [x] Provided first class support for ES modules
- [x] Prevented unencrypted secrets from ever touching the file system
- [x] Abstracted away the complexity of secrets fetching using community contributed [providers](./src/providers/)
- Allowed for a secrets payload of any size
- Could be up and running in minutes
- Scaled to work in any environment, including local development
- Could support the most restrictive serverless platforms
- Provided first-class support for ES modules
- Prevented unencrypted secrets from ever touching the file system
- Abstracted away the complexity of secrets fetching using community-contributed [providers](./src/providers/)
## Providers
A provider is designed to abstract away the complexities of fetching secrets from any secret manager or secrets store by exposing a single async `fetch` method.
A secrets provider returns a plain Key-Value Object to ensure that serializing to and from JSON during encryption and decryption produces the same object structure originally fetched from the provider.
A secrets provider returns a plain Key-Value Object to ensure that serializing to and from JSON during encryption and decryption produces the same object structure initially fetched from the provider.
The current list of providers are:
Expand All @@ -75,13 +75,11 @@ We'd love to see the list of providers grow! Please see our [contributing guide]
## Encryption and Decryption
There are two file formats available for bundling secrets into your build:
There are two file formats available for bundling encrypted secrets into your deployments:
- **JSON**: Encrypted JSON file.
- **JS Module**: Encrypted JSON embedded in JS module.
You may be forced to use the JS module format if reading static JSON at runtime is problematic, e.g. [Vercel prefers a JS module with a custom path](https://github.com/DopplerUniversity/vercel-gitops-secrets-nextjs), but otherwise, there isn't a compelling reason to use one format over another.
### JSON
To encrypt secrets to a JSON file:
Expand Down Expand Up @@ -119,7 +117,11 @@ secrets.populateEnv();
### JS Module
The JS module format suits restricted environments where reading static files is problematic and depending upon the platform, building with a custom path may be required.
The JS module format is ideal for restricted environments such as Vercel where application-wide access to reading static files is problematic.
Depending upon the deployment platform and framework, you can potentially omit the `path` parameter to have encrypted secrets access and storage managed internally for you.
But if using Vercel with Next.js for example, the `path` configures the module to be output in your codebase with the format of the module matching that of your application.
To encrypt secrets to a JS module:
Expand All @@ -129,27 +131,28 @@ const secrets = require("gitops-secrets");
async function main() {
const payload = await secrets.providers.doppler.fetch();
// Internally managed storage
// Option 1: Internally managed storage
secrets.build(payload);
// Custom path for restrictive environments
// Option 2: Custom path for restrictive environments
secrets.build(payload, { path: "lib/secrets.js" });
}
main();
```
Then to decrypt secrets from a JS module, you can rely on internally managed storage:
To decrypt secrets from a JS module using internally managed storage, use the package-level `loadSecrets` method:
```js
const { loadSecrets } = require("gitops-secrets");
const secrets = loadSecrets();
// Optionally merge secrets into environment variables
secrets.populateEnv();
```
Or import directly from the generated JS module:
Or use the `loadSecrets` method from the generated module (ES modules also supported):
```js
const { loadSecrets } = require("../lib/secrets");
Expand All @@ -159,18 +162,14 @@ const secrets = loadSecrets();
secrets.populateEnv();
```
### Getting Started
We recommend checking out the [Working around Vercel’s 4KB Environment Variables Limit for Node.js with GitOps Secrets article](https://hashnode.com/preview/623404babef4c71aa6f0d65e) which takes you through the entire process step-by-step.
## Examples
## Getting Started
Take a look at the [Vercel GitOps Secrets Next.js sample repository](https://github.com/DopplerUniversity/vercel-gitops-secrets-nextjs) and deploy to Vercel to see it in action.
We recommend checking out the [Working around Vercel's 4KB Environment Variables Limit for Node.js with GitOps Secrets](https://hashnode.com/preview/623404babef4c71aa6f0d65e) blog post which guides you through the entire process.
## Contributing
Or take a look at the [Vercel GitOps Secrets Next.js sample repository](https://github.com/DopplerUniversity/vercel-gitops-secrets-nextjs) to see a complete working example that you can test and deploy to Vercel.
As this package is still in developer preview, a huge contribution you can make is simply testing this with your preferred framework and serverless provider as we'd love your feedback!
## Support
You can get support in the [Doppler community forum](https://community.doppler.com/), find us on [Twitter](https://twitter.com/doppler), and for bugs or feature requests, [create an issue](https://github.com/DopplerHQ/gitops-secrets-nodejs/issues) on this repository.
You can get support in the [Doppler community forum](https://community.doppler.com/), find us on [Twitter](https://twitter.com/doppler), and for bugs or feature requests, [create an issue](https://github.com/DopplerHQ/gitops-secrets-nodejs/issues) on the [DopplerHQ/gitops-secrets-nodejs](https://github.com/DopplerHQ/gitops-secrets-nodejs) GitHub repository.
We'd also love to see the number of providers grow and you can check out our [contributing guide](CONTRIBUTING.md) to get started.
We'd also love to see the number of providers grow, and you can check out our [contributing guide](CONTRIBUTING.md) to get started.
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "gitops-secrets",
"version": "0.0.3",
"version": "0.0.4",
"author": "Ryan Blunden <[email protected]>",
"description": "Securely bundle encrypted secrets into your deployments and safely decrypt at runtime.",
"description": "SecretOps workflow for bundling encrypted secrets into your deployments to safely decrypt at runtime.",
"repository": {
"type": "git",
"url": "https://github.com/DopplerHQ/gitops-secrets-nodejs.git"
Expand Down

0 comments on commit a6353a0

Please sign in to comment.