Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]: Support Top-level await in drizzle.config.ts #1982

Open
np-kyokyo opened this issue Mar 8, 2024 · 11 comments · May be fixed by #4075
Open

[FEATURE]: Support Top-level await in drizzle.config.ts #1982

np-kyokyo opened this issue Mar 8, 2024 · 11 comments · May be fixed by #4075
Labels
drizzle/kit enhancement New feature or request

Comments

@np-kyokyo
Copy link

np-kyokyo commented Mar 8, 2024

Describe what you want

I want to write a configuration as follows. Here, fetchDatabaseUri is a function that retrieves authentication information from AWS Secrets Manager and returns the database URI.

drizzle.config.tableau.ts

import type { Config } from "drizzle-kit";
import { fetchDatabaseUri } from "./util/rds";

const uri = await fetchDatabaseUri("tableau");

export default {
  schema: "./src/tableau/schema.ts",
  out: "./drizzle/tableau",
  driver: "mysql2",
  dbCredentials: { uri },
} satisfies Config;

Currently, when I run introspect with these settings, I get the following error:

np@MNNP9035-2 ~/g/s/db (db)> bun introspect:dev:tableau
$ dotenv -c dev.ro -- drizzle-kit introspect:mysql --config=drizzle.config.tableau.ts
drizzle-kit: v0.20.14
drizzle-orm: v0.29.4

Custom config path was provided, using 'drizzle.config.tableau.ts'
Reading config file '/Users/np/gitrep/sample/db/drizzle.config.tableau.ts'
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: Transform failed with 1 error:
/Users/np/gitrep/sample/db/drizzle.config.tableau.ts:4:12: ERROR: Top-level await is currently not supported with the "cjs" output format
    at failureErrorWithLog (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:1651:15)
    at /Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:849:29
    at responseCallbacks.<computed> (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:704:9)
    at handleIncomingPacket (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:764:9)
    at Socket.readFromStdout (/Users/np/gitrep/sample/db/node_modules/esbuild/lib/main.js:680:7)
    at Socket.emit (node:events:515:28)
    at addChunk (node:internal/streams/readable:545:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:495:3)
    at Readable.push (node:internal/streams/readable:375:5)
    at Pipe.onStreamRead (node:internal/stream_base_commons:190:23) {
  errors: [
    {
      detail: undefined,
      id: '',
      location: {
        column: 12,
        file: '/Users/np/gitrep/sample/db/drizzle.config.tableau.ts',
        length: 5,
        line: 4,
        lineText: 'const uri = await fetchDatabaseUri("tableau");',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'Top-level await is currently not supported with the "cjs" output format'
    }
  ],
  warnings: []
}

Node.js v21.1.0
error: script "introspect:dev:tableau" exited with code 1
@np-kyokyo np-kyokyo added the enhancement New feature or request label Mar 8, 2024
@artem-simutin
Copy link

I am facing the same issue. I want to fetch db credentials from Azure Key Vault (same as AWS Secret Manager) and cannot do that... 🥲

@bennettcolecohen
Copy link

I tried to wrap a promise around it but couldn't get it to work either. Any thoughts?

import type { Config } from "drizzle-kit";
import { getDatabaseConnectionString } from "./src/db/utils.js";

const connectionStringPromise = getDatabaseConnectionString();

const configPromise: Promise<Config> = (async () => {
  const connectionString = await connectionStringPromise;
  return {
    schema: "./src/db/schema.ts",
    out: "./src/db/migrations",
    driver: "pg",
    dbCredentials: {
      connectionString,
    },
  };
})();

export default configPromise;

@kylerush
Copy link

kylerush commented May 8, 2024

I am also having this issue. I am attempting to pull secrets from AWS Secrets manager in my JavaScript and I need top-level await to do that with Drizzle Kit's configuration file. I will have to abandon this strategy and instead use the AWS SDK for Linux to pull and set the environment variables at the OS level. Would be nice for top-level wait to be supported.

@paolostyle
Copy link

paolostyle commented May 13, 2024

Workaround:

const getConfig = async () => {
  // simplified
  const dbURL = await getSecret(process.env.DB_CREDENTIALS_SECRET);

  return defineConfig({
    schema: './src/schema.ts',
    out: './drizzle',
    dialect: 'postgresql',
    dbCredentials: {
      url: dbURL,
    },
  });
};

export default getConfig();

But naturally would be great to just use top-level await.

@kylerush
Copy link

kylerush commented May 13, 2024

@paolostyle wouldn't the last line of your code snippet need await?

export default await getConfig();

Since getConfig is an async function, doesn't it need to be awaited?

@paolostyle
Copy link

No, it doesn't. You don't ever have to await an async function, even though you probably should in most cases, but this is a config file processed by external code (drizzle-kit in this case). Considering it works, it likely awaits whatever is exported by the config file. I can't check it because a) drizzle-kit isn't open-sourced b) it's quite likely that the config part is handled by an external dependency c) I don't really have time to do that.

That snippet isn't something I would use in my regular code but as I said, it's a workaround.

@alecmev
Copy link

alecmev commented May 15, 2024

Async config works for drizzle-kit migrate/studio but not drizzle-kit generate/check, unfortunately:

ZodError: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "promise",
    "path": [],
    "message": "Expected object, received promise"
  }
]

Current (terrible) workaround:

// drizzle.credentials.ts

import { CONNECTION_STRING } from './some-file-with-top-level-await.js';

console.log(CONNECTION_STRING);
// drizzle.config.ts

import { execSync } from 'node:child_process';
import { createRequire } from 'node:module';

import type { Config } from 'drizzle-kit';

const require = createRequire(import.meta.url);

const CONNECTION_STRING = execSync(
  `tsx --no-warnings ${require.resolve('./drizzle.credentials.ts')}`,
  { encoding: 'utf8' },
).trim();

export default {
  dbCredentials: { url: CONNECTION_STRING },
  // ...
} satisfies Config;

@keypuncherlabs
Copy link

keypuncherlabs commented Sep 7, 2024

I'm having the same issue as well, this seems like a very common use case, storing secrets in .env files is hard to share and keep out of version control than just using a better approach like using a secrets manager.

Another workaround which doesn't require a code change is to run the migration while setting the env variable in the same command:

Mac Terminal command:

$ DATABASE_URL='your-db-url' npx drizzle-kit generate

@owlyowl
Copy link

owlyowl commented Oct 24, 2024

Can't wait for this feature... It seems totally essential for any secrets manager.

@EffectDoplera
Copy link

Async config works for drizzle-kit migrate/studio but not drizzle-kit generate/check, unfortunately:

ZodError: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "promise",
    "path": [],
    "message": "Expected object, received promise"
  }
]

Current (terrible) workaround:

// drizzle.credentials.ts

import { CONNECTION_STRING } from './some-file-with-top-level-await.js';

console.log(CONNECTION_STRING);
// drizzle.config.ts

import { execSync } from 'node:child_process';
import { createRequire } from 'node:module';

import type { Config } from 'drizzle-kit';

const require = createRequire(import.meta.url);

const CONNECTION_STRING = execSync(
  `tsx --no-warnings ${require.resolve('./drizzle.credentials.ts')}`,
  { encoding: 'utf8' },
).trim();

export default {
  dbCredentials: { url: CONNECTION_STRING },
  // ...
} satisfies Config;

As an alternative way of requesting secrets from the vault.

// drizzle.secrets.ts
import { getSecrets } from './get-secrets.ts'

const secrets = await getSecrets()

console.log(JSON.stringify(secrets))
// drizzle.config.ts
import { defineConfig } from 'drizzle-kit'

Object.assign(process.env, JSON.parse(process.env.SECRETS))

export default defineConfig({
    schema: "./src/db/schema.ts",
    out: "./src/db/migrations",
    driver: "pg",
    dbCredentials: {
      url: process.env.DATABASE_URL,
    },
  })

package.json

"db:migrate": "SECRETS=$(tsx --no-warnings './drizzle.secrets.ts') drizzle-kit migrate --config=drizzle.config.ts"

@kravetsone
Copy link
Contributor

looks like not drizzle issue
you package is CJS which sync

guillaumervls added a commit to guillaumervls/drizzle-orm that referenced this issue Feb 5, 2025
defineConfig now accepts a promise for a config or function that returns one - fixes drizzle-team#1982
guillaumervls added a commit to guillaumervls/drizzle-orm that referenced this issue Feb 5, 2025
defineConfig now accepts a promise for a config or function that returns one - fixes drizzle-team#1982
@guillaumervls guillaumervls linked a pull request Feb 5, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
drizzle/kit enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.