Split data.yaml file into seperate *.connection and *.query files

[jaroet]

Currently the connection information and the query text is in the same file. This is somewhat nice but also makes you repeat the connection information when it is already available in the project. Also it complicates how to version control your project.

I would like to see another layer added, by splitting the data.yaml files into *.connection files and *.query files. This way you only have to configure your connection once in a project and the different query files you can point to the correct connection file to get the connection information.

This change would also make it easier to add those *.connection files to the .gitignore file and make the query, model and dashboard files available through version control to our users. Our scenario is that we have several data discovery project with Rill in GIT repositories. People can then pull those, add there personal connection information to there local environment and keep getting updates from GIT pulls (and push changes).

[begelundmuller]

Hello @jaroet, this is a good observation and actually something we rolled out support for a few months back (but might not have been reflected throughout the examples yet). You can take a look at the connector YAML reference docs here: https://docs.rilldata.com/reference/project-files/connectors.

For example, for Postgres, you should be able to do something like this:

# .env
postgres_database_url="postgresql://postgres:postgres@localhost:5432/postgres"

# connectors/postgres.yaml  
type: connector
driver: postgres
database_url: "{{ .vars.postgres_database_url }}"

# sources/query.yaml
type: source
connector: postgres
sql: SELECT * FROM foo