Skip to content

Commit

Permalink
chore: setup docs
Browse files Browse the repository at this point in the history
  • Loading branch information
retro committed Sep 5, 2024
1 parent 1e9e213 commit b0768f2
Show file tree
Hide file tree
Showing 25 changed files with 18,123 additions and 99 deletions.
2 changes: 2 additions & 0 deletions .cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"Carnaval",
"chrono",
"Cidade",
"clsx",
"Creedence",
"dagrejs",
"DATEPART",
Expand Down Expand Up @@ -56,6 +57,7 @@
"Sucesso",
"testcontainers",
"tsdoc",
"typeof",
"verybigthings"
],
"flagWords": [],
Expand Down
56 changes: 56 additions & 0 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Deploy docs to GitHub Pages

on:
push:
branches:
- main
# Review gh actions docs if you want to further define triggers, paths, etc
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on

defaults:
run:
working-directory: ./docs

jobs:
build:
name: Build Docusaurus
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 18
cache: npm

- name: Install dependencies
run: npm ci
- name: Build website

run: npm run build

- name: Upload Build Artifact
uses: actions/upload-pages-artifact@v3
with:
path: docs/build

deploy:
name: Deploy to GitHub Pages
needs: build

# Grant GITHUB_TOKEN the permissions required to make a Pages deployment
permissions:
pages: write # to deploy to Pages
id-token: write # to verify the deployment originates from an appropriate source

# Deploy to the github-pages environment
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}

runs-on: ubuntu-latest
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
30 changes: 30 additions & 0 deletions .github/workflows/test-deploy-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Test docs deployment

on:
pull_request:
branches:
- main
# Review gh actions docs if you want to further define triggers, paths, etc
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on

defaults:
run:
working-directory: ./docs

jobs:
test-deploy:
name: Test deployment
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 18
cache: npm

- name: Install dependencies
run: npm ci
- name: Test build website
run: npm run build
144 changes: 46 additions & 98 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,40 +4,33 @@
![NPM](https://img.shields.io/npm/v/@verybigthings/semantic-layer)
![GitHub Workflow Status](https://github.com/verybigthings/semantic-layer/actions/workflows/semantic-layer.yml/badge.svg?branch=main)

## Introduction
# Quick Start Guide

The `@verybigthings/semantic-layer` library is crafted to simplify interactions between applications and relational databases, by providing a framework that abstracts SQL query complexities into a more manageable form. It aids in constructing analytical queries while addressing common issues such as join fanout and chasm traps. The library intelligently determines optimal join strategies for requested models, based on their definitions within the database. Designed for direct integration into existing code bases, it operates without the need for deploying external services.
Welcome to the semantic layer library! Let's dive in and create a simple data model in just a few steps.

## Key Features
## Installation

- **Declarative Schema and Query Building:** Utilize a fluent, TypeScript-based API to define your database schema and queries declaratively.
- **Type Safety:** Minimize errors with type-safe interfaces for query construction, enhancing code reliability.
- **Dynamic SQL Query Generation:** Automatically construct complex SQL queries tailored to your application's business logic, eliminating the need for string concatenation.
First, let's get the library installed (use npm or a package manager of your choice):

## Getting Started

### Installation

To integrate the Semantic Layer Library into your project, run the following command with npm:

```shell
```bash
npm install @verybigthings/semantic-layer
```

## Usage Examples
## Building Your First Semantic Layer

### Defining Models and Fields
Imagine you're running a music store. You have customers, and they make purchases. Let's model this!

This library allows you to define models and their respective fields, including dimensions and metrics, which represent the various columns and computed values within your database.
### Step 1: Create Your Models

**Defining a Model:**
We'll create two models: `customers` and `invoices`.

```typescript
import * as semanticLayer from "@verybigthings/semantic-layer";

// Our Customers model
const customersModel = semanticLayer
.model()
.withName("customers")
.withName("customers")
.fromTable("Customer")
.withDimension("customer_id", {
type: "number",
Expand All @@ -53,128 +46,83 @@ const customersModel = semanticLayer
sql: ({ model }) => model.column("LastName"),
});

// Our Invoices model
const invoicesModel = semanticLayer
.model()
.withName("invoices")
.withName("invoices")
.fromTable("Invoice")
.withDimension("invoice_id", {
type: "number",
primaryKey: true,
sql: ({ model, sql }) => sql`${model.column("InvoiceId")}`,
})
.withMetric("total", {
// node-postgres returns string types for big integers
type: "string",
aggregateWith: "sum",
sql: ({ model }) => model.column("Total"),
});

const invoiceLinesModel = semanticLayer
.model()
.withName("invoice_lines")
.fromTable("InvoiceLine")
.withDimension("invoice_line_id", {
type: "number",
primaryKey: true,
sql: ({ model }) => model.column("InvoiceLineId"),
})
.withDimension("invoice_id", {
type: "number",
sql: ({ model }) => model.column("InvoiceId"),
})
.withDimension("track_id", {
.withDimension("customer_id", {
type: "number",
sql: ({ model }) => model.column("TrackId"),
sql: ({ model }) => model.column("CustomerId"),
})
.withMetric("quantity", {
// node-postgres returns string types for big integers
type: "string",
aggregateWith: "sum",
sql: ({ model }) => model.column("Quantity"),
})
.withMetric("total_unit_price", {
// node-postgres returns string types for big integers

type: "string",
aggregateWith: "sum"
sql: ({ model }) => model.column("UnitPrice"),
.withMetric("total", {
type: "number",
description: "Invoice total.",
sql: ({ model, sql }) => sql`SUM(COALESCE(${model.column("Total")}, 0))`,
});
```

**Defining a Repository and joining models:**
### Step 2: Create a Repository

Now, let's put these models together in a repository:

```typescript
const repository = semanticLayer
.repository()
.withModel(customersModel)
.withModel(invoicesModel)
.withModel(invoiceLinesModel)
.joinOneToMany(
"customers",
"invoices",
({ sql, dimensions }) =>
sql`${dimensions.customers.customer_id} = ${dimensions.invoices.customer_id}`
)
.joinOneToMany(
"invoices",
"invoice_lines",
({ sql, dimensions }) =>
sql`${dimensions.invoices.invoice_id} = ${dimensions.invoice_lines.invoice_id}`
({ sql, models }) =>
sql`${models.customers.dimension(
"customer_id"
)} = ${models.invoices.dimension("customer_id")}`
);

const queryBuilder = repository.build("postgresql");
```

### Data Querying
### Step 3: Build a Query

Leverage the library's querying capabilities to fetch dimensions and metrics, apply filters, and sort results efficiently.
With our repository set up, we can now build queries:

```typescript
// Dimension and metric query
const queryBuilder = repository.build("postgresql");

const query = queryBuilder.buildQuery({
dimensions: ["customers.customer_id"],
metrics: ["invoices.total"],
members: [
"customers.customer_id",
"customers.first_name",
"customers.last_name",
"invoices.total",
],
order: { "customers.customer_id": "asc" },
limit: 10,
});
```

// Metric query with filters
const query = queryBuilder.buildQuery({
metrics: ["invoices.total", "invoice_lines.quantity"],
filters: [
{ operator: "equals", member: "customers.customer_id", value: [1] },
],
});
### Step 4: Execute the Query

// Dimension query with filters
const query = queryBuilder.buildQuery({
dimensions: ["customers.first_name", "customers.last_name"],
filters: [
{ operator: "equals", member: "customers.customer_id", value: [1] },
],
});
The `query` object contains the SQL string and bindings. You can use these with your preferred database client:

// Filtering and sorting
const query = queryBuilder.buildQuery({
dimensions: ["customers.first_name"],
metrics: ["invoices.total"],
filters: [{ operator: "gt", member: "invoices.total", value: [100] }],
order: { "invoices.total": "desc" },
});
```typescript
const result = await someSqlClient.query(query.sql, query.bindings);
```

### Executing queries

Note: `@verybigthings/semantic-layer` focuses on SQL generation. Execute the generated queries with your SQL client:
For example, with the `pg` package for PostgreSQL:

```typescript
const result = await sqlClient.query(query.sql, query.bindings);
const result = await pg.query(query.sql, query.bindings);
```

### Limitations
And there you have it! You've just set up a semantic layer for your music store data. This layer will make it easy to analyze customer purchases without writing complex SQL queries each time.

At the moment, only PostgreSQL queries are generated correctly. We're working on adding support for additional dialects.
Read the [documentation](https://verybigthings.github.io/semantic-layer/) for more information.

## Acknowledgments

`@verybigthings/semantic-layer` draws inspiration from several BI libraries, particularly [Cube.dev](https://cube.dev). While our API is very close to that of Cube.dev, future development may change our approach.
`@verybigthings/semantic-layer` draws inspiration from several BI libraries, particularly [Cube.dev](https://cube.dev). While our API was initially inspired by Cube.dev, it has since diverged based on our own needs and preferences.
3 changes: 2 additions & 1 deletion biome.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
"node_modules",
"coverage",
"vitest.config.ts",
"scripts/build.ts"
"scripts/build.ts",
"docs"
]
},
"formatter": {
Expand Down
20 changes: 20 additions & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Dependencies
/node_modules

# Production
/build

# Generated files
.docusaurus
.cache-loader

# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
yarn-error.log*
41 changes: 41 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Website

This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.

### Installation

```
$ yarn
```

### Local Development

```
$ yarn start
```

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

### Build

```
$ yarn build
```

This command generates static content into the `build` directory and can be served using any static contents hosting service.

### Deployment

Using SSH:

```
$ USE_SSH=true yarn deploy
```

Not using SSH:

```
$ GIT_USER=<Your GitHub username> yarn deploy
```

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
3 changes: 3 additions & 0 deletions docs/babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
};
Loading

0 comments on commit b0768f2

Please sign in to comment.