Skip to content

Commit

Permalink
Merge branch 'main' into dep_replace
Browse files Browse the repository at this point in the history
  • Loading branch information
lmangani authored Oct 3, 2023
2 parents 8f485c0 + e2f29f9 commit d93fcfa
Show file tree
Hide file tree
Showing 104 changed files with 13,330 additions and 12 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Simple workflow for deploying static content to GitHub Pages
name: Deploy Flux docs to Pages

on:
# Runs on pushes targeting the default branch
push:
branches: ["main"]
paths:
- docs/**

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false

jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Pages
uses: actions/configure-pages@v3
- name: Upload artifact
uses: actions/upload-pages-artifact@v2
with:
# Upload entire repository
path: './docs'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
1 change: 0 additions & 1 deletion .github/workflows/go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ jobs:
if: steps.cacheflux.outputs.cache-hit != 'true'
run: |
cd flux
git checkout tags/v0.194.3
go mod tidy
go get ./...
make
Expand Down
75 changes: 67 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,25 @@

[![Build-n-Release](https://github.com/metrico/fluXpipe/actions/workflows/go.yml/badge.svg)](https://github.com/metrico/fluXpipe/actions/workflows/go.yml)

**FluxPipe** is an *experimental* stand-alone **Flux API** for *serverless workers* and *embedded datasources*
**FluxPipe** is an *experimental* stand-alone **Flux API** for *serverless workers* and *embedded datasources*<br>
Execute your Flux scripts locally, in serverless functions or anywhere else - _decoupled from the data and database._<br>

> [Flux](https://github.com/InfluxCommunity/flux) is a lightweight *scripting language* for querying databases and working with data. [^1]
Need a practical Flux introduction? Check out the [official page](https://www.influxdata.com/products/flux/) or [3 Minutes to Flux](flux.md)
_Fluxpipe runs at 141,6Km/h_[*](https://en.wikipedia.org/wiki/DeLorean_time_machine#:~:text=Various%20proposals%20have%20been%20brought,speedometer%2C%20modified%20for%20the%20movie.) and is compatible with **InfluxDB 3.0 / IOx, ClickHouse, Grafana** and _beyond!_

<br>

> InfluxDB [Flux](https://github.com/InfluxCommunity/flux) is a lightweight *scripting language* for querying databases and working with data. [^1]<br>
> Need a practical Flux introduction? Check out the [official page](https://www.influxdata.com/products/flux/) or [3 Minutes to Flux](flux.md)
### Demo
Try our [serverless demo](https://fluxpipe.fly.dev/) or launch your own instance to instantly fall in love with *flux*

<br>

<a href="https://flyctl.sh/shell?repo=metrico/fluxpipe" target="_blank">
<img src="https://user-images.githubusercontent.com/1423657/236479471-a1cb0484-dfd3-4dc2-8d62-121debd7faa3.png" width=300>
</a>

<br>

### Instructions
Download a [binary release](https://github.com/metrico/fluxpipe/releases/), [docker](https://github.com/metrico/fluXpipe/pkgs/container/fluxpipe) or build from source

Expand Down Expand Up @@ -56,7 +58,56 @@ Fluxpipe embeds a playground interface to instantly execute queries _(borrowed f
Fluxpipe serves a simple REST API loosely compatible with existing flux integrations and clients

##### Grafana Flux [^1]
Usage with native **Grafana InfluxDB/Flux datasource** _(url + organization fields are required!)_
Fluxpipe is compatible with the native **Grafana InfluxDB/Flux datasource** _(url + organization fields are required!)_

<br>

##### ⭐ FlightSQL
###### SQL
You can query InfluxDB 3.0 IOx with raw SQL using the native `sql.from` handler
```
import "sql"
sql.from(
driverName: "influxdb-iox",
dataSourceName: "iox://iox-server:443/qryn_logs",
query: "SELECT level, sender, body FROM logs WHERE body LIKE '%DELETE%' limit 10",
)
```

![image](https://github.com/metrico/fluXpipe/assets/1423657/b6c2dcbe-079b-4329-9fee-a8601a8c853c)


###### Flux
You can query InfluxDB 3.0 IOx with Flux using the `iox.from` handler
```
> import "contrib/qxip/iox"
iox.from(
bucket: "test",
host: "eu-central-1-1.aws.cloud2.influxdata.com:443",
token: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
limit: "10",
columns: "time, level, sender",
table: "logs",
start: -100d,
)
```
```
_time:time level:string sender:string
------------------------------ ---------------------- ----------------------
2023-08-31T00:00:00.091362490Z info logtest
2023-08-31T00:00:00.091380034Z info logtest
2023-08-31T00:00:00.091381374Z info logtest
2023-08-31T00:00:00.091382470Z info logtest
2023-08-31T00:00:00.091383354Z info logtest
2023-08-31T00:00:00.091384514Z info logtest
2023-08-31T00:00:00.091385496Z info logtest
2023-08-31T00:00:00.091387718Z info logtest
2023-08-31T00:00:00.091389187Z info logtest
2023-08-31T00:00:00.091390136Z info logtest
```



###### ⭐ ClickHouse SQL
```
Expand Down Expand Up @@ -153,7 +204,15 @@ cat scripts/sql.flux | ./fluxpipe-server -stdin

### Grafana Datasource
Configure your Grafana instance with our public demo endpoint _(limited resources)_
![image](https://user-images.githubusercontent.com/1423657/185748494-0c6a95da-d112-46ab-b9db-b09438b63740.png)
![image](https://github.com/metrico/fluXpipe/assets/1423657/c100050a-49a6-4788-86af-15eb7e59c5d1)



### Documentation
**Flux**(pipe) is built using the [InfluxCommunity/flux](https://github.com/InfluxCommunity/flux) fork which contains additional features and contributions.<br>
All the standard and additional functions available in Fluxpipe are included in the [Flux community documentation](https://metrico.github.io/fluXpipe/)

<br>


#### Status
Expand Down
4 changes: 4 additions & 0 deletions build.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
#!/bin/sh

# WARNING:
# Make sure you have a complete flux build at `/usr/src/flux`

export PKG_CONFIG_PATH=$(pwd)

echo "Building fluxpipe-server ..."
Expand Down
13 changes: 13 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
![logo](https://user-images.githubusercontent.com/1423657/231777629-38f9c8f5-7968-4ff3-ae66-2fd2459ffa69.png ':size=200')

# Flux Community Documentation

Flux is an open source functional data scripting language designed for querying, analyzing, and acting on data. Flux supports multiple data source types, including:

- Time series databases _(such as InfluxDB and IOx)_
- Relational SQL databases _(such as MySQL, PostgreSQL, ClickHouse, etc)_
- Local Files or Object Storage Data _(such as CSV, TSV, JSON, etc)_

Flux unifies code for querying, processing, writing, and acting on data into a single syntax.

The language is designed to be usable, readable, flexible, composable, testable, contributable, and shareable.
124 changes: 124 additions & 0 deletions docs/_sidebar.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
##

- array
- [array](array.md)
- bitwise
- [bitwise](bitwise.md)
- contrib
- [naiveBayesClassifier](contrib/RohanSreerama5/naiveBayesClassifier.md)
- [anomalydetection](contrib/anaisdg/anomalydetection.md)
- [statsmodels](contrib/anaisdg/statsmodels.md)
- [alerta](contrib/bonitoo-io/alerta.md)
- [hex](contrib/bonitoo-io/hex.md)
- [servicenow](contrib/bonitoo-io/servicenow.md)
- [tickscript](contrib/bonitoo-io/tickscript.md)
- [victorops](contrib/bonitoo-io/victorops.md)
- [zenoss](contrib/bonitoo-io/zenoss.md)
- [discord](contrib/chobbs/discord.md)
- [influxdb](contrib/jsternberg/influxdb.md)
- [clickhouse](contrib/qxip/clickhouse.md)
- [hash](contrib/qxip/hash.md)
- [iox](contrib/qxip/iox.md)
- [logql](contrib/qxip/logql.md)
- [bigpanda](contrib/rhajek/bigpanda.md)
- [opsgenie](contrib/sranka/opsgenie.md)
- [sensu](contrib/sranka/sensu.md)
- [teams](contrib/sranka/teams.md)
- [telegram](contrib/sranka/telegram.md)
- [webexteams](contrib/sranka/webexteams.md)
- [events](contrib/tomhollingworth/events.md)
- csv
- [csv](csv.md)
- date
- [date](date.md)
- [boundaries](date/boundaries.md)
- dict
- [dict](dict.md)
- experimental
- [experimental](experimental.md)
- [aggregate](experimental/aggregate.md)
- [array](experimental/array.md)
- [bigtable](experimental/bigtable.md)
- [bitwise](experimental/bitwise.md)
- [csv](experimental/csv.md)
- [boundaries](experimental/date/boundaries.md)
- [dynamic](experimental/dynamic.md)
- [geo](experimental/geo.md)
- [http](experimental/http.md)
- [requests](experimental/http/requests.md)
- [influxdb](experimental/influxdb.md)
- [iox](experimental/iox.md)
- [json](experimental/json.md)
- [mqtt](experimental/mqtt.md)
- [oee](experimental/oee.md)
- [polyline](experimental/polyline.md)
- [prometheus](experimental/prometheus.md)
- [query](experimental/query.md)
- [record](experimental/record.md)
- [table](experimental/table.md)
- [usage](experimental/usage.md)
- generate
- [generate](generate.md)
- http
- [http](http.md)
- [requests](http/requests.md)
- influxdata
- [influxdb](influxdata/influxdb.md)
- [monitor](influxdata/influxdb/monitor.md)
- [sample](influxdata/influxdb/sample.md)
- [schema](influxdata/influxdb/schema.md)
- [secrets](influxdata/influxdb/secrets.md)
- [tasks](influxdata/influxdb/tasks.md)
- [v1](influxdata/influxdb/v1.md)
- internal
- [boolean](internal/boolean.md)
- [debug](internal/debug.md)
- [gen](internal/gen.md)
- [influxql](internal/influxql.md)
- [location](internal/location.md)
- [promql](internal/promql.md)
- [testing](internal/testing.md)
- [testutil](internal/testutil.md)
- interpolate
- [interpolate](interpolate.md)
- join
- [join](join.md)
- json
- [json](json.md)
- kafka
- [kafka](kafka.md)
- math
- [math](math.md)
- pagerduty
- [pagerduty](pagerduty.md)
- planner
- [planner](planner.md)
- profiler
- [profiler](profiler.md)
- pushbullet
- [pushbullet](pushbullet.md)
- regexp
- [regexp](regexp.md)
- runtime
- [runtime](runtime.md)
- sampledata
- [sampledata](sampledata.md)
- slack
- [slack](slack.md)
- socket
- [socket](socket.md)
- sql
- [sql](sql.md)
- strings
- [strings](strings.md)
- system
- [system](system.md)
- testing
- [testing](testing.md)
- [expect](testing/expect.md)
- timezone
- [timezone](timezone.md)
- types
- [types](types.md)
- universe
- [universe](universe.md)
78 changes: 78 additions & 0 deletions docs/array.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
## `array` package

Package array provides functions for manipulating array and building tables from arrays.

Import the `array` package:

```flux
import "array"
```

### Functions

### `concat()`

concat appends two arrays and returns a new array.

#### Function type signature

```flux
(<-arr: [A], v: [A]) => [A]
```

#### Parameters

| Parameter | Description | Required |
| --- | --- | --- |
| arr | arr: First array. Default is the piped-forward array (`<-`). | No |
| v | v: Array to append to the first array. | Yes |
### `filter()`

filter iterates over an array, evaluates each element with a predicate function, and then returns
a new array with only elements that match the predicate.

#### Function type signature

```flux
(<-arr: [A], fn: (x: A) => bool) => [A]
```

#### Parameters

| Parameter | Description | Required |
| --- | --- | --- |
| arr | arr: Array to filter. Default is the piped-forward array (`<-`). | No |
| fn | fn: Predicate function to evaluate on each element.
The element is represented by `x` in the predicate function. | Yes |
### `from()`

from constructs a table from an array of records.

#### Function type signature

```flux
(<-rows: [A]) => stream[A] where A: Record
```

#### Parameters

| Parameter | Description | Required |
| --- | --- | --- |
| rows | rows: Array of records to construct a table with. Default is the piped-forward array (`<-`). | No |
### `map()`

map iterates over an array, applies a function to each element to produce a new element,
and then returns a new array.

#### Function type signature

```flux
(<-arr: [A], fn: (x: A) => B) => [B]
```

#### Parameters

| Parameter | Description | Required |
| --- | --- | --- |
| arr | arr: Array to operate on. Defaults is the piped-forward array (`<-`). | No |
| fn | fn: Function to apply to elements. The element is represented by `x` in the function. | Yes |
Loading

0 comments on commit d93fcfa

Please sign in to comment.