Skip to content

Commit

Permalink
chore: update docs (#815)
Browse files Browse the repository at this point in the history
Co-authored-by: Luis Moreno <[email protected]>
  • Loading branch information
github-actions[bot] and morenol authored Jun 19, 2024
1 parent 2f1c957 commit 8889580
Show file tree
Hide file tree
Showing 3 changed files with 43 additions and 16 deletions.
12 changes: 6 additions & 6 deletions embeds/connectors/inbound/http.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ This is an example of simple connector config file for polling an endpoint:
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.3.3
version: 0.3.7
name: cat-facts
type: http-source
topic: cat-facts
Expand Down Expand Up @@ -75,7 +75,7 @@ Fluvio HTTP Source Connector supports Secrets in the `endpoint` and in the `head
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.3.3
version: 0.3.7
name: cat-facts
type: http-source
topic: cat-facts
Expand All @@ -101,7 +101,7 @@ The previous example can be extended to add extra transformations to outgoing re
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.3.3
version: 0.3.7
name: cat-facts
type: http-source
topic: cat-facts
Expand Down Expand Up @@ -141,7 +141,7 @@ Provide the `stream` configuration option to enable streaming mode with `delimit
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.3.3
version: 0.3.7
name: wiki-updates
type: http-source
topic: wiki-updates
Expand All @@ -159,10 +159,10 @@ Connect to a websocket endpoint using a `ws://` URL. When reading text messages,
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.3.3
version: 0.3.7
name: websocket-connector
type: http-source
topic: websocket-updates
http:
endpoint: ws://websocket.example/websocket
```
```
8 changes: 4 additions & 4 deletions embeds/connectors/inbound/mqtt.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ JSON Serialized string with fields `mqtt_topic` and `payload`
This is an example of connector config file:

```yaml
# sample-config.yaml
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.2.5
Expand All @@ -67,7 +67,7 @@ mqtt:
Run connector locally using `cdk` tool (from root directory or any sub-directory):

```bash
cdk deploy start --config sample-config.yaml
cdk deploy start --config config-example.yaml
cdk deploy list # to see the status
cdk deploy log my-mqtt-connector # to see connector's logs
Expand Down Expand Up @@ -101,7 +101,7 @@ Fluvio MQTT Source Connector supports [Transformations](https://www.fluvio.io/do

The previous example can be extended to add extra transformations to outgoing records:
```yaml
# sample-config.yaml
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.2.5
Expand All @@ -117,7 +117,7 @@ mqtt:
nanos: 0
payload_output_type: json
transforms:
- uses: infinyon/jolt@0.1.0
- uses: infinyon/jolt@0.4.1
with:
spec:
- operation: shift
Expand Down
39 changes: 33 additions & 6 deletions embeds/connectors/outbound/http.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ HTTP sink connector reads records from data streaming and generates an HTTP requ
HTTP Sink is configured using a YAML file:

```yaml
# sample-config.yaml
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.2.5
version: 0.2.9
name: my-http-sink
type: http-sink
topic: http-sink-topic
Expand All @@ -34,7 +34,7 @@ http:
| method | POST | String | POST, PUT |
| endpoint | - | String | HTTP URL endpoint |
| headers | - | Array\<String\> | Request header(s) "Key:Value" pairs |
| user-agent | `fluvio/http-sink 0.2.2` | String | Request user-agent |
| user-agent | `fluvio/http-sink 0.2.9` | String | Request user-agent |
| http_request_timeout | 1s | String | HTTP Request Timeout |
| http_connect_timeout | 15s | String | HTTP Connect Timeout |

Expand All @@ -57,7 +57,7 @@ HTTP request to `http://httpbin.org/post`.
# config.yaml
apiVersion: 0.1.0
meta:
version: 0.2.5
version: 0.2.9
name: httpbin
type: http-sink
topic: httpbin-send-post
Expand Down Expand Up @@ -130,10 +130,10 @@ Fluvio HTTP Sink Connector supports [Transformations](https://www.fluvio.io/docs

The previous example can be extended to add extra transformations to outgoing records:
```yaml
# sample-config.yaml
# config-example.yaml
apiVersion: 0.1.0
meta:
version: 0.2.5
version: 0.2.9
name: my-http-sink
type: http-sink
topic: http-sink-topic
Expand All @@ -158,6 +158,33 @@ In this case, additional transformation will be performed before records are sen

Read more about [JSON to JSON transformations](https://www.fluvio.io/smartmodules/certified/jolt/).

### Offset Management
Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
To activate it, you need to provide the `consumer` name and set the `strategy: auto`.
See the example below:
```yaml
apiVersion: 0.2.0
meta:
version: 0.2.9
name: my-http-sink
type: http-sink
topic:
meta:
name: http-sink-topic
consumer:
id: my-http-sink
offset:
strategy: auto
http:
endpoint: "http://127.0.0.1/post"
```

After the connector processed any records, you can check the last stored offset value via:
```bash
$ fluvio consumer list
CONSUMER TOPIC PARTITION OFFSET LAST SEEN
my-http-sink http-sink-topic 0 0 3s
```
## Contributing

Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
Expand Down

0 comments on commit 8889580

Please sign in to comment.