Skip to content

Commit e05ad05

Browse files
committed
Pipeline:input:kafka: style
Signed-off-by: Lynette Miles <[email protected]>
1 parent f6822f1 commit e05ad05

File tree

2 files changed

+26
-24
lines changed

2 files changed

+26
-24
lines changed

pipeline/inputs/kafka.md

Lines changed: 25 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,38 @@
11
# Kafka
22

3-
The Kafka input plugin allows subscribing to one or more Kafka topics to collect messages from an [Apache Kafka](https://kafka.apache.org/) service.
4-
This plugin uses the official [librdkafka C library](https://github.com/edenhill/librdkafka) \(built-in dependency\).
3+
The _Kafka_ input plugin subscribes to one or more Kafka topics to collect messages from an [Apache Kafka](https://kafka.apache.org/) service.
54

6-
## Configuration Parameters
5+
This plugin uses the official [librdkafka C library](https://github.com/edenhill/librdkafka) as a built-in dependency.
6+
7+
## Configuration parameters
78

89
| Key | Description | default |
910
| :--- | :--- | :--- |
10-
| brokers | Single or multiple list of Kafka Brokers, e.g: 192.168.1.3:9092, 192.168.1.4:9092. | |
11-
| topics | Single entry or list of topics separated by comma \(,\) that Fluent Bit will subscribe to. | |
12-
| format | Serialization format of the messages. If set to "json", the payload will be parsed as json. | none |
13-
| client\_id | Client id passed to librdkafka. | |
14-
| group\_id | Group id passed to librdkafka. | fluent-bit |
15-
| poll\_ms | Kafka brokers polling interval in milliseconds. | 500 |
16-
| Buffer\_Max\_Size | Specify the maximum size of buffer per cycle to poll kafka messages from subscribed topics. To increase throughput, specify larger size. | 4M |
17-
| rdkafka.{property} | `{property}` can be any [librdkafka properties](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) | |
18-
| threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
11+
| `brokers` | Single or multiple list of Kafka Brokers, e.g: 192.168.1.3:9092, 192.168.1.4:9092. | _none_ |
12+
| `topics` | Single entry or list of comma-separated topics (`,`) that Fluent Bit will subscribe to. | _none_ |
13+
| `format` | Serialization format of the messages. If set to `json`, the payload will be parsed as JSON. | _none_ |
14+
| `client_id` | Client id passed to librdkafka. | _none_ |
15+
| `group_id` | Group id passed to librdkafka. | `fluent-bit` |
16+
| `poll_ms` | Kafka brokers polling interval in milliseconds. | `500` |
17+
| `Buffer_Max_Size` | Specify the maximum size of buffer per cycle to poll Kafka messages from subscribed topics. To increase throughput, specify larger size. | `4M` |
18+
| `rdkafka.{property}` | `{property}` can be any [librdkafka properties](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) | _none_ |
19+
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
1920

20-
## Getting Started
21+
## Get started
2122

22-
In order to subscribe/collect messages from Apache Kafka, you can run the plugin from the command line or through the configuration file:
23+
To subscribe to or collect messages from Apache Kafka, run the plugin from the command line or through the configuration file:
2324

24-
### Command Line
25+
### Command line
2526

26-
The **kafka** plugin can read parameters through the **-p** argument \(property\), e.g:
27+
The Kafka plugin can read parameters through the `-p` argument (property):
2728

28-
```text
29-
$ fluent-bit -i kafka -o stdout -p brokers=192.168.1.3:9092 -p topics=some-topic
29+
```shell
30+
fluent-bit -i kafka -o stdout -p brokers=192.168.1.3:9092 -p topics=some-topic
3031
```
3132

32-
### Configuration File
33+
### Configuration file
3334

34-
In your main configuration file append the following _Input_ & _Output_ sections:
35+
In your main configuration file append the following `Input` and `Output` sections:
3536

3637
```text
3738
[INPUT]
@@ -44,7 +45,7 @@ In your main configuration file append the following _Input_ & _Output_ sections
4445
Name stdout
4546
```
4647

47-
#### Example of using kafka input/output plugins
48+
#### Example of using Kafka input and output plugins
4849

4950
The Fluent Bit source repository contains a full example of using Fluent Bit to
5051
process Kafka records:
@@ -69,10 +70,10 @@ process Kafka records:
6970
topics fb-sink
7071
```
7172

72-
The above will connect to the broker listening on `kafka-broker:9092` and subscribe to the `fb-source` topic, polling for new messages every 100 milliseconds.
73+
The previous example will connect to the broker listening on `kafka-broker:9092` and subscribe to the `fb-source` topic, polling for new messages every 100 milliseconds.
7374

74-
Since the payload will be in json format, we ask the plugin to automatically parse the payload with `format json`.
75+
Since the payload will be in JSON format, the plugin is configured to parse the payload with `format json`.
7576

7677
Every message received is then processed with `kafka.lua` and sent back to the `fb-sink` topic of the same broker.
7778

78-
The example can be executed locally with `make start` in the `examples/kafka_filter` directory (docker/compose is used).
79+
The example can be executed locally with `make start` in the `examples/kafka_filter` directory (`docker/compose` is used).

vale-styles/FluentBit/Spelling-exceptions.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@ Kube
9797
Kubernetes
9898
Kusto
9999
labelset
100+
librdkafka
100101
loadgenerator
101102
Logstash
102103
Lua

0 commit comments

Comments
 (0)