Doc: Add clarification about API key format for Logstash #17688
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vishaangelova
added
enhancement
docs
backport-active-9
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
vishaangelova
force-pushed
the
serverless-clarify-api-key-when-using-logstash
branch
from
ee532b4 to
aea8d98
Compare
vishaangelova
force-pushed
the
serverless-clarify-api-key-when-using-logstash
branch
from
aea8d98 to
c6d674e
Compare
|
Looking... |
ppf2
reviewed
Jun 5, 2025
ppf2
reviewed
Jun 5, 2025
ppf2
reviewed
Jun 5, 2025
ppf2
reviewed
Jun 5, 2025
ppf2
reviewed
Jun 5, 2025
vishaangelova
force-pushed
the
serverless-clarify-api-key-when-using-logstash
branch
from
ef40eff to
96e4b47
Compare
karenzone
reviewed
Jun 24, 2025
| @@ -19,7 +19,8 @@ Logstash accelerates your insights by harnessing a greater volume and variety of | |||
| You’ll use the {{ls}} [{{es}} output plugin](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md) to send data to {{serverless-full}}. | |||
| Note these differences between {{es-serverless}} and both {{ess}} and self-managed {{es}}: | |||
|
|
|||
| * Use **API keys** to access {{serverless-full}} from {{ls}}. Any user-based security settings in your [{{es}} output plugin](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md) configuration are ignored and may cause errors. | |||
| * You must use [**API keys**](/reference/secure-connection.md#ls-api-keys) to access {{serverless-full}} from {{ls}} as it does not support native user authentication. Any user-based security settings in your [{{es}} output plugin](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md) configuration are ignored and may cause errors. | |||
There was a problem hiding this comment.
Suggested change
| * You must use [**API keys**](/reference/secure-connection.md#ls-api-keys) to access {{serverless-full}} from {{ls}} as it does not support native user authentication. Any user-based security settings in your [{{es}} output plugin](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md) configuration are ignored and may cause errors. | |
| * Use [**API keys**](/reference/secure-connection.md#ls-api-keys) to access {{serverless-full}} from {{ls}}. | |
| Any user-based security settings in your [{{es}} output plugin](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md) configuration are ignored and may cause errors. |
There was a problem hiding this comment.
The link to the relevant section is a nice addition. Let's keep the simpler, imperative statement format, and apply the one-sentence-per-line standard for easier GitHub edits and suggestions in the future.
| @@ -74,6 +74,7 @@ output { <3> | |||
| 1. Use `filter-elastic_integration` as the first filter in your pipeline | |||
| 2. You can use additional filters as long as they follow `filter-elastic_integration` | |||
| 3. Sample config to output data to multiple destinations | |||
| 4. The format of the value is `id:api_key`, where `id` and `api_key` are the values returned by the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key) | |||
There was a problem hiding this comment.
This topic is a high-level snippet designed to illustrate what's involved in using the elastic_integration filter. It's an incomplete configuration, and the details about one config setting out of 75 don't seem appropriate.
| @@ -37,6 +37,8 @@ Configuration example: | |||
|
|
|||
| * `output {elasticsearch { cloud_id => "<cloud id>" api_key => "<api key>" } }` | |||
|
|
|||
| Note that the value of the [`api_key` option](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md#plugins-outputs-elasticsearch-api_key) is in the format `id:api_key`, where `id` and `api_key` are the values returned by the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). | |||
There was a problem hiding this comment.
Suggested change
| Note that the value of the [`api_key` option](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md#plugins-outputs-elasticsearch-api_key) is in the format `id:api_key`, where `id` and `api_key` are the values returned by the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). |
| @@ -53,6 +55,8 @@ Configuration example: | |||
| * `output {elasticsearch { cloud_id => "<cloud id>" cloud_auth => "<cloud auth>" } }` | |||
| * `output {elasticsearch { cloud_id => "<cloud id>" api_key => "<api key>" } }` | |||
|
|
|||
| Note that the value of the [`api_key` option](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md#plugins-outputs-elasticsearch-api_key) is in the format `id:api_key`, where `id` and `api_key` are the values returned by the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). | |||
There was a problem hiding this comment.
Suggested change
| Note that the value of the [`api_key` option](logstash-docs-md://lsr/plugins-outputs-elasticsearch.md#plugins-outputs-elasticsearch-api_key) is in the format `id:api_key`, where `id` and `api_key` are the values returned by the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-api-key). |
There was a problem hiding this comment.
The api_key link with this information is already available in the previous sentence, and makes this same information available to anybody who clicks it.
| @@ -292,11 +296,20 @@ Tips for creating API keys: | |||
| * {{ls}} can send both collected data and monitoring information to {{es}}. If you are sending both to the same cluster, you can use the same API key. For different clusters, you need an API key per cluster. | |||
| * A single cluster can share a key for ingestion and monitoring purposes. | |||
| * A production cluster and a monitoring cluster require separate keys. | |||
| * When you create an API key for Logstash on the UI of your deployment, once the API key is generated, make sure you select **Logstash** from the dropdown to copy the API key in the correct `id:api_key` format. Note that base64 encoded API keys are not supported in the {{ls}} configuration. | |||
There was a problem hiding this comment.
Suggested change
| * When you create an API key for Logstash on the UI of your deployment, once the API key is generated, make sure you select **Logstash** from the dropdown to copy the API key in the correct `id:api_key` format. Note that base64 encoded API keys are not supported in the {{ls}} configuration. | |
| * When you create an API key for Logstash, select **Logstash** from the **API key format** dropdown. | |
| This option formats the API key in the correct id:api_key format required by Logstash. |
There was a problem hiding this comment.
ToDo: Is this applicable for serverless, Cloud, and self-managed?
| :width: 400px | ||
| ::: | ||
|
|
||
| Depending on the deployment, the UI for creating API keys and the API key format dropdown may be slightly different. |
There was a problem hiding this comment.
Suggested change
| Depending on the deployment, the UI for creating API keys and the API key format dropdown may be slightly different. | |
| The UI for API keys may look different depending on the deployment type. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release notes
[rn:skip]
What does this PR do?
This PR clarifies the API key format required for Logstash in the following docs:
docs/reference/secure-connection.mddocs/reference/connecting-to-cloud.mddocs/reference/using-logstash-with-elastic-integrations.mdRelated issues
Closes #17687
Changes
(All links are to the PR preview)
Previews
https://docs-v3-preview.elastic.dev/elastic/logstash/pull/17688/reference
https://docs-v3-preview.elastic.dev/elastic/logstash/pull/17688/reference/secure-connection
https://docs-v3-preview.elastic.dev/elastic/logstash/pull/17688/reference/connecting-to-cloud
https://docs-v3-preview.elastic.dev/elastic/logstash/pull/17688/reference/using-logstash-with-elastic-integrations