Skip to content
This repository has been archived by the owner on Oct 28, 2024. It is now read-only.

POC granularity change #18

Closed
wants to merge 292 commits into from

Conversation

andrewma2
Copy link
Collaborator

@andrewma2 andrewma2 commented Aug 15, 2024

POC vector event log change; Iterates through a prepared set of batched events to record the count by the specified keys

neuronull and others added 30 commits April 1, 2024 18:50
…extract is enabled (vectordotdev#20213)

* fix(splunk_hec_logs sink): don't attempt to remove timestamp if auto extract is enabled

* typo

* fix integration test expectation

* spellcheck
* chore(deps): Bump chrono from 0.4.34 to 0.4.37

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.34 to 0.4.37.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](chronotope/chrono@v0.4.34...v0.4.37)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* clippy

Signed-off-by: Jesse Szwedko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Jesse Szwedko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Szwedko <[email protected]>
… expected by DD logs intake (vectordotdev#20198)

* chore(datadog_agent): align ddtags parsed output with DD logs intake

* changelog

* feedback- changelog

* component docs

* tags reconstruct in dd logs

* upgrade guide note

* spell check

* upgrade/changelog notes about dd logs

* rm extraneous upgrade guide

* feedback bruce

* changelog

* Revert "component docs"

This reverts commit 2096c73.

* Revert "feedback- changelog"

This reverts commit dda9512.

* Revert "changelog"

This reverts commit a278981.

* Revert "chore(datadog_agent): align ddtags parsed output with DD logs intake"

This reverts commit 4c657c6.

* Revert "upgrade guide note"

This reverts commit 10cbbf9.

* use correct attribute name

* one more ddtag rename

* feedback pavlos and const usage

* fix agent unit tests

* feedback bruce- constify ddtags

* better path compare

* feedback bruce- itertools ftw

* Compare the owned path to the field name without creating an `OwnedTargetPath`

* Revert "fix agent unit tests"

This reverts commit f2d2160.

* restore dd agent

---------

Co-authored-by: Bruce Guenter <[email protected]>
…#20221)

Bumps [async-compression](https://github.com/Nullus157/async-compression) from 0.4.6 to 0.4.7.
- [Release notes](https://github.com/Nullus157/async-compression/releases)
- [Changelog](https://github.com/Nullus157/async-compression/blob/main/CHANGELOG.md)
- [Commits](Nullus157/async-compression@async-compression-v0.4.6...async-compression-v0.4.7)

---
updated-dependencies:
- dependency-name: async-compression
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): Bump hostname from 0.3.1 to 0.4.0

Bumps [hostname](https://github.com/svartalf/hostname) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/svartalf/hostname/releases)
- [Changelog](https://github.com/svartalf/hostname/blob/master/CHANGELOG.md)
- [Commits](svartalf/hostname@v0.3.1...v0.4.0)

---
updated-dependencies:
- dependency-name: hostname
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* regenerate licenses

Signed-off-by: Jesse Szwedko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Jesse Szwedko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jesse Szwedko <[email protected]>
… reserved attributes (vectordotdev#20226)

* chore(datadog_logs sink): properly encode all semantically defined DD reserved attributes

* fix comment

* feedback and caught one other thing

* fix bug

* fix unit test

* grr
…ordotdev#19398)

* feat(vectordotdev#19183): add namespace input to chronicle sink

* docs(vectordotdev#19183): add changelog reference for the enhancement

* style(vectordotdev#19183): fix linting error in the changelog

* docs(vectordotdev#19183): fix docs check error

* Update src/sinks/gcp/chronicle_unstructured.rs

Co-authored-by: Stephen Wakely <[email protected]>

* feature(vectordotdev#19183): make chronicle namespace optional

* fix linting errors

---------

Co-authored-by: Stephen Wakely <[email protected]>
…20227)

Update docker.md so the command can be executed

It was missing --name  for the docker run command
…dec (vectordotdev#20074)

* Try to use the new VRL protobuf library

* Run fmt/clippy

* Some more changes

* Fix tests

* Add changelog entry

* Address comment
…p attributes (vectordotdev#20211)

- Currently the splunk hec logs sink has default paths for the timestamp_key and host_key configurable settings which statically point to the global log namespace location for these keys.
- In order to support log namespacing for these settings, the logic for determining the default case is moved from the configuration serialization down into the sink's encoding of the event. That is necessary because we must determine at runtime if each event received is namespaced or not.
- Note that this necessitated a small change to the Humio sinks as well simply due to those sinks being wrappers over Splunk HEC, to preserve existing behavior.
To resolve `RUSTSEC-2024-0332`.

Signed-off-by: Jesse Szwedko <[email protected]>
* fix(kafka source): fix source span instrumentation

* instrumenting the consumer async block covers the relevant emit! calls for Kafka{Bytes,Events}Received

* fix clippy warning about too many arguments by putting the span into consumer inner state struct

* add changelog entry

* spelling

* more better spelling :P

* cargo fmt
* feat(databend sink): service use databend-client

* fix: better docs for config

* fix: update stage attachment ref in docs

* chore: add changelog file

* chore: update license file
…orm when all_metrics=true (vectordotdev#20228)

* Fixing bug - set missing in log-to-metric transform

* Adding a test case

* Adding unit test

* Create 20228_fix-log-to-metric-set-supported.fix.md

* Update 20228_fix-log-to-metric-set-supported.fix.md

* Update changelog.d/20228_fix-log-to-metric-set-supported.fix.md

Co-authored-by: Jesse Szwedko <[email protected]>

* Update changelog.d/20228_fix-log-to-metric-set-supported.fix.md

Co-authored-by: Jesse Szwedko <[email protected]>

* cargo fmt

Signed-off-by: Jesse Szwedko <[email protected]>

---------

Signed-off-by: Jesse Szwedko <[email protected]>
Co-authored-by: Jesse Szwedko <[email protected]>
Co-authored-by: Jesse Szwedko <[email protected]>
…al metrics (vectordotdev#20193)

* Adds test illustrating the incorrect result of a mix of absolute and incremental gauges.

* track all values through metric normalizer when converting to absolute

Signed-off-by: Luke Steensen <[email protected]>

* inline normalizer logic to share state

Signed-off-by: Luke Steensen <[email protected]>

* Revert "track all values through metric normalizer when converting to absolute"

This reverts commit 173f88f.

Signed-off-by: Luke Steensen <[email protected]>

* add changelog entry

Signed-off-by: Luke Steensen <[email protected]>

---------

Signed-off-by: Luke Steensen <[email protected]>
Co-authored-by: Russell Rollins <[email protected]>
…ev#20220)

Bumps [security-framework](https://github.com/kornelski/rust-security-framework) from 2.9.2 to 2.10.0.
- [Release notes](https://github.com/kornelski/rust-security-framework/releases)
- [Commits](kornelski/rust-security-framework@v2.9.2...v2.10.0)

---
updated-dependencies:
- dependency-name: security-framework
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the aws group with 1 update: [aws-smithy-async](https://github.com/smithy-lang/smithy-rs).


Updates `aws-smithy-async` from 1.2.0 to 1.2.1
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

---
updated-dependencies:
- dependency-name: aws-smithy-async
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: aws
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ordotdev#20238)

Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.30.0 to 1.30.1.
- [Release notes](https://github.com/bufbuild/buf-setup-action/releases)
- [Commits](bufbuild/buf-setup-action@v1.30.0...v1.30.1)

---
updated-dependencies:
- dependency-name: bufbuild/buf-setup-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [cached](https://github.com/jaemk/cached) from 0.49.2 to 0.49.3.
- [Changelog](https://github.com/jaemk/cached/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jaemk/cached/commits)

---
updated-dependencies:
- dependency-name: cached
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [mock_instant](https://github.com/museun/mock_instant) from 0.3.2 to 0.4.0.
- [Commits](museun/mock_instant@v0.3.2...v0.4.0)

---
updated-dependencies:
- dependency-name: mock_instant
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [warp](https://github.com/seanmonstar/warp) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/seanmonstar/warp/releases)
- [Changelog](https://github.com/seanmonstar/warp/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/warp@v0.3.6...v0.3.7)

---
updated-dependencies:
- dependency-name: warp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [crc](https://github.com/mrhooray/crc-rs) from 3.0.1 to 3.2.1.
- [Release notes](https://github.com/mrhooray/crc-rs/releases)
- [Commits](mrhooray/crc-rs@3.0.1...3.2.1)

---
updated-dependencies:
- dependency-name: crc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [getrandom](https://github.com/rust-random/getrandom) from 0.2.12 to 0.2.14.
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md)
- [Commits](rust-random/getrandom@v0.2.12...v0.2.14)

---
updated-dependencies:
- dependency-name: getrandom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the prost group with 4 updates: [prost](https://github.com/tokio-rs/prost), [prost-reflect](https://github.com/andrewhickman/prost-reflect), [prost-types](https://github.com/tokio-rs/prost) and [prost-build](https://github.com/tokio-rs/prost).


Updates `prost` from 0.12.3 to 0.12.4
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](tokio-rs/prost@v0.12.3...v0.12.4)

Updates `prost-reflect` from 0.13.0 to 0.13.1
- [Changelog](https://github.com/andrewhickman/prost-reflect/blob/main/CHANGELOG.md)
- [Commits](andrewhickman/prost-reflect@0.13.0...0.13.1)

Updates `prost-types` from 0.12.3 to 0.12.4
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](tokio-rs/prost@v0.12.3...v0.12.4)

Updates `prost-build` from 0.12.3 to 0.12.4
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](tokio-rs/prost@v0.12.3...v0.12.4)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prost
- dependency-name: prost-reflect
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prost
- dependency-name: prost-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prost
- dependency-name: prost-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prost
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…rdotdev#20260)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This was accidentally dropped in e9815e1 resulting in the SHA256SUMS not being uploaded to the
GitHub releases.

Signed-off-by: Jesse Szwedko <[email protected]>
@andrewma2 andrewma2 force-pushed the andrewma2/POCEventLogs branch 5 times, most recently from 6c7315e to 2311679 Compare August 19, 2024 18:30
@andrewma2 andrewma2 force-pushed the andrewma2/POCEventLogs branch 2 times, most recently from 814402a to e26fa44 Compare August 23, 2024 22:04
@andrewma2
Copy link
Collaborator Author

Closing this to rebase it onto v0.39

@andrewma2 andrewma2 closed this Aug 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.