Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(telemetry)_: replace telemetry with prometheus metrics #6256

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

adklempner
Copy link
Contributor

@adklempner adklempner commented Jan 15, 2025

Replace telemetry with local metrics using prometheus. Add parameters to InitializeApplication for enabling waku metrics over prometheus and specifying which port to use.

This commit removes the telemetry functionality with a Prometheus client. Most of the metrics that were collected by telemetry now have their corresponding Prometheus gauges, counters, and histograms.

They still require a telemetry url to be set in order to be enabled. Additionally, the parameter WakuMetricsEnabled needs to be set as true in the request for InitializeApplication in order to start Prometheus at port 9305 (can be changed using WakuMetricsPort).

status-desktop Dogfooding PR: status-im/status-desktop#17020

@status-im-auto
Copy link
Member

status-im-auto commented Jan 15, 2025

Jenkins Builds

Click to see older builds (32)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 7cf1f93 #1 2025-01-15 21:30:58 ~3 min macos 📦zip
✖️ 7cf1f93 #1 2025-01-15 21:31:03 ~4 min tests 📄log
✔️ 7cf1f93 #1 2025-01-15 21:31:28 ~4 min ios 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:32:17 ~5 min linux 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:32:30 ~5 min android 📦aar
✔️ 7cf1f93 #1 2025-01-15 21:32:32 ~5 min macos 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:32:38 ~5 min windows 📦zip
✔️ 7cf1f93 #1 2025-01-15 21:33:32 ~6 min tests-rpc 📄log
✖️ 9c43971 #2 2025-01-16 00:30:24 ~2 min tests 📄log
✔️ 9c43971 #2 2025-01-16 00:31:32 ~3 min windows 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:11 ~4 min macos 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:28 ~4 min linux 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:43 ~5 min ios 📦zip
✔️ 9c43971 #2 2025-01-16 00:32:57 ~5 min macos 📦zip
✔️ 9c43971 #2 2025-01-16 00:33:15 ~5 min android 📦aar
✔️ 9c43971 #2 2025-01-16 00:34:15 ~6 min tests-rpc 📄log
✔️ 7c4ff40 #3 2025-01-16 00:52:34 ~3 min windows 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:52:55 ~4 min macos 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:53:07 ~4 min ios 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:53:20 ~4 min linux 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:53:45 ~5 min macos 📦zip
✔️ 7c4ff40 #3 2025-01-16 00:54:09 ~5 min android 📦aar
✔️ 7c4ff40 #3 2025-01-16 00:54:41 ~6 min tests-rpc 📄log
✖️ 7c4ff40 #3 2025-01-16 01:18:34 ~29 min tests 📄log
✔️ 3763072 #4 2025-01-16 01:28:53 ~3 min windows 📦zip
✔️ 3763072 #4 2025-01-16 01:29:24 ~4 min macos 📦zip
✔️ 3763072 #4 2025-01-16 01:29:40 ~4 min linux 📦zip
✔️ 3763072 #4 2025-01-16 01:29:49 ~4 min ios 📦zip
✔️ 3763072 #4 2025-01-16 01:30:11 ~5 min macos 📦zip
✔️ 3763072 #4 2025-01-16 01:31:02 ~6 min android 📦aar
✔️ 3763072 #4 2025-01-16 01:31:19 ~6 min tests-rpc 📄log
✔️ 3763072 #4 2025-01-16 01:56:45 ~31 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 5f8d5f6 #5 2025-01-16 22:52:02 ~3 min windows 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:53:00 ~4 min linux 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:53:17 ~5 min ios 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:53:29 ~5 min macos 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:54:17 ~6 min macos 📦zip
✔️ 5f8d5f6 #5 2025-01-16 22:54:32 ~6 min android 📦aar
✔️ 5f8d5f6 #5 2025-01-16 22:54:33 ~6 min tests-rpc 📄log
✖️ 5f8d5f6 #5 2025-01-16 23:18:05 ~29 min tests 📄log
✔️ 07798db #6 2025-01-16 23:05:31 ~3 min windows 📦zip
✔️ 07798db #6 2025-01-16 23:06:31 ~4 min linux 📦zip
✔️ 07798db #6 2025-01-16 23:07:07 ~5 min macos 📦zip
✔️ 07798db #6 2025-01-16 23:07:13 ~5 min ios 📦zip
✔️ 07798db #6 2025-01-16 23:07:28 ~5 min android 📦aar
✔️ 07798db #6 2025-01-16 23:07:35 ~5 min macos 📦zip
✖️ 07798db #6 2025-01-16 23:08:04 ~6 min tests-rpc 📄log
✖️ 07798db #6 2025-01-16 23:48:08 ~29 min tests 📄log

@adklempner adklempner force-pushed the feat/replace-telemetry-prometheus branch 3 times, most recently from 7c4ff40 to 3763072 Compare January 16, 2025 01:24
Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 37.43017% with 112 lines in your changes missing coverage. Please review.

Project coverage is 61.41%. Comparing base (4a5a338) to head (3763072).

Files with missing lines Patch % Lines
metrics/wakumetrics/client.go 44.21% 53 Missing ⚠️
wakuv2/waku.go 11.53% 19 Missing and 4 partials ⚠️
protocol/messenger.go 21.42% 10 Missing and 1 partial ⚠️
mobile/status.go 0.00% 9 Missing and 1 partial ⚠️
metrics/metrics.go 0.00% 8 Missing ⚠️
wakuv2/message_publishing.go 0.00% 2 Missing and 1 partial ⚠️
metrics/wakumetrics/metrics.go 89.47% 1 Missing and 1 partial ⚠️
protocol/messenger_pairing_and_syncing.go 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #6256      +/-   ##
===========================================
- Coverage    61.55%   61.41%   -0.15%     
===========================================
  Files          842      843       +1     
  Lines       110604   110400     -204     
===========================================
- Hits         68086    67799     -287     
- Misses       34562    34640      +78     
- Partials      7956     7961       +5     
Flag Coverage Δ
functional 21.55% <3.91%> (+0.04%) ⬆️
unit 59.93% <37.43%> (-0.16%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
protocol/requests/initialize_application.go 50.00% <ø> (ø)
wakuv2/common/metrics.go 100.00% <100.00%> (ø)
metrics/wakumetrics/metrics.go 89.47% <89.47%> (ø)
protocol/messenger_pairing_and_syncing.go 59.00% <0.00%> (ø)
wakuv2/message_publishing.go 72.30% <0.00%> (-9.24%) ⬇️
metrics/metrics.go 0.00% <0.00%> (ø)
mobile/status.go 9.66% <0.00%> (-0.08%) ⬇️
protocol/messenger.go 63.76% <21.42%> (-0.29%) ⬇️
wakuv2/waku.go 63.63% <11.53%> (-5.36%) ⬇️
metrics/wakumetrics/client.go 44.21% <44.21%> (ø)

... and 41 files with indirect coverage changes

@adklempner adklempner requested a review from a team January 16, 2025 16:39
)

var (
MessagesSentTotal = prometheus.NewHistogramVec(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is SomethingSomethingTotal a histogram? Shouldn't it be just a normal counter? How do you plan to use it as histogram?

},
)

PeersByOrigin = prometheus.NewGaugeVec(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these currently connected peers or overall number? Cause I'd expect PeersByOrigin to be only increasing as new are discovered (i.e. Counter)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is based on currently connected peers obtained by periodically looking at Waku's peer store

@adklempner adklempner force-pushed the feat/replace-telemetry-prometheus branch from 3763072 to 5f8d5f6 Compare January 16, 2025 22:47
Replace telemetry with local metrics using prometheus client.
Add parameters to InitializeApplication for enabling waku metrics
over prometheus and specifying which port to use.
@adklempner adklempner force-pushed the feat/replace-telemetry-prometheus branch from 5f8d5f6 to 07798db Compare January 16, 2025 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants