-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: kapinger use service IP instead of name #1283
base: main
Are you sure you want to change the base?
Conversation
What was the issue with CoreDNS? Have we informed the correct teams of a possible issue with CoreDNS? |
@@ -5,6 +5,7 @@ import ( | |||
"fmt" | |||
"log" | |||
"net/http" | |||
_ "net/http/pprof" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why import this? Comment for future users?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@huntergregory that's fairly common with pprof, but also not my most favorite thing. It uses init()
to attach pprof handlers to the default net/http
handler. Probably fine here, where it's for testing, but for production pprof handlers I usually like to explicitly attach those handlers to my routes. This is documented in the package docs here: https://pkg.go.dev/runtime/pprof .
@@ -3,10 +3,12 @@ module github.com/microsoft/retina/hack/tools/kapinger | |||
go 1.22.5 | |||
|
|||
require ( | |||
golang.org/x/exp v0.0.0-20250128182459-e0ece0dbea4c |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
non-blocking: I'm not a fan of the experimental library because it makes breaking changes all the time
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From that repo:
Warning: Packages here are experimental and unreliable. Some may one day be promoted to the main repository or other subrepository, or they may be modified arbitrarily or even disappear altogether.
We probably should make attempts to just move away from this entirely.
@@ -41,7 +41,7 @@ func NewKapingerHTTPClient(clientset *kubernetes.Clientset, labelselector string | |||
Transport: &http.Transport{ | |||
DisableKeepAlives: true, | |||
}, | |||
Timeout: 3 * time.Second, | |||
Timeout: 15 * time.Second, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if there's a networkpolicy dropping traffic in our testing? This change would reduce that amount of traffic by 80%?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perhaps make this timeout configurable?
for i := 0; i < k.volume; i++ { | ||
for _, url := range k.urls { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we killing the TARGET_TYPE=pod
functionality? I'd remove any dead code or add comments to it to fix later
retina/hack/tools/kapinger/clients/http.go
Lines 61 to 73 in d1378d1
switch k.targettype { | |
case Service: | |
k.urls, err = k.getServiceURLs() | |
if err != nil { | |
return nil, fmt.Errorf("error getting service URLs: %w", err) | |
} | |
case Pod: | |
k.urls, err = k.getPodURLs() | |
default: | |
return nil, fmt.Errorf("env TARGET_TYPE must be \"service\" or \"pod\"") | |
} |
Description
CoreDNS wasn't playing nice with massive bursts of service name fqdn lookups, this moves to using the ip of the service(s) directly
Please provide a brief description of the changes made in this pull request.
Related Issue
If this pull request is related to any issue, please mention it here. Additionally, make sure that the issue is assigned to you before submitting this pull request.
Checklist
git commit -S -s ...
). See this documentation on signing commits.Screenshots (if applicable) or Testing Completed
Please add any relevant screenshots or GIFs to showcase the changes made.
Additional Notes
Add any additional notes or context about the pull request here.
Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.