Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

panic: duplicate metrics collector registration attempted #6813

Open
wazerstar opened this issue Jan 28, 2025 · 5 comments · May be fixed by #6820
Open

panic: duplicate metrics collector registration attempted #6813

wazerstar opened this issue Jan 28, 2025 · 5 comments · May be fixed by #6820
Assignees
@mohammed90
Labels
bug 🐞 Something isn't working

Comments

@wazerstar
Copy link

Created latest development with xcaddy v0.4.4 on Windows 11 x64

xcaddy build master --with github.com/caddy-dns/cloudflare --with github.com/caddyserver/transform-encoder --with github.com/WeidiDeng/caddy-cloudflare-ip --with github.com/porech/caddy-maxmind-geolocation --with github.com/mholt/caddy-webdav

caddy run

2025/01/28 07:45:17.438 INFO    maxprocs: Leaving GOMAXPROCS=12: CPU quota undefined
2025/01/28 07:45:17.438 INFO    GOMEMLIMIT is updated   {"package": "github.com/KimMachineGun/automemlimit/memlimit", "GOMEMLIMIT": 61564166553, "previous": 9223372036854775807}
2025/01/28 07:45:17.452 INFO    using adjacent Caddyfile
2025/01/28 07:45:17.464 INFO    adapted config to JSON  {"adapter": "caddyfile"}
2025/01/28 07:45:17.464 WARN    Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies   {"adapter": "caddyfile", "file": "Caddyfile", "line": 201}
2025/01/28 07:45:17.480 INFO    admin   admin endpoint started  {"address": "localhost:2019", "enforce_origin": false, "origins": ["//127.0.0.1:2019", "//localhost:2019", "//[::1]:2019"]}
2025/01/28 07:45:17.480 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc0005ea380"}
2025/01/28 07:45:17.481 INFO    http.auto_https server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2025/01/28 07:45:17.481 INFO    http.auto_https automatic HTTP->HTTPS redirects are disabled    {"server_name": "srv0"}
2025/01/28 07:45:17.485 INFO    maxprocs: No GOMAXPROCS change to reset
panic: duplicate metrics collector registration attempted

goroutine 1 [running]:
github.com/prometheus/client_golang/prometheus.(*Registry).MustRegister(0xc0005f0a00, {0xc0007fa610?, 0x199dcb6?, 0xd?})        github.com/prometheus/[email protected]/prometheus/registry.go:405 +0x66
github.com/prometheus/client_golang/prometheus/promauto.Factory.NewGaugeVec({{0x1e4c9f0?, 0xc0005f0a00?}}, {{0x1988a69, 0x5}, {0x199dcb6, 0xd}, {0x19a7e20, 0x11}, {0x19e3598, 0x29}, ...}, ...)
        github.com/prometheus/[email protected]/prometheus/promauto/auto.go:308 +0x163
github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.initReverseProxyMetrics(0xc0007fc000, 0xc0005f0a00)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/reverseproxy/metrics.go:24 +0x7c
github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.newMetricsUpstreamsHealthyUpdater(0xc0007fc000, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x4, 0x4}, {0x0, 0x0, ...}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/reverseproxy/metrics.go:42 +0x65
github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.(*Handler).Provision(0xc0007fc000, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x4, 0x4}, {0x0, 0x0, ...}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/reverseproxy/reverseproxy.go:385 +0x12b8
github.com/caddyserver/caddy/v2.Context.LoadModuleByID({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x4, 0x4}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:391 +0x695
github.com/caddyserver/caddy/v2.Context.loadModuleInline({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:441 +0xf0
github.com/caddyserver/caddy/v2.Context.LoadModule({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:222 +0x706
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Route).ProvisionHandlers(0xc0004457a8, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, {0x0, 0x0, ...}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:154 +0x71
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.ProvisionHandlers({0xc000445688, 0x3, 0x29495bc2ea8?}, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:212 +0xa9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Provision({0xc000445688, 0x3, 0x4}, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:189 +0xb2
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Subroute).Provision(0xc0000833a0, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, {0x0, 0x0, ...}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/subroute.go:58 +0x5f
github.com/caddyserver/caddy/v2.Context.LoadModuleByID({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0002d06c0, 0x3, 0x4}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:391 +0x695
github.com/caddyserver/caddy/v2.Context.loadModuleInline({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:441 +0xf0
github.com/caddyserver/caddy/v2.Context.LoadModule({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:222 +0x706
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Route).ProvisionHandlers(0xc0004455b8, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, {0x0, 0x0, ...}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:154 +0x71
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.ProvisionHandlers({0xc000445408, 0x4, 0x29495bc2ea8?}, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:212 +0xa9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Provision({0xc000445408, 0x4, 0x4}, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:189 +0xb2
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Subroute).Provision(0xc000082e60, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, {0x0, 0x0, ...}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/subroute.go:58 +0x5f
github.com/caddyserver/caddy/v2.Context.LoadModuleByID({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc000082ea0, 0x2, 0x2}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:391 +0x695
github.com/caddyserver/caddy/v2.Context.loadModuleInline({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0009be400, 0x1, 0x1}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:441 +0xf0
github.com/caddyserver/caddy/v2.Context.LoadModule({{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0009be400, 0x1, 0x1}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:222 +0x706
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Route).ProvisionHandlers(0xc0002115b8, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0009be400, 0x1, 0x1}, {0x0, 0x0, ...}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:154 +0x71
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.ProvisionHandlers({0xc000211408, 0x6, 0x19a78e1?}, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0009be400, 0x1, 0x1}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/routes.go:212 +0xa9
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Provision(0xc000651110, {{0x1e4f7a8, 0xc00016ecf0}, 0xc0005eecc0, 0xc0001551a0, {0xc0009be400, 0x1, 0x1}, {0x0, 0x0, ...}, ...})
        github.com/caddyserver/caddy/[email protected]/modules/caddyhttp/app.go:367 +0xf25
github.com/caddyserver/caddy/v2.Context.LoadModuleByID({{0x1e4f7e0, 0xc0005f0a50}, 0xc0005eecc0, 0xc0001551a0, {0xc0009be400, 0x1, 0x1}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:391 +0x695
github.com/caddyserver/caddy/v2.Context.App({{0x1e4f7e0, 0xc0005f0a50}, 0xc0005eecc0, 0xc0001551a0, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, ...}, ...)
        github.com/caddyserver/caddy/[email protected]/context.go:467 +0xee
github.com/caddyserver/caddy/v2.provisionContext.func3(...)
        github.com/caddyserver/caddy/[email protected]/caddy.go:545
github.com/caddyserver/caddy/v2.provisionContext(0xc0005fb3e8?, 0x1)
        github.com/caddyserver/caddy/[email protected]/caddy.go:550 +0x53c
github.com/caddyserver/caddy/v2.run(0xc?, 0x1)
        github.com/caddyserver/caddy/[email protected]/caddy.go:400 +0x78
github.com/caddyserver/caddy/v2.unsyncedDecodeAndRun({0xc0005a8a80, 0x6714, 0x6a80}, 0x1)
        github.com/caddyserver/caddy/[email protected]/caddy.go:343 +0x145
github.com/caddyserver/caddy/v2.changeConfig({0x19870e5, 0x4}, {0x198df60, 0x7}, {0xc0005a2000, 0x6714, 0x6a80}, {0x0, 0x0}, 0x1)
        github.com/caddyserver/caddy/[email protected]/caddy.go:234 +0x6b6
github.com/caddyserver/caddy/v2.Load({0xc0005a2000?, 0x0?, 0x0?}, 0x0?)
        github.com/caddyserver/caddy/[email protected]/caddy.go:133 +0x114
github.com/caddyserver/caddy/v2/cmd.cmdRun({0x0?})
        github.com/caddyserver/caddy/[email protected]/cmd/commandfuncs.go:235 +0x831
github.com/caddyserver/caddy/v2/cmd.init.1.func2.WrapCommandFuncForCobra.1(0xc0000e4908, {0x1987145?, 0x4?, 0x1987115?})        github.com/caddyserver/caddy/[email protected]/cmd/cobra.go:141 +0x2f
github.com/spf13/cobra.(*Command).execute(0xc0000e4908, {0x2b441e0, 0x0, 0x0})
        github.com/spf13/[email protected]/command.go:985 +0xaaa
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000e4308)
        github.com/spf13/[email protected]/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/[email protected]/command.go:1041
github.com/caddyserver/caddy/v2/cmd.Main()
        github.com/caddyserver/caddy/[email protected]/cmd/main.go:96 +0x46b
main.main()
        caddy/main.go:16 +0xf
@mohammed90 mohammed90 self-assigned this Jan 28, 2025
@mohammed90 mohammed90 added the bug 🐞 Something isn't working label Jan 28, 2025
@mohammed90
Copy link
Member

mohammed90 commented Jan 28, 2025 •
edited
Loading

Sigh, I guess #6806 wasn't perfect. Can you share how to reproduce it? Because I did try multiple reloads and change of upstreams in my tests. It helps me to know what I missed.

Oh, and thanks for trying the bleeding edge!

@wazerstar
Copy link
Author

Sigh, I guess #6806 wasn't perfect. Can you share how to reproduce it? Because I did try multiple reloads and change of upstreams in my tests. It helps me to know what I missed.

Oh, and thanks for trying the bleeding edge!

I just narrowed it down to simple caddyfile having one reverse_proxy in in each and it will fail

192.168.1.2 {
	reverse_proxy 192.168.1.5 {
	}
}

192.168.1.3 {
	reverse_proxy 192.168.1.6 {
	}
}

@WJQSERVER

This comment has been minimized.

Sign in to view
@JeDaYoshi

This comment has been minimized.

Sign in to view
@AetherMagee

This comment has been minimized.

Sign in to view
@mohammed90 mohammed90 linked a pull request Jan 31, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mohammed90 mohammed90

Labels
bug 🐞 Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

reverseproxy: ignore duplicate collector registration error caddyserver/caddy
5 participants
@mohammed90 @wazerstar @JeDaYoshi @AetherMagee @WJQSERVER