Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cgtracker test failure #3467

Open
kkourt opened this issue Mar 5, 2025 · 0 comments
Open

cgtracker test failure #3467

kkourt opened this issue Mar 5, 2025 · 0 comments
Labels
kind/ci-flake A flake in CI

Comments

@kkourt
Copy link
Contributor

kkourt commented Mar 5, 2025

Ubuntu on ARM (https://github.com/cilium/tetragon/actions/runs/13673348873/job/38228265809):

    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Loaded generic kprobe program: /home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_generic_kprobe_v61.o -> __arm64_sys_lseek"
    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Loaded BPF maps and events for sensor successfully" maps="[Map{Name:fdinstall_map PinPath:mynamespace:lseek-test/generic_kprobe/fdinstall_map Owner:true} Map{Name:config_map PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/config_map Owner:true} Map{Name:kprobe_calls PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/kprobe_calls Owner:true} Map{Name:filter_map PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/filter_map Owner:true} Map{Name:argfilter_maps PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/argfilter_maps Owner:true} Map{Name:addr4lpm_maps PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/addr4lpm_maps Owner:true} Map{Name:addr6lpm_maps PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/addr6lpm_maps Owner:true} Map{Name:string_maps_0 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_0 Owner:true} Map{Name:string_maps_1 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_1 Owner:true} Map{Name:string_maps_2 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_2 Owner:true} Map{Name:string_maps_3 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_3 Owner:true} Map{Name:string_maps_4 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_4 Owner:true} Map{Name:string_maps_5 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_5 Owner:true} Map{Name:string_maps_6 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_6 Owner:true} Map{Name:string_maps_7 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_7 Owner:true} Map{Name:string_maps_8 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_8 Owner:true} Map{Name:string_maps_9 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_9 Owner:true} Map{Name:string_maps_10 PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_maps_10 Owner:true} Map{Name:string_prefix_maps PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_prefix_maps Owner:true} Map{Name:string_postfix_maps PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/string_postfix_maps Owner:true} Map{Name:retprobe_map PinPath:mynamespace:lseek-test/generic_kprobe/retprobe_map Owner:true} Map{Name:process_call_heap PinPath:mynamespace:lseek-test/generic_kprobe/process_call_heap Owner:true} Map{Name:tg_mb_sel_opts PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/tg_mb_sel_opts Owner:true} Map{Name:tg_mb_paths PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/tg_mb_paths Owner:true} Map{Name:stack_trace_map PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/stack_trace_map Owner:true} Map{Name:socktrack_map PinPath:mynamespace:lseek-test/generic_kprobe/socktrack_map Owner:true} Map{Name:ratelimit_map PinPath:mynamespace:lseek-test/generic_kprobe/ratelimit_map Owner:true} Map{Name:tg_cgtracker_map PinPath:tg_cgtracker_map Owner:false} Map{Name:override_tasks PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek/override_tasks Owner:true} Map{Name:policy_conf PinPath:mynamespace:lseek-test/policy_conf Owner:true} Map{Name:execve_map PinPath:execve_map Owner:false}]" progs="[Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_generic_kprobe_v61.o Attach:__arm64_sys_lseek Label:kprobe/generic_kprobe PinPath:mynamespace:lseek-test/generic_kprobe/__arm64_sys_lseek}]" sensor=generic_kprobe
    cgroup_fs.go:21: '/sys/fs/cgroup' is cgroup v2
    cgtracker_test.go:321: created cgroup dir '/sys/fs/cgroup/cgtracker-test-3541718773'
    cgtracker_test.go:357: cgid for a is 8917
    cgtracker_test.go:357: cgid for a/x is 8935
    cgtracker_test.go:357: cgid for b is 8953
    cgtracker_test.go:357: cgid for b/y is 8971
    tester.go:147: tester stderr> runtime: marked free object in span 0xff105073fd98, elemsize=208 freeindex=32 (bad use of unsafe.Pointer or having race conditions? try -d=checkptr or -race)
    tester.go:147: tester stderr> 0x40004aa000 alloc marked  
    tester.go:147: tester stderr> 0x40004aa0d0 alloc marked  
    tester.go:147: tester stderr> 0x40004aa1a0 alloc marked  
    tester.go:147: tester stderr> 0x40004aa270 alloc marked  
    tester.go:147: tester stderr> 0x40004aa340 alloc marked  
    tester.go:147: tester stderr> 0x40004aa410 alloc marked  
    tester.go:147: tester stderr> 0x40004aa4e0 alloc marked  
    tester.go:147: tester stderr> 0x40004aa5b0 alloc marked  
    tester.go:147: tester stderr> 0x40004aa680 alloc marked  
    tester.go:147: tester stderr> 0x40004aa750 alloc marked  
    tester.go:147: tester stderr> 0x40004aa820 alloc marked  
    tester.go:147: tester stderr> 0x40004aa8f0 alloc marked  
    tester.go:147: tester stderr> 0x40004aa9c0 alloc marked  
    tester.go:147: tester stderr> 0x40004aaa90 alloc marked  
    tester.go:147: tester stderr> 0x40004aab60 alloc marked  
    tester.go:147: tester stderr> 0x40004aac30 alloc marked  
    tester.go:147: tester stderr> 0x40004aad00 alloc marked  
    tester.go:147: tester stderr> 0x40004aadd0 alloc marked  
    tester.go:147: tester stderr> 0x40004aaea0 alloc marked  
    tester.go:147: tester stderr> 0x40004aaf70 alloc marked  
    tester.go:147: tester stderr> 0x40004ab040 alloc marked  
    tester.go:147: tester stderr> 0x40004ab110 alloc marked  
    tester.go:147: tester stderr> 0x40004ab1e0 alloc marked  
    tester.go:147: tester stderr> 0x40004ab2b0 alloc marked  
    tester.go:147: tester stderr> 0x40004ab380 alloc marked  
    tester.go:147: tester stderr> 0x40004ab450 alloc marked  
    tester.go:147: tester stderr> 0x40004ab520 alloc marked  
    tester.go:147: tester stderr> 0x40004ab5f0 alloc unmarked
    tester.go:147: tester stderr> 0x40004ab6c0 alloc unmarked
    tester.go:147: tester stderr> 0x40004ab790 alloc marked  
    tester.go:147: tester stderr> 0x40004ab860 alloc marked  
    tester.go:147: tester stderr> 0x40004ab930 alloc marked  
    tester.go:147: tester stderr> 0x40004aba00 free  marked   zombie
    tester.go:147: tester stderr> 0x00000040004aba00:  0x0000016900000002  0x0000000000000010 
    tester.go:147: tester stderr> 0x00000040004aba10:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004aba20:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004aba30:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004aba40:  0x0000016a00000007  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004aba50:  0x0000004000462a00  0x0000000000000526 
    tester.go:147: tester stderr> 0x00000040004aba60:  0x0000000000000540  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004aba70:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004aba80:  0x000001ea00000002  0x0000000000000011 
    tester.go:147: tester stderr> 0x00000040004aba90:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004abaa0:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004abab0:  0x0000000000000000  0x0000000000000000 
    tester.go:147: tester stderr> 0x00000040004abac0:  0x0000016c00000002  0x0000000000000010 
    tester.go:147: tester stderr> 0x40004abad0 free  unmarked
    tester.go:147: tester stderr> 0x40004abba0 free  unmarked
    tester.go:147: tester stderr> 0x40004abc70 free  unmarked
    tester.go:147: tester stderr> 0x40004abd40 free  unmarked
    tester.go:147: tester stderr> 0x40004abe10 free  unmarked
    tester.go:147: tester stderr> fatal error: found pointer to free object
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> runtime stack:
    tester.go:147: tester stderr> runtime.throw({0x19c081c?, 0x40004abad0?})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/panic.go:1096 +0x38 fp=0xff104de0e360 sp=0xff104de0e330 pc=0x865f8
    tester.go:147: tester stderr> runtime.(*mspan).reportZombies(0xff105073fd98)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgcsweep.go:877 +0x2f0 fp=0xff104de0e3e0 sp=0xff104de0e360 pc=0x3e0e0
    tester.go:147: tester stderr> runtime.(*sweepLocked).sweep(0x0?, 0x0)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgcsweep.go:652 +0x414 fp=0xff104de0e500 sp=0xff104de0e3e0 pc=0x3d094
    tester.go:147: tester stderr> runtime.(*mcentral).uncacheSpan(0x0?, 0x0?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mcentral.go:236 +0xac fp=0xff104de0e530 sp=0xff104de0e500 pc=0x2c5ec
    tester.go:147: tester stderr> runtime.(*mcache).releaseAll(0xff10972205c0)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mcache.go:292 +0x190 fp=0xff104de0e5a0 sp=0xff104de0e530 pc=0x2be40
    tester.go:147: tester stderr> runtime.(*mcache).prepareForSweep(0xff10972205c0)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mcache.go:329 +0x4c fp=0xff104de0e5d0 sp=0xff104de0e5a0 pc=0x2bf5c
    tester.go:147: tester stderr> runtime.procresize(0x2)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:5795 +0x57c fp=0xff104de0e6b0 sp=0xff104de0e5d0 pc=0x5f6bc
    tester.go:147: tester stderr> runtime.startTheWorldWithSema(0x9e18962857, {0xd8?, 0x8ea88?, 0x2a7?, 0x10862f7?})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:1692 +0xf4 fp=0xff104de0e720 sp=0xff104de0e6b0 pc=0x55de4
    tester.go:147: tester stderr> runtime.gcMarkTermination.func3()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1186 +0x30 fp=0xff104de0e760 sp=0xff104de0e720 pc=0x32510
    tester.go:147: tester stderr> runtime.systemstack(0x0)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:244 +0x6c fp=0xff104de0e770 sp=0xff104de0e760 pc=0x8b53c
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 5 gp=0x4000003880 m=6 mp=0x400038d008 [running]:
    tester.go:147: tester stderr> runtime.systemstack_switch()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:201 +0x8 fp=0x4000086440 sp=0x4000086430 pc=0x8b4b8
    tester.go:147: tester stderr> runtime.gcMarkTermination({0x0?, 0x9e18932dd8?, 0x2?, 0x20?})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1176 +0x590 fp=0x4000086670 sp=0x4000086440 pc=0x31cf0
    tester.go:147: tester stderr> runtime.gcMarkDone()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1014 +0x340 fp=0x4000086710 sp=0x4000086670 pc=0x314f0
    tester.go:147: tester stderr> runtime.gcBgMarkWorker(0x4000101f80)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1559 +0x384 fp=0x40000867b0 sp=0x4000086710 pc=0x32bd4
    tester.go:147: tester stderr> runtime.gcBgMarkStartWorkers.gowrap1()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1339 +0x28 fp=0x40000867d0 sp=0x40000867b0 pc=0x32818
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40000867d0 sp=0x40000867d0 pc=0x8d9e4
    tester.go:147: tester stderr> created by runtime.gcBgMarkStartWorkers in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1339 +0x140
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 1 gp=0x40000021c0 m=nil [runnable, locked to thread]:
    tester.go:147: tester stderr> encoding/json.stateEndValue(0x400044b600, 0x2c?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/encoding/json/scanner.go:279 +0x640 fp=0x40000dd1c0 sp=0x40000dd1c0 pc=0x2161e0
    tester.go:147: tester stderr> encoding/json.(*Decoder).readValue(0x400044b540)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/encoding/json/stream.go:103 +0x11c fp=0x40000dd210 sp=0x40000dd1c0 pc=0x21895c
    tester.go:147: tester stderr> encoding/json.(*Decoder).Decode(0x400044b540, {0x159a560, 0x400078a630})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/encoding/json/stream.go:63 +0x5c fp=0x40000dd240 sp=0x40000dd210 pc=0x2185ac
    tester.go:147: tester stderr> sigs.k8s.io/yaml.jsonUnmarshal({0x1cc6780, 0x400073ac00}, {0x19582c0, 0x40000fea08}, {0x0, 0x0, 0x40000dd308?})
    tester.go:147: tester stderr> 	/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/vendor/sigs.k8s.io/yaml/yaml.go:94 +0xe8 fp=0x40000dd2a0 sp=0x40000dd240 pc=0x6dedd8
    tester.go:147: tester stderr> sigs.k8s.io/yaml.unmarshal({0x2d9d620?, 0x7e31c?, 0x40000dd398?}, {0x19582c0, 0x40000fea08}, 0x7e31c?, {0x0, 0x0, 0x0})
    tester.go:147: tester stderr> 	/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/vendor/sigs.k8s.io/yaml/yaml.go:77 +0x158 fp=0x40000dd340 sp=0x40000dd2a0 pc=0x6dec68
    tester.go:147: tester stderr> sigs.k8s.io/yaml.Unmarshal(...)
    tester.go:147: tester stderr> 	/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/vendor/sigs.k8s.io/yaml/yaml.go:56
    tester.go:147: tester stderr> github.com/cilium/tetragon/pkg/k8s/crdutils.NewCRDBytes({_, _}, {_, _}, {_, _, _})
    tester.go:147: tester stderr> 	/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/vendor/github.com/cilium/tetragon/pkg/k8s/crdutils/register.go:61 +0x94 fp=0x40000ddb40 sp=0x40000dd340 pc=0x1397594
    tester.go:147: tester stderr> github.com/cilium/tetragon/pkg/k8s/apis/cilium.io/client.init()
    tester.go:147: tester stderr> 	/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/vendor/github.com/cilium/tetragon/pkg/k8s/apis/cilium.io/client/register.go:17 +0x54 fp=0x40000dde10 sp=0x40000ddb40 pc=0x13978b4
    tester.go:147: tester stderr> runtime.doInit1(0x2d12970)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:7350 +0xd4 fp=0x40000ddf40 sp=0x40000dde10 pc=0x62834
    tester.go:147: tester stderr> runtime.doInit(...)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:7317
    tester.go:147: tester stderr> runtime.main()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:254 +0x340 fp=0x40000ddfd0 sp=0x40000ddf40 pc=0x52b80
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40000ddfd0 sp=0x40000ddfd0 pc=0x8d9e4
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 2 gp=0x4000002c40 m=nil [force gc (idle)]:
    tester.go:147: tester stderr> runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:435 +0xc8 fp=0x4000084f90 sp=0x4000084f70 pc=0x86718
    tester.go:147: tester stderr> runtime.goparkunlock(...)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:441
    tester.go:147: tester stderr> runtime.forcegchelper()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:348 +0xb8 fp=0x4000084fd0 sp=0x4000084f90 pc=0x52e18
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x4000084fd0 sp=0x4000084fd0 pc=0x8d9e4
    tester.go:147: tester stderr> created by runtime.init.7 in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:336 +0x24
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 3 gp=0x4000003180 m=nil [runnable]:
    tester.go:147: tester stderr> runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:435 +0xc8 fp=0x4000085760 sp=0x4000085740 pc=0x86718
    tester.go:147: tester stderr> runtime.goparkunlock(...)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:441
    tester.go:147: tester stderr> runtime.bgsweep(0x4000062080)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgcsweep.go:316 +0x108 fp=0x40000857b0 sp=0x4000085760 pc=0x3c668
    tester.go:147: tester stderr> runtime.gcenable.gowrap1()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:204 +0x28 fp=0x40000857d0 sp=0x40000857b0 pc=0x30498
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40000857d0 sp=0x40000857d0 pc=0x8d9e4
    tester.go:147: tester stderr> created by runtime.gcenable in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:204 +0x6c
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 4 gp=0x4000003340 m=nil [GC scavenge wait]:
    tester.go:147: tester stderr> runtime.gopark(0x10000?, 0x1cbd3f8?, 0x0?, 0x0?, 0x0?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:435 +0xc8 fp=0x4000085f60 sp=0x4000085f40 pc=0x86718
    tester.go:147: tester stderr> runtime.goparkunlock(...)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:441
    tester.go:147: tester stderr> runtime.(*scavengerState).park(0x2e44da0)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgcscavenge.go:425 +0x5c fp=0x4000085f90 sp=0x4000085f60 pc=0x3a12c
    tester.go:147: tester stderr> runtime.bgscavenge(0x4000062080)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgcscavenge.go:658 +0xac fp=0x4000085fb0 sp=0x4000085f90 pc=0x3a6ac
    tester.go:147: tester stderr> runtime.gcenable.gowrap2()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:205 +0x28 fp=0x4000085fd0 sp=0x4000085fb0 pc=0x30438
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x4000085fd0 sp=0x4000085fd0 pc=0x8d9e4
    tester.go:147: tester stderr> created by runtime.gcenable in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:205 +0xac
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 18 gp=0x4000104380 m=nil [finalizer wait]:
    tester.go:147: tester stderr> runtime.gopark(0x16f28c0?, 0xc?, 0xc0?, 0x5?, 0x1000000000000?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:435 +0xc8 fp=0x4000084590 sp=0x4000084570 pc=0x86718
    tester.go:147: tester stderr> runtime.runfinq()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mfinal.go:196 +0x108 fp=0x40000847d0 sp=0x4000084590 pc=0x2f428
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40000847d0 sp=0x40000847d0 pc=0x8d9e4
    tester.go:147: tester stderr> created by runtime.createfing in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mfinal.go:166 +0x80
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 19 gp=0x4000105a40 m=nil [chan receive]:
    tester.go:147: tester stderr> runtime.gopark(0x400033e280?, 0x400000fed8?, 0x48?, 0x7?, 0x1578b8?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:435 +0xc8 fp=0x40000806f0 sp=0x40000806d0 pc=0x86718
    tester.go:147: tester stderr> runtime.chanrecv(0x4000100fc0, 0x0, 0x1)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/chan.go:664 +0x42c fp=0x4000080770 sp=0x40000806f0 pc=0x219dc
    tester.go:147: tester stderr> runtime.chanrecv1(0x0?, 0x0?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/chan.go:506 +0x14 fp=0x40000807a0 sp=0x4000080770 pc=0x21574
    tester.go:147: tester stderr> runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1796
    tester.go:147: tester stderr> runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1799 +0x3c fp=0x40000807d0 sp=0x40000807a0 pc=0x336bc
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40000807d0 sp=0x40000807d0 pc=0x8d9e4
    tester.go:147: tester stderr> created by unique.runtime_registerUniqueMapCleanup in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1794 +0x78
    tester.go:147: tester stderr> 
    tester.go:147: tester stderr> goroutine 20 gp=0x400044e540 m=nil [GC worker (idle)]:
    tester.go:147: tester stderr> runtime.gopark(0x9e188f91a0?, 0x0?, 0x0?, 0x0?, 0x0?)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/proc.go:435 +0xc8 fp=0x4000080f10 sp=0x4000080ef0 pc=0x86718
    tester.go:147: tester stderr> runtime.gcBgMarkWorker(0x4000101f80)
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1423 +0xdc fp=0x4000080fb0 sp=0x4000080f10 pc=0x3292c
    tester.go:147: tester stderr> runtime.gcBgMarkStartWorkers.gowrap1()
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1339 +0x28 fp=0x4000080fd0 sp=0x4000080fb0 pc=0x32818
    tester.go:147: tester stderr> runtime.goexit({})
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/asm_arm64.s:1223 +0x4 fp=0x4000080fd0 sp=0x4000080fd0 pc=0x8d9e4
    tester.go:147: tester stderr> created by runtime.gcBgMarkStartWorkers in goroutine 1
    tester.go:147: tester stderr> 	/opt/hostedtoolcache/go/1.24.0/arm64/src/runtime/mgc.go:1339 +0x140
    cgtracker_test.go:284: 
        	Error Trace:	/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/pkg/cgtracker/test/cgtracker_test.go:284
        	Error:      	Received unexpected error:
        	            	failed to read output: EOF
        	Test:       	TestCgTrackerPolicyFilter
    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Unloading sensor test-sensor-2"
    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Sensor unloaded" maps="[]" maps-error="[]" progs="[Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_lseek.o Attach:syscalls/sys_enter_lseek Label:tracepoint/sys_enter_lseek PinPath:test-sensor-2/test-sensor-2-test_lseek_prog}]"
    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Unloading sensor __base__"
    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Sensor unloaded" maps="[Map{Name:execve_map PinPath:execve_map Owner:true} Map{Name:tg_execve_joined_info_map PinPath:tg_execve_joined_info_map Owner:true} Map{Name:execve_map_stats PinPath:execve_map_stats Owner:true} Map{Name:tg_execve_joined_info_map_stats PinPath:tg_execve_joined_info_map_stats Owner:true} Map{Name:execve_calls PinPath:__base__/event_execve/execve_calls Owner:true} Map{Name:tcpmon_map PinPath:tcpmon_map Owner:true} Map{Name:tg_conf_map PinPath:tg_conf_map Owner:true} Map{Name:tg_stats_map PinPath:tg_stats_map Owner:true} Map{Name:tg_mbset_map PinPath:tg_mbset_map Owner:true} Map{Name:tg_errmetrics_map PinPath:tg_errmetrics_map Owner:true} Map{Name:tg_cgtracker_map PinPath:tg_cgtracker_map Owner:true}]" maps-error="[]" progs="[Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_exit.o Attach:acct_process Label:kprobe/acct_process PinPath:__base__/event_exit} Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_fork.o Attach:wake_up_new_task Label:kprobe/wake_up_new_task PinPath:__base__/kprobe_pid_clear} Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_execve_event_v61.o Attach:sched/sched_process_exec Label:tracepoint/sys_execve PinPath:__base__/event_execve} Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_execve_bprm_commit_creds.o Attach:security_bprm_committing_creds Label:kprobe/security_bprm_committing_creds PinPath:__base__/tg_kp_bprm_committing_creds} Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_cgtracker.o Attach:cgroup/cgroup_mkdir Label:raw_tracepoint/cgroup_mkdir PinPath:__base__/tg_cgtracker_cgroup_mkdir} Program{Name:/home/runner/work/tetragon/tetragon/go/src/github.com/cilium/tetragon/bpf/objs/bpf_cgtracker.o Attach:cgroup/cgroup_release Label:raw_tracepoint/cgroup_release PinPath:__base__/tg_cgtracker_cgroup_release}]"
    base.go:206: cleanup: unloading base sensor
    logcapture.go:25: time="2025-03-05T10:10:05Z" level=info msg="Unloading sensor __base__"
FAIL
coverage: [no statements]
time="2025-03-05T10:10:05Z" level=info msg="`mynamespace:lseek-test` still exists after test"
time="2025-03-05T10:10:05Z" level=info msg="`policy_filter_cgroup_maps` still exists after test"
time="2025-03-05T10:10:05Z" level=info msg="`tg_cgroup_namespace_map` still exists after test"
time="2025-03-05T10:10:05Z" level=info msg="map dir `/sys/fs/bpf/testcgtracker-test` still exists after test. Removing it."
FAIL	github.com/cilium/tetragon/pkg/cgtracker/test	4.532s
FAIL
make: *** [Makefile:266: test] Error 1
Error: Process completed with exit code 2.
@kkourt kkourt changed the title CI failure cgtracker test failure Mar 5, 2025
@kkourt kkourt added the kind/ci-flake A flake in CI label Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/ci-flake A flake in CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kkourt