Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parca-agent crashes at the start with BPF verifier error #2715

Open
shakhat opened this issue Apr 17, 2024 · 0 comments
Open

parca-agent crashes at the start with BPF verifier error #2715

shakhat opened this issue Apr 17, 2024 · 0 comments

Comments

@shakhat
Copy link

shakhat commented Apr 17, 2024

Describe the bug
Parca-agent crashes at the start with error message "load bpf program: failed to load BPF object: permission denied".
The issue occurs on main branch starting with commit 31bdda2

To Reproduce
Compile parca-agent from the source code and run as sudo ./dist/parca-agent

Logs

; LOG(\"[info] requesting that the user-space process attempt to fault in the memory at 0x%lx\", previous_rip_addr);
2128: (15) if r1 == 0x0 goto pc+5
last_idx 2128 first_idx 2128
 R0=inv(id=0) R1_rw=invP0 R2=inv0 R6=inv(id=0) R7_w=map_value(id=0,off=4,ks=4,vs=4151,imm=0) R8=inv(id=0) R9_w=inv4294967282 R10=fp0 fp-8=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-176=map_value fp-184=map_value fp-192=invP6 fp-200=map_value fp-208=map_value fp-216=ctx fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm fp-264=mmmmmmmm
parent didn't have regs=2 stack=0 marks
last_idx 2127 first_idx 2113
regs=2 stack=0 before 2127: (71) r1 = *(u8 *)(r7 +1)
; if (!unwinder_config.rate_limit_reads) {
2134: (61) r1 = *(u32 *)(r7 +20)
 R0=inv(id=0) R1_w=invP0 R2=inv0 R6=inv(id=0) R7=map_value(id=0,off=4,ks=4,vs=4151,imm=0) R8=inv(id=0) R9=inv4294967282 R10=fp0 fp-8=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-176=map_value fp-184=map_value fp-192=invP6 fp-200=map_value fp-208=map_value fp-216=ctx fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm fp-264=mmmmmmmm
; if (!unwinder_config.rate_limit_reads) {
2135: (15) if r1 == 0x0 goto pc+276
; payload_t payload = {REQUEST_READ, user_pid, addr};
2136: (7b) *(u64 *)(r10 -144) = r6
2137: (79) r3 = *(u64 *)(r10 -240)
2138: (63) *(u32 *)(r10 -148) = r3
2139: (b7) r1 = 3
2140: (73) *(u8 *)(r10 -152) = r1
; if (event_rate_limited(payload_for_rate_limiting, unwinder_config.rate_limit_reads)) {
2141: (18) r2 = 0xffff93669fa440d4
2143: (61) r6 = *(u32 *)(r2 +20)
 R0=inv(id=0) R1_w=inv3 R2_w=map_value(id=0,off=4,ks=4,vs=4151,imm=0) R3_w=inv(id=0) R6_w=inv(id=0) R7=map_value(id=0,off=4,ks=4,vs=4151,imm=0) R8=inv(id=0) R9=inv4294967282 R10=fp0 fp-8=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-144_w=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-176=map_value fp-184=map_value fp-192=invP6 fp-200=map_value fp-208=map_value fp-216=ctx fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm fp-264=mmmmmmmm
2144: (71) r2 = *(u8 *)(r10 -15)
invalid read from stack off -15+0 size 1
processed 7935 insns (limit 1000000) max_states_per_insn 4 total_states 306 peak_states 306 mark_read 120
-- END PROG LOAD LOG --
"
level=warn name=parca-agent ts=2024-04-17T14:44:59.946186552Z caller=logger.go:87 component=cpu_profiler component=libbpf msg="libbpf: prog 'native_unwind': failed to load: -13
"
level=warn name=parca-agent ts=2024-04-17T14:44:59.947440769Z caller=logger.go:87 component=cpu_profiler component=libbpf msg="libbpf: failed to load object 'parca-native'
"
level=error name=parca-agent ts=2024-04-17T14:44:59.947469422Z caller=cpu.go:465 component=cpu_profiler msg="could not create unwind info shards" lastError="failed to load BPF object: permission denied"
level=warn name=parca-agent ts=2024-04-17T14:44:59.947853868Z caller=discovery_manager.go:196 msg="unable to start provider" provider=systemd/0 error="context canceled"
level=error name=parca-agent ts=2024-04-17T14:44:59.948004403Z caller=main.go:539 err="load bpf program: failed to load BPF object: permission denied"

Note: BPF log is unwrapped for readability.

Software (please complete the following information):

  • Parca Agent Version: [parca-agent --version] :
    parca-agent, version (commit: 5249391d0b767f8c3bad291c6de9221c0d25841f, date: 2024-04-17T13:31:47Z), arch: amd64

Environment (please complete the following information):

  • Linux Distribution: [cat /etc/*-release]
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
  • Linux Version: [uname -r] - 5.4.0-170-generic
  • Arch: [uname -m] - x86_64
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant