Possibility of integrating otel-profiling-agent as a library #33
Comments
|
Hi @i-Pear Thank you for your interest in this project. At the moment we are focusing and working on donating this project to OTel. In general it should be already possible to use this project as dependency. One can take this projects main.go as an example and start and load the components. |
|
Hi,
|
|
Hey, I agree with @alban on the points he recorded in the previous comment. The only thing I'd open is the way of integration with the profiling agent. Correct me if I am wrong, but as I see, options no. 1 and no. 2 are not mutually exclusive. Projects, that use Go should be able to enjoy the benefits of the ecosystem if it doesn't create extra burden. |
@slashben I also think both options are viable, each with its own advantages. Option 1 may focus more on "isolation," as the profiler requires some userspace agent threads to assist the eBPF program. If the profiler and other applications are compiled into the same process, I'm not sure if this would impose limitations on syscalls like The clear advantage of Option 2 is the elimination of communication overhead, as there would be no need to design a complex protocol. It also allows for easier and deeper customization. Moreover, the For the Inspektor Gadget project, another reason to favor Option 1 is that IG has disabled CGO, while the profiler heavily relies on this feature. |
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
This is the code that backs open-telemetry#144. It can be reused to add features like requested in open-telemetry#33 and therefore can be an alternative to open-telemetry#192. The idea that enables off CPU profiling is, that perf event and kprobe eBPF programs are quite similar and can be converted. This allows, with the dynamic rewrite of tail call maps, the reuse of existing eBPF programs and concepts. This proposal adds the new flag '-off-cpu-threshold' that enables off CPU profiling and attaches the two additional hooks, as discussed in Option B in open-telemetry#144. Outstanding work: - [ ] Handle off CPU traces in the reporter package - [ ] Handle off CPU traces in the user space side Signed-off-by: Florian Lehner <[email protected]>
Thank you for developing such an amazing profiling agent!
I am interested in integrating this project into other eBPF projects to support unwinding for multiple languages, replacing the
bpf_get_stackid()helper function. Due to the complex architecture of this project, I would prefer not to fork the code and create my own version. However, it seems that this project does not account for being called by other eBPF programs.Considering that the eBPF part of this project requires significant assistance from the userspace agent (such as extracting
eh_frameinformation), I envision two possible integration methods:The userspace component of
otel-profiling-agentruns as an independent daemon process, providing the necessary support for the eBPF part. Other projects could obtain a file descriptor pointing tonative_tracer_entryfrom the daemon process and use eBPF extensions or tail calls to invoke it on the eBPF side, similar to callingbpf_get_stackid(). The daemon process would then provide an API for reading the required call stack information.Other Golang projects integrate
otel-profiling-agentas a package, with this package creating separate threads to maintain the information needed for the eBPF program at runtime. This approach might be easier in terms of interaction but less friendly for projects written in languages other than Go.What do you think about this? I am curious if you are interested in making the unwinder part of this project a general-purpose component. If some work is needed to achieve this, I am more than willing to help.
The text was updated successfully, but these errors were encountered: