replace BCC with golang native ebpf libs?

[SamYuan1990]

ref to https://ebpf.io/infrastructure
there golang native ebpf libs as
https://github.com/cilium/ebpf
or
https://github.com/aquasecurity/libbpfgo

do we need to consider that replace bcc by golang native libs for ebpf?

[rootfs]

I looked at cilium library before, I like the pure go library. There is a more detailed analysis written a couple of years ago (things may have already changed since then)

[SamYuan1990]

do we have a feature list for currently BCC features usages?
if all features are covered by pure go library, I suppose we can think about the replacement?
@marceloamaral what do you think?

[rootfs]

Pure Go or not is not the only criteria.
This is the conclusion from the blog post above.

Cilium library supports loading more BPF program types than its iovisor counterpart. Although the iovisor library provides more abstraction it does this by relying on CGO which incurs cost in terms of performance as a call to CGO is much slower then a call to native GO. The iovisor library has an interesting and useful feature. It changes the kernel version from your BPF program version section to the kernel version it is running on when it loads the ELF.That means that if you specify the “magic” version of 0xFFFFFFFE in your version section like: u32 _version SEC("version") = 0xFFFFFFFE when the library will read the ELF and encounter that version it will replace it with the version it finds on the system it is running on. This helps avoid compiling a BPF program for different kernel versions or recompiling on every minor kernel version change. Of course the BPF developers added this limitation with good reason as, for example, for a kprobe program, kernel function names may change at any time and so your program will not work anymore so use it at your own risk. But if we take the chance and assume that these won’t actually change so often then we have a more portable BPF program. This is a cool feature that could be also added to the cilium library.

[SamYuan1990]

Pure Go or not is not the only criteria. This is the conclusion from the blog post above.

Cilium library supports loading more BPF program types than its iovisor counterpart. Although the iovisor library provides more abstraction it does this by relying on CGO which incurs cost in terms of performance as a call to CGO is much slower then a call to native GO. The iovisor library has an interesting and useful feature. It changes the kernel version from your BPF program version section to the kernel version it is running on when it loads the ELF.That means that if you specify the “magic” version of 0xFFFFFFFE in your version section like: u32 _version SEC("version") = 0xFFFFFFFE when the library will read the ELF and encounter that version it will replace it with the version it finds on the system it is running on. This helps avoid compiling a BPF program for different kernel versions or recompiling on every minor kernel version change. Of course the BPF developers added this limitation with good reason as, for example, for a kprobe program, kernel function names may change at any time and so your program will not work anymore so use it at your own risk. But if we take the chance and assume that these won’t actually change so often then we have a more portable BPF program. This is a cool feature that could be also added to the cilium library.

how about https://github.com/aquasecurity/libbpfgo ?
and can we list all the things we need think about ?
for example, performance we can try to use benchmark to see impacts to us?

[marceloamaral]

inspektor-gadget.io

Choosing a Golang eBPF Library
There are three libraries we are aware of: iovisor/gobpf, cilium/ebpf and aquasecurity/libbpfgo.. We decided to use cilium/ebpf, because it’s well maintained, supports CO-RE and doesn’t use gco. It also provides bpf2go , a tool that reduces the amount of manual work to interact with eBPF programs.

[marceloamaral]

Apart from the advantage that cilium ebpf can be compiled with static libraries,
the .c code follows the same syntax as the linux kernel uses to write ebpf codes.
For example linux, cilium. Which is much better to follow examples.

[marceloamaral]

There is a big limitation of using the cilium/ebpf library, it doesn't support reading hardware counters. I opened an issue requesting HC support, but it may take some time.

Regarding aquasecurity/libbpfgo., although it can be compiled with static libraries without dependency on kernel libraries, this library requires BTF support from the kernel. Which is only available by default on newer Linux distributions, for example ubuntu 20.10. Otherwise we need to recompile the kernel.

However, aquasecurity/libbpfgo. also created a tool called BTFGEN for kernels that don't support BTF. I will try to use it.

[marceloamaral]

Libbpf support the CO-RE (Compile Once and Run Everywhere), however, this means that we must have an environment with OS that support BTF during the compilation. But to run everywhere, we can use the BTFGEN.

Unfortunately, my development env does not support BTF. Of course, we can use a virtual machine for compilation, but the development process will be a little bit harder. I am stopping investigating this now and we can return to it in the future.

Btw, the cilium community are discussing the possibility add support to collect HC.