eBPF from Go - IV - Code generation
Played more with TinyGo, but it always generates code for (minimal) reflection - i.e. the reflect
package, because it requires that for its map
implementation. I could skip or not generate that,
but then that would mean using a map would be … weird? Anyhow it looked that even TinyGo does too
much for eBPF. So plan D: generate eBPF code (and Go code) using Go. Cilium has done a lot of work
in this regard.
Also looking at eBPF and - once again - reading a lot, it dawned on me that eBPF is a hybrid, there
is always a kernel and userland component. Take for instance .rodata
(containing literal
strings, constants, etc, as used by libbpf
and bpftool
) this is currently a map that is
populated in userspace and then used in the eBPF assembler (libbpf sets this up for you). Going
further, there is no “map” use in eBPF programs, because the interface that happens via file
descriptors.
So even if I could create an ELF object, I’m not sure how the eBPF machine code and the (userland, native) machine code should be separated. If this would done in the Go compiler you have the same problem - you need some way of knowing which bits of your program are eBPF and which are native.
The current example I am working with is, just calling bpf_trace_printk
which is proving a
challenge, because:
- Go strings need to be converted to C strings (null terminated), I think Cilium does this, but
not sure. Using
import C
would work, but I don’t need CGO. - Pointers everywhere - I had thought a ARRAY map in eBPF could be indexed with u32s, but no, it is
a pointer to an u32…, so instead of a
asm.StoreImm(...)
this needs a put value on the stack and useRFP
to create a pointer to that value. Five instructions instead of one. - The eBPF verifier excels it helping you with its error messages:
6: (bf) r0 = r1: R1 !read_ok
, turns out I was reading from R1, but hadn’t written to it yet, which is illegal… Or10: (85) call bpf_trace_printk#6: R1 type=map_value_or_null expected=fp, pkt, pkt_meta, map_key, map_value, mem, ringbuf_mem, buf, trusted_ptr_
, OK, R1 has the wrong value, but lets no tell anyone what that is. Also this(bf)
and(85)
probably means something - no idea what yet. - I’m working with the Go AST (go/ast package), and walking the structure will probably involve
keeping track of nested function calls, to keep the ordering of the
asm
instructions correct.
In my test project, I’ve got the following in examples/helloworld:
package main
import "github.com/miekg/bpf"
func main() {
bpf.TracePrintk("Hello world!\n")
}
This now (almost) generates the following, which doesn’t work, the eBPF verifier rejects it (see above):
func setupProg() {
progSpec.Instructions = asm.Instructions{
asm.Mov.Reg(asm.R6, asm.R1),
asm.LoadMapPtr(asm.R1, ctx.FD()),
asm.StoreImm(asm.RFP, -16, 0, asm.Word),
asm.Mov.Reg(asm.R2, asm.RFP),
asm.Add.Imm(asm.R2, -16),
asm.FnMapLookupElem.Call(),
asm.Mov.Reg(asm.R1, asm.R0),
asm.LoadImm(asm.R2, 12, asm.DWord),
asm.FnTracePrintk.Call(),
asm.Mov.Imm(asm.R0, 0),
asm.Return(),
}
}
The rest of the Go program is also generated, but this is specifically geared towards the outcome I want at this moment. Will be interesting to see if that can be made more generic. The steps now are:
% ../../cmd/asm/asm helloworld.go > helloworld-ebpf.go
% go build -o helloworld-ebpf helloworld-ebpf.go
% sudo ./helloworld-ebpf
<verifier complains>