A GDB stub for Fuchsia that supports the GDB Remote Serial Protocol.
Refer to this page for further documentation on debugging with GDB.
Notes on reading this document:
The stub currently communicates with GDB over a TCP connection using BSD sockets and thus requires a working Fuchsia network stack and a network connection between the target and host environments. Follow the general network set up instructions (click here for enabling network under QEMU)
N.B. If using QEMU make sure you are using version 2.5.0 or later.
See this document for setting up netstack.
Fuchsia support is not in upstream GDB yet, so the Fuchsia copy of GDB must be used: https://fuchsia.googlesource.com/third_party/gdb.
devhost$ cd /path/to/fuchsia devhost$ jiri import gdb https://fuchsia.googlesource.com/manifest devhost$ jiri update devhost$ sh scripts/gdb/build-gdb.sh
The gdb binary can be found in out/toolchain/x86_64-linux/gdb/bin/gdb.
If using qemu run the following to make sure dnsmasq is running:
devhost$ scripts/start-dhcp-server.sh qemu
Make sure to pass the -N argument to enable networking.
devhost$ fx run -N
Make sure that the TCP stack is running and has acquired an ipv4 address. Zircon should print its ipv4 address when it boots. GDB currently doesn't support ipv6.
[00014.920] 02933.03643> netstack: NIC 2: DHCP acquired IP 192.168.3.53 for 24h0m0s
If zircon is started with netsvc disabled (uncommon), then you need the following:
> netstack & > ethernetif_init: opened an ethernet device ip4_addr: 192.168.3.53 netmask: 255.255.255.0 gw: 192.168.3.1 ip6_addr[0]: FE80::5054::FF:FE12:3456
192.168.3.53 is the address that we will use to connect from the host gdb later.
Next, run the stub with your port number of choice (e.g. 7000) and a path to a binary. The stub currently supports debugging a single inferior during its life time. The program, with optional arguments, can be passed on the command line to gdbserver, or can be specified from within gdb with the set remote exec-file /target/path/to/program command. Attaching to an already running process is currently not supported.
The gdbserver binary is named debugserver.
debugserver 7000 /path/to/program
The stub has a --verbose=<verbosity-level> option that can be used to increase log verbosity. Currently only levels 1 and 2 are supported (but any positive integer is currently accepted.
This example uses the debugger unit test program, passing “segfault” as an argument to trigger a segfault (to help exercise the debugger).
debugserver --verbose=2 7000 /system/test/sys/debugger-test segfault
Use the target extended-remote command to connect to the IP and port number that stub acquired earlier:
devhost$ out/toolchain/x86_64-linux/gdb/bin/gdb GNU gdb (GDB) <version> ... Hi, this is fuchsia.py. Adding Fuchsia support. Setting fuchsia defaults. 'help set-fuchsia-defaults' for details. (gdb) target extended-remote 192.168.3.53:7000 Remote debugging using 192.168.3.53:7000
From here we can use gdb as usual:
(gdb) file out/build-zircon/build-x64/system/utest/debugger/debugger.elf
...
(gdb) break main
(gdb) run
Starting program: ...
...
Breakpoint 1, main (argc=2, argv=0x10f3e20)
at system/utest/debugger/debugger.c:530
530 {
(gdb) continue
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x00000000010f5d79 in test_segfault_leaf (n=10, p=0x10f3cc0)
at system/utest/debugger/debugger.c:471
471 *crashing_ptr = x[0];
(gdb) backtrace
#0 0x00000000010f5d79 in test_segfault_leaf (n=10, p=0x10f3cc0)
at system/utest/debugger/debugger.c:471
#1 0x00000000010f5df1 in test_segfault_doit1 (p=0x10f3cc0)
at system/utest/debugger/debugger.c:485
#2 0x00000000010f5e39 in test_segfault_doit2 (p=0x10f3cc0)
at system/utest/debugger/debugger.c:490
#3 0x00000000010f5dcc in test_segfault_doit1 (p=<optimized out>)
at system/utest/debugger/debugger.c:483
#4 0x00000000010f5e39 in test_segfault_doit2 (p=0x10f3d00)
at system/utest/debugger/debugger.c:490
#5 0x00000000010f5dcc in test_segfault_doit1 (p=<optimized out>)
at system/utest/debugger/debugger.c:483
#6 0x00000000010f5e39 in test_segfault_doit2 (p=0x10f3d40)
at system/utest/debugger/debugger.c:490
#7 0x00000000010f5dcc in test_segfault_doit1 (p=<optimized out>)
at system/utest/debugger/debugger.c:483
#8 0x00000000010f5e39 in test_segfault_doit2 (p=0x10f3d80)
at system/utest/debugger/debugger.c:490
#9 0x00000000010f5dcc in test_segfault_doit1 (p=<optimized out>)
at system/utest/debugger/debugger.c:483
#10 0x00000000010f5e23 in test_segfault ()
at system/utest/debugger/debugger.c:499
#11 0x00000000010f5617 in main (argc=2, argv=0x10f3e20)
at system/utest/debugger/debugger.c:540
(gdb) do stuff
On Fuchsia:
$ debugserver 7000
On Linux:
devhost$ out/toolchain/x86_64-linux/gdb/bin/gdb
(gdb) file out/build-zircon/build-x64/system/utest/debugger/debugger.elf
(gdb) tar ext 192.168.3.53:7000
(gdb) set remote exec-file /system/test/sys/debugger-test
(gdb) r segfault
...
Program received signal SIGSEGV, Segmentation fault.
0x00000000010f5d79 in test_segfault_leaf (n=10, p=0x10f3cc0)
at system/utest/debugger/debugger.c:471
471 *crashing_ptr = x[0];
(gdb) # do stuff
Debugserver recognizes the gdb “monitor” command:
(gdb) monitor help help - print this help exit - quit debugserver quit - quit debugserver set <parameter> <value> show <parameter> Parameters: verbosity - useful range is -2 to 3 (-2 is most verbose)
This is not supported yet. We just need to wire up support for suspend/resume in the debugserver stub.
gdbserver‘s logging output is a bit verbose at the moment, and some output that claims to be “errors” aren’t really errors per se. In general, ignore them unless your gdb session isn't working as expected.
At the moment Fuchsia only supports the “non-stop” mode of GDB: When a thread stops all other threads are left running. See https://sourceware.org/gdb/current/onlinedocs/gdb/Non_002dStop-Mode.html#Non_002dStop-Mode
You may get complaints about unable to load symbols for shared libraries.
warning: Could not load shared library symbols for 3 libraries, e.g. libtest-utils.so. Use the "info sharedlibrary" command to see the complete listing. Do you need "set solib-search-path" or "set sysroot"?
Depending on the library you may be able to ignore these. The error occurs because the library isn‘t in gdb’s search path. If you need the library (say you want to set a breakpoint in a function in it) then find the library in the build tree, e.g.,
devhost$ find out/build-zircon/build-x64 -name libtest-utils.so out/build-zircon/build-x64/system/ulib/test-utils/libtest-utils.so
and then in gdb:
(gdb) show solib-search-path The search path for loading non-absolute shared library symbol files is out/build-zircon/build-x64/sysroot/debug-info. (gdb) set solib-search-path out/build-zircon/build-x64/sysroot/debug-info:out/build-zircon/build-x64/system/ulib/test-utils
The current Fuchsia GDB port is very preliminary. Expect problems and missing features! But you should be able to set breakpoints, run, and examine program state when it crashes.
It is absolutely necessary (at the moment) that gdb be able to load symbols for ld.so.1. If things aren't working at all, that is the first place to look.
Do both of these files exist?
devhost$ ls -l out/build-zircon/build-x64/sysroot/debug-info/{libc.so,ld.so.1}
lrwxrwxrwx 1 dje eng 7 Oct 1 12:53 out/build-zircon/build-x64/sysroot/debug-info/ld.so.1 -> libc.so
-rwxr-x--- 2 dje eng 4934096 Oct 1 12:53 out/build-zircon/build-x64/sysroot/debug-info/libc.so
Does libc.so have debug information? A check for it being > 3M in size is normally good enough.
The reason for this is due to how Fuchsia runs programs. All programs in Fuchsia are PIE executables, and GDB needs to know its runtime load address in order for most things to work. [There are caveats and qualifiers of course, elided for brevity.] When a program is started the dynamic linker is loaded into memory but the program itself is not loaded yet. The dynamic linker will load the program into memory later but it‘s not until then that the load address of the program is known. One issue is “How much later?” There is a special function in the dynamic linker, dl_debug_state(), that the dynamic linker calls when the main executable, and any shared libraries it uses, have been loaded into memory. GDB sets a breakpoint on this function and when the program gets here GDB can then know where the program has been loaded. But what’s the address of dl_debug_state? This is why GDB needs the symbols of the dynamic linker: to determine the address of dl_debug_state. [There are other ways to make this work. Maybe in time GDB will use one of them.]
The following is an incomplete and unprioritized list of tasks that remain to be finished for basic debugging to work: