Using RR debugger with FRR

rr-with-frr.md

https://rr-project.org/

Useful to help debug intermittent FRR issues, or problems which only reproduce under certain network conditions that are difficult to recreate.

1. Install daemonize

2. Edit /etc/frr/daemons and set the following as a wrap command for the daemons you want:

   /usr/bin/daemonize /usr/bin/rr record -w
   

   Daemons are started with -d, which causes them to fork and the parent process to exit. The -w option to rr makes sure that rr follows child processes; without it, rr will exit with the parent and the recording will contain only the startup sequence for the daemon prior to its call to fork(). However, -w means that rr will stay in the foreground while following the child process, which is why daemonize is needed; daemonize backgrounds rr, which allows the init script to continue.

   Example wrap:

   bgpd_wrap="/usr/bin/daemonize /usr/bin/rr record -w"
   

3. Allow unprivileged users to use perf events by setting this sysctl:

   sysctl -w kernel.perf_event_paranoid=-1
   

   Because FRR drops privileges, if you don't set this sysctl it will crash after dropping privs when run under rr.

4. Restart FRR.

Notes:

• Don't use all_wrap. This will result in watchfrr being run with rr which causes it to block forever, for some reason.