Skip to content

Instantly share code, notes, and snippets.

@xh4n3
Created July 7, 2022 02:39
Show Gist options
  • Save xh4n3/0a848898064d9a1587528d41e69c5803 to your computer and use it in GitHub Desktop.
Save xh4n3/0a848898064d9a1587528d41e69c5803 to your computer and use it in GitHub Desktop.
traces the ARP requests in containernetworking/plugins issue#756
#!/usr/bin/env bpftrace
/*
* arp_announcement traces inetdev_event/br_handle_frame/br_set_state.
*
* Background:
* 1. This script addresses https://github.com/containernetworking/plugins/issues/756
* 2. br state definitions at https://github.com/torvalds/linux/blob/v4.19/include/uapi/linux/if_bridge.h#L49-L53
* 3. inetdev_event event type definitions at https://github.com/torvalds/linux/blob/v4.19/include/linux/netdevice.h#L2413-L2450
*
* Usage:
* arp_announcement.bt
*/
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/ip.h>
#include <uapi/linux/if_ether.h>
struct net_bridge_port {
struct net_bridge *br;
struct net_device *dev;
struct list_head list;
unsigned long flags;
struct net_bridge_vlan_group __rcu *vlgrp;
struct net_bridge_port __rcu *backup_port;
u8 priority;
u8 state;
}
struct net_bridge {
spinlock_t lock;
spinlock_t hash_lock;
struct list_head port_list;
struct net_device *dev;
}
kprobe:inetdev_event {
time("%H:%M:%S ");
$event = (arg1);
$info = (struct netdev_notifier_info *)(arg2);
$dev = $info->dev;
printf("inetdev_event triggered: ");
printf("dev_ifindex %d", $dev->ifindex);
printf("\t");
printf("event_type %d", $event);
printf("\n");
}
kprobe:br_handle_frame {
time("%H:%M:%S ");
$skb = (struct sk_buff *)(*arg0);
printf("br_handle_frame triggered: ");
printf("dev_ifindex %d", $skb->dev->ifindex);
printf("\t");
$hdr = (struct ethhdr *)($skb->head + $skb->mac_header);
printf("source_mac %r", buf($hdr->h_source));
printf("\t");
printf("dest_mac %r", buf($hdr->h_dest));
printf("\t");
$brp = (struct net_bridge_port *) ($skb->dev->rx_handler_data);
printf("bridge_port_state %d", $brp->state);
printf("\t");
$br = $brp->br;
printf("bridge_ifindex %d", $br->dev->ifindex);
printf("\n");
}
kprobe:br_set_state {
time("%H:%M:%S ");
$port = (struct net_bridge_port *)(arg0);
$state = (arg1);
printf("br_set_state triggered: ");
printf("dev_ifindex %d", $port->dev->ifindex);
printf("\t");
printf("new_state %d", $state);
printf("\n");
}
@xh4n3
Copy link
Author

xh4n3 commented Jul 7, 2022

As described in containernetworking/plugins#756 (comment), when I trigger a sandbox removal/recreation, above script outputs:

# bpftrace time.bt
Attaching 3 probes...
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 9
10:03:16 inetdev_event   triggered: dev_ifindex 58	event_type 9
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 9
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 2
10:03:16 inetdev_event   triggered: dev_ifindex 58	event_type 2
10:03:16 br_set_state    triggered: dev_ifindex 58	new_state 0
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 2
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 6
10:03:16 inetdev_event   triggered: dev_ifindex 58	event_type 6
10:03:16 br_set_state    triggered: dev_ifindex 58	new_state 0
10:03:16 inetdev_event   triggered: dev_ifindex 58	event_type 25
10:03:16 inetdev_event   triggered: dev_ifindex 58	event_type 20
10:03:16 inetdev_event   triggered: dev_ifindex 11	event_type 11
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 6
10:03:16 inetdev_event   triggered: dev_ifindex 11	event_type 4
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 16
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 5
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 13
10:03:16 inetdev_event   triggered: dev_ifindex 1	event_type 1
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 16
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 5
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 16
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 5
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 13
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 1
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 4
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 0
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 19
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 25
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 20
10:03:16 inetdev_event   triggered: dev_ifindex 11	event_type 11
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 4
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 3
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 4
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 0
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 13
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 1
10:03:16 br_handle_frame triggered: dev_ifindex 60	source_mac \xce\xb1}N\xda\x12	dest_mac \xff\xff\xff\xff\xff\xff	bridge_port_state 0	bridge_ifindex 11
10:03:16 inetdev_event   triggered: dev_ifindex 2	event_type 4
10:03:16 inetdev_event   triggered: dev_ifindex 60	event_type 4
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 4
10:03:16 br_set_state    triggered: dev_ifindex 60	new_state 3
10:03:16 inetdev_event   triggered: dev_ifindex 11	event_type 4

The interface are listed below, note the interface #58 is the old sandbox being removing, the #60 is the newly-recreated sandbox.

# ip -d link
11: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 36:42:38:5c:3f:08 brd ff:ff:ff:ff:ff:ff promiscuity 0
    bridge forward_delay 1500 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id 8000.36:42:38:5c:3f:8 designated_root 8000.36:42:38:5c:3f:8 root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer    0.00 tcn_timer    0.00 topology_change_timer    0.00 gc_timer  194.81 vlan_default_pvid 1 vlan_stats_enabled 0 group_fwd_mask 0 group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 4 mcast_hash_max 512 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3125 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

58: veth28ab8fb8@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni0 state UP mode DEFAULT group default
    link/ether 5a:97:d3:73:b2:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 1
    veth
    bridge_slave state forwarding priority 32 cost 2 hairpin on guard off root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0 designated_bridge 8000.36:42:38:5c:3f:8 designated_root 8000.36:42:38:5c:3f:8 hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

60: veth741e6f0d@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni0 state UP mode DEFAULT group default
    link/ether 16:89:c0:b3:c4:64 brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 1
    veth
    bridge_slave state forwarding priority 32 cost 2 hairpin on guard off root_block off fastleave off learning on flood on port_id 0x8001 port_no 0x1 designated_port 32769 designated_cost 0 designated_bridge 8000.36:42:38:5c:3f:8 designated_root 8000.36:42:38:5c:3f:8 hold_timer    0.00 message_age_timer    0.00 forward_delay_timer    0.00 topology_change_ack 0 config_pending 0 proxy_arp off proxy_arp_wifi off mcast_router 1 mcast_fast_leave off mcast_flood on addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment