Skip to content

Instantly share code, notes, and snippets.

@thehajime

thehajime/net-setup.txt

Last active February 18, 2020 05:59
Show Gist options
  • Save thehajime/a71878cccf7830a23a23f8f8e8cc8753 to your computer and use it in GitHub Desktop.
Save thehajime/a71878cccf7830a23a23f8f8e8cc8753 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
net-setup.txt
+--docker0--+
| | 10GbEth
netperf +---tap0 eth0 +==========+ eth0 +---+ netserver
(client) (ixgbe) (ixgbe)
<-- Linux box (4.18.5) --> <-- Linux box (4.17.19) -->
Raw
um-lkl-tap.log
% for file in {lkl-hijack-um-tap.json,./lkl-hijack-vec-tap.json,lkl-hijack-virtio-tap.json}; do \
echo "==== $file ====" ; \
LKL_HIJACK_CONFIG_FILE=$file ./obj-x86/tools/lkl/bin/lkl-hijack.sh netperf -H 10.0.39.2 -- \
-o ELAPSED_TIME,THROUGHPUT,THROUGHPUT_UNITS ; \
LKL_HIJACK_CONFIG_FILE=$file ./obj-x86/tools/lkl/bin/lkl-hijack.sh netperf -H 10.0.39.2 -t TCP_MAERTS \
-- -o ELAPSED_TIME,THROUGHPUT,THROUGHPUT_UNITS; \
done
==== lkl-hijack-um-tap.json ====
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 () port 0 AF_INET : demo
Elapsed Time (sec),Throughput,Throughput Units
10.01,2290.42,10^6bits/s
MIGRATED TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 () port 0 AF_INET : demo
Elapsed Time (sec),Throughput,Throughput Units
10.03,1.04,10^6bits/s
==== ./lkl-hijack-vec-tap.json ====
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 () port 0 AF_INET : demo
Elapsed Time (sec),Throughput,Throughput Units
10.01,3699.98,10^6bits/s
MIGRATED TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 () port 0 AF_INET : demo
Elapsed Time (sec),Throughput,Throughput Units
10.00,5682.40,10^6bits/s
==== lkl-hijack-virtio-tap.json ====
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 () port 0 AF_INET : demo
Elapsed Time (sec),Throughput,Throughput Units
10.01,8029.13,10^6bits/s
MIGRATED TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 () port 0 AF_INET : demo
Elapsed Time (sec),Throughput,Throughput Units
10.00,9384.78,10^6bits/s
==> lkl-hijack-um-tap.json <==
{
"gateway": "172.17.0.1",
"interfaces": [
{
"ip": "172.17.0.39",
"masklen": "16",
"param": "tuntap,tap0,01:11:22:33:44:55",
"type": "um"
}
],
"debug": "0",
"singlecpu": "1",
"boot_cmdline": "mem=1G",
"sysctl": "net.ipv4.tcp_wmem=4096 87380 16777216"
}
==> lkl-hijack-vec-tap.json <==
{
"gateway": "172.17.0.1",
"interfaces": [
{
"ip": "172.17.0.39",
"masklen": "16",
"param": "transport=tap,ifname=tap0,gro=1",
"type": "um"
}
],
"debug": "0",
"singlecpu": "1",
"boot_cmdline": "mem=1G",
"sysctl": "net.ipv4.tcp_wmem=4096 87380 16777216"
}
==> lkl-hijack-virtio-tap.json <==
{
"gateway": "172.17.0.1",
"interfaces": [
{
"ip": "172.17.0.39",
"masklen": "16",
"param": "tap0",
"type": "tap",
"offload": "0xc803"
}
],
"debug": "0",
"singlecpu": "1",
"boot_cmdline": "mem=1G",
"sysctl": "net.ipv4.tcp_wmem=4096 87380 16777216"
}
Raw
um-netperf.sh
#!/bin/sh
ifconfig vec0 up
ip ad add 172.17.0.39/16 dev vec0
ip route add default via 172.17.0.1
sleep 2
/usr/local/bin/ethtool -k vec0
/usr/local/bin/netperf -H 10.0.39.2 -- -o ELAPSED_TIME,THROUGHPUT,THROUGHPUT_UNITS
/usr/local/bin/netperf -H 10.0.39.2 -t TCP_MAERTS -- -o ELAPSED_TIME,THROUGHPUT,THROUGHPUT_UNITS
Raw
um-tap.log
Core dump limits :
soft - NONE
hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking environment variables for a tempdir...none found
Checking if /dev/shm is on tmpfs...OK
Checking PROT_EXEC mmap in /dev/shm...OK
Adding 22847488 bytes to physical memory to account for exec-shield gap
Linux version 5.5.0-rc6-00047-ge9396182d667-dirty ([email protected]) (gcc version 8.3.1 20190223 (Red Hat 8.3.1-2) (GCC)) #14 Tue Feb 18 11:48:31 JST 2020
Built 1 zonelists, mobility grouping on. Total pages: 13581
Kernel command line: root=/dev/root rootfstype=hostfs rootflags=/home/tazaki/work/um-lkl/obj-x86-um/../../uml-rootfs/ ro init=/bin/netperf.sh eth0=tuntap,tap0,
Dentry cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
Inode-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 27448K/55080K available (2399K kernel code, 753K rwdata, 952K rodata, 101K init, 160K bss, 27632K reserved, 0K cma-reserved)
NR_IRQS: 24
clocksource: timer: mask: 0xffffffffffffffff max_cycles: 0x1cd42e205, max_idle_ns: 881590404426 ns
Calibrating delay loop... 8342.73 BogoMIPS (lpj=41713664)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
Checking that host ptys support output SIGIO...Yes
Checking that host ptys support SIGIO on close...No, enabling workaround
devtmpfs: initialized
random: get_random_u32 called from bucket_table_alloc+0xe6/0x10f with crng_init=0
umid_file_name : buffer too short
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 256 (order: 0, 6144 bytes, linear)
NET: Registered protocol family 16
clocksource: Switched to clocksource timer
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes, linear)
TCP established hash table entries: 512 (order: 0, 4096 bytes, linear)
TCP bind hash table entries: 512 (order: 0, 4096 bytes, linear)
TCP: Hash tables configured (established 512 bind 512)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered protocol family 1
printk: console [stderr0] disabled
mconsole (version 2) initialized on /home/tazaki/.uml/lMl3OJ/mconsole
Checking host MADV_REMOVE support...OK
workingset: timestamp_bits=62 max_order=13 bucket_order=0
io scheduler mq-deadline registered
io scheduler kyber registered
NET: Registered protocol family 17
Initialized stdio console driver
Console initialized on /dev/tty0
printk: console [tty0] enabled
Initializing software serial port version 1
printk: console [mc-1] enabled
Failed to initialize ubd device 0 :Couldn't determine size of device's file
Choosing a random ethernet address for device eth0
Netdevice 0 (c2:02:40:af:5a:d2) :
TUN/TAP backend -
VFS: Mounted root (hostfs filesystem) readonly on device 0:12.
devtmpfs: mounted
This architecture does not have kernel memory protection.
Run /bin/netperf.sh as init process
Features for eth0:
Cannot get device udp-fragmentation-offload settings: Not supported
rx-checksumming: off [fixed]
tx-checksumming: off
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: off
tx-scatter-gather: off [fixed]
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off [fixed]
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off [fixed]
tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off
generic-segmentation-offload: off [requested on]
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
random: netperf: uninitialized urandom read (16 bytes read)
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 (10.0.39) port 0 AF_INET
random: fast init done
random: crng init done
Elapsed Time (sec),Throughput,Throughput Units
10.00,2233.17,10^6bits/s
MIGRATED TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 (10.0.39) port 0 AF_INET
Elapsed Time (sec),Throughput,Throughput Units
10.00,7.85,10^6bits/s
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000
CPU: 0 PID: 1 Comm: netperf.sh Not tainted 5.5.0-rc6-00047-ge9396182d667-dirty #14
Stack:
6302bce0 6025baaa 00000001 63026000
602ea818 00000001 6302bcf0 6025bad5
6302bdf0 6002f977 631b2aa0 3000000010
Call Trace:
[<60016877>] show_stack+0xff/0x116
[<6025baaa>] ? dump_stack_print_info+0xa8/0xb1
[<6025bad5>] dump_stack+0x17/0x19
[<6002f977>] panic+0x12c/0x2a7
[<60051c9f>] ? __percpu_up_read+0x13/0x15
[<60036eb4>] ? percpu_up_read.constprop.57+0x23/0x25
[<600392d5>] ? exit_signals+0x7c/0xcf
[<60030c10>] do_exit+0x1ab/0x75e
[<600c6e0b>] ? __fdget+0xe/0x10
[<600af92c>] ? fdput_pos.isra.26+0x25/0x2c
[<60031dec>] sys_exit_group+0x0/0xf
[<60031dfb>] __wake_up_parent+0x0/0x1e
[<600182c3>] handle_syscall+0x6b/0x8b
[<6002a3e5>] userspace+0x391/0x3f1
[<60015958>] new_thread_handler+0xa2/0xa4
Raw
um-vec-tap.log
zakzak3:~/work/um-lkl/obj-x86-um% ./linux root=/dev/root rootfstype=hostfs rootflags=`pwd`/../../uml-rootfs/ mem=512M ro init='/bin/netperf.sh' vec0:transport=tap,ifname=tap0,gro=1
Core dump limits :
soft - NONE
hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking environment variables for a tempdir...none found
Checking if /dev/shm is on tmpfs...OK
Checking PROT_EXEC mmap in /dev/shm...OK
Adding 19533824 bytes to physical memory to account for exec-shield gap
Linux version 5.5.0-rc6-00047-ge9396182d667-dirty ([email protected]) (gcc version 8.3.1 20190223 (Red Hat 8.3.1-2) (GCC)) #14 Tue Feb 18 11:48:31 JST 2020
Built 1 zonelists, mobility grouping on. Total pages: 133983
Kernel command line: root=/dev/root rootfstype=hostfs rootflags=/home/tazaki/work/um-lkl/obj-x86-um/../../uml-rootfs/ mem=512M ro init=/bin/netperf.sh vec0:transport=tap,ifname=tap0,gro=1
Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 510864K/543364K available (2399K kernel code, 753K rwdata, 952K rodata, 101K init, 160K bss, 32500K reserved, 0K cma-reserved)
NR_IRQS: 24
clocksource: timer: mask: 0xffffffffffffffff max_cycles: 0x1cd42e205, max_idle_ns: 881590404426 ns
Calibrating delay loop... 8350.92 BogoMIPS (lpj=41754624)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
Checking that host ptys support output SIGIO...Yes
Checking that host ptys support SIGIO on close...No, enabling workaround
devtmpfs: initialized
random: get_random_u32 called from bucket_table_alloc+0xe6/0x10f with crng_init=0
umid_file_name : buffer too short
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 256 (order: 0, 6144 bytes, linear)
NET: Registered protocol family 16
clocksource: Switched to clocksource timer
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
TCP bind hash table entries: 8192 (order: 4, 65536 bytes, linear)
TCP: Hash tables configured (established 8192 bind 8192)
UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
NET: Registered protocol family 1
printk: console [stderr0] disabled
mconsole (version 2) initialized on /home/tazaki/.uml/Yphfcl/mconsole
Checking host MADV_REMOVE support...OK
workingset: timestamp_bits=62 max_order=17 bucket_order=0
io scheduler mq-deadline registered
io scheduler kyber registered
NET: Registered protocol family 17
Initialized stdio console driver
Console initialized on /dev/tty0
printk: console [tty0] enabled
Initializing software serial port version 1
Choosing a random ethernet address for device vec0
printk: console [mc-1] enabled
Failed to initialize ubd device 0 :Couldn't determine size of device's file
VFS: Mounted root (hostfs filesystem) readonly on device 0:12.
devtmpfs: mounted
This architecture does not have kernel memory protection.
Run /bin/netperf.sh as init process
sysctl: can't change directory to '/proc/sys': No such file or directory
uml-vector uml-vector.0 vec0: tap: using vnet headers for tso and tx/rx checksum
Features for vec0:
Cannot get device udp-fragmentation-offload settings: Not supported
rx-checksumming: on [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on [fixed]
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: on
tcp-segmentation-offload: on
tx-tcp-segmentation: on [requested off]
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off [fixed]
tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
random: netperf: uninitialized urandom read (16 bytes read)
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 (10.0.39) port 0 AF_INET
random: fast init done
random: crng init done
Elapsed Time (sec),Throughput,Throughput Units
10.00,5527.37,10^6bits/s
MIGRATED TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.39.2 (10.0.39) port 0 AF_INET
uml-vector uml-vector.0 vec0: vector_rx: error(1524)
uml-vector uml-vector.0 vec0: vector_rx: device stuck, remote end may have closed the connection
uml-vector uml-vector.0 vec0: vector_rx: error(1524)
uml-vector uml-vector.0 vec0: vector_rx: device stuck, remote end may have closed the connection
Elapsed Time (sec),Throughput,Throughput Units
10.00,9414.00,10^6bits/s
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000
CPU: 0 PID: 1 Comm: netperf.sh Not tainted 5.5.0-rc6-00047-ge9396182d667-dirty #14
Stack:
80437ce0 6025baaa 00000001 8042a000
602ea818 00000001 80437cf0 6025bad5
80437df0 6002f977 805d1aa0 3000000010
Call Trace:
[<60016877>] show_stack+0xff/0x116
[<6025baaa>] ? dump_stack_print_info+0xa8/0xb1
[<6025bad5>] dump_stack+0x17/0x19
[<6002f977>] panic+0x12c/0x2a7
[<60051c9f>] ? __percpu_up_read+0x13/0x15
[<60036eb4>] ? percpu_up_read.constprop.57+0x23/0x25
[<600392d5>] ? exit_signals+0x7c/0xcf
[<60030c10>] do_exit+0x1ab/0x75e
[<600c6e0b>] ? __fdget+0xe/0x10
[<600af92c>] ? fdput_pos.isra.26+0x25/0x2c
[<60031dec>] sys_exit_group+0x0/0xf
[<60031dfb>] __wake_up_parent+0x0/0x1e
[<600182c3>] handle_syscall+0x6b/0x8b
[<6002a3e5>] userspace+0x391/0x3f1
[<60015958>] new_thread_handler+0xa2/0xa4
zsh: abort (core dumped) ./linux root=/dev/root rootfstype=hostfs rootflags=`pwd`/../../uml-rootfs/ r
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment