early SocketSet numbers

aaa.md

Setup: SocketSet inverts the usual System.Net.Sockets.Socket approach of ad-hoc sockets with their own push-based write and pull-based receive, to a manager that presides over a set of sockets (hence SocketSet), that manages them as a sharded group (dedicated IO worker per shard, except in "managed" mode), making use of multi-operation efficiencies, pre-pinned buffers, etc - with tight control over the threading mode (except for the managed sockets fallback, which is a free-for-all). Receive becomes a push from the network rather than a pull from the receiving code.

Hardware: strictly consumer; all tests loopback:

• Bazzite 43, AMD 7900X (12-core)
• Windows 11, AMD 395 (16-core)

Implemented:

(OS-agnostic)

• managed sockets fallback using SAEA

Linux

• io_uring (pure kernel calls, no "liburing")
• TCP and UDS

Windows

• IOCP
• RIO
• TCP (UDS planned)

The model also maps cleanly to DPDK, but see "Observations".

Tests shown here:

• --latency is a non-pipelined small-payload ping/pong test
• --bandwidth is a heavily-pipelined larger-payload flood test

Observations: RIO has marginal latency gains, and probably isn't worth the overheads in most cases. It can have a negative effect in throughput scenarios with small sends (packet fragmentation from the sender), because it responds to every packet immediately; "slower" APIs tend to merge received data leading to higher avg receive bytes in pipelined scenarios, which means RIO makes more syscalls etc. This makes me very hesitant to invest time today looking at DPDK, which takes this a step further in the "latency at all costs" direction, but also pays the price of intentionally burning a CPU core (DPDK is hot-loop rather than event-backed). Plus I don't have the hardware to properly investigatge DPDK other than in fallback mode (i.e. not actually exploiting the hardware) - and DPDK really needs multiple machines with high-end enterprise networking gear.

linux TCP.txt

marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 12 --managed
backend=ManagedSocketFactory transport=tcp mode=ping/pong size=64 shards=12 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=     31,614,592 bytes (    30.1 MiB/s)  echoed=     31,646,784
[    2s] conns=64 round-trips=     64,959,680 bytes (    31.8 MiB/s)  echoed=     64,962,752
[    3s] conns=64 round-trips=     98,338,944 bytes (    31.8 MiB/s)  echoed=     98,341,312
[    4s] conns=64 round-trips=    131,320,960 bytes (    31.4 MiB/s)  echoed=    131,323,584
[    5s] conns=64 round-trips=    163,411,712 bytes (    30.6 MiB/s)  echoed=    163,415,232
done: 163,413,952 round-trip bytes over 5.0s across 64 connection(s)
recv: 5,106,907 completions, 326,870,272 bytes, avg 64 bytes/recv
marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 12
backend=IoUringFactory transport=tcp mode=ping/pong size=64 shards=12 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=     43,339,904 bytes (    41.3 MiB/s)  echoed=     43,380,544
[    2s] conns=64 round-trips=     87,165,120 bytes (    41.7 MiB/s)  echoed=     87,168,448
[    3s] conns=64 round-trips=    130,859,904 bytes (    41.7 MiB/s)  echoed=    130,862,528
[    4s] conns=64 round-trips=    174,686,464 bytes (    41.8 MiB/s)  echoed=    174,689,088
[    5s] conns=64 round-trips=    218,541,056 bytes (    41.8 MiB/s)  echoed=    218,544,448
done: 218,542,976 round-trip bytes over 5.0s across 64 connection(s)
recv: 6,829,711 completions, 437,102,784 bytes, avg 64 bytes/recv
marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --bandwidth -n 12 --managed
backend=ManagedSocketFactory transport=tcp mode=pipeline(window=64) size=4096 shards=12 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=  3,186,377,410 bytes ( 3,037.0 MiB/s)  echoed=  3,198,412,800
[    2s] conns=64 round-trips=  6,497,513,472 bytes ( 3,144.8 MiB/s)  echoed=  6,507,593,728
[    3s] conns=64 round-trips=  9,577,857,024 bytes ( 2,934.0 MiB/s)  echoed=  9,588,142,080
[    4s] conns=64 round-trips= 12,620,763,136 bytes ( 2,898.9 MiB/s)  echoed= 12,629,581,824
[    5s] conns=64 round-trips= 15,701,016,576 bytes ( 2,932.8 MiB/s)  echoed= 15,711,010,816
done: 15,701,200,896 round-trip bytes over 5.0s across 64 connection(s)
recv: 7,669,911 completions, 31,413,411,840 bytes, avg 4,096 bytes/recv
marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --bandwidth -n 12
backend=IoUringFactory transport=tcp mode=pipeline(window=64) size=4096 shards=12 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=  2,439,020,544 bytes ( 2,325.8 MiB/s)  echoed=  2,449,604,608
[    2s] conns=64 round-trips=  4,911,251,456 bytes ( 2,354.6 MiB/s)  echoed=  4,919,881,728
[    3s] conns=64 round-trips=  7,388,123,136 bytes ( 2,361.7 MiB/s)  echoed=  7,396,990,976
[    4s] conns=64 round-trips=  9,870,086,144 bytes ( 2,366.6 MiB/s)  echoed=  9,878,454,272
[    5s] conns=64 round-trips= 12,352,274,432 bytes ( 2,366.8 MiB/s)  echoed= 12,360,761,344
done: 12,352,413,696 round-trip bytes over 5.0s across 64 connection(s)
recv: 6,033,694 completions, 24,714,133,504 bytes, avg 4,096 bytes/recv

linux UDS.txt

marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 12 --managed -u @foo
backend=ManagedSocketFactory transport=uds mode=ping/pong size=64 shards=12 pin=True poke=False
listening on @foo
opened 64 client connection(s) to @foo
[    1s] conns=64 round-trips=     60,156,864 bytes (    57.3 MiB/s)  echoed=     60,205,568
[    2s] conns=64 round-trips=    119,175,296 bytes (    56.3 MiB/s)  echoed=    119,179,584
[    3s] conns=64 round-trips=    178,317,056 bytes (    56.4 MiB/s)  echoed=    178,320,064
[    4s] conns=64 round-trips=    237,112,448 bytes (    56.1 MiB/s)  echoed=    237,115,840
[    5s] conns=64 round-trips=    296,123,264 bytes (    56.3 MiB/s)  echoed=    296,127,104
done: 296,126,336 round-trip bytes over 5.0s across 64 connection(s)
recv: 9,254,161 completions, 592,268,480 bytes, avg 64 bytes/recv
marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 12 -u @foo
backend=IoUringFactory transport=uds mode=ping/pong size=64 shards=12 pin=True poke=False
listening on @foo
opened 64 client connection(s) to @foo
[    1s] conns=64 round-trips=    161,773,824 bytes (   154.3 MiB/s)  echoed=    161,915,072
[    2s] conns=64 round-trips=    326,439,104 bytes (   156.9 MiB/s)  echoed=    326,447,168
[    3s] conns=64 round-trips=    490,751,104 bytes (   156.8 MiB/s)  echoed=    490,756,416
[    4s] conns=64 round-trips=    654,904,384 bytes (   156.5 MiB/s)  echoed=    654,909,184
[    5s] conns=64 round-trips=    819,004,672 bytes (   156.5 MiB/s)  echoed=    819,011,200
done: 819,012,992 round-trip bytes over 5.0s across 64 connection(s)
recv: 25,594,869 completions, 1,638,077,696 bytes, avg 64 bytes/recv
marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --bandwidth -n 12 --managed -u @foo
backend=ManagedSocketFactory transport=uds mode=pipeline(window=64) size=4096 shards=12 pin=True poke=False
listening on @foo
opened 64 client connection(s) to @foo
[    1s] conns=64 round-trips=  5,092,532,224 bytes ( 4,852.9 MiB/s)  echoed=  5,105,860,608
[    2s] conns=64 round-trips= 10,542,067,712 bytes ( 5,181.6 MiB/s)  echoed= 10,550,231,040
[    3s] conns=64 round-trips= 15,947,325,440 bytes ( 5,153.9 MiB/s)  echoed= 15,956,779,008
[    4s] conns=64 round-trips= 21,302,849,536 bytes ( 5,096.3 MiB/s)  echoed= 21,312,802,816
[    5s] conns=64 round-trips= 26,751,356,928 bytes ( 5,195.2 MiB/s)  echoed= 26,759,548,928
done: 26,751,758,336 round-trip bytes over 5.0s across 64 connection(s)
recv: 13,064,927 completions, 53,514,612,736 bytes, avg 4,096 bytes/recv
marc@mgx:~/FastNet/FastNet-main/SmokeTest$ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --bandwidth -n 12 -u @foo
backend=IoUringFactory transport=uds mode=pipeline(window=64) size=4096 shards=12 pin=True poke=False
listening on @foo
opened 64 client connection(s) to @foo
[    1s] conns=64 round-trips=  6,785,671,168 bytes ( 6,470.7 MiB/s)  echoed=  6,799,966,208
[    2s] conns=64 round-trips= 13,869,588,480 bytes ( 6,746.4 MiB/s)  echoed= 13,877,092,352
[    3s] conns=64 round-trips= 20,946,759,680 bytes ( 6,748.1 MiB/s)  echoed= 20,953,968,640
[    4s] conns=64 round-trips= 28,025,384,960 bytes ( 6,749.8 MiB/s)  echoed= 28,032,643,072
[    5s] conns=64 round-trips= 35,102,691,328 bytes ( 6,748.4 MiB/s)  echoed= 35,109,761,024
done: 35,102,986,240 round-trip bytes over 5.0s across 64 connection(s)
recv: 17,142,323 completions, 70,215,340,032 bytes, avg 4,096 bytes/recv

windows.txt

(numbers don't change hugely with higher client count; this is a saturation test)

➜ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 16 --managed
backend=ManagedSocketFactory transport=tcp mode=ping/pong size=64 shards=16 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=     19,082,816 bytes (    17.9 MiB/s)  echoed=     19,107,712
[    2s] conns=64 round-trips=     37,861,824 bytes (    17.7 MiB/s)  echoed=     37,865,088
[    3s] conns=64 round-trips=     56,191,424 bytes (    16.7 MiB/s)  echoed=     56,193,728
[    4s] conns=64 round-trips=     74,676,480 bytes (    17.3 MiB/s)  echoed=     74,679,296
[    5s] conns=64 round-trips=     93,529,600 bytes (    16.9 MiB/s)  echoed=     93,532,032
done: 93,537,856 round-trip bytes over 5.2s across 64 connection(s)
recv: 2,923,105 completions, 187,079,040 bytes, avg 64 bytes/recv
   08:06:59  SmokeTest main  ?1  1   11.0.100-preview.3.26207.106   7.409s 
➜ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 16 --iocp
backend=WindowsIocpFactory transport=tcp mode=ping/pong size=64 shards=16 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=     38,241,536 bytes (    36.3 MiB/s)  echoed=     38,300,416
[    2s] conns=64 round-trips=     74,789,184 bytes (    34.6 MiB/s)  echoed=     74,793,536
[    3s] conns=64 round-trips=    112,147,072 bytes (    34.9 MiB/s)  echoed=    112,150,592
[    4s] conns=64 round-trips=    148,911,168 bytes (    34.9 MiB/s)  echoed=    148,914,752
[    5s] conns=64 round-trips=    185,510,912 bytes (    34.9 MiB/s)  echoed=    185,515,648
done: 185,608,960 round-trip bytes over 5.0s across 64 connection(s)
recv: 5,802,170 completions, 371,340,480 bytes, avg 64 bytes/recv
   08:07:15  SmokeTest main  ?1  1   11.0.100-preview.3.26207.106   7.472s 
➜ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --latency -n 16 --rio
backend=WindowsRioFactory transport=tcp mode=ping/pong size=64 shards=16 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=     41,007,872 bytes (    38.7 MiB/s)  echoed=     41,067,264
[    2s] conns=64 round-trips=     77,067,840 bytes (    34.1 MiB/s)  echoed=     77,071,808
[    3s] conns=64 round-trips=    117,238,080 bytes (    37.7 MiB/s)  echoed=    117,241,664
[    4s] conns=64 round-trips=    157,574,784 bytes (    38.0 MiB/s)  echoed=    157,578,176
[    5s] conns=64 round-trips=    197,312,704 bytes (    37.9 MiB/s)  echoed=    197,316,544
done: 197,330,560 round-trip bytes over 5.0s across 64 connection(s)
recv: 6,166,888 completions, 394,683,392 bytes, avg 64 bytes/recv
   08:07:25  SmokeTest main  ?1  1   11.0.100-preview.3.26207.106   7.299s 
➜ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --bandwidth -n 16 --managed
backend=ManagedSocketFactory transport=tcp mode=pipeline(window=64) size=4096 shards=16 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=  1,212,485,632 bytes ( 1,138.7 MiB/s)  echoed=  1,214,418,944
[    2s] conns=64 round-trips=  2,807,480,320 bytes ( 1,498.3 MiB/s)  echoed=  2,808,582,144
[    3s] conns=64 round-trips=  4,239,142,912 bytes ( 1,359.0 MiB/s)  echoed=  4,240,281,600
[    4s] conns=64 round-trips=  5,815,844,864 bytes ( 1,484.8 MiB/s)  echoed=  5,816,709,120
[    5s] conns=64 round-trips=  7,293,263,872 bytes ( 1,400.7 MiB/s)  echoed=  7,294,836,736
done: 7,295,668,224 round-trip bytes over 5.1s across 64 connection(s)
recv: 3,563,028 completions, 14,594,260,992 bytes, avg 4,096 bytes/recv
   08:07:41  SmokeTest main  ?1  1   11.0.100-preview.3.26207.106   7.296s 
➜ dotnet run -c Release -f net10.0 -- -c 64 -s -t 5 --bandwidth -n 16 --iocp
backend=WindowsIocpFactory transport=tcp mode=pipeline(window=64) size=4096 shards=16 pin=True poke=False
listening on 0.0.0.0:10000
opened 64 client connection(s) to 127.0.0.1:10000
[    1s] conns=64 round-trips=  2,373,222,400 bytes ( 2,252.2 MiB/s)  echoed=  2,385,014,784
[    2s] conns=64 round-trips=  4,699,729,920 bytes ( 2,213.9 MiB/s)  echoed=  4,708,737,024
[    3s] conns=64 round-trips=  7,044,562,944 bytes ( 2,203.4 MiB/s)  echoed=  7,053,357,056
[    4s] conns=64 round-trips=  9,384,996,864 bytes ( 2,200.5 MiB/s)  echoed=  9,392,218,112
[    5s] conns=64 round-trips= 11,690,872,832 bytes ( 2,195.6 MiB/s)  echoed= 11,699,286,016
done: 11,692,679,168 round-trip bytes over 5.0s across 64 connection(s)
recv: 5,711,777 completions, 23,396,261,888 bytes, avg 4,096 bytes/recv