Skip to content

Instantly share code, notes, and snippets.

@adamo1139

adamo1139/gist:6e5eed4112397b88082e06066937f971

Created April 29, 2026 20:47
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save adamo1139/6e5eed4112397b88082e06066937f971 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save adamo1139/6e5eed4112397b88082e06066937f971 to your computer and use it in GitHub Desktop.
Download ZIP
8x 3090 ti p2p driver on x399 taichi
Raw
gistfile1.txt
# p2p test from cuda samples repo
./p2pBandwidthLatencyTest
[P2P (Peer-to-Peer) GPU Bandwidth Latency Test]
Device: 0, NVIDIA GeForce RTX 3090 Ti, pciBusID: 8, pciDeviceID: 0, pciDomainID:0
Device: 1, NVIDIA GeForce RTX 3090 Ti, pciBusID: 9, pciDeviceID: 0, pciDomainID:0
Device: 2, NVIDIA GeForce RTX 3090 Ti, pciBusID: a, pciDeviceID: 0, pciDomainID:0
Device: 3, NVIDIA GeForce RTX 3090 Ti, pciBusID: b, pciDeviceID: 0, pciDomainID:0
Device: 4, NVIDIA GeForce RTX 3090 Ti, pciBusID: 42, pciDeviceID: 0, pciDomainID:0
Device: 5, NVIDIA GeForce RTX 3090 Ti, pciBusID: 43, pciDeviceID: 0, pciDomainID:0
Device: 6, NVIDIA GeForce RTX 3090 Ti, pciBusID: 44, pciDeviceID: 0, pciDomainID:0
Device: 7, NVIDIA GeForce RTX 3090 Ti, pciBusID: 45, pciDeviceID: 0, pciDomainID:0
Device=0 CAN Access Peer Device=1
Device=0 CAN Access Peer Device=2
Device=0 CAN Access Peer Device=3
Device=0 CAN Access Peer Device=4
Device=0 CAN Access Peer Device=5
Device=0 CAN Access Peer Device=6
Device=0 CAN Access Peer Device=7
Device=1 CAN Access Peer Device=0
Device=1 CAN Access Peer Device=2
Device=1 CAN Access Peer Device=3
Device=1 CAN Access Peer Device=4
Device=1 CAN Access Peer Device=5
Device=1 CAN Access Peer Device=6
Device=1 CAN Access Peer Device=7
Device=2 CAN Access Peer Device=0
Device=2 CAN Access Peer Device=1
Device=2 CAN Access Peer Device=3
Device=2 CAN Access Peer Device=4
Device=2 CAN Access Peer Device=5
Device=2 CAN Access Peer Device=6
Device=2 CAN Access Peer Device=7
Device=3 CAN Access Peer Device=0
Device=3 CAN Access Peer Device=1
Device=3 CAN Access Peer Device=2
Device=3 CAN Access Peer Device=4
Device=3 CAN Access Peer Device=5
Device=3 CAN Access Peer Device=6
Device=3 CAN Access Peer Device=7
Device=4 CAN Access Peer Device=0
Device=4 CAN Access Peer Device=1
Device=4 CAN Access Peer Device=2
Device=4 CAN Access Peer Device=3
Device=4 CAN Access Peer Device=5
Device=4 CAN Access Peer Device=6
Device=4 CAN Access Peer Device=7
Device=5 CAN Access Peer Device=0
Device=5 CAN Access Peer Device=1
Device=5 CAN Access Peer Device=2
Device=5 CAN Access Peer Device=3
Device=5 CAN Access Peer Device=4
Device=5 CAN Access Peer Device=6
Device=5 CAN Access Peer Device=7
Device=6 CAN Access Peer Device=0
Device=6 CAN Access Peer Device=1
Device=6 CAN Access Peer Device=2
Device=6 CAN Access Peer Device=3
Device=6 CAN Access Peer Device=4
Device=6 CAN Access Peer Device=5
Device=6 CAN Access Peer Device=7
Device=7 CAN Access Peer Device=0
Device=7 CAN Access Peer Device=1
Device=7 CAN Access Peer Device=2
Device=7 CAN Access Peer Device=3
Device=7 CAN Access Peer Device=4
Device=7 CAN Access Peer Device=5
Device=7 CAN Access Peer Device=6
***NOTE: In case a device doesn't have P2P access to other one, it falls back to normal memcopy procedure.
So you can see lesser Bandwidth (GB/s) and unstable Latency (us) in those cases.
P2P Connectivity Matrix
D\D 0 1 2 3 4 5 6 7
0 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 1
3 1 1 1 1 1 1 1 1
4 1 1 1 1 1 1 1 1
5 1 1 1 1 1 1 1 1
6 1 1 1 1 1 1 1 1
7 1 1 1 1 1 1 1 1
Unidirectional P2P=Disabled Bandwidth Matrix (GB/s)
D\D 0 1 2 3 4 5 6 7
0 893.37 2.12 2.12 2.12 3.17 2.12 2.12 1.31
1 2.15 896.44 1.61 1.61 2.15 1.61 1.61 1.09
2 2.15 1.61 895.42 1.60 2.15 1.61 1.61 1.09
3 2.13 1.59 1.59 894.90 2.11 1.59 1.60 1.08
4 3.17 2.13 2.13 2.12 896.44 2.12 2.12 1.31
5 2.15 1.61 1.61 1.61 2.15 896.44 1.61 1.09
6 2.15 1.61 1.61 1.61 2.15 1.61 896.41 1.09
7 1.30 1.08 1.08 1.08 1.30 1.08 1.08 895.42
Unidirectional P2P=Enabled Bandwidth (P2P Writes) Matrix (GB/s)
D\D 0 1 2 3 4 5 6 7
0 893.37 3.30 3.30 3.29 6.55 3.30 3.30 1.65
1 3.30 897.99 3.30 3.29 3.30 3.30 3.30 1.65
2 3.30 3.30 896.44 3.29 3.30 3.30 3.30 1.65
3 3.22 3.27 3.25 897.47 3.25 3.25 3.22 1.64
4 6.59 3.30 3.30 3.29 896.44 3.30 3.30 1.65
5 3.30 3.30 3.30 3.29 3.30 897.47 3.30 1.65
6 3.30 3.30 3.30 3.29 3.30 3.30 897.39 1.65
7 1.65 1.65 1.65 1.64 1.65 1.65 1.65 896.44
Bidirectional P2P=Disabled Bandwidth Matrix (GB/s)
D\D 0 1 2 3 4 5 6 7
0 900.58 3.10 3.10 3.08 4.65 3.10 3.10 1.91
1 3.06 901.36 2.36 2.34 3.06 2.36 2.35 1.59
2 3.06 2.36 902.10 2.33 3.06 2.35 2.36 1.59
3 3.04 2.35 2.35 902.14 3.04 2.36 2.35 1.58
4 4.64 3.11 3.11 3.08 901.62 3.11 3.11 1.92
5 3.05 2.35 2.35 2.34 3.05 901.31 2.36 1.59
6 3.05 2.35 2.35 2.33 3.06 2.36 901.88 1.59
7 1.83 1.55 1.55 1.54 1.83 1.55 1.55 901.10
Bidirectional P2P=Enabled Bandwidth Matrix (GB/s)
D\D 0 1 2 3 4 5 6 7
0 901.09 6.33 6.33 6.31 12.84 6.33 6.33 3.20
1 6.33 901.88 4.57 4.55 6.33 6.30 6.30 3.20
2 6.33 4.57 901.88 4.55 6.33 6.29 6.30 3.20
3 6.31 4.55 4.50 901.10 6.31 6.27 6.27 3.20
4 12.85 6.33 6.33 6.31 902.40 6.34 6.33 3.20
5 6.33 6.30 6.29 6.27 6.33 901.10 4.57 3.21
6 6.33 6.30 6.30 6.28 6.33 4.57 901.62 3.21
7 3.20 3.20 3.20 3.20 3.20 3.21 3.21 901.10
P2P=Disabled Latency Matrix (us)
GPU 0 1 2 3 4 5 6 7
0 1.60 16.26 15.91 15.24 15.55 16.27 16.71 16.59
1 16.22 1.54 15.94 17.13 15.98 16.02 15.59 15.98
2 16.06 15.64 1.66 14.01 15.89 15.98 16.78 16.03
3 16.04 16.59 36.38 1.47 17.38 16.73 16.18 16.54
4 14.12 14.45 14.13 37.05 1.42 13.42 14.32 13.30
5 13.79 14.28 14.12 15.60 13.42 1.39 14.65 17.04
6 16.78 12.27 12.20 13.97 12.65 12.27 1.45 16.78
7 16.08 13.30 16.87 14.63 13.68 13.30 13.49 1.44
CPU 0 1 2 3 4 5 6 7
0 3.07 8.10 7.93 7.96 8.19 7.70 8.19 7.61
1 8.04 3.13 7.71 7.24 8.05 7.57 7.94 7.60
2 7.42 7.86 2.83 7.48 7.78 7.73 7.86 7.99
3 7.93 7.45 7.55 2.91 8.07 7.74 7.65 7.80
4 7.99 7.94 7.59 7.92 3.21 8.09 8.24 7.88
5 7.67 7.50 7.79 27.63 7.88 3.13 8.24 7.87
6 7.83 7.89 7.43 7.41 7.86 8.23 3.13 8.17
7 7.68 7.42 7.84 7.67 8.22 7.83 7.97 3.19
P2P=Enabled Latency (P2P Writes) Matrix (us)
GPU 0 1 2 3 4 5 6 7
0 1.60 1.18 1.19 1.22 1.19 1.20 1.18 1.21
1 1.18 1.53 1.16 1.18 1.18 1.16 1.15 1.18
2 1.20 1.21 1.65 1.19 1.18 1.18 1.19 1.18
3 1.10 1.09 1.12 1.50 1.11 1.11 1.09 1.10
4 1.15 1.16 1.16 1.19 1.43 1.18 1.20 1.17
5 1.21 1.18 1.14 1.21 1.16 1.39 1.14 1.14
6 1.17 1.19 1.15 1.17 1.14 1.18 1.44 1.17
7 1.24 1.22 1.21 1.20 1.18 1.18 1.19 1.44
CPU 0 1 2 3 4 5 6 7
0 3.07 2.33 2.19 2.32 2.39 2.17 2.32 2.32
1 2.47 3.13 2.37 2.26 2.18 2.16 2.27 2.39
2 2.26 2.21 2.99 2.22 2.32 2.38 2.32 2.32
3 2.43 2.28 2.37 3.09 2.23 2.24 2.19 2.33
4 2.53 2.32 2.31 2.39 2.93 2.43 2.57 2.48
5 2.43 2.36 2.44 2.54 2.55 3.31 2.40 2.38
6 2.41 2.43 2.53 2.39 2.52 2.63 3.22 2.54
7 2.63 2.40 2.39 2.40 2.47 2.55 2.39 3.06
NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment