Hands On with the Gigabyte Maple Ridge Thunderbolt 4 Card

I’ve been exploring Thunderbolt quite a bit lately. As a follow up to my post about using Thunderbolt in a NAS, I recently bought a Gigabyte Maple Ridge Thunderbolt 4 add on card. I’ve captured a few thoughts, observations and pics here.

Gigabyte GC-Maple Ridge

Observations

Physical Characteristics

The first thing I noticed about this card is that it’s nearly identical to it’s cousin, the Thunderbolt 3 Titan Ridge Card:

The actual form factor is identical, including the etching on the PCI brackets. Under the aluminum shroud, however, lay obvious differences. Most relevant among them is the Intel Thunderbolt™ JHL8540 Controller IC:

Additionally, the addition of a 100 Watt power delivery controller (I think):

I did not need to short the pins on the TB_HEADER that I did in my previous post for the Titan Ridge card. I simply inserted the card into the server and booted it up. I haven’t made any BIOS changes either. I wanted to visually inspect the pinouts before shorting the pins, hence why I took pictures and removed the cover plate (shroud).

I’m going to install the connector with the pins shorted (ground tied?) to see if anything changes with regards to capability or recognized features.

Operating System Support (Linux)

In Linux (Centos 7) the device was recognized as seen by the output from lspci:

02:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
03:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
03:01.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
03:02.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
03:03.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
04:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
06:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]

Since this card is Thunderbolt 4, it uses the thunderbolt and xhci_hcd kernel modules for the USB4 Host Controller.

04:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020] (prog-if 40 [USB4 Host Interface])
06:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020] (prog-if 30 [XHCI])

I’ll explore that further and document it eventually.

Here’s another pci tree view (lspci -tv):

\-[0000:00]-+-00.0  Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMI2
	+-01.1-[02-07]----00.0-[03-07]--+-00.0-[04]----00.0  Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
	|                               +-01.0-[05]--
	|                               +-02.0-[06]----00.0  Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]
	|                               \-03.0-[07]--

Verbose output (lspci -vvv) can be viewed here.

Here is the system device tree:

[user@nas-04:/sys/bus/thunderbolt] $ tree
.
├── devices
│   ├── 0-0 -> ../../../devices/pci0000:00/0000:00:01.1/0000:02:00.0/0000:03:00.0/0000:04:00.0/domain0/0-0
│   ├── 0-1 -> ../../../devices/pci0000:00/0000:00:01.1/0000:02:00.0/0000:03:00.0/0000:04:00.0/domain0/0-0/0-1
│   ├── 0-1.0 -> ../../../devices/pci0000:00/0000:00:01.1/0000:02:00.0/0000:03:00.0/0000:04:00.0/domain0/0-0/0-1/0-1.0
│   └── domain0 -> ../../../devices/pci0000:00/0000:00:01.1/0000:02:00.0/0000:03:00.0/0000:04:00.0/domain0
├── drivers
│   └── thunderbolt-net
│       ├── 0-1.0 -> ../../../../devices/pci0000:00/0000:00:01.1/0000:02:00.0/0000:03:00.0/0000:04:00.0/domain0/0-0/0-1/0-1.0
│       ├── bind
│       ├── module -> ../../../../module/thunderbolt_net
│       ├── uevent
│       └── unbind
├── drivers_autoprobe
├── drivers_probe
└── uevent

I grabbed a dump of the nvm firmware at /sys/bus/thunderbolt/devices/0-0/nvm_active0 using dd. I parsed it through various tools (od, strings, binwalk) but didn’t see anything notable. I also looked at it visually with mc just for grins.

Here’s some other miscellaneous output:

root@nas-04:/sys/bus/thunderbolt/devices/0-0 # cat nvm_version
28.0

root@nas-04:/sys/bus/thunderbolt/devices/0-0 # cat vendor_name
GIGABYTE

root@nas-04:/sys/bus/thunderbolt/devices/0-0 # cat generation
4

root@nas-04:/sys/bus/thunderbolt/devices/0-0 # cat device_name
GC-MAPLE RIDGE

I expect that since Centos 7 with Linux Kernel latest (at the time of this writing: 5.13.4-1.el7.elrepo.x86_64) discovered the device, it should work in most Linux distributions with a newer linux kernel and fresh userspace tools.

Benchmarks

Networking Capability

I still only see about 16.8 Gigabit/sec when using the Thunderbolt4 network interface:

root@nas-04:~ # iperf3 -c 10.10.10.2
Connecting to host 10.10.10.2, port 5201
[  4] local 10.10.10.4 port 34898 connected to 10.10.10.2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.91 GBytes  16.4 Gbits/sec    0   3.76 MBytes
[  4]   1.00-2.00   sec  1.94 GBytes  16.7 Gbits/sec    0   3.76 MBytes
[  4]   2.00-3.00   sec  1.95 GBytes  16.8 Gbits/sec    0   3.76 MBytes
[  4]   3.00-4.00   sec  1.94 GBytes  16.7 Gbits/sec    0   3.94 MBytes
[  4]   4.00-5.00   sec  1.95 GBytes  16.8 Gbits/sec    0   3.94 MBytes
[  4]   5.00-6.00   sec  1.95 GBytes  16.8 Gbits/sec    0   3.94 MBytes
[  4]   6.00-7.00   sec  1.95 GBytes  16.7 Gbits/sec    0   3.94 MBytes
[  4]   7.00-8.00   sec  1.95 GBytes  16.8 Gbits/sec    0   3.94 MBytes
[  4]   8.00-9.00   sec  1.94 GBytes  16.7 Gbits/sec    0   3.94 MBytes
[  4]   9.00-10.00  sec  1.95 GBytes  16.8 Gbits/sec    0   3.94 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  19.4 GBytes  16.7 Gbits/sec    0             sender
[  4]   0.00-10.00  sec  19.4 GBytes  16.7 Gbits/sec                  receiver

iperf Done.

I do see twice the number of lanes as compared to the Thunderbolt 3 card (I think):

root@nas-04:/sys/bus/thunderbolt/devices/0-3 # cat rx_lanes
2

root@nas-04:/sys/bus/thunderbolt/devices/0-3 # cat tx_lanes
2

root@nas-04:/sys/bus/thunderbolt/devices/0-3 # cat rx_speed
20.0 Gb/s

root@nas-04:/sys/bus/thunderbolt/devices/0-3 # cat tx_speed
20.0 Gb/s

The PCIe limitation of 4 lanes is likely the reason for this.

Direct Attached Storage

Unfortunately, I don’t have a benchmark for this yet. If you’re interested in Thunderbolt 3, you can see some insane benchmarks using RAID-0 over here. I’ve connected the one of the same aVolution Thunderbolt 3 disks to the controller using an active Thunderbolt 4 cable, but it doesn’t enumerate. I haven’t explored it beyond just seeing the outcome of plug-n-play. No dmesg output, nada. I’ll follow up on this and post an update later.

Additional Pictures