Is 2.5G-Ethernet a realistic way to set up faster home networking? Fortunately the new mainboard I just procured has built-in support for this standard. Networking hardware also has become more affordable recently. The cheapest PCIe based adapters start at around 26€ and USB based ones at around 33€. The first passively cooled switch is available at a slightly more affordable price than it’s expensive competition: QNAP QSW-1105-ST for around 115€. All this seems to be at least in part possible thanks to Realtek’s line-up of interesting controllers. The Realtek RTL8125B can be found on many mainboards and the Realtek RTL8156 powers USB based adapters. Fortunately these chips are supported out of the box in recent Linux kernels (Ubuntu 20.04 and Fedora 33 didn’t need any kernel/driver updates). To do some benchmarks I also bought a DeLOCK 65990 RJ-45 USB-C adapter. I chose this adapter mostly because of it’s availability. Since all affordable USB adapters use the same Realtek chipset, there shouldn’t be a difference in the performance.

Gigabyte B550 Aorus Pro

Before we begin with the benchmarks. Let’s have a brief look at the mainboard or rather it’s pros and cons.

• A lot of reviews claim very good overclocking capabilities. Especially memory overclocking should be good on this board.
• The IO on the back is quite extensive. There are loads of legacy USB2.0 ports and six USB3.2 (Gen1/2) ports. A lot of B550 boards only ship with four of those.
• The Audio output is powered by the Realtek AC1220. The signal quality is quite good, especially compared to my previous board.
• Internally we’ve got six SATA ports, five PCIe slots and two m.2 PCIe slots. Although not all of these extension ports can be used simultaneously.
• As indicated the mainboard supports 2.5G-Ethernet.
• With ~160€ the board isn’t cheap, but still more affordable than other mainboards with a similar feature set.

The only significant improvement this board could benefit from is another PCIe layout. Unfortunately every slot except for the top-most 16-lane slot is connected to the chipset. The chipset has to share it’s 4-lane connection to the cpu amongst many connected peripherals and extensions cards. In a situation with loads of disks and PCIe adapters, this could pose a bandwidth limitation. Although I have to say that this is likely not a common issue. In fact most B550 boards have a similar PCIe configuration. One of the few which don’t, is the Asus ROG Strix B550-E Gaming. It’s capable to split the 16 PCIe-4.0 lanes from the cpu amongst two mechanical full size PCIe slots. Unfortunately at the same time it cuts back on the rear-IO and is significantly more expensive.

2.5G-Ethernet Benchmarks

Coming back to the main feature discussed in today’s post. The 2.5G-Ethernet technology and performance. The test systems are the desktop machine partially described above and the AsRock DeskMini A300 powered by a Ryzen 3 2200G and a bunch of SSDs. For the sake of curiosity I also added the Raspberry PI 4 as comparison.

When thinking about network bandwidth, the first thought often is the maximum throughput. Naturally this is where significant time can be saved. Large video files aren’t uncommon, especially when talking about network storage. Looking at the numbers with classic 1G-Ethernet the NFS share performed close to it’s theoretical limit by reading and writing with up to ~110MiB/s. The 2.5G-Ethernet set-up obviously surpassed this with up to ~270MiB/s read and write throughput. To ensure that this was not limited by the SSD mirror in the NAS, I also tested it with the faster system drive in the NAS. As we’ve seen in my previous tests with the Raspberry PI4, the 1G-Ethernet connection can be saturated. Using 2.5G-Ethernet on the Raspberry PI4 showed significant issues with power draw and only an externally powered USB-hub could resolve the issue. The less than satisfying throughput is likely caused by USB limitations on the board and/or the relatively old USB-hub.

The second test consisted of a few directories with photos. The most interesting result here is the significantly lower write performance using the 2.5G-Ethernet connection. The read performance is still close to the maximum we’ve seen before, but the write performance seems to suffer from the significantly smaller file size. The Raspberry PI4 with it’s built-in 1G-Ethernet still keeps up quite well.

What happened here? Transmitting extremely small files increases the protocol overhead significantly and response times start to have a major impact on the total achievable performance. In order to test this I unpacked the Linux kernel source and copied it back and forth. Like the plot above shows the performance is horrid no matter what technology is used. Truth be told, nevertheless the overall results were pretty much as expected, but yet we still have one result which is very surprising. Comparing the 1G and 2.5G performance, the latter failed spectacularly in this test. The only explanation I can come to think of is that the 2.5G-Adapter behaves differently due to being USB based instead of directly connected to the PCIe bus. While it is only a guess, the network controller itself in the USB-adapter should be similar to the one on the desktop mainboard, as they are part of the same line-up. I also don’t see a reason why the controllers should perform worse than when running in the 1Gbit/s mode. Leaving the USB as a potential culprit.

Trouble with the Raspberry PI 4 and Power Consumption

When trying to use the network adapter with the Raspberry PI 4, I ran into several power issues. I even tried to power the PI4 with a 385 Watt ATX power supply ending up with the same conclusion. As soon as I tried to run the network adapter and the external SSD from the PI4’s usb ports, I got I/O errors on the SSD. The SSD alone works just fine.

Since the Raspberry PI4 should be able to deliver a few Watts on the USB ports I checked the power consumption of the network adapter. Surprisingly it already consumed 3 Watts after just hooking it up. The whole set-up pulled in over 12 Watts while attempting to write to the SSD through the network connection. Of course this does not include any short spikes in power usage which might not be shown by the rather cheap power meter I used. Although the 12 Watts are theoretically still below the limit of what the official PI4 power supply can deliver, I believe that I either didn’t see the short spikes on the power meter or the PI4 might not be able to pass through enough power to the USB ports.

Verdict

Considering 2.5G-Ethernet when buying new infrastructure is definitely a good idea. The standard will in my opinion become mainstream over the next years, since the power usage is acceptable, cheap controllers are available and existing Ethernet cables can be reused (CAT5e or better).

The few limitations found while creating the benchmarks will likely become less of an issue with 2.5G being more tightly integrated. Nevertheless it might be interesting information to consider when buying an adapter. Choosing a PCIe based adapter card might be a better option over yet another USB dongle.

The Raspberry PI4 is as amazing as it always has been. It simply has a few limitations and today it showed some of those. Realistically in many uses cases where the RPI4 is used, 1G-Ethernet will be more than good enough. If faster network is required, having a look at the ODROID H2+ might be an option. Both it’s network ports are powered by the Realtek RTL8125B, the same controller used on the Gigabyte B550 Aorus Pro.