Jumbo Frames
From ArchWiki
Contents |
Introduction
What are Jumbo Frames
The following paragraph was taken from the [Wikpedia Jumbo Frame Article]: In computer networking, jumbo frames are Ethernet frames with more than 1,500 bytes of payload (MTU). Conventionally, jumbo frames can carry up to 9,000 bytes of payload, but variations exist and some care must be taken when using the term. Many, but not all, Gigabit Ethernet switches and Gigabit Ethernet network interface cards support jumbo frames, but all Fast Ethernet switches and Fast Ethernet network interface cards support only standard-sized frames. Most national research and education networks (such as Internet2/NLR, ESnet, GÉANT, and AARNet) support jumbo frames, but most commercial Internet service providers do not support jumbo frames.
Using a larger mtu value (jumbo frames) can significantly speed up your network transfers. See [this webpage] for more.
Requirements for Jumbo Frames
- Must be GigLAN backbone (i.e. 1000baseT)
- Local NICs in the local PCs must support JFs
- Switches must support JFs
Real Word Examples
It is important that you test various frame sizes on your hardware doing typical file transfers to determine the optimal setting for you. There isn't a one-size-fits-all setting for jumbo frames. In fact, as illustrated below, depending on the hardware you're testing, a frame size that's too large can actually hurt your network throughput.
I tested three different frame sizes (standard 1500, jumbo 4000, and jumbo 9000) under several different test conditions (single large file and multiple small files). The results indicate that for my hardware, a 4000 frame size gave the best results. This result is not general; you will need to evaluate the effect that different frame sizes have on different file transfers on your own hardware to pick the right size for your application.
Both machine A and machine B are using approx. 5-year-old EDIE hardrives; a modern SATAII HDD and M/B will give higher throughput for sure, but the data are still valid. Also, both machines are using [Dlink 530T] NICs (PCI bus) and are set to the frame size indicated in the tables below (1500, 4000, and 9000).
Test 1 - Single large file (1,048,522 kb) via Samba
Times and throughput represent the average of three runs
mtu=1500 | t (sec) | Kb/sec | mtu=4000 | t (sec) | Kb/sec | mtu=9000 | t (sec) | Kb/sec | ||
A to B | 48 | 21,844 | A to B | 44 | 23,830 | A to B | 49 | 21,398 | ||
B to A | 81 | 12,945 | B to A | 41 | 25,574 | B to B | 41 | 25,574 |
Summary of Test 1
4k vs. 1.5k | % change | 9k vs. 1.5k | % change | |
A to B | +9 % | A to B | -2 % | |
B to A | +98 % | B to A | +98 % |
Test 2 - Several small files (1,283,439 kb total) via Samba
Times and throughput represent the average of three runs
mtu=1500 | t (sec) | Kb/sec | mtu=4000 | t (sec) | Kb/sec | mtu=9000 | t (sec) | Kb/sec | ||
A to B | 59 | 21,753 | A to B | 51 | 25,165 | A to B | 57 | 22,516 | ||
B to A | 94 | 13,654 | B to A | 46 | 27,901 | B to B | 49 | 26,193 |
Summary of Test 2
4k vs. 1.5k | % change | 9k vs. 1.5k | % change | |
A to B | +16 % | A to B | +4 % | |
B to A | +4% | B to A | +92 % |
Sustained Transfer Speed
I got an average speed of 26.6 MB/s on a 30+ GB xfer using 4k jumbo frames. The slowest drive in the link was a Western Digital WD3200JB - 5-year-old, EIDE 320 GB (ATA100/7200 RPM/8 MB cache). Burst speeds (i.e. 300-400 meg files) through the network are about 80 % of the burst speed drive-to-drive which isn't too bad (approx. 30 MB/s network vs. approx 38 MB/s drive-to-drive).
Example LAN Architecture using Jumbo Frames
PC1 (JF Enabled)
PC2 (JF Enabled) <----> GigLAN Switch (JF Enabled) <----> Router (JF Unenabled) <----> Cable modem (JF Unenabled)
PC3 (JF Enabled)
In the above example, all the PCs have NICs set to use JFs and are all connected to a GigLAN switch that can also use JFs. The switch is in turn connect via the uplink port to a router that cannot use jumbo frames which is in turn connected to a cable modem which also cannot use jumbo frames.
Contrary to what some websites state, this setup works 100 % fine. Xfers inside the JF portion on the network (i.e. behind the switch) are very fast. Xfers to the WAN (the Internet) from PCs behind the switch are just as fast as a PC without JFs enabled connected either directly to the cable modem, or to the router.
Using Jumbo Frames on Arch Linux
Temporarily Enabling Jumbo Frames
Again if both your NIC and your router/switch supports jumbo frames, making a temporary change to the mtu size for the NIC is trivial. Simple invoke ifconfig with the mtu parameter as follows:
# ifconfig ethx mtu y
Where ethx is the ethernet adapter in question (eth0, eth1, etc.) and y is the size of the frame you wish to use (1500, 4000, 9000). You can verify that the setting has been applied like this:
$ ifconfig | grep MTU
Mine returned:
UP BROADCAST RUNNING MULTICAST MTU:4000 Metric:1 UP LOOPBACK RUNNING MTU:16436 Metric:1
To set the mtu size on a system boot, see the next section.
Setting Your Network to Enable Jumbo Frames at Boot
Dynamic IP Configuration =
See [this forum post] for a suggestion by tomk to enable Jumbo Frames using a DHCP setup via netcfg and the IFOPTS profile parameter. Alternatively, try the /etc/rc.local method described below. I don't use a dynamic config so I can guarantee that it works.
Static IP Configuration
If you're able to use a static IP configuration, enabling jumbo frames is pretty straightforward. Edit your /etc/rc.conf to use a static IP. As an example, the relevant lines from mine are shown below:
eth1="eth1 192.168.1.3 netmask 255.255.255.0 broadcast 192.168.1.255" INTERFACES=(eth1) gateway="default gw 192.168.1.1" ROUTES=(gateway)
See the [Network] wiki page for a complete guide.
Next place a line in your /etc/rc.local to setup the jumbo frames of a size of your choosing via an ifconfig line. In my example, I'm setting 4k jumbo frame to eth1 (modify your line accordingly):
#!/bin/bash # # /etc/rc.local: Local multi-user startup script. # /sbin/ifconfig eth1 mtu 4000
Modify your /etc/reslov.conf to contain lines to your the DNS's that your ISP uses.
search my.ips.net. nameserver a.b.c.d nameserver a.b.c.d nameserver a.b.c.d
Finally, /etc/conf.d/dhcpcd needs to be tweaked to keep the dhcpcd from overwriting your /etc/reslov.conf which will happen without this modification:
# Arguments to be passed to the DHCP client daemon # # DHCPCD_ARGS="-q" DHCPCD_ARGS="-C resolv.conf -q"
Reboot your machine to verify that:
- Your network is functional.
- It is using your jumbo frame size.
To test functionality, try to ping www.google.com for example. To see your mtu size, use ifconfig and grep as described above:
$ ifconfig | grep MTU
The result should reflect the mtu sze you specified in your /etc/rc.local.