Jumbo Frames

From ArchWiki

Jump to: navigation, search


Contents

Introduction

What are Jumbo Frames

The following paragraph was taken from the [Wikpedia Jumbo Frame Article]: In computer networking, jumbo frames are Ethernet frames with more than 1,500 bytes of payload (MTU). Conventionally, jumbo frames can carry up to 9,000 bytes of payload, but variations exist and some care must be taken when using the term. Many, but not all, Gigabit Ethernet switches and Gigabit Ethernet network interface cards support jumbo frames, but all Fast Ethernet switches and Fast Ethernet network interface cards support only standard-sized frames. Most national research and education networks (such as Internet2/NLR, ESnet, GÉANT, and AARNet) support jumbo frames, but most commercial Internet service providers do not support jumbo frames.

Using a larger mtu value (jumbo frames) can significantly speed up your network transfers. See [this webpage] for more.

Requirements for Jumbo Frames

  1. Must be GigLAN backbone (i.e. 1000baseT)
  2. Local NICs in the local PCs must support JFs
  3. Switches must support JFs

Real Word Examples

It is important that you test various frame sizes on your hardware doing typical file transfers to determine the optimal setting for you. There isn't a one-size-fits-all setting for jumbo frames. In fact, as illustrated below, depending on the hardware you're testing, a frame size that's too large can actually hurt your network throughput.

I tested three different frame sizes (standard 1500, jumbo 4000, and jumbo 9000) under several different test conditions (single large file and multiple small files). The results indicate that for my hardware, a 4000 frame size gave the best results. This result is not general; you will need to evaluate the effect that different frame sizes have on different file transfers on your own hardware to pick the right size for your application.

Both machine A and machine B are using approx. 5-year-old EDIE hardrives; a modern SATAII HDD and M/B will give higher throughput for sure, but the data are still valid. Also, both machines are using [Dlink 530T] NICs (PCI bus) and are set to the frame size indicated in the tables below (1500, 4000, and 9000).

Test 1 - Single large file (1,048,522 kb) via Samba

Times and throughput represent the average of three runs

mtu=1500 t (sec) Kb/sec mtu=4000 t (sec) Kb/sec mtu=9000 t (sec) Kb/sec
A to B 48 21,844 A to B 44 23,830 A to B 49 21,398
B to A 81 12,945 B to A 41 25,574 B to B 41 25,574

Summary of Test 1

4k vs. 1.5k  % change 9k vs. 1.5k  % change
A to B +9 % A to B -2 %
B to A +98 % B to A +98 %


Test 2 - Several small files (1,283,439 kb total) via Samba

Times and throughput represent the average of three runs

mtu=1500 t (sec) Kb/sec mtu=4000 t (sec) Kb/sec mtu=9000 t (sec) Kb/sec
A to B 59 21,753 A to B 51 25,165 A to B 57 22,516
B to A 94 13,654 B to A 46 27,901 B to B 49 26,193

Summary of Test 2

4k vs. 1.5k  % change 9k vs. 1.5k  % change
A to B +16 % A to B +4 %
B to A +4% B to A +92 %


Sustained Transfer Speed

I got an average speed of 26.6 MB/s on a 30+ GB xfer using 4k jumbo frames. The slowest drive in the link was a Western Digital WD3200JB - 5-year-old, EIDE 320 GB (ATA100/7200 RPM/8 MB cache). Burst speeds (i.e. 300-400 meg files) through the network are about 80 % of the burst speed drive-to-drive which isn't too bad (approx. 30 MB/s network vs. approx 38 MB/s drive-to-drive).

Example LAN Architecture using Jumbo Frames

PC1 (JF Enabled)

PC2 (JF Enabled) <----> GigLAN Switch (JF Enabled) <----> Router (JF Unenabled) <----> Cable modem (JF Unenabled)

PC3 (JF Enabled)

In the above example, all the PCs have NICs set to use JFs and are all connected to a GigLAN switch that can also use JFs. The switch is in turn connect via the uplink port to a router that cannot use jumbo frames which is in turn connected to a cable modem which also cannot use jumbo frames.

Contrary to what some websites state, this setup works 100 % fine. Xfers inside the JF portion on the network (i.e. behind the switch) are very fast. Xfers to the WAN (the Internet) from PCs behind the switch are just as fast as a PC without JFs enabled connected either directly to the cable modem, or to the router.

Using Jumbo Frames on Arch Linux

Temporarily Enabling Jumbo Frames

Again if both your NIC and your router/switch supports jumbo frames, making a temporary change to the mtu size for the NIC is trivial. Simple invoke ifconfig with the mtu parameter as follows:

# ifconfig ethx mtu y

Where ethx is the ethernet adapter in question (eth0, eth1, etc.) and y is the size of the frame you wish to use (1500, 4000, 9000). You can verify that the setting has been applied like this:

$ ifconfig | grep MTU

Mine returned:

UP BROADCAST RUNNING MULTICAST  MTU:4000  Metric:1
UP LOOPBACK RUNNING  MTU:16436  Metric:1
Note: This setting will not be retained after a reboot.

To set the mtu size on a system boot, see the next section.

Setting Your Network to Enable Jumbo Frames at Boot

Dynamic IP Configuration =

See [this forum post] for a suggestion by tomk to enable Jumbo Frames using a DHCP setup via netcfg and the IFOPTS profile parameter. Alternatively, try the /etc/rc.local method described below. I don't use a dynamic config so I can guarantee that it works.

Static IP Configuration

If you're able to use a static IP configuration, enabling jumbo frames is pretty straightforward. Edit your /etc/rc.conf to use a static IP. As an example, the relevant lines from mine are shown below:

eth1="eth1 192.168.1.3 netmask 255.255.255.0 broadcast 192.168.1.255"
INTERFACES=(eth1)
gateway="default gw 192.168.1.1"
ROUTES=(gateway)

See the [Network] wiki page for a complete guide.

Next place a line in your /etc/rc.local to setup the jumbo frames of a size of your choosing via an ifconfig line. In my example, I'm setting 4k jumbo frame to eth1 (modify your line accordingly):

#!/bin/bash
#
# /etc/rc.local: Local multi-user startup script.
#
/sbin/ifconfig eth1 mtu 4000

Modify your /etc/reslov.conf to contain lines to your the DNS's that your ISP uses.

search my.ips.net.
nameserver a.b.c.d
nameserver a.b.c.d
nameserver a.b.c.d
Tip: If you are unsure what values to use in the /etc/reslov.conf simply configure your system to use DHCP and look at the /etc/reslov.conf which should be automatically set for you. You may use these values.

Finally, /etc/conf.d/dhcpcd needs to be tweaked to keep the dhcpcd from overwriting your /etc/reslov.conf which will happen without this modification:

# Arguments to be passed to the DHCP client daemon
#

# DHCPCD_ARGS="-q"
DHCPCD_ARGS="-C resolv.conf -q"

Reboot your machine to verify that:

  1. Your network is functional.
  2. It is using your jumbo frame size.

To test functionality, try to ping www.google.com for example. To see your mtu size, use ifconfig and grep as described above:

$ ifconfig | grep MTU

The result should reflect the mtu sze you specified in your /etc/rc.local.

Personal tools