Skip to content

Various MTU fixes. #749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 8, 2025
Merged

Various MTU fixes. #749

merged 4 commits into from
May 8, 2025

Conversation

FelixMcFelix
Copy link
Collaborator

First, we missed one branch in the overlay action for setting the mtu_unrestricted property on VPC-local packets. This ended up getting missed because the bench/xde-test runner set up the routing table differently to omicron.

Second, when we open up some limited ICMP families for Nexus (intending to support PMTUD), the illumos TCP stack will have its own (and better!) idea about what MSS is actually assigned to a flow. We now reuse this MSS when not explicitly boosting a flow into jumbo frame territory.

Closes #747.
Closes #748.

First, we missed one branch in the overlay action for setting the
`mtu_unrestricted` property on VPC-local packets. This ended up getting
missed because the bench/xde-test runner set up the routing table
differently to omicron.

Second, when we open up some limited ICMP families for Nexus (intending
to support PMTUD), the illumos TCP stack will have its own (and better!)
idea about what MSS is actually assigned to a flow. We now reuse this
MSS when not explicitly boosting a flow into jumbo frame territory.

Closes #747.
Closes #748.
@FelixMcFelix FelixMcFelix requested a review from rcgoodfellow May 7, 2025 16:49
@FelixMcFelix
Copy link
Collaborator Author

So far, this change has been tested by bringing up a local omicron setup on my helios workstation. Given that this is in single-sled mode, all traffic delivery between guests must go via the guest_loopback pathway. The simplest way to check the new behaviour was to insert a few extra statements into xde_mc_tx_one:

if mtu_unrestricted {
    opte::engine::err!("yay!");
} else {
    opte::engine::err!("nay...");
}

Crude, but we can see this is actually being set now. It wasn't before (but was in cargo kbench):

May  7 16:30:11 farme xde: [ID 130198 kern.warning] WARNING: yay!
May  7 16:33:34 farme last message repeated 2332670 times

Instance connectivity worked fine in this setup. I'm going to pile this up with a few extra OPTE/stlouis changes on a racklette shortly.

@FelixMcFelix
Copy link
Collaborator Author

Some further proof from berlin, using an anti-affinity group to guarantee placement:

BRM42220023 # snoop -c10 -rd cxgbe0 'udp and port 6081'
Using device cxgbe0 (promiscuous mode)
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=3096
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
fd00:1122:3344:102::1 -> fd00:1122:3344:101::1 UDP D=6081 S=42966 LEN=8960
10 packets captured

I don't yet have speeds with promisc recorded, but we should get that on dogfood. Results without promisc (via the siphon branch) will be in https://github.com/oxidecomputer/stlouis/issues/744.

@FelixMcFelix FelixMcFelix merged commit 9e79588 into master May 8, 2025
10 checks passed
@FelixMcFelix FelixMcFelix deleted the various-mtu-fixes branch May 8, 2025 21:17
FelixMcFelix added a commit to oxidecomputer/omicron that referenced this pull request May 8, 2025
* Set MEOI packetinfo on Rx paths (oxidecomputer/opte#750)
* Various MTU fixes. (oxidecomputer/opte#749)
FelixMcFelix added a commit to oxidecomputer/omicron that referenced this pull request May 9, 2025
Includes some perf-related bugfixes. Sled-to-sled TCP traffic should now
be making use of the full ~9kiB MTU.

* Set MEOI packetinfo on Rx paths (oxidecomputer/opte#750)
* Various MTU fixes. (oxidecomputer/opte#749)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

XDE should listen to MSS suggestions made by senders for rack-external traffic Rack-internal traffic is using a 1500B MTU for LSO
2 participants