R.Pi PXE Bootloader doesn't work on IPv6 only network

[owendelong]

Describe the bug
The PXE Loader on the Pi appears to be IPv4 only. Please add IPv6 support.

To reproduce

1. Create an IPv6 only network with properly configured PXE Boot server
2. Attempt to boot a Pi via PXE on said network.

Expected behaviour
I expect the Pi to attempt to get an IPv6 address via appropriate RS/NS messages and given the A and/or M flags in the RA, to follow those instructions and request appropriate data from the DHCPv6 server.

Actual behaviour
No IPv6 packets are emitted by the R.Pi and IPv6 Router Advertisements are ignored by the Pi during the PXE boot.

System

• Which model of Raspberry Pi? e.g. Pi3B+, PiZeroW
  ALL PXE capable Raspberry Pis are effected.

• Which OS and version (cat /etc/rpi-issue)?
  ALL OS are effected -- This problem occurs prior to OS Boot.

• Which firmware version (vcgencmd version)?
  All firmware versions tested to date (April 28, 2023) behaved identically

• Which kernel version (uname -a)?
  All kernels are effected... This problem occurs prior to loading the kernel.

Logs
This problem occurs before the kernel is loaded, there is no logging available during PXE Boot.

Additional context
I believe the above is a sufficient description of the situation. I will endeavor to monitor comments or requests for additional information on this issue and respond timely. While I think this is a bug in this day and age, I understand that some may think it more of a feature request.

[peterharperuk]

Are you expecting it to follow some "standard" for IPv6 network boot? If you are - do you have any details? RPi 4 has some unproven IPv6 boot functionality if that's any use https://www.raspberrypi.com/documentation/computers/remote-access.html#network-booting-using-ipv6

[owendelong]

Yes. I’m not sure why nfsroot is protocol specific to IPv4, but it sounds like the newer firmware solves the issues I felt necessary to report here. Hopefully that will soon move from experimental to standard.

In terms of DHCP server support, we are already using KEA which is a vastly superior alternative to dnsmasq (which I consider fairly awful, tbh) and has several advantages over the old isc dhcpd. (KEA is from ISC also and is a ground-up fresh implementation incorporating decades of operational experience and lessons learned in a well designed and more consistent framework).

[peterharperuk]

Interesting - I might do some testing with KEA

[owendelong]

FWIW, our entire environment is here: https://github.com/socallinuxexpo/scale-network

Our KEA environment is built in NIX from here: https://github.com/socallinuxexpo/scale-network/blob/master/nix/machines/core/common.nix

Most of the important KEA configuration fragments are here: https://github.com/socallinuxexpo/scale-network/tree/master/facts/servers/examples/dhcp

In case that helps.

This is the automation and configuration for the Southern California Linux Expo. Next year will be our 21st annual conference. It generally runs from a Thursday through a Sunday in March. Setup of the network usually starts the Monday before and Teardown usually starts about 5PM on Sunday and we usually have the truck fully unloaded with everything back in storage before 9PM on Sunday.

The network consists of 150+ WAPs (Open WRT), 50+ 48 Port 1U PoE switches (Juniper EX4200-48P), 3 Juniper SRX-300 routers, and about 4 miles of deployed Ethernet cables (Plus some fiber jumpers).

All VLANs are either Dual Stack or IPv6 Only. (Signs is the primary IPv6-only VLAN where this is an issue for us).

I know this is a bit of a digression from the actual issue here, but I figure sometimes understanding the deployment environment helps gain insight into solutions.

[owendelong]

You may run into additional troubles since the decision was made to close raspberrypi/rpi-eeprom#182 without properly obtaining an IANA number for Option93.
However, according to https://www.iana.org/assignments/dhcpv6-parameters/dhcpv6-parameters.xhtml#processor-architecture there is Value 0x0029 for Raspberry Pi.

I couldn't find a corresponding map for DHCPv4. It looks like that's usually handled as a substring of Option 60 containing "PXEClient:Arch:xxxxx" where xxxxx is the 5 digit hex version of the assigned architecture number.

The most recent document I could find lists the following:

# 00:06, 00:07, 00:09 EFI (x86, x64, ByteCode) respectively
# 00:00, 00:02 BIOS (x86, Itanium)
# 00:03 thru 00:05, 00:08 -- Deprecated
# 00:0a thru 00:0b, 00:15 thru 00:18 -- ARM
# 00:0c thru 00:0e -- PowerPC / Power OPAL
# 00:0f thru 00:14 -- Boot from HTTP
# 00:19 thru 00:1e -- RISC-V
# 00:1f thru 00:20 -- S390
# 00:21 thru ff:ff -- Unassigned (13 Jan 2020)

So if the Pi firmware is updated to request netbook over IPv6, I suggest properly setting option 93 to 0x0029 for DHCP6 requests. I would argue that Any of the ARM choices (000a, 000b, 0015-0018) would be a better choice than the current report of x86 BIOS, but I believe a better solution would be to seek a proper IANA registration.

I'm honestly not sure where I found that table and I haven't been able to find it searching this evening. A greatly abbreviated version is present in RFC4758 which defines that particular syntax for Option 60. The RFC for Option 60 in DHCP4, however, is basically a free-for-all allowing any vendor to put whatever they want there.

[peterharperuk]

However, according to https://www.iana.org/assignments/dhcpv6-parameters/dhcpv6-parameters.xhtml#processor-architecture there is Value 0x0029 for Raspberry Pi

I had that added and it IS used if you boot using IPV6 for CLIENT_ARCH_TYPE (option 61). I don't believe option 93 is relevant for DHCPv6.

[owendelong]

OK, so if we upgrade the firmware, with the PXE on the R.Pi now try to get an IPv6 address? The firmware we were running never even asked for an IPv6 address (No RS, no DHCPv6 Request, and no apparent processing of RA packets).

If that's the case, I'll try to test this weekend.

[peterharperuk]

Currently it's IPV4 or IPV6 not both. You have to modify the bootloader configuration for IPV6 (see https://www.raspberrypi.com/documentation/computers/remote-access.html#network-booting-using-ipv6)

Bootloader
Modify the configuration to tell it to attempt network boot via IPv6 rather than IPv4.

BOOT_ORDER=0xf21 # 2=Network boot
USE_IPV6=1 # Enable IPv6 network boot
BOOT_UART=1 # Debug

To revert to IPv4 network boot just remove the USE_IPV6 line from boot.conf.

[timg236]

Closing as resolved because ipv6 is support is available if really desired. No plans to extend it further right now

[MatthewCroughan]

https://github.com/raspberrypi/documentation/blob/develop/documentation/asciidoc/computers/raspberry-pi/boot-net.adoc

You must set option 43 to "Raspberry Pi Boot "

[owendelong]

I think this should be reopened...

1. It would be nice if there were additional information reported to the user on what failed on the boot screen.

• We had systems trying to boot over IPv6 and apparently never getting an address.
• We tried this both with O and with M+O bits set in the RA.
• We definitely saw (we believe proper) DHCPv6 responses being sent back to the client.
• The client merely timed out as though those responses were never received. Some detail as to why responses were discarded would be useful.

2. If there is DHCP magic required, having that documented would be worth while.
3. Correct, Option 93 does not apply to DHCPv6
4. Option 43 also is not the Vendor-Specific option for DHCPv6, it uses 17 instead.
5. We aren't certain, but we believe this may be related to the fact that our DHCPv6 environment is using a relay agent to reach the DHCPv6 server (Juniper SRX300).
6. I'm happy to help document this once I can get it working.

[owendelong]

If it would be useful, I'm willing to post our KEA configuration and any other requested data here. (it's all in a public GitHub repository anyway -- socallinuxexpo/scale-network).

One difficulty for us is that while we have dev and test environments, the full-blown network only exists for 7 days per year. We start setting it up on Monday before the conference, the conference runs from Thursday to Sunday, then it all gets torn down and put back into storage for a year.

Nonetheless, I will replicate whatever is needed to solve this. We are really hoping to have a solution before next year's conference.

[peterharperuk]

So you're asking for IPV6 boot to work with the KEA DHCP server? Are there any guides for getting this running on a Raspberry PI? I vaguely remember looking at this years a go but wasn't easily able to get anything running so gave up as . I will have a look if you have proper support for IPV6 boot.

[sarcasticadmin]

So you're asking for IPV6 boot to work with the KEA DHCP server? Are there any guides for getting this running on a Raspberry PI? I vaguely remember looking at this years a go but wasn't easily able to get anything running so gave up as . I will have a look if you have proper support for IPV6 boot.

@peterharperuk The Kea DHCP server just happens to be what we've been running for the conference network that @owendelong mentioned. The issue is that there are outstanding unknowns for what is needed to make the rpi boot via an ipv6 only network @owendelong has called these out above: #1808 (comment)

Attached is a slimmed down version of our kea config (excuse the .txt prefix, github upload make me add it):
slim-dhcp6-server.conf.txt You will need to modify this config for your particular kea server ip, tftp server, and ipv6 subnet(s) to run the Kea DHCPv6 service.

[sarcasticadmin]

follow up: #1938