PXE on vanilla ASUSwrt firmware

As a previous dd-wrt user, I love the idea of adding services to my home router, but just lacked the time to commit to moving from ASUSwrt to merlin.

Since ASUSwrt uses dnsmasq for DNS and DHCP and offers telnet login by default, adding the necesary configurations for PXE was relatively simple, albeit it took some tinkering.

Most PXE-live implementations rely on read-write access to a filesystem via NFS – but I was after something that was truly read-only.

The setup:

• On the ASUS WRT:
  • DHCP server PXE options via dnsmasq
  • TFTP server via dnsmasq
  • tftp-root folder on a mounted USB-drive (plugged into the WRT – it will automount on /mnt/sdX)
  • lpxelinux.0 – for loading kernel/initrd via HTTP (/usr/lib/PXELINUX/lpxelinux.0)
  • syslinux64.efi – for x64 EFI boot support (/usr/lib/SYSLINUX.EFI/efi64/syslinux.efi)
• On an separate HTTP-server (RPi, in my case):
  • Full ISOs (live-filesystems, etc.) exposed via HTTP
    • Each ISO is mounted in a subdirectory of /var/www/html/ for easy fetching

OS’ supporting live-boot over PXE/HTTP

Debian was by far the most PXE-friendly OS to get running. Kali took some doing, but I got there eventually – thanks to this post and kali-linux-2017.1-amd64.iso!

I was also able to get debian working with using debian-live-9.3.0-amd64-lxde.iso

RAW-iso mode worked for getting Arch Linux (archlinux-2018.01.01-x86_64.iso @ 522MB) and TinyCore (Core-current.iso @ 12MB) booted (command-line only) – see the append arguments in ‘default’, below.

For some reason, even though Ubuntu, Lubuntu and Tails are all based on Debian, I was not able to get these to ‘fetch’ the filesystem over HTTP at all (vanilla iso); even trying to load the iso into memory ‘raw’ failed to find the live filesystem once the initramfs loaded…

I was also unsuccessful at getting Puppy Linux to boot from PXE (failed to find/mount the filesystem), despite following (similar) instructions from a few different places on how to rebuild the initrd to include the filesystem.

Functional images (Jan-2021 edit)

New additions/updates below; Haven’t got around to trying a more recent Kali 🤷‍♂️

HTTP-Fetch:

• Debian10 (debian-live-10.x.x-amd64-lxde.iso)
• Debian10 (debian-live-10.x.x-amd64-standard.iso)

ISO-Raw

• TinyCore[64]
• ArchLinux
• Docker (cli – boot2docker.iso) 🐳
• Win10PE (ampe.iso)

UEFI + PXE -friendly OS’ (Jan-2018 edit)

After a good while of tinkering, I found the following post in a discussion on the ArchLinux wiki:

The currently generated archiso file does not contain the required syslinux files for booting a PXE image. I have filed a bug for this at https://bugs.archlinux.org/task/50188

Bummer…

Custom dnsmasq options

Fortunately, customising the dnsmasq configuration is as easy as:

• adding additional options (1 per line) to /etc/dnsmasq.conf, and
• restarting the service (killall dnsmasq && dnsmasq –log-async -C /etc/dnsmasq.conf).

(Lower down I share the script I built to apply (new) options and restart the service with one incantation.)

The dnsmasq options I added were:

dhcp-boot=lpxelinux.0	default image
enable-tftp
tftp-root=/mnt/sdxX/<tftp-root>
dhcp-option=vendor:PXEClient,6,2b	“kill multicast”
tftp-no-blocksize	(not sure if this is necessary)
log-dhcp	outputs to /tmp/syslog.log
pxe-prompt=PXE-Booting,5	(not sure if this is necessary)
dhcp-match=set:bios-x86_64,option:client-arch,0	detect BIOS machines
dhcp-boot=tag:bios-x86_64,lpxelinux.0	assign bootfile for BIOS
dhcp-match=set:efi-x86_64,option:client-arch,7	detect EFI machines
dhcp-match=set:efi-x86_64,option:client-arch,9	detect EFI machines
dhcp-boot=tag:efi-x86_64,syslinux64.efi	assign bootfile for EFI
pxe-service=X86PC,lpxelinux.0,lpxelinux.0	assign bootfile for BIOS

EDIT (13-Jan): Corrected the dnsmasq EFI-detection/assignemnt options based on this git project.
EDIT (20-Jan): Had to fall-back to BIOS-only to make sure everything works. (EFI line removed are struck-out, above)

minimal tftp-root filestructure:

<tftp-root>/	ldlinux.c32 lpxelinux.0 pxelinux.0 syslinux64.efi ldlinux.e64
pxelinux.cfg/	default menu.msg
bootimage/	dos622.img dosnet.img memdisk memtest-kali memtest86p-5.01.bin vmlinuz-kali initrd-kali.img

With the upgrade to lpxelinux.0, I was able to eliminate all vmlinuz and initrd images from the TFTP server, instead pulling them from the HTTP server directly (read: much more quickly, too!). The kali vmlinuz/initrd entries are noted as an excercise for the reader.

Theoretically, one could also verify the checksum of the mounted .iso/squashfs to ensure no modifications had been injected, but I digress.

‘menu.msg’ contents

This text-file gets displayed by [l]pxelinux.0, and is meant to present the user with options provided by the PXE-server. Put in as little or as much as you want!

my pxelinux.cfg/menu.msg:

Local Boot options
 - local (boot from local storage)

PXE-BOOT Options
 > Tools
 - dos (dos 6.22)
 - dosnet (dos with networking support)
 - memtest (memtest86+ v5)
 - memtest-kali (memtest86+ v5)
> Live ISOs
 - kali (Kali in forensic-live mode)
 - debian (Debian in live-mode)

‘default’ contents

These instruct [l]pxelinux.0 which kernel/initrd/etc. files should be pulled (and from where), based on user-input (or lack thereof).

All paths are relative to <tftp-root> – ‘http://‘ paths (for kernel and initrd) are only supported with lpxelinux.0 (not pxelinux.0)

The ‘fetch’ option tells the initrd to download the filesystem from the provided path. if ‘httpfs’ is used with the same path

my ‘default’:

serial 0 115200
default local
timeout 150
prompt 1
display pxelinux.cfg/menu.msg

label local
  localboot 0

## Tools
label dos
  kernel bootimage/memdisk
  append initrd=bootimage/dos622.img
label dosnet
  kernel bootimage/memdisk
  append initrd=bootimage/dosnet.img
label memtest
  kernel bootimage/memdisk append initrd=bootimage/memtest86p-5.01.bin
label memtest-kali
  kernel bootimage/memtest-kali

## Live ISOs
label kali
  kernel bootimage/vmlinuz-kali
  append initrd=bootimage/initrd-kali.img edd=off boot=live noconfig=sudo username=root hostname=klive device=eth0 noswap components fetch=http://<HTTP-server IP>/pxe/kali/live/filesystem.squashfs
  IPAPPEND 2

label deb10
 kernel http://<HTTP-server IP>/pxe/deb10lxde/live/vmlinuz-4.19.0-11-amd64
 append initrd=http://<HTTP-server IP>/pxe/deb10lxde/live/initrd.img-4.19.0-11-amd64 boot=live components fetch=http://<HTTP-server IP>/pxe/deb10lxde/live/filesystem.squashfs

label debian
  #kernel bootimage/vmlinuz-debian
  kernel http://<HTTP-server IP>/pxe/debian/live/vmlinuz-4.9.0-4-amd64
  append initrd=http://<HTTP-server IP>/pxe/debian/live/initrd.img-4.9.0-4-amd64 boot=live components fetch=http://<HTTP-server IP>/pxe/debian/live/filesystem.squashfs
  IPAPPEND 2

label debcli
  kernel http://<HTTP-server IP>/pxe/debian10/live/vmlinuz-4.19.0-8-amd64
  append initrd=http://<HTTP-server IP>/pxe/debian10/live/initrd.img-4.19.0-8-amd64 boot=live components fetch=http://<HTTP-server IP>/pxe/debian10/live/filesystem.squashfs

## Direct ISO
label tc
  kernel bootimage/memdisk
  append initrd=http://<HTTP-server IP>/pxe/tinycore.iso iso raw

label arch
  kernel bootimage/memdisk
  append initrd=http://<HTTP-server IP>/pxe/archlinux.iso iso raw

label docker
  kernel bootimage/memdisk
  append initrd=http://<HTTP-server IP>/pxe/boot2docker.iso

The notice how the kali entries instruct lpxelinux.0 to pull vmlinuz/initrd via TFTP, while the debian ones pull directly from the HTTP server.

ISOLINUX ‘append’ options (as a reference) from their respective iso’s .cfg files:

Debian (isolinux.cfg)	#LABEL English (en) linux /live/vmlinuz-4.9.0-4-amd64 APPEND initrd=/live/initrd.img-4.9.0-4-amd64 boot=live components locales=en_US.UTF-8
Kali (live.cfg)	#label live-forensic append boot=live noconfig=sudo username=root hostname=kali noswap #label live-persistence append boot=live noconfig=sudo username=root hostname=kali persistence

HTTP-server files:

The HTTP server basically has a mount folder for each ISO: Kali and Debian, serving all files (including vmlinuz, initrd and filesystem.squashfs)

When using lpxelinux.0 (or gpxelinux.0) vmlinuz and initrd can also be loaded via HTTP, which is much quicker than doing so via TFTP.

Pulling it all together

IMHO, the key advantages of merlin over ASUSwrt would (could) have been HTTP-server hosted on the router and reboot-resilience; As things stand, if my router reboots (it’s on a UPS, so a very rare occurrence), I need to reapply the settings to /etc/dnsmasq.conf and resart the service. Fortunately, I am the only one using PXE at home, so connecting via telnet and running the script is a trivial operation.

The script itself is rather straightforward (for anyone with a little BASH experience) – and there are probably some inefficiencies, but I can take constructive criticism 🙂

#!/bin/sh

default_conf="/etc/dnsmasq.conf"

local_pxe_options='dhcp-boot=lpxelinux.0
enable-tftp
tftp-root=/mnt/sda1/tftpboot
#kill_multicast
dhcp-option=vendor:PXEClient,6,2b
tftp-no-blocksize
log-dhcp
#pxe-prompt=PXE-Booting,5
#PXEClient:Arch:00000
pxe-service=X86PC,lpxelinux.0,lpxelinux.0
#PXEClient:Arch:00007
#pxe-service=BC_EFI,syslinux64.efi,syslinux64.efi
#PXEClient:Arch:00009
#pxe-service=X86-64_EFI,syslinux64.efi,syslinux64.efi
#modified_by_pxe-enable_bios.sh'

usb_mountpoint="/tmp/mnt/sda1"
usb_mount_str="/dev/sda1 on /tmp/mnt/sda1 type vfat"

# Check USB is mounted
if $(mount | grep -qi "$usb_mount_str"); then
        cd ${usb_mountpoint}/tftpboot
else
        echo "Error: USB not mounted - exiting(1)"
        exit 1
fi

# check/apply dnsmaq config
dnsmasq_ps=$(ps w | grep dnsmasq | grep -v grep)
dnsmasq_args=$(echo "$dnsmasq_ps" | sed 's/^.*dnsmasq //g')
if ! $(echo "$dnsmasq_args" | grep -qi ".conf"); then
        echo "no .conf specified: \"$dnsmasq_args\" (assuming \"/etc/dnsmasq.conf\")"
        dnsmasq_conf_file="/etc/dnsmasq.conf"
else
        setopt shwordsplit
        for word in $dnsmasq_args; do
                if echo "$word" | grep -qi ".conf"; then
                        unsetopt shwordsplit
                        dnsmasq_conf_file="$word"
                        break
                fi
        done
        unsetopt shwordsplit
        if [ "$dnsmasq_conf_file" == "" ]; then
                echo "error parsing .conf filename from \"$dnsmasq_args\" - exiting(1)"
                exit 1
        fi
fi

modified="0"
for cmd in $local_pxe_options; do
        #echo "testing $cmd"
        if ! grep -qi $cmd $dnsmasq_conf_file; then
                # command not found
                echo "adding: $cmd"
                echo $cmd >> $dnsmasq_conf_file
                modified="1"
        fi
done

if [[ $modified -ne "0" ]]; then
        echo "Restarting dnsmasq"
        killall dnsmasq
elif [ "$(ps w | grep dnsmasq | grep -v grep)" == "" ]; then
        echo "Starting dnsmasq"
else
        echo "All pxe-related options are already present ($dnsmasq_conf_file)"
fi

if [ "$dnsmasq_conf_file" != "$default_conf" ]; then
        /usr/sbin/dnsmasq --log-async -C "$dnsmasq_conf_file"
else
        /usr/sbin/dnsmasq --log-async
fi

Next steps:

Diskless thin client(s) with NFS-root? Maybe Clonezilla to quickly restore a family member’s PC when they bungle it up? Only time will tell 🙂

 

(Edit Mar-2021: Clarifying The Setup – which services go where, and Custom dnsmasq Options)

• ASUSwrt
• Linux