Don't use diskimage-builder to build IPXE image

IPXE's build process already creates bootable images, so we just use one of
those. This builds much faster, has far fewer dependencies, and produces
a much smaller image. The largest difference in output is that the
resulting image no longer uses grub, but as we weren't using any
features of grub this also means a faster boot time.

For reference, a full build takes 62 seconds on my laptop. The new
Makefile also handles dependencies much better so, for example, an
incremental rebuild after changing the embedded script takes under 3
seconds.

The ipxe repo changes to being a submodule, which continues to allow us
to pin it to a specific commit.

We no longer build the -41 image, as that was oddly specific to some
undescribed use-case. It's also trivial for whoever needed it to do
themselves if the use case is still relevant.

Change-Id: I5a6a990e6966d6eeaa2aa87cd89bfbac6738f6ee

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "ipxe/ipxe"]
+	path = ipxe/ipxe
+	url = https://git.ipxe.org/ipxe.git

ipxe/.gitignore

@@ -1 +0,0 @@
-ipxe-boot.d

ipxe/Makefile

@@ -1,10 +1,19 @@
-# git ref to checkout from the iPXE git repository
-# This is the current SHA for HEAD on 9 Aug 2019
+IPXE_IMG=ipxe/src/bin/ipxe.iso
 
-IPXE_GIT_REF=c63ef427
+all: ipxe-boot.img ipxe-boot.qcow2
 
-ipxe-boot.qcow2:
-	ELEMENTS_PATH=./elements IPXE_GIT_REF=$(IPXE_GIT_REF) disk-image-create -x -o ipxe-boot ipxe-boot-image
+ipxe-boot.img: $(IPXE_IMG) script.ipxe
+	cp $(IPXE_IMG) $@
+
+ipxe-boot.qcow2: ipxe-boot.img
+	qemu-img convert -f raw -O qcow2 $< $@
+
+# We disable -Werror so we can build older commits with newer gcc
+$(IPXE_IMG):
+	$(MAKE) -C ipxe/src NO_WERROR=1 EMBED=../../script.ipxe bin/ipxe.iso
 
 clean:
-	rm -f ipxe-boot.qcow2
+	$(MAKE) -C ipxe/src clean
+	rm -f ipxe-boot.img ipxe-boot.qcow2
+
+.PHONY: $(IPXE_IMG) clean all

ipxe/README

@@ -1,13 +1,5 @@
-IPXE images and tools
----------------------
+IPXE image-building tools
+-------------------------
 
-This directory contains IPXE images and tools for building those images.
-If you are an OVB user you are probably most interested in the pre-built
-images, ipxe-boot.qcow2 and ipxe-boot-41.qcow2.  The former is a minimal
-version of an image that only contains the necessary bits to PXE boot a
-cloud instance.  The latter image is the same, except its virtual size has
-been changed to 41 GB so it can be used as the base image for instances whose
-flavor bases the instance root disk size on the image size.
-
-There is also a diskimage-builder element and a makefile that can be used to
-rebuild the ipxe-boot.qcow2 image.  Simply run ``make`` to build the image.
+This directory contains tools for for building an IPXE image. Run ``make`` to
+build the image. It is provided as both raw and qcow2.

ipxe/elements/ipxe-boot-image/README.rst

@@ -1,13 +0,0 @@
-===============
-ipxe-boot-image
-===============
-Builds an image which contains *only* a grub2 based boot configured to run a
-custom built ipxe.lkrn image.
-
-While this element depends on centos7, all remnants of the OS are removed apart
-from the /boot grub configuration and modules.
-
-Optional parameters:
-
- * IPXE_GIT_REF a git reference to checkout from the iPXE git repository before
-   building the iPXE image. If not specified then current master will be built.

ipxe/elements/ipxe-boot-image/cleanup.d/98-cleanup-all-except-grub-ipxe

@@ -1,32 +0,0 @@
-#!/bin/bash
-#
-# Copyright 2014 Hewlett-Packard Development Company, L.P.
-# All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
-# not use this file except in compliance with the License. You may obtain
-# a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-# License for the specific language governing permissions and limitations
-# under the License.
-
-if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
-    set -x
-fi
-set -eu
-set -o pipefail
-
-ROOT_RM_PATHS="bin lib64 media opt sbin usr root etc lib mnt run srv var"
-
-for _DIR in $ROOT_RM_PATHS ; do
-    sudo rm -rf $TARGET_ROOT/$_DIR
-done
-# keep 99-tidy-logs happy
-sudo mkdir -p $TARGET_ROOT/var/log
-sudo mkdir $TARGET_ROOT/root
-

ipxe/elements/ipxe-boot-image/element-deps

@@ -1,3 +0,0 @@
-block-device-mbr
-centos7
-vm

ipxe/elements/ipxe-boot-image/install.d/50-grub2-boot-ipxe

@@ -1,102 +0,0 @@
-#!/bin/bash
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-# implied.
-#
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
-    set -x
-fi
-set -eu
-set -o pipefail
-
-SCRIPTDIR=$(dirname $0)
-
-cd /root
-git clone git://git.ipxe.org/ipxe.git
-cd ipxe/src
-if [ -n "${IPXE_GIT_REF:-}" ]; then
-  git checkout ${IPXE_GIT_REF}
-fi
-IPXE_GIT_REF_CURRENT=$(git rev-parse --short HEAD)
-
-cat << EOF >>script.ipxe
-#!ipxe
-#
-# This is the iPXE boot script that we embed into the iPXE binary.
-#
-# The default behaviour is to get DHCP and assume that DHCP includes
-# the filename option. If it fails, then it just stops.
-#
-# This script implements more retries until both DHCP and the filename
-# option are present. This makes sure that the underlying network has
-# plenty of time to be ready. If this still fails, then rather than
-# halt, then host will reboot to try again.
-#
-# Based on:
-# https://github.com/danderson/netboot/blob/master/pixiecore/boot.ipxe
-#
-set attempts:int32 10
-set x:int32 0
-
-:loop
-dhcp || goto nodhcp
-isset \${filename} || goto nobootconfig
-goto boot
-
-:nodhcp
-echo No DHCP response, retrying (attempt \${x}/\${attempts}).
-sleep 1
-goto retry
-
-:nobootconfig
-echo No filename option present, retrying (attempt \${x}/\${attempts}).
-sleep 1
-goto retry
-
-:retry
-iseq \${x} \${attempts} && goto fail ||
-inc x
-goto loop
-
-:boot
-chain \${filename}
-
-:fail
-echo Failed to get a correct DHCP response after \${attempts} attempts.
-echo
-echo Rebooting in 5 seconds...
-sleep 5
-reboot
-EOF
-
-make bin/ipxe.lkrn EMBED=script.ipxe
-cp bin/ipxe.lkrn /boot/
-
-cat << EOF >>/etc/grub.d/40_custom
-menuentry "iPXE boot ${IPXE_GIT_REF_CURRENT}" {
-linux16 /boot/ipxe.lkrn
-}
-EOF
-if [ -s /etc/default/grub ]; then
-    sed -i -e 's/^GRUB_TIMEOUT.*/GRUB_TIMEOUT=5/' /etc/default/grub
-    sed -i -e 's/^GRUB_DEFAULT.*/GRUB_DEFAULT="iPXE boot"/' /etc/default/grub
-fi
-
-# remove all unneeded /boot files
-rm -f /boot/vmlinuz*
-rm -f /boot/initramfs*
-rm -f /boot/initrd.*
-rm -f /boot/System.map*
-rm -f /boot/config-*
-rm -f /boot/symvers-*
-rm -rf /boot/grub2/locale

ipxe/elements/ipxe-boot-image/package-installs.yaml

@@ -1,3 +0,0 @@
-xz-devel:
-git:
-gcc:

ipxe/ipxe

@@ -0,0 +1 @@
+Subproject commit c63ef427a2b18d318b313a4adf6267bb4dfa0c1c

ipxe/script.ipxe

@@ -0,0 +1,47 @@
+#!ipxe
+#
+# This is the iPXE boot script that we embed into the iPXE binary.
+#
+# The default behaviour is to get DHCP and assume that DHCP includes
+# the filename option. If it fails, then it just stops.
+#
+# This script implements more retries until both DHCP and the filename
+# option are present. This makes sure that the underlying network has
+# plenty of time to be ready. If this still fails, then rather than
+# halt, then host will reboot to try again.
+#
+# Based on:
+# https://github.com/danderson/netboot/blob/master/pixiecore/boot.ipxe
+#
+set attempts:int32 10
+set x:int32 0
+
+:loop
+dhcp || goto nodhcp
+isset ${filename} || goto nobootconfig
+goto boot
+
+:nodhcp
+echo No DHCP response, retrying (attempt ${x}/${attempts}).
+sleep 1
+goto retry
+
+:nobootconfig
+echo No filename option present, retrying (attempt ${x}/${attempts}).
+sleep 1
+goto retry
+
+:retry
+iseq ${x} ${attempts} && goto fail ||
+inc x
+goto loop
+
+:boot
+chain ${filename}
+
+:fail
+echo Failed to get a correct DHCP response after ${attempts} attempts.
+echo
+echo Rebooting in 5 seconds...
+sleep 5
+reboot