From b2f1d4e2af01c60e24980ef885355d676959c0a1 Mon Sep 17 00:00:00 2001 From: Robert Collins Date: Thu, 18 Jul 2013 09:17:01 +1200 Subject: [PATCH] Document an interface for offline operation. Complex image builds can download hundreds of MB of data from the internet with many separate lookups. It would be nice to allow users to ask for a fast build where those lookups are entirely avoided, using locally cached resources (where possible). This new interface allows users to signal to elements that they wish to operate without updating cached resources, which will in turn allow us to avoid checking for stale data at all. As part of this I've also documented where we cache data, so that things like the ccache cache dir and image cache files are not a surprise to users. Change-Id: I27f5de6ceaa4e9c6390721b7c434fe0908df84f5 --- README.md | 23 +++++++++++++++++++++++ bin/disk-image-create | 2 ++ lib/common-defaults | 1 + 3 files changed, 26 insertions(+) diff --git a/README.md b/README.md index 0ede05d12..810031bb5 100644 --- a/README.md +++ b/README.md @@ -79,6 +79,21 @@ uncompressed cloud image and about 20% of that for working files. \* As reported by /proc/meminfo MemTotal +Caches and offline mode +======================= + +Since retrieving and transforming operating system image files, git +repositories, Python or Ruby packages, and so on can be a significant overhead, +we cache many of the inputs to the build process in ~/.cache/image-create/. The +writing an element documention describes the interface within +disk-image-builder for caching. When invoking disk-image-builder the --offline +option will instruct disk-image-builder to not refresh cached resources. + +Note that we don't maintain operating system package caches, instead depending +on your local infrastructure (e.g. Squid cache, or an APT or Yum proxy) to +facilitate caching of that layer, so you need to arrange independently for +offline mode. + Design ====== @@ -222,6 +237,14 @@ Ramdisk elements support the following files in their element directories: * udev.d : udev rules files that will be copied into the ramdisk. +Global image-build variables +---------------------------- + +* DIB\_OFFLINE : this is always set. When not empty, any operations that + perform remote data access should avoid it if possible. If not possible + the operation should still be attempted as the user may have an external + cache able to keep the operation functional. + Structure of an element ----------------------- diff --git a/bin/disk-image-create b/bin/disk-image-create index 3c2908c83..9e0dd3784 100755 --- a/bin/disk-image-create +++ b/bin/disk-image-create @@ -43,6 +43,7 @@ function show_options () { echo " -u -- uncompressed; do not compress the image - larger but faster" echo " -c -- clear environment before starting work" echo " --no-tmpfs -- do not use tmpfs to speed image build" + echo " --offline -- do not update cached resources" if [ "$IS_RAMDISK" == "0" ]; then echo " -n skip the default inclusion of the 'base' element" echo " -p package[,package,package] -- list of packages to install in the image" @@ -71,6 +72,7 @@ while true ; do -n) shift; export SKIP_BASE="1";; -p) IFS="," read -a INSTALL_PACKAGES <<< "$2"; export INSTALL_PACKAGES ; shift 2 ;; --no-tmpfs) shift; export DIB_NO_TMPFS=1;; + --offline) shift; export DIB_OFFLINE=1;; --) shift ; break ;; *) echo "Internal error!" ; exit 1 ;; esac diff --git a/lib/common-defaults b/lib/common-defaults index dc9550008..23aa0acc0 100644 --- a/lib/common-defaults +++ b/lib/common-defaults @@ -39,3 +39,4 @@ export DIB_NO_TMPFS=${DIB_NO_TMPFS:-0} _BASE_ELEMENT_DIR=$(dirname $0)/../elements ELEMENTS_PATH=${ELEMENTS_PATH:+"$ELEMENTS_PATH:$_BASE_ELEMENT_DIR"} export ELEMENTS_PATH=${ELEMENTS_PATH:-$_BASE_ELEMENT_DIR} +export DIB_OFFLINE=''