Build arm64 images using arm64 wheels from openstack if available

We updated python-base and python-builder to include arm64 images in
support of nodepool's arm64 python-builder image. In doing so we have
discovered a number of issues, but the biggest is slowness of building
python packages in an emulated environment.

In order to speed up package builds we consume the OpenDev linaro
cloud arm64 wheel cache. This doesn't have wheels for every package we
need, but for the things that it does have it will speed up our builds.

One of the risks with this setup is that we're relying on wheels built
for openstack on arm64 and those follow openstack's contraints. In order
to mitigate this risk we set pip install's --prefer-binary flag in the
pip.conf. This means that if openstack's constraints lag what is
availale on pypi we should use the existing wheels as long as they are
valid version according to requirements rather than trying to build from
sdist.

Co-Authored-By: James E. Blair <corvus@inaugust.com>
Co-Authored-By: Ian Wienand <iwienand@redhat.com>
Change-Id: I3b358721eebbceafc12daf9d706306634048b196

.zuul.yaml

@@ -156,6 +156,7 @@
     name: nodepool-build-image
     parent: opendev-build-docker-image
     description: Build Docker images.
+    timeout: 4800
     dependencies:
       - opendev-buildset-registry
     requires:
@@ -171,19 +172,29 @@
             # If zuul.tag is defined: [ '3', '3.19', '3.19.0' ].  Only works for 3-component tags.
             # Otherwise: ['latest']
             &imagetag "{{ zuul.tag is defined | ternary([zuul.get('tag', '').split('.')[0], '.'.join(zuul.get('tag', '').split('.')[:2]), zuul.get('tag', '')], ['latest']) }}"
+          arch:
+            - linux/amd64
+            - linux/arm64
         - context: .
           repository: zuul/nodepool-launcher
           target: nodepool-launcher
           tags: *imagetag
+          arch:
+            - linux/amd64
+            - linux/arm64
         - context: .
           repository: zuul/nodepool-builder
           target: nodepool-builder
           tags: *imagetag
+          arch:
+            - linux/amd64
+            - linux/arm64
 
 - job:
     name: nodepool-upload-image
     parent: opendev-upload-docker-image
     description: Build Docker images and upload to Docker Hub.
+    timeout: 4800
     requires:
       - python-builder-3.8-container-image
       - python-base-3.8-container-image

Dockerfile

@@ -13,17 +13,26 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM docker.io/opendevorg/python-builder:3.8 as builder
+FROM docker.io/opendevorg/python-builder:3.7 as builder
 # ============================================================================
 
 ARG ZUUL_SIBLINGS=""
 COPY . /tmp/src
+RUN if [ `uname -m` = "aarch64" ] ; then \
+      echo "Installing arm64 pip.conf" ; \
+      cp /tmp/src/tools/pip.conf.arm64 /etc/pip.conf ; \
+      cp /tmp/src/tools/pip.conf.arm64 /output/pip.conf ; \
+    fi
 RUN assemble
 
-FROM docker.io/opendevorg/python-base:3.8 as nodepool-base
+FROM docker.io/opendevorg/python-base:3.7 as nodepool-base
 # ============================================================================
 
 COPY --from=builder /output/ /output
+RUN if [ -f /output/pip.conf ] ; then \
+      echo "Installing pip.conf from builder" ; \
+      cp /output/pip.conf /etc/pip.conf ; \
+    fi
 RUN /output/install-from-bindep nodepool_base
 
 RUN useradd -u 10001 -m -d /var/lib/nodepool -c "Nodepool Daemon" nodepool

tools/pip.conf.arm64

@@ -0,0 +1,5 @@
+[global]
+extra-index-url = https://mirror.regionone.linaro-us.opendev.org/wheel/debian-10-aarch64/
+
+[install]
+prefer-binary = true