starlingx/root
Code Issues Proposed changes
root/build-tools/build-docker-images/loci/patches/0001-starlingx-enable-disable-package-repos.patch
Davlet Panech 4183a924cf docker-images: don't embed wheel tar in images 
This patch fixes a problem introduced by [1]: docker image script
embeds the wheels tarball in the docker image via a COPY command; then
deletes it during the build. This makes docker images unnecessarily
large. This lets us use local tar files with the docker image(s) being
built.

Recently [2] a patch was merged that adds a web server to the Debian
build environment, allowing us to access local files over HTTP and
making the COPY step unnecessary in the Docker file.

This patch removes the explicit downloading of the wheels tarball and
the COPY directive, while passing the wheel tarball to Loci as a URL,
allowing Loci to download it during the build w/o contributing to the FS
layer size.

This reduces the size of stx-fm-rest-api image by ~220 MB. Other Loci
images should experience similar savings.

[1] https://review.opendev.org/c/starlingx/root/+/857505
[2] https://review.opendev.org/c/starlingx/tools/+/873010

TESTS
=================

* Build a loci image with wheels tarball passed as a URL
* Build a loci image with wheels tarball passed as a local file name

Story: 2010294
Task: 47343

Change-Id: I6dca6f95b0845e549ad35c9b6e95aa58c86fc774
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
2023-02-10 15:23:40 -05:00

188 lines
6.0 KiB
Diff
Raw Blame History

From 7462c9467cd0a1e98ced03517646a4e00f65ddc3 Mon Sep 17 00:00:00 2001
From: Davlet Panech <davlet.panech@windriver.com>
Date: Thu, 8 Sep 2022 21:04:55 +0000
Subject: [PATCH] starlingx: enable/disable package repos
Dockerfile: new parameter DIST_REPOS that allows one to
enable/disable package repos when building
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
---
Dockerfile | 5 +-
stx-scripts/install.sh | 11 +++
stx-scripts/setup-package-repos.sh | 126 +++++++++++++++++++++++++++++
stx-wheels/.keep | 0
4 files changed, 141 insertions(+), 1 deletion(-)
create mode 100755 stx-scripts/install.sh
create mode 100755 stx-scripts/setup-package-repos.sh
create mode 100644 stx-wheels/.keep
diff --git a/Dockerfile b/Dockerfile
index 3a026a3..3baea6c 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -32,4 +32,7 @@ ARG SPICE_REF=${SPICE_REF:-spice-html5-0.1.6}
COPY scripts /opt/loci/scripts
ADD bindep.txt pydep.txt $EXTRA_BINDEP $EXTRA_PYDEP /opt/loci/
-RUN /opt/loci/scripts/install.sh
+#RUN /opt/loci/scripts/install.sh
+ARG DIST_REPOS
+COPY stx-scripts /opt/loci/stx-scripts
+RUN /opt/loci/stx-scripts/install.sh
diff --git a/stx-scripts/install.sh b/stx-scripts/install.sh
new file mode 100755
index 0000000..da11b75
--- /dev/null
+++ b/stx-scripts/install.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -ex
+
+LOCI_DIR="/opt/loci"
+
+# configure apt/yum repos
+"$LOCI_DIR/stx-scripts/setup-package-repos.sh"
+
+# run Loci installer
+"$LOCI_DIR/scripts/install.sh" "$@"
diff --git a/stx-scripts/setup-package-repos.sh b/stx-scripts/setup-package-repos.sh
new file mode 100755
index 0000000..dd43612
--- /dev/null
+++ b/stx-scripts/setup-package-repos.sh
@@ -0,0 +1,126 @@
+#!/bin/bash
+
+set -ex
+
+#
+# This script enables or disables package repos specified
+# by the DIST_REPOS environment variable, which must contain
+# a space-separated list of repos (in CentOS) or list files
+# (Debian) to enable or disable.
+#
+# In CentOS repo names refer to the names in square brackets
+# in any repo files under /etc/yum.repos.d.
+#
+# In Debian repo names refer to individual files under
+# /etc/apt/sources.list.d/$NAME.list.
+#
+# Repo names may be prefixed with
+# a "+" (enable) or a "-" (disable). The leading "+" may be
+# omitted.
+#
+# Additionally, the following keywords are treated specially:
+#
+# STX - enable or disable all StarlingX repos, ie
+# the locally-built package repos, the mirror/download
+# repo, and any repo's passed on the command-line
+# to "build-stx-image.sh" script.
+#
+# OS - same as "base updates extras" in CentOS
+# same as "debian" in Debian
+#
+#
+# These keywords have the same meaning in all distros, while actual
+# repo names are distro-specific.
+#
+# Any repos not included in $DIST_REPOS will remain unchanged (ie
+# they will remain enabled or disabled as defined in the base image).
+#
+# If a repo doesn't match an existing repository, this script will
+# fail.
+#
+# CentOS Example
+# ==============
+# DIST_REPOS="-base -updates"
+# disable "base" and "updates" repos normally defined
+# in /etc/yum.repos.d/CentOS-Base.repo
+#
+# DIST_REPOS="-STX +OS -updates"
+# disable all local repos, enable core OS repos, except "updates"
+#
+# Debian Example
+# ==============
+# DIST_REPOS="debian"
+# enable core OS repos (ie /etc/apt/sources.list.d/debian.list)
+#
+# DIST_REPOS="OS -STX"
+# enable core OS repos (ie /etc/apt/sources.list.d/debian.list),
+# disable STX repos (ie /etc/apt/sources.list.d/stx.list)
+#
+#
+
+if [[ -n "$DIST_REPOS" ]] ; then
+ # basenames of files under /etc/apt/sources.list.d
+ declare -A DEBIAN_REPO_GROUPS=(
+ [OS]="debian"
+ [STX]="stx"
+ )
+ # yum repo IDs
+ declare -A CENTOS_REPO_GROUPS=(
+ [OS]="base updates extras"
+ [STX]="/etc/yum.repos.d/stx.repo" # ie, all repos defined in this file
+ )
+
+ distro=$(awk -F= '/^ID=/ {gsub(/\"/, "", $2); print $2}' /etc/*release)
+ # enable or disable each repo
+ for base in $DIST_REPOS ; do
+ # starts with "-": disable this repo
+ if [[ "${base#-}" != "$base" ]] ; then
+ base="${base#-}"
+ enable=0
+ # starts with "+": enable this repo
+ elif [[ "${base#+}" != "$base" ]] ; then
+ base="${base#+}"
+ enable=1
+ # doesn't start with +/-: assume "+"
+ else
+ enable=1
+ fi
+
+ # enable or disable a repo
+ case ${distro} in
+ debian)
+ list_files="${DEBIAN_REPO_GROUPS[$base]:-$base}"
+ for list_file in $list_files ; do
+ if [[ $enable -eq 1 ]] ; then
+ cp -f /etc/apt/sources.list.d/${list_file}.list.disabled /etc/apt/sources.list.d/${list_file}.list
+ else
+ rm /etc/apt/sources.list.d/${list_file}.list
+ fi
+ done
+ ;;
+ centos)
+ specs="${CENTOS_REPO_GROUPS[$base]:-$base}"
+ for spec in $specs ; do
+ # repo id begins with a "/" - assume its a full path to a .repo file
+ # and enable/disable all repos defined in that file
+ if [[ "${spec#/}" != "$spec" ]] ; then
+ repos=$(sed -r -n 's/^\s*[[]([^]]+)[]]\s*$/\1/gp' "$spec")
+ else
+ repos=$spec
+ fi
+ for repo in $repos ; do
+ if [[ $enable -eq 1 ]] ; then
+ yum-config-manager --enable "$repo"
+ else
+ yum-config-manager --disable "$repo"
+ fi
+ done
+ done
+ ;;
+ *)
+ echo "error: unsupported OS \"$distro\"" >&2
+ exit 1
+ esac
+ done
+fi
+
diff --git a/stx-wheels/.keep b/stx-wheels/.keep
new file mode 100644
index 0000000..e69de29
--
2.30.2
View Git Blame Copy Permalink