e9e59e57e7
Remove extra OS and python packages from the generated images as a post-build step. New options for the build recipes: * POSTBUILD_REMOVE_OS_PACKAGES : which OS packages to remove. Default: python3-pip python-pip-whl * POSTBUILD_REMOVE_PYTHON_PACKAGES : which pip3 packages to remove Default: pip * POSTBUILD_COMMAND : arbitrary modification command Default: <none> This is needed because some docker images include packages that are required at build time, but not at runtime. This is a kludge to remove them after building. A much better solution would be to re-write every Dockerfile into a multi-stage build, so that the final image includes only the software it needs. DESIGN ========================== After building, create and build a Docker file that inherits from the image we are trying to modify, and: * Reset USER to root * RUN: remove the specified python packages, except ones owned by the package manager * RUN: remove the specified OS packages * RUN: execute arbitrary modification command configured in the build recipe * Reset USER back to what it was in the base image * If anything was removed or modified, retag the image These actions are handled by a new stand-alone script: docker-image-postbuild.sh and a number of helper scripts to be executed in the derived image. TESTS ========================== * Manually test the main script with various options * Rebuild a few select Starlingx images and make sure the post-build script gets called * Make sure overriding the config options in build recipes works as expected * Manually execute the main post-build script on every StarlingX and StarlingX/Openstack image generated by Jenkins. Make sure the script succeeds in all of them. * Manually ensure "pip" is removed at the end LIMITATIONS ========================== There are some exceptions/special cases: * Some images are very minimal and don't include /bin/sh ; the main script ignores these with a warning * Some images based on "foreign" distros leave multiple copies of pip behind and would require special handling in their own build recipes. Example: stx-ceph-manager. * Only rpm and dpkg based distributions are supported for auto-removal. Alpine/apk only allows the removal of pip modules, and not apk packages. This may be fixed in a separate commit in the future. Story: 2011452 Task: 52073 Signed-off-by: Davlet Panech <davlet.panech@windriver.com> Change-Id: Idc75fc3a2b7fbc752d6997035e356314716c9609
88 lines
2.0 KiB
Bash
Executable File
88 lines
2.0 KiB
Bash
Executable File
#!/bin/sh
|
|
#
|
|
# Copyright (c) 2025 Wind River Systems, Inc.
|
|
#
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
#
|
|
|
|
#
|
|
# This script is called inside a docker container by
|
|
# docker-image-postbuild.sh
|
|
#
|
|
# Usage: OUTPUT_TOKEN="..." remove-pip-packages.sh package1 package2 ...
|
|
#
|
|
|
|
if [ -z "$OUTPUT_TOKEN" ] ; then
|
|
echo "ERROR: OUTPUT_TOKEN must be defined" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if ! pip3 --version >/dev/null 2>&1 ; then
|
|
echo "WARNING: pip3 not found, can't remove any python packages" >&2
|
|
exit 0
|
|
fi
|
|
|
|
. `dirname "$0"`/utils.sh
|
|
|
|
sys_pkg_owned() {
|
|
if [ $PKG_MAN = dpkg ] ; then
|
|
dpkg -S "$1" >/dev/null 2>&1
|
|
elif [ $PKG_MAN = rpm ] ; then
|
|
rpm -qf "$1" >/dev/null 2>&1
|
|
else
|
|
apk info -W "$1" >/dev/null 2>&1
|
|
fi
|
|
}
|
|
|
|
in_list() {
|
|
in_list_item="$1"
|
|
shift
|
|
while [ "$#" -gt 0 ] ; do
|
|
if [ "$in_list_item" = "$1" ] ; then
|
|
return 0
|
|
fi
|
|
shift
|
|
done
|
|
return 1
|
|
}
|
|
|
|
trim() {
|
|
sed 's/^[ \t]*//;s/[ \t]*//'
|
|
}
|
|
|
|
sep=
|
|
rm_list=
|
|
for mod in $* ; do
|
|
mod_info=`pip3 show "$mod" 2>/dev/null` || continue
|
|
rdepends=`echo "$mod_info" | sed -n "s/^Required-by://ip" | tr -d ',' | trim`
|
|
if [ -n "$rdepends" ] ; then
|
|
for rmod in $rdepends ; do
|
|
pip3 show "$rmod" 2>/dev/null 2>&1 || continue
|
|
if ! in_list "$rmod" $modules ; then
|
|
echo "ERROR: can't uninstall pip module "$mod" because another installed module "$rmod" requires it" >&2
|
|
exit 1
|
|
fi
|
|
done
|
|
fi
|
|
location=`echo "$mod_info" | sed -n "s/^Location://ip" | trim`
|
|
if sys_pkg_owned "$location" ; then
|
|
echo "WARNING: can't uninstall pip module "$mod" because it is owned by the OS package manager" >&2
|
|
continue
|
|
fi
|
|
rm_list="${rm_list}${sep}$mod"
|
|
sep=" "
|
|
done
|
|
|
|
if [ -n "$rm_list" ] ; then
|
|
echo "Removing python packages [$rm_list]" >&2
|
|
( set -x ; pip3 uninstall --yes $rm_list ; ) || exit 1
|
|
echo "
|
|
|
|
$OUTPUT_TOKEN $rm_list
|
|
|
|
"
|
|
else
|
|
echo "No removable python packages found" >&2
|
|
fi
|
|
|