root/build-tools/modify-build-cfg
Scott Little b2286535c9 Update to build tools to support Build layering.
1: Building rpms now produces an rpm.lst file.  This file serves to
list all rpms produced by the build.  A seperate rpm.lst file
is producede for each build type (std, rt, installer). The file is
co-resident with the matching repodata directory.

The rpm.lst files will need to be published by cengn for each layer build.

The download tools in stx-tools will need to find the rpm.lst files
of lower layer builds, and use it to direct the download of rpms
from the lower layer.

2: Building rpms now produces an image.inc file.  This file serves
to list all rpms that the layer recommends be built into an iso.
The file is stored under the $MY_WORKSPACE/<build-type> subdirectory,
although it has identical content for any build-type.

The image.inc file will need to be published by cengn for each layer
build.

The download tools in stx-tools will need to download the per-layer
image.inc files to the $MY_REPO/cgcs-centos-repo/layer_image_inc/
sub-directory, renaming the file in some layer specific way.

The build-iso tool will process any *.inc files found under
$MY_REPO/cgcs-centos-repo/layer_image_inc/ ,
adding those rpms to the iso.

3) Add a mechanism for layer specific mock.cfg prototypes.
This will allow special handling of the real-time (aka 'rt'),
repositories for layer builds that need to do a 'rt' build.

4) Add support for a $MY_REPO/cgcs-centos-repo/rt subdirectory,
a place to build a repo of real-time rpms originating from
lower layer builds.

The download tools in stx-tools will need to populate the new
rt repos.

As of this writing, non-rt rpms remain in $MY_REPO/cgcs-centos-repo/.
i.e. there is not a new $MY_REPO/cgcs-centos-repo/std/ directory.

5) Some changes to make us more flexible about where we find
realease and bsp files.

6) Found that kernel mudules were not reliably building against
the hearnel-headers of our modified kernel.  Found that adding '--update'
to our mock build command was not working.  Does mock expect '--update'
to only be used independently of a build command?  It does work when
used in that manner, so that's what we will do.

7) The build-pkgs, build-srpms, build-rpms family of commands
can take a layer argument and/or will read the LAYER environment
variable.  Current the only use of this variable is to modify
the build-info.  It does NOT limit the compile to packages
for a specific layer.

Story: 2006166
Task: 37094

Depends-On: https://review.opendev.org/698756
Depends-On: https://review.opendev.org/700819
Change-Id: I817e08a19cdabe08b3fcc47dee63a36b461c13c0
Co-Authored-by: Martin Chen <haochuan.z.chen@intel.com>
Signed-off-by: Scott Little <scott.little@windriver.com>
2020-02-07 16:36:40 -05:00

160 lines
6.0 KiB
Bash
Executable File

#!/bin/sh
# This script modifies a mock configuration file (typically $MY_BUILD_CFG)
# to add build time environment variables to the mock environment (things
# like what branch we're building on, etc).
#
# For reasons of security, the host environment variables cannot normally be
# passed through to the mock environment, so this scripts sets the variables
# to literal values.
#
# usage: modify-build-cfg [file.cfg] [<layer>]
#
LAYER=${2:-$LAYER}
# Try to find a layer specific mock.cfg.proto
MOCK_CFG_PROTO="$MY_REPO/cgcs-centos-repo/mock.cfg.${LAYER}.proto"
echo "==== Try MOCK_CFG_PROTO=$MOCK_CFG_PROTO ===="
if [ ! -f "$MOCK_CFG_PROTO" ]; then
# Not present, Use default mock.cfg.proto
MOCK_CFG_PROTO="$MY_REPO/cgcs-centos-repo/mock.cfg.proto"
fi
echo "==== Use MOCK_CFG_PROTO=$MOCK_CFG_PROTO ===="
if [ ! -f "$MOCK_CFG_PROTO" ]; then
echo "ERROR: Couldn't find mock config prototype at '$MOCK_CFG_PROTO'"
exit 1
fi
if [ "${1}x" == "x" ]; then
FILE=$MY_BUILD_CFG
else
FILE=$1
fi
if [ -f $MOCK_CFG_PROTO ]; then
if [ -f $FILE ]; then
NEWER=$(find "$MOCK_CFG_PROTO" -newer "$FILE")
if [ "x$NEWER" != "x" ]; then
\rm -f -v "$FILE"
fi
fi
fi
if [ ! -f $FILE ]; then
if [ -z $MY_BUILD_ENVIRONMENT ] || [ -z $MY_BUILD_DIR ] || [ -z $MY_REPO ]; then
echo "Can't create $FILE without MY_BUILD_ENVIRONMENT, MY_BUILD_DIR and MY_REPO environment variables"
exit 1
fi
echo "Recreating $FILE"
\cp -f -v "$MOCK_CFG_PROTO" "$FILE"
if [ $? -ne 0 ]; then
echo "Couldn't find config file '$FILE', nor construct it from '$MOCK_CFG_PROTO'"
exit 1
fi
sed -i "s%LOCAL_BASE%http://127.0.0.1:8088%g" "$FILE"
sed -i "s%MIRROR_BASE%http://127.0.0.1:8088%g" "$FILE"
sed -i "s%BUILD_ENV%$MY_BUILD_ENVIRONMENT%g" "$FILE"
sed -i "s%/MY_BUILD_DIR%$MY_BUILD_DIR_TOP%g" "$FILE"
sed -i "s%/MY_REPO_DIR%$MY_REPO%g" "$FILE"
# Disable all local-* repos for the build-types other than the current one
for bt in std rt; do
if [ "$bt" != "$BUILD_TYPE" ]; then
# Use the range of lines starting with pattern [local-$bt] until the next line starting with []
sed -i "/^\[local-$bt\]/,/^\[/ s/enabled=1/enabled=0/" $FILE
sed -i "/^\[StxCentos7Distro-$bt\]/,/^\[/ s/enabled=1/enabled=0/" $FILE
fi
done
fi
# Add environment variables to mock config if they don't exist
grep -q "config_opts\['environment'\]\['BUILD_BY'\]" $FILE || \
echo "config_opts['environment']['BUILD_BY']" >> $FILE
grep -q "config_opts\['environment'\]\['BUILD_DATE'\]" $FILE || \
echo "config_opts['environment']['BUILD_DATE']" >> $FILE
grep -q "config_opts\['environment'\]\['REPO'\]" $FILE || \
echo "config_opts['environment']['REPO']" >> $FILE
grep -q "config_opts\['environment'\]\['WRS_GIT_BRANCH'\]" $FILE || \
echo "config_opts['environment']['WRS_GIT_BRANCH']" >> $FILE
grep -q "config_opts\['environment'\]\['CGCS_GIT_BRANCH'\]" $FILE || \
echo "config_opts['environment']['CGCS_GIT_BRANCH']" >> $FILE
if [ -z $FORMAL_BUILD ]; then
grep -q "config_opts\['macros'\]\['%_no_cgcs_license_check'\] = '1'" $FILE || \
echo "config_opts['macros']['%_no_cgcs_license_check'] = '1'" >> $FILE
else
sed -i "/config_opts\['macros'\]\['%_no_cgcs_license_check'\] = '1'/d" $FILE
fi
grep -q "config_opts\['macros'\]\['%_tis_build_type'\] = '$BUILD_TYPE'" $FILE || \
echo "config_opts['macros']['%_tis_build_type'] = '$BUILD_TYPE'" >> $FILE
if [ -f /usr/lib64/nosync/nosync.so ]; then
grep -q "config_opts\['nosync'\] = True" $FILE || \
echo "config_opts['nosync'] = True" >> $FILE
fi
NETWORK_PKGS=""
if [ "containers" == "$BUILD_TYPE" ]; then
NETWORK_PKGS="bind-utils"
fi
grep -q "config_opts\['chroot_setup_cmd'\] = 'install @buildsys-build pigz lbzip2 yum $NETWORK_PKGS'" $FILE || \
echo "config_opts['chroot_setup_cmd'] = 'install @buildsys-build pigz lbzip2 yum $NETWORK_PKGS'" >> $FILE
# Special case for containers.
# rpmbuild_networking is required for invoking helm commands within mock
# building containers requires the std repo to be enabled.
if [ "containers" == "$BUILD_TYPE" ]; then
grep -q "config_opts\['rpmbuild_networking'\] = True" $FILE || \
echo "config_opts['rpmbuild_networking'] = True" >> $FILE
grep -q "config_opts\['use_host_resolv'\] = True" $FILE || \
echo "config_opts['use_host_resolv'] = True" >> $FILE
sed -i "/^\[local-std\]/,/^\[/ s/enabled=0/enabled=1/" $FILE
fi
#
# Read macros from tis.macros to add to the build config file,
# for use in RPM spec files
#
RPM_MACROS=$MY_REPO/build-tools/tis.macros
sed 's/#.*//' $RPM_MACROS | grep '=' | while IFS='=' read name value; do
# Check if the entry already exists. If so, go to next line
grep -q "^config_opts\['macros'\]\['${name}'\] = '${value}'$" $FILE && continue
# Update or add the entry
grep -q "^config_opts\['macros'\]\['${name}'\]" $FILE
if [ $? -eq 0 ]; then
sed -i -r "s#^(config_opts\['macros'\]\['${name}'\]).*#\1 = '${value}'#" $FILE
else
echo "config_opts['macros']['${name}'] = '${value}'" >> $FILE
fi
done
# okay, now we have lines for each env var. Generate the correct values
BUILD_DATE=`date "+%F %T %z"`
CGCS_GIT_BRANCH=`cd $MY_REPO/stx/; git rev-parse --abbrev-ref HEAD`
WRS_GIT_BRANCH=`cd $MY_REPO; git rev-parse --abbrev-ref HEAD`
REPO=$MY_REPO
# Finally, our good friend sed will place the values in the mock config file
sed -i \
-e "s#config_opts\['environment'\]\['BUILD_BY'\].*#config_opts\['environment'\]\['BUILD_BY'\] = '$USER'#" \
-e "s#config_opts\['environment'\]\['BUILD_DATE'\].*#config_opts\['environment'\]\['BUILD_DATE'\] = '$BUILD_DATE'#" \
-e "s#config_opts\['environment'\]\['REPO'\].*#config_opts\['environment'\]\['REPO'\] = '$REPO'#" \
-e "s#config_opts\['environment'\]\['WRS_GIT_BRANCH'\].*#config_opts\['environment'\]\['WRS_GIT_BRANCH'\] = '$WRS_GIT_BRANCH'#" \
-e "s#config_opts\['environment'\]\['CGCS_GIT_BRANCH'\].*#config_opts\['environment'\]\['CGCS_GIT_BRANCH'\] = '$CGCS_GIT_BRANCH'#" \
$FILE