RoundTripLoader in ruamel.yaml will omit quotation marks. In some
cases, the result by loading a yaml with RoundTripLoader and
dumping it out will be unreadable by PYYAML. For example, the
pattern:
hosts: ["mon-logstash:port"]
will be transformed to:
hosts: [mon-logstash:port]
Currently, the Armada project is using PYYAML. It's better to keep
the quotes to allow the following process have to use PYYAML.
Tested with rebuild the helm chart with the yaml has the pattern
in the example, and the quotation marks are preserved.
Change-Id: I7375a1ad498336baa337cca5af8f90028b753110
Partial-Bug: 1896530
Signed-off-by: Yuxing Jiang <yuxing.jiang@windriver.com>
I want the build to work with either local-repo or cgcs-tis-repo.
In many places we will be testing for the existance local-repo as
the prefered path, then fall back to cgcs-tis-repo as an
alternative. If neither are present, either exit or continue but
assuming the new path is intended.
Story: 2006387
Task: 36910
Change-Id: I2e97dc8cd5d4a54158d4b1cefeecd2000ec11cf2
Signed-off-by: Scott Little <scott.little@windriver.com>
I want the build to work with either centos-repo or cgcs-centos-repo.
In many places we will be testing for the existance centos-repo as
the prefered path, then fall back to cgcs-centos-repo as an alternative.
If neither are present, either exit or continue but assuming the new
path is intended.
NOTE: The patch_rebase_1/2/3/4 scripts remain broken, but I hope
to salvage them one day. The current coding assumes content under
centos-repo/cgcs-centos-repo is managed by a git, which is not
currently true.
Story: 2006387
Task: 36912
Change-Id: I8f694814c41957c5b37eb2e64b653b7d42f2e2c9
Signed-off-by: Scott Little <scott.little@windriver.com>
One of the steps under build-helm-charts.sh is to modify
upstream manifests by substituting docker image locations and tags
with ones that reference the images we have built and published.
Some newer manifests we would like to include are now using a
multi-line format that isn't supported by the current script.
Examples would be ...
image: <host>[:<port>]/<path>/<image-name>
imageTag: <tag>
or
image:
repository: <host>[:<port>]/<path>/<image-name>
tag: <tag>
Our fix is to abandon sed, and substitute a routine
that walks the data structures looking substitutions
in either one or two line format.
Closes-Bug: 1893261
Depends-On: https://review.opendev.org/750710
Change-Id: I81be885e0c2460be9319fe403ef8fe938fe35549
Signed-off-by: Scott Little <scott.little@windriver.com>
Build avoidance does not work reliably because directory
.repo/repo is being included in the build context.
This directory is the repo tool itself, not any of the
gits managed by the repo manifest. The sha reported for
.repo/repo might not be available in users environmnet,
causing build avoidance to reject an otherwise valid
reference context.
Fix is in two parts.
1) Do not include .repo/repo in future build contexts.
2) Filter .repo/repo out of downloaded contexts before
use in build avoidance calculations. This addresses
contexts that are already published.
Closes-Bug: 1893243
Change-Id: I7bd20597cb7bc5ee93ae49728176791c51e89c53
Signed-off-by: Scott Little <scott.little@windriver.com>
The symlinks under .tox were being included for md5sum
calculations.
The md5sum calculations would report an error when a file containing
a space was encountered (which can happen in some .tox libraries such
as setuptools
Change-Id: Ib9bfe55a66e1f27b1e39596b14303c0ef00852c6
Signed-off-by: albailey <Al.Bailey@windriver.com>
- Options to set default IP v4 & v6 routes
- Option to set root & sysadmin passwords and disable password change on
1st login
Story: 2007858
Task: 40733
Change-Id: I59a7f112e8a01b00693aabbb804a80ef0cb8253e
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
build.starlingx.cengn.ca is not a public web server. It's use
results in 'HTTP Error 502 - Bad Gateway' outside of CENGN.
The correct url should refer to mirror.starlingx.cengn.ca.
Closes-Bug: 1892538
Change-Id: I6021ee3d97815e70ee461c2c1a8297dfdefc7769
Signed-off-by: Scott Little <scott.little@windriver.com>
Changes made previously to upversion grub2 in starlingx resulted in
some restructuring of the pxe-network-installer package, and
corresponding changes in build-iso. These changes were not reflected
in patch-iso, however. This update fixes the handling of
pxe-network-installer extraction, and includes additional grub2
packages in the patch support.
Change-Id: Ib72a44d2d5a39fe41023be8feb83b88e5f39435b
Closes-Bug: 1892424
Signed-off-by: Don Penney <don.penney@windriver.com>
- Added new option --graphical to generate images that boot into a
graphics card, rather than the serial port
- Added "_graphical" suffix to the generated image file name when
--graphical is in effect
- Comment out global_filter parameter in lvm.conf, otherwise LVM fails
to start when image is booted in different emulation environment
Story: 2007858
Task: 40440
Change-Id: I33e4899873c4f915417e0ab24f60a540d5a68642
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Include the build date of the tarball as underlying RPMs may not have
changed between builds.
Change-Id: I719f9f0224f00a408daf8385c3d5a346dc282521
Story: 2007960
Task: 40686
Signed-off-by: Robert Church <robert.church@windriver.com>
Users in some places like China need to use proxy during building image.
So, it is better to let user be able to use proxy by adding these proxy
arguments.
Closes-bug: 1890383
Change-Id: If9ef7d09a5f53148252b4ab62c40ab0f143e1c3f
Signed-off-by: Liu, ZhipengS <zhipengs.liu@intel.com>
Unlike "declare -i" and "local -i", the bash "let" does not support a
"-i" option. Rather, it takes it as a variable reference. If no "i"
variable is defined in scope, it does not cause an issue. If "i" has
been defined somewhere, however, it may cause a syntax issue, as the i
is evaluated.
A recent update to build-stx-images.sh added a loop that defines an
"i" variable without limiting its scope. In a current image build,
this loop ends with having "i" defined as a URL. As a result, a
"syntax error in expression" occurs, causing the "with_retries"
function to fail to increment the counter. Should a build error occur,
the "with_retries" will never hit the retry limit, looping until it
has a successful result.
This update removes the -i from all "let -i" occurrences in the build
scripts.
Change-Id: I34ad49f8872a81659ff4caf8087b256ea9fb3d32
Closes-Bug: 1891189
Signed-off-by: Don Penney <don.penney@windriver.com>
- when looking for an error message in qemu log, accept more patterns,
because QEMU executable may be named slightly differently than
literally "qemu"
- disable QEMU's terminal features when STDOUT is not a terminal
- allow mounting the ISO via sudo, because the default method (udisks)
needs additional PolicyKit configuration
Story: 2007858
Task: 40441
Change-Id: I19ec7e55aaa80b6ed98162b65e41ce1a63f6da40
build-stx-base.sh and build-stx-images.sh need to accept a
new argument (--config-file <path>) specifying a path to a
config file.
These commands automatically look for the config files at
the predetermined location, and apply them if found. If not
found, continue without error. If optional argument
--config-file is added, These commands will use specified
configfile instead of predetermined one.
Test pass on local build enviroment.
Closes-Bug: 1886062
Change-Id: I476e38064d1515d3d0a0d49c789f1d472d8ca1cc
Signed-off-by: zhipengl <zhipengs.liu@intel.com>
Tag docker.io/starlingx/portieris image.
master-centos-stable-20200726T013628Z.0 is retagged as stx.5.0-v0.7.0
Story: 2007348
Task: 40415
Change-Id: I93472bcd030fec782717f9db5cc3b2223e9c654a
Signed-off-by: Joseph Richard <joseph.richard@windriver.com>
Rather than maintain tag management files for each release, use a
single file that is updated as needed. This update drops the r3
image-tags file and renames the r4 file to drop the release number.
Change-Id: I37d6d4b805d488db6702c48170805df54fca7300
Signed-off-by: Don Penney <don.penney@windriver.com>
Rewrote script (build-img) that generates a QCOW2 (virtual machine)
image with StarlingX pre-installed. This will allow users to create
virtual machines more easily using a disk where verything is already
installed.
This script updates a standard iso file with a default menu choice
(e.g., AIO/serial console) and adds a kickstart script that configures
the network; then boots the ISO in QEMU and lets it install to a disk
image.
The generated image only works in QEMU because it has a few things that
are specific to that emulation:
* /etc/lvm/lvm.conf restricts LVM to only the bus/port/device type that
QEMU emulates SATA as
* It includes configuration for the network device named "ens3", which
is QEMU's default
Story: 2007858
Task: 40163
Change-Id: I9fa6960eadbeca9481dc91c0878153d3fe95f0c1
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Add support for building docker images with an explicit
configuration for specifying the docker context and Dockerfile.
This is required for projects that build from 3rd party source
that maintains a Dockerfile outside of the project repo and
builds from source that is cloned by the repo tool into the
project path.
New Configuration Parameters:
DOCKER_CONTEXT=<relative-path-to-source>
DOCKER_FILE=<relative-path-to-docker-file>
NOTE: Paths are relative to the image build file.
Story: 2006537
Task: 40286
Change-Id: I6520fd31218aaf849f4d1e803bb1e0bfb614b29f
Signed-off-by: Matt Peters <matt.peters@windriver.com>
Add python2 based services to docker_image_build_cfg_file.
The config file could then be modified to individually shift
images from python2 to python3 build method without having
to sync with cengn build scripts when there is a change.
Add --python2-wheels opt for build-stx-images.sh to specify link
to python2 based wheels tarball.
Story: 2007474
Task: 40139
Depends-on: https://review.opendev.org/#/c/712880/
Change-Id: Ic4313a2ac6f7ec2c8b87bef960f4a796f6a10738
Signed-off-by: zhipengl <zhipengs.liu@intel.com>
Remove the f/ or r/ from a branch name so we can create valid
prefix for the new manifest that gets created with the --manifest
flag.
Change-Id: I4eb8f3c70f85566463fb114944309597241a7a89
Signed-off-by: Saul Wold <sgw@linux.intel.com>
The sub-script fails when a double '==' is used inside of a repo
forall -c option. The single '=' is preferred when used in 'test'
or '['.
The error is similar to this:
if [ $REPO_PATH == cgcs-root ]; then echo $REPO_PROJECT; fi: 1: [: cgcs-root: unexpected operator
Closes-Bug: 1883772
Change-Id: If17de30a03939240f0cc329179d452820a691e32
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Changes in brief:
1) TIS_PATCH_VER will now accept new automatic variables
'OTHER_GITREVCOUNT' and 'SRC_GITREVCOUNT'.
2) New BASE_SRCREV variables are supported, 'SRC_BASE_SRCREV' and
'BASE_SRCREV_FOR_PATH'
3) OPT_DEP_LIST_FOR_BUILD_TYPE[<build-type>] joins OPT_DEP_LIST as
a way to list files that affect the build, but are not included
in the assembled src.rpm.
4) Some code relating to interpreting build_srpm.data files,
and producing a list of files that affect the build, are relocated
to 'srpm-util' and restructured for greater utility.
The details:
TIS_PATCH_VER=<expression>
# An integer, or one of the supported
# variables listed below, or the sum of
# variables and integers.
# e.g.
# TIS_PATCH_VER=PKG_GITREVCOUNT+SRC_GITREVCOUNT+5
PKG_GITREVCOUNT # Count git revisions relative to PKG_BASE.
# Optionally only count from PKG_BASE_SRCREV
SRC_GITREVCOUNT # Count git revisions relative to SRC_DIR.
# Optionally only count from SRC_BASE_SRCREV
GITREVCOUNT # Deprecated, please use SRC_GITREVCOUNT instead.
# Count git revisions relative to SRC_DIR.
# Optionally only count from TIS_BASE_SRCREV
OTHER_GITREVCOUNT # count git revisions from all sources excluding
# PKG_BASE and SRC_DIR
# Optionally only count from
# BASE_SRCREV_FOR_PATH[<path>]
PKG_BASE_SRCREV=<sha> # Limit PKG_GITREVCOUNT revision count to
# commits since <sha>
SRC_BASE_SRCREV=<sha> # Limit SRC_GITREVCOUNT revision count to
# commits since <sha>
TIS_BASE_SRCREV=<sha> # Deprecated, please use SRC_BASE_SRCREV
# instead
# Limit GITREVCOUNT revision count to commits
# since <sha>
BASE_SRCREV_FOR_PATH[<path>]=[<sha>|OTHER_PKG_BASE_SRCREV]
# Limit OTHER_GITREVCOUNT revision count for
# commits under <path> to commits since <sha>.
# If <path> is the PKG_BASE of another package
# (not the current package) then the keyword
# 'OTHER_PKG_BASE_SRCREV' can be used to extract
# the 'PKG_BASE_SRCREV' value of the other
# package.
#
# The <path> can reference variables like
# $STX_BASE and $GIT_BASE.
OPT_DEP_LIST=<path-list> # Add a space separated list of paths that
# don't contribute to the content of a src.rpm
# but do contribute to triggering a rebuild,
# and possibly modifying the TIS_PATCH_VER via
# use of OTHER_GITREVCOUNT.
OPT_DEP_LIST_FOR_BUILD_TYPE[<build-type>]=<path-list>
# For a specific build type only, add a space
# separated list of paths that don't
# contribute to the content of src.rpm,
# but do contribute to triggering a
# rebuild, and possibly modifying the
# TIS_PATCH_VER via use of OTHER_GITREVCOUNT.
Story: 2006166
Task: 39765
Change-Id: I9e7d409d4eefdb41a7083db1b801d531c443b678
Signed-off-by: Scott Little <scott.little@windriver.com>
Currently all flock packages have a common base version of 1.0 or 1.0.0
defined in specfiles. The build system creates tarballs and whl files
with the base version (without TIS_PATCH_VER) and the version is also
attached the generated RPM file (with TIS_PATCH_VER).
This patch addes PBR (OpenStack's Python Build Reasonableness) support,
which can generate the version by parsing semantically-versioned Git
tags and walking the Git history back to the last release tag.
A new variable named PBR_VERSION is supported in build_srpm.data. If its
is not set, the build system still uses TIS_PATCH_VER like before.
If PBR_VERSION=auto, the build system will generate the version by
PBR and create tarballs, whl files and RPM files with this version.
If PBR_VERSION is set manually, the build system will use it directly.
Story: 2006703
Task: 37014
Change-Id: Ic377e3ae3344b291d2d9c0d0fca8f681d1a007b5
Signed-off-by: Bin Yang <bin.yang@intel.com>
Signed-off-by: Yang, Bin <bin.yang@intel.com>
Builds a single package in a clean environment to detect missing
build dependencies.
Usage:
build-pkgs --dep-test <pkg>
Note:
Should only be run after a full 'build-pkgs'.
We want the build to fail because a dependency wasn't
been listed, not because a dependency hasn't been built yet.
Closes-Bug: 1880248
Change-Id: Icf64be37a77de0992647c37ce392fd5a2a458fba
Signed-off-by: Scott Little <scott.little@windriver.com>
