The wheel and image build tools hit intermittent issues such as:
* Network or server issues that cause file download failures
* Repo access failures due to repo updates or other issues
To avoid formal build failures on such issues, this update adds
retries around specific commands, like wget and docker build.
Change-Id: Ifafdc6201872f43b2dd77efd4dcb033456477c2e
Closes-Bug: 1823986
Signed-off-by: Don Penney <don.penney@windriver.com>
The openstack/requirements master/upper-constraints.txt was
recently updated to point to pecan-1.3.3, but pypi.org does
not have a wheel for this newly released version. This, in
turn, caused a dev image build failure.
This update adds pecan-1.3.3 to the dev-wheels.cfg to build
a wheel for this version and add it to the dev wheels tarball.
Change-Id: I165a3b3e31dad275459d015936e4c9ea5a0e2d78
Closes-Bug: 1823984
Signed-off-by: Don Penney <don.penney@windriver.com>
/usr/bin/getfattr is provided by attr-2.4.46-13.el7.x86_64, which
was being pulled into the ISO by openstack-nova-compute.
openstack-nova-compute is removed from ISO since openstack is
containerized.
Add attr rpm to the image.inc lst, to explicitly require it be
included in ISO.
Test:
Pass build and confirm attr rpm package is included in ISO.
Closes-Bug: 1823075
Change-Id: Ie40af4417ef48151eba5ed89bb0c0311bfbcc155
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Update the base stable-wheels.cfg to align with the
Openstack Stein release upper-constraints.txt, to
support building stable stein Openstack service images.
Change-Id: I6852620b05f58509c93877d9d8c5754e9aa64c0e
Story: 2005248
Task: 30353
Signed-off-by: Don Penney <don.penney@windriver.com>
This update removes pike and master wheel and/or image files,
as these have been replaced with stable/dev.
Change-Id: I1f44dc69de2819eba8dadf1639c6a4ce8f6f2031
Story: 2005248
Task: 30328
Signed-off-by: Don Penney <don.penney@windriver.com>
This update resyncs the dev-wheels.cfg to pick up recent changes
to master-wheels.cfg.
In addition, we're adding croniter-0.3.29 to the master and dev
wheels.cfg files to handle the upstream removal of croniter-0.3.27.
Change-Id: I4b25c18bee0514fff88d32b91ae11b9e586d735c
Story: 2005248
Task: 30215
Signed-off-by: Don Penney <don.penney@windriver.com>
This update restructures the loci and docker image build functions
to move the common post-build steps to a common function.
Change-Id: I8a540093e369cadcdb5f88fbd10b131acafb4331
Story: 2005248
Task: 30129
Signed-off-by: Don Penney <don.penney@windriver.com>
This update includes two changes:
* Due to timing of reviews of two recent commits, changes in the
second commit impacted the first, renaming a variable used in
new code introduced by the first commit. The second commit merged
soon after, leaving the new code using the now wrong variable.
This update corrects the new code, replacing the new uses of
OPENSTACK_RELEASE with BUILD_STREAM in the image build tool.
* The new stx-fm-rest-api image installs some openstack modules, and
builds in the stable build stream. This means we now need the
stable-wheels.cfg to be a copy of the pike-wheels.cfg, until
Stein is released. This update copies pike-wheels.cfg to the new file.
Change-Id: I538bae32a68e43d07e5c64ead8e2884204509194
Story: 2005248
Task: 30151
Signed-off-by: Don Penney <don.penney@windriver.com>
With the StarlingX move to supporting pure upstream OpenStack, the
majority of the SDK Modules are related to functionality no longer
supported. The remaining SDK Modules will be moved to StarlingX
documentation.
Story: 2005275
Task: 30138
Change-Id: Id5e830937854ecb6995ef715fcab1e4cbb3abd99
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
The use of pike/master naming when building StarlingX docker images
does not properly reflect the differences in how images are built,
resulting in some confusion for developers. Partly to address this,
and to allow for independence between images, this is being migrated
to use stable/dev build streams.
The stable build stream uses a base image (ie. stx-centos) that has
software repo configuration that points to StarlingX build output as a
source for software package installation. Images that use an upstream
repo as the main project code would generally use a stable branch for
this build stream, as well. For example, once Openstack Stein is
released, the stable builds for StarlingX Openstack services will be
updated to build from a stable/stein branch.
The dev build stream uses only upstream sources for software
installation, with images building from upstream master branches,
generally.
The stx-libvirt image, for example, is built solely from StarlingX
build output, and is therefore built in the stable build stream.
An image lineup for the stx-openstack application could therefore be a
mixture of stable and dev build stream images. The "stable"
application lineup (once we have Stein) would be all stable build
stream images, while the "dev" lineup would be dev build stream images
plus stable images for any that are stable-only (such as stx-libvirt).
To roll out these changes without impacting designers or builds, the
pike/master builds will still be supported for a short period.
This update provides the tool and base configuration changes to
support the introduction of the stable/dev build streams.
Change-Id: Ib5dc9a7c1d8c37368d073e28d83eeb883766ee3f
Story: 2005248
Task: 30098
Signed-off-by: Don Penney <don.penney@windriver.com>
This update enhances build-stx-images.sh to record image
versions after building and pushing images, to be used as
input for building application tarballs with versions.
Additionally, after the images are built, the build tool
will collect lists of installed packages and python modules,
for reference purposes.
Story: 2005248
Task: 30042
Change-Id: Ie9e15af0669ce15fabe7896f78809221811bc3f7
Signed-off-by: Don Penney <don.penney@windriver.com>
Update wheels.cfg to build the python-openstackclient wheel using the
upstream repo, rather than temporary fork. In addition, we need to
explicitly grab the pre-release networking_sfc-8.0.0.0b2 wheel due to
recent neutron changes.
Change-Id: I7459026ca5edd60dce355a47a3ebeab53696e88b
Story: 2004751
Task: 30062
Signed-off-by: Don Penney <don.penney@windriver.com>
A missing wheel RPM caused get-stx-wheels.sh to hang on a call
to rpm2cpio due to an empty string. This update improves the
error handling to fix the bash syntax on the file check to protect
against the empty string.
This update also corrects the usage of "local" keyword added to
the wheel and image scripts in a recent update, outside a function.
Change-Id: Ib8b83b376eacf44235a3d7fcd63d2451ca919210
Story: 2004751
Task: 29787
Signed-off-by: Don Penney <don.penney@windriver.com>
There have been intermittent failures building StarlingX
docker images due to failures to reach the mirrors within
the default timeout, during the formal CENGN builds. This
update adds a step to the docker files for the stx-centos
base image and the base wheel builder image to set a new
timeout value in the yum.conf file, increasing from the
default 30 seconds to 120 seconds.
Change-Id: Ide52cdebb053f639cfe6d4c3a6c5eef71e6fa8ab
Story: 2004751
Task: 29636
Signed-off-by: Don Penney <don.penney@windriver.com>
Build wheels process fails while is copying the wheel to the tarball due
to a mismatch in the name, this commit changes the wheelname
to upper case.
Closes-Bug: 1816666
Change-Id: Ie9e1c41d921447f15788d3c2a5cb068d4794ee61
Signed-off-by: Jesus Ornelas Aguayo <jesus.ornelas.aguayo@intel.com>
The build-stx-images.sh utility references a fixed loci SHA in order
to isolate the image builds from tool churn.
A recent update to virtualenv has broken loci builds, as described here:
http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002592.html
This update sets the loci SHA to the commit that addresses this issue.
Change-Id: I4c5404c72935433b217d534fc9129f6ea69f23b1
Story: 2004751
Task: 29461
Signed-off-by: Don Penney <don.penney@windriver.com>
The fm-api application requires pecan and wsme python
modules, this commit adds them to pike-wheels.cfg in
order to be built and they can be available in the wheels
tarball.
Story: 2004008
Task: 29400
Change-Id: Ieb15f49f18e8a6598e91819a5b76b4a6763e3e5e
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
This update enhances build-base-wheels.sh to support using a
wildcard in wheels.cfg for the wheel name, to allow for whatever
the version is when building from master for a given repo.
This also updates master-wheels.cfg to build the following modules
from master, to pick up the latest code for these supporting modules:
neutron-lib
python-openstackclient
openstacksdk
Story: 2004751
Task: 29395
Change-Id: Ifab8a209170a00f58e0f4a53b6cbd7e4fe86e8f0
Signed-off-by: Don Penney <don.penney@windriver.com>
This enforces google-api-python-client 1.7.7 which
is available on pypi site as a wheel that is both python2
and python3.
This affected only master docker images, pike was not
impacted
Closes-Bug: 1814835
Change-Id: I1bae1a5a3a1b0fcd323d072a113381c3340f9134
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
When the building scripts involved in the containerization
processes are executed behind a proxy, the user must set the
proxy manually making modifications in the scripts or dockerfiles.
These changes allows to set it as an argument.
Story: 2004925
Task: 29305
Change-Id: I52e13e4832daf488d8ff29334686a20daa7d2826
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
Partial cleanup of the python code.
Story: 2004825
Task: 29286
Change-Id: I146c626342a8be2c93da21cf6cb229a39e8eac10
Signed-off-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
The stx-ceilometer master image requires a specific
version of panko, which does not have a pre-built wheel
available in pypi, so we're adding it to the master
wheels tarball to make it available to the image build.
Change-Id: I854b75577b6dbc3f0a8171190f5a1aa839412dc8
Story: 2004520
Task: 29046
Signed-off-by: Don Penney <don.penney@windriver.com>
This commit adds 644 permissions for all the files under
/www/pages/feed/<release>/Packages, allowing the web server to reach
all the required files to provision compute nodes.
The internal web server used to provide rpms for the installation of
computes and additional controller can fail to serve packages if these
has different permission less than the required by the web server.
lighttpd runs as a non-root user, however the content under
/www/pages/feed/<release>/Packages is owned by root. If the files hasn't
the "read for other" bit set, then this files won't be provided by the
web server and thus causing a failure in the installation of the
computes.
We ensure that all the RPM's has the 644 permissions in that
location after to build the StarlingX ISO image.
Closes-Bug: #1794894
Change-Id: I928b156dad1cffcda2d9a8d41631a56ed7ee29f7
Signed-off-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
openstack-helm-infra, openstack-helm and stx-openstack-helm
no longer require network access, and can therefore be built
as part of build-pkgs in std mode.
The build-helm-charts.sh now packages from std results
Story: 2004005
Task: 28792
Change-Id: If39686cfad5f0f92f6ddfb2351684294b134b5fa
Depends-On: I113ec91b64faebf2e7e8154e4bfbe75acc3fbf43
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
Update the build-stx-base.sh tool to support building
stx-centos for use with images based on Openstack master.
Change-Id: Ic77036e15529dadf39df57e5e88a2000b88bd6cb
Story: 2004520
Task: 28726
Signed-off-by: Don Penney <don.penney@windriver.com>
In order to get correct "$PLATFORM_RELEASE" value
Change-Id: I842411baf7c2b9bb862fceb920919334e40194aa
Closes-Bug: #1809637
Signed-off-by: lihaijing <lihaijing@fiberhome.com>
StarlingX requires a higher version of the keyring and os-vif
python modules than is specified in the stable/pike
upper-constraints.txt file. This update specifies a newer
version for these modules in the pike-wheels.cfg file to
override the constraints for the docker image build.
Story: 2004520
Task: 28585
Change-Id: I9bdaaad2575f8832f19f4fdac26dae76fd83c442
Signed-off-by: Don Penney <don.penney@windriver.com>
When both --latest and --clean flags are passed to build-stx-base.sh
we expect the docker image tagged as '...-latest' to be deleted
from the local build environment.
This did not occure ...
starlingx/stx-centos dev-latest 279c713e0b5d 7
hours ago 331MB
starlingx/stx-centos <none> dda342637517 19
hours ago 331MB
Adding a code block to handle the case when these flags are combined.
Closes-Bug: 1808544
Change-Id: Ic2d17bc285f3746965c2a5e7ec49c278f0fbf066
Signed-off-by: Scott Little <scott.little@windriver.com>
build-pkgs --container fails when run within docker
due to DNS lookup failure.
Need to add pkg bind-utils to mock when building in
the 'container' context, and to add an /etc/resolv.conf
via 'config_opts['use_host_resolv'] = True'
Story: 2003907
Task: 28370
Change-Id: I5f9ec16d29545ba74457a060323e07967e085556
Signed-off-by: Scott Little <scott.little@windriver.com>
The existing user interface for docker image and wheel creation
is not flexible enough to name images is described in the spec,
and to push those images to the starlingx organization at docker.hub
This update adds new arguements that will allow creation of
docker images with compliant names, and will allow them to
be pushed to https://hub.docker.com/r/starlingx.
I will also change build-base-wheels.sh to allow the
'os-version' to be pushed through to build-wheels-tarball.sh.
Story: 2003907
Task: 28370
Change-Id: I4378db63d5ff906ffd636034351e580afa5e3472
Signed-off-by: Scott Little <scott.little@windriver.com>
This update adds support to the docker image and wheels build tools
to allow the build of StarlingX service images from openstack master
branches.
The master wheels tarball makes use of the loci-built master-OS wheel
image as a base, with any additional base wheels not provided by loci.
Change-Id: I6de67e29a9b05189f2a0bad3dfdd1668fbdab42b
Story: 2003907
Task: 28179
Signed-off-by: Don Penney <don.penney@windriver.com>
Closes-Bug: 1792001
Fix edit issue for package puppet-nova/bash/resource-agents.
The cause of the issue is, there is "rm -rf %{buildroot}"
or similar line in the spec, which cause the buildroot folder
be deleted, and git info be deleted also. So lead to the
"git add" cmd failure.
Fix:
Try to ignore this cmd to avoid git info be deleted.
Test:
This script is for edit cmd only, no impact to normal build.
Tested several packages locally, no issue found yet.
Change-Id: I667c363bd82e68f107a6c51f65a2a9dd0d2434d4
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
When the build-base-wheels.sh is run via Jenkins, there
is no tty, so docker rejects the -i and -t options.
These are not necessary for the build, and are removed
from the command execution.
Story: 2003907
Task: 27531
Signed-off-by: Don Penney <don.penney@windriver.com>
Change-Id: I4b1f4c2573df7b62acd48ee404d5aa3c098a50b1
This update adds tools for building a base image (ie. stx-centos)
and the set of images for the StarlingX services.
build-stx-base.sh will build the base image
build-stx-images.sh will build the StarlingX services
Change-Id: I499583552f633b9e29566dde9233f5c5208d0593
Story: 2003907
Task: 27938
Signed-off-by: Don Penney <don.penney@windriver.com>
This update provides the following enhancements to the wheel tools:
- Add a prefix to config files for openstack release
- Change auto-generated version format
- Skip base wheel build if all base wheels are present
- Clean unneeded images from docker after build
- Fix bash globbing issue in get-stx-wheels.sh
Change-Id: I17dbf01288c29662fddae4fd3d9d90ab2521037f
Story: 2003907
Task: 27531
Signed-off-by: Don Penney <don.penney@windriver.com>
Problem:
User 'jenkins' hit a build-iso failure in sign-rpms.
Problem was a test that assumes user jenkins will always
provide environment variable setting for SIGNING_USER,
SIGNING_SERVER and SIGNING_SERVER_SCRIPT. This is not
a valid assumption.
Solution:
Explicitly test that all three environment variables are set
before invoking sign_packages_on_server.
Story: 2003906
Task: 27974
Change-Id: If9eb61b8001ca58f9262c2eccab96b4820c68102
Signed-off-by: Scott Little <scott.little@windriver.com>
Move the cd statement to the correct location above the sanity
check that the operation completed.
the validation below, comparate the latest statement was sucessfull.
so this, should be the cd statement.
Change-Id: Idd093cf215cf1b637e37b89e8496231c64260972
Signed-off-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
The helm charts are packaged in rpms by the build process
and the purpose of this script is to package all those
artifacts together into a single tarball, for use by
other components in StarlingX.
Uses centos_helm.inc in each repo to determine what to search for.
Currently tested with packages from stx-config and stx-upstream:
openstack-helm-infra
openstack-helm
stx-openstack-helm
Multiple tarballs are created, one per manifest.
A calculated checksum.md5 is included in each tarball
Story: 2004005
Task: 27933
Depends-On: I3727a15fbdadc345a104c2af3446aee845171309
Change-Id: I06259d27e8f25d0b05eecdea49b2365246ac3112
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
