There are some cases where packages (or its dependencies) are missing.
Cache them on a best-effort basis, and ignore any that are missing.
They will be downloaded and installed if and when they are needed.
Change-Id: I5cd7f5f35b4fef6002c6cc3d66543a33a6972ef8
This commit adds a print statement with the stdout from the run_local()
call which executes build_wheels.sh in devstack. Right now when the
script is executed all it does is print that it's running but we are
unable to debug anything that could potentially go wrong during the
execution of the script without the output from build_wheels.
Change-Id: I7433f054127a42fa73598db219c6e351efc98fb7
Update manual dib mounting instructions to match changes made in the
tools/mount-image.sh script (I0deba1907f390f0d308ae6c8ed5448afbbf7688a)
and add a reference to the script itself in the documentation.
Change-Id: Ic72bf05f21cecb8bdff36956f3635c2f504c4dff
This commit adds calling tools/build_wheels.sh to the cache devstack
step in the nodepool scripts. This will generate and cache the wheels
before the devstack run, which is a length process normally involving C
compilations.
Change-Id: I9064d7d9b9b879a05d665c3b002a631fcc953f52
Use GIT_BASE variable to be consistent with:
nodepool/elements/openstack-repos/extra-data.d/50-create-repo-list
Change-Id: I2ae933aef62cb1f9009483406bb15405efaf4001
pkgs-to-install is written to /tmp/pkgs-to-install not
~/tmp/pkgs-to-install/.
From nodepool logs:
2015-02-26 14:24:11,465 INFO nodepool.image.build.devstack-trusty:
/tmp/in_target.d/install.d/50-download-pkgs:
line 28: /root/tmp/pkgs-to-install: No such file or directory
Change-Id: If367be96bbc2761faa9bd4b4bb173a3ecbe9b406
Allow supplying filename and paths with '**' recursive glob matches
to zuul-swift-upload. Since bash (or shell etc) will expand on any
filenames provided to the program this needs to be used in quotes.
Usage example:
./zuul_swift_upload.py my_results.txt '**/sdist/*.zip' output.log
The hierarchy is always flattened meaning the supplied list is
placed in the topmost generated index.html. Sub-folders still keep
their hierarchy.
Change-Id: I9ba04f7e46b579dcf3f8ad0bd188f41fa5dbcad9
When we build our images we attempt to checkout all of the valid
devstack branches so that we can list all of the images necessary to be
cached for each branch. Unfortunately this devstack repo is owned by
root during dib and snapshot image builds. This means we must use sudo
when updating the repo (either changing branches or updating the
current branch).
Change-Id: I8cd09cfed4d586648dcbd34fa04bfc030c31ee45
Since grub isn't installed by DIB until finalise.d/51 we need to
make sure the restrict-memory element happens after that, so it can
find and alter the grub configs accordingly.
Also make it apply the updated configuration, similar to how
nodepool/scripts/restrict_memory.sh does it.
Change-Id: I854f3bd1850594811cc8957f7a9263c33dfe6826
There doesn't appear to be anything in TripleO that relies
on these clones. Also, this is a bit confusing since people
may expect that things copied into ~workspace-cache would
eventually end up at /opt/git/ (this is not the case).
The correct way to add projects to a job is to export a
PROJECTS variable like devstack-gate does.
Change-Id: I0a96855ed9a38bcade6e506b37621b6b7b348b49
Unlike before where the jenkins user was editing files that it owns in
its homedir the /etc/pip.conf editing needs root permissions since that
file is owned by root.
Change-Id: I485a08cb243cc0abf4c848e6061606b4320a59cb
These are are too long to be properly viewed in github.
Also, by putting each puppet argument on a separate line
will help visual clarity and future maintenance.
Change-Id: I4b04d54ef57fd10660eae0b217433880551e86dd
This change is adapted from Ief25f224fbe15530a503f7c53acef8759df281b4
to fix rspec installs when puppet runs during old school nodepool
snapshot builds.
Note we are not updating DIB and snapshot together because we do not yet
use DIB builds for the bare-* $THIN=false builds.
Change-Id: I4d0ab20d437da0c779bddc286664f9def98484e1
This patch adds check test coverage on tripleo-puppet-elements
and puppet-tripleo.
As part of this change we also drop the normal tripleo check
tests from tripleo-puppet-elements because it isn't needed
and is a waste of resources. Only the puppet tests would
actually make use of this code.
Change-Id: I079e7b3bcbb36828390a034970092e3e88de4acf
Because of the testrepository bug 1416512 using the aggregate stream
generated by sql2subunit the timing data would be ignored by the testr
scheduler. The long term fix is to address this in testrepository
eventually. But, in the short to get any utility out of the preseed
data this commit switches the preseed to not use the aggregate view
stream and instead take the 10 most recent runs and use those as
the preseed data. While this data isn't as good for performing the
optimization, but it will at least use the data.
Change-Id: Ib0e815ca1966f5b5806fb257b91ac7a25d4db18d
These are no longer being used to run jobs, so clean up references
to them.
Change-Id: Iac7ffde66a2d5d1a1361ed9bcadb9144e034f10f
Depends-On: I40d5d09f28ef53583d239d2e852e9c50b5962cf8
The "yum repolist" command was failing with what I think was the
same error that the update command was designed to fix. I am
unsure how or if that ever worked correctly, or what might have
changed to cause it to start impacting that command.
At any rate, specifying "epel*" as the repo name to disable will
cause the update command to work regardless of whether it is
installed.
Change-Id: I1d978ade76ec78e8895e00bbf64e5e40529223f8
When the testrepository directory is created by calling the preseed
script the permissions are different from the rest of the tempest
repo. When the dir is rsync'd by devstack-gate it errors out with
permission denied on the .testrepository dir. This commit fixes this
by setting the ownership to be the same as the rest of the cached
repo.
Change-Id: Ie12f3f0d824401f2ed53a77f8f307ea0f178854e
We cache pip in cache_devstack but not in dib element cache-devstack
Fetching get-pip.py across the internet breaks surprisingly often in hp
cloud, so precache this.
Change-Id: I86d8e54b5a3129bedf7e472d4463009c139f0656
Related-Bug: #1413034
The zuul swift log uploader script needs argparse on python2.6 and
requests on all python versions to work properly. Install these two
packages to the virtualenv configured for this script at image build
time.
Note that `pip install argparse` on python >= 2.7 is not an error
it just reports that argparse is installed as part of the python
installation and moves on.
Change-Id: Ie79b59e1c687cce2b435c9704959b02911f94778
The DIB cache-devstack element's early-source-repos script was using a
hardcoded cache path rather than rooting at $DIB_IMAGE_CACHE. This meant
that for this portion of the DIB builds they insisted on writing to the
current users homedir and not the overridden cache path. Root the cache
path properly against $DIB_IMAGE_CACHE so that an overridden path is
properly used.
Change-Id: I9c84de7497bd9259b5660e78e84d2adc86f9d61c
When seeding tempest with testrepository subunit data use the correct
path to the tempest repo in the imgae build. Because we moved to
install.d we ues a path relative to the chroot and not relative to the
mount path.
Change-Id: I8e296595945c8cfe91a88f69ae46fe33c53df2db
DIB needs its scripts to be executable or they are not run. Update
permissions on two files that were missing the executable bit.
Change-Id: Ie172e50de57b5168264964644cd28530f023542a
Also rename the element to one which runs in the context of the
diskimage chroot rather than in the root context, and make it
executable.
Change-Id: I093d02706f2965e071e2a76ab1131ac29016d03c
This commit adds a new nodepool script to use subunit2sql to preseed
testrepository with timing data to use for scheduler optimization.
This uses aggregate data from all the tests stored in the subunit2sql
db from all the test runs to generate a subunit stream with test
execute times which equal the rolling average stored in the tests
table.
This also adds a call out to the new prepare_tempest_testrepository
script in order to pre-seed tempest's testrepository with data from
the subunit2sql DB. This will enable the testr scheduler to perform
some worker balance optimization based on the average run_times from
all the previous gate runs.
Change-Id: I04404a55baf4da2745760246b8606ed3db52c03a
Set up a venv for zuul-swift-logs and install its dependencies
(python-magic) there instead of as system packages.
For DIB use 90 as uploading logs may be considered more important
than installing zuul.
Change-Id: Id0fc01729853e65c81cdd50e4ffa4d0d6de00ae9
Recent versions of pip will create ~/.cache if it doesn't already exist.
If this happens while running "sudo pip" the resulting dir will be owned
by root and 700 - which breaks anything else on the system that wants to
use ~/.cache
I *think* this finds all instances of "sudo pip" in this repo and fixes
them to use -H. This should mean they always run in the right ~
Change-Id: I47ddb3b591df6ac2100f09b38c9b8a03cb1ba1ff
closes-bug: #1405732
This will enable the flexible configuration of the XVA and the URL
through nodepool, thus it's not needed to push an infra change for
changing those values. Also if you do not have those values configured,
sensible defaults are provided.
Change-Id: Ie2be50f842b4220d4e7f9d8e830d3df3262a6353