Because we are using the 'zuul' user to SSH into our zuul workers, we
need to add it to configure_mirror.sh and still support the jenkins
user.
Change-Id: I79eff670f56d06c49a16e1330a73889e7b8482f1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Now that we have migrated all our jobs to fedora-25, we can remove
fedora-24 from project-config.
Change-Id: I3a58c785c12f779060a6dca5cd40ddcc46429971
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Now that fedora-25 is online, we can start using the new AFS mirror for
fedora. Keep in mind, we didn't mirror fedora-24, since fedora-25 will
be replacing it.
Once fedora-24 has been removed from nodepool.yaml, we can remove our
conditional check.
Change-Id: I1bd510c6a568b71b604c27d6cae16d2654063d0c
Depends-On: I762e55fb7fe0c5ebfb7810545ddb4d5ccfedf2ff
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We incorrectly setup our mirrors for UCA. Now we support 3 openstack
releases.
Change-Id: Ie3a05398d9fdc7a7f7c0c74366a7c4dd70200a2b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This review cleanup scripts that were exclusively used by
snapshot scripts for nodepool which were removed in:
https://review.openstack.org/#/c/331494/
Change-Id: I37900a5d698a43bc0d510100289a5a7ec527021d
Update our script to provide additional debug information, such as the
builddate of our DIB images.
Change-Id: Ie399b1ac9cd6c6ea4372378ae7b7bf930fac16a3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
A cosmetic change to reduce the debug output on DNS failures. First
check for valid DNS, then proceed with configuration.
Change-Id: I3370a155bf6b49088c398dc9d4d572b6210b84b2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
After noticing ubuntu-xenial launch failures in osic-cloud1, it looks
like our unbound service is taking up to 1 minute to start properly.
So wait 30 seconds for host to timeout, and try 10 times. This gives
us a 300 second timeout window to configure DNS properly.
Change-Id: Id0432c91cc853fb4ecab43da991948c2e9d84b7d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because how often we run into nodes having DNS issues, a large amount
of debug info is dumped into nodepool logs when the ready-script
fails.
As such, move our dns checks soon so the ready-script failure output
is minimized.
Change-Id: I003f8ee816f279ef30075524c1b806e93c56e7f9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Now that we have debian-openstack-newton-backports online, we need to
update our source.list file for debian.
Change-Id: I55ef64ff7f1ed0c3fbd01e9666d3adab76928f6d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We now offer debian-openstack packages[1] for jessie-newton. Give a
way for users to enable them.
[1] http://mirror.dfw.rax.openstack.org/debian-openstack/
Change-Id: Ie3343608fb4c3af77680fb959c717785bc2567c6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
To make it easier to follow additions to configure_mirror.sh as we
grow more and more mirrors of things, separate the configuration
file contents into separate variable definitions so they don't make
the script logic as hard to follow.
Change-Id: I421da8eefeb040d9daded59b8d62158f0aa1610b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We have successfully migrated the devstack-plugin-ceph repo over to
using the opt in apt sources list for ceph mirroring. They were the
consumer of this so we can remove having it installed by default now.
Change-Id: Ibb3f906e673aa86ad1222e52d3a1c96104b0a00c
Rather than enabling ceph and cloud archive repos by default when when
configuring test instances to use our apt repo mirrors write out
configures for them in another directory which can be optionally
enabled.
This prevents growing dependencies on these repositories except for
where explicitly desired.
To use a mirror either symlink to the sources.list.available.d/ files or
copy them into sources.list.d/ and rerun apt-get update.
Change-Id: I410f45064080e1bb93684899028e9af34bd555a1
* add NODEPOOL_CENTOS_MIRROR to set custom CentOS mirror in same manner
as for Ubuntu
* add NODEPOOL_EPEL_MIRROR to set custom EPEL mirror
* use $releasever for CentOS and EPEL repositories
Change-Id: I93c92aea60725bb9b172e6fed12655e4a94b265b
Ever since we started populating the subunit2sql db with data from all
gate and periodic jobs this script has been kinda useless. This was
added with the intent to preseed tempest runs with data from past runs
to hopefully have a positive impact on the testr scheduler. (its
actual usefulness is limited because of bug 1416512 and other
scheduler limitations) However, because we're not filtering now we
end up preseeding tempest runs with data from things like zuul unit
tests which are just a waste. This commit adds the necessary
filtering to the DB queries to ensure we only preseed with data
from previous tempest runs.
Note: For this patch to work the depends-on below must be included in
a subunit2sql release
Change-Id: I32866d63b5f0fa406213709fc88a9029d0318645
Depends-On: Id6ca7e7a4cd1f913d9f84efae5f48d247dd9ed64
Currently these 9 scripts are no longer used by nodepool. While we
could leave them in tree, I'd like to remove them as not to encourage
users to use / reference them.
They have served us well in the past, but no longer of use today.
Change-Id: Ib954ad59988b81a9b60ab4d2dede7f1008cea21a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This patch loosens the timeout requirements for npm on our jenkins
nodes, in order to accomodate a cold AFS cache.
Change-Id: Icba1366aac3cb831b4b00441f691cfaca38ddb6e
This patch adds the npm mirror location to jenkins' .npmrc file,
which should supercede anything except an explicit declaration in
a similar file at a project root. All javascript jobs from this
point forward will use our npm mirrors rather than the public
registry.
Change-Id: I2b6463225ac86b83fa0c11ecdd390968b2209c4b
Since this image is going to be short lived, we don't want to add
ubuntu-precise AFS mirrors for ubuntu packages.
Change-Id: I0dc0e6bbb9f695584945056528cfb7e02793bdcd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This reverts commit b4a933c18f4791d297a42a7d6f4fcff71b48b4a5.
Currently all our DIBs are failing to build, as a result we are
reverting stackviz until we can properly test.
Change-Id: Id336c1c8df7123c67f821a3a72da77c470a3a1f1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
In order to get Stackviz running against tempest gate run data, it
will need to be pulled in its pre-built form (i.e. with npm deps
installed) from the nodepool images. This change adds a couple steps
to build Stackviz so that it is in a ready state for retrieval and use
on the logs server.
A future devstack-gate patch will pull the stackviz directory from the
nodepool images onto the logs server. This nodepool patch is required
first in order to prepare stackviz for download. Installing the npm
modules takes a considerable amount of time (~5-10 mins), so it is
best that this work is done on the images before gate jobs are run.
Change-Id: I9bc50b8eba696264d2b97ed20a37f28f4768abb8
Reprepro now as local repos for xenial-updates and xenial-backports on
our mirrors. Lets use them now.
Change-Id: Ic1dbb9cf7e041ecce1a872386e0e3ce15757e261
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When cleaning up comments from lines of the form:
foo bar # this is a comment
You want to remove remove everything after the first comment character
not after the last comment character. This is because:
foo bar # this is # a comment
Should get everything after bar removed not everything after is.
Unfortunately when you use str.rfind() you remove everything after is
not everything after bar. Switch to regular trusty find to fix this.
Change-Id: I78aa6b51b5be03bd3b8ce7885415442171218977
Because reprepro will not mirror repositories with 0 packages,
xenial-backports, xenial-updates, xenial-security are missing from
AFS. Until we have them populated, we'll only configure APT to use
the xenial mirror.
Change-Id: Id4d788097ae4e1a5d0f13cb84a46fbb9ac9a839e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When we removed the restrict memory grub stuff because hpcloud was no
longer in use we also removed the configuration updates that set the
grub timeout to 0. This can save many seconds per instance boot purely
waiting for grub to timeout and continue booting the host.
Change-Id: I2d79ec4ebd55f927d868e06c8f00cf50f34f3b92
At the end of gate jobs, we get warnings from zuul_swift_upload that
are not actionable for us.
Use requestsexceptions to silence these warnings. Add requestsexceptions
to the venv that is created for zuul_swift_upload to run in.
Example file
http://logs.openstack.org/48/298048/1/check/gate-ha-guide-tox-checkniceness/62e2d16/console.html
The example shows 6 warnings starting with:
/usr/zuul-swift-logs-env/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:315:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject
Name Indication) extension to TLS is not available on this platform.
This may cause the server to present an incorrect TLS certificate, which
can cause validation failures. For more information, see
https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/zuul-swift-logs-env/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:120:
InsecurePlatformWarning: A true SSLContext object is not available. This
prevents urllib3 from configuring SSL appropriately and may cause
certain SSL connections to fail. For more information, see
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
...
Change-Id: I02b4b6c7b426f3e9aa498941e4d75f67066d4d27
This reverts commit 468f17c4f6afac507dba9d4783c7da6fde24f656.
This isn't working on fedora-23 or centos7 slaves for some reason making
it very difficult to get those nodes ready. Revert in order to do more
testing across the platforms.
Change-Id: I634e784b71a2d92a745f8d9c89532e953f48b9cf
Update iptables rules to allow slaves to talk to themselves over their
known IP addresses. This should allow services to listen on public or
private addresses in addition to the loopback address and talk to those
services over these IPs.
Specifically this should address a problem with rabbitmq coming up on
the public IP but not being able to talk to it due to iptables.
Change-Id: If851eb8ce97ca0c0c113f0a924ec0916c993cf87
This patch adds Debian Jessie diskimage-builder image to nodepool.
debian-packageinstall element is based on debian-minimal and provides
additional required packages, change is introduced in
I9486b1356eac74272e79655c9024d86ee486628c
Depends-On: Ic39ba2b5ceb5018efb75742547b2babf80827e56
Depends-On: I7a661c8b5d4bb632eb32b5018c55474c1c66a4e8
Depends-On: I4a5a8352388de46e9e7713fa11951fcde27eaad9
Change-Id: I851cb47e844a72a430b578dd63d8b230ee6e3ed9
Co-Authored-By: Igor Belikov <ibelikov@mirantis.com>
We have attic and stackforge projects all of which are now inactive.
Because they are inactive we don't need to be caching these repos on
every test slave image.
Change-Id: I4df78c3d542758ce6159c195c1407f1d56f565a0
