The ipxe file can have different extensions based on the distro.
Changing from .pxe to generic extension should cover them all.
Change-Id: I6da5bf86a022315861d140029ce25321a5b57469
Fix hardcoded yum command by package manager variable, PKG_MANAGER
in scripts/install-deps.sh. That was problem on Fedora where default
package manager is dnf and not yum.
Change-Id: I2dbec079adbfcab71e46e086fcd201a5175f22d5
Story: 2005451
Task: 30503
Currently all of the opensuse 150 jobs are failing due to being unable
to install virtualbmc via pip. The failure signature is:
Cannot uninstall 'six'. It is a distutils installed project and thus we
cannot accurately determine which files belong to it which would lead to
only a partial uninstall.
The usual solution to this problem is to install Python packages in a
virtual environment. This change modifies the zuul CI jobs to use a
virtual environment for all openSUSE jobs.
Change-Id: I2840ec32cffcaf5d7f32a91907d51c1a500f19f1
The install-deps.sh script is sometimes used in a server sitting
behind a corporated network, for those scenarios is necessary to
pass proxy global environment variables. This change enables that
for fedora/centos OSes.
Change-Id: Ic327168eec892970cd2d7dd4b24d92fa0deee184
We make the following changes in order to support openSUSE Leap 15.0
- Updates the bindep.txt file to match the SUSE family.
- The netstat binary has been moved to net-tools-deprecated
package.
- We fix a bug when checking the status to extra dependencies. The
variable is an array, but the code was only checking the first
entry of the array.
- Replace spaces with '_' on ansible_distribution since openSUSE Leap
15.0 is being reported as 'openSUSE Leap' by Ansible.
Change-Id: I800331e29e0d47a3121229cd66a5814277e34b84
We're not capturing the logs as we likely can't
copy the contents of the folder. So lets use
sudo to exec the cp command, and for good measure
we'll pull a list in case there just is nothing
present to be uploaded for logs
Change-Id: I73ad58037518e6d334efe0bad99458a2fabfc047
Without this test, the Ironic services might end up
misconfigured and machines can not be provisioned
properly. The Ironic configuration templates
that Bifrost uses should be validated for both
Keystone enabled and disabled.
Change-Id: Iab4457a6dcea232b5c19e681dc87b08570c2a120
Depends-On: I4c733dba8e65fe624c9de5b9c5722cef2df59109
The [[:digit:]] does not work correctly when using sudo
2018-10-04 14:31:33.319558 | opensuse-423 | + sudo cp '/var/log/libvirt/baremetal_logs/testvm[[:digit:]]_console.log' /home/zuul/workspace/logs
2018-10-04 14:31:33.328756 | opensuse-423 | cp: cannot stat '/var/log/libvirt/baremetal_logs/testvm[[:digit:]]_console.log': No such file or directory
As we can see [[:digit:]] is not expanded so no logs are collected. This
is because the expansion happens by the user but since he does not have
access to the directory the command fails. We can fix this by opening a
subshell so it gets run with sudo privileges.
Change-Id: Ibdf8606a3b72c48faffdf2b58b122b690008ba98
As get-pip.py evolves based on pip 10, while we're still using
pip 9, changes in the way it can be used are causing problems.
For example, the ability to use --download is no longer there.
As such, let's pin to a known good version and leave it at that
until we no longer need to use this script. Version 3.2 maps to
pip 7.1.2 which fulfills our needs.
Change-Id: I2fa1c67ecd712364a822edcf7470e5489447d990
Commit 50c7edb9d8c5383303207026d30b4badc0c5f398 ("Only install
libvirt-python and python-lxml via pip") added support for installing
libvirt-python and lxml on the virtualenv but this leads to problem like
the following one when executing all the virt_* modules.
The `libvirt` module is not importable. Check the requirements.
The problem is that libvirt-python and python-lxml are installed in the
virtualenv, but when Ansible operates on localhost, the default
interpreter is /usr/bin/python. As such, it's unable to find the
necessary requirements for all the virt_* Ansible modules. In order to
fix that, we need to pass the correct Ansible python interpreter when
operating on localhost.
Since we now pass the venv interp to Ansible, we also need to have
python-apt in the virtualenv in order to be able to use the 'apt'
module. However, in [1] it's clear that python-apt can't be installed in
a virtualenv so we have no other option than switching the ansible
python interpreter back to the host one for the tasks which need the
'apt' module.
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "Could not
import python modules: apt, apt_pkg. Please install python-apt
package."}
[1]: https://github.com/ansible/ansible/issues/14468
Change-Id: I59949b0ff821aac1767f56844082c2e8a32035f6
Update bifrost to handle fedora as minor changes between
releases have been slightly breaking, and have gone largely
un-noticed since the bulk of bifrost users seem to use
centos or ubuntu.
Added a fedora 26 job to help enforce support moving forward.
Change-Id: I9a654b91edfa52321adccfa5df0c260eb0c9e487
Closes-Bug: #1695969
Closes-Bug: #1695968
Before trying to install any packages for the tests, we should ensure
that the package metadata is updated otherwise some packges may be
reported as missing.
Change-Id: Idc2033eee917b3024e392712aec9feb7611c097a
Sometimes we need to troubleshoot CI a little more,
So lets preserve IPA failure logs so we can investigate further.
Change-Id: I47e7f45e65997a61e41c73e072e3c8c31ef9a244
This helps fixing the following problem when installing MySQL on
Ubuntu hosts:
Preparing to unpack .../mysql-server-5.7_5.7.19-0ubuntu0.16.04.1_amd64.deb ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs
shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Configuring mysql-server-5.7
[...]
New password for the MySQL "root" user:
The package configuration becomes interactive and asks for configuring
the root password on MySQL and this breaks automated testing.
Change-Id: Icacd843950eee1b87bac2c3387e3562f8f584b77
Relying on the installed package manager to detect the distribution
is not reliable since it's possible to install multiple package managers
at the same time. As such, lets simply use the information in the
os-release files.
Change-Id: I285c6cd94932f84bb9133b39d282c39d9a25b301
When trying to install bifrost on CentOS , shade could not be
installed properly, because the system had an old version of
setuptools. Add this upgrade to install-deps, to have the latest
version of it.
Closes-Bug: #1695969
Change-Id: I31e2d3af04eb6fa74a03e7fa731f58dbb30bd3bc
Newer bindep versions are not working with languages not
set to en_US. As a workaround, export the LANG setting.
Workaround for: https://storyboard.openstack.org/#!/story/2001205
Change-Id: I60be976ff151bb61014f0272cdf02a4fb4851d0a
The nodes in the OpenStack CI may already have the epel repo present
but it could be disabled by default. As such, try to enable it as
early as possible.
Change-Id: Ib06f5aaf7f577d101b5b87f8cd81441380e56783
Use the --no-recommends option on zypper to avoid pulling too many
indirect dependencies. All dependencies need to explicitely listed in
the bindep.txt file instead.
Depends: I06b9682751a4aefe21866336ed9726541f74b22a
Change-Id: Ieeaada5585fa411edf3993534a94b60ed46439ed
OpenStack CI uses bindep to figure out which packages need to be
installed on the host. However, when a bindep.txt file is not
available, a default one is used which may or may not be good enough
for all cases. It's best to have control over the packages which are
going to be installed on the host and as such we add our own bindep.txt
file.
Change-Id: Idab36eb82d916183f2fb40d326d095c5a8ee34b2
Fixes the following problem:
2017-06-19 09:21:19.720993 | + mysql_setup
2017-06-19 09:21:19.721006 | + local DB_ROOT_PW=insecure_slave
2017-06-19 09:21:19.721023 | + local DB_USER=openstack_citest
2017-06-19 09:21:19.721045 | + local DB_PW=openstack_citest
2017-06-19 09:21:19.721075 | + sudo -H mysqladmin -u root password insecure_slave
2017-06-19 09:21:19.768612 | mysqladmin: connect to server at 'localhost' failed
2017-06-19 09:18:42.671268 | error: 'Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2 "No such file or directory")'
2017-06-19 09:18:42.671317 | Check that mysqld is running and that the socket: '/var/lib/mysql/mysql.sock' exists!
Change-Id: I06b9682751a4aefe21866336ed9726541f74b22a
- on Ubuntu Xenial, `python-virtualenv` package no longer installs
`virtualenv` shell script.
Change all `virtualenv` invocations to 'python -m virtualenv',
which is more portable.
- in `pip_install.yml`, add `virtualenv_command` argument using
the same portable invocation.
- in `install-deps.sh`, the condition to install `python-virtualenv`
was missing the `not` part
- the venv is created with `sudo` elevation in `install-deps.sh`,
thus in `env-setup.sh` Ansible must be installed into venv
using `sudo` elevation as well.
- in `install.yml`, fix passing `bifrost_venv_dir` var to `pip_install.yml`
Change-Id: Ifaa54a0dc97ed59ca8360e62878b603a04105e46
To move bifrost testing to *_ipmitool drivers + virtualbmc, we need
the baremetal data file to support ipmi port setting, as in the case of
virtualbmc, all test VM nodes use the same local IPMI address with
different port per-node.
Unfortunately, the CSV baremetal inventory format that is used in our
testing procedures does not support setting IPMI ports.
As the CSV format is considered a legacy format, let's move testing to use
JSON-formatted baremetal data instead of fixing the legacy format parser.
Changes are mostly to 'bifrost-create-vm-nodes' role and it's callers.
Some level of backward compatibility is provided:
- baremetal_csv_file var is still accepted, and its value is used as
path to write data, although the data will nevertheless be in JSON format.
An extra helper script is added to reduce the number of nodes in
inventory when testing DHCP.
Also the script in 'bifrost-test-dhcp' role is changed to support
loading data from JSON.
This change officially deprecates using CSV formatted baremetal inventory
files.
Handling CSV baremetal inventory files will be removed in the Queens
release.
Change-Id: If2dcf43857195611ef342fe602330878378b021b
Partial-Bug: #1659876
Commit afa9951dd92e2 ("Move database setup to bifrost") moved the
openstack_citest database setup from project-config to bifrost.
However, we need to provide a default (even if it's an empty one)
value for ZUUL_BRANCH (eg when running outside of zuul) in order
to not fail due to using 'set -u'. Fixes the following error:
./scripts/test-bifrost.sh: line 76: ZUUL_BRANCH: unbound variable
Change-Id: Iea1193ce0ac83013f4e41398b4e7ad9527d427ac
Fixes: afa9951dd92e2 ("Move database setup to bifrost")
Move openstack_citest database setup from project-config to bifrost.
The code is conditional on running in OpenStack CI - developers can
set up the database with setting ZUUL_BRANCH. This is the same logic
as used elsewhere in bifrost testing.
OpenStack CI will not setup databases anymore in project-config,
this needs to be done by the projects themselves now.
Change-Id: I8485751eda745841c1436ecd766c9bb95446f407
Needed-By: I339d2a301a76ad34ea4bcbfb26c13a21ecd86b6f
The script uses 'isdecimal' method on strings but this is actually valid
for unicode strings only.
Cast to six.text_type before accessing this method.
Change-Id: I032f4cc90140aba4eecd4317a9cd153989ed5add
Always exit with success code when collecting various logs. This allows
this script to be used to collect partial logs in case a deployment
fails since it may help with debugging the root cause of the failure.
Change-Id: I4d5a2c71303e9ed466b5fd5dbc63469677f2380f
Instaling and using Ansible from source for bifrost has several
drawbacks, mainly due to how Ansible's 'ansible/hacking/env-setup'
script mangles with PATH and PYTHONPATH, which complicates running it as
part of other scripts. Besides, cloning the whole repo and it's
submodules is somewhat longer.
The main reason why we were doing that at all was a necessity to install
some additional Ansible modules from newer Ansible versions, which we
dropped right into the source of Ansible code - but this does not have to
be so.
Luckily for us, all Ansible versions we target to support can load
modules from 'library' directory next to playbooks/roles,
and we already use that for 'os_ironic_facts' module.
The need to install a particular module can be assessed by running
ad-hoc 'ansible' command against localhost with the module in question
and without any arguments ('ansible localhost -m <module>'):
- if the module is available in Ansible, the stderr will contain
"changed" substring (as part of the standard module output)
- if the module is absent form Ansible, "changed" string will be absent
from stderr too, in which case we can download the module from github
directly into 'playbooks/library' directory.
This patch removes possibility of installing Ansible from source, and
always installs a released Ansible version via pip.
If not installed into venv, Ansible will be installed in user's ~/.local
directory via 'pip install --user'.
The missing but needed modules are downloaded as described above.
Some level of backward compatibility is provided:
- when the ANSIBLE_GIT_BRANCH has form of 'stable-X.Y', the
env-setup.sh script will do the next best thing and install latest
available Ansible version of X.Y.w.z
Also, ANSIBLE_PIP_VERSION can now accept a full pip version specifier:
- if ANSIBLE_PIP_VERSION starts with a digit, this exact version will be
installed (as 'ansible==X.Y.W.Z')
- otherwize this whole variable is assigned as Ansible version specifier
for pip, e.g
env ANSIBLE_PIP_VERSION="<2.2" env-setup.sh
will result in pip being called as
pip install -U "ansible<2.2"
Closes-Bug: #1663562
Change-Id: I2c9f47abbbb6740d03978f684ad2c876749655b7
The VMs which are created by the bifrost test scripts are not meant to
be used in production so improve the overall CI job performance by
switching to the 'unsafe' disk cache mode.
Change-Id: I0e6fe8de3f2306ad3cf69708d3fe31f0bda9693c
The original os_ironic_node module, nor bifrost as a whole
lacks a concept of waiting for a node deployment to reach an
active state where the conductor no longer has to take any
additional action upon the node.
In order to allow users to utilize the Ansible serial option,
we need to support the ability to wait so the conductor is not
overloaded by the user in specific edge cases.
Change-Id: I1fe3353a56a54ecde25f9f237b85ca009813f541
Depends-On: I69eee2d254cde2fffcf0c1ac7679a623fa7f97a5
