We've seen jobs where tests fail due to what appears to be rng
starvation. Enable virtio rng device to try and alleviate this.
Change-Id: I70d800cdc45b6008f775110f22c0000736421529
This reverts commit 168ca7f0a474f1207ee01dab0ca2e70f34783e9c.
Removing postgresql support from devstack was unnecessary
since it's not broken and not causing maintenance issues
as far as I know. The commit being reverted said that pg
support was deprecated in Pike but nothing in the docs or
commit message refer to official deprecation of postgres
support in devstack or openstack in general. Not to mention
that there are still postgres-based jobs that will no
longer work *and* the notification to the mailing list about
doing this happened *after* it was already done [1] leaving
stakeholders with no time to reply.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010196.html
Change-Id: Ie7036d37d79e6aba462b7c97f917e2e7aed108f9
This was deprecated for removal in Pike. It's probably time to drop it.
Note that the 'postgresql-devel'/'postgresql-server-dev-all' packages
are retained since some packages still include 'psycopg2' in their
general requirements.
Change-Id: I51e8354e99972757253ce259e6c03c91da24398c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Per the referenced bug, $NOVA_CPU_CONF was previously being initialized
by copying $NOVA_CONF, thereby trashing any values already configured in
$NOVA_CPU_CONF.
With this commit, we merge the values from $NOVA_CPU_CONF in after the
copy.
Note that this makes use of the merge_config_file function, which is
defined in inc/meta-config, which wasn't being sourced from every code
path that hit start_nova_compute; so this commit also moves that import
from stack.sh to functions (next to the other imports from inc/, which
makes sense anyway).
Change-Id: Id3e2baa2221e13f512f8dcf1248e1e15b6a7597f
Closes-Bug: #1802143
Remove nova cells v1 support, which also allows/necessitates removing
support for nova networks (which was only supported with cells v1) and
nova-consoleauth (which was required by cells v1 but is unnecessary
otherwise).
The Depends-On isn't really necessary, but it's here to make sure this
doesn't merge until we _really_ have killed cells v1.
I honestly expected this patch would be bigger.
Change-Id: I90316208d1af42c1659d3bee386f95e38aaf2c56
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-On: Ib0e0b708c46e4330e51f8f8fdfbb02d45aaf0f44
Don't specify hypervisor_type=vz property for hds images as nova
now reports only really connected hypervisors and not all supported
by a compute host
Change-Id: Ibec0856519ffe593d44d123f3b401eae19f7d95a
When we change the name of compute host then devstack is breaking
because it is using default host name from host.
How to change compute host name in local.conf
[[post-config|$NOVA_CONF]]
[DEFAULT]
host = foo
Change-Id: I4d4392f1f58f0431b10764610668565af88d392f
Signed-off-by: Prabhat Ranjan <pranjank@in.ibm.com>
/tmp is a world writeable directory, so using hardcoded filenames
in there is just a bad coding style (susceptible to symlink attacks).
Avoid using it to not give a bad precedent.
Change-Id: Ia66763a0e4714f2226e98dbd85600b2035bd5088
In Change-Id Ia3843818014f7c6c7526ef3aa9676bbddb8a85ca the 'host'
setting used for each of the fake compute hosts was accidentally named
'nhost' ('\nhost' was edited poorly), so the setting doesn't actually do
anything: you create multiple nova-compute processes they think they are
all on the same host and only one hypervisor and resource provider is
created.
With the correction in place, the wait_for_compute function needs to be
updated to be aware of the fact that the hostnames on the compute
services will have a numeric prefix when the fake virt driver is used.
Change-Id: I5e8430d170c0b1c4f195ebe510aff8be59e4a3bc
In function 'get_instance_ip', 'nova' client command is used to get
instance information in order to retrive IP address of the instance.
There is no need to use the nova command, since 'openstack' client
already supports such basic operation.
Moreover, 'openstack' client has an option to get value of specified
column. That brings more accurate way of retriving IP address.
This patch replaces nova command in 'get_instance_ip' by 'openstack'
command. Note, this nova command is the only one in devstack tree.
Change-Id: Iee0b81a994a4da5b3f4572c2e8eb30514cd43f89
Signed-off-by: Ryota MIBU <r-mibu@cq.jp.nec.com>
The loopback device is created for ceph osd. If the directory
${storage_data_dir} has been mounted when create disk, we should
umount ${storage_data_dir} instead of ${storage_data_dir}/drives/sdb1.
Change-Id: Ie9fe81c820c485dab9f049cf5a81c02424925728
Closes-Bug: #1689089
Reason is to be identical to the upstream KVM CI. Some Tempest
tests rely on vdX virtio-blk device naming. Others simply create
their own with a brand new image. Also, the scsi support on the
CirrOS image is limited, tests booting from volume fail.
Change-Id: I389147a58042aa6098a695e6dd32f3e697fbbbab
(1) when checksum fails, better delete the broken files and try the second time;
(2) amazon s3 is not good in mainland China, better try one more time with wget
Change-Id: I24ee73f216b78bd80564863cd335e5d5a9b56360
We need this for every devstack run now, so downloading it from github
every time isn't the most awesome thing in the world.
Add an extra variable EXTRA_CACHE_URLS which will be appended to the
output of tools/image_list.sh. This way, these files will be
downloaded during the daily nodepool build, but they will not be in
the IMAGE_LIST and hence be considered as images to upload.
Add a function get_extra_file which echos the path to a file given the
URL. It will first check the cache at $FILES, and if not present
download it.
Update the documentation in image_list.sh to reflect what's happening.
Move the defaults for etcd variables into stackrc, since it is a base
service now.
Change-Id: I86104824a29d973a6288df1f24b7891feb86267c
With cell v2, on initial bring up, discover hosts can't run unless all
the compute nodes have checked in. The documentation says that you
should run ``nova service-list --binary nova-compute`` and see all
your hosts before running discover hosts. This isn't really viable in
a multinode devstack because of how things are brought up in parts.
We can however know that stack.sh will not complete before the compute
node is up by waiting for the compute node to check in before
completing. This happens quite late in the stack.sh run, so shouldn't
add any extra time in most runs.
Cells v1 and Xenserver don't use real hostnames in the service table
(they encode complex data that is hostname like to provide more
topology information than just hostnames). They are exempted from this
check.
Related-Bug: #1708039
Change-Id: I32eb59b9d6c225a3e93992be3a3b9f4b251d7189
With cell v2, on initial bring up, discover hosts can't run unless all
the compute nodes have checked in. The documentation says that you
should run ``nova service-list --binary nova-compute`` and see all
your hosts before running discover hosts. This isn't really viable in
a multinode devstack because of how things are brought up in parts.
We can however know that stack.sh will not complete before the compute
node is up by waiting for the compute node to check in before moving
forward. This puts a few more seconds into the run, but ensures
everything is solid in multinode environments.
Change-Id: I667e6a9be3fee8bb5bfd73426eef567489e3d88d
As a follow on to I4c269a7f3d63ee9a976e7c3636fc3e5e8dab9ae3; the
quoting gets tricky when putting arbitrary command-substitution
strings into saved echo-able strings. As they say, "the only winning
move is not to play" :)
An alternative proposal is to not write this into a script but just
dump info into a file. To my mind, this has several advantages --
avoid getting involved in quoting, not dropping a script into the
global environment -- it's just as easy to "cat" -- and the plain-text
file can be collected as an artifact during log collection (also moved
git commit line to separate line for easier parsing during log search,
etc).
Change-Id: Ic7391dd087657c0daf74046e4a052c53f4eb6e1a
This commit adds a new function get_random_port to return a randomly
available port from the local port range.
Change-Id: Icaed180cc14602a74cdb3fd3456b690d8a4c729c
With cinder supporting this now, start logging global_request_id in
systemd logs. It will be None for all the services until the work
starts coming together, but it is safe to do.
Change-Id: Ic6ba1a42da88c03e43d89658b453f6a0b353e0db
truorfalse function from common-functions accepts default as the first
parameter. The arguments for USE_JOURNAL were mixed up and this commit
restores correct order.
Change-Id: Id3621b0e1910a625d6cfb8e81bd27bea82543ae9
This change ensures that when uploading vhdx images, we use the
proper format.
At the moment, vhdx images are uploaded as vhd, which can be
troublesome: first because this is misleading, second because the
actual image format may be checked, having the image rejected.
Change-Id: I9578be41ea9dc252404b7553679ac527e08a0ff6
One of the pending issues with the conversion to systemd was the loss of
log coloring. It turns out that journalctl by default strips out
characters it considers "unprintable" - including the color codes
emitted by the old-style logging. However, journalctl can be made to
print them by adding the `-a` flag.
This change makes devstack's log formatter conf settings include color
codes like the old screen-based setup used to
We also remove stackrc's setting of JOURNALCTL_F, whose usage was
removed via I6af6d1857effaf662a9d72bd394864934eacbe70.
Change-Id: I2401e267913a24d18dae355aa933072dbbdab1d8
This sets our default ulimit NOFILE to 2048, which is double what we
set things like mysql'd max_connections to.
Change-Id: I5126bed1e6b9f8c64db00eae4151ac61e47b1bf8
use_journal uses the systemd native path for logging, however there
are concerns that this might be negatively interacting with
eventlet. To be on the safe side fall back to stdout.
This introduces a USE_JOURNAL option which will let folks turn this
back on for testing.
This also adjusts the debug lines. When using the journal the pid
reported by systemd is correct. When using stdout, it will be the
parent process id, so we need to keep it to see which child each thing
is coming from.
Change-Id: Id7891c532bf99c099252e82d511a37a49506fea9
This is going to be a new option in oslo.log, which we can start
setting early to make it take effect.
Change-Id: If0e5e4717a1810c759058f33608fbac7543f2d85
During the PTG there was a discussion that the screen developer
workflow wasn't nearly as useful as it once was. There were now too
many services to see them all on one screen, and one of the most
common service restart scenarios was not restarting one service, but a
bunch to get code to take effect.
This implements a 3rd way of running services instead of direct
forking via bash, or running under screen, which is running as systemd
units.
Logging is adjusted because it's redundant to log datetime in oslo.log
when journald has that.
Swift needed to have services launched by absolute path to work.
This is disabled by default, but with instructions on using it. The
long term intent is to make this the way to run devstack, which would
be the same between both the gate and local use.
Some changes were also needed to run_process to pass the run User
in. A hack around the keystone uwsgi launcher was done at the same
time to remove a run_process feature that only keystone uwsgi uses.
Change-Id: I836bf27c4cfdc449628aa7641fb96a5489d5d4e7
This commit change check of 'running_in_container'
method so that other services ironic, nova and neutron
will not break.
Change-Id: I42eb587cfaebf37944cb10e459b8b8f7b4b4e4ba
Instead of grepping for 'lxc' in /proc/1/cgroup, use systemd's
features. This now at least also works in LXD containers.
Change-Id: I35e807c26f0b1fbba83ddbe04cfb4901a7a95cbe
When variables use the 'declare' directive, it is by default a local
variable. While other variables have global scope.
For example:
declare -A AN_ARRAY # local in scope
foo=1 # global in scope
This causes errors to occur as some of the variables will be local only
and others will be global.
Update the code, as appropriate, so that variables using the 'declare'
directive also include the '-g' flag to have them also be global. Not
every instance of a declared variable has been updated.
Closes-Bug: #1669509
Co-Authored-By: John L. Villalovos <john.l.villalovos@intel.com>
Change-Id: I2180b68fe861ad19c6d4ec0df0f9f8a528347862
Calling enable_kernel_bridge_firewall inside a
container, devstack will crash because it tries to
load a kernel module by calling 'sudo modprobe' on
net.bridge.
Change-Id: Id4718c065d5a8c507d49f38e19c2796a64221aa4
Closes-Bug: #1662194
This provides a single setup_logging function which builds consistent
colorization if the config supports it, otherwise builds the identity
strings that we need to actually keep track of requests.
Change-Id: Iffe30326a5b974ad141aed6288f61e0d6fd18ca9
This makes setup_colorized_logging be a thing which takes a single
parameter and doesn't let projects do things differently. It also
changes the order of values from user / project to project / user to
represent the hierachy more clearly.
Change-Id: I8c0ba7da54be588e3e068734feb4f78ed7c5a14a
Neutron doesn't use any arptables based firewall rules. This should
somewhat optimize kernel packet processing performance.
I think the setting came from:
http://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf
but does not apply to the way we use iptables.
Change-Id: I41796c76172f5243e4f9c4902363abb1f19d0d12
Closes-Bug: #1651765
With the plan [1] to stop enabling it by Neutron iptables firewall
driver itself, deployment tools should catch up and enable the firewall
themselves.
This is needed for distributions that decided to disable the kernel
firewall by default (upstream kernel has it enabled). This is also
needed for distributions that ship newer kernels but don't load the
br_netfilter module before starting nova-network or Neutron iptables
firewall driver. In the latter case, firewall may not work, depending on
the order of operations executed by the driver.
To isolate devstack setups from the difference in distribution
kernel configuration and version, the following steps are done:
- we load bridge kernel module, and br_netfilter if present, to get
access to sysctl knobs controlling the firewall;
- once knobs are available, we unconditionally set them to 1, to make
sure the firewall is in effect.
More details at:
http://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf
[1] I9137ea017624ac92a05f73863b77f9ee4681bbe7
Change-Id: Id6bfd9595f0772a63d1096ef83ebbb6cd630fafd
Related-Bug: #1622914
This variable can be used to accommodate for underlying infrastructure
that does not provide full 1500-sized traffic, or maybe instead gives
access to Jumbo frames.
Change-Id: I38a80bac18673a30842a7b997d0669fed5aff976
Related-Bug: #1603268
This is necessary to make it possible to filter out compute nodes,
which don't support such type of images.
Change-Id: I347953876e2057e6f3dca71c2f5e8b638b85aaf8
