The trove development environment is typically a linux VM within which
openstack is installed, and trove launches guest vm's within that
environment. To make it possible for these vm's to launch in one human
lifetime, one must set vt/x and enable nested hypervisors to use with
kvm; qemu emulation will take way too long.
The new libvirtd (v2.5.0) in Ubuntu Cloud Archive doesn't handle
nested hypervisors well and if you use it, you end up with a guest
hanging on the GRUB line.
To enable that use-case, we provide ENABLE_UBUNTU_CLOUD_ARCHIVE which
the trove developer can set (to False) before running devstack.
Change-Id: Ia0265c67bb7d2a438575a03c0ddbf2d9c53266ed
Closes-Bug: #1689370
In master branch of noVNC project file vnc_auto.html was renamed to
vnc_lite.html Because of that nova-novncproxy looks for file that
actually doesn't exist.
We need to change branch of noVNC to latest stable, because other
projects are not ready yet to rename the path. Those projects
depends on noVNC package installed in system, but it is too old
for now for both CentOS (version 0.5) and Ubuntu (version 0.4).
The only way to make noVNC console working on Devstack is to change
the branch to stable one.
Unit test also has to be modified in order to ignore novnc repo
from checking against cloning non-master branch.
Change-Id: Iaf4761aedf93bc6b914a6a0c5cf1cfedcc29583c
Closes-bug: #1692513
ETCD_DOWNLOAD_URL is set to github url, in our CI, we can point
ETCD_DOWNLOAD_URL to a url in tarballs.openstack.org possibly
in devstack-gate
Download the etcd binaries and drop them into /opt/stack/bin and
use it from there. Cache the tgz for subsequent use (local workflow)
daemon-reload is called twice once from inside the write_user_unit_file
and then when we adjust the entries with additional things recommended
by the etcd team. We need a better way to do this in the future.
Added a TODO to verify the downloaded artifact later. The etcd team
posts gpg signature, we could verify that or run sha256sum and hard
code that in lib/etcd3 file. We would have to update it whenever we
bump the etcd3 version.
We use the public key "CoreOS Application Signing Key <security@coreos.com>"
with ID FC8A365E to verify the integrity of the downloaded file
Any jobs that need to be run on architectures where v3.1.7 is not available
should rey the v3.2.0-rcX release candidates. We can switch to v3.2.0
when it gets released.
Initial version of this code was borrowed from the dragonflow
repo:
http://git.openstack.org/cgit/openstack/dragonflow/tree/devstack
Change-Id: Ibbb430fb1dbf66942168e0cb52d990ab6a2eb8d7
One of the pending issues with the conversion to systemd was the loss of
log coloring. It turns out that journalctl by default strips out
characters it considers "unprintable" - including the color codes
emitted by the old-style logging. However, journalctl can be made to
print them by adding the `-a` flag.
This change makes devstack's log formatter conf settings include color
codes like the old screen-based setup used to
We also remove stackrc's setting of JOURNALCTL_F, whose usage was
removed via I6af6d1857effaf662a9d72bd394864934eacbe70.
Change-Id: I2401e267913a24d18dae355aa933072dbbdab1d8
This sets our default ulimit NOFILE to 2048, which is double what we
set things like mysql'd max_connections to.
Change-Id: I5126bed1e6b9f8c64db00eae4151ac61e47b1bf8
This removes some remnant heat code and references now that heat is
running in a plugin. Before merging this patch the heat team should
verify they got everything they needed into their heat plugin, as
there were more parts left than I was expecting.
Change-Id: I477e3a6e75591aa8ff836c28f7ef56aa1b5f8727
This moves the developer use case over to systemd, and updates all the
relevant docs to discuss the systemd workflow instead of screen. It
does so by defaulting USE_SCREEN=False, so will not impact people that
set it explicitly.
Change-Id: I6d664612bc2b850eb7f56852afbc841867223ab7
In order to start making the transition in the gate make
USE_SCREEN=False also mean USE_SYSTEMD=True. We'll never actually
declare USE_SYSTEMD=True in the gate (as that doesn't exist for stable
branches), but this will let us roll over the existing transition.
We also have to install systemd-python 234 because we are recording
exception info in the journal, and all versions before that had a bug
in processing that.
Remove the somewhat pointless screen following journalctl commands. We
really don't want or need those, and they tend to build up over time.
Depends-On: I24513f5cbac2c34cf0130bf812ff2df6ad76657c
Change-Id: I6af6d1857effaf662a9d72bd394864934eacbe70
The DEFAULT_IMAGE_NAME variable is used to reference the name of the
default image in glance after it has been uploaded by devstack. It is
used both inside and outside of devstack for that purpose. However, when
configuring tempest there are some tests which also do image uploads and
need a filename for specifying which file they should upload into glance
for testing purposes. Previously we were just using DEFAULT_IMAGE_NAME
for both purposes, but this causes a conflict if the name of the image
we upload into glance does not have a file extension. So instead of
conflating the things this commit differentiates between them and adds a
new DEFAULT_IMAGE_FILE_NAME variable to use for this purpose.
Change-Id: Icf74badcf2093d8c75db538232b10b3ac7b86eb8
This makes keystone use the proxy uwsgi module when running in uwsgi
mode. It also introduces a new stackrc variable which is WSGI_MODE
that we can use to control the conditionals in services that current
work with mod_wsgi.
Also update retry timeouts on proxy pass so that workers don't disable
their connections during polling for initial activity.
Change-Id: I46294fb24e3c23fa19fcfd7d6c9ee8a932354702
tls-proxy is the way we're now doing a standard install using https
between services. There is a lot more work to make services directly
handle https, and having python daemons do that directly is a bit of
an anti pattern. Nothing currently tests this in project-config from
my recent grepping, so in the interest of long term maintenance,
delete it all.
Change-Id: I910df4ceab6f24f3d9c484e0433c93b06f17d6e1
Allow cursive to be installed from git instead of pip.
The barbican-tempest-plugin, which uses cursive indirectly
through nova and glance, would benefit from the ability to
use cursive from git instead of pip.
Change-Id: Icae7d310f1ee392d080e7c8e421a26d7c0ef4727
Allow castellan to be installed from git instead of pip.
Castellan has recently been moved under the oslo framework,
and the barbican-tempest-plugin tests which use castellan
would benefit from the ability to usd castellan from git
instead of pip.
Change-Id: I96edca90c61aec84637b7b1ce842eff04c521923
We have had issues with libvirt 1.3.1 which is stock on Xenial. Try
using 2.5.0 from UCA instead.
Related-Bug: 1643911
Related-Bug: 1646779
Related-Bug: 1638982
Change-Id: Ia4434541c71f050fe1ffb54f4c4c1e302391d00b
This is required to install python-openstacksdk from source for testing
other projects against master.
Change-Id: Iee7b043ac7d381dadf89d26098f69e935ed81d6b
In an effort to reduce memory consumption enable KSM by default. The
biggest win here is when using libvirt with nova or ironic with its
fake baremetal instances. In theory any process that runs duplicates
with mergeable memory will benefit though.
Change-Id: I4c5addfd3e83b6516023b36cbaabd5169f0d5ceb
During the PTG there was a discussion that the screen developer
workflow wasn't nearly as useful as it once was. There were now too
many services to see them all on one screen, and one of the most
common service restart scenarios was not restarting one service, but a
bunch to get code to take effect.
This implements a 3rd way of running services instead of direct
forking via bash, or running under screen, which is running as systemd
units.
Logging is adjusted because it's redundant to log datetime in oslo.log
when journald has that.
Swift needed to have services launched by absolute path to work.
This is disabled by default, but with instructions on using it. The
long term intent is to make this the way to run devstack, which would
be the same between both the gate and local use.
Some changes were also needed to run_process to pass the run User
in. A hack around the keystone uwsgi launcher was done at the same
time to remove a run_process feature that only keystone uwsgi uses.
Change-Id: I836bf27c4cfdc449628aa7641fb96a5489d5d4e7
Now that we don't support Ubuntu Trusty anymore, we can remove
the ebtables race workaround.
Closes-Bug: #1675714
Change-Id: I70483f871e35fcaa933d1b7bac7dbb396aa22cef
When variables use the 'declare' directive, it is by default a local
variable. While other variables have global scope.
For example:
declare -A AN_ARRAY # local in scope
foo=1 # global in scope
This causes errors to occur as some of the variables will be local only
and others will be global.
Update the code, as appropriate, so that variables using the 'declare'
directive also include the '-g' flag to have them also be global. Not
every instance of a declared variable has been updated.
Closes-Bug: #1669509
Co-Authored-By: John L. Villalovos <john.l.villalovos@intel.com>
Change-Id: I2180b68fe861ad19c6d4ec0df0f9f8a528347862
openvz is not in the nova tree, and is referencing a crazy old image,
we're going to assume that if anyone is using this they can build a
devstack plugin.
drop doing anything by default because this actually requires that we
special case things like ironic in tree to *not* do anything by
default.
Change-Id: I9d33b98263c3d52a95b9983e90eb0b341fa1d363
Add python-openstackclient to the list of packages installed under
Python 3 by default, so that jobs running with Python 3 exercise the
client that way.
Change-Id: I9778a6810bb3e4850132cfc19e583d50fed23ef5
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This commit switches devstack to use the published qcow2 cirros image
instead of the AMI version. Using AMI was mostly a historical artifact
dating pretty far back, but in the real world no one really uses AMI
images with openstack clouds. This change reflects that and also
enables tempest ro remove its deprecated config options for using AMI
as a fallback on misconfiguration (which was just there to support
devstack's defaults)
Change-Id: Id65ebae73b28da7185cb349b714b659af51ef77f
This new version of CirrOS is built on top of 0.3 branch, so wrt
version 0.3.4 it includes only two commits:
- Cherry-pick of the fix for https://launchpad.net/bugs/1564948
- One extra fix to make the above working on 0.3
Cherry-picked commit is http://bazaar.launchpad.net/~cirros-dev/cirros/trunk/revision/366
A Tempest test for hard reboot in some cases hits the case where
host key are empty. This triggers bugs/1564948, i.e. the ssh
daemon does not start at all, and the Tempest test fails with
"connection refused", which is misleading.
The new version of CirrOS solves this problem as it ensure host
keys are generated if missing, and the sshd deamon started.
I tested the scenario of missing host keys in Iea74c63925be17a1df894c1a2c23f5ba2793e0c6
using a private build of what then became 0.3.5.
Change-Id: I5c154ec25555e768954538fc22b4f5d5975b2deb
Add variables ENABLED_PYTHON3_PACKAGES and DISABLED_PYTHON3_PACKAGES to
work like ENABLED_SERVICES and DISABLED_SERVICES and to manage which
packages are installed using Python 3. Move the list of whitelisted
packages in pip_install to the default for ENABLED_PYTHON3_PACKAGES,
except swift which is not enabled by default for now.
Add enable_python3_package and disable_python3_package functions to make
editing the variables from local.conf easier.
Add python3_enabled_for and python3_disabled_for functions to check the
settings against packages being installed by pip.
Update pip_install to check if python3 is disabled for a service, then
see if it is explicitly enabled, and only then fall back to looking at
the classifiers in the packaging metadata.
Update pip_install messages to give more detail about why the choice
between python 2 and 3 is being made for a given package.
Change-Id: I69857d4e11f4767928614a3b637c894bcd03491f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
The logic to set the default image for Ironic has been moved into the
Ironic tree. This patch is just removing it from DevStack.
Change-Id: Iaeb177f194adc83e40d86696e5553f9f72bbd1f9
Depends-On: Id828b41dc44113ce1cd094ce5fc245989699d4ff
According to the feedback in the TC meeting [1], we renamed the Nova
virt driver from "docker" to "zun" [2] to avoid name collision
to nova-docker. This rename also help to clarify the difference
between these two drivers.
[1] http://eavesdrop.openstack.org/meetings/tc/2016/
tc.2016-11-29-20.01.log.html
[2] https://review.openstack.org/#/c/414651/
Change-Id: I747080953ae4d1d35ed334831100413b6e4466c4
Use trueorfalse to normalize the values for USE_PYTHON3
Install 3.5 instead of 3.4 When USE_PYTHON3 is specified.
Also, since not many packages are classified correctly, fallback
to looking for just "Programming Language :: Python :: 3" and
log a message for the package to highlight the problem.
Also special case some services that are *almost* ready
Depends-On: Id48e1b328230fcdf97ed1cb4b97f4c3f9cf6eb8a
Depends-On: Ib7d9aa0e0b74a936002e0eea0b3af05102b06a62
Change-Id: I243ea4b76f0d5ef57a03b5b0798a05468ee6de9b
Now that the placement service is mandatory for running Nova in Ocata,
we want to enable it by default when running devstack by default.
In the past, we added a placement-client service with
I04a655fbc58913b3d607400a7f677be299499142
Devstack-gate will also be able to run a multinode devstack with the
help of Ibd760c642e3c1ffff2dd61be48e30530b0d24720
Change-Id: I273c3c8299ee329bed425f3e7cd4b583ed1187a4
If 'RECREATE_KEYSTONE_DB=False' database won't be recreated.
It would be useful for multinode Grenade tests for Keystone. This parameter
will help us to deploy multiple services on different machines talking to
the same DB.
Devstack recreates Keystone DB each time during Keystone service installation.
If our controller node is already deployed, Keystone DB already contains
important information about OpenStack services and their endpoints. When
the second Keystone node is being deployed, we don't want to delete
records about controllers' services endpoints.
Partially-Implements: bp rolling-upgrade-testing
Change-Id: Ia8d07b4295ca165be01e44466c95d5275f596e83
No need to explain how to enable Neutron now that it's enabled by
default. Keep but reformat the 'how to enable swift' part though.
Change-Id: I3f9b7796fad10abf1039e4c68eb2cd5ef6cdbc99
The switch to using subnetpools caused quite a bit of confusion
because it didn't respect the value of FIXED_RANGE. This caused
conflicts in the gate with it's default IPv4 value of 10.0.0.0/8.
This patch does a few things to address the issue:
* It introduces the IPV4_ADDRS_SAFE_TO_USE and IPV6_ADDRS_SAFE_TO_USE
values and adjusts all of the FIXED_RANGE and SUBNETPOOL_PREFIX values
to dervive from them by default.
* This addresses the concern that was raised about implying that
SUBNETPOOL_PREFIX and FIXED_RANGE are equivalent when setting
SUBNETPOOL_PREFIX=FIXED_RANGE by default. Now we have a new value
for the operator specify a chunk of addresses that are safe to
use for private networks without implementation implications.
* Backwards compatibility is maintained by alloing users to override
override all of these values.
* The default for IPV4_ADDRS_SAFE_TO_USE uses /22 instead of /24
* Because we want to be able to use subnetpools for auto allocated
topologies and we want to be able to have a large chunk of
instances on each network, we needed a little more breathing room
in the default v4 network size.
* SUBNET_POOL_SIZE_V4 default is changed from 24 to 26
* In conjuction with this change and the one above, the default
subnetpool will support up to 16 64-address allocations.
* This should be enough to cover any regular gate scenarios.
* If someone wants a bigger/smaller subnet, they can ask for that
in the API request, change this value themselves, or use a different
network entirely.
* FIXED_RANGE_V6 defaults to a max prefix of /64 from IPV6_ADDRS_SAFE_TO_USE
* This avoids the private subnet in the non-subnetpool case from being
larger than /64 to avoid issues identified in rfc 7421.
* Users can still explicitly set this value to whatever they want.
This 'max' behavior is only for the default.
* This allows IPV6_ADDRS_SAFE_TO_USE to default to a /56, which leaves
tons of room for v6 subnetpools.
Closes-Bug: #1629133
Change-Id: I7b32804d47bec743c0b13e434e6a7958728896ea
This removes all of the heat code from the devstack tree, in favor of the
devstack plugin in Heat's tree.
Depends-On: I4bed1e5cef5afa7b049b07640086a86a3f881e13
Depends-On: Ic392bcc24bc374ee8511a94f1d8f6ac23131c7e3
Change-Id: I5b60422bf1f5fa78aa8f3383f7a222e0356d9e42
This patch sets Keystone v3 as default in services
configuration files and in the openrc and stackrc scripts.
Change-Id: I24546f02067ea23d088d383b85e3a78d7b43f166
Partially-Implements: bp keystonev3
Sets the port_security feature flag in tempest.conf
if the port_security extension is enabled, which it's not
by default in neutron but is set by default in devstack.
This adds global variable for setting the port_security
extension in ml2.conf and in tempest.conf so we only have
to set this in one place.
Depends-On: I1efd5c838aa0d73cc6e8864e3041eea25850198d
Change-Id: I6334b200e42edd785f74cfb41520627393039619
Related-Bug: #1624082
