Add a function which tests if a plugin has been enabled with
enable_plugin. This is helpful if two co-ordinating projects want to run
specific setup in devstack in one only if the other is enabled.
Change-Id: Ibf113755595b19d028374cdc1c86e19b5170be4f
This reverts commit 15b0a5f1eb849a30bcea9a1e437e9a88ae2c6f92.
The change had an assumption that a service
is properly configured even when it isn't enabled.
The assumption is not true.
Change-Id: Ib5a8ffe63eaec15bc29bfdd133db7169507bab82
Closes-Bug: #1698129
The OSC number remain high, and it's useful to understand how much
time we spend making OSC calls, especially to surface it relative to
other items. The way we embed this in our code makes it hard to
instrument.
This patch creates a wrapper function for OSC which collects the timings
then aliases `openstack` to that function. This means any invocations of
the openstack utility goes through our function while devstack is
running. Because this is an alias it only affects the stack.sh shell and
any subshells.
This also moves the time tracking infrastructure to count in ms,
instead of s, because some of these operations are close enough to a
second that rounding early is losing way to many significant
digits. We divide by 1000 before reporting to the user.
Change-Id: Ic5f1844ce732d447ee980b3c9fdc417f72482609
In the helper functions to check if roles are set and if not add the
role and return the id we weren't actually checking if the role was set.
The reason for this was we grepped for name values while outputing only
uuid values with OSC. Fix for this is straightforward, we just add the
--role argument to OSC which will filter for us then we don't have to
use a grep on the wrong value type.
Change-Id: I2691b347d2a6273100deb4a1750ab353a8e49673
Commit 5edae54 introduced the usage of systemd in Devstack. This allowed
the transition away from 'screen'. Systemd needs "user unit files" to
describe the services. Currently, those unit files get only created when
an openstack service (n-cpu, c-sch, g-api, ...) is in the list of enabled
services (`ENABLED_SERVICES`). This means, when Devstack is fully stacked,
there is no way to start the systemd unit of an openstack service which
is *not* in that list.
This commit changes that behavior, and creates the systemd unit files
independently of the list ENABLED_SERVICES. This means, when Devstack
is fully stacked, I can start a systemd unit of an openstack service which
wasn't in the ENABLED_SERVICES list. This allows more flexible lifecycle
management of openstack services in the gate, which is useful for tests
which test components which are not in the "default configuration" (e.g.
the "nova-serialproxy" service).
The `clean.sh` script purges all traces of systemd user unit files created
by devstack.
Change-Id: I0f7e1ee8723f4de47cbc56b727182f90a2b32bfb
Now that we aren't using native journal messages by default, the
syslog identifier of all the uwsgi processes is the same. We should be
more explicit with those.
Change-Id: Id5406d02407b022d4016517c2e18890973876d88
There are a couple of issues that have ended up being hit by devstack
plugin authors which we can detect and error in a much nicer way
instead of them having a cryptic systemd failure.
Change-Id: I45e4ac363aeefb4503015f9e1b57c58f79b58f40
When transitioning between different wsgi modes, or service modes, we
should really safely stop and cleanup things that are started in any
service mode, which makes it easier to ensure that we don't leave
things around from past runs.
Change-Id: I33acbee39e1a2da2bfd79a5dd54b84a12a778be1
In order to start making the transition in the gate make
USE_SCREEN=False also mean USE_SYSTEMD=True. We'll never actually
declare USE_SYSTEMD=True in the gate (as that doesn't exist for stable
branches), but this will let us roll over the existing transition.
We also have to install systemd-python 234 because we are recording
exception info in the journal, and all versions before that had a bug
in processing that.
Remove the somewhat pointless screen following journalctl commands. We
really don't want or need those, and they tend to build up over time.
Depends-On: I24513f5cbac2c34cf0130bf812ff2df6ad76657c
Change-Id: I6af6d1857effaf662a9d72bd394864934eacbe70
KEYSTONE_SERVICE_API is the keystone endpoint and it is what we should
use. The admin url should DIAF - but until it does, it CERTAINLY should
not be the thing we put into clouds.yaml.
Change-Id: If8196a04f852f633e0b7548793f68c92376aa6da
It turns out that we ended up with duplicate versions of this function
merging on top of each other within 3 days, and gerrit didn't catch
it. Boo gerrit. Boo bash.
Change-Id: Ic6aa2f9bafdec906de2bc51d5929beeec48a6a40
Things like SERVICE_PROTOCOL and KEYSTONE_AUTH_PROTOCOL shouldn't
really be exported in openrc as they encourage using them directly to
build up keystone urls instead of actually using the OS_AUTH_URL.
Remove them.
Change-Id: I4b7cc680f7f14dae29b706a227be540c9e212cad
uwsgi is a different service type under systemd and shouldn't be run as
a standard oneshot type. The uwsgi docs outline a good pattern for
writing systemd unit files:
http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html
This commit takes those suggestions and creates a separate path for
writing uwsgi unit files.
Change-Id: I9b541b86781afdded311dba058cedd783e1a0dfa
During the PTG there was a discussion that the screen developer
workflow wasn't nearly as useful as it once was. There were now too
many services to see them all on one screen, and one of the most
common service restart scenarios was not restarting one service, but a
bunch to get code to take effect.
This implements a 3rd way of running services instead of direct
forking via bash, or running under screen, which is running as systemd
units.
Logging is adjusted because it's redundant to log datetime in oslo.log
when journald has that.
Swift needed to have services launched by absolute path to work.
This is disabled by default, but with instructions on using it. The
long term intent is to make this the way to run devstack, which would
be the same between both the gate and local use.
Some changes were also needed to run_process to pass the run User
in. A hack around the keystone uwsgi launcher was done at the same
time to remove a run_process feature that only keystone uwsgi uses.
Change-Id: I836bf27c4cfdc449628aa7641fb96a5489d5d4e7
The package libvirt-bin is a transitional package in Debian and should
not be used anymore.
Ubuntu Xenial is an exception here.
Because of that this change also adds the possibility to use "not:" to
exclude distros in files/debs/* just as "dist:" limits distros.
Depends-On: Icc59ea79f54d4ff8751f2e353ee3530fff3d961e
Closes-Bug: #1673840
Change-Id: I3998a7178d14ec40eae5cb199d66da9546cd6ccf
When variables use the 'declare' directive, it is by default a local
variable. While other variables have global scope.
For example:
declare -A AN_ARRAY # local in scope
foo=1 # global in scope
This causes errors to occur as some of the variables will be local only
and others will be global.
Update the code, as appropriate, so that variables using the 'declare'
directive also include the '-g' flag to have them also be global. Not
every instance of a declared variable has been updated.
Closes-Bug: #1669509
Co-Authored-By: John L. Villalovos <john.l.villalovos@intel.com>
Change-Id: I2180b68fe861ad19c6d4ec0df0f9f8a528347862
Nova is going to increase the minimum required libvirt
in Pike to 1.2.9 in change:
I9a972e3fde2e4e552f6fc98350820c07873c3de3
Based on the libvirt distro support matrix wiki [1] that
drops support for Ubuntu Trusty and Debian 7.0/Wheezy.
Trusty has libvirt 1.2.2 and Wheezy has 0.9.12 (the Wheezy
support should have been removed long ago apparently). The
7.0 removed here is for Wheezy also based on commit
b2ef890db3d78b24f9da2f4dd80502165c669ad0.
This does not undo the check for "trusty" with the
EBTABLES_RACE_FIX in lib/nova_plugins/function-libvirt
since you can still force devstack to run on Trusty if
you specify the FORCE=yes variable.
Note that RHEL 7.1 has libvirt 1.2.8 so it won't technically
work with devstack and nova + pike + libvirt, but with the
way os_RELEASE is calculated the minor version is dropped
for RHEL distros so we just get "rhel7".
Also note that this doesn't attempt to continue supporting
Trusty or Wheezy if nova is not configured to use libvirt,
simply in order to start moving forward on devstack distro
support in general and to keep some sanity and closeness
to what we test with in the CI system.
While we're in here, we also drop Fedora 23 and add
Ubuntu Zesty.
[1] https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix
Depends-On: I9a972e3fde2e4e552f6fc98350820c07873c3de3
Depends-On: If69f99bd789e646b0261e27a8a061efde32436f7
Change-Id: I6617283afd798af37e64913b7865cea3c8a62aba
The proliferation of internal/admin endpoints is mostly legacy and
based on some specific deployment patterns. These are not used by
everyone, and for the devstack case aren't really that useful. We
should simplify our service catalog down to the minimum we need for
development.
Change-Id: Ided7a65c81b3a0b56f0184847fc82e17c29a771e
When command failed and component failed to start, original exit code
was overwritten due to original command being executed in background.
This commit adds information about command's exit code to echoed message
about component's start up failure.
Change-Id: I8a3dd485b1b1f2d70d42c5610baac7c0c713f53a
Signed-off-by: Joanna Taryma <joanna.taryma@intel.com>
Services that run inside Apache use tail -f on the corresponding log
file to display output in the screen session. However, they also use sed
to replace some control characters, and this means that the output is
buffered. This results in debugging experiences that range from
"impossible" (the log you want isn't shown) to "Kafkaesque nightmare"
(the log you want isn't shown, except that sometimes it is, and
sometimes it isn't even though you double-checked and you're completely
sure that you must have output a log, but when you check back later you
realise it actually is and you wonder if history is actually not mutable
after all and begin to question what is real and what is not).
This adds the --unbuffered option to ensure streaming output.
Change-Id: I665ff5f047156401d8152f478d834ac40ff31658
So that it can be used by functions like _determine_config_server,
which is used like RESULT=$(_determine_config_server).
Change-Id: Ia4e641c5529b95ada30ae662221f370bc7fa88a7
Since 4b49e409f853104dae021dfca1a9342ec9ac4709 devstack
started to use reload instead of restart.
Using reload in devstack for a fresh install,
does not makes too much sense unless multiple service
plugin touches the same service configs.
Systemd rejects to reload something,
which was not loaded before.
$ sudo /bin/systemctl reload httpd
httpd.service is not active, cannot reload.
We will switch to `reload-or-restart` action instead of `reload`,
it is more likely the action what the previous patch wanted.
Change-Id: I70d597fbe4a8923d937ba8432e29edefb27d1058
Without this parameter, when we set GIT_DEPTH,
it may happen that we clone only master and
then cannot checkout branch
Change-Id: I39376914f8bfc286a308c99db6bc92cddab195b5
Currently devstack assumes that the network used for ssh
validation is the private network. This patch adds a hook that
sets the network used for ssh validation based on whether or not
provider networking is being used. It also moves the function
'is_provider_network' into functions-common as it will now be
used by both tempest and neutron.
Change-Id: I265c9e26c9bfb18b7e201f27d8912b8bec235872
When using zypper remove, include the -y option to avoid stack.sh from
hanging waiting for user confirmation. Due to output buffering, the
script could hang before giving the user the prompt to enter Y to
continue, making it unclear why the script was hanging.
Change-Id: I5ea761e5ae0829439953c385f8e7d0546acba886
Closes-Bug: 1642736
When using the enable_plugin command and grenade jobs it can be
easy to enable the same plugin twice, as the grenade job has a
registration section and the configuration in project-config can also
enable it due to code-reuse in project-config.
If a plugin is enabled twice it will likely fail, though it won't be
obvious that it was due to the plugin being enabled multiple times.
This change makes it so if it sees the same plugin name is enabled
more than once it will die and an error message outputted.
Change-Id: I9f1d7e58b861b04473b6a57c9ad404203fb7277a
urllib3 1.18 was released today and contains new more correct hostname
matching that takes into account the ipAddress portion of a certificate
and disallows matching an IP Address against a DNS hostname.
Change-Id: I37d247b68911dc85f55adec6a7952ed321c1b1d8
TIL:
Similarly, all the END rules are merged, and executed when all the
input is exhausted (or when an exit statement is executed).
i.e. matching YUM_FAILED calls "exit", which falls through to the END
rules which calls "exit result" ... which is zero. i.e. if the return
code is 1 then we actually hide that and return with zero.
This is rather annoying because errors that should halt to alert us of
a package install failure pass through, only for you to have to debug
much later on seemingly unrelated problems.
This always sets "result" and thus should be returning the right
thing. I've updated the documentation to hopefully make it clearer
what's going on.
Change-Id: Ia15b7dc55efb8d3e3e945241b67a468b8a914672
Stud is now abandonware (see https://github.com/bumptech/stud) and is
not packaged in xenial. Lets use Apache for SSL termination since its
there already.
Change-Id: Ifcba410f5969521e8b3d30f02795541c1661f83a
