This renames the log files in logs/screen that contain timestamps to put
the timestamp after '.log' and '.log.summary' in the names. This will
simplify devstack-gate's search for log files to copy to '*.log'.
dstat.txt is also renamed to dstat.log
Make LOGDIR and LOGFILE local
bp:devstack-logging-and-service-names
Change-Id: I02aba9ca82c117a1186dafc1d3c07aa04ecd1dde
This makes a bunch of variable cleanups that will let -o nounset
function, for the time being we hide nounset behind another setting
variable so that it's not on by default.
Because this is bash, and things are only executed on demand, this
probably only works in the config it was run in. Expect cleaning up
all the paths to be something that takes quite a while.
This also includes a new set of unit tests around the trueorfalse
function, because my change in how it worked, didn't. Tests are good
m'kay.
Change-Id: I71a896623ea9e1f042a73dc0678ce85acf0dc87d
Slowly trying to introduce more v3 concepts into a generic
devstack installation.
Work with description of none and description with spaces
Change-Id: I7d2fde58363698ff020f92f129f1ff7378f945a8
Fix warning:
DEPRECATION: --download-cache has been deprecated and will be removed in the
future. Pip now automatically uses and configures its cache.
1. Since version 6.0 (2014-12-22) pip has deprecated PIP_DOWNLOAD_CACHE
and now automatically uses and configures its cache.
Default new location is $HOME/.cache/pip.
2. pip gets upgraded to the latest version in tools/install_pip.sh
but if pip version<6, exit with error: "Currently installed pip version
${pip_version} does not meet meet minimum requirements"
Change-Id: I8b203ffc6d9cf588462d0d65a9703a9941d8fa71
Sourcing the tools/install-prereqs.sh script with TOP_DIR set results
in GetDistro being called in get_packages and echoing the result.
Since all output from get_packages is assumed to be package names,
this results in the attempted installation of the non-existant 'Found'
'Distro' and '[distro name]' packages. This change removes the echo
statement to avoid this problem.
Change-Id: Idd05c31b9eec9e6209666fa16fa425cdf1f35aa2
This patch fixes the problem in loading the plugins in devstack
by defining the "plugins" variable in "run_plugins" function of
devstack/functions-common file.
Change-Id: I9fb0e24bf1fd282931a8489e0a8ec0c9ea078520
Closes-Bug: 1408571
This is an initial pass at plugin infrastructure for devstack which
allows specifying an external repository via:
enable_plugin <name> <giturl> [branch]
It implements the devstack specification for this at
I173dee3d57967b1d2ffd30e4868a2832aeac97ce
Change-Id: I8e4175313b3cf0b12e981122358b1288a7eb0746
Also use sudo -H with pip so that it doesn't create a ~stack/.cache
other things can't write to as the stack user later.
Change-Id: I2134c7d8f58f8b83f33150c9ed86d87f8ccba2f3
With multiline support for local.conf, the first line is created with
iniset, which will set *all* previous lines to the same thing, and then
subsequent lines will be added. Modify the multiline support to first
clear existing lines from the section.
This causes fatal errors with neutron.conf, which defines drivers with a bunch
of service_provider= options, and the current code ends up with the first
driver defined in local.conf being present twice.
Change-Id: If132a94e53545d9134859aa508da7b9819ede2f8
Since Fedora 20 it has been possible to use 'dnf' as a drop-in
replacement for 'yum', and it is targetted to become the default
in Fedora 22
http://fedoraproject.org/wiki/Changes/ReplaceYumWithDNF
There are many benefits of 'dnf' over 'yum' but the biggest
from the POV of an openstack developer is its speed.
Assuming an existing running devstack install ie all required
RPMs already installed on the system. Now look at how long it
takes to run stack.sh, during which yum does not have to
actually install anything
# ./unstack.sh
# time ./stack.sh
real 11m12.193s
user 10m17.129s
sys 0m15.275s
Now, with 'export YUM=dnf' set in local.conf, run the same
test again
# ./unstack.sh
# time ./stack.sh
real 0m48.610s
user 0m28.939s
sys 0m7.801s
So, this is showing that devstack is wasting 10 minutes just
for yum to figure out that everything is already installed.
The overhead of yum vs dnf is even worse when yum has to
acutally depsolve to install new packages.
Change-Id: Ia01a5f330a47b32207586902a861bedfc8a0f6e2
With gerrit 2.8, and the new change screen, this will trigger syntax
highlighting in gerrit. Thus making reviewing code a lot nicer.
Change-Id: Id238748417ffab53e02d59413dba66f61e724383
Exploit the --or-show functionality of openstackclient,
rather than issuing two different commands at the shell level,
let the CLI handle things.
Change-Id: I1db239fd3473eb580def1c5fb28ce472a1363569
apts aren't a thing, debs are. apt-get installs debs, like yum installs
rpms. Another option would be to rename rpms to yums, but that is even
sillier, considering then for suse we'd have to call it "zypper-suse" or
something, even though it would want to consume yums-general as a base.
Include a symlink to the old name to help out with grenade. The symlink
should go away later.
This is a long-standing annoyance of mine.
Related-to: I915f0e59c13d8cf5fb3549c6202f8aa2f8be0856
Change-Id: I0416180db5b6add996ce5b48c6966c1b68adbcb0
We're using all the magic variables based on python-fooclient, however
all the inline code was using fooclient for variables. So we had a
mismatch, which was kindly pointed out by some of the 3rd party ci
testers.
Change-Id: I27a56222c7e8e610fba8bf97672d2a42f5cf14ca
The PIP_USE_MIRRORS does not do anything else than adding
the --use-mirrors option to the pip command line.
The --use-mirrors is deprecated since pip-1.5,
and does not do anything else than printing a warning message.
For using alternate pypi index url this option is not required and
not sufficient. The current way for using alternate mirrors
is to defining them in the .pip/pip.conf either manually (before
devstack), or by using the PYPI_ALTERNATIVE_URL and PYPI_OVERRIDE
environment variables.
Change-Id: Ia33e783360e5661c2ef03b77e9f7af32b2633f2f
expand the devstack support for libraries from released versions to
support python-* clients and tempest_lib.
Depends-On: I81b0d228e7769758c61e5b0323ecfce8c8886d39
Change-Id: I26fac0ccf8fd4818e24618d56bf04b32306f88f6
INSTALL_TESTONLY_PACKAGES is assumed to be set to `True` or `False`.
However, in devstack-gate this variable is set to 0 or 1. The patch uses
the already existing `trueorfalse` function to evaluate
INSTALL_TESTONLY_PACKAGES and normalize its value.
Change-Id: I0e4a31e422bad2a31d919d9f871e24833a0faa99
The recent GIT_DEPTH change introduced a [[ ]] construct
which doesn't work for zsh 5.0.2. Workaround it by tweaking the test.
The following is a demonstration to show how zsh behaves:
% if [[ "" ]];then echo hoge;fi
zsh: parse error near `]]'
% if [[ "x" ]];then echo hoge;fi
zsh: parse error near `]]'
% if [[ -n "" ]];then echo hoge;fi
% if [[ -n "x" ]];then echo hoge;fi
hoge
%
Closes-Bug: #1387943
Change-Id: Ia88de876dacb3664a7c3d8f5a035e8e50fddb678
This creates a devstack REQUIREMENTS_MODE which is how we handle
syncing of global requirements. The default is 'strict', which is
current behavior. There is a new 'soft' mode which does a
--soft-update for projects *not* found in projects.txt, which lets
them specify additional requirements.
Change-Id: I4aa606514131b5dde67d87f5c8db5a3f3e50fc03
Depends-On: I1f195ef9ff1509659848e14ec9936ff6f66a6496
Depending on how you are using devstack you probably don't need the
entire history of a project so we should allow people to specify a clone
depth to speed up the devstack process.
Change-Id: I804a5abcc80f6a81e915c0bb4dceae72486441a7
Blueprint: git-depth
Swift has functional tests that check access controls
between users and projects in differing domains. Those tests
are currently skipped by default since swift tests are
configured to use keystone v2 API. In order for those
tests to pass when using keystone v3 API, a user and
project must be setup in a non-default domain.
This patch creates a domain, and a user and project in
that domain, in support of swift functional tests moving
to using keystone v3 API.
Changes:
lib/swift
- create a new domain, project and user for
swift testing
- add new project and user credentials to swift
test config file
- set correct identity service url in swift test
config file according to kesytone API version
functions-common
- add function get_or_create_domain
- modify get_or_create_user and get_or_create_project
functions to optionally specify a domain
Change-Id: I557de01bf196075f2f3adcdf4dd1b43756d8a0ae
screen_service() can currently only be used to launch things that
pass the 'is_service_enabled' check, even though its calling functions
will have already done this. This removes such check, renames it
to screen_process() and updates its usage elsewhere.
Change-Id: I480a4560a45b131a95c1b2d2d2379aeba542a9bc
It would behave such as the contents from each meta-section in
local.conf is copied to the destination files. One exception is the multiline
options not grouped together. In that case, the contents will be grouped
together in its destination config file.
Check tests/test_config.sh for examples.
This was originally committed in https://review.openstack.org/128805.
But the original change used AWK syntax that is not supported in AWK
3.1.8, and caused syntax error on servers with that AWK version. This
patch makes the necessary change so that it's compatible with AWK
3.1.8.
Change-Id: Id1e1fe01f05bd0f19ea6e89c4f4c0f8be695dfce
Partial-Bug: #1374118
Provide support for named proposed branches for selection logic
This syncs the get_release_name_from_branch() function from grenade.
Change-Id: I1adabf07cdc3ea6863cd30d8b6454fb40fc20288
This reverts commit 6ff21acf4c4d4ef08bbef419ba582cade4da8da7.
This commit has broken config options with colons in them.
The following is a sample configuration that no longer works:
[[post-config|/etc/neutron/plugins/ml2/ml2_conf.ini]]
[restproxy]
server_ssl=False
servers=10.211.1.9:80
server_ssl=False
With the above config and the code present that this reverts,
the 'servers' option will come out blank.
Change-Id: I328852d2d941605051a1bf5eaf0f7674191f8c48
It would behave such as the contents from each meta-section in
local.conf is copied to the destination files. One exception is the multiline
options not grouped together. In that case, the contents will be grouped
together in its destination config file.
Check tests/test_config.sh for examples.
Change-Id: I8c046b558eeb98ed221f6f1a59182d4179956ced
Partial-Bug: #1374118
Now that we are on pip 1.5.6 lets drop the workaround to make pip 1.4
work. As this is a development/testing tool requiring a newer pip
shouldn't be an issue. Also stack.sh installs pip by default.
Work around introduced in https://github.com/pypa/pip/issues/709
Change-Id: I0e7aad1d21f4fce4c020ce36685bb56893c66bdc
This patch provides a new path for installing libraries in devstack so
that it's possible to either test with upstream released libraries, or
with git versions of individual libraries.
Libraries are added by name to 3 associative arrays GITREPO,
GITBRANCH, GITDIR. When we get to the library install phase we inspect
LIBS_FROM_GIT and look for libraries by name (i.e. "oslo.config") and
if they exist we'll clone and install those libraries from
git. Otherwise we won't, and just let pip pull them as dependencies
when it needs them.
This patch provides the conversion of the oslo libraries, including
pbr.
Devstack-gate jobs for these libraries will need to change to support
actually forward testing their content.
Change-Id: I6161fa3194dbe8fbc25b6ee0e2fe3cc722a1cea4
run_process will use screen if USE_SCREEN=True (the default),
otherwise it will simply start the requested service. Therefore
wherever screen_it used, run_process can be instead.
Where stop_screen was found it has been replaced with stop_process.
A tail_log function has been added which will tail a logfile in a
screen if USE_SCREEN is True.
lib/template has been updated to reflect the use of the new
functions.
When using sg the quoting in run_process gets very complicated.
To get around this run_process and the functions it calls accepts
an optional third argument. If set it is a group to be used with sg.
Change-Id: Ia3843818014f7c6c7526ef3aa9676bbddb8a85ca
This introduces new run_process() and screen_service() functions and sets the
groundwork to change how DevStack starts services. screen_service() is simply a
direct call to the screen portion of the old screen_it() function and is intended
to run commands that only need to run under screen, such as log file watchers.
run_process() is a replacement for screen_it() (which remains until all of the
services are updated). The usage is similar but requires updates to every current
screen_it() call to remove everything that requires the command to be interpreted
by a shell.
The old run_process() and _run_process() functions are still present as
old_run_process() and _old_run_process() to support the deprecated screen_it()
function. These will all go away in the future once all services have been
confirmed to have been changed over.
There is a similar new set of stop process functions stop_process() and
screen_stop_service(). The old screen_stop() will also remain for the deprecation
period.
As an initial test/demostration this review also includes the changes for
lib/cinder to demonstrate what is required for every service.
I included the scripts I used to test this; tests/fake-service.sh and
tests/run-process.sh are quite rough around the edges and may bite. They should
mature into productive members of the testing ecosystem someday.
Change-Id: I03322bf0208353ebd267811735c66f13a516637b
