isset function was moved to config file related functions by accident,
this change also simplfies the isset in a bash >=4.2 way.
All supported distro has at least bash 4.2. (RHEL6 used 4.1)
Change-Id: Id644b46ff9cdbe18cde46e96aa72764e1c8653ac
* config/INI functions from functions-common to to inc/ini-config
* local.conf meta-config functions from lib/config to inc/meta-config
Change-Id: I00fab724075a693529273878875cfd292d00b18a
Ensure that the debtcollector library is pulled in
like the other oslo libraries so that devstack can
use it where appropriate.
Also fixes 'test_libs_from_pypi.sh' to not have a huge
single line of libraries; and splits it into multiple
lines so the diffs and code can be easily looked at.
Change-Id: I35ab0ed0e20b6092a41ecb3b6f1aaf0a05f5180e
As per the graduation work items, any new libraries should be
added to lib/oslo and stackrc
partially implements bp graduate-policy
Change-Id: Ief8f28715ecff5a602d6d840d736ea07b5e7ff39
The HOST_IP check in stackrc is useful for real users, but some cloud
nodes that infra runs don't detect it correctly, and it's not actually
required for unit tests to work right.
Change-Id: Id2aaa713aea91825021e9d8d49d19f3e40a8e6c9
This makes a bunch of variable cleanups that will let -o nounset
function, for the time being we hide nounset behind another setting
variable so that it's not on by default.
Because this is bash, and things are only executed on demand, this
probably only works in the config it was run in. Expect cleaning up
all the paths to be something that takes quite a while.
This also includes a new set of unit tests around the trueorfalse
function, because my change in how it worked, didn't. Tests are good
m'kay.
Change-Id: I71a896623ea9e1f042a73dc0678ce85acf0dc87d
With multiline support for local.conf, the first line is created with
iniset, which will set *all* previous lines to the same thing, and then
subsequent lines will be added. Modify the multiline support to first
clear existing lines from the section.
This causes fatal errors with neutron.conf, which defines drivers with a bunch
of service_provider= options, and the current code ends up with the first
driver defined in local.conf being present twice.
Change-Id: If132a94e53545d9134859aa508da7b9819ede2f8
This adds unit tests for all the GIT* definitions, ensuring that for
libraries we think should be defined, they are. It exposed a bug in
glance_store definitions in the process.
The GITDIR definition for python-openstackclient is moved to stackrc
for testability.
Change-Id: Ibd9ab665f0362a84f4f7e80e80da56a4737f584e
run_tests.sh was created as an entry point for bashate. We stopped
using it now that we know how to run bashate directly in the gate.
We lost running an unrelated bashate test when that happened.
Devstack does have unit tests. We don't run them. We should.
Especially when working on things like the LIBS_FROM_GIT which is
sufficiently tricky that we really need to add unit tests to ensure
that we don't break it.
Change-Id: Ife067855569e2eae4c085471d326e8086de37332
This reverts commit e2c9fee8ed846aba124a2fc1bba245790ed7ba90.
We have decided that we don't want to support the json-style argument
as described by bug#1374118 (see thread at [1]).
This restores the old behavior of sending the argument in
double-quotes so environment variables get expanded. As a bonus,
tests for this are added.
[1] http://lists.openstack.org/pipermail/openstack-dev/2014-October/049341.html
Change-Id: I9fc99f3716cc53366907878adb00ae6cf3898f14
Closes-Bug:#1386413
It would behave such as the contents from each meta-section in
local.conf is copied to the destination files. One exception is the multiline
options not grouped together. In that case, the contents will be grouped
together in its destination config file.
Check tests/test_config.sh for examples.
This was originally committed in https://review.openstack.org/128805.
But the original change used AWK syntax that is not supported in AWK
3.1.8, and caused syntax error on servers with that AWK version. This
patch makes the necessary change so that it's compatible with AWK
3.1.8.
Change-Id: Id1e1fe01f05bd0f19ea6e89c4f4c0f8be695dfce
Partial-Bug: #1374118
This reverts commit 6ff21acf4c4d4ef08bbef419ba582cade4da8da7.
This commit has broken config options with colons in them.
The following is a sample configuration that no longer works:
[[post-config|/etc/neutron/plugins/ml2/ml2_conf.ini]]
[restproxy]
server_ssl=False
servers=10.211.1.9:80
server_ssl=False
With the above config and the code present that this reverts,
the 'servers' option will come out blank.
Change-Id: I328852d2d941605051a1bf5eaf0f7674191f8c48
It would behave such as the contents from each meta-section in
local.conf is copied to the destination files. One exception is the multiline
options not grouped together. In that case, the contents will be grouped
together in its destination config file.
Check tests/test_config.sh for examples.
Change-Id: I8c046b558eeb98ed221f6f1a59182d4179956ced
Partial-Bug: #1374118
When attempting to add a libvirt section with a volume_drivers entry
to $NOVA_CONF, via a post-config block in the local.conf file, I
encountered problems; the value for this attribute takes the form
driver=python.import.path.to.driver
but the value actually populated in the $NOVA_CONF was truncated at the
equals.
Taking the iscsi driver setting specified in the official nova.conf
documentation as an example, if I have the following in my local.conf
file:
[[post-config|$NOVA_CONF]]
[libvirt]
volume_drivers = iscsi=nova.virt.libvirt.volume.LibvirtISCSIVolumeDriver
I will see that the generated $NOVA_CONF has the following:
[libvirt]
volume_driver = iscsi
This occurs because the existing handling for a post-config setion, as
implemented in merge_config_file(), splits the line on the equals sign,
and then uses the first and seconds elements of the resulting array as
attribute name and value respectively.
However when an equals occurs as part of the value this results in the
value being truncated at the first equals in the value.
The fix I've implemented, based upon review feedback, extracts the
contents of $0 before the first equals as the attr name, and extracts
the remainder after the equals as the value. Then it strips the leading
and trailing whitespaces from both as appropriate.
I've also added test5 to tests/test_config.sh to test for, and verify,
correct operation when this scenario is encountered. Similarly I've
added test6 to ensure that trailing spaces in values are stripped
correctly.
Change-Id: Id0cb1e6e1cece21bc5dbf427c4d756af86fbd927
Closes-Bug: #1374482
If we don't single quote the extracted argument to iniset we drop any
quotes from the source. Add a simple test-case for this.
Partial-bug: #1374118
Change-Id: If2f47b64b11015e727a011c7e5f6e8ad378b90eb
run_process will use screen if USE_SCREEN=True (the default),
otherwise it will simply start the requested service. Therefore
wherever screen_it used, run_process can be instead.
Where stop_screen was found it has been replaced with stop_process.
A tail_log function has been added which will tail a logfile in a
screen if USE_SCREEN is True.
lib/template has been updated to reflect the use of the new
functions.
When using sg the quoting in run_process gets very complicated.
To get around this run_process and the functions it calls accepts
an optional third argument. If set it is a group to be used with sg.
Change-Id: Ia3843818014f7c6c7526ef3aa9676bbddb8a85ca
This introduces new run_process() and screen_service() functions and sets the
groundwork to change how DevStack starts services. screen_service() is simply a
direct call to the screen portion of the old screen_it() function and is intended
to run commands that only need to run under screen, such as log file watchers.
run_process() is a replacement for screen_it() (which remains until all of the
services are updated). The usage is similar but requires updates to every current
screen_it() call to remove everything that requires the command to be interpreted
by a shell.
The old run_process() and _run_process() functions are still present as
old_run_process() and _old_run_process() to support the deprecated screen_it()
function. These will all go away in the future once all services have been
confirmed to have been changed over.
There is a similar new set of stop process functions stop_process() and
screen_stop_service(). The old screen_stop() will also remain for the deprecation
period.
As an initial test/demostration this review also includes the changes for
lib/cinder to demonstrate what is required for every service.
I included the scripts I used to test this; tests/fake-service.sh and
tests/run-process.sh are quite rough around the edges and may bite. They should
mature into productive members of the testing ecosystem someday.
Change-Id: I03322bf0208353ebd267811735c66f13a516637b
Since merge_config_file() tries to create an unsubstituted config file,
stack.sh fails due to permission denied in the file creation when you
use '[[post-config|/$Q_PLUGIN_CONF_FILE]]' and the file does not exist.
This patch deletes unnecessary 'touch' command, because the file will
be made by 'iniset' function in the next command line with evaled
string from 'configfile'.
This patch also fixes merge_config_group() to use evaled 'configfile'
when it checks the directory of the config file exists.
Change-Id: I608847a34143b5c6a1708c180186dd88a32dd44b
Closes-bug: #1294213
Check that function calls look like ^function foo {$ in bash8, and fix
all existing failures of that check. Add a note to HACKING.rst
Change-Id: Ic19eecb39e0b20273d1bcd551a42fe400d54e938
we mostly have a consistent style on if/then & for/do in devstack,
except when we don't. This attempts to build a set of rules to
enforce this.
Because there are times when lines are legitimately long, and there
is a continuation, this starts off ignoring if and for loops with
continuations. But for short versions, we should enforce this.
Changes to make devstack pass are included. The fact that the
cleanup patch was so small is pretty solid reason that this is
actually the style we've all agreed to.
Part of a git stash from hong kong that I finally cleaned up.
Change-Id: I6376d7afd59cc5ebba9ed69e5ee784a3d5934a10
iniset did not handle the case of "|" in the value
to be injected. Fix this by replacing | with \000 (NULL).
Fixes bug #1258050
Change-Id: I8882c2f3f177ebdfa0c66270dbbc7fd50f30b065
* iniset() bails if no section or option (attribute) is supplied
* merge_config_file() properly skips lines with only whitespace
* Also split the ini-tests into their own script
Bug 1257954
Change-Id: Ie31c5bd0df8dfed129fbcf1e37228aaf25e9305d
This defines a new local.conf file that is designed to take the place of all
of the 'pass-through'[1] configuration options that have been defined in DevStack.
* new local.conf file can contain multiple config file settings to be
merged in to existing project config files
* localrc can be embedded into local.conf and will auto-extract if
localrc does not exist
* Adds functions get_meta_section(), get_meta_section_files(),
merge_config_file() and merge_config_group()
* Adds EXTRA_OPTS, EXTRA_BAREMETAL_OPTS, Q_DHCP_EXTRA_DEFAULT_OPTS and
Q_SRV_EXTRA_DEFAULT_OPTS to the deprecated warning list at the end of stack.sh
[1] Pass-through options are those that do not configure or change DevStack's behaviour
but simply set a value in a project config file. This includes most of the EXTRA_XXX_OPTS
configuration variables.
Change-Id: I367cadc86116621e9574ac203aafdab483d810d3
is_package_installed() incorrectly returned '0' for packages that
had 'un' status in the dpkg database.
Change-Id: I81b77486c2ed7717ed81cb2c2572fe6c4b394ffc
Remove python-netaddr as a DevStack (stack.sh) requirement,
this does not change any individual project requirements.
Specifically it replaces address_in_net() and adds cidr2netmask()
and maskip() functions.
Change-Id: Ic604437fde2e057faced40a310ab282f3eb27726
Fixed bug #1136964.
1. Added 3 ini functions to support MultiStrOpt:
Function "iniset_multiline config-file section option value1 value2
value3 ..." sets a MultiStrOpt option in an ini file.
Function "iniget_multiline config-file section option" gets the
MultiStrOpt option values.
Function "iniadd config-file section option value1 value2 value3..."
appends an option without relacing the old values, which would result
the option to be MultiStrOpt.
2. Modified the nova configuation to correctly enable notification for
ceilometer.
Change-Id: I1c27db1a6e58b35bc4428e761f40627988f69e37
* In python the white spaces are part of the section name
* Handle options with empty value
* Support paths with white spaces
Change-Id: I69a584608853cfdb8b7dce1e24d929216ef2fc41
Between is_fedora, is_ubuntu and is_suse, we can make the code a bit
simpler to read. We also use exit_distro_not_supported to identify
places where we need implementation details for new distros.
As "/sbin/service --skip-redirect" is Fedora-specific, guard this with a
is_fedora test too.
Change-Id: Ic77c0697ed9be0dbb5df8e73da93463e76025f0c
This allows section names to look like:
[ default ]
OpenSSL is the primary offender for this usage.
Change-Id: If5c711107e73cebab9d4a26ca02a7ce572224377
This helps reduce the distro-dependent code in stack.sh, and also fixes
the bug where "rpm -qa | grep sudo" will work if gnome-sudoku is
installed.
Rebased
Change-Id: Ib1330b29b915b41d9724197edd791f0d4e0fe373
Editing ENABLED_SERVICES directly can get tricky when
the user wants to disable something. This patch includes
two new functions for adding or removing services
safely, and a third (for completeness) to clear the
settings entirely before adding a minimal set of
services.
It also moves the logic for dealing with "negated"
services into a function so it can be tested and
applied by the new functions for manipulating
ENABLED_SERVICES.
Change-Id: I88f205f3666b86e6f0b6a94e0ec32a26c4bc6873
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
* Put all config files in /etc/keystone
* keystone.conf rewritten
* logging.conf.sample rewritten to logging.conf
* default_catalog.templates copied from devstack/files
* iniset() now properly adds options that do not previously exist
Fixed to re-configure the catalog templated backend; sql is the
default in trunk now but DevStack needs a bit more work before
it can use it.
Change-Id: Ic7060ef897e47495cd08ca3786e49fdebadf6723
* Replace die_if_error() with the simpler die()
* Attempt to clean up unnecessary trace output
* Formatting cleanups on all exercise scripts
Change-Id: I72a542b3a59ee9bf12bee6bcc605edd7579205e0
* Test returns and exit codes on most command invocations
* Add start and end banners to make output easier to find in
long log files
* Adds die_if_error(), die_if_not_set() and is_set() to functions
* Add some function tests
Fixes bug 944593
Change-Id: I55e2962c5fec9aad237b674732b1e922ad37a62e
