An initial install for devstack-tools, this will need to use all the
fun pip extra variables for installation, however the current
pip_install always prefers python2, and we only want to do python3
here.
Change-Id: I3dcdb35130f76fad81cb7b0d4001b7e96efbbd84
When installing a library from source and python 3 is enabled, first run
the installation process with python 2 enabled to ensure the library is
also installed under python 2 for any services not yet running under
3. The python 3 version is installed second so that command line tools
installed with libraries are installed under python 3 when python 3 is
enabled.
Change-Id: Ibb0f7a68d21081bf7652a0c1515080c0c54888ca
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Add variables ENABLED_PYTHON3_PACKAGES and DISABLED_PYTHON3_PACKAGES to
work like ENABLED_SERVICES and DISABLED_SERVICES and to manage which
packages are installed using Python 3. Move the list of whitelisted
packages in pip_install to the default for ENABLED_PYTHON3_PACKAGES,
except swift which is not enabled by default for now.
Add enable_python3_package and disable_python3_package functions to make
editing the variables from local.conf easier.
Add python3_enabled_for and python3_disabled_for functions to check the
settings against packages being installed by pip.
Update pip_install to check if python3 is disabled for a service, then
see if it is explicitly enabled, and only then fall back to looking at
the classifiers in the packaging metadata.
Update pip_install messages to give more detail about why the choice
between python 2 and 3 is being made for a given package.
Change-Id: I69857d4e11f4767928614a3b637c894bcd03491f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Previous this was set the zake, but that was revert to missing
dependencies issue and because zake is a test fixture and not somthing
to deploy.
This change configures the Cinder dlm with this one is zookeeper.
And it installs tooz and the extra dependencies needed for the
zookeeper driver.
To do it, this commit have to introduce a new method for package
installation: 'pip_install_gr_extras package extra1,extra2'.
Change-Id: Idca310c08e345db59840eb31434c6cb1f849fa70
Really close to getting swift and keystone under uwsgi working, so
let's white list them. Won't affect any existing jobs, so we should
be good.
Change-Id: I51d56d16a5b175bd45dee09edc0b2748d72a5d06
Use trueorfalse to normalize the values for USE_PYTHON3
Install 3.5 instead of 3.4 When USE_PYTHON3 is specified.
Also, since not many packages are classified correctly, fallback
to looking for just "Programming Language :: Python :: 3" and
log a message for the package to highlight the problem.
Also special case some services that are *almost* ready
Depends-On: Id48e1b328230fcdf97ed1cb4b97f4c3f9cf6eb8a
Depends-On: Ib7d9aa0e0b74a936002e0eea0b3af05102b06a62
Change-Id: I243ea4b76f0d5ef57a03b5b0798a05468ee6de9b
When using local.conf in multinode envs not everything is going to be
defined in all places. Eventually we probably want to make it so we
have a host role for these sections or something. But for now warn
instead of die when we can't find a config var.
Change-Id: I6959099373f035fbfe9e540a44e4c52b8e7c95c0
Closes-Bug: #2000824
The current coding fails to process local.conf like
the following. Note: This example is taken from a
real use case. [1]
[[post-config|$NEUTRON_CONF]]
[qos]
notification_drivers = midonet
[[post-config|$NEUTRON_CONF]]
[quotas]
# x10 of default quotas (at the time of writing)
quota_network=100
quota_subnet=100
quota_port=500
quota_router=100
quota_floatingip=500
quota_security_group=100
quota_security_group_rule=1000
[1] https://review.openstack.org/#/c/400627/
Closes-Bug: #1583214
Change-Id: Ie571b5fa5a33d9ed09f30ba7c7724b958ce17616
In the 25.0.0 release [1] of setuptools during any install
operation the package in not overwritten. If a package is
installed from another requirement via pip and then it is
installed again from git, it is not updated causing
check_libs_from_git to fail.
[1] https://setuptools.readthedocs.io/en/latest/history.html#v25-0-0
Change-Id: Ibaa1d4157816ea649f4452756fbde25951347001
Closes-Bug: #1605998
This adds a set of local.conf modifying functions which make it easier
for consuming projects like devstack-gate to programatically add
elements to local.conf structured files.
Change-Id: I3427968c2bd43aba12b3619acc27f73c74f0dabb
Co-Authored-By: fumihiko kakuma <kakuma@valinux.co.jp>
For client debugging that invokes multiple libs it can be useful
to have all libs directly in git and not listing all of them
in LIBS_FROM_GIT.
TrivialFix
Change-Id: Ie631cc4045231ebbe8177d2d113e47e4bf83f61c
It seems pip distinguishes paths with .. or extra / for constraints.
For example, the following directories are considered different.
/path/to/dir
/path/to//dir
/path/to/dir/subdir/..
This commit tries to normalize the given directory name to avoid
"Could not satisfy constraints for 'xxxx': installation from path
or url cannot be constrained to a version" error due to directory
name mismatch.
Reference: https://github.com/pypa/pip/pull/3582
Closes-Bug: #1542545
Change-Id: Iae9d58c27d3b10bca16e4a471507c4d5c16439a0
We don't have a new bashate release yet, but this fixes some minor
issues when used with bashate trunk.
The only two things triggered are stricter checking of assignment in
local declartions (catches quotes) and one instance of evaluating
arithmetic result in tools/xen/functions.
Therefore, hopefully, this change has no effect!
Change-Id: I3fd5d55236e90c2f2c25243af87adaf0c1b763e2
Recent pip supports using [extras] to install extra dependencies
from the project setup.cfg. Add support so that projects can take
advantage of it.
For example, if devstack is configured to use ldap, install the
extra ldap dependencies using:
setup_develop $KEYSTONE_DIR ldap
Partial-Bug: 1479962
Change-Id: Ic13d95b99aaa4d3854b2723343e90f5de6b98aa2
Change I855ffe9c7a75e9943106af0f70cf715c34ae25c5 and
I368fec44858bd97fc6a314fb20eed2b10932cbb1 added timing
infrastructure which hides the return value of
the main commands. Restore the prior behaviour.
Change-Id: I4a536abefa7824ac6d42e7bbbf1079d9cc055984
Closes-Bug: #1518545
This function returns list of sections from specified INI file.
It is needed for I7938f26015d953efdf720774b5ad2cee4068ac90.
'iniget_sections' is needed for automatic node enrollment to
Ironic from INI syntax file.
Change-Id: I52a69ad3bc8fc528e7f20e6b03a98f327acaaa74
Implements: blueprint ironic-ml2-integration
XenServer install with devstack doesn't support local.conf, this fix
is to add support for using local.conf and backward-compatibility of
localrc
Change-Id: Ie494e01f8f1ecb8720e14392ef3f12d5a5a01dcd
Closes-Bug: #1528520
Add USE_PYTHON3 and PYTHON3_VERSION variables to allow services to use
python 3 if they indicate support in their python package metadata.
Tested in Heat here -> I837c2fba682ab430d50e9f43913f2fed20325a7a.
Project config change to add a dedicated job to Heat is here -> I0837e62d6ccc66397a5e409f0961edd4be31f467
Change-Id: I079e18b58b214bf8362945c253d6d894ca8b1a6b
There are two bugs in add_sudo_secure_path.
Firstly we don't properly check if the file exists, so always append
the new line. This will overwrite any existing changes.
Secondly the logic for checking if the path exists is inverted, so we
miss adding paths when we should. This particularly causes failures
when installing with virtualenv's since the paths are inside the
virtualenv, rather than the standard system locations.
Change-Id: I646fe0c68958470d464fe4f3d81d5c17dd6f2ab6
Closes-bug: #1521241
I noticed this when debugging some grenade issues failures.
An include of grenade/functions stores the current value of XTRACE
(on) and disables xtrace for the rest of the import.
We then include devstack's "functions" library, which now overwrites
the stored value of XTRACE the current state; i.e. disabled.
When it finishes it restores the prior state (disabled), and then
grenade restores the same value of XTRACE (disabled).
The result is that xtrace is incorrectly disabled until the next time
it just happens to be turned on.
The solution is to name-space the store of the current-value of xtrace
so when we finish sourcing a file, we always restore the tracing value
to what it was when we entered.
Some files had already discovered this. In general there is
inconsistency around the setting of the variable, and a lot of obvious
copy-paste. This brings consistency across all files by using
_XTRACE_* prefixes for the sotre/restore of tracing values.
Change-Id: Iba7739eada5711d9c269cb4127fa712e9f961695
If local.conf specifies a config file addition in the following way...
[[post-config|$MY_CONF_FILE]]
[xyz]
foo=bar
...and $MY_CONF_FILE points to a file whose directory is not writable by
the user running the script, then stack.sh aborts with the following
obscure message:
2015-09-01 08:20:08.113 | touch: setting times of '/': Permission denied
This patch modifies inc/meta-config to provide a useful error message,
such as:
2015-09-01 08:20:08.114 | could not create config file / ($MY_CONF_FILE)
This patch also modifies inc/meta-config so that it provides an error
message if $MY_CONF_FILE is empty (instead of silently ignoring this local.conf
statement):
2015-09-01 09:38:53.406 | bogus config file specification: $MY_CONF_FILE
is undefined
Change-Id: I9b78407420318548561012a8672762bc7fdd6db6
Closes-Bug: 1490881
Ia0957b47187c3dcadd46154b17022c4213781112 proposes to have bashate
find instances of setting a local value. The issue is that "local"
always returns 0, thus hiding any failure in the commands running to
set the variable.
This is an automated replacement of such instances
Depends-On: I676c805e8f0401f75cc5367eee83b3d880cdef81
Change-Id: I9c8912a8fd596535589b207d7fc553b9d951d3fe
Ia0957b47187c3dcadd46154b17022c4213781112 detects setting local
variables with subshell commands.
Although this is a particuarly benign case, it trips the test. Rather
than putting in an ignore for this, we can easily change it to make
the test pass. This seems better than putting in special work-arounds
to bashate, etc.
Change-Id: I37c3967c0f2d780a636a7d26cda83755085c5c69
When moving to Python 3, we also need to install test-requriements
to allow projects to install any python 3 test dependencies they
might be missing otherwise.
Change-Id: I2d19aa2f7ec8de869a82aa7764ab72cc8693101f
after the glance_store vs. upper-constraints bug, it's probably worth
actually enforcing and sanity checking that devstack is doing what
it's being asked of with LIBS_FROM_GIT. This will hopefully reduce
user generated error.
This *might* not work with the current oslo naming, we'll have to test
and normalize that.
Change-Id: Iffef2007f99a0e932b68c4c897ebbfb748cac2b4
Having behavior on your laptop diverge from behavior in the gate is
confusing. Just use constraints on every devstack run to be consistent.
Users of devstack can edit the requirements repo in order to change
these constraints locally if necessary.
Change-Id: I843208e2e982eb04931b76f5cb4bd219fbcd70de
Add a -sudo option to allow these functions to operate on root-owned
files. Test-case is updated, but not enabled by default as we can't
expect test-runner to have sudo access.
Change-Id: I134c3397314c7d9395996eb6c825ecb7e7fdfc69
These return paths doesn't renable xtrace, so output mysteriously goes
missing until the next time it is enabled.
Change-Id: I3a8018dfa9397c07534970c39fba8dc10afcbe41
Libraries were resulting in two edit-constraints runs:
- one for the library, which adds a non-editable file path
- then one for the editable servers, but that fails becuse
pkg-resources couldn't parse the prior entry.
This is fixed in two parts: the dependent patch which supports parsing
file urls that have egg names, and this patch which changes from a
file path to a file url with an egg name.
Change-Id: I0f07858e96ea3baf46f8a453e253b9ed29c7f7e2
Constraints files allow a global view of dependencies for devstack
without the side effect that requirements files have of installing
everything everytime. This is part of the cross project
requirements-management spec.
Change-Id: If089d30146629e6cf817edd634e5c2b80f1366dd
Before the fix, requirements soft-update was used for projects that are
in the file.
Change-Id: I095d42521f54b45a6b13837e2f8375fa04532faa
Closes-Bug: #1469067
We're adding the ability to have devstack plugins, which should be
much more free to require new things not in global requirements. Our
old thinking of locking down all the requirements doesn't really work
in a plugin model.
Instead, if the project is in projects.txt, continue with the old
behavior. If it is not, do a soft update (update all the requirements
we know about, leave the ones we don't). This was previously the SOFT
requirements update mode, but now it will just be the default.
Change-Id: Ic0c6e01a6d7613d712ac9e7e4a378cc3a8ce75e6
