Old-style test fails on Ubuntu when python3 enabled, with:
.../devstack/inc/python: line 52: [: 16.04: integer expression expected
Use bash-style test, which doesn't attempt to evaluate the RHS if the
LHS evaluates to false
Change-Id: If18031ab98c9060e5825c3a8d3c647bd3705cd9c
Closes-Bug: #1796174
If a project shows up in zuul's required-projects list, add it
to LIBS_FROM_GIT automatically. This way, when a user specifies
that a job requires a zuul-project, it gets used in testing, but
otherwise, it doesn't (pypi is used instead).
Also add information about what happens behind the scenes for both
LIBS_FROM_GIT and plugin dependencies.
This moves the check performed in check_libs_from_git to
a helper function which is installed for most kinds of
installations. This means that if someone sets LIBS_FROM_GIT to
"foobar", devstack won't error anymore, as nothing is going to
try to install foobar, therefore the check won't run on that.
However, as we move to automated generation of the local config,
that error is not likely to happen. This check was originally
added due to an error in the upper-constraints file (where a
constraint name did not match a package name). This location of
the check would still catch that type of error.
Change-Id: Ifcf3ad008cf42d3d4762cfb3b6c31c93cfeb40db
python-openstacksdk does not match its pip name which is openstacksdk.
So setting python-openstacksdk in LIBS_FROM_GIT leads to devstack
thinking there is a problem.
Put in a workaround for now. It would be better to either:
a) rename python-openstacksdk repo to openstacksdk
b) rename the pip name for openstacksdk back to python-openstacksdk
c) add general support in the various GIT hashes for a pip name
Change-Id: I57cf95763d54ad2060a4ce2af91c3ba18ca04db0
The 'pip list' command prints the "safe name" which converts _'s to
-'s amongst other things; e.g. glance_store becomes
"glance-store 0.21.1.dev22 /opt/stack/glance_store"
Because people may use these more familiar "file system" names in
LIBS_FROM_GIT automatically convert names when checking if libraries
are installed.
Change-Id: I30524f80a341f38dfa794a8f629d859e85a4a448
In commit f0cd9a8b08d92524fc8e2c3f05d08cdebc638e2a we changed to
use column format, but it checks for zero length string and
check_libs_from_git fails.
Change-Id: I97b52b80efb33749647229a55147a08afa112dd2
The old code was strip()ing the version string instead of split()ing the
version string so we always got the first character of the version
string. This worked fine as long as the pip version was single digit but
as soon as it rolls over to '10.stuff' we will compare:
pip version 1 (instead of 10) > 6
Which fails bceause 1 is less than six. Instaed we really do want to
compare 10 > 6 so use split on '.' instead.
Change-Id: Ic7d0c04d7fa77774ab2d70fb9d11f182becec553
The pip list command supports a --format=columns option which outputs
things in space delimited columns. Switch to using that.
Change-Id: I5140a7d83bf567b1c3c67516112eb4c57074fa53
Strip the [<extras>] string from a <package_dir>[<extras>] argument
when looking for the package directory. Explain what the heck is
going on.
Change-Id: I79beb5c3e9e7c35c91cdd0d5a1d91532bebc4b6d
Closes-Bug: #1721638
As described in the change, "pip freeze" has issues with the way
zuulv3 clones repos without a remote. This is an attempt to use "pip
list" to check for local install
Change-Id: I33d25f86b6afcadb4b190a0f6c53311111c64521
lib/nova does a pip re-install of libvirt-python to rebuild the python
library incase the underlying libvirt version changed during package
installs. In offline mode, the underlying version of libvirt can't
have changed; so we have the situation that we've removed the libvirt
python bindings but can't reinstall them (because we're offline).
This fixes that particular situation, but skipping uninstalls in
offline mode seems generically OK.
Change-Id: I2b75d45d94d82f87d996c7570c125d46f5f99f6a
Closes-Bug: #1708369
Some URLs are broken, so fix them.
The others are redirect to new URLs,
so replace them with new ones.
The config options of nova serial console proxy
have been gathered in nova/conf/serial_console.py.
So the description in doc/source/guides/nova.rst
is fixed.
Change-Id: Ifd81cc09969341fbf8f135a913fc6003b94e0acc
To reduce the total number of invocations necessary for pip which isn't
the quickest thing ever (due to needing to evaluate constraints and deps
lists and what is currently installed) combine the main installation of
software with its test-requirements.txt file which should roughly halve
our pip invocations.
Change-Id: Ibcc3264136e66d34a879ad1c90a62e1bb6a84243
'pip uninstall' will hang running stack.sh if it has to
prompt the user for input, use -y.
Change-Id: Ic94639e444b87fd3538463d5a51c01a0208a2ab2
Closes-bug: #1691172
libvirt-python compiles against the currently installed libvirt. If
you upgrade that, it needs to rebuild, however it won't change
versions, so pip install just noops. Force an uninstall / reinstall of
it every time to handle potential upgrades of libvirt.
Change-Id: If34541b34aa6d55eedaf6c603fd1fe92eb887308
When variables use the 'declare' directive, it is by default a local
variable. While other variables have global scope.
For example:
declare -A AN_ARRAY # local in scope
foo=1 # global in scope
This causes errors to occur as some of the variables will be local only
and others will be global.
Update the code, as appropriate, so that variables using the 'declare'
directive also include the '-g' flag to have them also be global. Not
every instance of a declared variable has been updated.
Closes-Bug: #1669509
Co-Authored-By: John L. Villalovos <john.l.villalovos@intel.com>
Change-Id: I2180b68fe861ad19c6d4ec0df0f9f8a528347862
An initial install for devstack-tools, this will need to use all the
fun pip extra variables for installation, however the current
pip_install always prefers python2, and we only want to do python3
here.
Change-Id: I3dcdb35130f76fad81cb7b0d4001b7e96efbbd84
When installing a library from source and python 3 is enabled, first run
the installation process with python 2 enabled to ensure the library is
also installed under python 2 for any services not yet running under
3. The python 3 version is installed second so that command line tools
installed with libraries are installed under python 3 when python 3 is
enabled.
Change-Id: Ibb0f7a68d21081bf7652a0c1515080c0c54888ca
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Add variables ENABLED_PYTHON3_PACKAGES and DISABLED_PYTHON3_PACKAGES to
work like ENABLED_SERVICES and DISABLED_SERVICES and to manage which
packages are installed using Python 3. Move the list of whitelisted
packages in pip_install to the default for ENABLED_PYTHON3_PACKAGES,
except swift which is not enabled by default for now.
Add enable_python3_package and disable_python3_package functions to make
editing the variables from local.conf easier.
Add python3_enabled_for and python3_disabled_for functions to check the
settings against packages being installed by pip.
Update pip_install to check if python3 is disabled for a service, then
see if it is explicitly enabled, and only then fall back to looking at
the classifiers in the packaging metadata.
Update pip_install messages to give more detail about why the choice
between python 2 and 3 is being made for a given package.
Change-Id: I69857d4e11f4767928614a3b637c894bcd03491f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Previous this was set the zake, but that was revert to missing
dependencies issue and because zake is a test fixture and not somthing
to deploy.
This change configures the Cinder dlm with this one is zookeeper.
And it installs tooz and the extra dependencies needed for the
zookeeper driver.
To do it, this commit have to introduce a new method for package
installation: 'pip_install_gr_extras package extra1,extra2'.
Change-Id: Idca310c08e345db59840eb31434c6cb1f849fa70
Really close to getting swift and keystone under uwsgi working, so
let's white list them. Won't affect any existing jobs, so we should
be good.
Change-Id: I51d56d16a5b175bd45dee09edc0b2748d72a5d06
Use trueorfalse to normalize the values for USE_PYTHON3
Install 3.5 instead of 3.4 When USE_PYTHON3 is specified.
Also, since not many packages are classified correctly, fallback
to looking for just "Programming Language :: Python :: 3" and
log a message for the package to highlight the problem.
Also special case some services that are *almost* ready
Depends-On: Id48e1b328230fcdf97ed1cb4b97f4c3f9cf6eb8a
Depends-On: Ib7d9aa0e0b74a936002e0eea0b3af05102b06a62
Change-Id: I243ea4b76f0d5ef57a03b5b0798a05468ee6de9b
In the 25.0.0 release [1] of setuptools during any install
operation the package in not overwritten. If a package is
installed from another requirement via pip and then it is
installed again from git, it is not updated causing
check_libs_from_git to fail.
[1] https://setuptools.readthedocs.io/en/latest/history.html#v25-0-0
Change-Id: Ibaa1d4157816ea649f4452756fbde25951347001
Closes-Bug: #1605998
For client debugging that invokes multiple libs it can be useful
to have all libs directly in git and not listing all of them
in LIBS_FROM_GIT.
TrivialFix
Change-Id: Ie631cc4045231ebbe8177d2d113e47e4bf83f61c
It seems pip distinguishes paths with .. or extra / for constraints.
For example, the following directories are considered different.
/path/to/dir
/path/to//dir
/path/to/dir/subdir/..
This commit tries to normalize the given directory name to avoid
"Could not satisfy constraints for 'xxxx': installation from path
or url cannot be constrained to a version" error due to directory
name mismatch.
Reference: https://github.com/pypa/pip/pull/3582
Closes-Bug: #1542545
Change-Id: Iae9d58c27d3b10bca16e4a471507c4d5c16439a0
Recent pip supports using [extras] to install extra dependencies
from the project setup.cfg. Add support so that projects can take
advantage of it.
For example, if devstack is configured to use ldap, install the
extra ldap dependencies using:
setup_develop $KEYSTONE_DIR ldap
Partial-Bug: 1479962
Change-Id: Ic13d95b99aaa4d3854b2723343e90f5de6b98aa2
Change I855ffe9c7a75e9943106af0f70cf715c34ae25c5 and
I368fec44858bd97fc6a314fb20eed2b10932cbb1 added timing
infrastructure which hides the return value of
the main commands. Restore the prior behaviour.
Change-Id: I4a536abefa7824ac6d42e7bbbf1079d9cc055984
Closes-Bug: #1518545
Add USE_PYTHON3 and PYTHON3_VERSION variables to allow services to use
python 3 if they indicate support in their python package metadata.
Tested in Heat here -> I837c2fba682ab430d50e9f43913f2fed20325a7a.
Project config change to add a dedicated job to Heat is here -> I0837e62d6ccc66397a5e409f0961edd4be31f467
Change-Id: I079e18b58b214bf8362945c253d6d894ca8b1a6b
Ia0957b47187c3dcadd46154b17022c4213781112 proposes to have bashate
find instances of setting a local value. The issue is that "local"
always returns 0, thus hiding any failure in the commands running to
set the variable.
This is an automated replacement of such instances
Depends-On: I676c805e8f0401f75cc5367eee83b3d880cdef81
Change-Id: I9c8912a8fd596535589b207d7fc553b9d951d3fe
Ia0957b47187c3dcadd46154b17022c4213781112 detects setting local
variables with subshell commands.
Although this is a particuarly benign case, it trips the test. Rather
than putting in an ignore for this, we can easily change it to make
the test pass. This seems better than putting in special work-arounds
to bashate, etc.
Change-Id: I37c3967c0f2d780a636a7d26cda83755085c5c69
When moving to Python 3, we also need to install test-requriements
to allow projects to install any python 3 test dependencies they
might be missing otherwise.
Change-Id: I2d19aa2f7ec8de869a82aa7764ab72cc8693101f
after the glance_store vs. upper-constraints bug, it's probably worth
actually enforcing and sanity checking that devstack is doing what
it's being asked of with LIBS_FROM_GIT. This will hopefully reduce
user generated error.
This *might* not work with the current oslo naming, we'll have to test
and normalize that.
Change-Id: Iffef2007f99a0e932b68c4c897ebbfb748cac2b4
Having behavior on your laptop diverge from behavior in the gate is
confusing. Just use constraints on every devstack run to be consistent.
Users of devstack can edit the requirements repo in order to change
these constraints locally if necessary.
Change-Id: I843208e2e982eb04931b76f5cb4bd219fbcd70de
Libraries were resulting in two edit-constraints runs:
- one for the library, which adds a non-editable file path
- then one for the editable servers, but that fails becuse
pkg-resources couldn't parse the prior entry.
This is fixed in two parts: the dependent patch which supports parsing
file urls that have egg names, and this patch which changes from a
file path to a file url with an egg name.
Change-Id: I0f07858e96ea3baf46f8a453e253b9ed29c7f7e2
Constraints files allow a global view of dependencies for devstack
without the side effect that requirements files have of installing
everything everytime. This is part of the cross project
requirements-management spec.
Change-Id: If089d30146629e6cf817edd634e5c2b80f1366dd
Before the fix, requirements soft-update was used for projects that are
in the file.
Change-Id: I095d42521f54b45a6b13837e2f8375fa04532faa
Closes-Bug: #1469067
