As a new feature, os-brick now supports setting the location of file
locks in a different location from the locks of the service.
The functionality is intended for HCI deployments and hosts that are
running Cinder and Glance using Cinder backend. In those scenarios the
service can use a service specific location for its file locks while
only sharing the location of os-brick with the other services.
To leverage this functionality the new os-brick code is needed and
method ``os_brick.setup`` needs to be called once the service
configuration options have been loaded.
The default value of the os-brick ``lock_path`` is the one set in
This patch adds support for this new feature in a non backward
compatible way, so it requires an os-brick version bump in the
The patch also ensures that ``tox -egenconfig`` includes the os-brick
configuration options when generating the sample config.
Port the legacy legacy-tempest-dsvm-multibackend-matrix job to the
native Zuul v3 syntax, and rename it following the guidelines
This job tests the migration between two different backends
specified through the volume.backend_names configuration
key in tempest.conf.
Now the job leverages the existing zuul code, namely the
run-tempest role, which is called multiple times with all
the possible combinations of the 3 tested backends
(lvm, ceph, nfs) where the source and the destination differ.
The final JUnitXML output summarizes the test results
for each of the tested combinations.
The cinder-mypy job is failing because library stubs aren't installed
for requests . Modify the mypywrap.sh to accept options specified
in an environment variable named OS_MYPY_OPTS to the mypy invocation,
and set this var in tox.ini.
The value is "--install-types --non-interactive" which is suitable for
CI purposes, and seems to make sense for local tox use as well .
The downside is it basically runs mypy twice, once to determine
whether there are any library stubs missing and then install them, and
againto do the actual check. If we don't want this setting in
tox.ini, we can move it to .zuul.yaml for the cinder-mypy job run.
Also, update the version of mypy in test-requirements to a version
that supports the above options.
And, run mypy in its own env (instead of reusing pep8) so that the
tox logs are preserved during CI runs.
Tox issues a warning because bash is not
in whitelist_externals and indicates that this
will fail in tox 4+.
Just call mypywrap.sh directly to prevent this.
Add a "mypy" tox environment which runs mypy
type checking against Cinder code.
Taken from Stephen Finucane's Nova work at
Added "show_error_codes" and "pretty" options.
Generates an html report in ./mypy-report/
This adds stubs for oslo.i18n, so that _() calls
are annotated as intended. It may be possible to
do this with less .pyi files carried along here.
Starting from mysql version 8 it's not possible to create a user
implictly when using GRANT.
This patch makes the behavior compatible with that.
Signed-off-by: Sean McGinnis <firstname.lastname@example.org>
There's some remainings of Python 2 in a few .py that this patch
intends to fix. This patch has been in Debian for the last 2
years, it's probably about time to get this in upstream Cinder too.
This adds usage of the flake8-import-order extension to our flake8
checks to enforce consistency on our import ordering to follow the
overall OpenStack code guidelines.
Since we have now dropped Python 2, this also cleans up a few cases for
things that were third party libs but became part of the standard
library such as mock, which is now a standard part of unittest.
Some questions, in order of importance:
Q: Are you insane?
Q: Why should we touch all of these files?
A: This adds consistency to our imports. The extension makes sure that
all imports follow our published guidelines of having imports ordered
by standard lib, third party, and local. This will be a one time
churn, then we can ensure consistency over time.
Q: Why bother. this doesn't really matter?
A: I agree - but...
We have the issue that we have less people actively involved and less
time to perform thorough code reviews. This will make it objective and
automated to catch these kinds of issues.
But part of this, even though it maybe seems a little annoying, is for
making it easier for contributors. Right now, we may or may not notice
if something is following the guidelines or not. And we may or may not
comment in a review to ask for a contributor to make adjustments to
follow the guidelines.
But then further along into the review process, someone decides to be
thorough, and after the contributor feels like they've had to deal with
other change requests and things are in really good shape, they get a -1
on something mostly meaningless as far as the functionality of their
code. It can be a frustrating and disheartening thing.
I believe this actually helps avoid that by making it an objective thing
that they find out right away up front - either the code is following
the guidelines and everything is happy, or it's not and running local
jobs or the pep8 CI job will let them know right away and they can fix
it. No guessing on whether or not someone is going to take a stand on
following the guidelines or not.
This will also make it easier on the code reviewers. The more we can
automate, the more time we can spend in code reviews making sure the
logic of the change is correct and less time looking at trivial coding
and style things.
Q: Should we use our hacking extensions for this?
A: Hacking has had to keep back linter requirements for a long time now.
Current versions of the linters actually don't work with the way
we've been hooking into them for our hacking checks. We will likely
need to do away with those at some point so we can move on to the
current linter releases. This will help ensure we have something in
place when that time comes to make sure some checks are automated.
Q: Didn't you spend more time on this than the benefit we'll get from
A: Yeah, probably.
Signed-off-by: Sean McGinnis <email@example.com>
This allows "tox -e pep8" to succeed on
This is only called from the pep8 env which
already specifies a basepython of python3, so
it should be ok.
Adds a doc about configuring and troubleshooting a service user
to send service tokens.
Also adds the config necessary for successful use of a service token
to the sample configuration file. Only adds keystone v3 options
because v3 is required for service tokens. Lists the service user
options before the session conf opts because the former are required
while the latter are less likely to be changed.
On el7 w/git 1.8.3, "tox -e pylint" just returns:
fatal: ambiguous argument '*.py': unknown revision
or path not in the working tree.
Before this change.
"-j 0" runs on the number of CPUs available, just
pass that instead of counting CPUs ourselves.
(This removes a stealthy python2 dependency for us,
- colorizer.py: stestr (and ostestr) can be used instead,
as they include its code;
- enable-pre-commit-hook.sh: no references in the documentation,
it looks like a relic;
- with_venv.sh: its only usage was removed when run_tests.sh
was removed, and it's being phased out from other repositories
The upgrade checker is going to need to be able to load
configuration settings to check for things that will
interfere with an upgrade. This will cause genopts
to incorrectly think an entry needs to be added in
cinder/opts.py . This change add status.py to the
This patch adds a new static method to drivers to expose their
configuration options. This also adds the same call to backup drivers
as well as the zone manager drivers.
Updated the generate_driver_list to expose the options as well.
This patch also orders the driver list alphabetically to make it easier
and consistent to find drivers. The driver list is now broken down
into supported and unsupported drivers as well.
Determines how many commits to check based
on the $FAST8_NUM_COMMITS env variable.
If set to "smart", it uses git to try to run
against all unsubmitted commits. This allows
fast8 to be more useful when actively developing
a series of patches.
This commit does several things:
- Setup and run pylint directly rather than running
through a script. This allows the user to see what is happening
while the user is running through pylint.
- Allow the user to either run pylint on a particular changeset,
or the entire cinder tree.
- Allow the user to run on a particular changeset. Using like
- Since the pylint gate check I disabled the tests that were
reported by pylint. The thought here would be go through
the failures and correct them.
- Update pylint to 2.1.1.
- Removed old pylintrc.
Signed-off-by: Chuck Short <firstname.lastname@example.org>
The default process concurrency for pylint is 1. Since our gate
images used to run the job have 8 cores, this results in slower
job execution with many cores sitting idle. To speed things up,
this sets the process count to match the number of cores available.
Now cinder had some Versioned Objects which names do not
match their ORM counterparts. In method: get_model_for_versioned_object,
we handles those exceptions.
This patch fix this issue to keep the names match.
This switches the pylint target to run under python3. In
order to work right, it also raises the version of pylint
used to a newer version.
The long term goal for OpenStack is to support python3 for
all things except explicit python2 testing by the T release.
Part of preparing for that is ensuring all ancillary things
like docs jobs and other tooling work with python3.
This switches the default for our docs jobs to python3 so
anything that does not specify another version will end up
Further work is necessary for the pylint job due to changes
between the runtimes. That will be done in a follow up patch.
There was also a difference in behavior with the genopts job
where it ends up trying to recreate the venv under which it
is currently running under, resulting in a corrupted venv
and a failure. This cleans up that script and changes it so
rather than a tox job calling a tox job it just runs the
The driver list generation takes driver docstrings and attempts
to parse that into rst formatted output for publication. With the
way this content is formatted within the docstring, this would
cause some slightly off formatting in the html output.
This attempts to better extract that information to make the
output cleaner and more readable.
The location of the generate_cinder_opts script was moved in
d3abafdee4. There is a check in
the script to exclude the script to its parsing, but this path
was not updated when it was moved. This updated the path to
the new location.
The sections in the cinder/opts.py file that are currently
upper case (I.E. BACKEND) cause a sphinx warning to be
generated. When we switch to treating warnings as errors
this will break our doc build.
This patch updates the code that generates the cinder/opts.py
file to make sure that all of the section names are lower-case
with the exception of the 'DEFAULT' section. With this
change in place we will not getting Sphinx warnings during
a doc build from the automatic config file generation.
In the project, some of the terminology, like URL, URLs, API, APIs, OpenStack,
UUID, Cinder are neglectfully written as url, api, openstack, uuid, cinder.
This patch is to keep consistent of naming convention.
The command to edit tempest.conf should be run as the same
user that owns that file. This is causing the scripts to run always
with the same backends.
This test was left behind as it wasn't passing the LVM -> NFS
migration setup. As it wasn't possible to reproduce and fix localy,
let's add this to the gate so we can reproduce the error scenario.
Additionaly to that, some bug fixes need this to be fully tested.
Add simple script to setup mysql and postgresql databases, this script
can be run by users during testing and will be run by CI systems for
specific setup before running unit tests. This is exactly what is
currently done by OpenStack CI in project-config.
This allows to change in project-config the python-db jobs to
python-jobs since python-jobs will call this script initially.
Update devref for this.