This makes bandit requirements work like our
pylint requirements. (Neither are handled
in the typical way by openstack/requirements.)
This prevents needless installation of bandit
for each unit test job run.
Change-Id: I513de53520a4531067cea4e7a225fb1ad2e9ba06
Sync with the max version specified by
openstack dependency constraints.
This brings in a handful of fixes and features
over 0.910.
Change-Id: I6aa753818ed35dccd47c22d05fa2ce80473edfb8
Some of the yoga backend drivers require the yoga os-brick release.
Also update other requirements to match the minima requested by
os-brick (since that's what we'll have to use anyway).
The os-brick requirements were updated by change Iec14833ab502.
Change-Id: I694d01bcdd2ea83627d375cc4501bff774fa82c2
Raise min version of ddt to 1.4.4, which fixes a regression in the
'idata' decorator in 1.4.3 that breaks our unit tests.
This change does not need to be backported as ddt is capped at
1.4.2 in xena.
Depends-on: https://review.opendev.org/c/openstack/requirements/+/811555/
Change-Id: Ide8665722bdf7f55f92d2e2c95724cf88e8db657
We get test failures with the current requirements, so raise
them to the current allowable for stable/xena.
Change-Id: I6cb38cc72a9d85076a79f2499171dfb593e5f419
The cinder-mypy job is failing because library stubs aren't installed
for requests [0]. Modify the mypywrap.sh to accept options specified
in an environment variable named OS_MYPY_OPTS to the mypy invocation,
and set this var in tox.ini.
The value is "--install-types --non-interactive" which is suitable for
CI purposes, and seems to make sense for local tox use as well [1].
The downside is it basically runs mypy twice, once to determine
whether there are any library stubs missing and then install them, and
againto do the actual check. If we don't want this setting in
tox.ini, we can move it to .zuul.yaml for the cinder-mypy job run.
Also, update the version of mypy in test-requirements to a version
that supports the above options.
And, run mypy in its own env (instead of reusing pep8) so that the
tox logs are preserved during CI runs.
[0] https://zuul.opendev.org/t/openstack/build/b66ee6c21e594940941585b0e9e5082a
[1] https://mypy.readthedocs.io/en/stable/running_mypy.html#library-stubs-not-installed
Change-Id: Id69cb519ee7300b33ff087de4e7d46cdad67d162
Update min brick version to the wallaby brick release, and adjust
requirements files and lower constraints to accommodate the versions
of dependencies requested by os-brick 4.3.0.
Change-Id: I74b897a6ec27ca2ef00811fbbc2e6c43d31e307f
Bring the versions specified in requirements.txt and test-req.txt
closer to what's actually being used in the py36 and py38 testenvs,
and update lower-constraints to reflect the updated requirements.
What has made this necessary right now is that the latest pip has
introduced a much stricter resolver, and the cinder lower-constraints
job is failing because cinder is imposing some constraints which are
too low for some dependencies, and we can't get away with that under
the stricter resolver.
Change-Id: I42af21b1c4247d04d479f1fc1ecd6f9baac0cfc9
Add a "mypy" tox environment which runs mypy
type checking against Cinder code.
Taken from Stephen Finucane's Nova work at
https://review.opendev.org/#/c/676208/
Added "show_error_codes" and "pretty" options.
Generates an html report in ./mypy-report/
This adds stubs for oslo.i18n, so that _() calls
are annotated as intended. It may be possible to
do this with less .pyi files carried along here.
Change-Id: I2589d22c1f16f2e177d34730a520591743c0c1e3
We had a very old line in our test-requirements to install
oslo.versionedobjects with its [fixtures] extra to pick up additional
test requirements. That extra was removed with
e44431cff6b76d33e8da1812c31069a1075a97a7 several releases ago. Our
installs would silently ignore this, but with the upcoming pip resolver
changes, this will now cause an error installing dependencies.
Change-Id: I6bb5378377dba19cae119ebb3c146732a8b3844c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
As per victoria cycle testing runtime and community goal,
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).
Bump lower constraints to make testing work for Focal.
Co-Author: tushargite96 <tushargite96@gmail.com>
Story: #2007865
Task: #40179
Change-Id: I5f37fb5611362e550610e2094d9cb3778548bf47
Several version specified in our lower-constraints files had conflicting
dependencies. This updates a few packages to avoid those conflicts. It
also removes the linters that are tracked in the global requirements
blocklist since we do not need to enforce lower-constraints for linters.
Change-Id: Iaa2b3e1518614caf8664af017b2a2e1a7c005b07
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* parallelizing building of documents
Update Sphinx version as well.
Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.
openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.
Set openstackdocs_pdf_link to link to PDF file.
Set openstackdocs_auto_name to use project as name.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: I3544c1f95dd3ce510c00bfeda4f3ced87a3cd60c
Python 3.6 is now our minimum Python runtime version to support.
This raises several lower constraints versions to the minimum that
supports 3.6 or later.
This also raises greenlets and lxml versions to account for updates to
cpython changes that break installation when installing under 3.7.
Depends-on: https://review.opendev.org/725412
Change-Id: I34f7346a183e662f862a79df54a28d608a93c6c8
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.
Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.
To avoid similar gate break in future, we need to bump the hacking min
version.
- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html
Change-Id: I0a0504f9ab35cf35fb02d4edf2564f23a278c5a2
We want to limit the maximum version of hacking installed since the
global upper constraints does not do this for linters. Otherwise when a
new release of hacking is available in the future, stable branches may
suddenly be broken.
This also raises the hacking version to be at least the 3.0 release.
Change-Id: I8e8732bcba1c5fb034b50083a03b79ed833ab7ce
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This version has been blacklisted in global-requirements. It's not
likely that it will get used since pip will try to grab the latest, but
let's be explicit and make sure it never is.
Change-Id: I644926e6300f646b9ddc9ac4bdf0253caef18177
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
When a volume record is soft-deleted in the database,
dependent records in other tables (for example,
Transfers, VolumeGlanceMetadata, etc.) must be soft
deleted as well. Otherwise, we will get FK dependency
errors when the database is purged.
This patch adds that support for VolumeAttachment table.
(other tables were already covered, just refactored)
Also adds tests.
Co-authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Change-Id: Ibfa6c4ba2f162681756ec3203991351345b65346
Related-Bug: #1542169
The flake8-logging-format extension includes several checks for things
we've had to try to catch in code reviews until now. This enables the
extension and fixes the few cases where things had slipped through code
review.
G200: Logging statements should not include the exception in logged string
is disabled since that triggers a lot more issues, some of which may be
acceptable. That can be left as a follow up exercise if we want to clean
those up and enable all checks.
Change-Id: I1dedc0b31f78f518c2ab5dee5ed7abda1c1d9296
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
testresources and testscenarios were added in ec40c3b6 to address an
issue with oslo.db not pulling in its own requirements.
tempest was used when we had the tempest plugin in the main cinder repo
and was not removed when the plugin switched to its own repo.
os-api-ref is used for building the API reference and was a leftover
from before doc/requirements.txt was added.
Change-Id: Ib4fc7cb6199c9581c19d34ea43a2d15ec9003f13
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This adds usage of the flake8-import-order extension to our flake8
checks to enforce consistency on our import ordering to follow the
overall OpenStack code guidelines.
Since we have now dropped Python 2, this also cleans up a few cases for
things that were third party libs but became part of the standard
library such as mock, which is now a standard part of unittest.
Some questions, in order of importance:
Q: Are you insane?
A: Potentially.
Q: Why should we touch all of these files?
A: This adds consistency to our imports. The extension makes sure that
all imports follow our published guidelines of having imports ordered
by standard lib, third party, and local. This will be a one time
churn, then we can ensure consistency over time.
Q: Why bother. this doesn't really matter?
A: I agree - but...
We have the issue that we have less people actively involved and less
time to perform thorough code reviews. This will make it objective and
automated to catch these kinds of issues.
But part of this, even though it maybe seems a little annoying, is for
making it easier for contributors. Right now, we may or may not notice
if something is following the guidelines or not. And we may or may not
comment in a review to ask for a contributor to make adjustments to
follow the guidelines.
But then further along into the review process, someone decides to be
thorough, and after the contributor feels like they've had to deal with
other change requests and things are in really good shape, they get a -1
on something mostly meaningless as far as the functionality of their
code. It can be a frustrating and disheartening thing.
I believe this actually helps avoid that by making it an objective thing
that they find out right away up front - either the code is following
the guidelines and everything is happy, or it's not and running local
jobs or the pep8 CI job will let them know right away and they can fix
it. No guessing on whether or not someone is going to take a stand on
following the guidelines or not.
This will also make it easier on the code reviewers. The more we can
automate, the more time we can spend in code reviews making sure the
logic of the change is correct and less time looking at trivial coding
and style things.
Q: Should we use our hacking extensions for this?
A: Hacking has had to keep back linter requirements for a long time now.
Current versions of the linters actually don't work with the way
we've been hooking into them for our hacking checks. We will likely
need to do away with those at some point so we can move on to the
current linter releases. This will help ensure we have something in
place when that time comes to make sure some checks are automated.
Q: Didn't you spend more time on this than the benefit we'll get from
it?
A: Yeah, probably.
Change-Id: Ic13ba238a4a45c6219f4de131cfe0366219d722f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
We've kept hacking capped for a long time now. This raises the hacking
package version to the latest release and fixes the issues that it
found.
Change-Id: I933d541d9198f9742c95494bae6030cb3e4f2499
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
If86dd619402495d9d4470b14cb270fcf53db6794 moved reno to the docs
requirements file, so it is now only installed into the docs virtual
environment. Our instructions for adding release notes state to use "tox
-e venv -- reno new [slug]" to generate the new release note. This now
fails due to reno not being present in the default virtual environment.
This add reno back to test-requirements to make sure it is present for
generating new release notes.
Change-Id: I394c1f802409137d95a82ed6d1a4b82cf755b9e7
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Cleanup doc/requirements to just use what is needed for doc building.
Move reno to doc/requirements, add doc8 to test-requirements.
update tox.ini:
* doc8 should be in pep8 as linter and not docs according to PTI [1]
* Only build main doc in docs environment, CI uses this and we waste
just time building api-ref as well, there's a separate environment for
this. This follows PTI [1]
* Do not include requirements file for docs building, it's not needed
with apidoc.
* Use common deps for all docs build environments.
Cleanup doc/source/conf.py, with the switch to api-doc a lot of settings
are not needed anymore, also the eventlet bug is fixed.
Update openstackdocstheme to 1.20.0 and which allows to remove obsolete
setting of html_last_updated_fmt, project, latex_elements from conf.py.
Add doc8 to lower-constraints to make requirments-check happy.
[1] https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation
Change-Id: If86dd619402495d9d4470b14cb270fcf53db6794
E731s are be fixed, since use of lambdas makes code
harder to debug.
Ignore E402 and W503/W504 since these don't make a
sensible case for us to change code.
This also requires fixing some E501 errors that should
have already been detected, but weren't.
E117 and E305 errors are disabled for the moment,
and are fixed in a subsequent patch.
Pin pycodestyle to 2.5.0.
Change-Id: Ia06940cff7c3b71938eddb271d2cb8170e02e954
Bandit 1.6.0 changes the behavior of the '-x' option so that it now
supports glob patterns. Update our tox file to use to correctly exclude
test code from bandit scans. This requires bumping our minimum bandit
version.
For additional details, refer to ML Thread[1]
[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006116.html
Change-Id: I0b61934067bfb69ed3375f14bb1e05c9eeb694ad
Removes a non-exception from the map and adds tests to check the
consistency of message_field.Action and .Detail fields.
Updates test-requirements and lower-constraints to use the most
recent version of ddt allowed by current upper-constraints. (An
included test uses the idata decorator, which allows passing an
iterator to the test data.)
Closes-bug: #1822025
Change-Id: I0cad6589b145fac430942ccbb27279db7b3b152f
Due to migration of gate jobs to bionic, several packages were
incompatible with the distro.
This patch bumps the version of incompatible packages to the
lowest compatible version.
Change-Id: I9f0fec25444ed865d56d0d250fb6d840ab5b4095
Reno was moved over to doc/requirements.txt when that was created
because it is needed for releasenotes builds. But with its removal
from the other requirements files, doing something like the
recommended "tox -e venv -- reno new xxx" to create a new release
note fails with reno missing from any venv's created since it was
removed.
This adds reno back in to test-requirements.txt so it is installed
when creating the venv, ummm... venv.
Change-Id: Id2e0325b0ce8e18bb9a65318fb3fc52777753536