162 Commits

Author SHA1 Message Date
Eric Harney
5899ccd023 Update hacking to 5.0 series
Update hacking to 5.0.

Change-Id: I5a3ece77717e88c7ae3e8854d89b4a351c8cf087
2023-02-10 11:46:57 -05:00
Eric Harney
ace090cc9c Update to hacking 4.1.0
Update to hacking version 4.1.0.

Change-Id: Idb72458baff89eeb3545dd2a9d9258c1a945be8b
2023-01-30 11:26:25 -05:00
Eric Harney
1af7eefebb Update mypy to 0.981
Depends-On: I6ec5a3e7a3626d1fcfdc9f2034c765e617e2057f
Change-Id: I8d6ca5a032806ddd3ebb8cd65b42642c06b3be3c
2022-10-03 14:09:51 +00:00
Eric Harney
a287eb35a6 Bump mypy version to 0.960
Sync with the max version specified by
openstack dependency constraints.

Change-Id: Ie1975b4d5a88a955f0b16742acd532aea6710ab0
2022-07-18 12:41:27 -04:00
Zuul
5a765deaa8 Merge "Bump moto version to support py3.9" 2022-07-15 18:24:40 +00:00
Eric Harney
befa5b5be8 Move bandit requirements to tox.ini
This makes bandit requirements work like our
pylint requirements.  (Neither are handled
in the typical way by openstack/requirements.)

This prevents needless installation of bandit
for each unit test job run.

Change-Id: I513de53520a4531067cea4e7a225fb1ad2e9ba06
2022-06-27 10:51:48 -04:00
Eric Harney
6d0a4432ae Bump moto version to support py3.9
Cinder states support for Python 3.9.  Bump
to a version of moto that supports the same.

Change-Id: I1e42ca64ba3a9082e78bfa46c9ab76a8af4841ec
2022-06-27 10:39:10 -04:00
Eric Harney
18d833dc88 Bump mypy version to 0.942
Sync with the max version specified by
openstack dependency constraints.

This brings in a handful of fixes and features
over 0.910.

Change-Id: I6aa753818ed35dccd47c22d05fa2ce80473edfb8
2022-04-11 10:36:26 -04:00
Brian Rosmaita
a74513db9a Specify yoga os-brick in requirements
Some of the yoga backend drivers require the yoga os-brick release.
Also update other requirements to match the minima requested by
os-brick (since that's what we'll have to use anyway).

The os-brick requirements were updated by change Iec14833ab502.

Change-Id: I694d01bcdd2ea83627d375cc4501bff774fa82c2
2022-03-07 11:39:01 -05:00
Brian Rosmaita
8881599184 Raise min version of ddt
Raise min version of ddt to 1.4.4, which fixes a regression in the
'idata' decorator in 1.4.3 that breaks our unit tests.

This change does not need to be backported as ddt is capped at
1.4.2 in xena.

Depends-on: https://review.opendev.org/c/openstack/requirements/+/811555/
Change-Id: Ide8665722bdf7f55f92d2e2c95724cf88e8db657
2021-10-05 11:22:00 -04:00
Brian Rosmaita
72c471408a Update SQLAlchemy related requirements
We get test failures with the current requirements, so raise
them to the current allowable for stable/xena.

Change-Id: I6cb38cc72a9d85076a79f2499171dfb593e5f419
2021-09-20 11:04:30 -04:00
Brian Rosmaita
c8c4fc953c Add installation of mypy stubs packages
The cinder-mypy job is failing because library stubs aren't installed
for requests [0].  Modify the mypywrap.sh to accept options specified
in an environment variable named OS_MYPY_OPTS to the mypy invocation,
and set this var in tox.ini.

The value is "--install-types --non-interactive" which is suitable for
CI purposes, and seems to make sense for local tox use as well [1].
The downside is it basically runs mypy twice, once to determine
whether there are any library stubs missing and then install them, and
againto do the actual check.  If we don't want this setting in
tox.ini, we can move it to .zuul.yaml for the cinder-mypy job run.

Also, update the version of mypy in test-requirements to a version
that supports the above options.

And, run mypy in its own env (instead of reusing pep8) so that the
tox logs are preserved during CI runs.

[0] https://zuul.opendev.org/t/openstack/build/b66ee6c21e594940941585b0e9e5082a
[1] https://mypy.readthedocs.io/en/stable/running_mypy.html#library-stubs-not-installed

Change-Id: Id69cb519ee7300b33ff087de4e7d46cdad67d162
2021-07-15 10:33:36 -04:00
Brian Rosmaita
b14cb5e13a Use os-brick 4.3.0
Update min brick version to the wallaby brick release, and adjust
requirements files and lower constraints to accommodate the versions
of dependencies requested by os-brick 4.3.0.

Change-Id: I74b897a6ec27ca2ef00811fbbc2e6c43d31e307f
2021-03-25 13:15:43 -04:00
Eric Harney
2ae735bff4 Update to hacking 4.0.0
Update to hacking version 4.0.0.

Change-Id: I0ec3005eef5df45c45eea406ee2102ce10b6239b
2021-02-16 10:54:51 -05:00
Jegor van Opdorp
337248138e created s3 cinder backup driver
Co-Authored-By: Masahiro Okuri <okuri.masahiro@jp.fujitsu.com>
Change-Id: If24871ad6bf7558ae71e560141bf61ede61f81f5
Implements: blueprint support-s3-backup-driver
2021-01-28 10:56:43 -05:00
Brian Rosmaita
d3ffa90baa Update requirements and lower-constraints
Bring the versions specified in requirements.txt and test-req.txt
closer to what's actually being used in the py36 and py38 testenvs,
and update lower-constraints to reflect the updated requirements.

What has made this necessary right now is that the latest pip has
introduced a much stricter resolver, and the cinder lower-constraints
job is failing because cinder is imposing some constraints which are
too low for some dependencies, and we can't get away with that under
the stricter resolver.

Change-Id: I42af21b1c4247d04d479f1fc1ecd6f9baac0cfc9
2020-12-08 17:04:46 -05:00
Eric Harney
6e53c264b6 Add mypy tox env
Add a "mypy" tox environment which runs mypy
type checking against Cinder code.

Taken from Stephen Finucane's Nova work at
https://review.opendev.org/#/c/676208/

Added "show_error_codes" and "pretty" options.

Generates an html report in ./mypy-report/

This adds stubs for oslo.i18n, so that _() calls
are annotated as intended.  It may be possible to
do this with less .pyi files carried along here.

Change-Id: I2589d22c1f16f2e177d34730a520591743c0c1e3
2020-10-14 08:24:13 -04:00
Sean McGinnis
9194f70fea Remove oslo.versionedobjects extra install
We had a very old line in our test-requirements to install
oslo.versionedobjects with its [fixtures] extra to pick up additional
test requirements. That extra was removed with
e44431cff6b76d33e8da1812c31069a1075a97a7 several releases ago. Our
installs would silently ignore this, but with the upcoming pip resolver
changes, this will now cause an error installing dependencies.

Change-Id: I6bb5378377dba19cae119ebb3c146732a8b3844c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-09-17 14:53:03 -05:00
Ghanshyam Mann
14e114d901 [goal] Fixing lower constraints for Ubuntu Focal
As per victoria cycle testing runtime and community goal,
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

Bump lower constraints to make testing work for Focal.

Co-Author: tushargite96 <tushargite96@gmail.com>
Story: #2007865
Task: #40179

Change-Id: I5f37fb5611362e550610e2094d9cb3778548bf47
2020-09-16 15:14:24 -05:00
Ivan Kolodyazhny
9fbdb3ba6c Uncomment psycopg2 in test-requirements.txt
This dependency was accidentally commented in
I7f26b270d5a85cd40ffbb3b33006a41b2e4852a1 commit.

Change-Id: Ia179d9b38a50e821c0b4ca76ae0cdbfbc84125b9
2020-08-04 17:03:18 +03:00
Sean McGinnis
4af68f0621 Fix lower-constraints conflicts
Several version specified in our lower-constraints files had conflicting
dependencies. This updates a few packages to avoid those conflicts. It
also removes the linters that are tracked in the global requirements
blocklist since we do not need to enforce lower-constraints for linters.

Change-Id: Iaa2b3e1518614caf8664af017b2a2e1a7c005b07
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-08-03 15:04:39 -04:00
Eric Harney
3418aba178 Bump hacking version to 3.1.0
Fixes a small number of issues required for this
upgrade.

Change-Id: Iafb120957b262bbbff8bf98aeeaed7662e6e4d34
2020-07-14 17:01:22 -04:00
Zuul
61709c5449 Merge "Move get_volume_stats impl to the base volume driver" 2020-07-01 20:21:35 +00:00
Ivan Kolodyazhny
fdd0a3bf5e Move get_volume_stats impl to the base volume driver
Change-Id: I7f26b270d5a85cd40ffbb3b33006a41b2e4852a1
2020-07-01 12:41:20 +03:00
Eric Harney
969d65945e bump pycodestyle to 2.6.0
Fixes E225, E741 failures

Change-Id: I0cb196012a35097709bd421361e0721e71e8c307
2020-06-08 19:40:38 -04:00
Andreas Jaeger
592772b85f Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* parallelizing building of documents

Update Sphinx version as well.

Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.

openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.

Set openstackdocs_pdf_link to link to PDF file.
Set openstackdocs_auto_name to use project as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I3544c1f95dd3ce510c00bfeda4f3ced87a3cd60c
2020-05-27 12:28:24 +00:00
Sean McGinnis
a2fc5fa677 Raise lower constraints to meet our minimums
Python 3.6 is now our minimum Python runtime version to support.
This raises several lower constraints versions to the minimum that
supports 3.6 or later.

This also raises greenlets and lxml versions to account for updates to
cpython changes that break installation when installing under 3.7.

Depends-on: https://review.opendev.org/725412

Change-Id: I34f7346a183e662f862a79df54a28d608a93c6c8
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-15 21:53:03 -04:00
Ghanshyam Mann
f1969109b1 Fix hacking min version to 3.0.1
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.

Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.

To avoid similar gate break in future, we need to bump the hacking min
version.

- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html

Change-Id: I0a0504f9ab35cf35fb02d4edf2564f23a278c5a2
2020-05-15 08:50:54 -05:00
Zuul
99c09bddce Merge "Update hacking to 3.x release" 2020-04-17 03:11:53 +00:00
Sean McGinnis
1065a84b71
Update hacking to 3.x release
We want to limit the maximum version of hacking installed since the
global upper constraints does not do this for linters. Otherwise when a
new release of hacking is available in the future, stable branches may
suddenly be broken.

This also raises the hacking version to be at least the 3.0 release.

Change-Id: I8e8732bcba1c5fb034b50083a03b79ed833ab7ce
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-30 10:46:31 -05:00
Sean McGinnis
88411cfc2f
Blacklist stestr 2.3.0
This version has been blacklisted in global-requirements. It's not
likely that it will get used since pip will try to grab the latest, but
let's be explicit and make sure it never is.

Change-Id: I644926e6300f646b9ddc9ac4bdf0253caef18177
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-03-03 12:18:27 -06:00
Zuul
2ff6bb4a00 Merge "Enable flake8-logging-format extension" 2020-01-20 13:44:10 +00:00
Zuul
64b9fa6f2c Merge "Clean up test requirements" 2020-01-16 23:52:59 +00:00
whoami-rajat
a5bb17bdfc Make volume soft delete more thorough
When a volume record is soft-deleted in the database,
dependent records in other tables (for example,
Transfers, VolumeGlanceMetadata, etc.) must be soft
deleted as well.  Otherwise, we will get FK dependency
errors when the database is purged.

This patch adds that support for VolumeAttachment table.
(other tables were already covered, just refactored)

Also adds tests.

Co-authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Change-Id: Ibfa6c4ba2f162681756ec3203991351345b65346
Related-Bug: #1542169
2020-01-11 13:01:11 -05:00
Sean McGinnis
1f7b0efdcc
Enable flake8-logging-format extension
The flake8-logging-format extension includes several checks for things
we've had to try to catch in code reviews until now. This enables the
extension and fixes the few cases where things had slipped through code
review.

G200: Logging statements should not include the exception in logged string
is disabled since that triggers a lot more issues, some of which may be
acceptable. That can be left as a follow up exercise if we want to clean
those up and enable all checks.

Change-Id: I1dedc0b31f78f518c2ab5dee5ed7abda1c1d9296
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-01-09 14:35:20 -06:00
Sean McGinnis
a90970da15
Clean up test requirements
testresources and testscenarios were added in ec40c3b6 to address an
issue with oslo.db not pulling in its own requirements.

tempest was used when we had the tempest plugin in the main cinder repo
and was not removed when the plugin switched to its own repo.

os-api-ref is used for building the API reference and was a leftover
from before doc/requirements.txt was added.

Change-Id: Ib4fc7cb6199c9581c19d34ea43a2d15ec9003f13
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-01-08 13:33:50 -06:00
Sean McGinnis
3eb9b422f4
Introduce flake8-import-order extension
This adds usage of the flake8-import-order extension to our flake8
checks to enforce consistency on our import ordering to follow the
overall OpenStack code guidelines.

Since we have now dropped Python 2, this also cleans up a few cases for
things that were third party libs but became part of the standard
library such as mock, which is now a standard part of unittest.

Some questions, in order of importance:

Q: Are you insane?
A: Potentially.

Q: Why should we touch all of these files?
A: This adds consistency to our imports. The extension makes sure that
   all imports follow our published guidelines of having imports ordered
   by standard lib, third party, and local. This will be a one time
   churn, then we can ensure consistency over time.

Q: Why bother. this doesn't really matter?
A: I agree - but...

We have the issue that we have less people actively involved and less
time to perform thorough code reviews. This will make it objective and
automated to catch these kinds of issues.

But part of this, even though it maybe seems a little annoying, is for
making it easier for contributors. Right now, we may or may not notice
if something is following the guidelines or not. And we may or may not
comment in a review to ask for a contributor to make adjustments to
follow the guidelines.

But then further along into the review process, someone decides to be
thorough, and after the contributor feels like they've had to deal with
other change requests and things are in really good shape, they get a -1
on something mostly meaningless as far as the functionality of their
code. It can be a frustrating and disheartening thing.

I believe this actually helps avoid that by making it an objective thing
that they find out right away up front - either the code is following
the guidelines and everything is happy, or it's not and running local
jobs or the pep8 CI job will let them know right away and they can fix
it. No guessing on whether or not someone is going to take a stand on
following the guidelines or not.

This will also make it easier on the code reviewers. The more we can
automate, the more time we can spend in code reviews making sure the
logic of the change is correct and less time looking at trivial coding
and style things.

Q: Should we use our hacking extensions for this?
A: Hacking has had to keep back linter requirements for a long time now.
   Current versions of the linters actually don't work with the way
   we've been hooking into them for our hacking checks. We will likely
   need to do away with those at some point so we can move on to the
   current linter releases. This will help ensure we have something in
   place when that time comes to make sure some checks are automated.

Q: Didn't you spend more time on this than the benefit we'll get from
   it?
A: Yeah, probably.

Change-Id: Ic13ba238a4a45c6219f4de131cfe0366219d722f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-01-06 09:59:35 -06:00
Sean McGinnis
d9ce598f0c Raise hacking version to 2.0.0
We've kept hacking capped for a long time now. This raises the hacking
package version to the latest release and fixes the issues that it
found.

Change-Id: I933d541d9198f9742c95494bae6030cb3e4f2499
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-01-02 14:42:49 -06:00
Sean McGinnis
d6cd56baca
Readd reno to test-requirements
If86dd619402495d9d4470b14cb270fcf53db6794 moved reno to the docs
requirements file, so it is now only installed into the docs virtual
environment. Our instructions for adding release notes state to use "tox
-e venv -- reno new [slug]" to generate the new release note. This now
fails due to reno not being present in the default virtual environment.

This add reno back to test-requirements to make sure it is present for
generating new release notes.

Change-Id: I394c1f802409137d95a82ed6d1a4b82cf755b9e7
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-12-02 04:11:45 -06:00
Andreas Jaeger
af04cc3392 Update docs building
Cleanup doc/requirements to just use what is needed for doc building.
Move reno to doc/requirements, add doc8 to test-requirements.

update tox.ini:
* doc8 should be in pep8 as linter and not docs according to PTI [1]
* Only build main doc in docs environment, CI uses this and we waste
  just time building api-ref as well, there's a separate environment for
  this. This follows PTI [1]
* Do not include requirements file for docs building, it's not needed
  with apidoc.
* Use common deps for all docs build environments.

Cleanup doc/source/conf.py, with the switch to api-doc a lot of settings
are not needed anymore, also the eventlet bug is fixed.

Update openstackdocstheme to 1.20.0 and which allows to remove obsolete
setting of html_last_updated_fmt, project, latex_elements from conf.py.

Add doc8 to lower-constraints to make requirments-check happy.

[1] https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation

Change-Id: If86dd619402495d9d4470b14cb270fcf53db6794
2019-08-10 11:41:14 +02:00
Eric Harney
830490916b Bump hacking version to 1.1.x
E731s are be fixed, since use of lambdas makes code
harder to debug.

Ignore E402 and W503/W504 since these don't make a
sensible case for us to change code.

This also requires fixing some E501 errors that should
have already been detected, but weren't.

E117 and E305 errors are disabled for the moment,
and are fixed in a subsequent patch.

Pin pycodestyle to 2.5.0.

Change-Id: Ia06940cff7c3b71938eddb271d2cb8170e02e954
2019-05-30 15:06:02 -04:00
whoami-rajat
0554ee1d47 Bump bandit version
Bandit 1.6.0 changes the behavior of the '-x' option so that it now
supports glob patterns. Update our tox file to use to correctly exclude
test code from bandit scans. This requires bumping our minimum bandit
version.
For additional details, refer to ML Thread[1]

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006116.html

Change-Id: I0b61934067bfb69ed3375f14bb1e05c9eeb694ad
2019-05-20 16:54:30 +00:00
Brian Rosmaita
a5b5d8e70e Remove non-exception from EXCEPTION_DETAIL_MAPPINGS
Removes a non-exception from the map and adds tests to check the
consistency of message_field.Action and .Detail fields.

Updates test-requirements and lower-constraints to use the most
recent version of ddt allowed by current upper-constraints.  (An
included test uses the idata decorator, which allows passing an
iterator to the test data.)

Closes-bug: #1822025
Change-Id: I0cad6589b145fac430942ccbb27279db7b3b152f
2019-04-03 11:34:46 +00:00
whoami-rajat
645ffab6bf Gate Fix: Bumping versions in lower-constraints.txt
Due to migration of gate jobs to bionic, several packages were
incompatible with the distro.
This patch bumps the version of incompatible packages to the
lowest compatible version.

Change-Id: I9f0fec25444ed865d56d0d250fb6d840ab5b4095
2019-03-15 16:12:24 +05:30
Sean McGinnis
f0f930b12e Add reno to test-requirements
Reno was moved over to doc/requirements.txt when that was created
because it is needed for releasenotes builds. But with its removal
from the other requirements files, doing something like the
recommended "tox -e venv -- reno new xxx" to create a new release
note fails with reno missing from any venv's created since it was
removed.

This adds reno back in to test-requirements.txt so it is installed
when creating the venv, ummm... venv.

Change-Id: Id2e0325b0ce8e18bb9a65318fb3fc52777753536
2018-04-27 09:21:06 -05:00
OpenStack Proposal Bot
f0d17457e0 Updated from global requirements
Change-Id: Iefa7a5e32739a6b9d448659cd8edd232de267aae
2018-02-01 07:06:24 +00:00
OpenStack Proposal Bot
91d6ea9bd5 Updated from global requirements
Change-Id: Ie64af4305605717af01f903b05352e2480569d25
2018-01-16 12:25:12 +00:00
junboli
1b50d73a43 Remove unused lib in test-rquirements.txt
python-subunit is unused now, so we can remove that, and
we have switched to use stestr, os-testr can be removed.
for details, please see [1] and [2]

[1] https://review.openstack.org/#/c/488441/
[2] https://review.openstack.org/#/c/491075/

Change-Id: I5798ebacf8367a5d2b539cc14ee6af505c5a58b6
2018-01-14 17:52:57 +08:00
jiansong
f877ec2917 Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:

https://governance.openstack.org/tc/reference/project-testing-interface.html

For more detials information, please refer to:

http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I84e614346cfa4d302f919d12bea9680c5e527475
2017-12-20 18:24:11 -08:00
OpenStack Proposal Bot
42f98148a4 Updated from global requirements
Change-Id: I615d12dfdced3230d3a4195e0323cf3ed9a04e6b
2017-12-05 02:58:54 +00:00