Commit Graph

765 Commits

Author SHA1 Message Date
Stephen Finucane
8ca40da4bb Cap protobuf < 4.x
tooz doesn't use upper-constraints so we get the newest version of all
dependencies. It seems one of these, etcd3, is incompatible with recent
versions of protobuf.

  Failed to import test module: tooz.tests.drivers.test_etcd3
  Traceback (most recent call last):
    File "/usr/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
      module = self._get_module_from_name(name)
    File "/usr/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
      __import__(name)
    File "/home/zuul/src/opendev.org/openstack/tooz/tooz/tests/drivers/test_etcd3.py", line 22, in <module>
      import tooz.drivers.etcd3 as etcd3_driver
    File "/home/zuul/src/opendev.org/openstack/tooz/tooz/drivers/etcd3.py", line 18, in <module>
      import etcd3
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/__init__.py", line 3, in <module>
      import etcd3.etcdrpc as etcdrpc
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/__init__.py", line 1, in <module>
      from .rpc_pb2 import *
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/rpc_pb2.py", line 16, in <module>
      from etcd3.etcdrpc import kv_pb2 as kv__pb2
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/kv_pb2.py", line 36, in <module>
      type=None),
    File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 755, in __new__
      _message.Message._CheckCalledFromGeneratedFile()
  TypeError: Descriptors cannot not be created directly.
  If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
  If you cannot immediately regenerate your protos, some other possible workarounds are:
   1. Downgrade the protobuf package to 3.20.x or lower.
   2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

The etcd3 dependency is only used for the etcd3 driver which was
deprecated in change Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac. The
protobuf dependency is only used for etcd3. We can't outright remove the
etcd3 driver yet as it hasn't been long enough since we deprecated it.
That will happen soon enough though and until then we can simply cap
protobuf to an older version that is compatible with the most recent
etcd3 release.

While we're here, we also sync the lower boundaries in the docs
requirments and add some comments indicating where they come from/what
they're for.

Change-Id: I8d6647118be22b0ce55e01b7e5451612ebe30e73
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
(cherry picked from commit e3de0e634a)
(cherry picked from commit 73dbe0e6fc)
2022-07-04 12:46:43 +01:00
Pavlo Shchelokovskyy
c1ae1649d2 Enable retries in redis driver
this is followup to Iaab5ce609c0dcf7085f5dd43efbd37eb4b88f17b

actually retry for specified number of retries instead of raising
error on first ConnectionError

Change-Id: Ibca3f568b65dfea252da4b67f6d5105ba7f1ecb1
(cherry picked from commit 47c4d56e44)
(cherry picked from commit 11526e594c)
2021-11-16 17:17:29 +00:00
Matthias Runge
1ddddbba73 Retry on redis connection errors
Sometimes, connections get closed by the server. This change
adds retrying to the code.

Conflict:
requirements.txt

NOTE(hberaud): This conflict is due to the cap of tenacity during wallaby
to unblock the gate. This requirement cap won't be an issue for victoria
and for previous stable branches.

fe9c430455

Change-Id: Iaab5ce609c0dcf7085f5dd43efbd37eb4b88f17b
(cherry picked from commit a025b4d4ee)
(cherry picked from commit 62d5d17a5d)
2021-05-17 13:25:37 +02:00
Zuul
e5025587a6 Merge "Update .gitreview for stable/victoria" into stable/victoria 2020-10-22 11:35:59 +00:00
Elod Illes
c8d77ee485 Blacklist etcd3gw 0.2.6
etcd3gw 0.2.6 was blacklisted in openstack/requirements [0], because
that version has a bug [1].

tooz does not use openstack/requirements' upper constraints, so the
same blacklisting needs to be introduced here in setup.cfg.

[0] Icb6873d8c5d3a3624c0ac3d76fc9125c5d8134b2
[1] https://github.com/dims/etcd3-gateway/issues/41

Change-Id: I22b955419014dd34c63e406c488e0636ffe9013b
Closes-Bug: #1891314
(cherry picked from commit 3f0759091c)
2020-10-13 09:18:48 +02:00
affcbdaad3 Update .gitreview for stable/victoria
Change-Id: I463460a800ba29fc125a6dc9032c565ad54a6c8d
2020-09-11 21:04:17 +00:00
Dmitriy Rabotyagov
9aa22c34ca Use safe_decode for decoding in zookeeper driver
Current behaviour leds to AttributeError because py3 don't have decode
attribut for the string objects. So it's better to use safe_decode from
oslo_utils.encodeutils for that purpose

Change-Id: I22ce4212381511e2b2631a97057d6da938024b99
2020-07-29 15:43:06 +00:00
Zuul
fca43df897 Merge "hashring: allow choosing hash function" 2020-07-21 12:12:11 +00:00
Dmitry Tantsur
d59b283440 Fix breakage with PyMySQL 0.10.0
In this version a Connection is no longer a context manager. Fix
it by simply getting a Cursor out of it (locks don't seem to
interact with transactions, at least according to MariaDB docs).

Change-Id: I5ea06ebd2b976465ff82f10a74e140f30e9e803f
2020-07-20 15:52:32 +02:00
Dmitry Tantsur
f5d32480d9 hashring: allow choosing hash function
In FIPS mode it's not possible to use md5 for any purposes.

Change-Id: Ia3d9035abb645e82083fe9a796d7d7df7cf4d7a7
2020-07-12 14:56:17 +02:00
Riccardo Pittau
5ac72fcb3d Remove six library
The tooz project doesn't support Python 2 anymore, using the
six library should not be necessary.

Change-Id: I9886cabcd13cbc615065ceb372efbf037dff1aeb
2020-07-01 15:56:53 +02:00
Nick Maludy
80f379bfa2
Implements Group API for the Consul driver
Also fixes tox env for py38-consul tests
Update zuul script to use consul 1.7.4 (2020) vs 0.6.3 (2016)

Adds ACL tokens to all session management calls.

Change-Id: Iaddf21f14c434129541e7c9ec7134e0661f7be52
2020-06-15 11:24:49 -05:00
Zuul
2ae8c4b4cc Merge "Add python 3.8 to classifiers" 2020-06-08 23:36:03 +00:00
Zuul
ed86c75cab Merge "Use unittest.mock instead of third party mock" 2020-06-03 11:25:12 +00:00
Zuul
34492ee608 Merge "Stop to use the __future__ module." 2020-06-03 10:17:54 +00:00
Zuul
130df13e18 Merge "Add release notes links to doc index" 2020-06-03 09:08:46 +00:00
Hervé Beraud
9c4f23a019 Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: I95809cf4035a6bc487b24f9e91aaa715921ad842
2020-06-02 21:02:18 +02:00
Andreas Jaeger
69fc39083f Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I6254b8b17eea752a5b1d6031c89de47ecc865a49
2020-06-02 13:55:46 +02:00
Hervé Beraud
5d60388fd2 Add python 3.8 to classifiers
Change-Id: Ife3021fe203fdb6a9a6f2f51d45a64773f5fe89c
2020-05-26 14:06:45 +02:00
Zuul
94bf257392 Merge "Update master for stable/ussuri" 2020-05-25 12:41:06 +00:00
Hervé Beraud
404fe100a6 Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I4ef04d18b3a6da5cbf3193330957db7846a7bc8d
2020-05-15 18:30:01 +02:00
Zuul
2c5a72a7d5 Merge "Switch to Victoria tests" 2020-05-15 11:28:10 +00:00
JP Bourget
08b2d93226 Add support for Consul ACL token parameter.
When running Consul with ACLs enabled, the best practice is to
default deny all. In order to support applying a policy to the
tooz coordination KV store path, passing through an ACL is needed.

Closes-Bug: #1752205
Change-Id: I98fc96468b21368ce66365e3fc38c495b1f2918a
2020-05-14 21:48:49 -04:00
Andreas Jaeger
b60eee68b4 Switch to Victoria tests
This repo has been using train tests and thus was missed by the bot to
update from ussuri to victoria, update manually. also, update other jobs
to use now py38.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Ie1ef901d7b52800912365cfeae96b0e66108e1fc
2020-05-14 13:13:57 +02:00
Andreas Jaeger
977c7b9ff9 Add nose as test-requirement
Add nose requirement, this is now needed by ddt for testing here,
but not required from ddt.

Change-Id: I99c029d7a153548b9cfc94f5d80d331ca87d2b70
2020-05-14 13:11:37 +02:00
Zuul
c47e8db010 Merge "ignore reno generated artifacts" 2020-04-29 11:20:39 +00:00
JP Bourget
2546cfc562 Fix mysql driver comparison operator
A test was failing due to L67 in tooz/drivers/mysql.py using `is`
instead of `==`.

Change-Id: I2baa101499f170ece99ccff368103b58f31ca79a
Closes-Bug: 1875749
2020-04-28 16:49:12 -04:00
Hervé Beraud
a5abf9ef08 Add release notes links to doc index
Change-Id: Ibd20fc5776b789bb869cdb1f7b1e58acc4c2c207
2020-04-16 17:54:15 +02:00
b11456a00d Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: I5e9b7258593e10ff1323ab408acaf2f075fdc111
Sem-Ver: feature
2020-04-14 10:20:27 +00:00
Zuul
1880be2c00 Merge "Add TLS support in etcd3 and etcd3gw drivers" 2020-04-01 18:27:03 +00:00
Alan Bishop
a598cce62b Add TLS support in etcd3 and etcd3gw drivers
The etcd3 and etcd3gw drivers parse CA, key and cert options from
the coordination URL, and pass them on to the backend clients. The
etcd3gw driver implements the "etcd3+https" scheme.

Change-Id: I78d8ca0583f883f7f746791f82fbcc116458ce2c
2020-04-01 06:56:51 -07:00
Elod Illes
7147939860 Drop requirements-check job
Since tooz project has chosen to *NOT* consume upper-constraints.txt
there is no need for the requirements check job, so let's drop it.

Change-Id: I372cf0c9c9b1ff3d4b83cd84997f780e5de9709a
2020-03-31 17:15:55 +02:00
Hervé Beraud
4be7780adb ignore reno generated artifacts
Change-Id: I4593085e60930f039d13db2a4c7799b16a1f055a
2020-03-25 13:17:59 +01:00
Zuul
6e7a8508fb Merge "Adds heartbeating to the consul driver." 2020-03-04 03:02:16 +00:00
Nick Maludy
6e750b6921
Adds heartbeating to the consul driver.
Change-Id: Ie22e944e270dfd37988724f76823f6fd38d19b21
Closes-Bug: #1752225
2020-03-03 15:43:00 -05:00
Zuul
27c4f158e5 Merge "Avoid redis lock's expire_time exceeding timeout." 2020-03-02 11:26:47 +00:00
Andreas Jaeger
ba27954b06 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Parent tox py36/py37 jobs to openstack-tox-py36/37 so that the correct
interpreter is installed.

Change-Id: I14a01f0a9989195e30c3ed0dadec35490f41bd48
2020-02-03 22:19:47 +01:00
Bob.Haddleton
842c7404b2 RedisLock release() should not check if the lock has been acquired
The RedisDriver coordinator _stop() method removes the lock from the
_acquired_locks set before calling the release() method on the lock,
so the release() method should not check to see if the lock is in the
set, since it never will be.

This was preventing the coordinator's _stop() method from releasing
the locks, so they had to timeout before another group member could
take over as leader.

Change-Id: Ie23d88f1a05983d7465f18a329cabc23d789a628
Closes-Bug: 1815330
2019-12-19 09:12:28 +00:00
Zuul
4c85e958a5 Merge "Fix membership lease issue on the etcd3gw driver" 2019-12-16 17:13:57 +00:00
Zuul
ea29fbfdab Merge "Drop os-testr test-requirement and pretty_tox.sh" 2019-10-10 15:37:52 +00:00
Zuul
841c4b1a76 Merge "Add shared arg in metaclass Lock" 2019-10-10 15:19:42 +00:00
Thomas Bechtold
4492bbaea8 Drop os-testr test-requirement and pretty_tox.sh
os-testr is only needed to provide the "subunit-trace" command which
is used in the tools/pretty_tox.sh script. And pretty_tox.sh is not
needed. "stestr" can be used directly in tox.ini. So drop
tools/pretty_tox.sh .

Change-Id: I6a18183f092689c8c9d283c19914da9baa59cc32
2019-09-24 13:05:10 +02:00
c430dbaa69 Update master for stable/train
Add file to the reno documentation build to show release notes for
stable/train.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.

Change-Id: I42bd5786c06043c21010ef7260184b5d9d4de00f
Sem-Ver: feature
2019-09-20 17:45:17 +00:00
yenai
76cd3e8e56 Add shared arg in metaclass Lock
Add the missing arg 'shared' in the abstract method 'acquire' of
the metaclass 'Lock'.

Change-Id: I8572b0df2615f8f6e8f666be988b49e63a549679
2019-09-05 11:17:19 +08:00
Zuul
5b280a85c5 Merge "Add Python 3 Train unit tests" 2019-08-12 14:57:01 +00:00
Kaifeng Wang
55864ed245 Fix membership lease issue on the etcd3gw driver
The backend driver of etcd3gw doesn't update membership lease
on heartbeat, this patch aligns the implementation with etcd3
driver.

Change-Id: I2a6a11d59c4c82124f4d42391176031656e26528
Closes-Bug: #1839691
2019-08-12 14:12:46 +00:00
Kaifeng Wang
a4f1288ee4 Fix wrong log level during heartbeat
It appears that tooz used log.warning which is a method
instead of an integer required by tenacity.

Change-Id: I0dd40f578ff0fc7bd1cd64c7889de2148ae6b06c
Closes-Bug: #1837693
2019-07-24 13:20:36 +08:00
Zuul
491dbee619 Merge "Move grpcio from requirements.txt to extras" 2019-07-08 16:56:07 +00:00
Corey Bryant
5f7a0e290e Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: Ie594497012dd65372c0014890f1fc989714a94b1
Story: #2005924
Task: #34234
2019-07-05 15:20:19 -04:00
Thomas Bechtold
535121d613 Move grpcio from requirements.txt to extras
It is only needed for the etcd3 driver.

Change-Id: I0a0215f56b37d9a4286097b7f748f68d59afcb35
2019-07-04 08:23:54 +02:00