Commit Graph

815 Commits (master)

Author SHA1 Message Date
Jan Gutter 738931d026 Update the default etcd3gw endpoint to v3
* Recent versions of etcd no longer support the /v3alpha/ endpoint
* It's probably better to skip directly to the /v3/ endpoint
  as a default.
* Note that this probably means a new minor version of tooz, while
  the behaviour can be worked around in config, this is enough to
  break existing systems on upgrade.
* Updated the test to test backwards compatibility.
* Updated the tests to use etcd 3.3.27, unfortunately we need
  a newer version of pifpaf that supports 3.4 before the default
  api can be used.
* The etcd backend tests have been removed: the deprecated backend
  uses the v2 api which is no longer provided - a future commit
  should remove the deprecated backend. The etcd3gw backend is the

This fix is required in pifpaf, but it's not in any release:

Change-Id: Ibb5e587f6b1348794b1bf4d6405bc4a32a653c2c
2023-09-27 09:26:52 +00:00
Pavlo Shchelokovskyy 0feead5deb Replace en-dash with ASCII minus in a docstring
currently Gitea at displays a warning about
ambiguous Unicode characters in tooz/

While really minor and benign, let's fix it so it does not trigger
and confuse future viewers.

Change-Id: Idbe9006e843bcedcdb98d735c49b157ab5ae6b12
2023-09-04 07:40:44 +00:00
Zuul 702ea408b2 Merge "Change StrictRedis usage to Redis" 2023-08-11 11:10:40 +00:00
ricolin bed303e9b4 Fix: Add timeout for mysql driver
The issue with current one is we try to use blocking as timeout
That make the timeout setting in mysql bound with blocking.
And the lock can't served with scenario that when we don't need timeout
blocking but still timeout required.

Change-Id: I36c80f882a91cd1c63d5a3bc2d1d8cb50f6637c6
2023-07-10 22:56:14 +08:00
Zuul d5bf20cbcc Merge "Revert "Moves supported python runtimes from version 3.8 to 3.10"" 2023-05-08 10:46:18 +00:00
Ghanshyam 67bc01af4f Revert "Moves supported python runtimes from version 3.8 to 3.10"
This reverts commit 4a18ae10b8.

Keeping Python 3.10 in setup.cfg classifier and testing py38 min and
py310 max version.

Reason for revert:


TC has been discussing about re-adding the python 3.8
testing in current master 2023.2 release testing.


While governance changes are under review, TC agreed to add py3.8 testing
so that we do not see more project/lib dropping python 3.8 and make them
uninstalable on python 3.8


Also adding py3.8 testing back in job

Change-Id: I18508891947649aa0a696372f71877c87b5c387e
2023-05-05 14:03:32 -05:00
ricolin db64c2175c Fix mysql timeout
mysql should support customize timeout when acquire lock.
Curently we only put 0, which will lead to in crush cases,
lock might releases after mysql socket timeout, which is long hrs.

Change-Id: I01aaff4af34b89fe82d985276ba356ce85a910de
2023-04-14 14:03:54 +08:00
Takashi Kajinami 4a18ae10b8 Moves supported python runtimes from version 3.8 to 3.10
Within 2023.2 python version 3.9 and 3.10 are the
supported python runtimes [1].
[1] https: //

Change-Id: I73a909411d4896adf1fe5903afa1dec4640f8a0b
2023-04-10 13:13:32 +09:00
Tobias Urdin 378566dab8 Change StrictRedis usage to Redis
The StrictRedis class is only an alias for
Redis in >= 3.0.0

Change-Id: I89146dcf63cf984410f29b31ac2492f5ccaf3468
2023-01-18 09:55:44 +00:00
Tobias Urdin 7698c582f3 tox: set allowlist_externals
Change-Id: I96c9822e54864cdc7cfb6e22ce069054cfef9a94
2023-01-17 15:32:49 +00:00
Dmitriy Rabotyagov f11666b66c Allow to pass ssl-related args for zookeeper
Zookeeper does support TLS encryption and authentication for client
connections. There's no reason not to pass these arguments to the kazoo
to allow encrypted connections.

We bump minimum kazoo version to 2.6.0 since change implementin SSL support
has been merged with [1] and was first released with 2.6.0 tag.

[1] 35ce10669a

Change-Id: Ied29512989f477a19753afcb789e5588877fd688
2022-12-08 19:54:11 +00:00
Dmitriy Rabotyagov 7d50893625 Loosen protobuf version that's used for docs/renos
protobuf package is present in upper-constraints, thus should not be
specified in requirements to version that is smaller that one on u-c,
otherwise pip will fail due to conflict between requirements and
We do that only for docs and releasenotes, since used workaround
decreases performance.

The issue is mainly raised by etcd3 module that we rely on, so not
to affect runtime things a left as is for now.

Change-Id: Ieeba8d53a8311ed6cc228f444c831d148ee29025
2022-11-30 18:16:48 +01:00
Christian Rohmann 0c145b4567 Add grouping support in etcd to compatibility matrix.
Support has been added with

Closes-Bug: 1995125
Change-Id: I507301618c2bb32fa989db3e5634016c8e15f305
2022-10-28 17:28:21 +02:00
Zuul 1a76dd6199 Merge "Enable watch functionality for Etcd3Driver" 2022-10-12 08:35:27 +00:00
Zuul cb0392ca6b Merge "Support etcd3gw api version" 2022-09-05 16:37:55 +00:00
Pavlo Shchelokovskyy b64623daeb Support etcd3gw api version
since etcd3 3.5 dropped support for v3alpha and v3beta api, replacing
those with v3.

etcd3gw library supports passing api_path (v3alpha by default), but tooz
lacks such possibility and thus can not work with etcd3 3.5.

This patch adds handling of "api_version" in the options to
"ectd3+https" connection URLs (defaults to v3alpha as it is today and
corresponds to etcd3 version shipped in major LTS distros at the

Closes-Bug: #1983668
Depends-On: I49c480f573a4ba8294627a3ce730b816ded10aed
Change-Id: Ib30c1e003f261cd7e1ac6fed87167f9974bf8542
2022-08-24 08:12:17 -07:00
jiaqi07 00972f3b09 remove unicode from code
Change-Id: Ie63625d79dff284ce985e1fad8c487e5281fbbb4
2022-08-02 16:11:32 +08:00
Zuul d145b503ca Merge "Add TLS support for MySQL driver" 2022-07-15 21:00:16 +00:00
Zuul a10db5d089 Merge "[etcd3gw] create new lease if expired." 2022-07-15 20:20:25 +00:00
Zuul 1e86b91035 Merge "Fix inappropriate logic in memcachedlock.release()" 2022-07-15 20:17:58 +00:00
Stephen Finucane e3de0e634a Cap protobuf < 4.x
tooz doesn't use upper-constraints so we get the newest version of all
dependencies. It seems one of these, etcd3, is incompatible with recent
versions of protobuf.

  Failed to import test module: tooz.tests.drivers.test_etcd3
  Traceback (most recent call last):
    File "/usr/lib/python3.7/unittest/", line 436, in _find_test_path
      module = self._get_module_from_name(name)
    File "/usr/lib/python3.7/unittest/", line 377, in _get_module_from_name
    File "/home/zuul/src/", line 22, in <module>
      import tooz.drivers.etcd3 as etcd3_driver
    File "/home/zuul/src/", line 18, in <module>
      import etcd3
    File "/home/zuul/src/", line 3, in <module>
      import etcd3.etcdrpc as etcdrpc
    File "/home/zuul/src/", line 1, in <module>
      from .rpc_pb2 import *
    File "/home/zuul/src/", line 16, in <module>
      from etcd3.etcdrpc import kv_pb2 as kv__pb2
    File "/home/zuul/src/", line 36, in <module>
    File "/home/zuul/src/", line 755, in __new__
  TypeError: Descriptors cannot not be created directly.
  If this call came from a file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
  If you cannot immediately regenerate your protos, some other possible workarounds are:
   1. Downgrade the protobuf package to 3.20.x or lower.
   2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

The etcd3 dependency is only used for the etcd3 driver which was
deprecated in change Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac. The
protobuf dependency is only used for etcd3. We can't outright remove the
etcd3 driver yet as it hasn't been long enough since we deprecated it.
That will happen soon enough though and until then we can simply cap
protobuf to an older version that is compatible with the most recent
etcd3 release.

While we're here, we also sync the lower boundaries in the docs
requirments and add some comments indicating where they come from/what
they're for.

Change-Id: I8d6647118be22b0ce55e01b7e5451612ebe30e73
Signed-off-by: Stephen Finucane <>
2022-06-28 15:11:35 +01:00
zhen 6b2216ebcc Fix inappropriate logic in memcachedlock.release()
Whether 'was_deleted' was 'TRUE' or not, eventually we have to remove
self from '_acquired_locks'.
For example:
1. App #1 with coordinator 'A' wants to release lock "b"
2. 'self.coord.client.delete()' failed for some reason(.e.g,
3. According to the former logic,lock "b" will not remove
from "_acquired_locks", so "self.heartbeat()" will make it alive
forever until App #1 was down or lock "b" turned expired.
4. Now App #1 with coordinator 'A' wants to acquire lock "c", who
have the same lock-name with lock "b",It is clear that this will
fail and prevent the locked program from continuing to execute.

Change-Id: I6fc33b8e0a88510027bcfc30d1504489d2a91b4e
2022-05-26 17:23:41 +08:00
Mitya_Eremeev ae5adcf387 [etcd3gw] create new lease if expired.
It can happen membership lease can be expired
to the moment of lease refreshing.
In this case heartbeat will try to refresh
expired lease forever without success.
The patch checks if lease expired and create new one.

Closes-Bug: 1975524
Change-Id: I5017724fd0134f3f0a51f059640e641dd9853ff2
2022-05-25 20:21:42 +03:00
Hervé Beraud b21c20f79f Drop python3.6/3.7 support in testing runtime
In Zed cycle testing runtime, we are targetting to drop the
python 3.6/3.7 support, project started adding python 3.8 as minimum,

example nova:
- 56b5aed08c/setup.cfg (L13)

Also indicates that we support python 3.9.

Change-Id: I9f9e63fcea02a90a08bfb381755ed76cdd35ddef
2022-05-05 16:01:30 +02:00
Ekaterina Chernova c0f6cc0fe5 Enable watch functionality for Etcd3Driver
Add run_watchers support to Etcd3Driver.

Raise NotImplemented during leader change watch methods
to let corresponding tests pass since only group
membership is supported

Closes-bug: #1968323
Change-Id: Ibd46a3697b59cd22f386e60d29649acc5c5cd88a
2022-04-11 17:17:22 +03:00
Ekaterina Chernova 96f91b961f Fix getting group with prefix in etcd3gw driver
Convert group_id from string to bytes while byte string concatenation

Change-Id: I29d8fe9f3d017da4f582b9bdce8631690e728c0a
Closes-Bug: #1967498
2022-04-01 13:55:50 +03:00
Tobias Urdin 0a7c3050e8 Bump tenacity dependency to >= 5.0.0
This bumps the tenacity dependency to
>= 5.0.0 because that is what should have
been done with the fix in [1] where that
new support was added.


Change-Id: I5aef75c3dba44b975b850c0745bfe317837d2c12
2022-03-29 09:06:40 +00:00
Zuul 39e5930b0f Merge "Support later tenacity versions" 2022-03-22 14:03:49 +00:00
Zuul ba16eadd8b Merge "Update CI to use unversioned jobs template" 2022-03-21 21:48:22 +00:00
Tobias Urdin 5fbb63fa4f Support later tenacity versions
Instead of lambda func use the defined
retry_never Retrying class.

Change-Id: I756b44443d8b5892c04bd1be9faa010e39c763fc
2022-03-21 12:08:19 +00:00
Stephen Finucane 377a9a8a75 Update CI to use unversioned jobs template
As part of the migration of this project to the independent release
model, we failed to notice that the job template was still tied to a
specific release. We've now introduced a new unversioned job template,
'openstack-python3-jobs' [1], which was can and should use. Do this.


Change-Id: I51acac9189a6631eb530a25aa10f56d28179c7d0
Signed-off-by: Stephen Finucane <>
2022-03-15 15:09:56 +00:00
Tobias Urdin 7ee2780af8 Deprecate the etcd3 driver
The etcd3 driver depends on the python-etcd3 [1]
library that is currently stale and no has no
active maintenance [2] with a non-working CI
that blocks all PRs.

It's also currently blocking Tooz from supporting
newer versions of tenacity [3] that a lot of
projects is now pinning because newer version of
tenacity dropped old legacy compatibility code
that has been in there for years.

We currently have etcd3gw as an alternative driver
where the library the driver depends on is a
deliverable that we maintain.


Change-Id: Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac
2022-03-15 10:35:09 +00:00
Arne Wiebalck e88e601f92 Fix docstring for get_members()
Add 'asynchronously', fix a typo, and correct a copy and paste error.

Change-Id: Ib11562fe3398fe5faeba0477994513f674ee5bd6
2021-12-09 10:01:22 +01:00
Pavlo Shchelokovskyy 81c629676c Add TLS support for MySQL driver
use pymysql TLS-related arguments as query params in DB connection URL.

Change-Id: I3136df2a9ac4e05daa46f33e47c1438e3b02a321
2021-11-08 19:41:16 +02:00
Pavlo Shchelokovskyy 47c4d56e44 Enable retries in redis driver
this is followup to Iaab5ce609c0dcf7085f5dd43efbd37eb4b88f17b

actually retry for specified number of retries instead of raising
error on first ConnectionError

Change-Id: Ibca3f568b65dfea252da4b67f6d5105ba7f1ecb1
2021-06-16 19:44:38 +03:00
Zuul 292d768b80 Merge "setup.cfg: Replace dashes with underscores" 2021-05-26 18:34:25 +00:00
Zuul 8e996b89a0 Merge "Fix formatting of release list" 2021-05-26 17:08:29 +00:00
yangyawei 0e0c11eef3 setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead


Change-Id: I4b4bace2f08ce960a789fb97cf7320fc07195526
2021-05-14 13:50:15 +08:00
Zuul 174065f875 Merge "Retry on redis connection errors" 2021-04-27 10:15:38 +00:00
maaoyu 0ba64e4499 setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead


Change-Id: Icbbd718896cdd3858a13a149f2a65d6a3aab1c1a
2021-04-27 15:19:59 +08:00
Zuul 01a394d56c Merge "Add Python3 wallaby unit tests" 2021-04-16 14:42:11 +00:00
Pierre Riteau 608ea7979d Fix formatting of release list
Change-Id: I5b599bcc101446df96c2aebcc66962263709ad49
2021-04-16 14:23:46 +02:00
Zuul c0a95686dc Merge "Update master for stable/wallaby" 2021-04-16 11:18:17 +00:00
Zuul c6082c9417 Merge "Move flake8 as a pre-commit local target." 2021-04-15 15:10:26 +00:00
Daniel Bengtsson 714e77e71f Move flake8 as a pre-commit local target.
The goal here is to avoid conflicts between flake8 and hacking version each
2 days.

Inspired from nova's approach[1].

The flake8 version to install will be determined by hacking and
requirements[2] will stay aligned instead of relying on different versions.


Change-Id: Ic3f3d05ce7bb10a7d29815444dac00dea4f9c9d1
2021-04-06 11:01:50 +02:00
Matthias Runge a025b4d4ee Retry on redis connection errors
Sometimes, connections get closed by the server. This change
adds retrying to the code.

Change-Id: Iaab5ce609c0dcf7085f5dd43efbd37eb4b88f17b
2021-03-31 10:49:35 +02:00
OpenStack Release Bot 420059e69a Update master for stable/wallaby
Add file to the reno documentation build to show release notes for

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on

Sem-Ver: feature
Change-Id: I84eb9877fd221a1c14bf02b00f38ed77c67ed7a9
2021-03-18 11:34:29 +00:00
Matthias Runge f2005df3d6 Cap tenacity to unblock the gate
Recent version 7 changed retry and dropped the
retry_state parameter.

Change-Id: I109d3b5770c3cd8fe0bdeae78517f4d36ba3b65f
2021-03-09 16:14:24 +00:00
Zuul fa2b6cc7ae Merge "Use py3 as the default runtime for tox" 2021-02-24 02:42:21 +00:00
manchandavishal a381d71423 Bump hacking and flake8 version to fix pep8 job
This patch update hacking and flake8 version to
fix pep8 job.

Change-Id: Id6999d06fb2e15b5c80bc27b995bde065b342eec
2021-02-08 08:43:26 +00:00