tooz doesn't use upper-constraints so we get the newest version of all
dependencies. It seems one of these, etcd3, is incompatible with recent
versions of protobuf.
Failed to import test module: tooz.tests.drivers.test_etcd3
Traceback (most recent call last):
File "/usr/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/zuul/src/opendev.org/openstack/tooz/tooz/tests/drivers/test_etcd3.py", line 22, in <module>
import tooz.drivers.etcd3 as etcd3_driver
File "/home/zuul/src/opendev.org/openstack/tooz/tooz/drivers/etcd3.py", line 18, in <module>
import etcd3
File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/__init__.py", line 3, in <module>
import etcd3.etcdrpc as etcdrpc
File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/__init__.py", line 1, in <module>
from .rpc_pb2 import *
File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/rpc_pb2.py", line 16, in <module>
from etcd3.etcdrpc import kv_pb2 as kv__pb2
File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/etcd3/etcdrpc/kv_pb2.py", line 36, in <module>
type=None),
File "/home/zuul/src/opendev.org/openstack/tooz/.tox/py37/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 755, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
The etcd3 dependency is only used for the etcd3 driver which was
deprecated in change Iff0cd3b51cfc45ecbddc831c55267e80e9c79cac. The
protobuf dependency is only used for etcd3. We can't outright remove the
etcd3 driver yet as it hasn't been long enough since we deprecated it.
That will happen soon enough though and until then we can simply cap
protobuf to an older version that is compatible with the most recent
etcd3 release.
While we're here, we also sync the lower boundaries in the docs
requirments and add some comments indicating where they come from/what
they're for.
Change-Id: I8d6647118be22b0ce55e01b7e5451612ebe30e73
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
In Zed cycle testing runtime, we are targetting to drop the
python 3.6/3.7 support, project started adding python 3.8 as minimum,
example nova:
- 56b5aed08c/setup.cfg (L13)
Also indicates that we support python 3.9.
Change-Id: I9f9e63fcea02a90a08bfb381755ed76cdd35ddef
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:
UserWarning: Usage of dash-separated 'description-file' will not be
supported in future versions. Please use the underscore name
'description_file' instead
[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb
Change-Id: I4b4bace2f08ce960a789fb97cf7320fc07195526
etcd3gw 0.2.6 was blacklisted in openstack/requirements [0], because
that version has a bug [1].
tooz does not use openstack/requirements' upper constraints, so the
same blacklisting needs to be introduced here in setup.cfg.
[0] Icb6873d8c5d3a3624c0ac3d76fc9125c5d8134b2
[1] https://github.com/dims/etcd3-gateway/issues/41
Change-Id: I22b955419014dd34c63e406c488e0636ffe9013b
Closes-Bug: #1891314
When running Consul with ACLs enabled, the best practice is to
default deny all. In order to support applying a policy to the
tooz coordination KV store path, passing through an ACL is needed.
Closes-Bug: #1752205
Change-Id: I98fc96468b21368ce66365e3fc38c495b1f2918a
The etcd3 and etcd3gw drivers parse CA, key and cert options from
the coordination URL, and pass them on to the backend clients. The
etcd3gw driver implements the "etcd3+https" scheme.
Change-Id: I78d8ca0583f883f7f746791f82fbcc116458ce2c
Sphinx 2.0 no longer works on python 2.7, so we need to start capping it there.
grpcio-1.16.0 introduced a bug that is breaking the tooz unit tests.
cf. I8b026a0680a91a68281e354a59012565f4db1b95 for more info. 1.18.0
fixes the problem, so bump our minimum requirement to that version.
Change-Id: I4b672563f4a80ce0e0478c867b7f85842ac3b0d3
After n heartbeats, the expireation time of the lock key becomes
(1 + 0.5n) * timeout! Therefore, when the lock holder performs a
long task and exits abnormally, the lock will not be automatically
released in time.
Change-Id: I2437071bb575e95083b0d57a6a0a33aaf8356113
We no longer test with Python 3.5 and we shouldn't claim we somehow
support it. We do support Python 3.7 now though so mark that as such.
Change-Id: If4b3cc1be30e5d47cc7db9197be81d6790891ff2
- Update sphinx so that warning-is-error option is available
- Fix sphinx warnings
- Install python modules required for automodule into docs env
- Fix docstring warning in tooz/coordination.py
- pbr warnerrors is no longer needed in favor of sphinx warning-is-error
Change-Id: I5d54c452522b6ee00884d7d17105af22b4c95189
The code in python-etcd3 for locks has been adjusted to reflect how tooz wants
his lock to work. That's amazing because now we can use them directly and not
implement our own locking mechanism!
Change-Id: Ia5741bae83cddd90f9bb78fba9173dca8509e6d9
Using grpc to directly access etcd3 does not work well with
eventlet based services. So we need to use the grpc HTTP
gateway API (/v3alpha).
https://coreos.com/etcd/docs/latest/dev-guide/api_grpc_gateway.html
For this we use the etcd3gw package:
https://pypi.python.org/pypi/etcd3gw
The structure of the code is very similar to the one we
already have that uses the etcd3 package.
Change-Id: I97bd7ffb05a7e40cb08c9b9d62cc45236ad292aa
This patch adds support to Tooz for taking advantage of the etcd3 gRPC-based
API (instead of the etcd2 HTTP/REST-based API) via the python-etcd3 library.
Change-Id: Ic7c3d9be42a9912fcb09c43e7637270db4011c4a
The release management team have indicated this is their preferred way
to collect release notes in future.
Closes-Bug: #1634356
Change-Id: I10fbc4cb80d53d4bf5b8b6365c7472b7706d46b2
This makes sure we install the correct and only the correct dependencies
when installing tooz, based on the driver one wants to use.
Change-Id: I58825cf228771a00b28d515a552141b42f884efa
For now it only supports locks, in the
future it may support more functionality
as time allows (and/or if people want to
add it).
Co-Authored-By: Joshua Harlow <harlowja@gmail.com>
Implements: bp add-consul-driver
Change-Id: I6064383e9db4e90fd16f17065bd7b0934ca7e125
By setting this pbr option in setup.cfg, the doc build will fail in case
of any warnings or errors occur during the build process.
Change-Id: Ia6b9cef8933ade60810a48d1813a6f3b3dc72103
We don't test against 3.3 in the integrated CI anymore
so we likely shouldn't say that we support it anymore
since it's not something we are guaranteeing.
We still support 3.4 (and that's what we are testing
against so we use that version in the setup.cfg
file).
Change-Id: I9fadcac92ea3f43e811818cf013f93445ec1b5e9
Add the foundations of a redis driver that can be used
along side the rest of the driver types.
Redis provides a few nice benefits that act as a poormans
zookeeper.
- Durability (when setup with AOF mode).
- Consistent, note that this is still restricted to only
one redis server, without the recently released redis (alpha)
clustering > 1 server will not be consistent when partitions
or failures occur (even redis clustering docs state it is
not a fully AP or CP solution, which means even with it there
will still be *potential* inconsistencies).
- Master/slave failover (when setup with redis sentinel), giving
some notion of HA (values *can* be lost when a failover transition
occurs).
Further resources/links:
- http://redis.io/
- http://redis.io/topics/sentinel
- http://redis.io/topics/cluster-spec
Change-Id: I2d90be1afae98f40ff5c4230563e686b23d42c8f
That is needed to prevent Zake driver load in real-life Zookeeper
usage - currently we had the situation when even if you want to use
just KaZoo, you needed to install partially test-requierements
(zake package).
Change-Id: I9664fda2d6e76a6e48776b4a6d92af4c4baf1425
This patch add a new driver which mimic a Kazoo client
without the need to use a real zookeeper server.
The dependency Zake (written by Joshua Harlow:harlowja@gmail.com)
has been added which provides the Kazoo API.
Change-Id: I53ca9ac972d6e2dca2f9e04b6187ee0545813e74
