Our version of hacking was ancient, and it loads a pep8 check that
is both very inconsistent in reporting errors and has weaker
constraints. Fix the pep8 errors revealed and disable some of the
irrelevant tests in tox.ini.
Change-Id: I4ca71aa88456e4ac2baf0cfb3ac058b0a19e3b6b
Some minor updates to certain functions to allow mypy to work, but
largely a matter of adding formatted typing comments that mypy can do
static typing against.
Change-Id: I65e88ba493599091fa657181f3f3ad5595dacbe1
The 'SignedEtcdJSONWriter' is, depending on config, a 'signed etcd json
writer' or 'a proxy to the unsigned etcd json writer'. The two classes
should be instances of the same base type and the correct one
instantiated based on circumstances.
This change converts users of the JSON writers to use a function that
instantiates the right JSON writer based on circumstances.
Change-Id: If01c68d93d06c62bea8e55568d5049a3aa8c2c97
Correctly handle mac-address changes during vpp-agent restarts
by removing the mac-address of a port from the etcd key.
The port mac-address could be computed dynamically by the vpp-agent
and could change after an agent restart. If the mac-address is
present in the etcd key, it results in duplicate GPE keys and GPE
stops working. This patch resolves this issue by removing the
mac-address from the key and storing it in the value.
Closes-Bug: #1761023
Change-Id: I3f1f4168f0c77080edc076bbb660021b81fb62dc
A problem with the 'get_subtree' method resulted in use of .children
returning native EtcdResult types with no augmented parsing of data.
There are a number of attributes that ParsedEtcdResult didn't expose but
there's no reason they shouldn't be exposed. These have been added, in
case future code depends on them.
Change-Id: Ia879d4e97d8bdbef835202507a9a2d662611c163
This recently (Pike-Queens) moved from neutron.plugins.ml2.config to
neutron.conf.plugins.ml2.config and this patch adds a layer in compat.py
to insulate us from that, and also rejigs ml2_vpp config so that the
options for etcd live nearer the code that uses them and in general all
option registering works in a similar fashion.
Addresses-the config problems that were band-aided over in change
Idbf622591e8d92d8e29e3a92799d0e7ff460354a.
Change-Id: I18767336f05a0fdcebfe16ddd46c2a5325d4e661
The etcd messages are (optionally) stored as JWT (JSON Web tokens) with
RSA signature and X509 certificate. It guarantees the authenticity of
the etcd messages.
Change-Id: Iaa1a70a7e87b935b6ff48e2b1f27784ed27ecc97
This patch fixes the bug in which GPE key_space directories were
left behind in etcd after a port was unbound. It removes the empty
etcd directories and cleans up etcd.
Closes-Bug: 1710737
Change-Id: Ie146dcbee18da1e236e004dd7e2ab6719d6aca0e
Ensure that we grab the election for a certain time before proceeding
with jobs that can take extended periods. This ensures that the election
does not expire (which would let another thread become master).
Adding a configurable values for various timeouts and general refactoring.
Change-Id: I480a4ec44b571c24fcf42520220de0439743ef81
Closes-bug:#1694723
The utils module contained a number of originally general purpose
functions, but they ultimately ended up being all about etcd. The
exceptions were also exclusively about etcd and its config. The
two modules have been combined with etcdhelper and networking_vpp's
own exceptions module.
Change-Id: Ieaa574cf7b335c43062419845e9ebd96a38b4bdb
This is imperfect, because our compatibility layer that allows
us to work with older versions doesn't cover all the necessary
Neutron inclusions, but excluding N530 we have a pass.
Changes fit into categories:
- LOG.warn -> LOG.warning
- log messages now have kwargs if more than one argument
(not essential, but used to be for translation and seems
to be a good practice anyway)
- exception messages must be translated
- translated messages must have named positional arguments
- oslo_serialization.jsonutils must be used, not json
- [] can't be a default argument
- compat is marked with N530 as, there, neutron imports are
expected. In other places the mark has been left off, and
the tox check has been disabled (for later enablement via an
RFE)
This should not include changes outside of that set.
Change-Id: I1dc050f83a5199bf40117eac6c9adae221ae6857
This still doesn't break backward compatibility, but
makes the resync_end default behaviour sane - a removal of
keys that we know are known by the controlled element but are
no longer in etcd.
Change-Id: If9c6c2a1445a301ba7fd101c3b171e7955d17568
If etcd client has connect timeout, rv is never defined and vpp-agent will
throw exception afterward. This patchset is to predefine rv to avoid
unknown variable exception.
Change-Id: I8cc6ddd86ed3579da06d623d847b262bbd186be4
This addes a new subclass to EtcdWatcher that attempts to analyse
changes in etcd itself, regardless of whether a watch or resync
happened, and will then pass on the calls to its listening function
according to what is different.
This has the advantage that resyncs because of history problems,
syncs because of initial setup and watches are all reduced to the same
problem.
This makes the system more susceptible to port-bind-before-secgroup
problems.
Change-Id: Ib49a5efde84171a387d2ba468bb16999a42db56f
- Remove wait_until_elected argument to EtcdElection creation
This argument was removed in a previous patch but not from one
specific call - remove it there so that the election works.
The default and now only behaviour is wait_until_elected, so we will
get the same behaviour here as we did before when passing =True.
- Separate election and watcher types
Watcher is-not-an election; watcher uses-an election
- Change the recovery and the job time
Watches are timing out because we spend more time in the work than
the expiry time of the key
- Stop clearing the key space on mechdriver restart
Multiple Neutron processes on one or multiple hosts may be running,
so previous elections should be respected. We can't assume that the
reset of this neutron process has removed an elected thread.
- Add debug to the election process
- Move thread_id into the EtcdElection
- Make the journal draining thread kick work across hosts and processes
Change-Id: I84a2c543aceea3ceb096b4ef5ed1aa46a47ca3c8
Closes-Bug: 1666308
Closes-bug: #1644371
Implement an election algorithm to elect two threads
as masters for doing work - one for DB journal (forward workers)
and the other for etcd (reverse workers). The threads that are
not elected as master go to sleep and check if the master is
alive periodically. If they cannot detect a master, a new master
is elected to do the work.
Change-Id: I13a4dbe4e58c8156231acf7fab2ca293f7b4e4cb
Tuple in place from move from % to , on the log call, and this means
we're an arg short. Simple typo.
Change-Id: Ie4f1beaa758cce2e4fc190eec4d57f377d71fde4
String interpolation should be delayed to be handled by the logging code,
rather than being done at the point of the logging call.
Ref:http://docs.openstack.org/developer/oslo.i18n/guidelines.html#log-translation
For example:
LOG.info(_LI('some message: variable=%s') % variable)
LOG.info(_LI('some message: variable=%s'), variable)
Change-Id: I1cdd1eb10aa6ea59461436b91cc966606f7358c2
We've already found several interesting ways to add race conditions
to the watch loop, and additionally resyncs are a little hard to
grasp; pull the code out to a common location so that fixes benefit
all watchers and new users don't make all the rookie mistakes.
This includes fixes to address the race window between startup reads
and the first watch.
Change-Id: I55846b346644bb33179ed77979409d38c93fd6a5
