The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* The nova.rpcclient module is removed; it was only a helper class
to allow us split a lot of the more tedious changes out of this
patch.
* Finalizing the port from RpcProxy to RPCClient is straightforward.
We put the default topic, version and namespace into a Target and
contstruct the client using that.
* Porting endpoint classes (like ComputeManager) just involves setting
a target attribute on the class.
* The @client_exceptions() decorator has been renamed to
@expected_exceptions since it's used on the server side to designate
exceptions we expect the decorated method to raise.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in nova.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
The porting of cells code is particularly tricky:
* messaging.TransportURL parse() and str() replaces the
[un]parse_transport_url() methods. Note the complication that an
oslo.messaging transport URL can actually have multiple hosts in
order to support message broker clustering. Also the complication
of transport aliases in rpc.get_transport_url().
* proxy_rpc_to_manager() is fairly nasty. Right now, we're proxying
the on-the-wire message format over this call, but you can't supply
such messages to oslo.messaging's cast()/call() methods. Rather than
change the inter-cell RPC API to suit oslo.messaging, we instead
just unpack the topic, server, method and args from the message on
the remote side.
cells_api.RPCClientCellsProxy is a mock RPCClient implementation
which allows us to wrap up a RPC in the message format currently
used for inter-cell RPCs.
* Similarly, proxy_rpc_to_manager uses the on-the-wire format for
exception serialization, but this format is an implementation detail
of oslo.messaging's transport drivers. So, we need to duplicate the
exception serialization code in cells.messaging. We may find a way
to reconcile this in future - for example a ExceptionSerializer
class might work, but with the current format it might be difficult
for the deserializer to generically detect a serialized exception.
* CellsRPCDriver.start_servers() and InterCellRPCAPI._get_client()
need close review, but they're pretty straightforward ports of code
to listen on some specialized topics and connect to a remote cell
using its transport URL.
blueprint: oslo-messaging
Change-Id: Ib613e6300f2c215be90f924afbd223a3da053a69
Remove rootwrap code copied from oslo-incubator, make the
nova-rootwrap console_script entrypoint point to oslo.rootwrap
code instead.
Change-Id: Iec7aaf2e1599b2faf403d11b779eeab199a89486
Implements: blueprint nova-oslo-rootwrap
Baremetal deploy helper (nova/cmd/baremetal_deploy_helper.py) uses
wsgiref module, so it should be in requirements.
Closes-bug: #1263480
Change-Id: Ibdd917d8065c085411a21a65e2d9ae016e265ee5
Pull in middleware modules required to support CADF auditing
reports from Nova. audit.py requires notify.py, and
notify.py requires base.py. This is why those 3 files are
being synced, and not others in the
openstack/common/middleware directory from oslo.
The following 3 change IDs included in this sync provide
the main parts of the CADF functions being synced:
Ie67cf4a7 Add audit middleware
Iead2431a Middleware: add notifier
I05ff58 Move wsgi module to deprecated package
The following change IDs additionally included in this sync
provide minor updates to the files that were made after the
main parts of the CADF functions were merged in oslo:
I032616 Changed header from LLC to Foundation based on
trademark policies
I840c43bf add support to ignore certain request
I2099e1ee Enable hacking H404 test
I04b0cf middleware: code cleanup
blueprint support-standard-audit-formats
Change-Id: Id15f95a0c354270ba2d5aecb12c4f910905e8c20
This allows us to drop some unsightly
0.7.3 patching and also avoids potential
issues in using 0.8.1 which had a bad
release version string (it was called '0.7.3dev' by mistake).
This will require a requirements change here:
https://review.openstack.org/56662
Change-Id: I5f80dfc07f8256716d025002fc1a1f47e58b2142
Closes-Bug: 1251700
Now that 1.2.0 has been released, require it rather than one of the beta
release tarballs.
Closes-Bug: #1182861
Change-Id: I75ffc4a572d14488d19491a38742d6920951c171
Latest boto 2.13.0 has a bug which lead to unit tests fails, before
it's fixed avoid use this version.
Fix bug #1224832
Change-Id: I8a81adfc0cb0ad37d27b11db3c881735a099a95c
Only direct dependencies need to be listed in requirements.txt.
pyparsing was previously added to workaround installation ordering
issues, which is no longer necessary.
Change-Id: I45bdd1d0dabfcc7c80314c28072950cfc5a59615
Cheetah is unmaintained since 2010 and it's unlikely to get Python3
support soon. Also, the rest of OpenStack (mostly) standardized on
Jinja2.
Implements: blueprint jinja-templating-conversion
Change-Id: Ia15f00ee96d3c1d55d7c290f20ccc988e4c52e1a
The websockify package was recently upgraded to version 0.5.1, which added two
parameters that provide additional security:
* no_parent - returns 403 response if contents out of the web root are
requested. Required to disable directory traversal.
* file_only - returns 404 response if non-file contents are requested. Required
to disable directory listing.
This version is available in PyPi.
Nova should upgrade it's required version for websockify, and set these
parameters while creating and starting the NovaWebSocketProxy to ensure
better security.
DocImpact
Fixes bug #1195700
Change-Id: I9c4b28eb1b384048f10d365ed204b916afd371bb
Allow exceptions to propagate up through the stevedore map
function used by the V3 API servers core extension. This allows
for functionality such as extension specific parsing of client
supplied data formerly required to be handled by the core API to
be handled by the extension itself instead.
This functionality requires stevedore 0.10 or later
Partially implements blueprint nova-v3-api
Change-Id: Ib848147225707f1c7eda27b1ba796022161ba20f
Fixes bug 1182271.
This reverts commit c39687edb9138c6eac675b8781a5d4bf5a51c968. The
original workaround is no longer needed, as pip is now configured to
install with the --upgrade flag and will therefore pull a new enough
python-jinja2 into the tox virtualenv even though site packages is
enabled in nova and an older distribution package of the same
library is still present on CentOS 6.4.
Change-Id: I33335314c7c1b26d630137edfa0cdc2d1947432e
if any of the projects specify a capped client, it has the
potential for preventing that client from being tested in the
gate. To fix this we have to uncap maximum versions of all
openstack client code in all openstack projects.
(Note Neutronclient needs special dispensation for now, so
it is left capped.)
(Rebased after a neutron rename landed)
Fixes bug #1200214
Change-Id: I0168cb645279a819942a442c800afb1d0336203c
This change updates to use the neutron client library and provides
backwards compatiblity with the network_api_class.
implements bug: 1197208
Change-Id: Ia6c74e40eff516a0fcbe53fdc48e16fb0f98e864
Fixes bug #1194807
Firstly, we update the oslo.config dep to 1.2.0a3 because of the issue
with namespace packages (bug #1194742).
But the main issue here is that when we previously depended on 1.2.0a3
we found that if you did:
$> pip install -r nova/requirements.txt
then you end up with the oslo.config 1.1.1 code installed. This is
because oslo.config>=1.1.0 gets pulled in as a transitive dep and pip
gets confused.
See I977700d73342e81ee962019b76238d2cb2b1fff4
You can reproduce with e.g.
$> pip install \
http://.../oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3 \
python-keystoneclient
$> pip freeze | grep oslo.config
oslo.config-1.2.0a3
$> python -c 'from oslo.config.cfg import DeprecatedOpt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: cannot import name DeprecatedOpt
This is because of a bug with pip where it sees oslo.config-1.2.0a3 and
oslo.config as two unrelated things. It should strip the version part of
the egg= fragment before using it as a package name, but it doesn't.
However, we can simply use the -f/--find-links pip option in our
requirements.txt to add the tarball URL to the list of URLs considered
and also add the oslo.config>=1.2.0a3 dependency:
$> pip install \
-f http://.../oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3 \
'oslo.config>=1.2.0a3' \
python-keystoneclient
$> pip freeze | grep oslo.config
oslo.config-1.2.0a3
$> python -c 'from oslo.config.cfg import DeprecatedOpt'
This is actually exactly the semantics we want and we go to great
lengths in pbr to get these semantics while using a single tarball URL.
The only downside to this --find-links strategy is that we gain an extra
line in our requirements.txt ... but it does work around the pip bug.
Change-Id: I6f3eb5fd2c75615d9a1cae172aed859b36b27d4c
The version of cliff we are using depends on pyparsing>=1.5.7 but
python-quantumclient pulls 1.5.6 because it appears first in the
requirements processing.
The requirements change is in
https://review.openstack.org/#/c/33406/
Fixes bug #1191866
Change-Id: Ib1c11b53813ff96b750f5ffe4997ad01907d32c3
This partially reverts commit:
43b7f22b30244114f809e4c3b44dc337cade95f1.
The oslo.config change in that patch seems to break running unit
tests with tox. This reverts just that part of the change.
Change-Id: I977700d73342e81ee962019b76238d2cb2b1fff4
Use a version of stevedore that does not try to
install distribute.
The requirements change is in
https://review.openstack.org/#/c/32738/
Change-Id: I277a282ed86743a25138c431045af1f0ccb11b39
Signed-off-by: Doug Hellmann <doug.hellmann@dreamhost.com>
Rename tools/pip-requires to requirements.txt and tools/test-requires
to test-requirements.txt. These are standard files, and tools in the
general world are growing intelligence about them.
Change-Id: I68ece8406fb1d4e082a42db8e76e17b1aaa7e775
Fixes: bug #1179008