26 Commits

Author SHA1 Message Date
Zuul
2e846ed0f3 Merge "Fix rfc3986.is_valid_uri deprecation warnings" 2018-12-27 18:15:04 +00:00
Takashi NATSUME
2d2dc78975 Fix rfc3986.is_valid_uri deprecation warnings
The rfc3986.is_valid_uri has been deprecated in 1.1.0.
It generates the following warnings.

DeprecationWarning: Please use rfc3986.validators.Validator instead.
This method will be eventually removed.

So this patch replaces rfc3986.is_valid_uri with
rfc3986.validators.Validator.

Even after applying this patch, the warnings are still output
because it is caused by oslo.config.
The fix for oslo.config will be done in another patch.

Change-Id: I70aebad6c6bd384dbd11ef732226356922bf1913
Closes-Bug: #1809755
2018-12-26 10:48:39 +09:00
Matt Riedemann
5316a8a0cd Fix jsonutils.to_primitive UserWarning
The CheatingSerializer fixture used in nova tests keeps
the RequestContext.db_connection set on the context object
which otherwise wouldn't normally happen. The context object
can show up in error notification payloads because of how
nova.exception_wrapper.wrap_exception works. That payload
is eventually serialized for notifications and since the
cheated RequestContext.db_connection is set and cannot be
serialized, it results in a UserWarning from the
jsonutils.to_primitive method (called via JsonPayloadSerializer).

This will eventually result in failures when that UserWarning
is made into an error.

To fix this, we can pass a fallback method to to_primitive()
which will serialize a RequestContext object the same way that
RequestContextSerializer serializes a context - by simply
converting it to dict form.

Since this only affects test runs, because of using the
CheatingSerializer fixture, it should have no impact on
runtime serializations.

Error logging is added to the FakeNotifier since it's hard
to know what is wrong in the payload unless it is logged.

Also, the WarningsFixture is updated to make sure we don't
introduce new UserWarnings for the serialization issue.

The jsonutils.to_primitive() fallback method was added to
oslo.serialization via commit cdb2f60d26e3b65b6370f87b2e9864045651c117
in 2.21.1 so we have to bump our minimum required version
of that library as well.

Change-Id: Id9f960a0c7c8897dbb9edf53b4723154341412d6
Closes-Bug: #1799249
2018-12-17 22:56:50 -05:00
Zuul
2e9356374e Merge "Migrate upgrade checks to oslo.upgradecheck" 2018-12-17 23:23:17 +00:00
Ben Nemec
c9436c3846 Migrate upgrade checks to oslo.upgradecheck
The common upgrade check code has been moved to oslo.upgradecheck.
This change switches the Nova upgrade checks to use the common code
and removes the tests for functionality that is now the responsibility
of the library.

Change-Id: I0dc2044286dbe78314c650a92c4654f7f50642d2
2018-12-14 21:45:34 +00:00
Balazs Gibizer
1f2a80c195 Replace ThreadPoolExecutor with GreenThreadPoolExecutor
Based on eventlet issue [1] ThreadPoolExecutor doesn't play nice with
eventlet in python 3.7. We saw deadlocks in the functional-py37
execution in live migration tests due to live migration using
ThreadPoolExecutor.

The [1] suggests to replaces ThreadPoolExecutor with
futurist.GreenThreadPoolExecutor to avoid deadlocks. So this patch does
the replacement and adjusts the unit tests accordingly.

As the ThreadPoolExecutor was the last used class from the futures
module we remove that from the requirements and add the futurist module
instead.

[1] https://github.com/eventlet/eventlet/issues/508

Change-Id: Ia56ab43be739e677760bbad5c40caad924425fa5
2018-12-13 10:41:42 -05:00
melanie witt
fded752b16 Use SleepFixture instead of mocking _ThreadingEvent.wait
Recently, the _ThreadingEvent class in oslo.service was removed [1] and
our unit test patching is preventing us from moving to a newer version
of oslo.service [2].

We have patching of the _ThreadingEvent.wait method to bypass the sleep
time in the looping call of RetryDecorator, which adds several seconds
to the run time of unit tests.

This changes things to use the new SleepFixture from oslo.service
instead.

Depends-On: https://review.openstack.org/616371

[1] I62e9f1a7cde8846be368fbec58b8e0825ce02079
[2] https://review.openstack.org/615676

Change-Id: I45dd7602068eb0ce1331cfefd5a0cf6418bc8e88
2018-11-08 16:50:39 -05:00
Balazs Gibizer
d4f8974f87 Add request_spec.RequestGroup versioned object
Later patches will introduce a field in RequestSpec using this type as
the field type to store the resource requests coming from outside of
Nova like the bandwidth request coming from the Neutron ports.

This patch refactors the usage of placement.lib.RequestGroup. Until now
this class was used both by placement and nova services and they used
it only as a util class. However after this series the nova services
would like to use such a class via RPC which requires an OVO. This
patch makes sure that the new OVO is used by nova and the old plain
object is used by placement. This way placement is not forced to use
an OVO where no OVO functionality is required.

The minimum required version of oslo.versionedobjects is updated to
1.33.3 to include the fix for bug 1771804.

Change-Id: I46c97d2641d9685ef59771314665a17a5236097d
blueprint: bandwidth-resource-provider
2018-11-05 17:47:06 -05:00
Zuul
1827f2b7b2 Merge "Fix ironic client ironic_url deprecation warning" 2018-10-30 07:31:08 +00:00
Daniel Abad
35f49f4035 Fix ironic client ironic_url deprecation warning
In preparation for stein, use 'endpoint' instead of 'ironic_url'
when calling get_client in order to remove the following warning:

WARNING ironicclient.client The argument "ironic_url" passed to
get_client is deprecated and will be removed in Stein release,
please use "endpoint" instead.

For reference:
In the python-ironicclient code, in the ironicclient/client.py#L24
TODO(vdrok): remove in Stein
[...]
  ('ironic_url',): 'endpoint',

Introduced in commit:
58c39b7a80

Change-Id: I1b3ce1955622c40b780c0b15ec7e09be3e8ace72
2018-10-24 17:59:10 +02:00
melanie witt
37d046c511 Bump os-brick version to 2.6.1
This will get us bug fixes for gate bug:

  https://bugs.launchpad.net/nova/+bug/1732199

and another bug that looks like it could affect the gate:

  https://bugs.launchpad.net/os-brick/+bug/1794829

Closes-Bug: #1732199

Change-Id: I709b4a1438aedcfc95d191462b4ec3f1f8c4355a
2018-10-16 20:29:51 +00:00
Eric Fried
8e1ca5bf34 Use uuidsentinel from oslo.utils
oslo.utils release 3.37.0 [1] introduced uuidsentinel [2]. This change
rips out nova's uuidsentinel and replaces it with the one from
oslo.utils.

[1] https://review.openstack.org/#/c/599754/
[2] https://review.openstack.org/#/c/594179/

Change-Id: I7f5f08691ca3f73073c66c29dddb996fb2c2b266
Depends-On: https://review.openstack.org/600041
2018-09-05 09:08:54 -05:00
Stephen Finucane
964832d37d Revert "Don't use '_TransactionContextManager._async'"
This reverts commit bd7d991309ea2bea5d175cb1f2710519936fd0c2 and bumps
the minimum version of oslo.db to 4.40.0, as that is the first version
of the library to include the renamed attribute.

Change-Id: Ic9e7864be3af7ef362cad5648dfc7bdecd104465
Related-Bug: #1788833
2018-08-28 17:18:51 +01:00
Kevin_Zheng
79dac41fee Use ThreadPoolExecutor for max_concurrent_live_migrations
This changes the max_concurrent_live_migrations handling
to use a ThreadPoolExecutor so that we can control a bounded
pool of Futures in order to cancel queued live migrations
later in this series.

There is a slight functional difference in the unlimited
case since starting in python 3.5, ThreadPoolExecutor will
default to ncpu * 5 concurrently running threads. However,
max_concurrent_live_migrations defaults to 1 and assuming
compute hosts run with 32 physical CPUs on average, you'd
be looking at a maximum of 160 concurrently running live
migrations, which is probably way above what anyone would
consider sane.

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Part of blueprint abort-live-migration-in-queued-status

Change-Id: Ia9ea1e164fb3b4a386405538eed58d94ad115172
2018-07-16 13:57:09 -04:00
Huang Rui
e5acf4f961 z/VM Driver: Initial change set of z/VM driver
This is the first change that implements basic virt.driver methods
to allow nova-compute process start successfully.

A set of subsequent changes will implement spawn, snapshot, destroy
and instance power actions.

Change-Id: Ica6117c2c64f7518b78b7fb02487622250638e88
blueprint: add-zvm-driver-rocky
2018-07-16 11:35:10 +08:00
Stephen Finucane
d1820d14c6 Revert "docs: Disable smartquotes"
Bump the minimum version of oslo.config to 6.1.0, which adds proper
support for parsing Opt.help as rST [1]. This in turn allows us to
revert commit 75fc30090133c31316e6ae790568f1b622807d0c, which is a
temporary fix relying on deprecated features of Sphinx.

[1] https://review.openstack.org/#/c/553860/

Change-Id: I8f56bdce37cfc538348490052a24e463164c86a3
2018-07-11 14:02:32 +01:00
Sylvain Bauza
c4e0ad1433 Fix placement incompatible with webob 1.7
In Ie4d81fa178b3ed6b2a7b450b4978009486f07810 we started using a new WebOb API
for introspecting headers but since this new API isn't supported by older
versions than 1.8, we need to only accept 1.8.1 or 1.8.2 for Nova
(because 1.8.0 was having a bug fixed by 1.8.1 at least).

Change-Id: I345f372815aef5ac0fb6fc607812ce81587734bf
Closes-Bug: #1773225
2018-06-29 16:55:03 +02:00
Eric Fried
68caa292ce Bump keystoneauth1 minimum to 3.9.0
...so we can set raise_exc=False on Adapter creation instead of having
to do it on every request.

Change-Id: Icaf6e67de00108578a1a264570b60a9139a92f98
2018-06-22 09:13:19 -05:00
Zuul
5b56434517 Merge "Use oslo.messaging per-call monitoring" 2018-06-14 21:55:37 +00:00
jianghua wang
f7593ded8f XenAPI: define a new image handler to use vdi streaming
With the new image handler, it creates an image proxy which
will use the vdi streaming function from os-xenapi to
remotely export VHD from XenServer(image upload) or import
VHD to Xenerver(image download).
The existing GlanceStore uses custom functionality to directly
manipulate files on-disk, so it has the restriction that SR's
type must be file system based: e.g. ext or nfs. The new
image handler invokes APIs formally supported by XenServer
to export/import VDI remotely, it can support other SR
types also e.g. lvm, iscsi, etc.

Note:
vdi streaming would be supported by XenServer 6.5 or above.
The function of image handler depends on os-xenapi 0.3.3 or
above, so bump os-xenapi's version to 0.3.3 and also declare
depends on the patch which bump version in openstack/requirements.

Blueprint: xenapi-image-handler-option-improvement
Change-Id: I0ad8e34808401ace9b85e1b937a542f4c4e61690
Depends-On: Ib8bc0f837c55839dc85df1d1f0c76b320b9d97b8
2018-06-13 01:59:07 +00:00
Dan Smith
fe26a52024 Use oslo.messaging per-call monitoring
This change makes nova configure oslo.messaging's active call monitoring
feature if the operator increases the rpc_response_timeout configuration
option beyond the default of 60 seconds. If this happens, oslo.messaging will
heartbeat actively-running calls to indicate that they are still running,
avoiding a false timeout at the shorter interval, while still detecting
actual dead-service failures before the longer timeout value.

In addition, this adds a long_rpc_timeout configuration option that we
can use for known-to-run-long operations separately from the base
rpc_response_timeout value, and pre_live_migration() is changed to use
this, as it is known to suffer from early false timeouts.

Depends-On: Iecb7bef61b3b8145126ead1f74dbaadd7d97b407
Change-Id: Icb0bdc6d4ce4524341e70e737eafcb25f346d197
2018-06-11 14:44:10 -07:00
Zuul
40a0b579c4 Merge "Implement granular policy rules for placement" 2018-06-01 21:06:42 +00:00
Rawan Herzallah
a833bcd05f Adding NVMEoF for libvirt driver
Adding NVMEoF libvirt driver for supporting NVMEoF initiator CLI.
Libvirt NVMe volume driver is added to handle required calls for
attaching and detaching volume from instaces through calling
os-brick's NVMe Connector.

Implements: blueprint nvme-over-fabirc-nova
Co-Authored-By: Ivan Kolodyazhny <e0ne@e0ne.info>
Change-Id: I67a72c4226e54c18b3a6e4a13b5055fa6e85af09
2018-05-29 11:26:01 +00:00
Matt Riedemann
0a461979df Implement granular policy rules for placement
This adds a granular policy checking framework for
placement based on nova.policy but with a lot of
the legacy cruft removed, like the is_admin and
context_is_admin rules.

A new PlacementPolicyFixture is added along with
a new configuration option, [placement]/policy_file,
which is needed because the default policy file
that gets used in config is from [oslo_policy]/policy_file
which is being used as the nova policy file. As
far as I can tell, oslo.policy doesn't allow for
multiple policy files with different names unless
I'm misunderstanding how the policy_dirs option works.

With these changes, we can have something like:

  /etc/nova/policy.json - for nova policy rules
  /etc/nova/placement-policy.yaml - for placement rules

The docs are also updated to include the placement
policy sample along with a tox builder for the sample.

This starts by adding granular rules for CRUD operations
on the /resource_providers and /resource_providers/{uuid}
routes which use the same descriptions from the placement
API reference. Subsequent patches will add new granular
rules for the other routes.

Part of blueprint granular-placement-policy

Change-Id: I17573f5210314341c332fdcb1ce462a989c21940
2018-05-17 11:12:16 -04:00
Matt Riedemann
ee6a257f58 Cleanup ugly stub in TestLocalDeleteAllocations
With a fix in wsgi-intercept 1.7.0 we can properly
use the PlacementFixture as a context manager to
simulate when placement is configured for a given
operation. This allows us to remove the ugly stub
that one of the tests in here had to rely on.

While in here, the CastAsCall fixture is removed
since we shouldn't rely on that in these tests
where we're trying to simulate the user experience.

Change-Id: I2074b45126b839ea6307a8740364393e9dddd50b
2018-05-16 09:23:55 -04:00
Doug Hellmann
e95af3f91c add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: Ic28558ee6481d49a9b4e5dc2c4182504e330448f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Co-Authored-by: Eric Fried <efried@us.ibm.com>
Co-Authored-by: Jim Rollenhagen <jim@jimrollenhagen.com>
2018-05-14 16:47:07 +00:00