1378 Commits

Author SHA1 Message Date
Lucian Petrut
afc3a16ce3 Fix os-simple-tenant-usage result order
nova usage-list can return incorrect results, having resources counted
twice. This only occurs when using the 2.40 microversion or later.

This microversion introduced pagination, which doesn't work properly.
Nova API will sort the instances using the tenant id and instance uuid,
but 'os-simple-tenant-usage' will not preserve the order when returning
the results.

For this reason, subsequent API calls made by the client will use the
wrong marker (which is supposed to be the last instance id), ending
up counting the same instances twice.

Change-Id: I6c7a67b23ec49aa207c33c38580acd834bb27e3c
Closes-Bug: #1796689
2018-10-26 14:47:52 +00:00
Balazs Gibizer
3a43a931d4 consumer gen: more tests for delete allocation cases
User confirming migration as well as a successfull live migration also
triggers the delete allocation code path. This patch adds test coverage
for these code paths.

If the deletion of the source allocation of a confirmed migration fails
then nova puts the instance to ERROR state. The instance still has two
allocations in this state and deleting the instance only deletes the one that
is held by the instance_uuid. This patch logs an ERROR describing that in this
case the allocation held by the migration_uuid is leaked. The same true
for live migration failing to delete allocaton on the source host.

As this makes every caller of _delete_allocation_after_move logging the
same error for AllocationDeleteFailed exception this patch moves that logging
into _delete_allocation_after_move.

Blueprint: use-nested-allocation-candidates
Change-Id: I99427a52676826990d2a2ffc82cf30ad945b939c
2018-09-26 13:30:07 +02:00
Balazs Gibizer
53ca096750 consumer gen: move_allocations
This patch renames the set_and_clear_allocations function in the
scheduler report client to move_allocations and adds handling of
consumer generation conflict for it. This call now moves everything from
one consumer to another and raises AllocationMoveFailed to the caller if
the move fails due to consumer generation conflict.

When migration or resize fails to move the source host allocation to the
migration_uuid then the API returns HTTP 409 and the migration is aborted.

If reverting a migration, a resize, or a resize to same host fails to move
the source host allocation back to the instance_uuid due consumer generation
conflict the instance will be put into ERROR state. The instance still has two
allocations in this state and deleting the instance only deletes the one that
is held by the instance_uuid. This patch logs an ERROR describing that in this
case the allocation held by the migration_uuid is leaked.

Blueprint: use-nested-allocation-candidates
Change-Id: Ie991d4b53e9bb5e7ec26da99219178ab7695abf6
2018-09-25 15:56:45 +00:00
Zuul
792cc425f2 Merge "Allow ability for non admin users to use all filters on server list." 2018-09-23 05:17:23 +00:00
Zuul
2274c08460 Merge "Remove deprecated hide_server_address_states option" 2018-09-21 13:58:57 +00:00
Zuul
ca39416b7c Merge "Making instance/migration listing skipping down cells configurable" 2018-09-21 13:53:20 +00:00
Zuul
235d03ca95 Merge "Imported Translations from Zanata" 2018-09-21 13:00:12 +00:00
Zuul
1c1a111e5a Merge "Resource retrieving: add changes-before filter" 2018-09-21 11:48:29 +00:00
OpenStack Proposal Bot
f610b9c5fe Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Icfb9c7bc90d3ac4b3d6f5c6ca604367a1997acf5
2018-09-21 07:23:02 +00:00
Surya Seetharaman
21c5f3e2e5 Making instance/migration listing skipping down cells configurable
Presently if a cell is down, the instances in that cell are
skipped from results. Sometimes this may not be desirable for
operators as it may confuse the users who saw more instances in
their previous listing than now. This patch adds a new api config
option called list_records_by_skipping_down_cells which can be set to
False (True by default) if the operator desires to just return an
API error altogether if the user has any instance in the down cell
instead of skipping. This is essentially a configurable revert of
change I308b494ab07f6936bef94f4c9da45e9473e3534d for bug 1726301 so
that operators can opt into the 500 response behaviour during listing.

Change-Id: Id749761c58d4e1bc001b745d49b6ff0f3732e133
Related-Bug: #1726301
2018-09-20 22:02:26 +02:00
Matt Riedemann
9b69afd457 Remove deprecated hide_server_address_states option
The hide_server_address_states config option and related
policy rule were deprecated in Queens:

  I6040e8c2b3e132b0dfd09f82ae041b4786a63483

They are now removed in Stein as part of the API extension
merge effort.

Part of blueprint api-extensions-merge-stein

Change-Id: Ib3582038274dedbf524ffcaffe818ff0e751489d
2018-09-19 11:36:44 -04:00
zhangbailin
28c1075b59 Resource retrieving: add changes-before filter
This adds the changes-before filter to the servers,
os-instance-actions and os-migrations APIs for
filtering resources which were last updated before
or equal to the given time. The changes-before filter,
like the changes-since filter, will return deleted
server resources.

Part of bp support-to-query-nova-resources-filter-by-changes-before
Change-Id: If91c179e3823c8b0da744a9363906b0f7b05c326
2018-09-19 09:56:56 -04:00
Kashyap Chamarthy
e155baefb0 libvirt: Use 'virt' as the default machine type for ARMv7
The existing Nova's default machine type for ARMv7 ('vexpress-15') was
added more than four years ago (in commit: 5b27fe7: "libvirt: Allow
specification of default machine type").  The 'vexpress-15' board is a
specific development board, which has hardware limitations (like only
single ethernet adapter).

The upstream QEMU recommendation[1] for the past couple of years is to
use the 'virt' machine type for both ARMv7, and AArch64, which was
specifically designed to be used with virutal machines.  Quoting a
write-up[2] from QEMU's ARM subsystem maintainer:

    "Why the 'virt' board?

    "QEMU has models of nearly 50 different ARM boards, which makes it
    difficult for new users to pick one which is right for their
    purposes.  This wild profusion reflects a similar diversity in the
    real hardware world: ARM systems come in many different flavours
    with very different hardware components and capabilities. A kernel
    which is expecting to run on one system will likely not run on
    another. Many of QEMU’s models are annoyingly limited because the
    real hardware was also limited — there’s no PCI bus on most mobile
    devices, after all, and a fifteen year old development board
    wouldn’t have had a gigabyte of RAM on it.

    "My recommendation is that if you don’t know for certain that you
    want a model of a specific device, you should choose the “virt”
    board. This is a purely virtual platform designed for use in virtual
    machines, and it supports PCI, virtio, a recent ARM CPU and large
    amounts of RAM. The only thing it doesn’t have out of the box is
    graphics, but graphical programs on a fully emulated system run very
    slowly anyway so are best avoided."

So, change the default machine type for ARMv7 arch to be 'virt'.

[1] https://wiki.qemu.org/Documentation/Platforms/ARM
[2] https://translatedcode.wordpress.com/2016/11/03/installing-debian-on-qemus-32-bit-arm-virt-board/

Change-Id: If9ffa5a019f67734a9f30ccaf3ab96ff41262dc8
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
2018-09-17 11:14:26 +02:00
Sam Morrison
7c56588647 Allow ability for non admin users to use all filters on server list.
Adds a new policy rule "os_compute_api:servers:allow_all_filters"
to control whether a user can use all filters when listing servers.

Closes-bug: #1737050

Change-Id: Ia5504da9a00bad689766aeda20255e10b7629f63
2018-09-14 19:04:08 -04:00
zhangshj
3ed77b1a25 fix a spelling error
Change-Id: Ia7f724c756a616b577d6233dd653bcf7b15881f5
2018-09-08 15:23:00 +08:00
Zuul
7116d8daf8 Merge "[placement] Add /reshaper handler for POST" 2018-08-29 09:22:10 +00:00
Zuul
6bf864df77 Merge "Deprecate Core/Ram/DiskFilter" 2018-08-28 12:47:37 +00:00
Zuul
c40bffa530 Merge "Make instance_list perform per-cell batching" 2018-08-28 04:55:17 +00:00
Dan Smith
c3a77f80b1 Make instance_list perform per-cell batching
This makes the instance_list module support batching across cells
with a couple of different strategies, and with room to add more
in the future.

Before this change, an instance list with limit 1000 to a
deployment with 10 cells would generate a query to each cell
database with the same limit. Thus, that API request could end
up processing up to 10,000 instance records despite only
returning 1000 to the user (because of the limit).

This uses the batch functionality in the base code added in
Iaa4759822e70b39bd735104d03d4deec988d35a1
by providing a couple of strategies by which the batch size
per cell can be determined. These should provide a lot of gain
in the short term, and we can extend them with other strategies
as we identify some with additional benefits.

Closes-Bug: #1787977
Change-Id: Ie3a5f5dc49f8d9a4b96f1e97f8a6ea0b5738b768
2018-08-27 06:44:32 -07:00
Matt Riedemann
243ba85130 Deprecate Core/Ram/DiskFilter
The time has come.

These filters haven't been necessary since Ocata [1]
when the filter scheduler started using placement
to filter on VCPU, DISK_GB and MEMORY_MB. The
only reason to use them with any in-tree scheduler
drivers is if using the CachingScheduler which doesn't
use placement, but the CachingScheduler itself has
been deprecated since Pike [2]. Furthermore, as of
change [3] in Stein, the ironic driver no longer
reports vcpu/ram/disk inventory for ironic nodes
which will make these filters filter out ironic nodes
thinking they don't have any inventory. Also, as
noted in [4], the DiskFilter does not account for
volume-backed instances and may incorrectly filter
out a host based on disk inventory when it would
otherwise be OK if the instance is not using local
disk.

The related aggregate filters are left intact for
now, see blueprint placement-aggregate-allocation-ratios.

[1] Ie12acb76ec5affba536c3c45fbb6de35d64aea1b
[2] Ia7ff98ff28b7265058845e46b277317a2bfc96d2
[3] If2b8c1a76d7dbabbac7bb359c9e572cfed510800
[4] I9c2111f7377df65c1fc3c72323f85483b3295989

Change-Id: Id62136d293da55e4bb639635ea5421a33b6c3ea2
Related-Bug: #1787910
2018-08-24 19:32:27 -04:00
Zuul
a34a18b747 Merge "Correct the release notes related to nova-consoleauth" 2018-08-24 07:57:23 +00:00
melanie witt
4f01f4ff88 Correct the release notes related to nova-consoleauth
The release notes said it was okay not to run the nova-consoleauth
service in Rocky, but that's not true because the Rocky code is storing
new console authorization tokens in both the database backend and the
existing nova-consoleauth backend. The use of nova-consoleauth will be
discontinued in Stein (for non-cells v1). We can't remove
nova-consoleauth until we remove cells v1.

Closes-Bug: #1788470

Change-Id: Ibbdc7c50c312da2acc59dfe64de95a519f87f123
2018-08-23 19:37:10 +00:00
Chris Dent
4d525b4ec1 [placement] Add /reshaper handler for POST
/reshaper provides a way to atomically modify some allocations and
inventory in a single transaction, allowing operations like migrating
some inventory from a parent provider to a new child.

A fair amount of code is reused from handler/inventory.py, some
refactoring is in order before things get too far with that.

In handler/allocation.py some code is extracted to its own methods
so it can be reused from reshaper.py.

This is done as microversion 1.30.

A suite of gabbi tests is provided which attempt to cover various
failures including schema violations, generation conflicts, and
data conflicts.

api-ref, release notes and rest history are updated

Change-Id: I5b33ac3572bc3789878174ffc86ca42ae8035cfa
Partially-Implements: blueprint reshape-provider-tree
2018-08-23 00:36:17 +00:00
Zuul
5ffc7f3fb8 Merge "Remove ChanceScheduler" 2018-08-22 22:54:00 +00:00
Sergii Golovatiuk
8b019d6f1e libvirt: Always escape IPv6 addresses when used in migration URI
If IPv6 is passed to URI it should be wrapped within square brackets.
This patch detects IPv6 to form migration URI properly. Domain name, IPv4 or
already bracketed IPv6 address will pass as is

Extend tests to include collapsed IPv6 addresses and IPv6 addresses with
port

Change-Id: I1201db996ea6ceaebd49479b298d74585a78b006
Closes-Bug: #1786058
2018-08-22 09:02:29 -04:00
zhangyangyang
92a459331f Remove ChanceScheduler
ChanceScheduler is deprecated in Pike [1] and will be removed in a
subsequent release.

[1] https://review.openstack.org/#/c/492210/

Change-Id: I44f9c1cabf9fc64b1a6903236bc88f5ed8619e9e
2018-08-20 15:56:13 +01:00
Zuul
c587a85acd Merge "Remove the deprecated API extensions policies" 2018-08-19 07:22:29 +00:00
ghanshyam
f72fa9a739 Remove the deprecated API extensions policies
API extensions policies have been deprecated in 17.0.0
release[1]. This commit removes them.

[1] Ie05f4e84519f8a00ffb66ea5ee920d5c7722a66b
Change-Id: Ib3faf85c78bc2cdee13175560dc1458ddb6cb7a8
2018-08-17 07:11:45 +00:00
Zuul
4027dc8821 Merge "Imported Translations from Zanata" 2018-08-16 05:39:59 +00:00
Doug Hellmann
857b5003cc use static pages for mitaka and newton release notes
The liberty, mitaka, and newton branches are closed so there is no
reason to scan for release notes dynamically. Use static content for
those pages to speed up the release notes build a little bit.

Change-Id: I983346c97df96fda988a2fefec89c3f0d6c14498
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-08-15 16:50:18 -04:00
Jim Rollenhagen
a985e34cde Ironic: report 0 for vcpus/memory_mb/disk_gb resources
Ironic nodes should all be using resource classes for scheduling by now,
which means reporting CPU/RAM/disk isn't useful. Report these as zero so
they cannot be scheduled.

Since we now require resource classes, raise an exception in
update_provider_tree for any nodes that don't have a resource class set
on the node.

Change-Id: If2b8c1a76d7dbabbac7bb359c9e572cfed510800
2018-08-14 21:32:07 +00:00
Zuul
395a227be7 Merge "Revert "libvirt: slow live-migration to ensure network is ready"" 2018-08-13 03:58:27 +00:00
OpenStack Proposal Bot
d5341bde49 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Ie7e0bdd0f7d29fc635004f91d1e4d28cc24e1c70
2018-08-12 07:33:01 +00:00
Dan Smith
411c45842f Revert "libvirt: slow live-migration to ensure network is ready"
This reverts commit 8e6d5d404cf49e5b68b43c62e7f6d7db2771a1f4.

As detailed in the bug, this is overly racy, waiting for the event
potentially a long ways (and a couple RPC calls) away from where
the event will be triggered. The compute manager now has a generic
mechanism to do this which conflicts and replaces this functionality,
if enabled (default is off in Rocky, to be defaulted to on in Stein).

Conflicts:
	nova/tests/unit/virt/libvirt/test_driver.py
	nova/virt/libvirt/driver.py

Change-Id: Ibf2b5eeafd962e93ae4ab6290015d58c33024132
Closes-Bug: #1786346
2018-08-10 12:18:08 -07:00
4e62958802 Update reno for stable/rocky
Change-Id: Ieb01aff6be1d747a94b29391821aed23d33730ae
2018-08-10 09:53:55 +00:00
Zuul
ddc7c466d7 Merge "Add a prelude release note for the 18.0.0 Rocky GA" 2018-08-09 20:39:46 +00:00
melanie witt
8bb8045b59 Add a prelude release note for the 18.0.0 Rocky GA
Change-Id: Iafcf8b6d90bb3d4607b1f55bab0e39bec52c6928
2018-08-09 16:16:29 +00:00
Zuul
509a2cca24 Merge "conf: Deprecate 'network_manager'" 2018-08-08 17:40:38 +00:00
Zuul
8688b25ca7 Merge "Deprecate upgrade_levels options for deprecated/removed services" 2018-08-04 19:00:31 +00:00
Takashi NATSUME
198885522c Reno for notification-transformation-rocky
Add a release note for blueprint
versioned-notification-transformation-rocky.

Change-Id: Ide95e93c2bcbfb8449fda9ee329409f71703ac01
Implements: bp versioned-notification-transformation-rocky
2018-08-03 11:57:09 -04:00
Matt Riedemann
f0d2925bc7 Deprecate upgrade_levels options for deprecated/removed services
* nova-cert was removed in Pike: 2bcee77e3

* nova-network has been deprecated since newton: 7d5fc4868

* nova-consoleauth was deprecated in rocky: e91dad3ee

So start the timer on the related upgrade_levels options
so we can remove them.

Change-Id: Ied416335d36fe3324fb1c5c78353affe1332ba90
2018-08-03 15:53:11 +00:00
Zuul
7eda21891b Merge "scheduler: Start utilizing RequestSpec.network_metadata" 2018-07-29 02:42:31 +00:00
Zuul
63c1f9a17a Merge "Add nova-status upgrade check for request spec migrations" 2018-07-28 02:39:19 +00:00
Zuul
77aa959896 Merge "Remove support for /os-floating-ip-dns REST API" 2018-07-26 01:14:04 +00:00
Zuul
cdf6560435 Merge "Remove support for /os-floating-ips-bulk REST API" 2018-07-26 01:11:33 +00:00
Zuul
99d2a34d1f Merge "Add nova-manage placement sync_aggregates" 2018-07-25 18:56:26 +00:00
Zuul
c2ef1c0863 Merge "Trim the fat on HostState.instances" 2018-07-24 21:18:50 +00:00
Stephen Finucane
803f85d7e6 scheduler: Start utilizing RequestSpec.network_metadata
Now that we have this information, we can use it as a pre-filtering for
suitable hosts.

With this patch we complete the blueprint. As a result, documentation
and release notes are bundled in the patch and previously inactive tests
are now enabled.

Part of blueprint numa-aware-vswitches

Change-Id: Ide262733ffd7714fdc702b31c61bdd42dbf7acc3
2018-07-24 15:05:07 -04:00
Zuul
88fb15c75c Merge "Add method to get cpu traits" 2018-07-24 16:42:38 +00:00
Matt Riedemann
aa6360d683 Add nova-manage placement sync_aggregates
This adds the "nova-manage placement sync_aggregates"
command which will compare nova host aggregates to
placement resource provider aggregates and add any
missing resource provider aggregates based on the nova
host aggregates.

At this time, it's only additive in that the command
does not remove resource provider aggregates if those
matching nodes are not found in nova host aggregates.
That likely needs to happen in a change that provides
an opt-in option for that behavior since it could be
destructive for externally-managed provider aggregates
for things like ironic nodes or shared storage pools.

Part of blueprint placement-mirror-host-aggregates

Change-Id: Iac67b6bf7e46fbac02b9d3cb59efc3c59b9e56c8
2018-07-24 11:19:23 -04:00