1187 Commits

Author SHA1 Message Date
Zuul
8a407bd288 Merge "Block deleting compute services which are hosting instances" 2018-04-20 05:55:36 +00:00
Zuul
e3a9efab86 Merge "Add hide_hypervisor_id flavor extra_spec" 2018-04-20 05:21:38 +00:00
Zuul
d079d26444 Merge "Log a warning and add nova-status check for old API service versions" 2018-04-20 05:21:00 +00:00
Zuul
b1dcfd8159 Merge "Provide framework for setting placement error codes" 2018-04-20 00:33:21 +00:00
Zuul
1622e23d20 Merge "xenapi: Use XAPI pool instead of aggregate pool for shared SR migration" 2018-04-19 23:28:14 +00:00
Konstantinos Samaras-Tsakiris
edf67cfda2 Add hide_hypervisor_id flavor extra_spec
Blueprint add-kvm-hidden-feature added the capability of hiding the kvm
signature from guests. However, it was implemented only through an image
property.

A major reason for this feature is to allow passed-through Nvidia GPUs
to work correctly. GPU pci-passthrough is specified on the flavor's
extra_specs, without requiring an image with special properties.
Therefore, hiding the KVM signature should also be specifiable through
the flavor's extra_specs, in order to not require a special image for
this use case.

If the new flavor extra_spec is present and set to 'true', the libvirt
driver will produce an additional element to hide kvm's signature on
the vm, in the same way as with the image property
`img_hide_hypervisor_id`.

Implements: blueprint hide-hypervisor-id-flavor-extra-spec
Closes-Bug: 1757424
Change-Id: I41c5913b4148629b448ea5fb43b7597dc067dc22
2018-04-19 22:36:26 +00:00
Zuul
318d353d64 Merge "conf: Remove deprecated 'multi_instance_display_name_template' opt" 2018-04-19 08:31:52 +00:00
naichuans
9b7affcac1 xenapi: Use XAPI pool instead of aggregate pool for shared SR migration
xenapi is going to support pool-based multi-hosts OpenStack
environments, this patch is used to remove dependency to the
old aggregate-based-pools and add support to xenapi pool.
Also include unit test changes.
Updated configuring migrations document:
https://docs.openstack.org/nova/latest/admin/configuring-
migrations.html#configuring-migrations-xenserver-shared-storage
Other related documents will be updated in another patch.

Implements: blueprint live-migration-in-xapi-pool
Change-Id: I2c492c46e85c1df96aa7fdc12cdee0b1c7ba775e
2018-04-19 05:50:31 +00:00
Matt Riedemann
42f62f1ed2 Block deleting compute services which are hosting instances
This change makes "DELETE /os-services/{service_id}" fail
with a 409 response when attempting to delete a nova-compute
service which is still hosting instances.

Deleting a compute service also results in deleting the
related compute_nodes table entry for that service host.
The compute node resource provider in placement is tied
to the compute node via the UUID, and if we allow deleting
the compute service and node then the resource provider for
that node is effectively orphaned in Placement, along with
the instances which have allocations against that resource
provider.

Furthermore, restarting the compute service will create a
new service and compute_nodes record, and the compute node
would have a new UUID and resource provider. This will
affect scheduling for that host since Placement will be
reporting it as having available capacity which in reality
is not accurate.

A release note is included for the (justified) behavior
change in the API. A new microversion should not be required
for this since admins should not have to opt out of broken
behavior. Since this API did not previously expect to return
a 409 response, the "expected_errors" decorator is updated
and again, should not require a microversion per the
guidelines:

https://docs.openstack.org/nova/latest/contributor/microversions.html#when-a-microversion-is-not-needed

Change-Id: I0bd63b655ad3d3d39af8d15c781ce0a45efc8e3a
Closes-Bug: #1763183
2018-04-18 19:16:00 -04:00
Chris Dent
2c51688558 Parse forbidden in extra_specs
Accept forbidden traits in the processing of extra_specs, with the
format of:

      trait:CUSTOM_MAGIC=forbidden

This will be transformed into required=!CUSTOM_MAGIC when the traits
are assembled into a request to GET /allocation_candidates.

Implements blueprint forbidden-traits-in-nova

Change-Id: I31e609aef47d2fea03f279e4bfdd30f072d062b4
2018-04-17 18:18:43 +01:00
Chris Dent
bd9f24b7aa Provide framework for setting placement error codes
The API-sig has a guideline[1] for including error codes in error
responses to help distinguish errors with the same status code
from one another. This change provides a simplest-thing-that-could-
possibly-work solution to make that go.

This solution comes about after a few different constraints and attempts:

* We would prefer to go on using the existing webob.exc exceptions, not
  make subclasses.
* We already have a special wrapper around our wsgi apps to deal with
  setting the json_error_formatter.
* Though webob allows custom Request and Response objects, it uses the
  default Response object as the parent of the HTTP exceptions.
* The Response object accepts kwargs, but only if they can be associated
  with known attributes on the class. Since we can't subclass...
* The json_error_formatter method is not passed the raw exception, but
  it does get the current WSGI environ
* The webob.exc classes take a 'comment' kwarg that is not used, but
  is also not passed to the json_error_formatter.

Therefore, when we raise an exception, we can set 'comment' to a code
and then assign that comment to a well known field in the environ and if
that environ is set in json_error_formatter, we can set 'code' in the
output.

This is done in a new microversion, 1.23. Every response gets a default
code 'placement.undefined_code' from 1.23 on. Future development will
add specific codes where required. This change adds a stub code for
inventory in use when doing a PUT to .../inventories but the name
may need improvement.

[1] http://specs.openstack.org/openstack/api-wg/guidelines/errors.html

Implements blueprint placement-api-error-handling

Change-Id: I9a833aa35d474caa35e640bbad6c436a3b16ac5e
2018-04-14 13:45:54 +01:00
Zuul
0b0754c64c Merge "[placement] Support forbidden traits in API" 2018-04-14 00:48:44 +00:00
Zuul
c6d7d92c93 Merge "Add microversion to support extra_specs in flavor API." 2018-04-13 22:47:15 +00:00
Chris Dent
4e07d81260 [placement] Support forbidden traits in API
In a new microversion (1.22) expose support for processing
forbidden traits in GET /resource_providers and GET
/allocation_candidates. A forbidden trait is expressed as
part of the required parameter with a "!" prefix:

    required=CUSTOM_FAST,!CUSTOM_SLOW

This change uses db and query processing code adjustments
already present in the code but guarded by a flag. If the
currently requested microversion matches 1.22 or beyond
that flag is True, otherwise False.

Reno, api-ref update and api history update are included.
Because this microversion changes the value of an existing
parameter it was unclear how to best express that in the
api-ref. In this case existing parameter references were
annotated.

Partially implements blueprint placement-forbidden-traits

Change-Id: I43e92bc5f97db7a2b09e64c6cb953c07d0561e63
2018-04-13 19:24:08 +01:00
Michael Still
7b43fb4ebd Move xenapi disk resizing to privsep.
The same pattern as the rest of the changes. This means that privsep now
needs to let you pass flags to e2fsck, which I don't love and will remove
in a later patch.

Change-Id: I6c695c04ae586fec6adc354257638116277dda88
blueprint: hurrah-for-privsep
2018-04-13 07:09:58 +10:00
Zuul
2c5da2212c Merge "Imported Translations from Zanata" 2018-04-11 17:57:13 +00:00
Zuul
7891acf809 Merge "Marker reset option for nova-manage map_instances" 2018-04-11 17:57:04 +00:00
Zuul
5244d25f89 Merge "Expose shutdown retry interval as config setting" 2018-04-11 16:41:42 +00:00
Yikun Jiang
0baba40b1b Add microversion to support extra_specs in flavor API.
Exposes flavor extra_specs in the flavor representation since
microversion 2.61. Now users can see the flavor extra-specs
in flavor APIs response only and do not need to call
``GET /flavors/{flavor_id}/extra_specs`` API.

Flavor extra_specs will be included in Response body of the
following APIs:

* ``GET /flavors/detail``
* ``GET /flavors/{flavor_id}``
* ``POST /flavors``
* ``PUT /flavors/{flavor_id}``

Part of blueprint add-extra-specs-to-flavor-list

Change-Id: I048747633babf690a63c6de9773bff5547872053
2018-04-11 17:47:57 +08:00
OpenStack Proposal Bot
dee8de3fab Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I65a0342bf82f9dd99e79ec5640513b48543fce69
2018-04-11 06:17:52 +00:00
Zuul
15f1caf98a Merge "PowerVM: Add proc_units_factor conf option" 2018-04-11 03:21:16 +00:00
esberglu
8a67fcd1c2 PowerVM: Add proc_units_factor conf option
This introduces a new PowerVM conf option, proc_units_factor, which can
range from 0.05 to 1.0 and will default to 0.1. It is used to calculate
the physical processing power to assign per vCPU, where 1.0 is a whole
physical processor and 0.05 is 1/20th of a physical processor.

Change-Id: I67bfe2a6eff86f1947ada7661fc7c3fed81ed28f
2018-04-10 11:24:20 -05:00
Raoul Hidalgo Charman
197539d7a0 Expose shutdown retry interval as config setting
This option is in the driver interface but was hard coded in the manager
class.

Defaults to old value (10 seconds) if not set in configuration file.

Change-Id: I0c8db2efec6098c017aad2f6588938bc548db139
2018-04-10 14:36:01 +01:00
Matt Riedemann
eaf6340847 Log a warning and add nova-status check for old API service versions
Change Ib984c30543acb3ca9cb95fb53d44d9ded0f5a5c8, which was added
in Newton when cells v2 was optional, added some transitional code
to the API for looking up an instance, which didn't rely on instance
mappings in a cell to find the instance if the minimum nova-osapi_compute
service version was from before Ocata.

People have reported this being a source of confusion when upgrading
from before Ocata, when cells v2 wasn't required, to Ocata+ where cells
v2 along with the mapping setup is required. That's because they might
have older nova-osapi_compute service version records in their 'nova'
(cell) database which makes the API think the code is older than it
actually is, and results in an InstanceNotFound error.

This change does two things:

1. Adds a warning to the compute API code in this scenario to serve
   as a breadcrumb if a deployment hits this issue.

2. A nova-status check to look for minimum nova-osapi_compute service
   versions across all cells and report the issue as a warning. It's
   not an upgrade failure since we don't know how the nova-api service
   is configured, but leave that investigation up to the deployer.

This is also written in such a way that we should be able to backport
this through to stable/ocata.

Change-Id: Ie2bc4616439352850cf29a9de7d33a06c8f7c2b8
Closes-Bug: #1759316
2018-04-09 14:43:34 -04:00
Zuul
e2d5dc4e2c Merge "Support extending attached ScaleIO volumes" 2018-04-09 15:18:22 +00:00
Zuul
8e6ecdb9dd Merge "conf: Remove 'db_driver' config opt" 2018-04-09 11:46:46 +00:00
Zuul
887dc23a17 Merge "libvirt: Allow to specify granular CPU feature flags" 2018-04-06 15:00:01 +00:00
Zuul
0120643def Merge "Add --enable and --disable options to nova-manage update_cell" 2018-04-05 23:16:13 +00:00
Zuul
9204f91c30 Merge "Add nova-status check for ironic flavor migration" 2018-04-05 17:38:00 +00:00
Kashyap Chamarthy
6b601b7cf6 libvirt: Allow to specify granular CPU feature flags
The recent "Meltdown" CVE fixes have resulted in a critical performance
penalty[*] that will impact every Nova guest with certain CPU models.

I.e. assume you have applied all the "Meltdown" CVE fixes, and performed
a cold reboot (explicit stop & start) of all Nova guests, for the
updates to take effect.  Now, if any guests that are booted with certain
named virtual CPU models (e.g. "IvyBridge", "Westmere", etc), then those
guests, will incur noticeable performance degradation[*], while being
protected from the CVE itself.

To alleviate this guest performance impact, it is now important to
specify an obscure Intel CPU feature flag, 'PCID' (Process-Context ID)
-- for the virtual CPU models that don't already include it (more on
this below).  To that end, this change will allow Nova to explicitly
specify CPU feature flags via a new configuration attribute,
`cpu_model_extra_flags`, e.g. in `nova.conf`:

    ...
    [libvirt]
    cpu_mode = custom
    cpu_model = IvyBridge
    cpu_model_extra_flags = pcid
    ...

NB: In the first iteration, the choices for `cpu_model_extra_flags` is
restricted to only 'pcid' (the option is case-insensitive) -- to address
the earlier mentioned guest performance degradation.  A future patch
will remove this restriction, allowing to add / remove multiple CPU
feature flags, thus making way for other useful features.

Some have asked: "Why not simply hardcode the 'PCID' CPU feature flag
into Nova?"  That's not graceful, and more importantly, impractical:

  (1) Not every Intel CPU model has 'PCID':

       - The only Intel CPU models that include the 'PCID' capability
         are: "Haswell", "Broadwell", and "Skylake" variants.

       - The libvirt / QEMU Intel CPU models: "Nehalem", "Westmere",
         "SandyBridge", and "IvyBridge" will *not* expose the 'PCID'
         capability, even if the host CPUs by the same name include it.
         I.e. 'PCID' needs to be explicitly when using the said virtual
         CPU models.

  (2) Magically adding new CPU feature flags under the user's feet
      impacts live migration.

[*] https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU

Closes-Bug: #1750829
Change-Id: I6bb956808aa3df58747c865c92e5b276e61aff44
BluePrint: libvirt-cpu-model-extra-flags
2018-04-05 16:05:27 +02:00
Eric Young
ce90bec66f Support extending attached ScaleIO volumes
Adding ability to extend ScaleIO volumes which are
attached to an instance.

Change-Id: Ib2debfe481d4c1921d14546f81b60d070e8f9f39
2018-04-05 09:01:16 -04:00
Zuul
4ba2249f95 Merge "Move xenapi xenstore_read's to privsep." 2018-04-05 09:35:25 +00:00
Zuul
12d0b49ac2 Merge "Move configurable mkfs to privsep." 2018-04-05 09:35:17 +00:00
Matt Riedemann
7eb6703521 Add nova-status check for ironic flavor migration
In Pike we started requiring that ironic instances have their
embedded flavor migrated to track the ironic node custom
resource class. This can be done either via the normal running
of the nova-compute service and ironic driver or via the
'nova-manage db ironic_flavor_migration' command.

This change adds a nova-status check to see if there are any
unmigrated ironic instances across all non-cell0 cells, and
is based mostly on the same logic within the nova-manage command
except it's multi-cell aware and doesn't use the objects.

Change-Id: Ifd22325e849db2353b1b1eedfe998e3d6a79591c
2018-04-04 17:23:06 -04:00
Surya Seetharaman
cd01cbe65e Add --enable and --disable options to nova-manage update_cell
Through these new options, users can enable or disable a cell
through the CLI.

Related to blueprint cell-disable

Change-Id: I761f2e2b1f1cc2c605f7da504a8c8647d6d6a45e
2018-04-04 20:23:51 +00:00
Zuul
2b05ee0226 Merge "Fix typos in release notes" 2018-04-04 18:33:47 +00:00
Zuul
2474f44b17 Merge "PowerVM Driver: Network interface attach/detach" 2018-04-04 06:49:19 +00:00
Michael Still
cc0d8ab9ff Move xenapi xenstore_read's to privsep.
Self explainatory really.

Change-Id: I1d63dd771e4f44ac131c37563bff42b970c44a09
blueprint: hurrah-for-privsep
2018-04-04 06:29:33 +10:00
Michael Still
0751ee19d8 Move configurable mkfs to privsep.
Nova allows deployers to configure the command line which is used to create
a filesystem of a given type. This is frankly a little bit weird, but its
also historical. Move this functionality to privsep, including doing a
dance at startup to load config flags into privsep in a hopefully secure
manner.

Honestly, all of this code should be deprecated, but that's above my pay
grade and would take time to do. Oh, and maybe deployers love it the way
it is.

Change-Id: Id8eeb21e10f98a448946f178c8c5a36e48c7cac6
blueprint: hurrah-for-privsep
2018-04-04 06:29:32 +10:00
Zuul
9c929662b3 Merge "Allow scheduling only to enabled cells (Filter Scheduler)" 2018-04-03 15:33:43 +00:00
Surya Seetharaman
ba083b0c98 Allow scheduling only to enabled cells (Filter Scheduler)
This by default would be scheduling to all cells since all cells
would be enabled at the time of creation unless specified otherwise.

Since the list of enabled cells are stored as a global cache on the
host_manager, a reset() handler for the SIGHUP signal has also been
added in the scheduler. Hence upon every create-cell/enable-cell/
disable-cell operation the scheduler has to be signaled so that the
cache is refreshed.

Co-Authored-By: Dan Smith <dms@danplanet.com>

Implements blueprint cell-disable

Change-Id: I6a9007d172b55238d02da8046311f8dc954703c5
2018-03-30 19:16:14 -04:00
Dan Smith
a27da62d82 Documentation for tenant isolation with placement
This explains how to actually wire up placement aggregates to allow
for filtering on tenant.

Change-Id: Idb06e7562d88957a00f52cba7d0a788dbff42a28
2018-03-29 11:56:39 -07:00
esberglu
ed525cc403 PowerVM Driver: Network interface attach/detach
This adds the ability to hotplug network interfaces for the powervm
virt driver.

Blueprint: powervm-network-hotplug
Change-Id: I78b94c9731c35e3291d46b9bf9f5554e21c2429e
2018-03-29 09:06:09 -04:00
Dan Smith
732e202e81 Add require_tenant_aggregate request filter
This adds a require_tenant_aggregate request filter which uses overlaid
nova and placement aggregates to limit placement results during scheduling.
It uses the same `filter_tenant_id` metadata key as the existing scheduler
filter we have today, so people already doing this with that filter will
be able to enable this and get placement to pre-filter those hosts for
them automatically.

This also allows making this filter advisory but not required, and supports
multiple tenants per aggregate, unlike the original filter.

Related to blueprint placement-req-filter

Change-Id: Idb52b2a9af539df653da7a36763cb9a1d0de3d1b
2018-03-28 15:58:46 -07:00
Zuul
9c7ebf90fa Merge "Add disabled option to create_cell command" 2018-03-26 13:26:35 +00:00
Surya Seetharaman
9e9520a4b0 Add disabled option to create_cell command
Specifying this new option will create pre-disabled cells.

Related to blueprint cell-disable

Change-Id: I87e06eeefd48f640f42fcc5ff5a48938adc2ac22
2018-03-23 20:26:02 +00:00
Zuul
9273b08202 Merge "deprecate fping_path config option" 2018-03-23 14:21:27 +00:00
jichenjc
497e0321f1 deprecate fping_path config option
/os-fping was deprecated at API version 2.36, ``fping_path`` was
only used by this API so it should be able to be deprecated.

Change-Id: I7d3faae0013315d595386ff262cadf8b18f70c68
2018-03-21 15:06:51 +08:00
Matt Riedemann
6482165bb1 Handle EndpointNotFound when building image_ref_url in notifications
Change I4e755b9c66ec8bc3af0393e81cffd91c56064717 made the
[glance]/api_servers option optional. If not set, we attempt
to get the image service endpoint via keystoneauth adapter and
the service catalog via the request context.

Periodic tasks run without an actual token so there is no way
to get the service catalog and the KSA adapter code to get the
endpoint raises EndpointNotFound when trying to build the
"image_ref_url" entry in the legacy instance notification payload.

This change simply handles the EndpointNotFound and sets the
image_ref_url to the instance.image_ref, which for non-volume-backed
instances is the image id (for volume-backed instances it's an empty
string).

This doesn't affect versioned notifications since those don't use the
"image_ref_url" entry in the payload that is created, they just have
an "image_uuid" entry in the versioned notification payload which is
populated via instance.image_ref.

An upgrade impact release note is added in the case that some consuming
service is actually relying on that legacy notification field being
a URL and parsing it as such. The thinking here, however, is this is
better than not sending the field at all, or setting it to None.

Long-term this code all gets replaced with versioned notifications.

Change-Id: Ie23a9c922776b028da0720c939846cba233ac472
Closes-Bug: #1753550
2018-03-20 17:22:50 -04:00
zhufl
0e061491aa Fix typos in release notes
This is to fix below typos in release notes:
  Keytone
  specifed
  availabilty
  maange
  expetected
  migratons
  maintanance

Change-Id: Ifbecf095f2f549d4ec40892484ec1b725927fb44
2018-03-19 18:02:30 +00:00