832 Commits

Author SHA1 Message Date
Radoslav Gerganov
fc0c6d2f9a VMware: add support for different firmwares
Add support for explicitly requiring UEFI or BIOS firmware from
images by taking hw_firmware_type metadata into account.

blueprint vmware-boot-uefi

Change-Id: Ibfb57c55569e430a846003472a49a536e94e6a48
2018-01-03 19:23:26 +00:00
Jenkins
fc454044d6 Merge "hardware offload support for openvswitch" 2017-08-03 19:08:06 +00:00
Moshe Levi
ccc1def0f5 hardware offload support for openvswitch
In Kernel 4.8 we introduced Traffic Control (TC see [1]) hardware offloads
framework for SR-IOV VFs which allows us to configure the NIC [2].
Subsequent OVS patches [3] allows us to use the TC framework
to offload OVS datapath rules. This feature is supported in OVS 2.8.0.

This patch adds support for VIF_TYPE_OVS with vnic_type direct

blueprint sriov-ovs-offload

[1] http://netdevconf.org/1.2/papers/efraim-gerlitz-sriov-ovs-final.pdf
[2] https://patchwork.ozlabs.org/patch/738176/
[3] https://mail.openvswitch.org/pipermail/ovs-dev/2017-April/330606.html

Change-Id: I3f38954bc5cf7b1690182dc8af45078eea275aa4
2017-08-03 13:30:30 +00:00
Jenkins
55a832b4cc Merge "Accept any scheduler driver entrypoint" 2017-08-02 16:57:52 +00:00
Jenkins
8e56bdb1dc Merge "Add release note for PUT /os-services/* for non-compute services" 2017-08-02 10:00:54 +00:00
Sylvain Bauza
1e5c7b52a4 Accept any scheduler driver entrypoint
We broke the possibility in Ocata with Icdcf839b6d28893694bfa1355e9dbe8dbb5ea8c3
to use other scheduler drivers but the ones we provided in tree.

Unfortunately, that was an incidental change without any communication.

Removing the choices kwarg will allow operators to run their own scheduler driver.
Whether Nova would stop supporting custom drivers would require a totally separate
change which would clearly communicate thru a deprecation notice but that is not
the intent for that bugfix, which aims only to bring back the capability.

Change-Id: I346881bc3bc48794b139cc471be1de11c49b8ee3
Closes-Bug: #1704788
2017-08-01 16:50:10 +00:00
Matt Riedemann
3414ab14ca Deprecate cells v1
With multi-cell support in Pike, we should deprecate cells v1
so we can at least start the deprecation signaling in the
docs and release notes. We may not end up removing cells v1
code in Queens, but this at least gives us the option.

Note that we also want to do this because nova-network cannot
be removed until we remove cells v1.

Change-Id: I1a173f7ce0715e684850e030c358e8175fa8724c
2017-08-01 11:07:55 -04:00
Matt Riedemann
598314a4bb Add release note for PUT /os-services/* for non-compute services
Change If1e03c9343b8cc9c34bd51c2b4d25acdb21131ff made the
"PUT /os-services/*" APIs only work with nova-compute services
since they now rely on finding the compute service in a cell
via the host_mappings record in the API database, and host
mappings only exist for compute services.

No one should have relied on disabling a non-compute service since
that does nothing, at least with in-tree code.

This change just adds a release note for awareness.

Change-Id: Idb5bbc31a0aabc328f11e5a9979e5b80651a8f67
Closes-Bug: #1700359
2017-08-01 10:21:43 -04:00
Jay Pipes
23c4eb3438 claim resources in placement API during schedule()
Adds logic to call the placement API's PUT /allocations/{consumer_uuid}
when selecting hosts in the filter scheduler's _schedule() method.

We only attempt the claim of resources if and only if the scheduler
driver uses allocation candidates (i.e. it isn't the caching scheduler)
and the conductor has passed in a list of instance UUIDs (otherwise,
there's no way to allocate in the placement API).

Change-Id: Ifc5cf482209e4f6f4e3e39b24389bd3563d86444
blueprint: placement-claims
2017-07-27 17:56:28 -07:00
Jenkins
d38c6f489b Merge "deprecate `wsgi_log_format` config variable" 2017-07-27 02:07:01 +00:00
Sean Dague
27c341c1bf deprecate `wsgi_log_format` config variable
Running under a real wsgi environment, such as uwsgi or apache /
mod_wsgi is preferred now, and there is a different logging solution
for that. This was always an eventlet leaky abstraction. So we'll just
deprecate this for now, and the code can be removed once eventlet on
the API side is removed.

Change-Id: I934a1693cbfd539d8575d022519510d19d13aaab
2017-07-26 21:14:44 +00:00
Jan Gutter
2bd008ef5c Netronome SmartNIC Enablement
These patches enable Netronome Agilio SmartNIC acceleration in Nova.

* This patch set adds VIF_TYPE_AGILIO_OVS to hook in the external ML2
  plugin.

* This patch set adds VNIC_TYPE_VIRTIO_FORWARDER to enable the
  virtio-forwarder plugging mode.

* nova/network/neutronv2/api.py has been modified to pass the bridge
  in the os-vif objects, similar to other OVS VIF types.

* os_vif_util.py has been extended to handle the agilio_ovs os-vif
  plugin.

* VIFPortProfileOVSRepresentor is used to pass the PCI address and vif
  name to the os-vif plugin.

* Note: this enables the external ML2 and OS-VIF plugin, hosted at
  https://github.com/Netronome/agilio-ovs-openstack-plugin
  to support the out-of-tree version of Agilio OVS. Consult the Agilio
  documentation for configuration recommendations.

Change-Id: I3da059fb62f085ff04532f8ec05857acfb628b06
Depends-On: Id95a9f311e71b3cefb170704728641a4a1a91144
Signed-off-by: Jan Gutter <jan.gutter@netronome.com>
2017-07-26 19:03:04 +00:00
Jenkins
bed59a47b7 Merge "Ironic: Support boot from Cinder volume" 2017-07-25 15:18:35 +00:00
Jenkins
62c4535a85 Merge "request_log addition for running under uwsgi" 2017-07-25 04:47:19 +00:00
Satoru Moriya
3e1a3c9f82 Ironic: Support boot from Cinder volume
This enables Ironic to boot bare metal machines from Cinder
volume. Ironic virt driver needs to pass the remote volume
connection information down to Ironic when spawning a new
bare metal instance requested to boot from a Cinder volume.

This implements get_volume_connector method for the Ironic
driver. It will get connector information from the Ironic service
and pass it to Cinder's initialize_connection method for attached
volumes. And then it puts the returned value into Ironic.

This patch changes the required Ironic API version to 1.32 for using
new API for volume resources.

Co-Authored-By: Satoru Moriya <satoru.moriya.br@hitachi.com>
Co-Authored-By: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
Change-Id: I319779af265684715f0142577a217ab66632bf4f
Implements: blueprint ironic-boot-from-volume
2017-07-25 03:57:51 +00:00
Jenkins
19f78e2f5f Merge "Add online migration to move quotas to API database" 2017-07-25 02:07:57 +00:00
melanie witt
b291061cb1 Add online migration to move quotas to API database
This moves quota limits and classes from the main database to the
API database in an online migration.

Part of blueprint cells-quota-api-db

Change-Id: I64b600b30f6e54db0ec9083c6c176e895c6d0cc2
2017-07-24 18:36:46 -04:00
Sean Dague
aa45a6f3ab request_log addition for running under uwsgi
If someone chooses to run under uwsgi/apache instead of eventlet, the
basic logging of requests goes away (as that was an eventlet.wsgi
function). This is a critical piece of information for understanding
how services are working, and we need to retain it under uwsgi/apache.

This creates a new request_log middleware, inspired by the one in
placement, to provide that functionality. This includes all the same
information as before: http method, uri, status, content length, time
for the request. It also includes the microversion the request was
processed as, "-" if no microversion.

The middleware does not emit anything if it detects that it's running
under eventlet, to prevent duplicate log messages.

Release notes provided as this will be a manual transition for folks
as it's an api-paste.ini change.

Change-Id: I3a597b06d3501c765e2d7805c6c1375d6f4e40db
2017-07-24 14:23:38 -04:00
Jenkins
716dabdb0e Merge "Deprecate useless quota_usage_refresh from nova-manage" 2017-07-23 16:52:16 +00:00
Jenkins
422f9b5646 Merge "Implement interface attach/detach in ironic virt driver" 2017-07-23 14:14:24 +00:00
Jenkins
8df1014a22 Merge "doc: Switch to openstackdocstheme" 2017-07-21 20:16:48 +00:00
Jenkins
fe46cfa301 Merge "Change default policy to view quota details" 2017-07-21 20:02:20 +00:00
Andrey Volkov
dcc2934921 Change default policy to view quota details
Default policy for quota details was an admin only privilege
but it was able to get used and reserved resource data via
nova limits call even for non admin owners.

Horizon worked around the issue by using the limits API instead of the
os-quota-sets API. And this removes the need to workaround it and just
use the single os-quota-sets API.

This patch sets the quotas detail default policy to RULE_ADMIN_OR_OWNER.

Change-Id: I65b9de24a1310079a67e033606eaf2dde796cd48
Closes-Bug: #1618513
2017-07-21 15:09:23 +00:00
Vasyl Saienko
66f962d9b9 Implement interface attach/detach in ironic virt driver
This patch implements attach_interface, detach_interface
methods in Ironic virt driver.

Implements: blueprint ironic-hotplug-interfaces
Co-Authored-By: Vladyslav Drok <vdrok@mirantis.com>
Depends-On: Idc9dc37b11950e3aa2864a0a90d6ec2e307926a3
Change-Id: I48c4706b3eb6e0a5105e463236870921d55dbd93
2017-07-21 14:54:57 +00:00
chenxing
fa0e8f69b4 doc: Switch to openstackdocstheme
Change-Id: If7afc2cb58759b16fc6f7caa44d0cf6b7bcf4d06
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2017-07-21 11:35:48 +01:00
Jenkins
4771fe2d8a Merge "Use URIOpt" 2017-07-20 04:04:56 +00:00
Jenkins
15cac9d7e5 Merge "Handle uuids in os-hypervisors API" 2017-07-20 01:03:27 +00:00
Jenkins
d3e6b94eb9 Merge "Use uuid for id in os-services API" 2017-07-20 01:02:42 +00:00
Jenkins
854053ab47 Merge "Removes getfattr from Quobyte Nova driver" 2017-07-19 20:16:21 +00:00
Dan Smith
f2606322f7 Deprecate useless quota_usage_refresh from nova-manage
Since we no longer use usage records, we don't need the nova-manage
command to sync usage records with actual resource consumption.

Co-Authored-By: melanie witt <melwittt@gmail.com>

Part of blueprint cells-count-resources-to-check-quota-in-api

Change-Id: I55f7a2b6e15eec6e6ec97be23903325ad22bd4d4
2017-07-19 19:33:23 +00:00
Jenkins
5c322924fa Merge "conf: remove *_topic config opts" 2017-07-19 19:32:48 +00:00
Matt Riedemann
622bfb2e95 Handle uuids in os-hypervisors API
There are quite a few changes here as this is not only handling
uuids for the hypervisor id but it's also a refactor in several
APIs for consistency.

The main changes are detailed in the REST API Version History
doc in this change, but to summarize the changes:

* Hypervisor and service IDs are handled as the UUIDs for those
  resources; this is necessary for accurately working with these
  resources across multiple cells.
* The 'servers' and 'search' routes are deprecated and folded into
  the index and detail methods as query parameters, validated using
  json schema.
* The show method will also be able to return the list of servers
  hosted on the given hypervisor using the with_servers query
  parameter.
* The marker used when paging over lists of hypervisors is the
  compute node UUID.
* Using the hypervisor_hostname_pattern query parameter will not
  work with paging parameters.
* API reference docs are updated for the detailed changes.
* Functional and unit tests are provided for all changes.

Part of blueprint service-hyper-uuid-in-api

Change-Id: I828350c179df8bcfa4739910abeafaba2f96982b
2017-07-18 17:08:01 -04:00
Dan Peschman
2f7bf29d47 Use uuid for id in os-services API
This patch introduces a new microversion to identify services by uuid
instead of id, to ensure uniqueness across cells. GET /os-services
returns uuid in the id field, and uuid must be provided to delete a
service with DELETE /os-services/{service_uuid}.

The old PUT /os-services/* APIs are now capped and replaced
with a new PUT /os-services/{service_uuid} which takes a uuid path
parameter to uniquely identify the service to update. It also restricts
updates to nova-compute services only, since disabling or forcing-down
a non-compute service like nova-scheduler doesn't make sense as it
doesn't do anything.

The new update() method in this microversion also avoids trying to
re-use the existing private action methods like _enable and _disable
since those are predicated on looking up the service by host/binary,
are confusing to follow for code flow, and just don't really make sense
with a pure PUT resource update method.

Part of blueprint service-hyper-uuid-in-api

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Change-Id: I45494a4df7ee4454edb3ef8e7c5817d8c4e9e5ad
2017-07-18 15:39:57 -04:00
Jenkins
430ec6504b Merge "Support tag instances when boot(4/4)" 2017-07-18 16:31:31 +00:00
Stephen Finucane
6ef30d5078 conf: remove *_topic config opts
All RPC topic opts were deprecated in Ocata.
This patch removes all *_topic opts from the code.

Change-Id: I41a6be48380999c49d6db3ee5221174a41014002
Implements: blueprint centralize-config-options-pike
2017-07-17 21:27:02 -07:00
Takashi NATSUME
463fd7a11b Use URIOpt
Replace StrOpt with URIOpt.
But 'serial_port_service_uri' in nova/conf/vmware.py
is excluded because it can be non URI format(*1).

*1: https://github.com/openstack/vmware-vspc/blob/master/README.rst

Change-Id: I595c55798b482009c42261c3603a58f82e89877c
Closes-Bug: #1599034
2017-07-18 00:38:24 +00:00
Kevin_Zheng
b50b5a660e Support tag instances when boot(4/4)
This is the 4th patch of the series,
this patch adds a new microversion
in API to support adding tags when
booting instances.

Implemetes: blueprint support-tag-instance-when-boot

Change-Id: Ifcaaf285c8f98a1d0e8bbbc87b2f57fbce057346
2017-07-17 15:59:42 -04:00
He Jie Xu
d0f91817eb Fix the releasenote and api-ref for quota-class API
This patch marks the network related quota in the quota-class API
as deprecated in the 2.50 microversion. Also rewrite the release note for
2.50 microversion, change to describe the API change instead of the history.

Change-Id: Ida5518b7d43e85d9f30b11ed2819025a190aefd6
2017-07-13 07:05:22 +00:00
Mathieu Gagné
bbe0f313bd Add ability to signal and perform online volume size change
Allow Cinder to use external events to signal a volume extension.

1) Nova will then call os-brick to perform the volume extension
   so the host can detect its new size.
2) Compute driver will resize the device in QEMU so instance can detect
   the new disk size without rebooting.

This change:

* Adds the 'volume-extended' external event.
  The event tag needs to be the extended volume id.
* Bumps the latest microversion to 2.51.
* Exposes non-traceback instance action event details for
  non-admins on the microversion. This is needed for the
  non-admin API user that initiated the volume extend
  operation to be able to tell when the nova-compute side
  is complete.

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Blueprint: nova-support-attached-volume-extend

Change-Id: If10cffd0dc4c9879f6754ce39bee5fae1d04f474
2017-07-12 11:53:20 -04:00
Jenkins
0ffe7b2789 Merge "Fix quota class set APIs" 2017-07-12 05:56:35 +00:00
Jenkins
17f9653f5d Merge "Add release note for xenapi virt device tagging support" 2017-07-11 21:59:06 +00:00
ghanshyam
92e0efeefd Fix quota class set APIs
v2.1 API which does not return the 'server_groups' and
'server_group_members' quotas in GET & PUT os-quota-class-sets
API response. v2 API used to return those keys in API response.

Also filter out the network related quotas from os-quota-class-sets
APIs

Fixing this with microversion.
Closes-Bug: #1701211
Closes-Bug: #1693168
implement-blueprint fix-quota-classes-api

Change-Id: I66aeb7a92fb5ee906fead78030bd84a2e97916e8
2017-07-10 21:44:08 +00:00
Jay Pipes
48268c73e3 placement: scheduler uses allocation candidates
This patch replaces the scheduler's use of GET
/resource_providers?resources=XXX with GET
/allocation_candidates?resources=XXX.

In doing so, we move the interaction with the placement API out of the
scheduler driver interface and up into the scheduler manager. This
allows us to make fewer changes to the underlying HostManager and
SchedulerDriver interfaces and isolate communication with the placement
API in a single place.

The provider_summaries part of the response from GET
/allocation_candidates is used to generate the UUIDs that winnow the
number of compute nodes retrieved by the filter scheduler during
scheduling. Following patches will add in support for actually doing
the claim from the scheduler against one or more resources providers by
examining the allocation_requests part of the HTTP response and picking
one that contains the host the scheduler picked during its _schedule()
loop.

Change-Id: I1c0bd2987dcbc38f23b71db2bc8e3267f85168c8
blueprint: placement-allocation-requests
2017-07-07 11:35:54 -04:00
Matt Riedemann
85e94cef77 Add release note for xenapi virt device tagging support
Nova change I565617e05acf33e6254ea091b88d975270ffde05 added
the support for creating a server with tagged VIFs and BDMs
with the xenapi driver, but did not contain a release note.

This adds the release note, and much of the text is copied
from Tempest change I9fe520bc9b68d0bc7f879617f2cd27dd1029e4de.

Part of blueprint virt-device-role-tagging-xenapi

Change-Id: I330087944203ab6aa9cf64fd4d3f1909ce44b435
2017-07-06 12:38:30 -04:00
Jenkins
de5c9e076d Merge "Don't delete neutron port when attach failed" 2017-07-05 01:29:31 +00:00
Jay Pipes
1d01a8811a placement: support GET /allocation_candidates
A new 1.10 API microversion is added to return information that the
scheduler can use to select a particular set of resource providers to
claim resources for an instance.

The GET /allocation_candidates endpoint takes a "resources" querystring
parameter similar to the GET /resource_providers endpoint and returns a
dict with two top-level elements:

"allocation_requests" is a list of JSON objects that contain a
serialized HTTP body that the scheduler may subsequently use in a call
to PUT /allocations/{consumer_uuid} to claim resources against a
related set of resource providers.

"provider_summaries" is a JSON object, keyed by resource provider UUID,
of JSON objects of inventory/capacity information that the scheduler
can use to sort/weigh the results of the call when making its
destination host decisions.

Change-Id: I8dadb364746553d9495aa8bcffd0346ebc0b4baa
blueprint: placement-allocation-requests
2017-07-03 10:57:59 -04:00
Kevin_Zheng
115cf068a6 Don't delete neutron port when attach failed
Currently, when attaching neutron pre-existing
port to an instance, if the attach failed, it
will also be deleted in Neutron side due to
bad judgement of the who created the port by
reading not up-to-date info_cache.

The workflow starts at:
https://github.com/openstack/nova/blob/9ed0d6114/nova/network/neutronv2/api.py#L881
ordered_ports and preexisting_port_ids are
the same when attaching a preexisting port
to an instance and it calls
https://github.com/openstack/nova/blob/9ed0d6114/nova/network/base_api.py#L246
which calls back into the neutronv2 api code
https://github.com/openstack/nova/blob/9ed0d6114/nova/network/neutronv2/api.py#L1274
and at this point, compute_utils.refresh_info_cache_for_instance(context,
instance) won't have the newly attached port in it(see
debug log: http://paste.openstack.org/show/613232/)
because _build_network_info_model() is going to
process it. The instance obj in memoryt with old
info_cache will be used at rollback process and
causing the miss-judging.

This patch fixed it by updating instance.info_cache
to the new ic after it is created.

Co-Authored-By: huangtianhua@huawei.com
Change-Id: Ib323b74d4ea1e874b476ab5addfc6bc79cb7c751
closes-bug: #1645175
2017-07-03 15:40:12 +08:00
Silvan Kaiser
f2d5af0b38 Removes getfattr from Quobyte Nova driver
Replaces reading quobyte.info xarg with a range
of checks on a given mountpoint. Adds several
unit tests to verify the different check results.
Adds --disable-xattrs option to mount calls and
a release notes file.

Closes-Bug: 1659328

Change-Id: I1c07e513af14b769c257c72e51132f81e61673c7
2017-07-03 09:26:23 +02:00
Jenkins
d7ddee03aa Merge "Remove wsdl_location configuration option" 2017-06-30 17:16:59 +00:00
Jenkins
5ea8857138 Merge "Only auto-disable new nova-compute services" 2017-06-30 17:15:27 +00:00