From bde4c0275f257949272e0cf7f870e3d3212417e9 Mon Sep 17 00:00:00 2001 From: Claudiu Belu Date: Thu, 10 Sep 2015 17:03:32 +0300 Subject: [PATCH] Adds spec for exposing host capabilities https://blueprints.launchpad.net/nova/+spec/expose-host-capabilities Ensuring proper scheduling can be difficult task, especially when the instances require several host features or capabilities. This would require the administrators to know what features are available for a certain hypervisor version and / or creating quite a few host aggregates, which can become tedious. Implements: blueprint expose-host-capabilities Change-Id: I7f31de7c5074561cbe2906c80d79ce8b825f1009 --- .../approved/expose-host-capabilities.rst | 253 ++++++++++++++++++ 1 file changed, 253 insertions(+) create mode 100644 specs/mitaka/approved/expose-host-capabilities.rst diff --git a/specs/mitaka/approved/expose-host-capabilities.rst b/specs/mitaka/approved/expose-host-capabilities.rst new file mode 100644 index 000000000..bafaecc4d --- /dev/null +++ b/specs/mitaka/approved/expose-host-capabilities.rst @@ -0,0 +1,253 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +======================== +Expose host capabilities +======================== + +https://blueprints.launchpad.net/nova/+spec/expose-host-capabilities + +Ensuring proper scheduling can be a difficult task, especially when the +instances require several host features or capabilities. This would require the +administrators to know what features are available for a certain hypervisor +version and / or creating quite a few host aggregates, which can become +tedious. + +Problem description +=================== + +Currently, the nova scheduler is not aware of all the host capabilities or +features the compute nodes might have. For example, certain features require a +minimum hypervisor version. The only way to handle this would be to set filters +(e.g. image property ``requires_hypervisor_version``) in order to ensure proper +scheduling. This is less than ideal, as it requires the administrators to know +which feature requires what hypervisor version. [1] + +On top of this, there are some features that are not available without proper +configuration. Choosing to deploy an instance with such a feature on such a +node would fail. For example, Hyper-V Shielded VMs cannot be created on a +host that does not have the Host Guardian Service enabled and it is not +Guarded [2]. Or, Hyper-V vTPM feature does not exist in Windows 10 at all, but +it exists in Windows / Hyper-V Server 2016 and they share the same hypervisor +version (10.0). + +Plus, the users might not be aware of each and every hypervisor specific +capability that they could have at their disposal. For example, Secure Boot +VMs for Windows guests are available starting with Windows / Hyper-V Server +2012 R2, while the newer versions offer this feature for both Windows and Linux +guests. This sort of capability can easily be reported by the compute nodes, +instead of having the look for this information online. + +Finally, in heterogenous deployments (different hypervisors, different +hypervisor versions, different hardware), there could be N total host +capabilities, each compute node having a subset from those N capabilities. +Instances could require any combination of those N capabilities; creating a +host aggregate for each combination of capabilities in order to ensure proper +scheduling is unfeasible. Ideally, the scheduler could do feature-matching +instead. + +Use Cases +---------- + +With this feature, most of the hypervisor specific features and other host +capabilities will be reported by the compute nodes. Hypervisor features will be +discovered automatically, meaning that users will not have to search for +information in order to find out what features are available to them. + +In heterogenous deployments, instances requiring a certain subset of the +available host capabilities will be easier to deploy, as they won't require +a specific host aggregate that contains those capabilities. + +Proposed change +=============== + +There are two types of host capabilities: + +* **Hypervisor version related capabilities**: newer hypervisor versions can + offer new features that can be added to the instances. (e.g.: secure boot, + generation 2 VMs, etc.) + +* **Undiscoverable capabilities**: cannot be determined easily or at all by + the nova-compute service, mostly hardware related capabilites (e.g.: SSD, + SR-IOV, fibre channel, etc.) + +The method ``get_hypervisor_capabilities`` must be added to virt.ComputeDriver. +The driver will have to implement this method and return a ``HostCapabilities`` +object containing the "hypervisor version related capabilities" mention in the +`Use Cases` section. + +As for the "undiscoverable capabilities", a config option in the group +``host_capabilities`` can be defined for each capability. + +All configured and reported capabilities must already exist as a field in the +HostCapabilities object. Any unknown capability detected will generate a +warning and will be ignored. + +As for the capabilities present in the ``HostCapabilities`` model, they will +mirror the properties that can be in the image metadata or flavor extra specs, +in order to easily match requested features to host capabilites. + +For example, the host could have the capability for instance secure boot. The +``HostCapabilities.os_secure_boot`` field will be set to True. In order to +request the instance secure boot feature, users will have to define the image +property ``os_secure_boot`` or flavor extra spec ``os:secure_boot`` as +``required`` [3]. + +A new filter will be implemented which will match the instance features +requested with the host capabilities as previously described. It will only +take into account fields defined in the ``HostCapabilities``. If a field in the +``HostCapabilities`` instance has not been set, that capability will be +considered as not present. + +The host capabilities can be expressed as follows: + +1. **Boolean**. For most cases, a host capability is simply a boolean: it is +present or not. In this case, an instance requiring a certain capability can +easily be matched with hosts which has that capability present or set to True. +For example, the instance's image metadata contains the property``os_vtpm`` +set to ``required``. If the ``HostCapabilities`` instance ``os_vtpm`` field is +set to True, then that host is appropriate for that instance. + +2. **Set of values**. In other cases, a host capability is expressed as a set +with different values. For example, the ``hw_machine_type`` capability [4], +which can have multiple values. In the Hyper-V's case, the mentioned field is +used to whether a VM is generation 1 or 2. Windows Hyper-V / Server 2012 R2 or +newer will report the values ['hyperv-gen1', 'hyperv-gen2'] for this +capability. For an instance with image metadata containing the property +``hw_machine_type`` set to ``hyperv-gen2``, a host will be considered +appropriate if the requested capability value exists in the +``HostCapabilities.hw_machine_type`` set. + +Instances can also require multiple values from the same capabilities set +by expressing image properties / flavor extra spec values as lists (e.g.: image +property ``os_foo=bar1,bar2``). + +3. **Ordered set of values**. In the case of host capabilities expressed as +ordered sets (e.g.: set element `i` is an enhanced version of element `i - 1`), +users can use logical operators to indicate what subset of a capability is +required by the instance. For example, If an instance requires the capability +``os_foo`` to be newer / greater or equal to ``bar1``, he will define the image +property as ``os_foo=>=bar1``. + +Alternatives +------------ + +* Image property ``requires_hypervisor_version``: it requires administrators to + know which feature requires what hypervisor version, plus there are features + that cannot be determined by version alone. + +* Host aggregates, grouping hosts by capabilities. This can become quite + difficult when trying to deploy instances requiring multiple capabilities. + +Data model impact +----------------- + +The Text column ``host_capabilities`` will be added to the ``compute_node`` +database table. This column will contain all the HostCapabilities objects, +reported by the compute nodes, serialized as a JSON blob. A database schema +migration is necessary in order to add the column. + +New ``HostCapabilities`` object will be added, containing all the capabilities +currently acceptable by Nova. If any of the object's fields is not set, then +that capability will be considered as not present. + +REST API impact +--------------- + +None + +Security impact +--------------- + +None + +Notifications impact +-------------------- + +None + +Other end user impact +--------------------- + +None + +Performance Impact +------------------ + +None + +Other deployer impact +--------------------- + +None + +Developer impact +---------------- + +Drivers will have to implement the new ``get_host_capabilities`` method. It +should return an instance of ``HostCapabilities``. + +In order for a new capability to be accepted in ``HostCapabilities``, a version +increment for will be necessary. If the capability's type is `undiscoverable`, +it will have to be added to the config option group ``host_capabilities``. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + Claudiu Belu + +Work Items +---------- + +* ``host_capabilities`` column in ``compute_nodes`` table. +* ``HostCapabilities`` object model. +* ``host_capabilities`` config option group. +* add ``host_capabilities`` attribute in HostState. +* ``nova.virt.driver.ComputeDriver.get_host_capabilities`` method. +* drivers' ``get_host_capabilities`` implementation. +* new scheduler filter. + +Dependencies +============ + +None + +Testing +======= + +* Unit tests. +* Jenkins. + +Documentation Impact +==================== + +The new scheduler filter and the host capabilities that can be scheduled using +the new filter will have to be documented. +The new config option group ``host_capabilities`` will have to be documented. +The new nova API microversion will have to be documented. +The deployer impact will have to be documented. + +References +========== + +[1] #openstack-nova IRC discussion: + http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2015-09-08.log.html#t2015-09-08T15:30:04 + +[2] Hyper-V vTPM / shielded VMs spec: + https://review.openstack.org/#/c/195068/ + +[3] Hyper-V UEFI Secure Boot spec: + https://review.openstack.org/#/c/190997/ + +[4] Hyper-V generation 2 VMs spec: + https://review.openstack.org/#/c/103945/ + +History +=======