openstack/nova-specs
Code Issues Proposed changes

Mark liberty specs as implemented

Browse Source 
This review syncs the state of launchpad with the spec repo for
things implemented up to liberty-3. There are no changes to the
specs, just things being moved around.

Change-Id: I930d33532b268b6e933c8be06a0569c20fd09586
This commit is contained in:
Michael Still
2015-09-04 08:46:08 +10:00
parent ee259ec008
commit 30d5bacb19
31 changed files with 31 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

222
specs/liberty/implemented/add-locking-information-in-server-get-response.rst Normal file
View File

@@ -0,0 +1,222 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
======================================
Add 'locked' in server Detail Response
======================================
https://blueprints.launchpad.net/nova/+spec/add-locking-information-in-server-get-response
Currently admin or owner can lock/unlock the instance but there is no way to
know whether instance is locked.
This spec is to propose a way to know lock information by adding new
'locked' attribute in server GET APIs Response.
Problem description
===================
Currently Instance can be locked and unlocked by admin or owner. But there is
no way to get to know who locked the instance. Even users cannot know
whether that instance is locked or not.
Use Cases
----------
User can know whether instance is locked. When user want to perform any action
on instance and it is locked then it return error about instance not in proper
state. If there is prior way to know whether instance is locked, it will be
easy for user/admin to do appropriate action accordingly.
As lock/unlock action can be performed by admin or owner (more than one role),
'locked' information can be very useful.
Project Priority
-----------------
None.
Proposed change
===============
Add new attribute 'locked' in server detail response which will
provide instance lock information.
Returned value by 'locked' attribute will be-
* True - If instance is locked.
* False - If instance is not locked.
When user will query about instance details (List Detail or Show), 'locked'
information will be returned in response.
When "locked" is "true", it means there is a lock on instance, it does not
mean instance is locked for requested users.
For example - When instance is locked by the owner but listed by an admin,
"locked" will be "true" even admin can override the owner lock.
So "locked" provides only concrete information whether instance is locked
or not.
We could have provide lock information by exposing "locked_by" (which
holds the lock owner information in current implementation) directly but
as in most cases instance will not be locked so its value will be null.
And its always better to expose the simple and concrete information than
implementation one which can be changed in future as lock things can be
expanded in future.
NOTE-
Lock can be made it's own resource at a later time which will help to know
complete information about instance lock (locked by whom, lock reason,
time stamp etc).
New lock API can looks like something - servers/server_id/lock.
Alternatives
------------
User can get to know locked status from exception returned on performing
invalid action on locked server. But there will not be any way to know who
locked instance.
Data model impact
-----------------
None.
REST API impact
---------------
* Specification for the method
* Description
* API Show server details & List servers details
* Method type
* GET
* Normal http response code
* 200, no change in response code
* Expected error http response code(s)
* No change in error codes
* URL for the resource
* /v2.1
* JSON schema definition for the body data if allowed
* A request body is not allowed.
* JSON schema definition for the response data if any
::
{
'status_code': [200],
'response_body': {
'type': 'object',
'properties': {
'server': {
'type': 'object',
'properties': {
'id': {'type': 'string'},
'name': {'type': 'string'},
'status': {'type': 'string'},
.
.
'locked': {'type': 'boolean'},
.
.
.
'OS-EXT-STS:task_state': {'type': ['string', 'null']},
'OS-EXT-STS:vm_state': {'type': 'string'},
'OS-EXT-STS:power_state': {'type': 'integer'},
.
.
.
},
}
}
'required': ['server']
}
}
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
python-novaclient needs to be updated in order to show the 'locked'
in the 'nova show' commands.
Performance Impact
------------------
None.
Locked by information is already present in Instance object, this will just
show that information to user.
Other deployer impact
---------------------
None.
Developer impact
----------------
None.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
gmann
Other contributors:
None
Work Items
----------
* Add 'locked' in server GET APIs (Show and List Detail)
Response.
* Modify Sample and unit tests accordingly.
Dependencies
============
None.
Testing
=======
Currently Nova functional test will cover these changes testing.
After discussion of micro version testing in Tempest, these changes
can be tested accordingly.
Documentation Impact
====================
server GET APIs doc will be updated accordingly.
References
==========

236
specs/liberty/implemented/adding-python34-support-to-nova.rst Normal file
View File

@@ -0,0 +1,236 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=================================
Adding Python 3.4 support to Nova
=================================
https://blueprints.launchpad.net/nova/+spec/nova-python3
Problem description
===================
It's time to add Python 3 support to Nova by generalizing the usage of the six
module, in addition to the Python 2 support.
Use Cases
----------
See the article `Why should OpenStack move to Python 3 right now?
<http://techs.enovance.com/6521/openstack_python3>`_ for the rationale.
Project Priority
-----------------
None
Proposed change
===============
This specification details the steps needed to add Python 3.4 support to Nova
by generalizing the usage of the six module.
Almost all Nova dependencies are ported to Python 3. For the few remaining
libraries, the port is in progress and should be done in a few weeks. It is
already possible to start porting Nova to Python 3. See the Dependencies
section below for more information.
The goal here is to make all Nova tests pass with Python 3: Nova unit tests and
Tempest tests.
.. note::
Python 2 support is kept.
Alternatives
------------
None
Data model impact
-----------------
None (changes must not impact the data model).
REST API impact
---------------
None (changes must not impact the REST API).
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
There is no impact on performances.
In March 2013, Brett Canon ran `the official Python benchmarks suite
<https://hg.python.org/benchmarks>`_ to compare Python 2.7 and 3.3 for his talk
at Pycon US: `Python 3.3: Trust Me, It's Better Than Python 2.7
<https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-than-python-2-dot-7-by-dr-brett-cannon>`_.
The result: "If you sorted all of the benchmarks and looked at the median
result ... Python 3 is the same".
See the "Optimizations" section of each "What's New in Python 3.x" document for
the full list of optimizations: `Python 3.1
<https://docs.python.org/3/whatsnew/3.1.html#optimizations>`_, `Python 3.2
<https://docs.python.org/3/whatsnew/3.2.html#optimizations>`_, `Python 3.3
<https://docs.python.org/3/whatsnew/3.3.html#optimizations>`_ and `Python 3.4
<https://docs.python.org/3/whatsnew/3.4.html#significant-optimizations>`_.
Other deployer impact
---------------------
Deployers using python 2.7 will see no changes.
Those able to run python 3.4 will now be try that out.
Developer impact
----------------
Once the Python 3 check job becomes voting, developers will have to write code
compatible with Python 2 and Python 3. During the transition period, only code
tested by the ``tox.ini`` whitelists will require Python 3 support.
Thanks to tox, it is trivial to run locally the Nova test suite on Python
2.7 and 3.4.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
victor-stinner
Other contributors:
None
Work Items
----------
* Fix most obvious Python 3 issues. Example of a patch fixing most Python 3
issues: `Port Nova to Python 3
<https://github.com/haypo/nova/commit/bad54bc2b278c7c7cb7fa6cc73d03c70138bd89d>`_.
This change is just a draft to test if porting Nova is feasible, it
should be splitted into smaller patches grouped by similar changes. See also
the `Port Python 2 code to Python 3
<https://wiki.openstack.org/wiki/Python3#Port_Python_2_code_to_Python_3>`_
section of the Python 3 wiki page.
* Add a Python 3 test environment to ``tox.ini`` to run a subtest of the tests
which pass on Python 3. It will use a whitelist of tests which are know to
pass on Python 3.
* Add a non-voting Python 3 check job for Nova
* When the Python 3 check job is stable enough, make it voting. From this
point, we should now be able to avoid Python 3 regressions while working.
* Fix failing tests, one by one, to enlarge the whitelist of tests
in ``tox.ini``.
* Once all tests work, remove the whitelist of tests from ``tox.ini``.
The transition period, when Python 3 is only supported partially, should be a
short as possible.
No voting Python 3 gate jobs will be added to not waste resources of the
OpenStack infra, Python 2 gate jobs are enough. We consider that there is a low
risk of having a Python 3 specific issue introduced by a conflict between the
Python 3 check job and the Python 2 gate job.
Dependencies
============
Remaining dependencies not compatible with Python 3 yet:
* ``oslo.messaging``: The development version works on Python 3, except of Qpid
and AMQP 1.0 drivers. To begin the Nova port to Python 3, we can start with
the RabbitMQ driver, until AMQP is ported to Python 3 too. A new version
of ``oslo.messaging`` will be released in a few weeks.
* ``mysql-python``: the fork `mysqlclient
<https://pypi.python.org/pypi/mysqlclient>`_ works on Python 3 and includes
bug fixes. There is also `PyMySQL <https://pypi.python.org/pypi/PyMySQL>`_,
a driver fully implemented in Python which works on Python 3 too, but it has
worse performances.
* ``python-memcached``: see the pull request `Port memcache to Python 3
<https://github.com/linsomniac/python-memcached/pull/67>`_. It blocks
``keystonemiddleware``. It may be replaced with ``pymemcache`` which is
already Python 3 compatible.
* ``websockify``: Python 3 classifier is missing in websockify 0.6.0, but it
is present in the development version. Tests are failing but they may be
issues with the tests, not with websockify directly.
All in all, there is no major issue with the dependencies.
Using the development version of ``oslo.messaging``, it's already possible to
work on fixing Nova tests on Python 3.
Testing
=======
The current test suite should be enough to test Nova on Python 3.
We will run tests with Nova running under Python 3.4 by the end of this
process: Nova unit tests and Tempest tests.
Documentation Impact
====================
Developers might be interested in reading the official `Python 3 page
<https://wiki.openstack.org/wiki/Python3>`_ on the Openstack wiki. It shows
the current progress of the OpenStack port of Python 3, and details some common
issues that arise when porting code from Python 2 to Python 3.
References
==========
* Related Liberty specifications:
- Heat: `Add Python 3.4 support <https://review.openstack.org/#/c/175340/>`_
by Sirushti Murugesan
- Neutron: `Porting to Python 3 spec
<https://review.openstack.org/#/c/172962/>`_ by Cyril Roelandt with the
support of Ihar Hrachyshka.
* `Python 3 page in OpenStack wiki <https://wiki.openstack.org/wiki/Python3>`_
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Liberty
- Introduced

247
specs/liberty/implemented/admin-query-any-keypair.rst Normal file
View File

@@ -0,0 +1,247 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===========================================================
Allow admins to query and create keypairs owned by any user
===========================================================
https://blueprints.launchpad.net/nova/+spec/admin-query-any-keypair
This change allows admin users to query keypairs owned by users
other than themselves, as well as to create/import new keypairs on
their behalf.
Problem description
===================
Currently, keypairs are only available to their owners. Admins should
be able to see the keypairs of other users, and create new keypairs
on their behalf, when needed.
Use Cases
----------
As the admin of an openstack cluster I need to see what keypairs a
user has.
As someone with admin permission, I want to be able to create an
instance with someone else's keypair assigned so that they can log
into it.
Allowing the administrators to create keypairs and importing public keys on
behalf of the users will allow the users to have access to an instance booted
with it.
For instances that require additional post-deployment configuration using
Configuration Management tools (Ansible, Puppet etc..), having a pre-installed
keypair deployed by the administrator is also very useful.
Project Priority
-----------------
(yet to be defined)
Proposed change
===============
For querying operations, this change doesn't affect the API format
or schema in any way, but merely adds a query key to select a specific
user for the keypair query.
However, an optional user_id parameter will need to be added to POST
operation to specify the user for which a keypair is being created.
The following requests are currently allowed for querying:
GET /os-keypairs
GET /os-keypairs/[keypair]
After this change, admins would be able to do this:
Get a list of keypairs for [user_id]:
GET /os-keypairs?user_id=[user_id]
View a specific user's keypair
GET /os-keypairs/[keypair]?user_id=[user_id]
* Future work:
Allowing the admins to list keypairs from all users will require additional
work, that will involve changes to the database scheme, will be submitted
in a separate spec.
Alternatives
------------
We could add a new admin API method to facilitate this, but doing so
would be a lot more work for little (if any) benefit.
Data model impact
-----------------
None.
REST API impact
---------------
* Specification for the querying methods
* The existing index operation for keypairs will be extended to
honor an optional "user_id" parameter, if the proper microversion
is active.
* Method type: GET
* Normal http response code(s): 200
* Expected error http response code(s)
* 403: If the user does not have permissions, per the policy file
* ``/v2.1/{tenant_id}/os-keypairs?user_id={user_id}``
* ``/v2.1/{tenant_id}/os-keypairs/{keypair_name}?user_id={user_id}``
* Parameters which can be passed via the url: The alternate user id
* JSON schema definition for the body data is unchanged
* JSON schema definition for the response data is unchanged
* Example use case:
Request:
GET http://127.0.0.1:8774/v2.1/e0c1f4c0b9444fa086fa13881798144f/os-keypairs?user_id=example_userid
* Specification for the create/import methods
* Create method will be extended to honor an optional "user_id" parameter,
that will be provided in the request body,
if the proper microversion is active.
* Method type: POST
* Normal http response code(s): 200
* Expected error http response code(s)
* 403: If the user does not have permissions, per the policy file
* ``/v2.1/{tenant_id}/os-keypairs``
* JSON schema definition for the response data is unchanged
* JSON schema definition for the body data will change to include the
optional user_id parameter:
.. code-block:: javascript
create:
{
"keypair": {
"name": "%(keypair_name)s",
"type": "%(keypair_type)s",
"user_id": %(user_id)s"
}
}
import:
{
"keypair": {
"name": "%(keypair_name)s",
"type": "%(keypair_type)s",
"public_key": "%(public_key)s,"
"user_id": %(user_id)s"
}
}
* This will add new policy elements which will allow assigning this
permission:
::
"os_compute_api:os-keypairs:index": "is_admin:True or user_id:%(user_id)s"
"os_compute_api:os-keypairs:show": "is_admin:True or user_id:%(user_id)s"
"os_compute_api:os-keypairs:create": "is_admin:True or user_id:%(user_id)s"
Security impact
---------------
Admin users will be able to see the public keys of other
users and create new keypairs on their behalf.
However, these are generally regarded as material suitable for
public viewing anyway.
Notifications impact
--------------------
None
Other end user impact
---------------------
* This change will imply changes to the python-novaclient to allow
specifying the user for which keypairs should be listed or created.
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
vladikr
Other contributors:
Dan Smith, Dan Radez
Work Items
----------
* Add a new microversion and make os-keypairs honor the user_id query/create
parameter
Dependencies
============
None
Testing
=======
Unit tests are sufficient to verify this functionality, as it is
extremely simple. API samples tests can be added to make sure that the
output of the list call does not differ when a user_id parameter is
passed. Add new API sample to verify the create/import request schemas.
Documentation Impact
====================
The nova/api/openstack/rest_api_version_history.rst document will be updated.
References
==========
* Bug https://bugs.launchpad.net/nova/+bug/1182965 requesting this
feature.

173
specs/liberty/implemented/allocation-ratio-to-resource-tracker.rst Normal file
View File

@@ -0,0 +1,173 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Move allocation ratios to resource tracker
==========================================
https://blueprints.launchpad.net/nova/+spec/allocation-ratio-to-resource-tracker
Move the definition and calculation of allocation ratios out of the scheduler
and into the resource tracker.
Problem description
===================
Allocation ratios are currently improperly defined in the scheduler. This leads
to efficiency problems due to the scheduler interfacing with the database
when it does not need to as well as recalculating adjusted resource usage
numbers when it does not need to.
The memory and CPU allocation ratios are currently controlled on a global or
per-aggregate basis, with the global configuration settings determined in the
core_filter and ram_filter filter modules in the scheduler, and the
per-aggregate allocation ratio overrides are stored in the aggregates table in
the database, with the core_filter scheduler filter performing repeated lookups
to the aggregates table to determine the allocation ratio to use when host
aggregates are in use in the deployment.
Allocation ratios are NOT scheduler policy, and should neither be defined
nor queried in the scheduler at all. Allocation ratios are simply a way for a
compute node to advertise that it has the ability to service more resources
than it physically has available: an overcommit ratio. Therefore, not only does
the total advertised amount of resources on a compute node NOT need to be
recalculated on each run of the scheduler to find a compute node for an
instance, but the resource tracker is the most appropriate place to set the
available resource amounts.
Use Cases
---------
Operator want to specify allocation ratios per compute node
Project Priority
----------------
Priorities have not yet defined for Liberty, but this spec should be related to
the former Kilo 'scheduler' priority effort.
Proposed change
===============
We propose to move the definition of CPU and memory allocation ratios out of
the scheduler filters where they are currently defined (core_filter and
ram_filter) and into the resource tracker (nova.compute.resource_tracker).
Further, we propose to remove all calculation of the compute node's overcommit
ratio for both CPU and RAM out of core_filter.py and ram_filter.py.
This calculation will initially be moved into the host_manager.HostManager
class, which will store the real and adjusted available resource amounts for
each compute node in its collection of HostState structs. Because the current
resource tracker in Nova only accounts for a single compute node, we must,
for now, use the scheduler's internal resource tracker (HostManager) to track
all compute nodes' allocation ratios.
As now host aggregate information for each compute node is provided into
HostState, we can direcly get the related overcommit ratios. If the compute
node belongs to any host aggregates, then the overcommit ratio for CPU and
memory shall be either the lowest ratio set for any of the host aggregates
OR the default global configuration ratio value.
The long-term goal is to enable each compute node in the system to have its
own settable allocation ratios and remove the need for this particular check
or calculation in the resource tracker or the scheduler itself. My personal
end goal is to align the scheduler's internal resource tracker with the code
in nova.compute.resource_tracker, but this particular blueprint is scoped
only to the relatively minor changes described above.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
Even with the relatively minor changes introduced here, there should be a
performance increase for a single scheduler request due to fewer
calculations being made in each scheduler request. For deployments that
use host aggregates, performance improvements will be much greater, as
the number of DB queries per scheduler request will be reduced.
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Work Items
----------
* Move the definition of the allocation ratios out of the filters and into
nova.compute.resource_tracker
* Provide ram_allocation_ratio and cpu_allocation_ratio as new fields for the
ComputeNode object ans the DB compute_nodes model
* Change the behavior in HostManager.get_all_host_states() to calculate
resources available from either the host aggregate's min ratio or the
global definition ratio populated correctly in HostState
Assignee(s)
-----------
Primary assignee:
sylvain-bauza
Dependencies
============
None
Testing
=======
Some minor adjustments to the existing unit tests would need to be performed.
Documentation Impact
====================
None
References
==========
Mailing list discussion:
http://lists.openstack.org/pipermail/openstack-dev/2014-June/036602.html

141
specs/liberty/implemented/allow-ip6-search-for-non-admin.rst Normal file
View File

@@ -0,0 +1,141 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=====================================
Allow ip6 server search for non-admin
=====================================
https://blueprints.launchpad.net/nova/+spec/allow-ip6-search-for-non-admin
Add ``ip6`` to the list of options that are allowed for non-admins when
listing servers.
Problem description
===================
Filtering by IPv6 address is currently only allowed for admins, but there
is no reason to treat this differently than IPv4. It is also quite surprising
for a user to find that ``nova list --ip6 xxx`` will list all servers.
Use Cases
---------
A user will want to list servers based on their IPv6 address, just like they
can already do based on IPv4.
Project Priority
----------------
None
Proposed change
===============
Add a new API microversion for which the ``ip6`` option will no longer be
filtered out from the server search for non-admins.
Alternatives
------------
Treat the bug fix as a minor patch that will not require a new API
microversion. However in the conversation about the fix (see References)
there seemed to be consensus that a microversion is needed in order for
a client to be able to tell whether filtering by IPv6 is available or not.
Data model impact
-----------------
None
REST API impact
---------------
The new API is added as a microversion.
Request::
GET /servers?ip6=<regex>
The request and response headers, body and possible codes are unchanged from
current behaviour. The ``ip6`` option will no longer be silently discarded
for non-admins.
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
python-novaclient will have to be updated so that it can request the new
microversion when the ``--ip6`` option is used.
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Jens Rosenboom <j.rosenboom@x-ion.de>
Work Items
----------
* Add a new microversion and change
``nova/api/openstack/compute/plugins/v3/servers.py`` to add ``ip6`` to the
list of allowed server search options.
Dependencies
============
None
Testing
=======
* Unit tests and API samples functional tests in the nova tree will be added.
Documentation Impact
====================
The nova/api/openstack/rest_api_version_history.rst document will be updated.
References
==========
[1] Originally reported as a bug: https://bugs.launchpad.net/nova/+bug/1450859
[2] ML thread discussing whether a microversion is needed:
http://lists.openstack.org/pipermail/openstack-dev/2015-May/065118.html
[3] Proof of concept code change: https://review.openstack.org/179569

199
specs/liberty/implemented/api-relax-validation.rst Normal file
View File

@@ -0,0 +1,199 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Relax API validation for v2.0 on v2.1
==========================================
https://blueprints.launchpad.net/nova/+spec/api-relax-validation
Currently v2.1 strongly validates all API requests.
This spec details how we will relax some validation for v2.0 API requests
served by the v2.1 code base.
Note requests being sent to /v2.1 will keep their full strong validation.
Problem description
===================
We hope that in the near future, all request to nova API can be processed by
the new API v2.1 code base.
At that point, we will be able to deprecate, then delete, the current v2.0
API implementation, and return to a single API implementation.
While all clients making valid requests to the v2.0 API will get the same
results talking to the v2.1 API, there are issues.
Various types of "invalid" requests are currently accepted by v2.0, but would
be rejected by v2.1.
Even tempest was found to be making invalid requests:
https://review.openstack.org/#/c/138245
Use Cases
----------
We need it to be low risk for users to deploy v2.1 to deal with the current
requests for v2.0.
While initial tests of some major SDKs have shown they appear to be making
correct requests to our v2.0 API, not all users use an SDK.
Given the problems found in the tempest test suite, where invalid requests
were being made to the v2.0 API, it must be assumed that users who have
written their own code to access our API will have made similar mistakes.
Where possible, we want these users to be unaffected by the change from
v2.0 to v2.1.
It is expected that SDKs will be updated to start adding the version headers
for all their requests to the API. At this point, they will start to get the
full benefits of strong API validation. Only those users that are still not
specifying the version headers would be getting the weaker validation.
Project Priority
-----------------
Part of the API v2.1 effort.
Proposed change
===============
The API v2.1 validation logic will change such that:
* requests to /v2.1 work the same as today after this change
* requests to /v2 will have relaxed validation, and will ignore
X-OpenStack-Nova-API-Version headers, and always return /v2 responses
* requests made to /v2 will never return X-OpenStack-Nova-API-Version headers,
even when powered by the v2.1 codebase
* if we keep /v1.1 it will remain the same as /v2
The relaxed validation consists of:
* no longer error out requests due to additionalProperties, instead when
the request if for the /v2 API we just ignore those additionalProperties.
In addition:
* any request to /v2 that includes headers for /v2.1 will be ignored when
v2.1 codebase is used to deliver the /v2 requests, so it matches what
the v2 codebase is doing today.
For more details see REST API impact.
Alternatives
------------
The main alternative is to not do this, which is likely to lead to slower
adoption of v2.1.
We could also allow /v2 requests to be sent to /v2.1, but that would
confuse matters, /v2 should just ignore the version headers.
We could ensure that any requests to /v2 error out if you sent the
X-OpenStack-Nova-API-Version header, but as python-novaclient already
sends that header for all /v2 requests, it would create another backwards
compatibility issue.
Instead of just ignoring parameters, it would be nice to also strip out
any invalid parameters before passing through the request to the v2.1 code.
It also feels slightly better from an input validation and security point
of view, but it does risk changing how the API behaves.
We could still look to add this at a later date, if it turns out to be a
good idea.
Data model impact
-----------------
None
REST API impact
---------------
This will have zero impact for the /v2.1 endpoint.
The /v2 endpoint powered by the v2.1 code gets the relaxed validation to
make it more compatible, as mentioned above.
In addition /v2 endpoint thats powered by the v2.1 code should never accept
any requests not accepted by /v2, and should only return /v2 like responses.
Basically, it should always ignore any X-OpenStack-Nova-API-Version
just like the the v2 code base does today.
For consistency, /v1.1 will be the same as /v2
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
johnthetubaguy and alex_xu
Work Items
----------
* relax validation for /v2 requests when using v2.1 codebase,
instead just ignore bad properties
* requests made to /v2 will never return X-OpenStack-Nova-API-Version headers,
even when powered by the v2.1 codebase
* ensure that /v2 served up by the v2.1 codebase ignores any
of the X-OpenStack-Nova-API-Version headers,
just like v2.0 code base does.
Dependencies
============
None
Testing
=======
Additional unit tests and integration tests should be enough to cover these
changes.
Documentation Impact
====================
None
References
==========
None

173
specs/liberty/implemented/cells-host-mapping.rst Normal file
View File

@@ -0,0 +1,173 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==================
Cells host mapping
==================
https://blueprints.launchpad.net/nova/+spec/cells-host-mapping
Since the scheduler will return a host rather than a cell we need to know which
cell that host is in. A new table will be created which can store this
mapping.
Problem description
===================
When Nova is partitioned into cells, the compute api needs to know which cell
to communicate with in order to build a scheduled instance, or for host API
requests. There is currently no mapping of host to cell so given just a host
there is no way to know how to pass information to that host.
Use Cases
----------
* Operators want to partition their deployments into cells for scaling, failure
domain, and buildout reasons. When partitioned, we need a lookup table to
know which partition a host is in.
Project Priority
-----------------
Priorities have not been decided for Liberty.
Proposed change
===============
The change being proposed is a new table in the 'nova_api' database for storing
a mapping of host to cell and an object to interact with it. Migration of data
into this table will be tackled in a separate spec.
The following diagram may help visualize it.::
api/cell boundary
scheduler returns a host |
| |
v |
nova-api+--------------------------->cell-db/rpc
+ |
+----+ |
| |
v |
host_mapping (joined with) cell_mapping |
Alternatives
------------
We could continue to use the nova-cells model in place today.
Data model impact
-----------------
A new 'host_mapping' table will be added to the 'nova_api' database.
The table will look like:::
CREATE TABLE `host_mapping` (
`created_at` datetime DEFAULT NULL,
`updated_at` datetime DEFAULT NULL,
`id` int(11) NOT NULL AUTO_INCREMENT,
`host` varchar(255) NOT NULL,
`cell_uuid` varchar(36) NOT NULL)
And host will be an indexed column. Other indexes are possible as well
and can be discussed in the code review.
It should be noted that there is no 'deleted' or 'deleted_at' column here. If
a host exists it should be mapped, it should only be deleted if the host is
being permanently removed in which case there is no reason to keep it here.
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
On its own this change does not introduce a performance impact. When it's used
by later specs it does introduce another database lookup for some actions
within Nova.
Other deployer impact
---------------------
This introduces a new table into the 'nova_api' database. And as described in
the "Data model impact" section above it should be considered when running any
cleanup of hosts. If hosts are removed from a deployment they can be removed
from the host_mapping table as well.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
alaski
Other contributors:
None
Work Items
----------
* Add database migration for 'host_mapping' table.
* Add HostMapping object.
Dependencies
============
None
Testing
=======
Since this is designed to be an internal re-architecting of Nova with no user
visible changes the current suite of Tempest or functional tests should
suffice. At some point we will want to look at how to test multiple cells or
potentially exposing the concept of a cell in the API and we will tackle
testing requirements then.
Documentation Impact
====================
Documentation should be added about the new table and what its usage will be.
References
==========
``https://etherpad.openstack.org/p/kilo-nova-cells``

152
specs/liberty/implemented/cells-instance-migration.rst Normal file
View File

@@ -0,0 +1,152 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
========================
Cells instance migration
========================
https://blueprints.launchpad.net/nova/+spec/cells-instance-migration
Now that there's a table to map instances to cells it needs to be populated
with data on instances that existed prior to its creation and usage.
Problem description
===================
When Nova is partitioned into cells the compute api needs to know which cell to
communicate with for a particular instance. Instances that existed before this
mapping was maintained need to have their location added to the table.
Use Cases
----------
* Operators want to partition their deployments into cells for scaling, failure
domain, and buildout reasons. When partitioned, we need a lookup table to
know which partition an instance is in. That lookup table needs to be
populated with information on instances that existed prior to its creation.
Project Priority
-----------------
Cells v2 is a priority for Liberty.
Proposed change
===============
The 'instance_mapping' table will be populated with data on which cell an
instance lives in.
A new nova-manage command will be added to look up instances in a database and
add an instance_mapping row for them. The command will take a cell name/uuid
as an argument and migrate instances within that cell. The cell name/uuid must
correspond to an entry in the cell_mapping table and the database connection
information in that row will be used for finding instances to be mapped.
For cells v1 setups where there are multiple cells a new cell_mapping entry
should be added for each cell and then the nova-manage command would need to be
run for each cell.
Alternatives
------------
The alternatives to cells v1/v2 have been discussed prior to this spec. In the
cells v2 effort there is no alternative for this mapping requirement.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
Deployers will be provided with a new nova-manage command to trigger the
creation of the mappings. This should be run once for a deployment not
currently using cellsv1, and once in each cell for a deployment currently using
cellsv1. This command will need to be run for each cell, or to migrate the
first cell for current non cellsv1 users, before the current hardcoded database
connection method can be dropped. The timeline for that is likely to be at the
end of a release cycle, either Liberty or M(ongoose).
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
alaski
Other contributors:
None
Work Items
----------
* Add nova-manage command to populate instance_mapping data for instances. The
command should migrate one cell at a time.
* Update grenade testing job, or add a new one, to call the new command and
verify that migration works properly.
Dependencies
============
https://blueprints.launchpad.net/nova/+spec/cells-instance-mapping
Testing
=======
Testing will be a combination of Nova in-tree functional tests and a grenade
test to verify that upgrades work.
Documentation Impact
====================
Documentation on the new nova-manage command will need to be written.
References
==========
``https://etherpad.openstack.org/p/YVR-nova-cells-v2``

154
specs/liberty/implemented/check-flavor-type-before-add-tenant.rst Normal file
View File

@@ -0,0 +1,154 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Check flavor type before add tenant access
==========================================
https://blueprints.launchpad.net/nova/+spec/check-flavor-type-before-add-tenant
Problem description
===================
We can't add tenant access to a public flavor.
Trying to do such a thing results in a confusing error message
when later we show the flavor::
$ nova flavor-access-add 1 2
+-----------+-----------+
| Flavor_ID | Tenant_ID |
+-----------+-----------+
| 1 | 2 |
+-----------+-----------+
$ nova flavor-access-list --flavor 1
ERROR (CommandError): Failed to get access list for public flavor type.
We should check whether the flavor is public or not
and reject add access to public flavor API call.
But this is backward incompatible bug, every API change need spec,
so a microversion is needed to handle this problem.
Use Cases
----------
User will be nofified they can't add an access to flavor if it's
public flavor with some exceptions.
Project Priority
-----------------
None
Proposed change
===============
In API layer, for "addTenantAccess" for flavor, we did following
to add access to flavor, so the proposed change is before
the access is added, add a validation to check the type and raise
exception in case the flavor is private type.
api/openstack/compute/plugins/v3/flavor_access.py
.. code:: python
flavor = objects.Flavor(context=context, flavorid=id)
try:
flavor.add_access(tenant)
This is not a backward compatible fix, so a new microversion will
be added and the check will be done only when the incoming API version
is equal or higher to this one.
Alternatives
------------
Let user continue to use existing code, though it's a bug.
Data model impact
-----------------
None
REST API impact
---------------
Per 'Proposed change' discussed, a check will added before
add_access is called.
It will be something like:
.. code:: python
def _check_flavor_type(self, req, flavor):
if req.version > target_version:
if flavor.type is public:
raise HTTPBadRequest
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
jichenjc <jichenjc@cn.ibm.com>
Work Items
----------
Add API change through microversion because it's backward incompatible.
1) Add microversion to os-flavor-access/addTenantAccess
2) Add validation before add access.
Dependencies
============
None
Testing
=======
None
Documentation Impact
====================
None
References
==========
[1] https://bugs.launchpad.net/nova/+bug/1361476
[2] https://review.openstack.org/#/c/124338/

160
specs/liberty/implemented/consolidate-console-api.rst Normal file
View File

@@ -0,0 +1,160 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Consolidate the APIs for getting consoles
==========================================
https://blueprints.launchpad.net/nova/+spec/consolidate-console-api
We have different public API for getting console access for each kind of
console that is supported in Nova. The proposal is to consolidate all these
APIs into one.
Problem description
===================
The APIs for getting console access are tightly coupled with the name of the
underlying protocol: os-getVNCConsole, os-getRDPConsole, etc. The result is
that every time we want to add support for a new console, we need to introduce
a new public API. A far better solution is to have only one API, get_console,
which can be used for obtaining access to all types of consoles.
Use Cases
----------
As a Nova developer I want to add support for a new console type and I don't
want to add more clutter to the public API.
Project Priority
-----------------
None
Proposed change
===============
The proposal is to introduce a single public API for getting console access and
deprecate all of the current public APIs that we have. The implementation will
inspect the request and will call the relevant get_XXX_console of the
ComputeManager.
Alternatives
------------
The alternative is to keep adding public APIs for each new console type which
is not really desired.
Data model impact
-----------------
None
REST API impact
---------------
The new API will be exposed with API microversion and will have the following
definition:
Request::
POST /servers/<uuid>/remote-consoles
{
"remote_console": {
"protocol": ["vnc"|"rdp"|"serial"|"spice"],
"type": ["novnc"|"xpvnc"|"rdp-html5"|"spice-html5"|"serial"]
}
}
The 'type' parameter in the request is optional and should be used when the
chosen protocol supports multiple connection types.
Response::
200 OK
{
"remote_console": {
"url": string,
"protocol": ["vnc"|"rdp"|"serial"|"spice"],
"type": ["novnc"|"xpvnc"|"rdp-html5"|"spice-html5"|"serial"]
}
}
Some of failure scenarios and their corresponding error code include:
* wrong values for protocol/type in the request - "400 Bad Request"
* the instance is not yet ready - "409 Conflict"
* the virt driver doesn't support this console type - "501 Not Implemented"
The old API 'os-getVNCConsole', 'os-getSPICEConsole', 'os-getSerialConsole'
and 'os-getRDPConsole' will be removed at the microversion which new API
added.
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
There will be a new 'console-get' subcommand for the Nova CLI that will support
all of the console types.
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
rgerganov
Work Items
----------
There is already a patch which implements the blueprint which didn't land in
Kilo: https://review.openstack.org/#/c/148509/
Dependencies
============
None
Testing
=======
A new test will be added to tempest which will exercise the new API.
Documentation Impact
====================
The new API should be documented and we should encourage users to use this
instead of the old APIs which will be deprecated.
References
==========
None

295
specs/liberty/implemented/db2-database.rst Normal file
View File

@@ -0,0 +1,295 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================
Add Support for DB2 (v10.5+)
============================
https://blueprints.launchpad.net/nova/+spec/db2-database
The community currently supports MySQL and PostgreSQL production databases.
Several other integrated projects already support DB2. This blueprint adds
support to Nova for DB2 as a production database.
Problem description
===================
* Currently there is no support in the community for a deployer to run Nova
against a DB2 backend database.
* For anyone running applications against an existing DB2 database that wants
to move to OpenStack, they'd have to use a different database engine to
run Nova in OpenStack.
* There is currently an inconsistent support matrix across the core projects
since the majority of core projects support DB2 but Nova does not yet.
Use Cases
---------
As a deployer, I want to run Nova against a DB2 backend database so I can use
a single DB2 database engine for multiple integrated OpenStack services.
Project Priority
----------------
None, however, most of the other integrated projects in OpenStack already
support a DB2 backend database or are working toward that.
The integrated projects that currently support DB2 today:
* Ceilometer
* Cinder
* Glance
* Heat
* Ironic
* Keystone
* Neutron
* Trove
The integrated projects that do not yet have DB2 support:
* Sahara
Also, oslo.db and sqlalchemy-migrate have DB2 support.
Proposed change
===============
Add code to support migrating the Nova database against a DB2 backend. This
would require a fresh deployment of Nova since there are no plans to migrate
an existing Nova database from another engine, e.g. MySQL to DB2.
Unit test code would also be updated to support running tests against a DB2
backend with the ibm_db_sa driver and all Nova patches will be tested against a
Tempest full run with 3rd party CI running DB2 that IBM will maintain.
There is already some code in Oslo's db.api layer to support common function
with DB2 like duplicate entry error handling and connection trace, so that is
not part of this spec.
Alternatives
------------
Deployers can use other supported database backends like MySQL or PostgreSQL,
but this may not be an ideal option for customers already running applications
with DB2 that want to integrate with OpenStack. In addition, you could run
other core projects with multiple schemas in a single DB2 OpenStack database,
but you'd have to run Nova separately which is a maintenance/configuration
problem.
Data model impact
-----------------
#. The 216 migration will be updated to handle conditions with DB2 like index
and foreign key creation. The main issue here is that DB2 does not support
unique constraints over nullable columns, it will instead create a unique
index that excludes null keys. Most unique constraints created in Nova are
on non-nullable columns, but the instances.uuid column is nullable and the
216 migration creates a unique index on it, but this will not allow any
foreign keys on the instances.uuid column to be created with DB2 since the
reference column has to be a unique or primary key constraint.
#. In order to support creating the same foreign keys that reference the
instances.uuid column as other database engines, the instances.uuid column
must be made non-nullable and a unique constraint must be created on it.
The dependent blueprint "Enforce unique instance uuid in data model" is
used to handle this change (completed in Kilo).
#. Finally, add another migration script which creates the previously excluded
foreign keys from the 216 migration script for DB2.
REST API impact
---------------
None.
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
None.
Performance Impact
------------------
The only performance impact on existing deployments is in the migration
script changes which would be tested with turbo-hipster.
Other deployer impact
---------------------
The new database migration which creates the missing foreign keys since the
control node needs to be down when running the migration. However, the new
migration only creates foreign keys if the backend is DB2, which would be a new
installation as noted in the `Proposed change`_ section so the impact should be
minimal.
Developer impact
----------------
The only impact on developers is if they are adding DB API code or migrations
that do not work with DB2 they will have to adjust those appropriately, just
like we do today with MySQL and PostgreSQL. IBM active technical contributors
would provide support/guidance on issues like this which require specific
conditions for DB2, although for the most part the DB2 InfoCenter provides
adequate detail on how to work with the engine and provides details on error
codes.
* DB2 SQL error message explanations can be found here:
http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/index.jsp?topic=%2Fcom.ibm.db2.luw.messages.sql.doc%2Fdoc%2Frsqlmsg.html
* Information on developing with DB2 using python can be found here:
http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/index.jsp?topic=%2Fcom.ibm.swg.im.dbclient.python.doc%2Fdoc%2Fc0054366.html
* Main contacts for DB2 questions in OpenStack:
* Matt Riedemann (mriedem@us.ibm.com) - Nova core member
* Brant Knudson (bknudson@us.ibm.com) - Keystone core member
* Jay Bryant (jsbryant@us.ibm.com) - Cinder core member
* Rahul Priyadarshi (rahul.priyadarshi@in.ibm.com) - ibm_db_sa maintainer
* The DB2 CI wiki page also provides contact information for issues with third
party testing failures:
https://wiki.openstack.org/w/index.php?title=IBM/IBM_DB2_CI
Implementation
==============
Assignee(s)
-----------
Primary assignee:
mriedem@us.ibm.com
Work Items
----------
#. Change the 216 migration to work with DB2.
#. Add a new migration to create the excluded foreign keys from the 216 script
for DB2.
#. Make the test_migrations.py module work with a configured DB2 backend for
running unit tests.
See the WIP patch for details: https://review.openstack.org/#/c/69047/
Dependencies
============
* Blueprint "Enforce unique instance uuid in data model" (completed in Kilo):
https://blueprints.launchpad.net/nova/+spec/enforce-unique-instance-uuid-in-db
* DB2 10.5 support was added to sqlalchemy-migrate 0.9 during Icehouse:
https://blueprints.launchpad.net/sqlalchemy-migrate/+spec/add-db2-support
* There are no requirements changes in Nova for the unit tests to work. The
runtime requirements are the ibm-db-sa and ibm_db modules, which are both
available from pypi. sqlalchemy-migrate optionally imports ibm-db-sa. The
ibm-db-sa module requires a natively compiled ibm_db which has the c binding
that talks to the DB2 ODBC/CLI driver.
* Note that only DB2 10.5+ is supported since that's what added unique index
support over nullable columns which is how sqlalchemy-migrate handles unique
constraints over nullable columns.
Testing
=======
There are three types of testing requirements, Tempest, unit test and
turbo-hipster performance/scale tests. Each have different timelines for when
they are proposed to be implemented.
* IBM is already running 3rd party CI for DB2 on the existing Nova WIP patch
that adds DB2 support. The same 3rd party CI is running against all
sqlalchemy-migrate changes with DB2 on py26/py27 and runs Tempest against
Keystone/Glance/Cinder/Heat/Neutron/Ironic patches with a DB2 backend. Once
the DB2 support is merged the DB2 3rd party CI would run against all Nova
patches with a full Tempest run. This is considered required testing for this
blueprint to merge in the Liberty release.
* While code will be added to make the Nova unit tests work against a DB2
backend, running Nova unit tests against DB2 with third party CI is not
considered in the scope of this blueprint for Liberty, but long-term this is
something IBM wants to get running for additional QA coverage for DB2 in
Nova. This is something that would be worked on after getting Tempest
running. The majority of the work here is actually in oslo.db. The plan for
delivering third party unit test coverage is in the 2016.1 'M' release.
* Running 3rd party turbo-hipster CI against DB2 is not in plan for this
blueprint in Liberty but like running unit tests against DB2 in 3rd party CI,
running turbo-hipster against DB2 in 3rd party CI would be a long-term goal
for QA and the IBM team will work on that after Tempest is running and after
unit test CI is worked on. The plan for delivering third party turbo-hipster
performance test coverage is in the 2016.1 'M' release.
* The proposed penalty for failing to deliver third party unit test and/or
turbo-hipster performance test coverage in the M release is that the Nova
team will turn off voting/reporting of DB2 third party CI and not allow DB2
fixes to Nova until the third party CI is available.
* More discussion in the mailing list here:
http://lists.openstack.org/pipermail/openstack-dev/2014-May/035009.html
Documentation Impact
====================
* The install guides in the community do not go into specifics about setting up
the database. The RHEL/Fedora install guide says to use the openstack-db
script provided by openstack-utils in RDO which uses MySQL. The other
install guides just say that SQLite3, MySQL and PostgreSQL are widely used
databases. So for the install guides, those generic statements about
supported databases would be updated to add DB2 to the list. Similar generic
statements are also made in the following places which would be updated as
well:
* http://docs.openstack.org/training-guides/content/developer-getting-started.html
* http://docs.openstack.org/admin-guide-cloud/compute.html
* http://docs.openstack.org/trunk/openstack-ops/content/cloud_controller_design.html
* There are database topics in the security guide, chapters 32-34, so there
would be DB2 considerations there as well, specifically:
* http://docs.openstack.org/security-guide/content/ch041_database-backend-considerations.html
* http://docs.openstack.org/security-guide/content/ch042_database-overview.html
* http://docs.openstack.org/security-guide/content/ch043_database-transport-security.html
References
==========
* Work in progress nova patch: https://review.openstack.org/#/c/69047/
* "Enforce unique instance uuid in data model" spec:
http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/enforce-unique-instance-uuid-in-db.html
* There are Chef cookbooks on stackforge which support configuring OpenStack
to run with an existing DB2 installation:
http://git.openstack.org/cgit/stackforge/cookbook-openstack-common/
* Mailing list thread on third party testing:
http://lists.openstack.org/pipermail/openstack-dev/2014-May/035009.html
* DB2 10.5 InfoCenter: http://pic.dhe.ibm.com/infocenter/db2luw/v10r5/index.jsp
* Some older manual setup instructions for DB2 with OpenStack:
http://www.ibm.com/developerworks/cloud/library/cl-openstackdb2/index.html
* ibm-db-sa: https://code.google.com/p/ibm-db/source/clones?repo=ibm-db-sa
* DB2 Third Party CI Wiki: https://wiki.openstack.org/w/index.php?title=IBM/IBM_DB2_CI

255
specs/liberty/implemented/libvirt-macvtap-vif.rst Normal file
View File

@@ -0,0 +1,255 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================================================================
Add MacVTap as new virtual interface type for libvirt virtualization
====================================================================
https://blueprints.launchpad.net/nova/+spec/libvirt-macvtap-vif
Add macvtap as new virtual interface (vif) type to Nova's libvirt driver.
This is required to attach libvirt managed KVM guests via macvtap to the hosts
network. It will be exploited by a new macvtap ml2 plugin and agent. The
current spec is hosted on github `[1]`_ but the final goal is to have this
under the big tent.
Macvtap guest attachments come with the following values
* Significantly higher throughput than the reference implementation ovs
* Significantly lower latency than the reference implementation ovs
* Significantly lower CPU consumption per throuput in the hypervisor than
the reference implementation ovs
* Built into each kernel - no additional packages required
* Very less configuration required
However, the disadvantage of an macvtap attachment is, that Security Groups
and Rules can technically not be supported for now. More details see section
'Security impact'.
Problem description
===================
The new macvtap neutron driver and agent `[1]`_ requires nova vif driver
integration. The corresponding vif_type should represent a general macvtap
device. It is responsible for
* creating the xml definition accordingly
* the correct plug/unplug operations
Use Cases
----------
* Attaching libvirt/KVM guests via macvtap to the hosts network
No additional impact on actors from the Nova side (configuration is done in
Neutron)
Project Priority
-----------------
None
Proposed change
===============
Adding support for a general macvtap vif_type that is represented by the
following domain.xml: ::
<interface type='direct'>
<source dev='<macvtap_src>' mode=’<mode>’/>
<model type='virtio' />
</interface>
The following attributes are variables that must be propagated by neutron to
nova using the vif dictionary:
* macvtap_src: This is the source device for the macvtap (the device the
macvtap sits on).
* mode: The mode, in which the macvtap device should be instantiated.
e.g. 'bridge' or 'vepa'
Bandwith configurations are supported like with other vif types using a macvtap
based connection (e.g. 'hw_veb').
This proposed change will consider the ongoing os-vif-library discussion
(`[5]`_ or alternative `[3]`_) and
* implement things in this new way, if it gets approved.
* implement things in the old way, if it will be moved out.
The corresponding neutron code will use the vnic_type 'normal'.
Alternatives
------------
**Reusing an exsiting vif_type**
Reusing an already existing type is not possible:
* It depends on `[5]`_ to allow different plug/unplug operations
* It requires a refactoring of existing vif_types xml generation method
* An existing vif_type would have to be renamed which breaks other ml2 plugins
**Creating a new generic vif_type for direct connections**
This will be part of the os-vif-library effort `[5]`_.
**Supported vnic types**
The plan is to support only the vnic_type 'normal'.
However two other vnic_types do exist: 'direct' and 'macvtap'.
These other vnic_types both trigger PCI Requests in nova, so they cannot be
reused. The new macvtap support should be hardware independet. But I see the
confusion with the existing 'macvtap' type by just looking at the name.An idea
would be to rename the type 'macvtap' to somehting like 'sriov-macvtap'. But
this requires another blueprint and probbly an nova api change.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
Macvtap does not provide a hook to apply iptables or ebtables on the guest's
traffic, and therefore does not support Neutron Security Groups and
anti-spoofing-rules. Technically, the linux macvtap driver could implement
such a hook in the future. But even without such a hook, macvtap prevents
MAC spoofing already today.
As a consequence, configuration of the NoopFirewallDriver is desired, like it
is with other integrated direct attachment types.
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None from a Openstack code point of view.
But the network performance of the guest should increase compared to the ovs
reference implementation.
Other deployer impact
---------------------
None for Nova
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
scheuran (andreas.scheuring@de.ibm.com)
Other contributors:
None
Work Items
----------
Dependencies
============
New vif-plug approach:
* `[5]`_ or alternatively
* `[3]`_
Testing
=======
* Unittest
* No tempest tests. They will be added with the corresponding neutron code and
run by a neutron Thirdparty CI.
Documentation Impact
====================
None for Nova
References
==========
* Spec of Macvtap driver/agent:
https://github.com/scheuran/networking-macvtap/blob/bp/initial-macvtap
-support/specs/liberty/macvtap-ml2.rst
.. _[1]: https://github.com/scheuran/networking-macvtap/blob/bp/initial-macvtap
-support/specs/liberty/macvtap-ml2.rst
* Core-Vendor decomposition:
https://github.com/openstack/neutron-specs/blob/master/specs/kilo/
core-vendor-decomposition.rst
.. _[2]: https://github.com/openstack/neutron-specs/blob/master/specs/kilo/
core-vendor-decomposition.rst
* VIF-plug-script proposal:
https://review.openstack.org/#/c/162468/
.. _[3]: https://review.openstack.org/#/c/162468/
* Request for networking-macvtap stackforge project:
https://review.openstack.org/#/c/189644/
.. _[4]: https://review.openstack.org/#/c/189644/
* os-vif-library:
https://review.openstack.org/#/c/193668/
.. _[5]: https://review.openstack.org/#/c/193668/
* Launcpad project of macvtap-agent:
https://launchpad.net/networking-macvtap
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Liberty
- Introduced

140
specs/liberty/implemented/libvirt-set-admin-password.rst Normal file
View File

@@ -0,0 +1,140 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===============================
Libvirt Set Admin Root Password
===============================
https://blueprints.launchpad.net/nova/+spec/libvirt-set-admin-password
Nova provides an API to let users set an administrator password on a
virtual machine which is already active. The purpose of this spec is
to take advantage of the libvirt API set-user-password provided with
version 1.2.16 to implement that feature for Qemu/KVM users.
Problem description
===================
Nova provides API to let users set an administrator password but
Qemu/KVM users cannot take advantage of it.
Use Cases
----------
Users want the ability to reset administrator password of an instance
which is already active by using the command "nova root-password
<instance>"
Project Priority
-----------------
None.
Proposed change
===============
To be noted this feature requires that the image have the qemu guest
agent installed to function. Most of the change will be done in the
libvirt driver of Nova.
In order to support both unix-like (GNU/Linux) virtual machines and
Windows the default behavior will be to update password of username
"root" for unix-like virtual machines and "Administrator" for Windows.
To give more flexibility and provide a way for users to change
administrator password of a different username. A new image property
"os_admin_user" will be introduced to let users define who is the
administrator username to update.
Alternatives
------------
The use case for this API is to allow an admin to re-gain control over
an already running guest for which they have lost the password, or for
an admin to bulk change the passwords across all their running guests,
without having to login to the console of each guest
manually/individually. The inject password feature doesn't really
satisfy that.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
* The end user will have to install a QEMU Guest Agent daemon program
inside the image and set the image property 'hw_qemu_guest_agent'.
* If the image request a different username to be updated, end user
will have to correctly set image property 'os_admin_user'.
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
sahid-ferdjaoui
Work Items
----------
* Implement method set_admin_password
* Extend the method set_admin_password to read in image property for
specific admin user
Dependencies
============
Libvirt 1.2.16
Testing
=======
* Unit tests will cover the new code
* The nova API is already covered by tests
Documentation Impact
====================
The new glance image property will need to be documented.
References
==========
* http://wiki.qemu.org/Features/QAPI/GuestAgent

196
specs/liberty/implemented/libvirt-virtiomq.rst Normal file
View File

@@ -0,0 +1,196 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
libvirt: virtio-net multiqueue
==========================================
https://blueprints.launchpad.net/nova/+spec/libvirt-virtio-net-multiqueue
This blueprint aims to enhance the libvirt driver to enable the virtio-net
multiqueue. This will allow the guest instances to increase the total network
throughput.
Problem description
===================
Today's high-end servers have more processors, and guests running on them often
have an increasing number of vCPUs. In a single virtio-net queue, the scale of
the protocol stack in a guest is restricted, as the network performance does
not scale as the number of vCPUs increases. Guests cannot transmit or
retrieve packets in parallel, as virtio-net has only one TX and RX queue.
Multiqueue virtio-net provides the greatest performance benefit when:
- Traffic packets are relatively large.
- The guest is active on many connections at the same time, with traffic
running between guests, guest to host, or guest to an external system.
- The number of queues is equal to the number of vCPUs. This is because
multi-queue support optimizes RX interrupt affinity and TX queue
selection in order to make a specific queue private to a specific vCPU.
Although the virtio-net multiqueue feature provides a performance benefit,
it has some limitations and therefore should not be unconditionally enabled:
- Guest OS is limited to ~200 MSI vectors. Each NIC queue requires a MSI
vector, as well as any virtio device or assigned PCI device.
Defining an instance with multiple virtio NICs and vCPUs might lead to a
possibility of hitting the guest MSI limit.
- virtio-net multiqueue works well for incoming traffic, but can
occasionally cause a performance degradation, for outgoing traffic.
- Enabling virtio-net multiqueue increases the total network throughput,
but in parallel it also increases the CPU consumption.
- Enabling virtio-net multiqueue in the host QEMU config, does not enable
the functionality in the guest OS. The guest OS administrator needs to
manually turn it on for each guest NIC that requires this feature, using
ethtool.
- MSI vectors would still be consumed (wasted), if multiqueue was enabled
in the host, but has not been enabled in the guest OS by the administrator.
- In case the number of vNICs in a guest instance is proportional to the
number of vCPUs, enabling the multiqueue feature is less important.
- Each virtio-net queue consumes 64 KB of kernel memory for the vhost driver.
Use Cases
---------
Guest instance users may benefit from increased network performance and
throughput.
Project Priority
-----------------
None
Proposed change
===============
In order to address the problem, an administrator should have the control
over the paralleled packet processing by disabling the multiqueue support
for certain workloads, where this feature may lead to a
performance degradation.
Introducing a new parameter to the image properties, for the users
to control the virtio-net multiqueue feature, and be able to disable it,
if desired.
hw_vif_multiqueue_enabled=true|false (default false)
Currently, the number of queues will match the number of vCPUs, defined for the
instance.
NOTE: Virtio-net multiqueue should be enabled in the guest OS manually, using
ethtool. For example:
ethtool -L <NIC> combined #num_of_queues
Alternatives
------------
Considering the limited amount of MSI vectors in the guest,
users could have the ability to create instances with vNICs operating at
various traffic rates, by controlling the use of the multiqueue feature.
For example, one NIC might be used solely for administrative traffic
and so will not need multiqueue enabled, while other NICs might be used for
high throughput traffic and will require multiple queues.
E.g. nova boot -nic net_id=<id>,queues=#
However, currently, there is no agreement on the interface the users use to
specify the number of queues per each vNIC.
Another option would be to replace "queues=#" with "throughput=high|medium|low"
which would set the number of queues to 100%|50%|1 of vCPUs.
There will be a need for a following spec to determine the correct interface.
Finally, the administrator could set the overall number of allowed queues on
the flavor extra_specs. However, since the amount of queues affects only the
guest resources, it should be up to the user to control the amount per each
nic.
Data model impact
-----------------
None
REST API impact
---------------
None.
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
None.
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
vromanso@redhat.com
Work Items
----------
* Update the vif config to set the correct number of vCPUs to
virtio-net devices
* Update the NetworkRequest object to add number of queues per port
Dependencies
============
None
Testing
=======
Unit tests
Requires Libvirt >= 1.0.5
Documentation Impact
====================
The new parameters in flavor extra specs and the new network port property
"queues" in nova boot command.
Some recommendations for an effective usage, should be mentioned as well.
References
==========
http://www.linux-kvm.org/page/Multiqueue

158
specs/liberty/implemented/make-resource-tracker-use-objects.rst Normal file
View File

@@ -0,0 +1,158 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Make Resource Tracker Use Objects
==========================================
https://blueprints.launchpad.net/nova/+spec/make-resource-tracker-use-objects
This blueprint was approved for Kilo but some patches missed the feature
freeze. This proposal is to complete the remaining patches in Liberty.
Nova is converting data structures it uses to communicate via RPC and through
the database to use an object encapsulation called Nova Objects. This supports
of multi-versioning for live-upgrade and database schema independence. This
blueprint covers the conversion of the resource tracker to use Nova Objects.
Problem description
===================
Conversion to Nova Objects will replace dict data structures that are currently
communicated via the conductor API with Nova Object versions. Where necessary
the Nova Objects will be extended to cover parameters that are not already
implemented.
Use Cases
---------
As an operator I want to be able to upgrade Nova without any downtime.
Project Priority
----------------
This blueprint fits under the Live Upgrades priority.
Proposed change
===============
The Nova Object classes that will be used include:
- ComputeNode
- Instance
- Migrations
- Flavor
- Service
- PciDevicePool
The ComputeNode object was modified during the Kilo cycle to include
required fields. A small amount of cleanup is needed around the
PciDevicePool object to correct assignment of tags and the API samples
and tests associated with the os-pci API. PciDevicePool is used
in the ComputeNode object so this is required before finally replacing
compute node data structures in the resource tracker with the ComputeNode
object.
The other objects were added to the resource tracker during Kilo.
Alternatives
------------
None.
Data model impact
-----------------
The objects isolate the code from the database schema. They are written to
operate with existing data model versions. At present the scheduler does not
the ComputeNode object, so code there will need to tolerate changes in
database schema or the format of data stored in fields.
REST API impact
---------------
None. The current tests and API samples for pci stats are incorrect and
will be fixed. This does not change the relvant API, so although the
samples change, the API doesn't.
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
None.
Performance Impact
------------------
None.
Other deployer impact
---------------------
The objects are written to be compatible with the database schema used in
Juno. There is no database migration associated with the remaining
changes in this blueprint and the format of data stored in the fields of
the database will not change. This means that a combination of Kilo and
Liberty versions of the compute nodes will be able to coexist and interact
with the scheduler.
Developer impact
----------------
Developers working on the resource tracker will be required to use the new
objects instead of directly making database calls to conductor.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
pmurray
Work Items
----------
These are the remaining work items left to complete the blueprint.
* Use ComputeNode object in resource tracker
Some of these work items are currently ready for review:
https://review.openstack.org/#/q/topic:bp/make-resource-tracker-use-objects,n,z
Dependencies
============
None
Testing
=======
This does not affect existing tempest tests. Unit tests will be
added for each object and existing tests will be modified to deal
with the new data structure.
Documentation Impact
====================
The os-pci API response samples will be correct. There will be no new feature
and the APIs will not change, but API samples appear in the documentation so
this will cause a minor documentation impact.
References
==========
https://blueprints.launchpad.net/nova/+spec/scheduler-lib

217
specs/liberty/implemented/mark-host-down.rst Normal file
View File

@@ -0,0 +1,217 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===========================================
New nova API call to mark nova-compute down
===========================================
https://blueprints.launchpad.net/nova/+spec/mark-host-down
New API call is needed to change the state of nova-compute service down
immediately. This allows usage of evacuate API without a delay. Also as
external system calling the API will make sure no VMs left running, there
will be no possibility to break shared storage or use same IPs again. API
usage applies mainly for cases where there is single host mapped to
nova-compute. Cases like in Ironic or vSphere would be out of scope.
Problem description
===================
Nova-compute state change for failed or unreachable host is slow and does
not reliably state host is down or not. Evacuation cannot happen fast and
as VMs might still be running, it might lead to reusing same IPs and to data
corruption in case of shared storage. Also there can be an impact on cloud
stability due to ability to schedule VMs on failed host.
Use Cases
----------
As a user I want to fast evacuate VMs in case nova-compute down.
As a user I want to trust VMs will be scheduled to a healthy compute node.
As a user I want to trust no VMs are left running in case nova-compute is
reported down. This can be the case if external system can mark nova-compute
down when notice fault, so it can be trusted that also the corresponding
VMs are really down.
As a deployer I want to deploy external fault monitoring system that can
detect different problems that can be translated as host fault to be informed
to OpenStack and make sure that host is fenced (powered down). Monitoring
system could monitor interfaces, links, services, memory, CPU, HW, hypervisor,
OpenStack services,... and make actions accordingly.
Project Priority
-----------------
Liberty priorities have not yet been defined.
Proposed change
===============
Introducing new services API extensions for setting the power state to up or
down of the nova-compute.
As future work there could be other BP made related to this:
* New notification of service state change.
Related to instances running on host there could also be BPs made:
* There could be an API to set 'power_state: shutdown' for all VMs related to
a single host.
* Currently there is an API to reset VM state one by one. There could be an
API to have the same for all VMs related to a single host.
Alternatives
------------
There is no attractive alternatives to detect all different host faults than
to have a external tool to detect different host faults. For this kind of tool
to exist there needs to be new API in Nova to report fault. Currently there
must have been some kind of workarounds implemented as cannot trust or get the
states from OpenStack fast enough.
Data model impact
-----------------
Nova DB service table will have a new Boolean column ``forced_down`` with false
as default value. Database servicegroup driver ``is_up`` method needs to be
updated to use this to determine service state is down in case value is true.
Otherwise current timestamp based usage is expected. Only when ``forced_down``
flag will be set back to false will nova-compute be allowed to come up and
have the state reported up.
REST API impact
---------------
New compute API to change nova-compute ``forced_down`` flag value to true or
false:
request::
PUT /v2.1/{tenant_id}/os-services/force-down
{
"binary": "nova-compute",
"host": "host1",
"forced_down": true
}
response::
200 OK
{
"service": {
"host": "host1",
"binary": "nova-compute",
"forced_down": true
}
}
request::
PUT /v2.1/{tenant_id}/os-services/force-down
{
"binary": "nova-compute",
"host": "host1",
"forced_down": false
}
response::
200 OK
{
"service": {
"host": "host1",
"binary": "nova-compute",
"forced_down": false
}
}
Service schema will have new optional parameter:
``forced_down``: parameter_types.boolean
This will be in response messages to forced_down requests.
Besides new call, response for list of services will also contain information
about state of forced_down field.
Security impact
---------------
Configurable by policy, defaulting to admin role.
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
Deployer can make use of any external system to detect host fault and report it
to OpenStack.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee: Tomi Juvonen
Other contributors: Ryota Mibu, Roman Dobosz
Work Items
----------
* Test cases.
* REST API and Service changes.
Implementation: https://review.openstack.org/#/c/184086/
* CLI API changes.
* Documentation.
Dependencies
============
None.
Testing
=======
Unit and functional test cases needs to be added.
Documentation Impact
====================
New API needs to be documented:
* Compute API extensions documentation.
http://developer.openstack.org/api-ref-compute-v2.1.html
* nova.compute.api documentation.
http://docs.openstack.org/developer/nova/api/nova.compute.api.html
References
==========
* OPNFV Doctor project: https://wiki.opnfv.org/doctor
* OpenStack Instance HA Proposal:
http://blog.russellbryant.net/2014/10/15/openstack-instance-ha-proposal/
* The Different Facets of OpenStack HA:
http://blog.russellbryant.net/2015/03/10/the-different-facets-of-openstack-ha/

301
specs/liberty/implemented/metadata-service-network-info.rst Normal file
View File

@@ -0,0 +1,301 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==================================================
API: Proxy neutron configuration to guest instance
==================================================
https://blueprints.launchpad.net/nova/+spec/metadata-service-network-info
Improve the networking info given in both config drive and the metadata
service to instances.
Problem description
===================
Use Cases
---------
Currently, cloud-init takes the Debian-style interfaces file that is
generated from a template and has to convert it to an interfaces file for
other OSs such as RHEL or Windows. This becomes more and more challenging as
network configurations get more complex.
Ironic is working with baremetal hardware. Their network configs might
require more complex network configurations such as multiple VLANs over bonded
interfaces. Translating network templates to multiple OS's then becomes much
more challenging than today. These aren't supported in Neutron as of today,
but there are multiple proposed changes to add support. Using a flexible
design will allow new network configurations much more easily.
Alternate Use Cases:
Consider a VM with the first interface configured by DHCP, and all other
interfaces on private networks where the interfaces are statically configured,
but you are not using config drive, just the metadata service, and not
cheating by doing file injection, presenting the data in a guest agnostic
format.
Setting up static routes without declaring a global route in the interfaces
template.
For Future Expansion:
Future use cases would be using this format to create bonded interfaces,
either of physical or virtual interfaces. Many hypervisors are deployed on
hardware with bonded interfaces, so it is sensible for Ironic/TripleO
to require bonds. To create these bonds today, assumptions have to be made
about the interface names that are being bonded, which can change depending
on the OS. With this change, the bonds can be described generically and
implemented in a consistent way for the OS.
Project Priority
----------------
None, but this combined with Neutron support for bonding, will increase the
fault tolerance of Ironic nodes. In the case of Triple O, that would also
increase the fault tolerance of the hardware running Nova in the overcloud.
Proposed change
===============
* Create a versioned network_data (like user_data and vendor_data already in
the metadata service and configdrive) providing more detailed network info
* A flexible JSON schema to deal with complex network layouts,
and which can be extended easily as Neutron supports more configurations
* Information comes from current network_info for instance
* Some things like bonds won't be supported until Neutron supports them
* We only really need concrete info: mac address, fixed IP address, subnet,
gateway, host routes, neutron port-id, neutron network-id, neutron subnet-id
* Links should be split out separate from network information to make tiered
structures like bonds more easily implemented
* VIFs will be supported as a Link
* Physical links will be supported when the neutron-external-attachment-points
blueprint is completed. [1]
* VLANs will be supported as another type of Link
* Add a "services" section for network services that aren't related to a
particular network or interface. The primary use will be DNS servers.
* In the future, bonds can be supported as another type of Link, pointing at
multiple other Links
Alternatives
------------
* Neutron could create the network_data.json and have Nova simply download
the file and add it to the metadata service and configdrive.
Data model impact
-----------------
None
REST API impact
---------------
Sample API for getting network information from metadata service
GET: http://169.254.169.254/openstack/$VERSION/metadata/network_data.json
JSON Response::
{
"links": [
{ // Example of VIF
"id": "interface2", // Generic, generated ID
"type": "vif", // Can be 'vif', 'phy' or (future) 'bond'
"ethernet_mac_address": "a0:36:9f:2c:e8:70", // MAC from Neutron
"vif_id": "E1C90E9F-EAFC-4E2D-8EC9-58B91CEBB53D",
"mtu": 1500 // MTU for links
},
{ // Example of physical NICs
"id": "interface0",
"type": "phy",
"ethernet_mac_address": "a0:36:9f:2c:e8:80",
"mtu": 9000
},
{
"id": "interface1",
"type": "phy",
"ethernet_mac_address": "a0:36:9f:2c:e8:81",
"mtu": 9000
},
{ // Bonding two NICs together (future support)
"id": "bond0",
"type": "bond",
"bond_links": [
"interface0",
"interface1"
],
"ethernet_mac_address": "a0:36:9f:2c:e8:82",
"bond_mode": "802.1ad",
"bond_xmit_hash_policy": "layer3+4",
"bond_miimon": 100
},
{ // Overlaying a VLAN on a bond (future support)
"id": "vlan0",
"type": "vlan",
"vlan_link": "bond0",
"vlan_id": 101,
"vlan_mac_address": "a0:36:9f:2c:e8:80",
"neutron_port_id": "E1C90E9F-EAFC-4E2D-8EC9-58B91CEBB53F"
},
],
"networks": [
{ // Standard VM VIF networking
"id": "private-ipv4",
"type": "ipv4",
"link": "interface0",
"ip_address": "10.184.0.244",
"netmask": "255.255.240.0",
"routes": [
{
"network": "10.0.0.0",
"netmask": "255.0.0.0",
"gateway": "11.0.0.1"
},
{
"network": "0.0.0.0",
"netmask": "0.0.0.0",
"gateway": "23.253.157.1"
}
],
"neutron_network_id": "DA5BB487-5193-4A65-A3DF-4A0055A8C0D7"
},
{ // IPv6
"id": "private-ipv4",
"type": "ipv6",
"link": "interface0",
// supports condensed IPv6 with CIDR netmask
"ip_address": "2001:cdba::3257:9652/24",
"routes": [
{
"network": "::",
"netmask": "::",
"gateway": "fd00::1"
},
{
"network": "::",
"netmask": "ffff:ffff:ffff::",
"gateway": "fd00::1:1"
},
],
"neutron_network_id": "DA5BB487-5193-4A65-A3DF-4A0055A8C0D8"
},
{ // One IP on a VLAN over a bond of two physical NICs (future support)
"id": "publicnet-ipv4",
"type": "ipv4",
"link": "vlan0",
"ip_address": "23.253.157.244",
"netmask": "255.255.255.0",
"dns_nameservers": [
"69.20.0.164",
"69.20.0.196"
],
"routes": [
{
"network": "0.0.0.0",
"netmask": "0.0.0.0",
"gateway": "23.253.157.1"
}
],
"neutron_network_id": "62611D6F-66CB-4270-8B1F-503EF0DD4736"
}
],
"services": [
{
"type": "dns",
"address": "8.8.8.8"
},
{
"type": "dns",
"address": "8.8.4.4"
}
]
}
The same JSON will be stored in the configdrive under
openstack/$VERSION/network_data.json
Security impact
---------------
The JSON data could give more insight into the network than would be
available otherwise to a guest instance. In a locked down environment,
a user may be able see more network details in the metadata service than they
could otherwise discover. An example could be a hardened SELinux VM. A
security note should be documented.
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
The intention is that this network metadata can be used by cloud-init and
other in-instance agents to configure the network in more advanced ways. It
is possible that, depending on the agent's implementation,
the network config could change slightly compared to configs generated prior
to this new metadata. An example is network interfaces being named slightly
differently than the OS would name them. This will be highly dependent on
changes to agents like cloud-init.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
JoshNang
Other contributors:
claxton
JayF
Work Items
----------
* Get basic networking info from neutron into Metadata Service
(list of: mac, IP, subnet, gateway, neutron-port-id, host-routes)
* Add above information into ConfigDrive as "network_data"
Dependencies
============
None
Testing
=======
Unit and functional tests will be added to check if network data is returned.
Documentation Impact
====================
Changes to the Metadata Service api to ask and return network data.
References
==========
[1] https://blueprints.launchpad.net/neutron/+spec/neutron-external-attachment-points
[2] https://etherpad.openstack.org/p/IcehouseNovaMetadataService

315
specs/liberty/implemented/nova-api-policy.rst Normal file
View File

@@ -0,0 +1,315 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=======================================================================
Policy should be enforced at API REST layer where possible (Final Part)
=======================================================================
https://blueprints.launchpad.net/nova/+spec/nova-api-policy-final-part
NOTE: This spec follow up the rest work of nova api policy blueprint:
https://blueprints.launchpad.net/nova/+spec/v3-api-policy , In the Kilo
The v2.1 policy improvement already finished, but only finished part of db
policy improvement and rest of those works will be continue in Liberty.
The detail is described at 'Work Items' section.
This BP proposes enforcing all policy checks only at the Nova REST API
layer. The extra permission checks at the lower layers of Nova will be
removed. There will be consistent policy naming for the V2.1 API and
backwards compatibility will be retained for existing policy checks
related to the V2 API so that the policy checks remain effectively the
same, even though they may in practice be implemented at different points.
This BP is already discussed at Icehouse summit:
https://etherpad.openstack.org/p/icehouse-summit-nova-v3-api
Problem description
===================
Currently policy permission checking is spread through the various
levels of the Nova code. There are also duplicated checks where
effectively the same sort of policy check under different names is
done at different levels such as both at the Nova REST API layer
and the Nova Compute API layer. In addition to this there are also
some cases where there are hard coded permission checks in the db
layer.
This situation makes it much harder for operators to correctly
configure the policy settings that they want and because of the multi
layered complexity of permission the implementation itself is more
vulnerable to security bugs.
A detailed description of the problem:
* Permission checking spread in different level of nova code
Example:
* REST API layer: pause server "compute_extension:admin_actions:pause"
* Compute API layer: pause in compute API "compute:pause"
* DB layer: require_admin_context decorator for db API service_get
* Duplicated policy checking for same API. Example:
* For server's pause action:
* REST API layer:
"compute_extension:admin_actions:pause": "rule:admin_or_owner"
* Compute API layer: "compute:pause": ""
* Hard code policy check at db layer
Example:
::
@require_admin_context
def service_get_all(context, disabled=None):
query = model_query(context, models.Service)
if disabled is not None:
query = query.filter_by(disabled=disabled)
return query.all()
This means it won't have any effect after you modify the policy at REST
API layer, it always enforced as admin at db layer.
Use Cases
---------
1. Operator want to specified role can access service API, but it's hard-code
as only admin can operator those API.
2. As developer view, Only one rule for pause server API at REST API layer.
Developer needn't be confused how to process permission checks in the nova.
Project Priority
----------------
None
Proposed change
===============
Enforce policy at REST API layer. Because REST API will access
different internal APIs, like compute API, DB API or other internal API, the
REST API layer is the place to enforce policy consistently.
* Remove policy check from compute API layer for EC2 and Nova V2.1 API
* For V2.1 API, there will only be policy checks in the nova REST API
layer. There will be a parameter 'skip_policy_check' for compute API to
control whether doing the policy checks. For V2.1 API,
skip_policy_check will be True.
https://review.openstack.org/#/c/100408/2/nova/api/openstack/compute/plugins/v3/shelve.py
* For Ec2, we want to keep the backwards-compatibility. Also we want to
move the compute API layer policy checking into REST API layer, the same
as V2.1 API. That means the old policy and new policy will be available
at sametime. At least after one release, we can remove the old polcy.
* For V2 API, we want to keep the backwards-compatibility. So we won't move
the compute API layer policy checking into REST API layer. We will set
compute API's parameter skip_policy_check to False, that means still
doing policy checking at compute API layer. It's because V2 API will be
depreciated. Before V2 API removed, we needn't take risk of breaking
existing code.
https://review.openstack.org/#/c/100408/2/nova/compute/api.py
* Remove hard-code permission check from db layer
* Example: https://review.openstack.org/#/c/73490/
* For the v2.1 API, we remove all the hard-code permission check from DB
layer. And we should ensure we have policy check at REST API layer.
* For the v2 API, we remove all the hard-code permission check from DB
layer, and move the hard-code permission checks into REST API layer to
keep back-compatibility. V2 API will removed once V2.1 ready, this
can reduce the risk we break something existed.
* Update policy configuration file to match the existing behavior for
EC2 and V2.1 API.
* Correct the policy rule name specification for the v2.1 api and ec2 api
The policy naming style as below:
For V2.1: api:[extension_alias]:[action]
For ec2: ec2_api:[action]
* We won't use 'compute' and 'compute_extension' to distinguish the core and
extension API. Because the core API may be changed in the future.
* We also remove the API version from the policy rule. Because after we have
Micro-version, the version will be changed often.
* For volume related extensions, there isn't any thing can do, there already
have policy checks at REST API layer, also have policy checks by cinder.
* For network related extensions, we are doing same change like compute API.
For nova-network, move all the policy enforcement into REST API layer from
network API, and remove the db layer hard-code permission checks.
For neutron, we didn't have too much can do, neutron has its own policy
enforcement. We just need ensure we have policy enforcement at nova REST
API layer.
Alternatives
------------
The alternative is the status quo which is confusing for both deployers as
well as developers having to maintain the current implementation
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
This BP will remove the policy permission checks in the compute API layer
and DB layer.
These patches will require very rigorous double checking and high
quality reviews to ensure that security bugs are not introduced as the
nova internal calls can be called from quite a few different code
paths (Ec2, V2 API, V2.1 API and other internals).
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
This BP will improve the error handling performance. Because the permission
checking occurs at the API level rather than at a lower level in Nova less
processing will occur before a request is rejected. Also potentially for newer
versions of the API redundant policy checks are removed which will also
improve performance.
Other deployer impact
---------------------
Every effort will be made to keep all existing policy permission
settings backwards compatible for v2 API, except the db hard-code permission
checks. Because v2 API will be removed once v2.1 API is ready.
As v2.1 API isn't ready yet, there isn't any user for v2.1 for now, so we
won't worry about any change will affect the user.
For EC2 API, we also want to keep backwards compatibility, just like v2 API.
The old policy rules will be keep at least for one release. If the user
just want to use the old policy, user can set all the new policy to empty.
Then all the policy will be skipped. If user want to use new policy, just
set the rule into new policy, then new policy will be enforced before old
policy.
The extension will be affect by remove db layer hard-code permission checks
as below:
* services
* certificates
* floating_ips
* floating_ips_bulk
* floating_ip_dns
* fixed_ips
* os-network
* network_associate
* quotas
* quota_classes
* security_group
* security_group_default_rule
* migrations
* flavor_manage
* flavor_access
* cell
* agent
* pci
For v2.1 and ec2 api, the policy rule name prefix is changed. So it need
Deployer update their policy config.
Developer impact
----------------
When a developer adds a new REST API for nova policy permission checks
will only be added at the REST API layer.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Alex Xu <hejie.xu@intel.com>
Other contributors:
Eli Qiao <liyong.qiao@intel.com>
ShaoHe Feng <shaohe.feng@intel.com>
YunTong Jin <yuntong.jin@intel.com>
Park Hei <heijlong@linux.vnet.ibm.com>
jichenjc <jichenjc@cn.ibm.com>
Work Items
----------
The tasks with "(Done)" mean already done at Kilo. Other tasks will be
continue.
* Add parameter to compute and network API to skip policy checks. (Done)
* Improve the EC2 API policy enforcement. (Abandon because EC2 deprecated)
* Add new policy rules at REST API layer
* Add new EC2 API rules
* Move EC2 API rules into separated file.
* Improve the V2.1 API policy enforcement. (Done)
* Remove compute API and network API layer policy enforcement
* Rename V2.1 API rules
* Move V2.1 API rules into separated file.
* Remove db layer hard-code permission checks.
The rest of part is most about nova-network and service/compute_nodes db
calls.
* Add back-compatibility rules into REST API layer for v2 API
* Add policy rules at REST API layer instead of hard-code checks for v2.1
* Move V2 API policy out of policy.conf
Working list:
https://etherpad.openstack.org/p/apipolicycheck
Dependencies
============
None
Testing
=======
No tempest changes. All the policy checks tests will be test by unittest,
as this is mostly an internal nova blueprint.
Documentation Impact
====================
The db layer permission checks will be deleted, this should be document at
upgrade documentation.
All the policy should enforce at API layer, this should be document at
developer documentation.
For the consistent configuration of policy rule, this should be document at
Cloud Admin documentation.
References
==========
https://etherpad.openstack.org/p/icehouse-summit-nova-v3-api

168
specs/liberty/implemented/nova-rootwrap-daemon-mode.rst Normal file
View File

@@ -0,0 +1,168 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
====================
rootwrap daemon mode
====================
https://blueprints.launchpad.net/nova/+spec/nova-rootwrap-daemon-mode
Nova is one of projects that heavily depends on executing actions on compute
and network nodes that require root priviledges on Linux system. Currently this
is achieved with oslo.rootwrap that has to be run with sudo. Both sudo and
rootwrap produce significant performance overhead. This blueprint is one of the
series of blueprints that would cover mitigating rootwrap part of the overhead
using new mode of operations for rootwrap - daemon mode. Neutron has already
adopted this approach.
Problem description
===================
As you can see in [#ne_ml]_ rootwrap presents big performance overhead for
Neutron. Impact on Nova is not as signigicant since most of the work is done
with libvirt's API but it is still there.
Details of the overhead are covered in [#rw_bp]_.
Use Cases
----------
This will eliminate bottleneck in nova-network, nova-compute at boot large of
number of nodes.
Project Priority
-----------------
None
Proposed change
===============
This blueprint proposes adopting functionality in oslo.rootwrap that would
allow to run rootwrap daemon. The daemon will work just as a usual rootwrap but
will accept commands to be run over authenticated UNIX domain socket instead of
command line and will run continuously in background.
Note that this is not usual RPC over some message queue. It uses UNIX socket,
so no remote connections are available. It also uses digest authentication with
key shared over stdout (pipe) with parent process, so no other processes will
have access to the daemon. Further details of rootwrap daemon are covered in
[#rw_bp]_.
``use_rootwrap_daemon`` configuration option should be added that will make
``utils.execute`` use daemon instead of usual rootwrap.
Alternatives
------------
Alternative approaches have been discussed for Neutron in [#ne_eth]_.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
This change requires additional endpoint to be available to run as root -
``nova-rootwrap-daemon``. It should be added to the ``sudoers`` file.
All security issues with using client+daemon instead of plain rootwrap are
covered in [#rw_bp]_.
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
This change introduces performance boost for disk and network operations that
are required to be run with root priviledges in ``nova-compute`` and
``nova-network``. Current state of rootwrap daemon shows over 10x speedup
comparing to usual ``sudo rootwrap`` call. Total speedup for Nova will be less
impressive but should be noticeable.
Looking at numbers from check-tempest-dsvm-full CI job ([#nova_perf]_) with
the rootwrap daemon mode on and off, here's what we see:
Daemon Off - Average 0.08981064764 seconds
Daemon On - Average 0.02984345922 seconds
Other deployer impact
---------------------
This change introduces new config variable ``use_rootwrap_daemon`` that
switches on new behavior. Note that by default ``use_rootwrap_daemon`` will be
turned off so to get the speedup one will have to turn it on. With it turned on
``nova-rootwrap-daemon`` is used to run commands that require root priviledges.
This change also introduces new binary ``nova-rootwrap-daemon`` that should
be deployed beside ``nova-rootwrap`` and added to ``sudoers``.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Davanum Srinivas <davanum@gmail.com>
Work Items
----------
The only work item here is to implement new config variable and run rootwrap in
daemon mode with it.
Dependencies
============
* rootwrap-daemon-mode blueprint in oslo.rootwrap [#rw_bp]_.
Testing
=======
This change doesn't change APIs so it doesn't require additional integration
tests. If tempest is happy with ``use_rootwrap_daemon`` turned on, the feature
works. We can turn this flag on for some of the jobs say the nova-network
job.
Documentation Impact
====================
None
References
==========
.. [#rw_bp] oslo.rootwrap blueprint:
https://blueprints.launchpad.net/oslo.rootwrap/+spec/nova-rootwrap-daemon-mode
.. [#ne_ml] Original mailing list thread:
http://lists.openstack.org/pipermail/openstack-dev/2014-March/029017.html
.. [#ne_eth] Original problem statement summarized here:
https://etherpad.openstack.org/p/neutron-agent-exec-performance
.. [#nova_perf] Nova check-tempest-dsvm-full comparison:
https://docs.google.com/spreadsheets/d/1sxhan2fRg6eshY4559O8z1g8sFPRXma00xz53nZ6sAI/edit#gid=870990378

331
specs/liberty/implemented/online-schema-changes.rst Normal file
View File

@@ -0,0 +1,331 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=====================
Online Schema Changes
=====================
https://blueprints.launchpad.net/nova/+spec/online-schema-changes
Make schema changes execute online (ie while services are running) when
safely and semantically possible. This will allow operators to reduce the
amount of downtime currently required during deploys.
Problem description
===================
* All database migrations are currently required to be run offline.
* Database migrations have historically been a source of lengthy downtime
during deployments.
* Schema changes are required to be repeated in two places: database
model defined in Nova and writing a migration script.
Use Cases
---------
* Any deployer that would like to reduce the amount of downtime during
deploys.
* Developers that would like to spend less time writing migration
scripts.
* Deployers that would like to maintain a set of local schema changes.
Project Priority
----------------
This fits under the 'Live Upgrades' liberty priorities.
Proposed change
===============
A new alternative workflow for applying schema changes will be added
that expands the schema, then contracts the schema (expand/contract
workflow).
The new expand/contract workflow will not utilize any migration scripts,
instead it will dynamically compare the running schema against the
database model defined in Nova. DDL statements will be generated, and
optionally executed, to make the running schema match the model.
The existing schema management workflow is the 'db sync' command to
nova-manage. This is managed by sqlalchemy-migrate and uses individual
migration scripts. This workflow will remain for now, but is expected
to be be removed at some future time, leaving the expand/contract
workflow as the way to manage the database schema.
Until the sqlalchemy-migrate workflow is removed, all schema changes
will still need sqlalchemy-migrate migration scripts to be written.
Three new nova-manage commands will be added:
#. expand. This would apply changes that are compatible with old running
code.
#. migrate. This would apply changes that are necessary to be run offline.
#. contract. This would apply changes that are compatible with new
running code.
Those schema changes that can be safely and semantically applied while
running online will be applied during the expand and contract phases.
Also, only those schema changes that will not acquire long running
locks in the database will be considered for the online phases (expand,
contract). All other schema changes will be applied during the migrate
phase.
The three new commands would be built by dynamically executing alembic's
autogenerate and DDL generating features. A list of differences would
be generated by alembic and then DDL statements would be generated using
a separate feature of alembic.
The set of DDL statements that can be run in each phase would be dictated
by the database software used (eg MySQL, PostgreSQL, etc), the version of
the database software (eg MySQL 5.5, 5.6, etc) and the storage engine used
(eg InnoDB, TokuDB, etc).
As an example, index additions can be executed online in MySQL 5.5, but
not 5.1. An index addition would be run during the expand phase for
MySQL 5.5 or higher, but during the migrate phase for MySQL 5.1.
It is intended that the initial set that will run online will be
conservative at first and a subset of what is possible to run safely.
This can be safely expanded at any time in the future.
Schema changes that will be potentially performed during expand:
- Table creates
- Column additions
- Non-Unique Index additions
Schema changes that will be potentially performed during migrate:
- Unique Index additions/drops
- Foreign Key additions/drops
Schema changes that will be potentially performed during contract:
- Table drops
- Column drops
- Non-Unique Index drops
Some schema changes that aren't currently used or are difficult to
automate will not be allowed initially. For instance, column type
changes will not be allowed initially. This is because not all column
type changes can be automated because of complexity and database
restrictions. A subset of column type changes may be implemented in
the future if they can be automated on all databases.
The migrate and contract phases would verify that the previous phases
(expand in the case of migrate, expand and migrate in the case of
contract) no longer need to be executed before continuing.
This would be performed by generating the list of needed changes for
the previous phases and verifying the list is empty. This indicates the
previous phases were either run or unnecessary.
A new '--dryrun' argument would print, instead of execute, each
generated DDL statement. This could be used by database administrators
to see what would be executed for a particular phase. These can be
optionally executed manually if desired. The schema synchronizer will
not generate that DDL statement since the running schema does not
have that difference anymore.
When 'db contract' is finally run and the running schema has been
verified to match the models, the version in the migrate_version
table would be updated to the latest shipped sqlalchemy-migrate
migration version. This would maintain compatibility with the
existing sqlalchemy-migrate workflow.
The fundamental difference between the two workflows is the
expand/contract workflow is declarative (by using the model) and the
sqlalchemy-migrate workflow is imperative (by using migration scripts).
By being declarative, it limits changes to one place (database model)
and allows for more intelligent decisions (by factoring in the database
software, engine, version, etc)
Alternatives
------------
Splitting the existing single stream of migrations into three separate
streams of migrations. This would allow some schema changes to be
executed online.
This limits the schema changes that can be safely executed online to
that of the lowest common denominator of databases supported by Nova.
This would also require changes to sqlalchemy-migrate to be able to
manage separate streams of migrations.
Another option would be remove the use of sqlalchemy-migrate for schema
changes altogether. The 'db sync' command to nova-manage would be
implemented by effectively calling 'db expand', 'db migrate' and
'db contract'.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
Running online DDL changes can affect the performance of a running system.
This is optional and is only done when the deployer explicitly requests
it.
This can mitigated by the deployer by scheduling the expand and contract
phases to be run during periods of low activity. The expand phase can
be run an arbitrary amount of time before the migrate phase. Likewise,
the contract phase does not need to be run immediately after the
migrate phase is run.
Other deployer impact
---------------------
Using the new expand/contract workflow is optional. If the deployer
does not want to perform database schema changes online, then they
can continue using the 'db sync' command with nova-manage.
Those deployers that want to take advantage of the online schema changes
will need to run the 'db expand', 'db migrate' and 'db contract' commands
at the appropriate steps in their deployment process.
Switching from the sqlalchemy-migrate workflow to the expand/contract
workflow can happen at any time. The reverse can only happen after
a final 'db contract' is run (to ensure all schema changes are applied
and the migrate_version table is updated).
If the expand/contract workflow is used, then 'db contract' is required
to be execute once for each formal release of Nova. This is to ensure
that SQL namespaces (table, column, etc) can be reused in the future.
Deployers that have made local schema changes (extra indexes, columns,
tables, etc) will need to update the model to ensure those additions
aren't dropped during the contract phase.
If using the expand/contract workflow, then deployers can run 'db expand'
before stopping or restarting any services. 'db migrate' might acquire
locks in the database and may affect running services. 'db contract' can
be run after all Nova services are running the new code.
Developer impact
----------------
Eventually no more sqlalchemy-migrate migrations would need to be written
leading to less work for developers.
No more migration compaction. The initial creation of tables for a
database is handled completely by the schema synchronizer.
Some schema changes will no longer be allowed. This is generally
restricted to schema changes that cannot be reasonably automated but
those schema changes are generally the ones with the most downtime
anyway.
Namespaces (table, column, index, etc) are not reusable in a formal
release cycle. The contract phase is only required to be executed once
per formal release, pinning old names until the next formal release.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
johannes.erdfelt
Other contributors:
None
Work Items
----------
- Implement schema synchronizer using alembic.autogenerate
- Implement new 'expand', 'migrate' and 'contract' commands to
'nova-manage db'
- Ensure grenade and turbo-hipster tests are update
Dependencies
============
This builds on top of the validate-migrations-and-model spec. The
existing use of alembic.autogenerate will now also be used to generate
the list of needed changes to make the schema match the model.
This also depends on dropping the use of sqlalchemy-migrate for data
migrations.
Testing
=======
No tempest tests will be added since tempest does not do any upgrade
testing.
A Nova unit test will be added to test starting from an empty database.
Grenade currently tests upgrades from older versions of Nova. A new
test to use the new 'db expand', 'db migrate' and 'db contract' commands
are necessary. This will be compared with the result of 'db sync' to
ensure that upgrades from past commits end up semantically identical.
turbo-hipster tests upgrades using production database snapshots. It
currently uses the 'db sync' command to nova-manage. The new
expand/contract workflow will be tested as well to ensure that both
workflows function correctly.
Documentation Impact
====================
Documentation will need to be updated to include the new 'expand',
'migrate' and 'contract' commands to 'nova-manage db'.
Release Notes will need to be updated to warn that the model will need
to be updated with local schema changes.
Instance Types would need to be manually created as the 216 migration
would not necessarily run anymore.
References
==========
https://etherpad.openstack.org/p/kilo-nova-zero-downtime-upgrades

130
specs/liberty/implemented/rename-parallels-to-virtuozzo.rst Normal file
View File

@@ -0,0 +1,130 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Rename Parallels Cloud Server to Virtuozzo
==========================================
https://blueprints.launchpad.net/nova/+spec/rename-pcs-to-virtuozzo
Problem description
===================
Within Parallels company rebranding, Parallels Cloud Server product, whose
support was merged in Kilo, was renamed to Virtuozzo [1]_.
Moreover, Parallels Service provider business was renamed
and became Odin [2]_. Thus, there was no Parallels name left for this product.
We need to address this change in nova/libvirt driver accordingly to avoid
confusion of users and make things be actual.
It is worth noting though, that Virtuozzo is based on opensource and free
OpenVz project and supporting Virtuozzo means supporting OpenVz also.
Use Cases
----------
Project Priority
-----------------
No priority defined for this change yet.
Proposed change
===============
Libvirt section in nova.conf will change ``virt_type`` from ``parallels`` to
``vz``. We set minimal required version of libvirt to work with ``vz``
virt_type to 1.3.0 since it is the first version that supports 'vz' uri.
The code will hold both names: ``parallels`` and ``vz`` for next release
cycle. In ``M`` release cycle there should be left only ``vz``.
Alternatives
------------
We can leave it as is and have ``parallels`` virt_type but this approach
will leave confusion for user forever.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
Deployer can use libvirt virt_type ``parallels`` in Liberty release cycle,
but will have to switch to ``vz`` in next ``M`` release. This action is
going to be fulfilled by Virtuozzo deployment system.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee: Maxim Nestratov <mnestratov@parallels.com>
Work Items
----------
* Test cases.
Implementation: https://review.openstack.org/#/c/184311/
* Documentation.
Dependencies
============
None.
Testing
=======
Unit test cases need to be updated.
Already addressed in pending implementation.
Documentation Impact
====================
* OpenStack Configuration Reference.
http://docs.openstack.org/kilo/config-reference/content/compute-nova-conf.html
* OpenStack Configuration Reference.
http://docs.openstack.org/kilo/config-reference/content/list-of-compute-config-options.html
References
==========
.. [1] http://www.odin.com/products/virtuozzo/#tab4
.. [2] http://www.odin.com/odin/

248
specs/liberty/implemented/request-spec-object.rst Normal file
View File

@@ -0,0 +1,248 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=========================
Create RequestSpec Object
=========================
https://blueprints.launchpad.net/nova/+spec/request-spec-object
Add a structured, documented object that represents a specification for
launching multiple instances in a cloud.
Problem description
===================
The main interface into the scheduler, the `select_destinations()` method,
accepts a `request_spec` parameter that is a nested dict. This nested dict is
constructed in `nova.scheduler.utils.build_request_spec()`, however the
structure of the request spec is not documented anywhere and the filters in the
scheduler seem to take a laisse faire approach to querying the object during
scheduling as well as modifying the `request_spec` object during loops of the
`nova.scheduler.host_manager.HostStateManager.get_filtered_hosts()` method,
which calls the filter object's `host_passes` object, supplying a
`filter_properties` parameter, which itself has a key called `request_spec`
that contains the aforementioned nested dict.
This situation makes it very difficult to understand exactly what is going on
in the scheduler, and cleaning up this parameter in the scheduler interface is
a pre-requisite to making a properly-versioned and properly-documented
interface in preparation for a split-out of the scheduler code.
Use Cases
----------
This is a pure refactoring effort for cleaning up all the interfaces in between
Nova and the scheduler so the scheduler could be split out by the next cycle.
Project Priority
-----------------
This blueprint is part of a global effort around the 'scheduler' refactoring
for helping it to be split out. It was formerly identified as a priority in
Kilo.
Proposed change
===============
A new class called `RequestSpec` will be created that models a request to
launch multiple virtual machine instances. The first version of the
`RequestSpec` object will simply be an objectified version of the current
dictionary parameter. The scheduler will construct this `RequestSpec` object
from the `request_spec` dictionary itself.
The existing
`nova.scheduler.utils.build_request_spec` method will be removed in favor of a
factory method on `nova.objects.request_spec.RequestSpec` that will construct
a `RequestSpec` from the existing key/value pairs in the `request_spec`
parameter supplied to `select_destinations`.
Alternatives
------------
None.
Data model impact
-----------------
This spec is not focusing on persisting the RequestSpec object but another
blueprint (and a spec) will be proposed with this one as dependency for
providing a save() method to the RequestSpec object which would allow it to be
persisted in (probably) instance_extra DB table.
REST API impact
---------------
None.
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
None.
Performance Impact
------------------
None.
Other deployer impact
---------------------
None.
Developer impact
----------------
None, besides making the scheduler call interfaces gradually easier to read
and understand.
Implementation
==============
The `request_spec` dictionary is currently constructed by the nova-conductor
when it calls the `nova.scheduler.utils.build_request_spec()` function, which
looks like this:
.. code:: python
def build_request_spec(ctxt, image, instances, instance_type=None):
"""Build a request_spec for the scheduler.
The request_spec assumes that all instances to be scheduled are the same
type.
"""
instance = instances[0]
if isinstance(instance, obj_base.NovaObject):
instance = obj_base.obj_to_primitive(instance)
if instance_type is None:
instance_type = flavors.extract_flavor(instance)
# NOTE(comstud): This is a bit ugly, but will get cleaned up when
# we're passing an InstanceType internal object.
extra_specs = db.flavor_extra_specs_get(ctxt, instance_type['flavorid'])
instance_type['extra_specs'] = extra_specs
request_spec = {
'image': image or {},
'instance_properties': instance,
'instance_type': instance_type,
'num_instances': len(instances),
# NOTE(alaski): This should be removed as logic moves from the
# scheduler to conductor. Provides backwards compatibility now.
'instance_uuids': [inst['uuid'] for inst in instances]}
return jsonutils.to_primitive(request_spec)
As the filter_properties dictionary is hydrated with the request_spec
dictionary, this proposal is merging both dictionaries into a single object.
A possible first version of a class interface for the `RequestSpec`
class would look like this, in order to be as close to a straight conversion
from the nested dict's keys to object attribute notation:
.. code:: python
class RequestSpec(base.NovaObject):
"""Models the request to launch one or more instances in the cloud."""
VERSION = '1.0'
fields = {
'image': fields.ObjectField('ImageMeta', nullable=False),
'root_gb': fields.IntegerField(nullable=False),
'ephemeral_gb': fields.IntegerField(nullable=False),
'memory_mb: fields.IntegerField(nullable=False),
'vcpus': fields.IntegerField(nullable=False),
'numa_topology': fields.ObjectField('InstanceNUMATopology',
nullable=True),
'project_id': fields.StringField(nullable=True),
'os_type': fields.StringField(nullable=True),
'availability_zone': fields.StringField(nullable=True),
'instance_type': fields.ObjectField('Flavor', nullable=False),
'num_instances': fields.IntegerField(default=1),
'force_hosts': fields.StringField(nullable=True),
'force_nodes': fields.StringField(nullable=True),
'pci_requests': fields.ListOfObjectsField('PCIRequest', nullable=True),
'retry': fields.ObjectField('Retry', nullable=True),
'limits': fields.ObjectField('Limits', nullable=True),
'group': fields.ObjectField('GroupInfo', nullable=True),
'scheduler_hints': fields.DictOfStringsField(nullable=True)
}
This blueprint targets to provide a new Scheduler API method which would only
accept RequestSpec objects in replacement of select_destinations() which would
be deprecated and removed in a later cycle.
That RPC API method could be having the following signature:
.. code:: python
def select_nodes(RequestSpec):
# ...
As said above in the data model impact section, this blueprint is not targeting
to persist this object at the moment.
Assignee(s)
-----------
Primary assignee:
bauzas
Other contributors:
None
Work Items
----------
- Add request spec class to `nova/objects/request_spec.py` w/ unit tests
- Add a factory classmethod on `nova.objects.RequestSpec` that constructs a
`RequestSpec` object from the *existing* set of instance type extra_specs,
scheduler_hints, flavor and image objects that are supplied to the
`nova.scheduler.utils.build_request_spec` function.
- Convert all filter classes to operate against the `RequestSpec` object
instead the nested `request_spec` dictionary.
- Add developer reference documentation for what the request spec models.
Dependencies
============
None.
Testing
=======
New unit tests for the request spec objects will be added. The existing unit
tests of the scheduler filters will be modified to access the `RequestSpec`
object in the `filter_properties` dictionary.
Documentation Impact
====================
Update any developer reference material that might be referencing the old
dictionary accesses.
References
==========
This blueprint is part of an overall effort to clean up, version, and stabilize
the interfaces between the nova-api, nova-scheduler, nova-conductor and
nova-compute daemons that involve scheduling and resource decisions.

205
specs/liberty/implemented/robustify_evacuate.rst Normal file
View File

@@ -0,0 +1,205 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==================
Robustify Evacuate
==================
https://blueprints.launchpad.net/nova/+spec/robustify-evacuate
Evacuate provides the ability for a deployer to relocate and rebuild
instances when a compute host goes down for an extended period of
time. It does this by simply rebuilding the instance elsewhere. When
the initial compute host comes back up, some logic on startup looks at
instances that appear to have been relocated and deletes them from the
local machine. This is problematic in practice, as we have very little
information on which to base a decision to destroy data -- making it
effectively a guess.
This spec proposes to improve the robustness of this process by
explicitly recording the event of an instance evacuation such that the
recovering host has a very clear indication that it should delete the
local copy of an instance.
Problem description
===================
The evacuate cleanup logic in Nova guesses about whether to destroy
data. It shouldn't guess, as it's relatively easy to cause Nova to
guess poorly, which results in data loss.
As an example of this: right now if you have a libvirt-based system, an
accidental hostname change will destroy data. Imagine a deployer that
has a compute node with a failed CPU. The easy fix is to swap the
disks into an identical machine and reboot. If the new machine causes
the host to get a different hostname (because it has a different MAC
on the NIC, or is in a different place in the rack), as soon as
nova-compute boots up, it will assume all the instances have moved and
will destroy them.
Another example is a vmware deployer that has multiple vcenters. When
bringing up vcenter03.foo.com, the config file contains a typo which
points nova-compute at vcenter02.foo.com. When nova starts, it will
assume that all the instances have been evacuated and will destroy all
their data (on shared storage, even).
Use Cases
----------
A developer wants to enable evacuation on their cloud and have Nova
only destroy data when it has been safely moved to another host.
Project Priority
-----------------
Robustness? Tech debt?
Proposed change
===============
I propose that we make the evacuation code record the action using the
existing migration data structure. That structure contains a
timestamped record of a source host and destination host, and
currently provides a small state machine for confirming or reverting
user-initiated migration requests. We need the exact same thing, but
for evacuation actions, which are confirmed by the source host when it
comes back up when it destroys the data.
When a host starts up, it should:
1. Check for unconfirmed evacuation migrations where it is the source
host
2. Delete the local data, if still present
3. Mark the migration as completed
This means that we need to add a "type" or "reason" field to the
current migration object so that we can keep user/admin-initiated
migration records separate from evacuation-initiated ones (when and
where appropriate).
Since we keep migration entries in the database, this becomes a sort
of log of actions, including the tombstones left for compute nodes to
clean up when they return. Even if there are multiple evacuations
before recovery begins, each host can make an informed decision about
what to do for recovery.
Alternatives
------------
One alternative is to do what we have now, which is that we guess
about whether we need to delete things based on the host field of an
instance having apparently changed since we last started up. That
means that you can trigger the deletion code by changing your
hypervisor hostname.
A permutation of the above is to use a globally-unique hypervisor
identifier provided by the virt driver attached to an instance. This
functions just like the host field, where we try to guess whether we
should delete something based on the current and expected values for
those fields. However, both of these approaches suffer from the fact
that they're guessing based on circumstantial data instead of a record
that tells us an evacuation was performed.
Data model impact
-----------------
This would re-use our existing migration data structure, so data model
impact would be minor. We would need to add at least one field to
track the method of the migration. This would be something like
"resize", "migrate", or "evacuate". This will let us extend the
migration log model to live migration as well as any other methods of
moving instances around so that the log becomes accurate enough to
make proper decisions.
REST API impact
---------------
No impact to the REST API is necessary as we would continue to only
show user-initiated migrations through the existing API interfaces (as
keyed by the type or reason field added above).
If it is desirable to expose these evacuation-related migrations to
the user/admin then we could either extend the existing os-migrations
API or create a new evacuation-specific interface.
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
Users will not have their data deleted based on a guess. Otherwise
they will not notice this change.
Performance Impact
------------------
There will be additional database traffic on compute node startup to
look up the evacuation records. Since we already look up all
instances, usually for no reason, this could actually reduce the
overall traffic and improve performance at startup.
Other deployer impact
---------------------
No real deployer impact, other than safer operation of the evacuation
recovery code.
Developer impact
----------------
None.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
danms
Work Items
----------
* Extend the existing migration object to support a reason/type field
* Make the current evacuation code create a migration record when
instances are evacuated
* Make the compute node startup code use and confirm these records on
startup instead of guessing based on hostname.
* Mark the destroy_after_evacuate workaround config option as
deprecated
Dependencies
============
None.
Testing
=======
Testing this in tempest is tricky because it effectively requires
downing a running compute node. Thus, functional tests will be used to
spin up a reasonably full environment with multiple computes in order
to test the feature.
Documentation Impact
====================
No real impact to users or deployers is expected, and thus no
documentation changes are expected.
References
==========

216
specs/liberty/implemented/service-version-number.rst Normal file
View File

@@ -0,0 +1,216 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================================
Add a service version number to the database
============================================
https://blueprints.launchpad.net/nova/+spec/service-version-number
We have previously identified online data migrations as critical for
supporting live upgrades. We do well with that today, as long as
control services are upgraded together (conductor, api, etc). In order
to extend this further, we need a way to determine that some of the
control services have been upgraded, but not all. This information
will allow us to avoid upgrading data online until all of the services
are upgraded to the point at which they can handle the new schema.
Problem description
===================
Partially upgrading control services will result in newer conductors
beginning to convert data to expand into new schema before older
conductors and other control services are ready. If you upgrade all
your control services together, this works well, but if you don't (as
would be more realistic) you have the potential to break some of the
older control services.
For example, in Kilo we can apply the database schema before starting
to upgrade any of the code. However, once we upgrade any one component
that talks to the database directly, online migrations will begin to
happen, and any other nodes that read from the database directly will
be confused as things are starting to move. Since services like
nova-api, nova-scheduler, nova-conductor, etc all talk directly to the
database, we're currently unable to upgrade these services
independently. If we had this service version number available, we
could avoid doing any online migrations until all the affected
services are upgraded to a new-enough point.
Use Cases
----------
As a deployer, I want to be able to stage my upgrades of control
services, without having to take down *all* of my conductor, api,
scheduler, etc nodes and restart them together spanning a live data
migration.
Project Priority
-----------------
This is an upgrades enhancement.
Proposed change
===============
The proposed change is adding a version number column to the services
table. Each service will report its version number when it updates its
service record. We will be able to determine if all services are on
the same level of code by checking to see if there is more than one
version in the table (optionally grouped by service). We can use this
information to conditionally enable online data migrations. For
example, we can check to see if all the conductors are upgraded to the
same level by doing something like this:
SELECT DISTINCT version FROM services WHERE binary='conductor';
If we do that at conductor startup, we can set a flag to not enable
migrations that require conductor services to be newer than a specific
version. We could also refresh this on SIGHUP like we currently do for
config reload.
Initially, the column will be created with a default of NULL, and the
Service object will treat NULL as "version 1". Subsequent changes will
set versions to 2. Any time we do an online data migration, we will
need to bump the service version number.
Longer-term we could try to tie RPC versions to this information to
make pinning easier. Because that has potentially hard-to-quantify
implications on backports (which occasionally do need to modify RPC
interfaces), I propose we leave that out of the scope of this work for
now. Even still, tying RPC versions to this service version would be
work for the next cycle once this is in place and we can depend on
it.
The other thing that this enables is the ability to have a service
start up and know that it is massively out of date. Presumably we
should be able to have a conductor start up and say "wow, I'm much
older than the other conductors, I should log an error and exit."
Determining what the minimum version is and should be is something we
would do in M when we have this for the N and N-1 releases such that
we can depend on it.
We need to do this in L so that we can leverage it in M. If we delay
this until M, we won't be able to rely on it until N.
Why not use semver?
-------------------
In things like our RPC API versions, we use semver-like version
numbers. This allows us to make decisions about incoming calls and
whether they're compatible with a newer node, and generally define
rules for what is a breaking (i.e. major bump) change.
This service version number doesn't imply any semantics itself, but
rather just provides a vector with which we can orient ourselves in
time to make other decisions. As described elsewhere in this spec,
that may mean that we can decide what RPC version to use, or whether
it is safe to start doing online data migrations. *Those* decisions
extract semantic meaning out of the service version vector, and they
may have significantly different rules (as would certainly be the case
with the aforementioned RPC and DB decisions).
Alternatives
------------
We can continue to do what we do today, which is start converting data
from one schema to the other as soon as we roll new code. We keep the
restriction of all control services being required to update together.
We could also codify this in config somehow, but that will require
much more operator intervention, and increases the likelihood of error.
Data model impact
-----------------
A schema migration will be written to add a version column to the
table. The version will be an integer, nullable, default to NULL.
The Service object will be extended to support this version, and will
treat a NULL version as version "1", which will avoid us having to do
any data migrations on existing service records. New saves will
initially write version "2" for all services.
REST API impact
---------------
None.
Security impact
---------------
None.
Notifications impact
--------------------
None.
Other end user impact
---------------------
None.
Performance Impact
------------------
None.
Other deployer impact
---------------------
This will make upgrading nova services easier and more
flexible/forgiving for deployers.
Developer impact
----------------
Developers (and reviewers) will need to ensure that the service
version number is bumped across any online data migration that we do
(like the recent flavor restructuring).
Implementation
==============
Assignee(s)
-----------
Primary assignee:
danms
Other contributors:
alaski
Work Items
----------
* Write the schema migration
* Update the sqlalchemy models and service object
* Write some object methods to help with querying service version
numbers in ways that will be friendly for determining upgrade
feasibility.
* Extend the service startup code to check version spread and persist
so that we can use that as a static flag for enabling migrations.
Dependencies
============
None. This is mostly early setup for being able to do more interesting
upgrade things in M.
Testing
=======
This should be fully testable with unit tests.
Documentation Impact
====================
Ideally, this should make upgrades require less documentation.
References
==========
* http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/flavor-from-sysmeta-to-blob.html

216
specs/liberty/implemented/show-reserved-status-in-os-fixed-ips-api.rst Normal file
View File

@@ -0,0 +1,216 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Show 'reserved' status in os-fixed-ips API
==========================================
https://blueprints.launchpad.net/nova/+spec/show-reserved-status-in-os-fixed-ips-api
Show the 'reserved' status on a FixedIP object in the os-fixed-ips API
extension. The extension allows one to reserve and unreserve a fixed IP but the
show method does not report the current status.
Problem description
===================
The os-fixed-ips API extension currently allows one to set the 'reserved'
field on a FixedIP object in the database but the show method does not return
the current value so if you want to write an application to reserve/unreserve
fixed IPs today, you have to keep track of this information externally or get
it from the database yourself.
Use Cases
---------
As a cloud administrator, I want to reserve/unreserve fixed IPs but I need to
know the current reserved status on a given fixed IP before I can act on it.
Project Priority
----------------
None
Proposed change
===============
Add a new API microversion to the os-fixed-ips API extension such that if the
version on the API GET request satisfies the minimum version required, include
the 'reserved' status in the fixed_ip response data.
Alternatives
------------
We could add this information to the 'nova-manage fixed list' output but the
nova-manage CLI is mostly deprecated for things that should be done through the
Nova API service.
Data model impact
-----------------
None
REST API impact
---------------
The proposed change just updates the GET response data in the os-fixed-ips
API extension to include the 'reserved' boolean field if the request has a
minimum supported version.
* Specification for the method
* Shows information for a specified fixed IP address.
* Method type: GET
* Normal http response code(s): 200
* Expected error http response code(s):
* 400: If the address on the request is invalid.
* 404: If the address on the request does not match a FixedIP entry in the
database.
* ``/v2.1/​{tenant_id}​/os-fixed-ips/​{fixed_ip}​``
* Parameters which can be passed via the url: The fixed IP address
* JSON schema definition for the response data:
::
get_fixed_ip = {
'status_code': [200],
'response_body': {
'type': 'object',
'properties': {
'fixed_ip': {
'type': 'object',
'properties': {
'address': {
'type': 'string',
'format': 'ip-address'
},
'cidr': {'type': 'string'},
'host': {'type': 'string'},
'hostname': {'type': 'string'},
'reserved': {'type': 'boolean'}
},
'required': ['address', 'cidr', 'host',
'hostname', 'reserved']
}
},
'required': ['fixed_ip']
}
}
* Example use case:
Request:
GET --header "X-OpenStack-Nova-API-Version: 2.4" \
http://127.0.0.1:8774/v2.1/e0c1f4c0b9444fa086fa13881798144f/os-fixed-ips/\
192.168.1.1
Response:
::
{
"fixed_ip": {
"address": "192.168.1.1",
"cidr": "192.168.1.0/24",
"host": "host",
"hostname": "openstack",
"reserved": false
}
}
* There should not be any impacts to policy.json files for this change.
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
* The v2.1 python-novaclient fixed-ip-get command could be updated to show the
'reserved' status in it's output if 'fixed_ip' dict response has the
'reserved' key in it.
Performance Impact
------------------
None
Other deployer impact
---------------------
None; if a deployer is using the required minimum version of the API to get
the 'reserved' data they can begin using it, otherwise they won't see a change.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Matt Riedemann <mriedem@us.ibm.com>
Work Items
----------
* Add a new microversion and change nova/api/openstack/plugins/v3/fixed_ips.py
to use it to determine if the 'reserved' attribute on the FixedIP object
should be returned.
Dependencies
============
None
Testing
=======
* Unit tests and possibly API samples functional tests in the nova tree.
* There are currently not any compute API microversions tested in Tempest
beyond v2.1. We could add support for testing the new version in Tempest
but so far the API is already at least at v2.3 without changes to Tempest.
Documentation Impact
====================
The nova/api/openstack/rest_api_version_history.rst document will be updated.
References
==========
* Originally reported as a bug: https://bugs.launchpad.net/nova/+bug/1249526
* Old ML thread for the bug:
http://lists.openstack.org/pipermail/openstack-dev/2013-November/019506.html
* Proof of concept code change: https://review.openstack.org/#/c/168966/

194
specs/liberty/implemented/use-os-brick-library.rst Normal file
View File

@@ -0,0 +1,194 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
===================================================
Use the os-brick library for libvirt volume drivers
===================================================
https://blueprints.launchpad.net/nova/+spec/use-os-brick-library
Introduce the usage of the os-brick library into Nova. The library
will be used inside the libvirt volume drivers for volume discovery
and removal for all of the supported protocols. os-brick will also
be used to collect the initiator information, which is the connector
object, passed to Cinder during for volume attaches.
Problem description
===================
For several OpenStack releases now, there has been duplicate code
between Cinder and Nova with respect to volume discovery and removal.
Since the Havana release, Cinder has been incubating an embedded
library called 'brick'. The brick library's purpose was to collect
initiator information, discover volumes being attached to a host and to
remove volumes already attached. This is essentially the purpose of
Nova's libvirt volume drivers.
Cinder has officially removed the embedded brick library and has switched
to using the pypi os-brick library at the start of the Liberty release.
This spec lays out the removal of the duplicate code in Nova's
libvirt volume drivers that exists in the os-brick library.
Use Cases
----------
The main use case for using the os-brick library is when Nova needs to attach
or detach a volume. The primary work flow is this, Nova collects the initiator
information into a connector object, which gets passed to Cinder to export the
volume. Cinder passes back the target information and Nova uses that target
information to discover the volume showing up. Instead of using the embedded
code in Nova to collect the initiator information, and discover the target
volume showing up, Nova will use the os-brick library to do that work. The
os-brick library will also be used for volume removal when a Nova needs to
detach a volume.
Project Priority
-----------------
None
Proposed change
===============
First os-brick needs to be a new requirement for Nova. Then rework the
libvirt volume drivers to remove their internal code for volume discovery
and removal, and replace the code with calls to os-brick Connector objects.
Finally, replace the Nova virt/utils code that collects the initiator
information that gets passed to Cinder at volume attach time.
Alternatives
------------
Nova can continue to have duplicate code that does the same work as os-brick.
This is less than desirable as fixing bugs and adding new features needs to
be done in Nova as well as os-brick.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
os-brick will be a required pypi library for Nova.
Developer impact
----------------
One of the positive benefits of using this external library is that the Nova
team won't have to maintain the internals of os-brick's code for volume
discovery and removal. os-brick is owned by the Cinder team and any bug
fixes and new features added will benefit Nova.
Implementation
==============
Assignee(s)
-----------
Who is leading the writing of the code? Or is this a blueprint where you're
throwing it out there to see who picks it up?
If more than one person is working on the implementation, please designate the
primary author and contact.
Primary assignee:
walter-boring
Work Items
----------
* Convert Nova's code to use os-brick for collecting the initiator information
into the connector object.
* Convert each of the supported libvirt volume drivers to use os-brick
Connector objects.
* Rework the libvirt volume driver unit tests.
Dependencies
============
Import the os-brick library as a requirement for Nova.
Testing
=======
To test this out correctly, we have to change the unit tests in Nova that
look at the internals of volume discovery for iSCSI, FC, etc to test out
the os-brick library APIs. Ensure that each of the os-brick Connector
objects are loaded correctly for each of the support transports.
We will rely on the os-brick unit tests and releases for volume discovery
and removal to work properly, so there is no need to test all of the
internals for how iSCSI volume attaches work.
Documentation Impact
====================
We are simply moving the existing functionality of volume discovery
and removal from Nova's embedded libvirt volume drivers to use os-brick.
So, there is no doc impact.
References
==========
* Cinder's adoption of os-brick:
https://review.openstack.org/#/c/155552/
* https://github.com/openstack/os-brick
* https://pypi.python.org/pypi/os-brick
History
=======
Back in the Havana time frame we discussed the idea of creating a shared
library for doing volume discovery and removal as well as local volume
management for Nova and Cinder. The os-brick library is the first step
in that direction that solves the volume discover and removal shared code.
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Liberty
- Introduced

152
specs/liberty/implemented/vif-driver-ib-passthrough.rst Normal file
View File

@@ -0,0 +1,152 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
=============================================
Add support for InfiniBand SR-IOV VIF Driver
=============================================
https://blueprints.launchpad.net/nova/+spec/vif-driver-ib-passthrough
Adding support for InfiniBand SR-IOV vif type allows Virtual PCI device (VF)
to be directly mapped to the guest, allowing higher performance
and advanced features such as RDMA (remote direct memory access).
Problem description
===================
Till Juno Release (including) InfiniBand (IB) SR-IOV networking was possible
via out of the tree nova vif driver with neutron Mellanox Ml2 Mechanism Driver.
Since Juno Ethernet SR-IOV device vNIC is supported by nova, and may be
leveraged for IB SR-IOV device.
IB SR-IOV vif plugging logic should be added to LibvirtGenericVIFDriver.
Use Cases
----------
InfiniBand (IB) is popular, open standard, high performance and
extreme efficiency interconnect protocol. To enable Big Data, High Performance
Computing (HPC) and other similar use cases the guest requires direct access
to IB NIC device.
Project Priority
-----------------
None
Proposed change
===============
This change adds vif_driver support for vif_type VIF_TYPE_IB_HOSTDEV
as part of the GenericLibvirtVifDriver.
Currently, there is no standard API to set InfiniBand vGuid (equivalent for
Ethernet mac address), therefore special driver utility will be used to set it.
This utility is already used in case of mlnx_direct vif_type.
In the neutron VIF_TYPE_HOSTDEV is already supported by ML2 Mellanox
Mechanism Driver and enables networking configuration of SR-IOV Virtual
Functions on IB Fabric. The vif_type name should be renamed to
VIF_TYPE_IB_HOSTDEV to indicate specific InfiniBand vif_type.
Alternatives
------------
Currently there is no valid alternative.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
Execution of the dedicated 'ebrctl' utility requires the use of sudo.
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
To use this feature the deployer must use Infiniband enabled network adapters.
Infiniband Subnet Manager should be running to enable IB Fabric.
The 'ebrctl' utility should be installed on compute nodes.
Deployers need to be aware of the limitations:
* There will be no smooth upgrade path from the out of tree solution [3] to
the current solution.
* The out of tree solution [3] will be deprecated.
Developer impact
----------------
This blueprint will have no developer impact.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
Moshe Levi
Other contributors:
Irena Berezovsky
Work Items
----------
* Add vif_type ib_hostdev support to LibvirtGenericVIFDriver
Dependencies
============
proprietary 'ebrctl' library is required. This is installed as part
of the Driver OFED package.
Testing
=======
* Unit tests will be added to validate these modifications.
* Adding Third party testing for nova with SR-IOV Infiniband NIC
* Third party testing for neutron is already in place:
https://wiki.openstack.org/wiki/ThirdPartySystems/Mellanox_CI
Documentation Impact
====================
No documentation changes for Nova are anticipated.
VIF_TYPE_IB_HOSTEV will be automatically enabled by Neutron where appropriate.
References
==========
[1] Infiniband openstack solution documentation:
https://wiki.openstack.org/wiki/Mellanox-Neutron-Icehouse-Redhat#InfiniBand_Network
[2] Mellanox ML2 Mechanism Driver:
https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/mlnx/mech_mlnx.py
[3] Out of the tree VIF_TYPE_HOSTDEV vif driver:
https://github.com/mellanox-openstack/mellanox-vif

216
specs/liberty/implemented/vif-type-tap.rst Normal file
View File

@@ -0,0 +1,216 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
============================================================
New VIF type to allow routing VM data instead of bridging it
============================================================
https://blueprints.launchpad.net/nova/+spec/vif-type-tap
We propose to add a new VIF type, VIF_TYPE_TAP, whose meaning is that
the host side of a VNIC to a VM is - at least initially - a simple TAP
interface that is not plugged into a bridge or vSwitch. A Neutron
agent can continue the process of deciding how to handle data from
that interface, and how to deliver data to it - but this is beyond the
scope of the initial TAP interface setup that Nova needs to provide.
Problem description
===================
For Project Calico (http://www.projectcalico.org/), we'd like the host
side of the pipe into a VM to be a simple, unbridged TAP interface,
and currently there is no VIF_TYPE that I can use to get this.
VIF_TYPE_MIDONET, VIF_TYPE_IOVISOR and VIF_TYPE_IVS (without firewall
or hybrid plug) all create an unbridged TAP interface, but then do
additional things to it (within the Nova code) to connect that TAP
interface into the host's networking system. Other VIF_TYPEs involve
bridges or vSwitches, or 'direct' attachments to physical host
interfaces.
Use Cases
---------
One application is that VIF_TYPE_TAP makes it possible for data
to/from VMs, and also between VMs on the same host, to be routed by
their immediate compute host instead of being bridged. This is of
interest in deployments where VMs only require services at layer 3
(IP) and above, and it is still possible to implement, through
iptables and route distribution filters, all of the detailed
connectivity and security policies that are implied by any given set
of OpenStack's networking, security group and router configurations.
For more on how that can work please see Project Calico
(http://www.projectcalico.org/).
The applicability of VIF_TYPE_TAP should however be wider than just
that one project. It enables a class of experimental future
networking implementations to be explored in Neutron (with plugin,
mechanism driver and agent code) without needing to change or patch
any Nova code.
Project Priority
----------------
Not applicable. (As advised by John Garbutt.)
Proposed change
===============
Add VIF_TYPE_TAP in nova/network/model.py.
Add get_config_tap, plug_tap and unplug_tap methods in
nova/virt/libvirt/vif.py, with implementations that simply configure,
create and delete a TAP device.
The libvirt config for VIF_TYPE_TAP would be an <interface
type="ethernet"> element with a null script, just as for the existing
VIF_TYPE_MIDONET and VIF_TYPE_IOVISOR cases, prepared by calling
self.get_base_config and
designer.set_vif_host_backend_ethernet_config.
When a VM is to be launched using VIF_TYPE_TAP:
- Nova creates the TAP interface, in plug_tap(), by calling
linux_net.create_tap_dev
- libvirt launches the VM, with the config described just above, to
match the created TAP interface.
Alternatives
------------
None. All existing VIF_TYPEs whose plugging is implemented in
nova/virt/libvirt/vif.py are unsuitable, as described in Problem
Description above. Pre-Juno it was possible to configure use of an
out-of-tree VIF driver (with the virt_driver setting), but this was
deprecated and has now been removed.
Data model impact
-----------------
None.
REST API impact
---------------
None.
Security impact
---------------
A compute host can guard against IP address spoofing (by a VM) on a
VIF_TYPE_TAP interface by installing iptables rules that require each
packet from a VM to have the expected source IP address.
If a VIF_TYPE_TAP interface is not plugged into a bridge, MAC address
spoofing by a VM has no impact. If a VIF_TYPE_TAP interface _is_
plugged into a bridge, that bridge can implement similar protections
against MAC spoofing as for existing bridged VIF_TYPEs.
Notifications impact
--------------------
None.
Other end user impact
---------------------
None.
Performance Impact
------------------
Unless the VIF_TYPE_TAP vif type is explicitly requested (e.g. by a
Neutron/ML2 mechanism driver class), there is no possible performance
impact on a standard OpenStack system.
Other deployer impact
---------------------
The Nova extension proposed here will have no effect on existing or
newly deployed OpenStack systems, unless the VIF_TYPE_TAP vif type
is explicitly requested somewhere (e.g. by a Neutron/ML2 mechanism
driver class).
Developer impact
----------------
None.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
neil-jerram
Other contributors:
lukasaoz
cliljenstolpe
Work Items
----------
The changes required for this spec have already been implemented by
us, based on the Icehouse and Juno release code, and extensively
tested.
Code that was submitted during the Kilo cycle can been seen at
https://review.openstack.org/#/c/146914/.
Remaining work items are as follows:
* Submit changes formally for review.
* Participate in resulting discussions, mark up and re-review
processes.
* Repeat until done!
Dependencies
============
None.
Testing
=======
Within the OpenStack ecosystem, this change will be tested at
unit-test level, by adding a test case to
nova/tests/virt/libvirt/test_vif.py, that creates and verifies a
virtual interface with type VIF_TYPE_TAP.
(It will also be extensively tested at the system level by continuing
related development and testing at Project Calico
(http://www.projectcalico.org/), which uses VIF_TYPE_TAP, and such
work will generally be conducted and reported in public.
We understand, though, that this is not formally verifiable testing
within the OpenStack ecosystem; so it is mentioned here for
information only.)
Documentation Impact
====================
No documentation changes for Nova are anticipated. VIF_TYPE_TAP will
be automatically enabled by a related Neutron/ML2 driver, where
appropriate.
References
==========
http://www.projectcalico.org/
https://github.com/Metaswitch/calico-nova
https://github.com/Metaswitch/calico-neutron

223
specs/liberty/implemented/vmware-limits.rst Normal file
View File

@@ -0,0 +1,223 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
VMware Limits, Shares and Reservations
==========================================
https://blueprints.launchpad.net/nova/+spec/vmware-limits
VMware Virtual Center provides options to specify limits, reservations and
shares for CPU, memory, disks and network adapters.
In the Juno cycle support for CPU limits, reservation and shares was added.
This blueprint proposes a way of supporting memory, disk and network
limits, reservations and shares.
For limits the utlization will not exceed the limit. Reservations will be
guaranteed for the instance. Shares are used to determine relative allocation
between resource consumers. In general, a consumer with more shares gets
proportionally more of the resource, subject to certain other constraints.
Problem description
===================
The VMware driver is only able to support CPU limits. Providing admins the
ability to provide limits, reservation and shares for memory, disks and
network adapters will be a very useful tool for providing QoS to tenants.
Use Cases
----------
* This will enable a cloud provider to provide SLA's to customers
* It will allow tenants to be guaranteed performance
Project Priority
-----------------
None
Proposed change
===============
Due to the different models for different drivers and the API's in which
the backends expose we are unable to leverage the same existings flavor
extra specs.
For example for devices libvirt makes use of: 'hw_rng:rate_bytes',
'hw_rng:rate_period'.
In addition to this there are the following disk I/O options are:
'disk_read_bytes_sec', 'disk_read_iops_sec', 'disk_write_bytes_sec',
'disk_write_iops_sec', 'disk_total_bytes_sec', and
'disk_total_iops_sec'.
For bandwidth limitations there is the 'rxtx_factor'. This will not enable
us to provide the limits, reservations and shares for vifs. This is used in
some bases to pass the information through to Neutron so that the backend
network can do the limitations. The following extra_specs can be configured
for bandwidth I/O for vifs:
'vif_inbound_average', 'vif_inbound_burst', 'vif_inbound_peak',
'vif_outbound_average', 'vif_outbound_burst' and 'vif_outbound_peak'.
None of the above of possible for the VMware driver due to VC API's. The
following additions below are proposed:
Limits, reservations and shares will be exposed for the following:
* memory
* disks
* network adapters
The flavor extra specs for quotas has been extended to support:
* quota:memory_limit - The memory utilization of a virtual machine will not
exceed this limit, even if there are available resources. This is
typically used to ensure a consistent performance of virtual machines
independent of available resources. Units are MB.
* quota:memory_reservation - guaranteed minimum reservation (MB)
* quota:memory_shares_level - the allocation level. This can be 'custom',
'high' 'normal' or 'low'.
* quota:memory_shares_share - in the event that 'custom' is used, this is
the number of shares.
* quota:disk_io_limit - The I/O utilization of a virtual machine will not
exceed this limit. The unit is number of I/O per second.
* quota:disk_io_reservation - Reservation control is used to provide guaranteed
allocation in terms of IOPS
* quota:disk_io_shares_level - the allocation level. This can be 'custom',
'high' 'normal' or 'low'.
* quota:disk_io_shares_share - in the event that 'custom' is used, this is
the number of shares.
* quota:vif_limit - The bandwidth limit for the virtual network adapter.
The utilization of the virtual network adapter will not exceed this limit,
even if there are available resources. Units in Mbits/sec.
* quota:vif_reservation - Amount of network bandwidth that is guaranteed to
the virtual network adapter. If utilization is less than reservation, the
resource can be used by other virtual network adapters. Reservation is not
allowed to exceed the value of limit if limit is set. Units in Mbits/sec.
* quota:vif_shares_level - the allocation level. This can be 'custom',
'high' 'normal' or 'low'.
* quota:vif_shares_share - in the event that 'custom' is used, this is the
number of shares.
Alternatives
------------
The alternative is to create an abstract user concept that could help hide
the details and of the difference from end users, and isolate the differences
to just the admin users.
This is really out of the scope of what is proposed and will take a huge
cross driver effort. This will not only be relevant for flavors but maybe for
images too.
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
Preventing instances from exhausting storage resources can have a significant
performance impact.
Other deployer impact
---------------------
None
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
garyk
Work Items
----------
* common objects for limits, reservation and shares
* memory support
* disk support
* vif support
Dependencies
============
None
Testing
=======
This will be tested by the VMware CI. We will add tests to validate this.
Documentation Impact
====================
This should be documented in the VMware section.
References
==========
The vCenter API's can be see the following links:
* Disk IO: http://goo.gl/uepivS
* Memory: http://goo.gl/6sHwIA
* Network Adapters: http://goo.gl/c2amhq
History
=======
None

210
specs/liberty/implemented/vmware-webmks-console.rst Normal file
View File

@@ -0,0 +1,210 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==========================================
Native HTML5 consoles for VMware
==========================================
https://blueprints.launchpad.net/nova/+spec/vmware-webmks-console
This is a proposal to add support for the native HTML5 console available in
vSphere 5.5 and later (referred as WebMKS console). This will solve some of the
current problems that we have with VNC consoles with respect to deployment,
scalability and security. The existing support for VNC consoles will remain but
users will be encouraged to use WebMKS when possible.
Problem description
===================
There are number of problems with the current approach which provides VNC
consoles to instances running on VMware hypervisors:
* Deployment - cloud operators need to open a huge range of ports (5900-6105)
in the firewall configuration on each ESXi host. Additionally, they need to
create and install a special VIB package for the firewall modifications to
persist after reboot. The whole thing creates a huge security risk because
we rely only on the fact that all ESXi hosts will run on a private network
that will be inaccessible to the cloud users.
* Scalability - the Nova driver needs to allocate a unique VNC port for each
instance and this is done by querying all allocated ports on the vCenter
while holding a lock in the driver. Needless to say, this doesn't scale well.
* Security - we can prevent race conditions for port allocations occurring on
the same nova-compute but there is no way to do this between several
nova-computes. This creates a possibility for allocating the same VNC port to
different instances and if these instances end up on the same ESXi host
somehow, one user can hijack the console of another user.
All of the above problems don't stand with WebMKS consoles.
Use Cases
----------
Deployers will be able to deploy OpenStack on top of vSphere much more easily
because they won't need to modify firewall configurations on the ESXi hosts.
Spawning many instances at the same time will be faster for End Users because
no ports will be allocated in the driver and thus no locks will be held.
Project Priority
-----------------
None
Proposed change
===============
The proposal is to add a new API that will return WebMKS connect information
similar to how we return connect information for other types of consoles like
VNC and RDP. The WebMKS connect information for a given instance is the
following:
* host - the hostname of the corresponding vCenter
* port - 7331 or 7343 depending on the type of the web socket (ws or wss)
* vmId - the Managed Object ID of the virtual machine
* vmName - the name of the virtual machine
* vcFQDN - the vCenter Server advanced setting "VirtualCenter.FQDN"
* sessionTicket - session ticket obtained with the AcquireCloneTicket API
* thumbprint - the vCenter Server SHA1 SSL thumbprint
Alternatives
------------
There is already an existing alternative for instances running on VMware which
is VNC consoles. We will continue to support this.
Data model impact
-----------------
None
REST API impact
---------------
We will leverage the new v2.1 API for getting console access:
Sample request::
POST /servers/<uuid>/console
{
"protocol": "mks"
"type": "webmks"
}
Sample response::
200 OK
{
"protocol": "mks",
"url": "http://example.com:6090/?token=c88be357-0992-40e8-8dd7-afc918ad34d5"
}
The updated JSON schema is as follows::
{
'type': 'object',
'properties': {
'protocol': {
'enum': ['vnc', 'spice', 'rdp', 'serial', 'mks'],
},
'type': {
'enum': ['novnc', 'xvpvnc', 'rdp-html5',
'spice-html5', 'serial', 'webmks'],
},
},
'required': ['protocol', 'type'],
'additionalProperties': False,
}
The URL in the response points to a WebMKS proxy which authorizes the given
token by calling nova-consoleauth and then starts proxying between the client
and the corresponding vCenter server where the instance runs.
Security impact
---------------
We will use the same security model as for the other types of consoles which is
leveraging nova-consoleauth to authorize tokens generated by the compute node.
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
If the VMware driver is configured to provide only WebMKS consoles it will
perform better because it won't need to allocate any VNC ports (allocating a
VNC port requires a vCenter query while holding a lock).
Other deployer impact
---------------------
WebMKS consoles will require deploying WebMKS proxy (similar to
nova-novncproxy) that will connect compute service users to the corresponsing
vCenter which provides console access to the given virtual machine.
There will be a new configuration option that will specify the base URL for the
WebMKS proxy (e.g. webmks_proxy_base_url).
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
rgerganov
Work Items
----------
The implementation can be split in two parts:
* add the new API to the compute manager and the virt driver interface
* implement the new API in the VMware driver
Dependencies
============
blueprint: consolidate-console-api
Testing
=======
Since this is a VMware specific feature, VMware NSX CI will be updated to run
tests that exercise this functionality.
Documentation Impact
====================
The Nova documentation for VMware needs to be updated to say that two types
of consoles are supported (VNC and WebMKS) and there is no need to modify
firewall configurations if deployers want to provide only WebMKS.
References
==========
None