46082 Commits

Author SHA1 Message Date
Jenkins
e18e23e100 Merge "Perform old-style local delete for shelved offloaded instances" 2017-04-07 01:56:30 +00:00
XieYingYun
35ff68fb12 Optimize the link address
Use https instead of http to ensure the safety without containing our
account/password information

Change-Id: Iff85d63353b858e0df28bf2e2c9557e6737b7aba
2017-04-07 08:56:37 +08:00
Anusha Unnam
f95a3acb89 Add description to policies in quota_sets.py
The 'discoverable' policy is not documented as it will be removed in a
future change.

blueprint policy-docs

Change-Id: If69440c4a792c7810e148312e8f3b8d788b496cd
2017-04-06 22:51:56 +00:00
Matt Riedemann
d52bcc616f Fix joins in instance_get_all_by_host
Some callers of instance_get_all_by_host are passing
in columns_to_join=[], like the _sync_scheduler_instance_info
periodic task in the compute manager, to avoid unnecessary
joins with other tables.

The problem was columns_to_join wasn't being passed through
to _instance_get_all_query which builds the actual query
method, and defaults to join on info_cache and security_groups.

This fixes the problem by passing through columns_to_join and
provides tests to show it working both with and without the joins.

Change-Id: I69f2ddca8fb0935e03b0f426891d01360940a85a
Closes-Bug: #1680616
2017-04-06 17:59:34 -04:00
Matt Riedemann
1521049dae Fix test_instance_get_all_by_host
This test was not testing instance_get_all_by_host,
it was testing _instance_get_all_uuids_by_host,
which is completely redundant with the same test
that comes right after it.

¯\_(ツ)_/¯

Change-Id: Ia0331e3f622f17067bf5900f92b3f8ff2aff966e
Closes-Bug: #1680614
2017-04-06 17:47:11 -04:00
Jenkins
6103ec7c11 Merge "Fix rest_api_version_history (2.40)" 2017-04-06 20:34:22 +00:00
Jenkins
eb69c1745c Merge "Make scheduler target cells to get compute node instance info" 2017-04-06 20:33:36 +00:00
Jenkins
beb47b8b1a Merge "Remove config opts for extension black/white list" 2017-04-06 20:22:57 +00:00
Jenkins
fd014e6f2f Merge "api-ref: make it clear that os-cells is for cells v1" 2017-04-06 19:41:47 +00:00
Jenkins
5a6d4e089d Merge "Deprecate the Cinder API v2 support" 2017-04-06 19:41:13 +00:00
He Jie Xu
56d7c05d3f Remove the stevedore extension point for server create
This patch remove the stevedore extension point for server
create.

This patch move all the extension points into ServersController
class attribute 'server_create_func_list'. This is for
backward-compatible with the stevedore extension interface.

The final goal is that merging all of those extended code into
the ServerController. So this is a middle step for refactor.

Partial-implement-blueprint api-no-more-extensions-pike

Change-Id: I9a8f56037b35e89543626922221b910ab8b1917e
2017-04-06 18:52:11 +00:00
He Jie Xu
4ca9e3667a Remove the json-schema extension point of server create
To get rid of all of stevedore extension point. The
jsons-chema extension point of server create should be
removed. This patch moves all of extended json-schema into
ServerControoler's class attributes 'schema_func_list'.

But the final goal is that merging all of those extended
json-schema into server's main json-schema. So this is a
middle step for all of those refactor for removing stevedore.

Partial-implement-blueprint api-no-more-extensions-pike

Change-Id: Id8ba6a0383cf236259224ace2df3c3d89f82f27d
2017-04-06 18:51:56 +00:00
He Jie Xu
ccf2a3e430 Remove the extension check for os-networks in servers API
The Nova API doesn't support extension anymore. There is no way
to disable any API in the Nova now. So the check for 'os-networks'
is useless now, also as the 'os-networks' is always enabled now,
the check for neutron is useless also.

Partial-implement-blueprint api-no-more-extensions-pike

Change-Id: I393ca6e91d3bd5b4799386db641366e9e4c3a171
2017-04-06 18:51:32 +00:00
Dan Smith
0af57a9d4b Make server_groups determine deleted-ness from InstanceMappingList
The server_groups API currently queries whole instance objects to
determine which of the group members are not deleted. This hits the
cell database and also grabs a ton of data. This patch makes it use
the newly-minted InstanceMappingList to draw the same conclusion,
which is both more effcient and does not require teaching it about
multiple cells.

Related to blueprint cells-aware-api

Change-Id: If571f9e7c9d0ef6265a249ff808d30a24ab462af
2017-04-06 10:57:06 -07:00
Dan Smith
38a0231065 Add get_by_instance_uuids() to InstanceMappingList
This provides a bulk lookup for things that need it, such as the
server_groups api module.

Related to blueprint cells-aware-api

Change-Id: I8bd5c9611b7ba076116aac50f4c9bfea6fb41034
2017-04-06 10:57:06 -07:00
Dan Smith
bff9aa2801 Remove Mitaka-era service version check
We added this to make sure that we were compatible with Mitaka computes
when get-me-a-network was added in Newton. That compatibility is no
longer a concern in Pike. Remove the check, which would otherwise need
to be multi-cell-ified.

Related to blueprint cells-aware-api

Change-Id: I3168616b7878915c0c33286cd5629e8d01366de3
2017-04-06 10:57:06 -07:00
Dan Smith
74c5bfea6f Teach HostAPI about cells
This makes the HostAPI module know enough about cells to support
the os-hypervisors, os-services, and several other API modules that
deal with hosts and services.

Note that this introduces a conflict where duplicate-id resources could
be returned from the API because they are in different DBs with different
key namespaces. For now, we return an error in the delete case where an
ambiguous result could cause us to delete the wrong service. This is
a fundamental problem across several of our APIs and is being worked as
a separate improvement.

Related to blueprint cells-aware-api

Change-Id: If1e03c9343b8cc9c34bd51c2b4d25acdb21131ff
2017-04-06 10:57:06 -07:00
Dan Smith
791cf06434 Make scheduler target cells to get compute node instance info
This extends the previous patch to target cells during compute node
discovery to also target cells when fetching information about
instances prior to scheduling.

Change-Id: Idb9b969620c24f2b83f3205f95a9aab9daa28513
2017-04-06 10:57:06 -07:00
Jenkins
a17659afb7 Merge "Regression test for local delete with an attached volume" 2017-04-06 17:43:55 +00:00
Jenkins
fb87f64a25 Merge "Set size/status during image create with FakeImageService" 2017-04-06 17:39:57 +00:00
Ildiko Vancsa
89f984d028 Deprecate the Cinder API v2 support
Change-Id: Ice67d29f31aa5bd5c84fad4e828435d8552a9629
2017-04-06 09:55:39 -07:00
Scott Moser
f559be35a0 Limit exposure of network device types to the guest.
Previously, the 'type' of the hypervisor network device, was exposed to
the guest directly. That does not make sense, as
a.) this leaks needless information into the guest
b.) the guest cannot be reasonably expected to make decisions
    based on a type of link that is present underneath the
    virtual device that is presented to the guest.
c.) guests then are forced to either continuously track these types
    or to assume that unknown type is "phy".

This limits the exposure of types to a specific list. Any other
type will be shown to the guest as 'phy'.

Change-Id: Iea458fba29596cd2773d8d3565451af60b02bcca
Closes-Bug: #1642679
2017-04-06 10:27:33 -04:00
Matt Riedemann
4777f9f265 Remove a fallacy in scheduler.driver config option help text
Change fe3d6dba3d1db8f97dab4baffcc11dda56368096 removed the
ability to specify a full classpath to load up your own
scheduler driver. After that point stevedore extensions had
to be used from setup.cfg.

Change 6832a417c93046750ec7eff51eb59be802fd70dc encoded the
supported in-tree scheduler driver options by using the
"choices" kwarg with the scheduler.driver option. This means
if you specify something not in that list, you get a ValueError.

So the part in the help text about registering your own
stevedore extension is a lie since it would still fail because
of "choices".

Change-Id: I60ce26f7370b71b2dc266d582d9b17d35e8cd9c1
2017-04-06 09:59:30 -04:00
Chris Dent
6dd047a330 [placement] Allow PUT and POST without bodies
We plan to allow PUT requests to create/update both custom traits
and custom resource classes, without bodies. Prior to this change,
the code would not all a PUT, POST or PATCH to not have a body. This was
added in I6e7dffb5dc5f0cdc78a57e8df3ae9952c55163ae which was fixing an
issue with how webob handles exceptions.

This change does two things:

* It address the problem from bug #1623517, fixed in the change id'd
  above, in a more narrow fashion, making sure the data source that
  causes the KeyError is non-empty right before it is used. This allows
  simplifying the following change.
* If a request has a content-length (indicating the presence of a body),
  verify that there is also a content-type. If not, raise a 400.

basic-http.yaml has been change to modify one gabbi test to check a
response body is correct and to add another test to confirm that the
code that is doing the content-length check is passed through.

Change-Id: Ifb7446fd02ba3e54bbe2676dfd38e5dfecd15f98
Closes-Bug: #1674392
Related-Bug: #1623517
2017-04-06 10:45:06 +00:00
He Jie Xu
de0a8440c8 Remove config opts for extension black/white list
The config opts of extension black/white list 'extensions_blacklist'
and 'extensions_whitelist' are deprecated for a long time. It's time
to remove it.

Partial-implement-blueprint api-no-more-extensions-pike

Change-Id: Ide8c302619825c7b41ee52baa7128c5bb2341aa4
2017-04-06 08:47:06 +00:00
Jenkins
8d9bf947a4 Merge "Remove the usage of extension black/white list opt in scheduler hints" 2017-04-06 08:17:19 +00:00
He Jie Xu
a2d1207032 Remove the usage of extension black/white list opt in scheduler hints
The Nova API doesn't support the extension anymore. The API
unittest shouldn't disable any part of Nova API. This patch
removes the tests which disable part of API in scheduler hints.

Partial-implement-blueprint api-no-more-extensions-pike

Change-Id: I90544eb3fa51a6cf6347ea4bcecee92c837f7888
2017-04-06 07:12:58 +00:00
Matt Riedemann
ea8321adc7 Cleanup wording on compute service version checks in API
These were originally added in Newton for get-me-a-network
but are now used for other purposes, like cells v2 and volume
reservations in Pike:

63805735c25a54ad1b9b97e05080c1a6153d8e22

This change just makes the wording more generic and removes
a TODO about removing the cache in Ocata, which is no longer
applicable.

Change-Id: I9cabe075c70424b167f5e340d05d28fa18f7d276
2017-04-05 19:20:43 -04:00
Jenkins
53296a0702 Merge "docs: update description for AggregateInstanceExtraSpecsFilter" 2017-04-05 22:08:00 +00:00
Jenkins
7660d0cc38 Merge "Add confirm resized server functional negative tests" 2017-04-05 21:55:37 +00:00
Jenkins
187a6f12a4 Merge "Commit usage decrement after destroying instance" 2017-04-05 21:55:04 +00:00
Jenkins
6cda9bc912 Merge "Add regression test for quota decrement bug 1678326" 2017-04-05 21:54:30 +00:00
Jenkins
dd1d94d85a Merge "Short-circuit local delete path for cells v2 and InstanceNotFound" 2017-04-05 21:54:00 +00:00
Matt Riedemann
f845e17968 Fix test_no_migrations_have_downgrade
This wasn't walking the correct directory and it wasn't
including the API database migrations.

Change-Id: Idff0e8089faf4025a080a5842ff65eaad7b6fdb8
Closes-Bug: #1680240
2017-04-05 17:00:05 -04:00
Jenkins
501a0fe65a Merge "Remove the usage of extension black/white list opt in user data" 2017-04-05 20:29:25 +00:00
Jenkins
e117260699 Merge "conf: Final cleanups in conf/network" 2017-04-05 20:28:54 +00:00
Jenkins
c5c600de4a Merge "conf: Deprecate 'allow_same_net_traffic'" 2017-04-05 20:28:22 +00:00
Jenkins
683295d740 Merge "libvirt: Ignore 'allow_same_net_traffic' for port filters" 2017-04-05 20:27:49 +00:00
Matt Riedemann
9245bbf79d Perform old-style local delete for shelved offloaded instances
This fixes a regression from some local delete code added for cells v2
where it assumed that if an instance did not have a host, it wasn't
scheduled to a cell yet. That assumption misses the fact that the
instance won't have a host if it was shelved offloaded. And to be
shelved offloaded, the instance had to have first been built on a host
in a cell.

So we simply duplicate the same check as later in the _delete() method
for instance.host or shelved-offloaded to decide what the case is.

Obviously this is all a giant mess of duplicate delete path code that
needs to be unwound, and that's the plan, but first we're fixing
regressions and then we can start rolling this duplication all back
so we can get back to the single local delete flow that we know and love.

Change-Id: Ie2063f621618c1d90aeb59f0f1d7da351862ea9f
Closes-Bug: #1678326
2017-04-05 16:27:41 -04:00
Jenkins
a1f152de69 Merge "placement: Enable attach traits to ResourceProvider" 2017-04-05 20:26:33 +00:00
Jenkins
acfbec7db9 Merge "API: accept None as content-length in HTTP requests" 2017-04-05 20:23:05 +00:00
Matt Riedemann
3ae12fdc6f Regression test for local delete with an attached volume
Once an instance is in a cell and we do a local delete from
the API, we aren't actually detaching volumes and destroying
BDMs.

Related-Bug: #1675570

Change-Id: Ie3e2dfd4b0f1bb3dff4080f460bf8bb40d69f4f4
2017-04-05 15:36:47 -04:00
Matt Riedemann
371a2512a7 Set size/status during image create with FakeImageService
This is needed for an upcoming change which introduces
a functional test which shelves a server. Shelving a server
creates a snapshot image and in the real world, glance sets
the size and status attributes on the image when it's created
in glance. Our FakeImageService wasn't doing that, so tests
that are running at the same time with the same fake noauth
credentials are listing images and picking up the shelve
snapshot image which doesn't have size or status set and
that produces a KeyError in the API code.

Change-Id: I70d1505c915e80156aaa4d27a7d47e437b40b6f1
2017-04-05 15:36:38 -04:00
Matt Riedemann
42d4ea0b62 Commit usage decrement after destroying instance
This fixes a regression in Ocata where we were always
decrementing quota usage during instance delete even
if we failed to delete the instance. Now the reservation
is properly committed after the instance is destroyed.

The related functional test is updated to show this working
correctly now.

Change-Id: I5200e72c195e12b5a069cbae3f209492ed569fb4
Closes-Bug: #1678326
2017-04-05 15:19:15 -04:00
Matt Riedemann
032937ce51 Add regression test for quota decrement bug 1678326
This was spotted from someone validating the fix for
bug 1670627. They reported that even though they failed
to delete an instance in ERROR state that was in cell0,
the quota usage was decremented.

This is because we committed the quota reservation
to decrement the usage before actually attempting to destroy
the instance, rather than upon successful deletion.

The rollback after InstanceNotFound is a noop because of
how the Quotas.rollback method noops if the reservations
were already committed. That is in itself arguably a bug,
but not fixed here, especially since the counting quotas
work in Pike will remove all of the reservations commit and
rollback code.

Change-Id: I12d1fa1a10f9014863123ac9cc3c63ddfb48378e
Partial-Bug: #1678326
2017-04-05 15:15:20 -04:00
Matt Riedemann
5a9cc2fb7a Short-circuit local delete path for cells v2 and InstanceNotFound
When we're going down the local delete path for cells v2 in the API
and instance.destroy() fails with an InstanceNotFound error, we are
racing with a concurrent delete request and know that the instance
is alread deleted, so we can just return rather than fall through to
the rest of the code in the _delete() method, like for BDMs and
console tokens.

Change-Id: I58690a25044d2804573451983323dde05be9e5d6
Closes-Bug: #1680211
2017-04-05 15:12:41 -04:00
Matt Riedemann
74db119220 api-ref: make it clear that os-cells is for cells v1
With cells v2, we need to be clear about what is cells v1 and
what is cells v2 in our documentation. This makes a note in the
os-cells api-ref about that being for cells v1 and also adds a
line saying the entire API is admin-only by default.

Change-Id: I2a06e2c4921ce7172aab33818054dd37b5ef9a65
2017-04-05 17:56:57 +00:00
Anusha Unnam
dd2c997be2 Add description to policies in security_group_default_rules.py
The 'discoverable' policy is not documented as it will be removed in a
future change.

blueprint policy-docs

Change-Id: I751cbde145ed486cd47e2ea6ff66d41914f0552b
2017-04-05 16:40:56 +00:00
He Jie Xu
0f0493d1ec Remove the usage of extension black/white list opt in user data
The Nova API doesn't support the extension anymore. The API
unittest shouldn't disable any part of Nova API. This patch
removes the tests which disable part of API in user data
unittests.

Partial-implement-blueprint api-no-more-extensions-pike

Change-Id: I7f9721d67aa978699c3d34dc36ade1288adf0667
2017-04-05 09:58:01 +00:00
ghanshyam
af1734b0dd Add empty flavor object info in server api-ref
In case flavor information is not present in sysmeta then
GET/PUT server response can have flavor as empty object.

ref- http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/views/servers.py#n252

Having that mentioned in api-ref will be nice to understand it clearly.
Closes-Bug: #1677999

Change-Id: I350dc33df0cb03eb86df1f142e58ae34b02a1334
2017-04-05 02:52:41 +00:00