11110 Commits

Author SHA1 Message Date
Zuul
2a21fbdae9 Merge "Move Inventory and InventoryList to own file" 2019-03-15 07:48:43 +00:00
Zuul
bea1afacf2 Merge "Reorder classes and methods in allocation_candidate" 2019-03-14 19:28:27 +00:00
Zuul
b872fc6b03 Merge "Move allocation candidate classes and methods" 2019-03-14 19:28:25 +00:00
Chris Dent
fbf9510907 Extract user and endpoint creation install doc
The placement user and service catalog endpoint creation information
in the three different distro docs is very nearly exactly the same
so it has been extracted to a file which is included by all three.
This file can then be reused elsewhere as well, such as planned
"from-pypi" install docs (for which there exists a stub).

The only difference in the three versions was the URL used for
the endpoint(s) which already required some cooking to work in
any environment. A note has been added to point out the installar's
responsibilities in this case.

Subsubheadings are introduced in the surrounding document so that
the numbered list is considered two separate lists. Otherwise the
output ends up looking like a single list with two initial number
one items.

Change-Id: Ib928162702a54f000161367c90172c730cb632c7
2019-03-14 19:18:56 +00:00
Zuul
31b9a67b2f Merge "Trivial: Update doc for _set_traits()" 2019-03-14 19:08:23 +00:00
Zuul
439bcca6e2 Merge "Remove the TraitList class" 2019-03-14 19:08:22 +00:00
Zuul
b1737f7100 Merge "Update get trait and traits calls that were writers to reader" 2019-03-14 14:04:57 +00:00
Zuul
5707b66997 Merge "Move Trait and TraitList to own module" 2019-03-14 13:45:21 +00:00
Chris Dent
2e2cb183a4 Remove unused table constants from allocation_candiate.py
The constants are not used in allocaion_candidate.py and
are not referenced from elsewhere. The reason they ended
up in there was because during the refactoring at [1] there
was indecision about whether some methods should be in
resource_provider.py or allocation_candidate.py. They
ended up in resource_provider.py but the list of tables
was not updated.

[1] I6539d7db87bc24b03842e3d28a4296002b3549d5

Change-Id: I3d501543f7e79a8c3c030369af1398ca68f2b811
2019-03-14 12:29:34 +00:00
Chris Dent
0a2a611d9f Move set_traits tests back to test_resource_provider
The set_traits related tests were moved in [1] and in the
review process it was determined that they should be
associated with resource provider-related tests.

Private methods used to assert traits in test results are moved
to test_base.

[1] Ibe11ebd7ccc97a46586edd6c897b173ed67ca134

Change-Id: I8e1e62518c1c2308a64bb19997fedc219bf8342a
2019-03-14 12:21:13 +00:00
Zuul
7f0efd3dbb Merge "Clean up the intro to the REST API section" 2019-03-14 04:06:54 +00:00
Zuul
04da5998f2 Merge "Clean up around links to database migrations scripts" 2019-03-14 04:06:45 +00:00
Zuul
e6064d977e Merge "Indicate existince of sync_on_startup option" 2019-03-14 04:06:40 +00:00
Zuul
e859cb46cb Merge "Link to install docs from deployment overview" 2019-03-14 04:06:36 +00:00
Chris Dent
a53aef47b2 Remove ResourceProviderList class
The get_all_by_filters class method is now module level. Callers
and tests are updated accordingly.

As there is no longer any package using common.py, it is removed.

Looking at this, it now seems obvious that 'get_all_by_filters' is
oxymoronic but changing it now didn't seem appropriate. We may
wish to do that in future work.

Change-Id: Ib9365200ebec58002c2456435f858466c6afea7f
2019-03-13 21:24:34 +00:00
Chris Dent
f23137beee Remove InventoryList class
The two methods from the class, 'find' and
'get_all_by_resource_provider' are made module-level and tests
and callers updated as required. The signature on find changes to
take a list of Inventory objects as its first arg.

Change-Id: I50e69aececf0d1afd3218bb2d63407b273757b29
2019-03-13 21:24:34 +00:00
Chris Dent
ca3ade7616 Move Inventory and InventoryList to own file
The Inventory and InventoryList classes are moved to an
inventory.py file. Tests and other callers are udpated
to reflect the new location.

A new unit.objects.test_inventory is created. A
functional.db.test_inventory is _not_ created because though
there are inventory related tests within test_resource_provider
they are within a ResourceProvider-related class and teasing
out the ones that are for the sake of just Inventory versus
for both Inventory and ResourceProvider is not entirely
clear.

Change-Id: Ia1ad18585194236647ec2c83480254531a619e91
2019-03-13 21:24:34 +00:00
Chris Dent
275fb28c20 Reorder classes and methods in allocation_candidate
Classes first, in alpha order, followed by private methods
in alpha order. There are no module level public methods.

Methods within classes have not been reordered.

Change-Id: I229d429ff4cadfabfbc53d89b57a89949901be89
2019-03-13 21:24:34 +00:00
Chris Dent
0f0d80b6f5 Move allocation candidate classes and methods
The AllocationCandidates class and the classes it uses to assemble
a suite of classes are moved to objects/allocation_candidate.
Also included are the several methods used solely by those classes.

Methods which are more directly about resource providers, even
if only used by the allocation_candidate modules, are kept in
resource_provider.py. As a result several methods in
resource_provider.py are made public.

Tests are updated accordingly.

As this is already a rather chaotic change, no reordering was
done of the classes and methods in allocation_candidate. That
will follow.

Change-Id: I6539d7db87bc24b03842e3d28a4296002b3549d5
2019-03-13 21:24:34 +00:00
Chris Dent
580cc15d9e Remove the TraitList class
The get_all and get_all_by_resource_provider methods are moved
to module level, callers and tests updated accordingly.

Note that this leaves some commonality between the ResourceClass
and Trait single-items classes, which we may wish to address in
future work.

Change-Id: I593bfa80007eb41803f09597377ae5c2a3540990
2019-03-13 21:24:34 +00:00
Chris Dent
07088532f8 Update get trait and traits calls that were writers to reader
There were initially writer because there had been an
ensure_trait_sync (no ensure_sync) in them some time ago.
When trait sync was moved to the start of the placement
web service process [1], the writers were not corrected to
a reader.

A related comment on _trait_sync is updated to reflect the
current syncing reality.

[1] Ic87518948ed5bf4ab79f9819cd94714e350ce265

Change-Id: I8f9413b346f6da221b4fff2d37980f00d82e3e3a
2019-03-13 21:24:34 +00:00
Chris Dent
5cf552ef4d Move Trait and TraitList to own module
This is a precursor to getting rid of the TraitList
class in favor of operating on native python lists.

With the move, also create new unit and functional test files
for the new module.

The contents of trait.py is a copy of what had been in
resource_provider.py with some changes that may be of note:

* classes are put before methods, public methods before private
* a direct import of sqlalchemy's null() is replaced by using
  the already imported sa.null(). It's the same thing.
* ensure_trait_sync renamed to ensure_sync.
* get_trait_ids_from_names becomes ids_from_names
* ResourceProviderTraitTestCase is renamed to TraitTestCase

Change-Id: Ibe11ebd7ccc97a46586edd6c897b173ed67ca134
2019-03-13 21:24:34 +00:00
Zuul
62a5e79a1b Merge "Make policy init more threadsafe" 2019-03-13 20:39:16 +00:00
Chris Dent
a844d97ff1 Clean up the intro to the REST API section
* remove the reference to gabbi, as that seems out of date now
  that the api-ref is so huge
* add discussion of "any client that works"
* add links to openstackclient and osc-placement

Story: 2005190
Task: 29941
Change-Id: I257c4dbcec9ed68e3562d6001174bff0d679609e
2019-03-13 18:06:50 +00:00
Chris Dent
ee27e3b1ff Clean up around links to database migrations scripts
* Link directly to the scripts in git.o.o
* Changes the information about needing the migration to a warning,
  for sake of visibility
* Note that an upgrade at the same time as everything else is not
  required

These changes aren't intended as a substitute for true "upgrading from
nova docs", rather that in this particular place in the docs, something
needs to be said to summarize. Later changes that write that true
doc will be linked to from here.

Story: 2005190
Task: 29939
Change-Id: Iae418d8bf385134ff1a93c6251bd9bf0eeee7b3e
2019-03-13 18:06:50 +00:00
Chris Dent
18a80b4e8e Indicate existince of sync_on_startup option
Clarify the options for database synchronization and clean up
how the relevant oslo.config options are linked.

Story: 2005190
Task: 29937
Change-Id: I3128203eed4ac9d0b5aa568993390212c62ffeeb
2019-03-13 18:06:50 +00:00
Chris Dent
def5159ffd Link to install docs from deployment overview
So that people know where to go for more info.

Story: 2005190
Task: 29936
Change-Id: Id9124287901a6697c0211ffbdf4ae4d7f6aeed5e
2019-03-13 18:05:37 +00:00
Zuul
834e86cded Merge "Be explicit about which conf is being used by policy enforcer" 2019-03-13 02:18:25 +00:00
Zuul
e0e106a96d Merge "Rename global _ENFORCER_PLACEMENT to _ENFORCER" 2019-03-12 21:35:24 +00:00
Zuul
377d59b0ea Merge "Do not separately configure logging for alembic" 2019-03-12 21:35:15 +00:00
Zuul
512a4b8318 Merge "Use sync_on_startup in placement-perfload job" 2019-03-12 21:35:14 +00:00
Chris Dent
3414f38413 Remove NAME in placement/deploy.py
It hasn't been used in a long time, and was pointing to 'nova'
so may as well get rid of it.

Change-Id: Iea846e5b69a7abc38c7db922183f6d5b1db948fb
2019-03-12 12:48:11 +00:00
Chris Dent
871b15ef93 Make policy init more threadsafe
In a high concurrency environment, it is possible for the policy
enforcer to be halfway ready, leading to policies not being loaded.
See the linked story for more details on how that can happen.

This change fixes that in two (redundant) ways:

* policy is initialized in the web app at WSGI application
  creation, once, before the web service starts

* The global that contains the _ENFORCER is not changed from
  None to assigned until policy is fully loaded.

Because of the change in when init() happens, the unit
test is changed to not assert the initial state of the
enforcer.

Change-Id: Iae0c5c3ccda7587087606c97c615b9e44e3a68f0
Story: 2005168
Task: 29905
2019-03-12 11:14:41 +00:00
Chris Dent
98f722a603 Be explicit about which conf is being used by policy enforcer
We've got a tradition of avoiding global config where possible, so
extract a _get_enforcer private method which allows us to do some
testing without relying on the implicit global config.

Change-Id: Icb2198ebedb2ff29347ebafda46690a908bf9d27
2019-03-12 11:11:42 +00:00
Chris Dent
ce1589235f Rename global _ENFORCER_PLACEMENT to _ENFORCER
Now that placement is extracted, we only have one enforcer, so no
need for the big name.

Change-Id: I1121a99601c6c32630a9a91518b72f72fe153d5d
2019-03-12 11:02:43 +00:00
Chris Dent
2588c7b8da Update docs bug links to storyboard
Update doc, api-ref and releasenotes conf.py to set 'use_storyboard' to
True. According to the docs of the theme [1] bug project and tag are
not used when using StoryBoard.

doc/requirements.txt (used by all the docs-related jobs) is updated
to make openstackdocstheme>=1.24.0. That version is the most recent
one to have bug fixes related to use_storyboard.

[1] https://docs.openstack.org/openstackdocstheme/latest/#using-the-theme

Change-Id: I3b28dd1da9e8e75eda151a3025e78a5a47c971f9
Story: 2005190
Task: 29948
2019-03-11 20:54:24 +00:00
Chris Dent
b895b02c6d Update CONTRIBUTING and HACKING
Update README.rst to not point to HACKING.rst, only CONTRIBUTING.rst.

Remove HACKING.rst because it is redundant in purpose, and incorrect
in content.

Update CONTRIBUTING.rst to point to the contributor docs on
docs.openstack.org and generally be a bit less generic than before.

Change-Id: Ib608b494e027091ae324793f275bad3b554e9710
Story: 2005190
Task: 29949
Task: 29950
2019-03-11 18:28:36 +00:00
Matt Riedemann
69b9659a45 Remove extraction warning from README
The warning in the readme is a bit old and confusing now
that placement is extracted and can be installed standalone
separate from nova in Stein and is its own project under
openstack TC governance, so this just removes the warning.

Change-Id: I69066cf21327677734c814fd7211135ff672ec3a
Story: #2005190
Task: 29946
2019-03-11 16:01:04 +00:00
Matt Riedemann
c70533f351 Update bug tracker link in README and CONTRIBUTING
Placement bugs are tracked in StoryBoard now.

Change-Id: I9fcf758229d14fa7e290069c7c6f0d3cf5756db4
Story: #2005190
Task: #29947
2019-03-11 10:30:20 -04:00
zhufl
adc8a6424e Add missing ws seperator between words
This is to add missing ws seperator between words in log message.

Change-Id: I8b9ab73c782d68b3217a50270dc6414dcf88b27f
2019-03-11 16:32:56 +08:00
Chris Dent
b24a1b973b Do not separately configure logging for alembic
Before this patch, when calling alembic commands (as done for db sync,
online migrations and the automatic sync when
[placement_database]/sync_on_startup is True) the env.py file would be
evaluated, clobbering the logging configuration already set in the
process. In the command line tools we didn't notice this because
there's not much logging, but when it happens in the web-service process
it means that while the process may start out logging at a DEBUG level
(if you asked for that) it would then stop once the db sync happened.

It turns out that we can remove the call to fileConfig in env.py and
get rid of the associated configuration settings in alembic.ini
we still get the behaviors we want from the placement-manage
and alembic command line tools.

Change-Id: Id9bc515cee71d629b605da015de39d1c9b0f8fc4
Story: 2005187
Task: 29932
2019-03-09 12:32:04 +00:00
Chris Dent
65da83aece Use sync_on_startup in placement-perfload job
Use the [placement_database]/sync_on_startup config setting to
have the database schema synchronized during web-service startup
rather than through a separate call to placement-manage.

This is done for two reasons:

* It provides a reaonable test that it works, which is not present
  in other integration tests.
* Until Id9bc515cee71d629b605da015de39d1c9b0f8fc4 merges it will
  demonstrate the bug described in the story linked below.

Couple of things to note:

* The tempest job will continue to exercise placement-manage, as it
  has always done.
* The bug (in the story) doesn't impact the behavior of the API, it
  merely impacts what is or is not logged. In the
  logs/placement-api.log generated in the perfload job for this change
  there will be an initial burst of DEBUG and INFO logging, but then
  only request logging. This should be corrected by
  Id9bc515cee71d629b605da015de39d1c9b0f8fc4

Change-Id: Ib7f5cdfa3b314af7681d594dccb553bddb764224
Story: 2005187
2019-03-09 12:24:22 +00:00
Zuul
6d42fc4baf Merge "Remove the ResourceClassList class" 2019-03-08 00:24:38 +00:00
Zuul
b8cfee9d05 Merge "Move ResourceClass and ResourceClassList to own module" 2019-03-08 00:24:36 +00:00
Tetsuro Nakamura
5a667c1f05 Trivial: Update doc for _set_traits()
This patch updates the doc for _set_traits()
because TraitList is now removed.

Change-Id: I29967aed8d3995d05110602fb382d282cd4b1a5b
2019-03-07 14:41:34 +00:00
Chris Dent
ba328870c5 Remove the ResourceClassList class
The ResourceClassList class had one method "get_all". Pull that
out to a module level function and update tests accordingly.

Change-Id: I43a382dcfaf2eb8b65b90c145ba8b186446cf497
2019-03-07 12:22:03 +00:00
Chris Dent
7964cd9cbc Move ResourceClass and ResourceClassList to own module
This is a precursor to getting rid of the ResourceClassList
class in favor of operating on native python lists.

With the move, also create new unit and functional test files
for the new module, and move resource class syncing into the new
module.

The ensure_resource_classes_sync is renamed to ensure_sync since
its membership in the resource_class module gives sufficient
namespacing.

Change-Id: I95c383e22f11fce2eb68ef29106fe045315644b6
2019-03-07 12:19:42 +00:00
Tetsuro Nakamura
9f452f6f57 Trivial: pull _normalize_trait_map() out
This patch refactors _get_provider_ids_for_traits_and_aggs() a bit to
reduce the complexity of the function to prevent it in advance from
having the max complexity in the project.

Change-Id: I8d9c79a89fb93f4497724d24d0984ab5d2aaf4c5
2019-03-06 22:27:59 +00:00
Tetsuro Nakamura
a6002659af Use oslo_utils.excutils for reraise exception
In some cases the exception context could be cleared, resulting in None
being attempted to be re-raised after an exception handler is run.

To avoid this, this patch changes some exception handlers to use
oslo_utils.excutils to run handler code before re-raising the original
exception.

Change-Id: Ife95eb62618321564260687869bc2a2e58264a27
2019-03-06 20:06:26 +00:00
Eric Fried
aa2c1442af Add explicit short-circuit in get_all_by_consumer_id
PyCharm complained of a potentially-unset local variable in
get_all_by_consumer_id, even though that wasn't actually possible given
the list comp would be iterating over zero values in that case. However,
adding an explicit short-circuit for this case makes the code clearer to
read, and gets rid of the IDE warning, so \o/.

An unrelated typo in a comment is fixed at the same time.

Change-Id: Ic7b5a223a8eab8f7dd651521b8723d89abf2ded1
2019-03-06 16:38:49 +00:00