490 Commits

Author SHA1 Message Date
Stephen Finucane
4b4dd5ecf8 Add whereto for testing redirect rules
Use whereto to test that the redirect rules do what we expect.

Change-Id: Id3a8f3b9f372ebe9176f1df917f7a6aac30a8e92
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2019-07-08 09:18:18 +01:00
Chris Dent
3659df6e75 Update implemented spec and spec document handling
Forbidden aggregates was added in microversion 1.32 (part of Train)
and allocation candidates mappings was added in 1.34 so those specs
have been moved to implemented.

An _extra/.htaccess is added to redirect the approved spec to the
implemented spec. If we're experiencing enough volume of specs to
warrant automating this, it should be easy to do, but at this point
we don't have, nor intend to have, large numbers of specs.

While doing that I noticed the management of spec documents can be
simplified somewhat by moving placeholders and templates to a
non-release specific location. The placeholder can then be used
as a symlink when in approved or implemented when those are first
created or otherwise empty (an empty directory when doing a table of
contents glob is an error so some kind of file is needed).

Contribution docs are updated to reflect the location of the
template.rst file. The template.rst file is updated to be release
generic.

Change-Id: Icb886d5062a52bfc757ed7bbe36ed8a63abe1387
2019-07-08 09:17:55 +01:00
Zuul
77000c1408 Merge "Spec: Support Consumer Types" 2019-07-01 21:19:50 +00:00
Surya Seetharaman
ed4a9a1d78 Spec: Support Consumer Types
This spec aims at providing support for services to model
``consumer types`` in placement. While placement defines a
consumer to be an entity consuming resources from a provider
it does not provide a way to identify similar "types" of
consumers and henceforth allow services to group/query them
based on their types. This spec proposes to associate each
consumer to a particular type defined by the service owning
the consumer.

Change-Id: I79ce8f1b20c60b89fc1ed0ece8bcc321e9435bad
Story: 2005473
Task: 30554
2019-07-01 10:50:31 -05:00
Zuul
ceae90feac Merge "Centralize and clarify pip in the docs" 2019-07-01 13:48:57 +00:00
Zuul
63dacfb59d Merge "Mention OsProfiler in the testing doc" 2019-06-28 02:38:40 +00:00
Eric Fried
180c83b52b Centralize and clarify pip in the docs
Add a clarifying note about running pip (e.g. you might have to spell it
pip3) in a centralized place and link it from elsewhere. We should link
to this whenever we mention pip so we don't duplicate information.

Change-Id: Ie94ab144005a5846fc951f4f387703bd241181c8
Story: #2005950
Task: #34315
2019-06-27 11:52:15 -05:00
Balazs Gibizer
243757dde6 Mention OsProfiler in the testing doc
Since I5a0e805fe04c00c5e7cf316f0ea8d432b940e560 OsProfiler can be used
with extracted placement service to trace the wsgi endpoint. This patch
adds documentations to help using the OsProfiler.

Story: 2005842
Task: 34191
Change-Id: I33a4f529660057e00e6da25fa4fb4263854819d1
2019-06-27 12:41:58 -04:00
Zuul
060b330b17 Merge "Miscellaneous doc/comment/log cleanups" 2019-06-27 05:47:50 +00:00
Zuul
d99039de9e Merge "Microversion 1.35: root_required" 2019-06-26 18:43:23 +00:00
Eric Fried
698645ca5c Miscellaneous doc/comment/log cleanups
This commit is a grab-bag of tweaks and cleanups noticed while working
in the neighborhood on something unrelated.

- Add a descriptive comment to the ResourceProviderNotFound exception.
  This is a marker exception used for inter-method communication.
- Add and enhance debug logs in the in_tree$S filter code path.
- Correct a SQL comment in get_providers_with_resource (missed when the
  code was tweaked).
- Removes a TODO about an optimization that became obsolete when
  RequestGroupSearchContext was integrated into
  get_provider_ids_matching.
- Corrects the get_sharing_providers docstring s/list/set/ to reflect
  reality.
- Changes a link in the provider-tree doc from the spec-in-progress to
  the merged and published version.

Change-Id: Id433b0540e2051ab27b2c22cc14c62105555ea8f
2019-06-25 08:33:28 -05:00
Zuul
93a7470d86 Merge "Spec for nested magic 1" 2019-06-24 15:45:54 +00:00
Eric Fried
b733786a0a Microversion 1.35: root_required
Microversion 1.35_ adds support for the ``root_required`` query
parameter to the ``GET /allocation_candidates`` API. It accepts a
comma-delimited list of trait names, each optionally prefixed with ``!``
to indicate a forbidden trait, in the same format as the ``required``
query parameter. This restricts allocation requests in the response to
only those whose (non-sharing) tree's root resource provider satisfies
the specified trait requirements.

This is to support use cases like, "Land my VM on a host that is capable
of multi-attach," or, "Reserve my Windows-licensed hosts for special
use."

Story: #2005575
Task: #33753
Change-Id: I76cad83248920fa71da122711f1f763c4ebdb1ba
2019-06-20 17:30:35 -05:00
Chris Dent
c00d043376 Spec for nested magic 1
This is a spec for several nested features which has been split
off from the original spec for nested magic [1] in an effort to
not delay progress on these features while 'can_split' is being
discussed.

This spec describes a cluster of Placement API work to support several
interrelated use cases for Train around:

* Modeling complex trees such as NUMA layouts, multiple devices,
  networks.
* Requesting affinity (in the NUMA sense) between/among the various
  providers/allocations in allocation candidates against such layouts.
* Describing granular groups more richly to facilitate the above.
* Requesting candidates based on traits/aggregates that are not
  necessarily associated with resources.

In particular, it describes the new GET /allocation_candidates
features:

* arbitrary group suffixes
* same_subtree query parameter
* resourceless request groups
* root_required, root_member_of

[1] I7c00b06e85879ab1bf877ce32979e8cc898bfd9e

Co-Authored-By: Eric Fried <openstack@fried.cc>
Change-Id: I55973aa7de4a85b63dff4a7d1afb6c36796af71a
Story: #2005575
Task: #30949
2019-06-20 10:18:06 -05:00
Colleen Murphy
99242b83ee Update SUSE install documentation
openSUSE and SLES have Placement packages, but they are named slightly
differently and the default Apache configuration is slightly different
than what was documented. See the package definition in OBS[1]. This
patch makes those corrections.

[1] https://build.opensuse.org/package/show/Cloud:OpenStack:Stein/openstack-placement

Change-Id: Id0eca721d850d849c7a273683c7f393e5a216682
2019-06-19 11:08:12 -07:00
Chris Dent
88c6ad9cb4 Optionally run a wsgi profiler when asked
This is the code used to run placement with the wsgi
profiling described in a blog post [1]. It has proven
useful enough that we may wish to include it in the
released code. It is added in a way that is off by
default and makes no changes to requirements.

Brief docs are included in testing.rst. They are
brief because it's assumed that someone who wants to
do this already mostly knows what they want to do and
merely needs the specifics on how to do it in this
environment.

[1] https://anticdent.org/profiling-wsgi-apps.html

Change-Id: I342512732b94bc19bd711684ba3ec9480cc51f81
2019-06-05 19:41:18 +00:00
Zuul
fb57f23bf7 Merge "Resource provider - request group mapping in allocation candidate" 2019-06-04 03:24:37 +00:00
Balazs Gibizer
7049e40786 Resource provider - request group mapping in allocation candidate
To support QoS minimum bandwidth policy during server scheduling
Neutron needs to know which resource provider provides the bandwidth
resource for each port in the server create request.

Story: 2005575
Task: 30804

Change-Id: Iafdb0eab9b41f4c34c93cada08a4da27cf4b1499
2019-06-03 13:52:21 -05:00
Eric Fried
289130fc3d Bump openstackdocstheme to 1.30.0
...to pick up many improvements, including the return of table borders.

Change-Id: Ic3a43e2e35f361234f081c0e1f34729cf6560828
2019-05-30 18:08:16 -05:00
Zuul
b0d1bb61f7 Merge "Allow [a-zA-Z0-9_-]{1,64} for request group suffix" 2019-05-29 02:53:34 +00:00
Matt Riedemann
4af1df9408 Change "Missing Root Provider IDs" upgrade check to a failure
We want to drop the REST API compability code for resource
providers with no root_provider_id in Train, so to start
we should make the related upgrade check a failure rather
than a warning.

Change-Id: Ifd3c84ea3348fc9e6653838d6fba4a5eb864f01e
Story: 2005613
Task: 30921
2019-05-22 14:45:36 -04:00
Chris Dent
fb0f6f2608 Allow [a-zA-Z0-9_-]{1,64} for request group suffix
Add a 1.33 microversion to move from numeric suffixes to string
suffixes that can be 64 chars longs made from '-', '_', and
mixed-case alphanumeric. The format is shared between schema
and RequestGroup parsing.

Docs, api-ref, api history and microversion upper limit are updated
to indicate the new form in the new microversion.

A release note is added.

Story: 2005575
Task: 30781
Change-Id: Ia44b0922d151695d406883262e891bd932536f38
2019-05-21 11:07:38 +01:00
Chris Dent
e98b5df3d9 Add 'docs' worklist to worklist table
Since some people may be looking for docs-related work,
add a link to the docs worklist. Stories in the placement
project group show up there if they have been tagged 'docs'.

Change-Id: Ie9ff8c3527c145b4cbc356444acc587931398d00
2019-05-20 11:33:10 +01:00
Qiu Fossen
43c859b421 Cap sphinx for py2 to match global requirements
Change-Id: I7d95fedf44dd0504a57ffa4c4ecd54fccf7a0230
2019-05-17 05:06:10 -04:00
Zuul
e481ee4655 Merge "Package db migration scripts in placement pypi dist" 2019-05-07 06:54:34 +00:00
Chris Dent
c62ed6ccba Package db migration scripts in placement pypi dist
Having the db migration scripts within the openstack-placement pypi
distribution is desirable for deployment tools, such as
openstack-ansible. It provides a known good location for the script,
available with a pip install.

There are several ways to distribute files with a python package. The
method used here was chosen because it works both with tarballs and
wheels (the files are already in the tarball, as a result of the way pbr
works, but not in the wheel).

Here's what's done:

* The db migrate scripts are put in their own direcory,
  placement_db_tools, so that only they are packaged, not the other
  tools.
* To preserve how grenade interacts with these files as well as not
  disrupt the docs, symlinks from tools to placement_db_tools have been
  created.
* placement_db_tools is added to the list of packages included in the
  openstack-placement distro. This means that when 'pip install
  openstack-placement' happens, the python environment will then include
  placement and placement_db_tools directories.

The end result is that the true path to the script can be found with:

  pkg_resources.resource_filename('placement_db_tools', 'mysql-migrate-db.sh')

This has been noted in the to-stein.rst document.

A different package was chosen to not muddy the waters of what is
"actually placement".

Similarly, the 'data_files' functionality provided by pbr was not used,
because that requires the file be written to a location on the local
filesystem, relative to the install prefix. Dirtying the filesystem
outside the python lib with this sort of thing is inappropriate.

Change-Id: Ie326ce8a2a0692d20793bc18be606e034fa94a44
Story: 2005535
Task: 30671
2019-05-07 00:08:39 +00:00
Zuul
7f92fdb729 Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-05 14:34:16 +00:00
Zuul
8f2a400208 Merge "Update worklist information for contributors" 2019-05-02 07:03:20 +00:00
Chris Dent
e9160abc20 Update worklist information for contributors
When the worklists were first created, cdent was unaware of
how to make the worklists filter by story state so a tag of
'fixed' was being used to indicate "done".

This change updates the docs to reflect new awareness. The
associated worklists have already been updated.

Change-Id: I445b67ff85be7979805d72ac953f0853ee738f46
2019-05-01 14:03:05 -06:00
Zuul
6137a69cbc Merge "Remind people to use postgresql-migrate-db.sh when migrating data" 2019-05-01 03:57:31 +00:00
Matt Riedemann
ae9bdbaa35 Remind people to use postgresql-migrate-db.sh when migrating data
The "Upgrading from Nova to Placement" doc mentions the
postgresql-migrate-db.sh script but within a big wall of
text and the rest of the examples in the guide are using
mysql, so people might miss the postgresql script which
could result in errors later if their sequence ids are
wrong.

This adds a note to serve as a reminder.

Change-Id: Id93531cc2d440b1adde1c0bd7c504f737b14012c
Story: #2005478
Task: #30568
2019-05-01 03:10:24 +00:00
caoyuan
a2513d8ea6 Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I62921ef9ee51c391bbd2b4e39941df5b68a9a908
2019-04-29 13:06:10 +08:00
Chris Dent
1b5710db74 Add links to storyboard worklists to contributing.rst
These worklists exist but are not easily discoverable so
they need to be linked from somewhere. contributing.rst seemed
like the best place.

Change-Id: If74cf2f510441f500c2e13612f99631d56cf4bfe
2019-04-18 12:38:09 +01:00
Chris Dent
c3a1aefefb Fix arg typos in contributing.rst
Review of changes to contributing.rst noticed an unrelated
typo in the arguments used in the examples for visual indent.

Change-Id: I2268fa6aa6c9272ffbb7b6d3b439539765197fed
2019-04-09 11:41:47 +01:00
Chris Dent
3d29a4b7d8 Correct task status when a task is under review
"Progress" is the wrong state for a task that has been
submitted for review. The correct state is "Review".

Update contributing.rst to reflect this.

Change-Id: I221fc533ba86e82dc6f119ef20fc25a54a162f92
2019-04-02 14:57:57 +01:00
Zuul
15e3e8c6c3 Merge "Spec: Support filtering by forbidden aggregates 2" 2019-03-28 17:09:11 +00:00
Zuul
f8089dfbec Merge "Spec: Support filtering by forbidden aggregates 1" 2019-03-28 17:09:06 +00:00
Zuul
1be6fab6f2 Merge "Fill in the Writing Code section of contributing.rst" 2019-03-28 17:09:00 +00:00
Zuul
087afea315 Merge "Fill in the New Features section of contributing.rst" 2019-03-28 17:08:56 +00:00
Zuul
e155061242 Merge "Fill in reviewing section of contributing.rst" 2019-03-28 15:59:07 +00:00
Zuul
f678512af3 Merge "Fill in the bugs section of contributing.rst" 2019-03-28 15:59:06 +00:00
Zuul
debb35668e Merge "Document alloc-candidates-in-tree" 2019-03-28 15:49:24 +00:00
Chris Dent
cc959d53de Fill in the Writing Code section of contributing.rst
Provide several rules and guidelines for constructing good
changes to submit to gerrit. This list can never be complete
but this gets us started. As with everything else, there's a
balance between being too prescriptive and not prescriptive
enough.

Change-Id: I30243e81ea8711296ce0884d93e985c6d2b5cefd
2019-03-28 12:22:33 +00:00
Chris Dent
73a585d7a5 Fill in the New Features section of contributing.rst
Write up general outlines on managing specs. As with the other
section, this avoids overloading the description with too many
details. As we discover gaps we can fill them in.

Change-Id: I2d980ea05be5067cd2c98919854cedeb438f6893
2019-03-28 12:12:05 +00:00
Chris Dent
aa1ca270f7 Fill in reviewing section of contributing.rst
This provides some links to more info about reviewing, and some
guidelines on how to be a good reviewer, with special
considerations for cores.

Not included is information on how a patch submitter should hassle
someone to get attention. With the low volume of patches that go
through placement, that should not be something the submitter should
have to care about. It's the job of regular reviewers to be aware
of new stuff. The guideline about latency covers that.

Change-Id: I61cf791c956a75304b87ffb4a16a8252fc36800b
2019-03-28 12:07:35 +00:00
Chris Dent
fcd99b3354 Fill in the bugs section of contributing.rst
For now just the very basics of how to create a good bug, what tags
to use, information to include, etc. And some encouraging words on
doing triage.

At this time no effort is made to describe a process for things like
"backport potential" or what release something is associated with.
We'll figure out that once, as a group, we have a better idea of what
StoryBoard can do for us. It is likely that a combination of tags
and tasks will be the order of the day.

Change-Id: Ie9e3a2c624357c846bf861ae6de39396a0fd958d
2019-03-28 11:58:30 +00:00
Chris Dent
a83c5a4c82 Add initial framing for a contributing doc
We have contributor documentation, that explains placement
from within, but we don't have much in the way of docs that
explain how to make contributions and the guidelines for
how do it well. This patch is the first in a series that will
add that information.

This first patch simply creates an initial contributing page
with a light intro and titles for four initial sections.

Change-Id: Iea127a052a42ed5cc5349b89137381f782717af3
2019-03-26 20:26:45 +00:00
Zuul
02fb9cf8c7 Merge "Initial structure for in-tree specs" 2019-03-25 23:24:32 +00:00
Ian Wienand
c99492cfc6 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I669e77c1cc1d626552939b8f65e093e7901ad754
2019-03-24 20:35:35 +00:00
Chris Dent
48948bce9e Initial structure for in-tree specs
We decided in IRC and email discussions that henceforth
placement specs will reside in the same tree as code and docs.
This change sets that up using the following structure:

* There is a docs/source/specs top-level index that should be
  updated with each release to include tables of contents generated
  from cycle directories (currently pointing to train) for
  approved and implemented specs.

* In the approved directory for any cycle there will be a
  <cycle-template>.rst file to use as the basis for any new
  spec. New specs go in this directory.

* In the implemented directory is a placeholder file which
  will be removed when the first (if any) spec moves from
  approved (meaning in-progress) to implemented. The PTL
  or their designee will be responsible for doing that moving.
  The placeholder file is required to avoid warnings (which
  will kill the tox docs job) from the table of contents
  globbing.

* The train-template.rst is a copy-with-edits from nova's
  version of the same thing. The main changes are to indicate
  that StoryBoard, not launchpad blueprints, are the starting
  point and to remove various nova-isms.

  It is likely we will want to further tweak the template as
  we decide what matters.

The next steps for this are for people with placement specs
that had been pending in nova in stein, to resubmit them to
placement as train specs (in doc/source/specs/train/approved).
It is quite likely and not surprising that it will sometimes
be confusing whether a spec should be in nova or placement.
Ideally we can identify the "just placement" aspects of a
change and make a placement-only spec for that which the nova
spec makes reference to. This may sound like a pain but it is
a feature, not a bug, which helps make sure that placement
evolves its API as a strong contract for all potential clients.

Change-Id: I85854d771409701cea2fcf7a218f02af60dba72e
2019-03-22 13:33:09 +00:00