Discussion in IRC suggested that
https://anticdent.org/quick-placement-development.html ought to do in
the contributor docs, so here is a version of it, translated from
Markdown to RST.
Change-Id: I7a014e96d2f4c01594d977c5a4af9894278dd511
This is the first in a series of changes to the content in
doc/source to make it more accurately reflect the current state of
placement, now that it is in its own repo.
There are small changes throughout the front page, mostly to make
it more clear and make useful stuff a bit more discoverable.
The two major changes are:
* Older (pre-stein) upgrade notes are found by linking back to
nova's rocky documentation.
* The REST API history is moved to its own sub-page, otherwise it
the page is overwhelming and noisy.
Close review is probably required. cdent typed this. That has
implications.
Change-Id: I66e0c7d18b253b0a5a8fdac65e30b5b3cef37db2
The tests for placement-manage, in unit/cmd/test_manage were not
verifying help message output when partial or incorrect commands
were provided. This led to some confusion when trying to verify
a fix related to different behaviors of argparse.
Here, some verification is added. Because of differences between
python 2 and 3, we need to look at stderr and stdout, respectively, to
verify the output.
Change-Id: I337a25cfdc23bc56fbf15f128a94d2b1b1d1b664
Related-Bug: #1804420
Doing so with cpython < 2.7.9 results in the following bug being hit:
argparse set_defaults on subcommands should override top level set_defaults
https://bugs.python.org/issue9351
As CentOS 7 continues to ship with cpython 2.7.5 this change simply
removes the parent parser call to set_defaults for py2 while leaving the
associated tests unchanged. This should ensure that the actual behaviour
of placement-manage is unchanged while allowing these tests to pass once
again on CentOS.
Change-Id: I87989ced31b1d39e30c992b34fe2ae6db06f7883
Closes-Bug: #1804420
The default bug tag for placement doc is defined
as 'docs' currently in doc/source/conf.py.
But it should be 'doc'(*1) instead of 'docs'.
So replace 'docs' with 'doc'.
*1: https://wiki.openstack.org/wiki/Nova/BugTriage#Tag_Owner_List
TrivialFix
Change-Id: I469ad687f655b96f7a0be11e1b92e30a31e3184d
This adds the basic framework for the
placement-status upgrade check command which
is a community-wide goal for the Stein release.
A simple placeholder check is added which should
be replaced later when we have a real upgrade
check.
Change-Id: I9291386fe7fcbfc035c104ea9fdbe5eb875c4776
Story: 2003657
Task: 27518
There are cases where ``root_provider_id`` of a resource provider is
set to NULL just after it is upgraded to the Rocky release. In such
cases getting allocation candidates raises a Keyerror.
This patch fixes that bug for cases there is no sharing or nested
providers in play.
Change-Id: I9639d852078c95de506110f24d3f35e7cf5e361e
Closes-Bug:#1799892
This change makes it easier to use the PlacementFixture from external
systems (mostly nova) that would like to run functional tests with a
stubbed in placement service, including a database.
A 'set_config' boolean flag is added to the Database fixture to allow
the caller to indicate they have not themselves set a connection string
in the placement_database group and want the fixture to do so.
That fixture attempts to register the group and opt but allows that to
fail if it has already been done. This provides some resiliency in the
face of nova currently having the group and option but not having it
later.
Because both nova and placement use global config, we need to avoid
registering and processing all of placement's config options anywhere
during the use of the fixture (from nova), otherwise errors will
happen. Therefore, we cannot use migration.upgrade in the Database
fixture, as env.py currently pulls in and processes a global config.
Instead we use a create_schema method. Migrations continue to be
tested by the migration specific tests.
The necessary functionality of syncing traits and resources classes is
handled by the fixture itself, rather than things that call it, allowing
some cleanups in other fixtures. This centralizes database handling in
one place, which is probalby how it should have been all along, but as
with all evolving things, it meandered.
The PlacementFixture is updated to use the Database fixture
automatically, unless a db keyword arg is set to False.
Change-Id: I7d4099c1da425bb4ea115ac51c82f5748203f2d3
This change adds a rudimentary command line tool, 'placement-manage'
that can do three things:
* placement-manage --version: report the current version of the software
* placement-manage db sync: sync the db to the 'head' alembic version
* placement-manage db version: report the current version of the db
It is written following the examples set by both nova and ironic for
using oslo config driven command line parsing. It is not as full
featured as the nova version (with decorators for argument handling and
help text etc) because we don't need that yet. There's plenty of room
for improvement, but the intention here is to keep things simple until
we need them to be otherwise.
The provided unit tests cover that the arg parsing behaves as expected
and that the right commands are connected the right args. The actual
functionality of the 'db sync' command (a migration.upgrade) is tested
in runs that use devstack as well as the test_migrations functional
test.
Errors during the migration simply raise exceptions which will output
noisily to the console. At this stage this is desirable and suitable.
A new doc for the cli is added, with basic information for
placement-manage. Note that at this time the deployment documentation
has not been updated. That will be done in a later change.
Future changes may include (if we can't avoid it) tools for initiating
online migrations from the command line.
Needed-By: https://review.openstack.org/600162/
Change-Id: I1ff472106610150a587f5286f26a6bd7c1aa84f4
With the migrations now gone, we can safely remove sqlalchemy-migrate
from requirements.txt and lower-constraints.txt.
Change-Id: Id8d03d9532bcf4c4f7ad1c1039fa6a42990f8d5d
This gives some basic documentation for creating a revision using
alembic. It then links to the alembic tutorial for more information.
Change-Id: I8647aaa55dca537d95c7c03753f2a24f364f0ddf
The api-ref-jobs has been added
since Ia4680f24d78af1260f2f0106a458b78a079c1287.
It includes build-openstack-api-ref jobs in check and gate.
So remove the build-openstack-api-ref job definitions
in .zuul.yaml because they are unnecessary.
Change-Id: I4b612fa87609240bd0a0f838163ee87c45a41a46
As the first step to implementing alembic for our data migrations, we
need to get rid of the old migration files.
Change-Id: Ib6ee4895a3c5b18b027f1ac44905dff0e28cdcf8
Added alembic, and created an initial database revision.
For the sake of testing with real databases, tools/test-setup.sh is added
so that CI can start up a mysql and/or postgresql database on which to
test migrations. The OpportunisticDBMixin from oslo_db was originally
used for this, but because of the way placement configures the database
this proved hard to debug and get correct, so we've used something
that leverages the available oslo_db tools, but with more visibility.
Because these tests mix sqlite, mysql and postgresql settings in the
potentially the same process we need a way to insure that global
settings for databases do not leak into other tests. This is done with
a reset() on the placement db fixture, called by the msyql and
postgresql tests before and after they run. We also need careful
management of that cleanup when these tests are skipped (because db
server or database is not there).
Those tests will confirm that the models match the migrations so we also
need to remove model files that no longer matter.
Since we no longer need to distinguish among multiple database files, we
can simplify the naming of these files.
Co-Authored-By: Chris Dent <cdent@anticdent.org>
Change-Id: I51ed1e4e7dbb76a3eab23af7d0d106f716059112
There are cases where ``root_provider_id`` of a resource provider is
set to NULL just after it is upgraded to the Rocky release. In such
cases getting allocation candidates raises a Keyerror.
This patch recreate that bug by simulating the situation by
inserting the records to the database directly.
Change-Id: Iaed912314f3e8fef2f46453a6bf12011702ae1dd
Related-Bug:#1799892
- This change tries to address an edge case discovered
when running placement under mod_wsgi where
if the placement wsgi application is re-initialize the db_api
configure method attempts to reconfigure a started transaction
factory.
- This since oslo.db transaction factories do not support
reconfiguration at runtime this result in an exception being
raised preventing reloading of the Placement API without
restarting apache to force mod_wsgi to recreate the
python interpreter.
- This change introduces a run once decorator to allow annotating
functions that should only be executed once for the lifetime of
an interpreter.
- This change applies the run_once decorator to the db_api configure
method, to suppress the attempt to reconfigure the current
TransactionFactory on application reload.
Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Closes-Bug: #1799246
Related-Bug: #1784155
Change-Id: I704196711d30c1124e713ac31111a8ea6fa2f1ba
One of the things Placement will need when it is properly publishing
itself is a working release notes job. Until this change there was
no source/conf.py or source/index.rst for releasenotes job to use.
This adds it, we can now create release notes output, with the
openstackdocs theme.
However: Because the placement repo has no branches, all the existing
release notes in releasenotes/notes are considered 'unreleased', which
makes for a lot of noise at the page generated by unreleased.rst.
What this probably means is that we will want to clear out all the
release notes that pre-date extraction and start anew. An experiment to
try removing some notes shows that the releasenotes will produce some
messages about missing release notes, but neither warn nor error, and
the output is good, so this should work out okay for us.
That will be easy to do now that we have a working releasenotes tox env.
Note: This change does not add any jobs to zuul, it just makes tox
work for releasenotes.
Change-Id: I61a7ce492395e0ae69dd0401a141ea4bb93516b9
The driving reason for this change is to allow the placement.db_api
module to be able, in subsequent patches, to import generic utility
routines from placement.util without causing an cyclical import.
It is moved into the handlers package because the only caller is
in there too (in allocation.py).
Relevant unit test are moved, and the callers of ensure_consumer are
updated to reflect the new location.
Change-Id: Ifd1b10e5568b9d6f641e396c3c49101901daff31
The "handle allocation updates in a safe way" item and
the link to "Add Generation to Consumers" spec have been added
since I77f34788dd7ab8fdf60d668a4f76452e03cf9888 in the nova doc
(doc/source/user/placement.rst).
But it has not been added yet in the placement doc
(doc/source/index.rst). So add it.
Change-Id: I4c70da34d8907cab2373b1349bb2c6e9b8cdd35a
Use publish-openstack-docs-pti job instead of
build-openstack-docs-pti job to publish placement documents
as https://docs.openstack.org/placement/latest/ .
Change-Id: I667387ec262680af899a628520c107fa0d4eec24
Some of the descriptions in the file ``doc/source/index.rst`` assume
placement still depends on nova, which isn't true, so this patch
updates it.
This patch also adds a upgrade note for Stein (19.0.0) in that file.
Change-Id: Ib98e370d6863cc8c679a07d04f053ba5a3723c43
The file ``doc/README.rst`` hasn't been changed since the placement
is extracted, so this patch de-nova-ify it.
Change-Id: I30bcfa987a87fd4672dcbcdf3f401295b2dad6f5
When you create the placement document locally with `tox -e docs`,
some of the files are created under the `doc/source/_static` directory
and are tracked by git because the .gitignore file is not set
appropriately. This patch fixes it cleaning up old nova configurations
in the .gitignore file.
Change-Id: I1ecae90c9ba1cccdcd4befec3513ba2fdacb4cea
The etc/placement/policy-generator.conf file still referred
to nova. Now that it does not, policy generation happens as
expected. The name of the generated file has been changed
to policy.yaml.sample, as we no longer need to distinguish
from nova's.
Note that the readme still points to nova docs, because that
is where we currently have the docs. This will need to change
later.
Change-Id: I633431933666c15eaf8da7e91769d0b4335e6c8d
This is based on the mysql-migrate-db.sh script but
with the necessary changes for interacting with postgresql.
Long-term it might be possible to refactor both the mysql
and postgresql scripts into a generic script but the
potential complexity in doing that makes it a lower priority
than simply getting pg support for the nova_api/placement
table migration done in Stein.
Change-Id: I1cab3183067ab2e41663ca369509d753442e775c