... because functionality of this parameter is effectively duplicate of
the HTTPProxyToWSGI middleware in oslo.middleware library.
Closes-Bug: #1967686
Change-Id: Ifebcfb6b5c1594c075bb9c152a06aa7af7c61bc8
The oslo.reports library provides some options under the [oslo_reports]
section. This change ensures these parameters are rendered by
the oslo-config-generator command.
Closes-Bug: #1940733
Change-Id: I135bce7bd557cd5067897486e91c97c1dc61aa2b
Merge these, removing an unnecessary layer of abstraction, and place
them in the new 'nova.db.main' directory. The resulting change is huge,
but it's mainly the result of 's/sqlalchemy import api/main import api/'
and 's/nova.db.api/nova.db.main.api/' with some necessary cleanup. We
also need to rework how we do the blocking of API calls since we no
longer have a 'DBAPI' object that we can monkey patch as we were doing
before. This is now done via a global variable that is set by the 'main'
function of 'nova.cmd.compute'.
The main impact of this change is that it's no longer possible to set
'[database] use_db_reconnect' and have all APIs automatically wrapped in
a DB retry. Seeing as this behavior is experimental, isn't applied to
any of the API DB methods (which don't use oslo.db's 'DBAPI' helper),
and is used explicitly in what would appear to be the critical cases
(via the explicit 'oslo_db.api.wrap_db_retry' decorator), this doesn't
seem like a huge loss.
Change-Id: Iad2e4da4546b80a016e477577d23accb2606a6e4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The 'oslo_db.concurrency.TpoolDbapiWrapper' wrapper provides a way to
enable the experimental use of thread pooling for all DB API calls. This
has been part of 'oslo.db' since that project was first split out on
nova back in change I0649539e071b2318ec85ed5d70259c949408e64b and has
been part of nova for even longer, having been first introduced in
change Id7de85dce11f5ea61dfccddf80de7800a2bc4e37. However, this has not
graduated to non-experimental status in the 7 years since and is
unlikely to so now. It also isn't used for any of the API DB calls.
Hasta la vista, 'TpoolDbapiWrapper'. You've had a good ride.
Change-Id: I1ce8f180d6bef465f9faa0a10c837841ed7ab528
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Make routes to the versioned discovery documents (/v2, /v2.1) go through
paste pipelines that don't require authentication, while leaving their
sub-URLs (/v2.1/servers etc) requiring authentication.
To make this work, our URLMap matcher gets support for a very
rudimentary wildcard syntax, whereby api-paste.ini can differentiate
between {/v2.1, /v2.1/} and /v2.1/$anything_else. The former points to
the unauthenticated discovery app pipeline; the latter points to the
existing "real API" pipeline. Similar for legacy v2.
This entails a slight behavior change: requests to /v2 and /v2.1 used to
302 redirect to /v2/ and /v2.1/, respectively. Now they just work.
Change-Id: Id47515017982850b167d5c637d93b96ae00ba793
Closes-Bug: #1845530
Closes-Bug: #1728732
Finish the job by removing all the now-unused modules. This also allows
us to - wait for it - kill mox at long last. It's a great day in the
parish.
Partial-Implements: blueprint remove-nova-network-ussuri
Partial-Implements: blueprint mox-removal-ussuri
Change-Id: Ia33ec2604b2fc2d3b6830b596cac669cc3ad6c96
Thanks to mikal, nova was fully transitioned to using privsep over
the past cycles. However the old rootwrap filter definitions have been
left behind, which basically means that there is limited security
benefit right now (code gone rogue could still call rootwrap to execute
some code as root).
As far as I can tell, those can be removed now. os-brick calls in
compute.filters should be covered by the os_brick.privileged.default
context.
Rootwrap is now only used to launch the two privileged privsep contexts.
Since those are one-shot at the start of the service, there is little
point in using rootwrap-daemon on nova nodes.
network.filters and api-metadata.filters are being removed in
https://review.opendev.org/#/c/696518 as part of the nova-net cleanup.
Change-Id: Ie5e5ab17dabad713d4ad1261ab8b5d62b017f31e
[api]auth_strategy defaults to `keystone`. The only other choice is
`noauth2`, which activates noauth paste pipelines, which go through
NoAuthMiddleware, which is crusty and bogus. It is used in our
functional tests to avoid having to fixture out keystone, but should not
be used in real deployments, ever. Deprecate the option for removal, and
add a deprecation warning in the paste pipeline if it is used.
When we remove the option, we could just hardcode to `keystone`. At that
time, we also need to move the middleware under the nova.tests package
-- or find a way to get rid of it entirely by instead stubbing out
keystone in tests if that's relatively easy.
Change-Id: I9e2be5423cc0821a628db7a68ad52bbd91264acd
The following file should have been removed
in I4181f39dea7eb10b84e6f5057938767b3e422aff.
* etc/nova/placement-policy-generator.conf
A description of generating the placement policy.yaml file
should also have been removed in the commit.
Therefore remove them.
And the 'wsgi-intercept' package is used for OSAPIFixture and
not for placement functional tests currently.
So a comment is removed in test-requirements.txt.
Change-Id: Ie8d0d7085ccd82a47ea27b98be75c1bc9ba6195b
This was added in change I7046ce55a0a294293c1b1a5fb0f092aeb891ee01 as an
example for how one could configure a cells v1 deployment via a JSON
file. It can be removed now.
Part of blueprint remove-cells-v1
Change-Id: I3235f38c432f1dbafde167ec0e856abe7a38aa8a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We recently exposed the privsep opts for config generator use, so
projects that depend on oslo.privsep should include them in their
sample configs.
Change-Id: I7fab7002d51b2aaf1b0a6545d07b616120e26461
Our sample configuration files are long. Really, really long. Make them
shorter through use of the summarize option added to oslo.config years
ago [1]. This turns a help text like the following:
[DEFAULT]
# This option helps you specify initial virtual disk to physical disk
# allocation ratio.
#
# This is only used when initially creating the ``computes_nodes``
# table record for a given nova-compute service.
#
# See https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html
# for more details and usage scenarios.
#
# Related options:
#
# * ``disk_allocation_ratio``
# (floating point value)
# Minimum value: 0
initial_disk_allocation_ratio = 1.0
Into:
[DEFAULT]
# This option helps you specify initial virtual disk to physical disk
# allocation ratio. For more information, refer to the documentation.
# (floating point value)
# Minimum value: 0
initial_disk_allocation_ratio = 1.0
Shorter, one line descriptions are kept as-is.
[1] https://github.com/openstack/oslo.config/commit/2f8025b22
Change-Id: Iaa51b6a139fc9b7ffc97cd6881a9d49b29858b9b
osprofiler is optional, but if it's installed we'll
load up the configuration options from the library,
but they weren't in the generated config sample so
people would have to find the osprofiler docs, or
worse the code, to figure out how to configure it.
This simply adds the osprofiler config options to the
nova config sample, which will also show up in the
config reference docs.
Change-Id: I28d35165ed77487cd49d560fb1eda4f1d640734e
Closes-Bug: #1774208
This adds a granular policy checking framework for
placement based on nova.policy but with a lot of
the legacy cruft removed, like the is_admin and
context_is_admin rules.
A new PlacementPolicyFixture is added along with
a new configuration option, [placement]/policy_file,
which is needed because the default policy file
that gets used in config is from [oslo_policy]/policy_file
which is being used as the nova policy file. As
far as I can tell, oslo.policy doesn't allow for
multiple policy files with different names unless
I'm misunderstanding how the policy_dirs option works.
With these changes, we can have something like:
/etc/nova/policy.json - for nova policy rules
/etc/nova/placement-policy.yaml - for placement rules
The docs are also updated to include the placement
policy sample along with a tox builder for the sample.
This starts by adding granular rules for CRUD operations
on the /resource_providers and /resource_providers/{uuid}
routes which use the same descriptions from the placement
API reference. Subsequent patches will add new granular
rules for the other routes.
Part of blueprint granular-placement-policy
Change-Id: I17573f5210314341c332fdcb1ce462a989c21940
xenapi likes enabling and disabling ext3 filesystem journals. They can
do that via privsep now.
Change-Id: Iad8198fbd01aa80bde0a6b295963391715c5cd48
blueprint: hurrah-for-privsep
Including updating their unit tests. We can now remove parted from
the rootwrap configuration.
Change-Id: I8cbfe296238976001e38997842059ec2f137f660
blueprint: hurrah-for-privsep
The same pattern as the others, except with a generated command line.
Change-Id: Icfbe3566d8cb82e6878ab4097ed747b18fd5e28a
blueprint: hurrah-for-privsep
The same as the mellanox example, but for midonet.
Disturbingly midonet appears to not have unit tests either, but
again I feel that correcting that is outside the scope of the privsep
blueprint.
Change-Id: I672534691a94a0ac294410ea12dd4ba2c327c0e0
blueprint: hurrah-for-privsep
This code isn't well labelled, but I am pretty sure it is for
Mellanox Infifiband VIFs. Same pattern as the others.
As best as I can see these methods had no test coverage, but I think
that's outside the scope of the current privsep work to fix.
Change-Id: I323399643c9978a115fdc1213876da2d85dcd8db
blueprint: hurrah-for-privsep
The same pattern as previous patches. Some of these unit tests
are starting to be a bit simpler as we finish the transition.
Change-Id: If0e1fe4c0466f2f88525dc575af2ef366d4bb59d
blueprint: hurrah-for-privsep
I can't see any evidence that anyone else uses our nova-idmapshift
binary, and it adds a lot of complexity (flags we never call for
example). Move the code we do actually use into the privsep
directory and simplify our calls to it. Remove the extra binary
from our install and documentation.
Change-Id: Ibce28d20d166da154833376cf51f1877b829925e
blueprint: hurrah-for-privsep
Add a new filesytem mounting helper in privsep, and then start
moving things across to it. This currently implements mount and
unmount. We get to cleanup some rmdir calls while we're at it
which is nice as well.
I've added an upgrade note mentioning that we no longer ignore
the value of stderr from mount calls, as requesed in code review.
Change-Id: Ib5e585fa4bfb99617cd3ca983674114d323a3cce
blueprint: hurrah-for-privsep
Instead of dd. The interesting bit here is that the read is
non-blocking, but apart from that its pretty cut and dried.
Change-Id: I14d2a15e4f5efbed605e581781873494f23393a4
The same pattern as the others, but with an added security concern.
Co-Authored-By: Evgeny Antyshev <eantyshev@virtuozzo.com>
Closes-Bug: #1717533
Change-Id: I1ac3a0ea4756ec68884866435c3da69171bbeb13
blueprint: hurrah-for-privsep
As discussed at the PTG, we're going to use one big context for
ease of management.
Change-Id: I951abd402736735730e0868f31b85b1817055b2f
blueprint: hurrah-for-privsep
As discussed at the PTG, squash the dac_admin privsep context into
the sysadmin context.
Change-Id: I10142be4baa404835fabebd50f7f976ca6ec402e
blueprint: hurrah-for-privsep
A nice simple example of how to move things to privsep (and the new set
of helpers). In a few of these cases I think a more complicated re-write
is actually required, but I've put TODOs there are will do those in a
followup patch.
Change-Id: Ibb6ef001e3f2add459b0e37dfbd9f51c9eff2eb7
I don't particularly care about this use case (although the localfs
code should perhaps go away), but it was a nice contained example
of a privsep user which wasn't just calling a command line.
This patch also starts to layout what an API to the privsep'd code
might look like. For now its modelled on python's os module, because
that's where all the operations we perform are coming from.
The rootwrap configuration is cleaned up as we remove users.
Co-Authored-By: Tony Breeds <tony@bakeyournoodle.com>
Change-Id: I911cc51a226d6af29d63a7a2c69253de870073e9
The form for rootwrap.d files is to include a comment before the filter
with the filename (and preferably) the command the filter matches.
This change ensures that these file comments refer to a valid file in the
tree. They can be checked with something like:
for i in `awk '/^#.*py *:/ {print $2}' etc/nova/rootwrap.d/compute.filters \
| sort -u` ; do
ls ${i/:}
done
* I13c701c390784fa1f7809705741abb46e40973be renamed
.../libvirt/connection.py to .../libvirt/drver.py
* I400db60fcc29c2d5e2d3b9dabc055649138468eb switched to os-brick and
removed nova/storage/linuxscsi.py
* I5fc2425d2c25076ea87686b2e41be35f66ebb923 moved .../libvirt/volume.py
into .../libvirt/volume/
* Update one comment to make the awk script above work.
* Add comments as 'chown' and 'tee' are used in nova/virt/libvirt/
Change-Id: I3d89830e4770a7cf88389fac5a2a684554a29bc5
If someone chooses to run under uwsgi/apache instead of eventlet, the
basic logging of requests goes away (as that was an eventlet.wsgi
function). This is a critical piece of information for understanding
how services are working, and we need to retain it under uwsgi/apache.
This creates a new request_log middleware, inspired by the one in
placement, to provide that functionality. This includes all the same
information as before: http method, uri, status, content length, time
for the request. It also includes the microversion the request was
processed as, "-" if no microversion.
The middleware does not emit anything if it detects that it's running
under eventlet, to prevent duplicate log messages.
Release notes provided as this will be a manual transition for folks
as it's an api-paste.ini change.
Change-Id: I3a597b06d3501c765e2d7805c6c1375d6f4e40db
The Scality volume driver was removed from Cinder in Ocata:
a931f9db79554630d8d71fcff1334bb4e37cb398
It's unmaintained and therefore no longer supported, and won't
even work with recent Cinder, so let's remove it.
Change-Id: I0cfa0c4b3d7138f989225a44bdedfa716a035a14
This change restores the rootwrap filter for cryptsetup that was
recently removed by I37ffc90c0bd57029fced251b5cfd7cd4318a0292 from
compute.filters, as it is still needed by dmcrypt. Without the rootwrap
filter, `cryptsetup` is not authorized to run with root permissions.
Change-Id: I5fe3e5d5e5a9694d0dbe5b59248e5eaf89858c62
Closes-Bug: #1688166
This change drops the encryptor classes and supporting code from the
codebase in favor of the classes provided by os-brick. This is made
possible by the following os-brick change that introduced new encryption
provider constants during Ocata :
Ic155bd29d46059832cce970bf60375e7e472eca6
Thanks to the following bugfix also released as part of 1.11.0 for Ocata
the constants present in os-brick also support the use of the deprecated
legacy class paths from Nova, for example
nova.volume.encryptors.luks.LuksEncryptor, while using the os-brick
provided classes :
I3ec6e3fe919bc03d158da04a18fb8b651002ed52
Implements: blueprint switch-to-os-brick-encryptor-classes
Change-Id: I37ffc90c0bd57029fced251b5cfd7cd4318a0292
Depends-On: Iae12605dc7d0607e78020a24b5b8801606c2f169
