This change includes unit tests for the ISO
format inspector using mkisofs to generate
the iso files.
A test for stashing qcow content in the system_area
of an iso file is also included.
This change modifies format_inspector.detect_file_format
to evaluate all inspectors until they are complete and
raise an InvalidDiskInfo exception if multiple formats
match.
Related-Bug: #2059809
Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712
This change adds a reproducer for the regression in iso
file support when
workarounds.disable_deep_image_inspection = False
Change-Id: I56d8b9980b4871941ba5de91e60a7df6a40106a8
This commit is a direct port of the format inspector
unit tests from glance as of commit
0d8e79b713bc31a78f0f4eac14ee594ca8520999
the only changes to the test are as follows
"from glance.common import format_inspector" was updated to
"from nova.image import format_inspector"
"from glance.tests import utils as test_utils"
was replaced with "from nova import test"
"test_utils.BaseTestCase" was replaced with "test.NoDBTestCase"
"glance-unittest-formatinspector-" was replaced with
"nova-unittest-formatinspector-"
This makes the test funtional in nova.
TestFormatInspectors requries qemu-img to be installed on the
host which would be a new depency for executing unit tests.
to avoid that we skip TestFormatInspectors if qemu-img
is not installed.
TestFormatInspectorInfra and TestFormatInspectorsTargeted
do not have a qemu-img dependency so
no changes to the test assertions were required.
Change-Id: Ia34203f246f0bc574e11476287dfb33fda7954fe
A Request-Too-Long error can be raised in case of too many aggregates
are attached to a host. The fix is to split the requests.
The request is executed by host via a periodic task. Result is set in
cache so no performances impact is expected.
The change introduce a new compute configuration option
`compute.sharing_providers_max_uuids_per_request = 200`.
Closes-bug: #2068893
Change-Id: I1355c0813cb99067ead60f9835938566289beb20
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
This restores the vmdk_allowed_types checking in create_image()
that was unintentionally lost by tightening the
qemu-type-matches-glance code in the fetch patch recently. Since we
are still detecting the format of base images without metadata, we
would have treated a vmdk file that claims to be raw as raw in fetch,
but then read it like a vmdk once it was used as a base image for
something else.
Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
Related-Bug: #2059809
There is an additional way we can be fooled into using a qcow2 file
with a data-file, which is uploading it as raw to glance and then
booting an instance from it. Because when we go to create the
ephemeral disk from a cached base image, we've lost the information
about the original source's format, we probe the image's file type
without a strict format specified. If a qcow2 file is listed in
glance as a raw, we won't notice it until it is too late.
This brings over another piece of code (proposed against) glance's
format inspector which provides a safe format detection routine. This
patch uses that to detect the format of and run a safety check on the
base image each time we go to use it to create an ephemeral disk
image from it.
This also detects QED files and always marks them as unsafe as we do
not support that format at all. Since we could be fooled into
downloading one and passing it to qemu-img if we don't recognize it,
we need to detect and reject it as unsafe.
Change-Id: I4881c8cbceb30c1ff2d2b859c554e0d02043f1f5
It has been asserted that we should not be calling qemu-img info
on untrusted files. That means we need to know if they have a
backing_file, data_file or other unsafe configuration *before* we use
qemu-img to probe or convert them.
This grafts glance's format_inspector module into nova/images so we
can use it to check the file early for safety. The expectation is that
this will be moved to oslo.utils (or something) later and thus we will
just delete the file from nova and change our import when that happens.
NOTE: This includes whitespace changes from the glance version of
format_inspector.py because of autopep8 demands.
Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
Closes-Bug: #2059809
Tempest currently defaults to disk_formats[0] for images it creates,
which is 'ami'. However, it's actually using a qcow2 disk image by
default, which means we're lying to glance when we create those.
Change-Id: I737e9aa51c268a387f1eed24cf717618d057d747
The 'rxtx_factor' is a float, not an integer, and should allow values <
1 so long as they are > 0. Correct this.
Change-Id: I3567f13d7a1bb6f42755e3f5739fa9ecd827aa49
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
There's no reason to inspect the API version requested (and reject
things if unset) if we don't have version constraints on the API
resource. This lets us add schema decorators to unversioned API
resources (i.e. the "versions" API at '/').
Change-Id: Ib4d4585e3e4664eba8cd3daa47b4349eb2bfed74
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
For APIs and resource actions that have been wholly removed and now
return a HTTP 4xx request regardless of microversion.
Change-Id: Ie3e90f4628dab4cc3e5983e0e8d8799dcf0465d8
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
OpenAPI 3.1 is a superset of JSON Schema Draft 2020-12. As a result, we
wish to migrate our current schemas to this. There are a couple of
issues to address:
- 'exclusiveMinimum' is now an integer and allows you to define an
minimum for an exclusive range, to complement 'minimum' which is used
for an inclusive range. We can drop it and use 'minimum' with a larger
number (since draft 6 [1]).
- 'integer' types can now accept a fractional part but only if it's zero
(i.e. '1.0' is permissible, '1.1' is not) (since draft 6 [1]).
- 'items' has been replaced with 'prefixItems' for describing the format
of an array item (since draft 2020-12 [2])
[1] https://json-schema.org/draft-06/json-schema-release-notes
[2] https://json-schema.org/draft/2020-12/release-notes
Change-Id: I1486701786960eef95c5c42674bff1b2d7d686e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Validate the validation by validating our schemas against the JSON
Schema meta schema. This is an important first step in getting us
of JSON Schema Draft 4 and onto Draft 2019-09, which OpenAPI is a
superset of.
Change-Id: I3b5a05aa0aa058e92c6927c9e3bee3cdd4477f8f
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
None of these should affect us at runtime but they are not technically
valid against JSON Schema Draft 4.
Change-Id: Iaae8998a34ff812b9f76a0cd1fa9be4555c89b98
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The idea here is to test that we are doing schema validation against
all routes that accept a request body (i.e. POST, PUT, PATCH). We can
use these schemas down the line to generate OpenAPI docs like all the
cool kids do.
Change-Id: Icdf20e7e5b38c9f5324eac39ec1d6327609bf6d4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This provides us a path to generate OpenAPI schemas for the nova API.
Thankfully most of the API documentation is pretty darn simple. Note
that we need to fix a lot of unit tests so that they pass correct bodies
to the controller methods under test. Also note that the
'reset_server_state' module in 'nova.api.openstack.compute.schemas' is
renamed to 'admin_actions' to match the name of the module containing
the controller itself, as is the case for all other schemas.
Change-Id: I39098fb621b120589ed6a923ce64207ee26e91aa
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These are deprecated but there's value in having a proper - if loose -
schema in place for API documentation purposes. Also, doing things this
way allows us to remove a whole load of hand-rolled stuff.
Change-Id: I4106cfa2a09d135f12892ed6d1f42f4151dc72e4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This change adds a retry_if_busy decorator
to the read_sys and write_sys functions in the filesystem
module that will retry reads and writes up to 5 times with
an linear backoff.
This allows nova to tolerate short periods of time where
sysfs retruns device busy. If the reties are exausted
and offlineing a core fails a warning is log and the failure is
ignored. onling a core is always treated as a hard error if
retries are exausted.
Closes-Bug: #2065927
Change-Id: I2a6a9f243cb403167620405e167a8dd2bbf3fa79
This change increases the swap in the emulation job form
1G to 8G
This change updates the default cirrors image form 0.5.2
to 0.5.3 to avoid know kernel bugs
this change reduces the tb_cache_size: to 128
The tempest concurrency is reduced to 4 to avoid
no valid host error in the resize tests
Change-Id: Ic1dde3d54f5ca12408ef53218773a27d55760705
This address review feedback on change
I7e1d10e66a260efd0a3f2d6522aeb246c7582178 to add some clarifying text
to the docs and release note.
Related to blueprint persistent-mdevs
Change-Id: I472552c64cc2c2ce06896158664faac0199d90bd
--before argument is currently described in ambiguous way: it
is not actually used to filter entries ARCHIVED before specified
date. Instead, it compares provided data with "deleted_at" value
for most rows and "updated_at" or "created_at" for remaining ones.
Since we already talk about time of deletion when describing
--before argument of "nova-manage db archive_deleted_rows" rows,
it make sense to not provide extra details here as well.
Change-Id: Ib5940e88a52dc8d32303e27237e567c3481fc3dc