2531 Commits

Author SHA1 Message Date
Zuul
b7721a9493 Merge "Remove non RE2 job config" into stable/2024.1 2024-11-09 19:50:11 +00:00
Dmitry Tantsur
7e9dd7e9f1 Warn when the provided checksum algorithm does not match the detected
I have a case where a user provided the checksum URL with SHA256
checksums, while Metal3 defaulted os_hash_algo to "md5". We're going
to change the Metal3 defaults in the next API version, but for now let
us issue a clear warning in such case.

Closes-Bug: #2085331
Change-Id: Ie4e62a378dc4a2089944f4302df3a8671b7c960f
(cherry picked from commit d8d32d93bd9fd0a9e759f7babe5db6fa804626de)
(cherry picked from commit aa01777ca8e4de7822bd8263ad85a0669bafbf55)
2024-10-23 13:31:57 +00:00
Steve Baker
b39d9d67a6 Remove non RE2 job config
This causees a linting failure, and the equivalent of this rule was
removed later in I64909aa932635b729cc85717dc241ae31798b558

Change-Id: Id8e1a0901090f062ff36101f07acafe01a98a67b
2024-10-18 11:51:25 +13:00
Dmitry Tantsur
349c9d39fe Trivial: fix variable in formatting
Change-Id: I6af5e6d2c4781c24345d456cec4d77c364ae2da5
(cherry picked from commit d7b2dcf66f16a1295613ad5fc1abb8e6c2ad66ee)
2024-09-24 10:41:04 +00:00
Jay Faulkner
06fe5ff178 Inspect non-raw images for safety
This is a backport of two changes merged together to facilitate
backporting:

The first is a refactor of disk utilities:

Import disk_{utils,partitioner} from ironic-lib

With the iscsi deploy long gone, these modules are only used in IPA and
in fact represent a large part of its critical logic. Having them
separately sometimes makes fixing issues tricky if an interface of
a function needs changing.

This change imports the code mostly as it is, just removing run_as_root and
a deprecated function, as well as moving configuration options to config.py.

Also migrates one relevant function from ironic_lib.utils.

The second is the fix for the security issue:

Inspect non-raw images for safety

When IPA gets a non-raw image, it performs an on-the-fly conversion
using qemu-img convert, as well as running qemu-img frequently to get
basic information about the image before validating it.

Now, we ensure that before any qemu-img calls are made, that we have
inspected the image for safety and pass through the detected format.

If given a disk_format=raw image and image streaming is enabled
(default), we retain the existing behavior of not inspecting it in
any way and streaming it bit-perfect to the device. In this case, we
never use qemu-based tools on the image at all.

If given a disk_format=raw image and image streaming is disabled, this
change fixes a bug where the image may have been converted if it was not
actually raw in the first place. We now stream these bit-perfect to the
device.

Adds two config options:
- [DEFAULT]/disable_deep_image_inspection, which can be set to "True" in
  order to disable all security features. Do not do this.
- [DEFAULT]/permitted_image_formats, default raw,qcow2, for image types
  IPA should accept.

Both of these configuration options are wired up to be set by the lookup
data returned by Ironic at lookup time.

This uses a image format inspection module imported from Nova; this
inspector will eventually live in oslo.utils, at which point we'll
migrate our usage of the inspector to it.

Closes-Bug: #2071740
Co-Authored-By: Dmitry Tantsur <dtantsur@protonmail.com>
Change-Id: I5254b80717cb5a7f9084e3eff32a00b968f987b7
9.11.1
2024-09-04 09:14:51 -07:00
Jay Faulkner
46744b1a74 Remove and disable examples job
It is not a valid check to import the hwm without it being loaded into
IPA first, as objects such as the configuration object won't be loaded
yet.

Change-Id: Icf20e71e8061bb886885c1b2e29bd13ccac37ade
(cherry picked from commit ba5c1bfe2abc7ee5891d7ec56489c15e63e9a413)
2024-09-04 09:14:51 -07:00
Jay Faulkner
e9c0578c7d Call evaluate_hardware_support exactly once per hwm
Fixes an issue where we could call evaluate_hardware_support multiple
times each run. Now, instead, we cache the values and use the cache
where needed.

Adds unit test coverage for get_managers and the new method.
Fixes issue where we were caching hardware managers between unit tests.

Closes-bug: 2066308
Change-Id: Iebc5b6d2440bfc9f23daa322493379bbe69e84d0
(cherry picked from commit c39517b04479df1aeaf96402840238236870fa74)
2024-05-22 09:00:02 -07:00
Julia Kreger
c1581df331 Unmount config drives
If this seems like deja vu, that is because it is. We had this
very same issue with the original CoreOS ramdisk. Since we don't
control the whole OS of the ramdisk, it only made sense to teach
the agent to umount the folder.

The folder is referenced already, and the agent does have safeguards
in place, but unfortunately this issue led to a rebuild breaking where
cloud-init, glean, and the agent were all trying do the right thing
as they thought, and there were just multiple /mnt/config folders
present in the OS. These are separate issues we also need to try and
remedy.

What happens is when the device is locked via a mount, the partition
table is never updated to the running OS as the mount creates a lock.
So the agent ends up thinking, in the case of a rebuild, that everything
including creating a configuration drive on that device has been
successful, but when you reboot, there is no partition table entry
for the new partition as the change was not successfully written.
This state prevented the workload from rebooting properly.
This change eliminates that possibility moving forward by attempting
to ensure that the cloud configuration folder is no longer mounted.

Change-Id: I4399dd0934361003cca9ff95a7e3e3ae9bba3dab
(cherry picked from commit 6ac3f350c049d9dc62c941702c20e67eff2c20a1)
2024-05-01 14:40:32 +00:00
Riccardo Pittau
f2b8a19cea Update ironic-lib min required version
The addition of the qemu-img module  that is used in ipa was
introduced in ironic-lib 6.0.0

Change-Id: I65b283d9abcfe2aa961baf9f686391c8dcc71d6b
9.11.0
2024-03-19 16:18:50 +00:00
ccf65889d8 Update TOX_CONSTRAINTS_FILE for stable/2024.1
Update the URL to the upper-constraints file to point to the redirect
rule on releases.openstack.org so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/2024.1 branch, tests will
continue to use the upper-constraints list on master.

Change-Id: I134abf6cff0579f501873df5b0a4de0b82c612fc
2024-03-19 11:10:42 +00:00
290d0c736f Update .gitreview for stable/2024.1
Change-Id: Iad4fb1447568a12e386cf63d5215b1fceb3e407c
2024-03-19 11:10:41 +00:00
Zuul
ee8340f2cb Merge "Update regex to detect closed branch" 9.10.0 2024-03-18 11:07:10 +00:00
Zuul
e28b3e72f7 Merge "Use assert_not_called" 2024-03-15 17:30:21 +00:00
Zuul
815e1f462f Merge "reno: Update master for unmaintained/victoria" 2024-03-14 12:05:39 +00:00
Zuul
aa76962b4e Merge "reno: Update master for unmaintained/wallaby" 2024-03-14 12:00:33 +00:00
38ba0d8508 reno: Update master for unmaintained/xena
Update the xena release notes configuration to build from
unmaintained/xena.

Change-Id: I3bbef10b65dc43596a59eaca5d792f5e451d5d4c
2024-03-14 11:27:13 +00:00
5a017ea84a reno: Update master for unmaintained/wallaby
Update the wallaby release notes configuration to build from
unmaintained/wallaby.

Change-Id: Iaf279482847d781d7d338c4923a672a5e9337332
2024-03-14 11:22:41 +00:00
6ebaf277a6 reno: Update master for unmaintained/victoria
Update the victoria release notes configuration to build from
unmaintained/victoria.

Change-Id: I00a9bcb8ee6d5160d2598fbecb8e585885212df7
2024-03-14 11:18:06 +00:00
Takashi Kajinami
bffa88acb8 Update regex to detect closed branch
... based on the change made in reno recently[1].

Also the overall regex is updated to be more consistent with the regex
used in ironic.

[1] https://review.opendev.org/c/openstack/reno/+/910547

Change-Id: I362de82fb5478b846df7a343da02a359f5f7dece
2024-03-13 19:40:40 +09:00
Zuul
9b760755ca Merge "Force constraints when installing a package during tox test" 2024-03-13 08:23:06 +00:00
Riccardo Pittau
95b3ed3fed Fix unit tests after ironic-lib changes
Updating tests after change [1] and [2] in ironic-lib.

[1] ae53e8e4b3
[2] 7644196e7d

Change-Id: I880b4f82beb117d8812e60c13040e19476cec32b
2024-03-12 09:13:14 +01:00
Thomas Goirand
ca6ff4706b Use assert_not_called
IPA still has 3 occurences of not_called() which are failing for me
when building the Ironic Debian package in Debian Unstable (ie: with
Python 3.12).

This patch uses assert_not_called() instead of not_called(), fixing
the problem.

Change-Id: I8bd27fa706b298b28ef5bef405134a2c9803d757
2024-02-26 11:57:10 +01:00
Riccardo Pittau
d4554e4ab5 Force constraints when installing a package during tox test
The tox deps option grants installation of single dependencies and
requirements, optionally pinned using constraints, before installing
a package, therefore not granting installation of the correct
constraint during the package installation.
To fix that tox 4.4.0 has introduced the constrain_package_deps
option [1]

[1] https://tox.wiki/en/4.12.1/faq.html#using-constraint-files

Change-Id: I770e55db5d6f53174c490749d27830f9209e98cc
2024-02-12 14:59:39 +01:00
Zuul
df7eccd7f1 Merge "Trivial: avoid deprecated utcnow" 2024-02-08 14:43:41 +00:00
Zuul
6d35c1e949 Merge "Make inspection URL optional if the collectors are provided" 2024-02-07 23:06:34 +00:00
614532d2a2 reno: Update master for unmaintained/yoga
Update the yoga release notes configuration to build from
unmaintained/yoga.

Change-Id: I0c5ab4348bd293ce77b04180247773412edbe179
2024-02-06 15:03:51 +00:00
Zuul
359ac636f0 Merge "Drop usage of run_as_root" 2024-01-31 16:29:06 +00:00
Dmitry Tantsur
8877e1f319
Trivial: avoid deprecated utcnow
Change-Id: I5dbe3c2be36e23e749fbeebbc448d413d276b401
2024-01-31 10:09:13 +01:00
Zuul
c3e3cf2aef Merge "Also retry inspection on HTTP CONFLICT" 9.9.0 2024-01-29 08:50:26 +00:00
Zuul
204381bbd9 Merge "[codespell] Adding git-blame-ignore-revs to clear codespell changes" 2024-01-27 00:17:47 +00:00
Dmitry Tantsur
0010f5c11a
Also retry inspection on HTTP CONFLICT
The new implementation can return it when unable to lock the node.

Other possible errors are 400 and 404 (should not be retried), as well as
5xx (already retried).

Change-Id: I74c2f54a624dc47e8e2d1e67ae4c6a6078e01d2f
2024-01-26 16:21:24 +01:00
Sharpz7
0ef3397bec [codespell] Adding git-blame-ignore-revs to clear codespell changes
Change-Id: Iac9895c78fe68b52521074065c5e6d0b0c15491c
2024-01-25 01:49:11 +00:00
Dmitry Tantsur
9f849472ca
Drop usage of run_as_root
IPA can only be run as root and does not use rootwrap. We need to
eventually remove support for rootwrap from ironic-lib.

Change-Id: Iffd5cae5e3dc8637bc6dd10b3bcc9fe33932b8cf
2024-01-23 14:23:23 +01:00
Zuul
1e107bd625 Merge "Add support for reporting CPU socket number" 2024-01-22 11:52:06 +00:00
Kaifeng Wang
9cafe76225 Add support for reporting CPU socket number
IPA reports a few cpu fields including cores, arch, flags etc.
There is a need that user wants to utilize the physical number in
a baremetal since cores are just a logical representation of the
compute resource.
The socket number is more suitable for the quota control in some
use cases.

Change-Id: I94be86d6b12a3a7e7ca1041d948427a073412a31
2024-01-19 21:24:37 +00:00
Sharpz7
b928b6601c [Codespell] Adding tox target to CI (Non-Voting)
Adding CI for previous codespell tox change

Related Bug: https://bugs.launchpad.net/ironic-python-agent/+bug/2047654

Change-Id: I02f7973c0e6799f7d7b44978009dfebbce3877a4
2024-01-17 17:15:33 +00:00
Zuul
be9477179b Merge "Support several API and Inspector URLs" 2024-01-16 19:17:36 +00:00
Zuul
3e066e3f3e Merge "Add missing headers to the inspection callback" 2024-01-16 19:17:35 +00:00
likui
dcf12d1b58 Update python classifier in setup.cfg
As per the current release tested runtime, we test
till python 3.11 so updating the same in python
classifier in setup.cfg

Change-Id: I699e08c268040d387b91ccca4e6505184d3d1b59
2024-01-15 14:17:44 +08:00
Zuul
64081ad36b Merge "Remove deprecated pbr options" 2024-01-10 09:32:35 +00:00
Zuul
dc33a97221 Merge "Remove unnecessary egg_info options" 2024-01-10 09:04:36 +00:00
Dmitry Tantsur
6cd36a750f
Make inspection URL optional if the collectors are provided
With the new in-band inspection, we can derive the callback URL from
the Ironic URL, there is no need to duplicate it. This change uses
the presence of collectors as a sign to run inspection.

The previous approach of setting an inspection URL, with or without
explicitly setting collectors, still works for compatibility with
ironic-inspector.

Change-Id: Ie4279ee6d2995c9686f1dcdef1d6e5dc1dd20871
2024-01-10 08:55:42 +01:00
Dmitry Tantsur
0d4ae976c2
Support several API and Inspector URLs
Allows nodes with a single IP stack to be deployed from a dual-stack
Ironic.

Detecting advertised address and usable Ironic URLs are done completely
independently which does open some space for a misconfiguration. I hope
it's not likely in the reality, especially since this feature is
targetting advanced standalone users.

Change-Id: Ifa506c58caebe00b37167d329b81c166cdb323f2
Closes-Bug: #2045548
2024-01-09 16:43:23 +01:00
Dmitry Tantsur
2bb74523ae
Add missing headers to the inspection callback
Somehow, it has worked correctly for years, but now I've discovered that
the new inspection is (no longer?) tolerant to the missing header.

While here, copy all headers from the heartbeat code.

Change-Id: I9e5c609eb4435e520bc225dea08aedfdf169744b
2024-01-09 16:38:46 +01:00
Zuul
a22d1fc411 Merge "Add tox target and configuration for codespell" 2024-01-08 17:22:04 +00:00
Zuul
d298e06b49 Merge "[codespell] Fix spelling issues in IPA" 2024-01-08 17:22:02 +00:00
Zuul
f1a4aeb29a Merge "Update to latest pep8/code style versions" 2024-01-03 14:57:53 +00:00
Zuul
7422a27de4 Merge "Reformat and update the section on injecting root credentials" 2023-12-29 22:20:49 +00:00
Jay Faulkner
dcaed43ef9 Update to latest pep8/code style versions
Update various linting programs to their latest version, and fix any
issues created by the update.

Change-Id: I014c846560663a76a1663b568ef48659d0ab6d4d
2023-12-28 14:19:27 -08:00
Jay Faulkner
4a1acae5bc Add tox target and configuration for codespell
Adds a tox target for codespell, `tox -e codespell`. Can optionally
be run as `tox -e codespell -- -w` to get automatic spelling fixes
applied where appropriate. Adds small amounts of configuration to
setup.cfg, including an ignore list of words.

Related-bug: #2047654
Change-Id: I98203b02a9c6b6fc36edd6b4bbcc7c92a634da8b
2023-12-28 11:43:10 -08:00