2433 Commits

Author SHA1 Message Date
Julia Kreger
b68a4c8a92 minor: fix release notes file path
Change-Id: I458d88bf14b55253179488cb771ae42e7b8c84d7
9.6.0
2023-08-07 12:57:34 -07:00
likui
c274869756 Add python3.10 support in testing runtime
In 2023.2 cycle testing runtime, project started adding python 3.10

[1] https://governance.openstack.org/tc/reference/runtimes/2023.2.html

Change-Id: I6e3eb0c9dec4c48e1bf1c7c53b8c177775ec91eb
2023-07-31 15:26:15 +08:00
Zuul
e493cad02c Merge "Log the number of bytes downloaded" 2023-07-27 21:39:12 +00:00
Julia Kreger
c65ad42ff1 Log the number of bytes downloaded
When troubleshooting download issues, which may present
as checksum validation failures, it is difficult to understand
if the *entire* file was downloaded due to the way HTTP works.

In that, a download may start with a successful result code,
and the content is streamed out until the socket is closed.

But with HTTP there is no way to know if that socket closed
prematurely and the original server size is *also* an optional
field, so just log the size we got to so we don't drive the
humans [more-]insane.

Also now logs the (optional) content-length field if
supplied by the server.

Change-Id: Id71b167f4e330d54b9afddf95f1a2ef9e40398bf
2023-07-19 16:20:40 +00:00
Zuul
0fb7fec56e Merge "Allow md5 to be disabled from the conductor" 2023-07-12 03:53:14 +00:00
Zuul
119981a818 Merge "Fix nvidia hardware manager url parser to permit https" 2023-06-26 10:11:55 +00:00
Zuul
bb156aad6c Merge "Fix Bandit errors" 2023-06-26 09:25:09 +00:00
Julia Kreger
b83678c968 Fix nvidia hardware manager url parser to permit https
Change-Id: I9a10e543d3256ceaa78c6fbdb01fc0d88c0ee6e6
2023-06-06 15:35:16 +00:00
Julia Kreger
78c1343a54 Fix Bandit errors
Bandit 1.7.5 released with a timeout check for all requests and
urllib calls.

Fixed those.

In the process, then exposed a bandit b310 issue, which was already
covered by the code, but explicitly marked it as such.

Also, enables bandit checks to be voting for CI..

Change-Id: If0e87790191f5f3648366d571e1d85dd7393a548
2023-06-06 08:34:55 -07:00
Zuul
4845fd04ba Merge "Follow-up Add documentation for MellanoxDeviceHardwareManager" 9.5.0 2023-05-25 15:03:19 +00:00
Julia Kreger
e6fd7e753e Allow md5 to be disabled from the conductor
Also fixes my use of set_override, as it is not on the actual
config object. You'd think I'd remember that, since I've done
that before...

Change-Id: I4b578c4319354001cbbd3b3856af96b30fd25555
2023-05-25 07:59:07 -07:00
waleedm
406c844aac Follow-up Add documentation for MellanoxDeviceHardwareManager
Add a follow-up documentation for
"update NVIDIA NIC firmware images and settings by ironic-python-agent"
Icfaffd7c58c3c73c3fa28cfc2a6c954d2c93c16e

Change-Id: I481cdd622f360cbba3312c6f3d4af45383bb7e1b
2023-05-25 10:55:11 +00:00
Jay Faulkner
6098747ec5 Ironic (and IPA) use launchpad now
Correct links to point to launchpad bug tracker, correct docs config

Change-Id: I5d46af2a9d94f3b2e05e4f937e0619a89fe04d4c
2023-05-17 15:38:57 -07:00
Zuul
141c5ff1c3 Merge "Add support for CentOS SUM files" 2023-05-09 09:03:25 +00:00
Zuul
03e88b579e Merge "Revert disabling MD5 checksums" 2023-05-05 08:44:37 +00:00
Zuul
44d9c2219f Merge "Add network interface speed to the inventory" 2023-05-04 09:04:30 +00:00
Dmitry Tantsur
c1c5537ba2 Revert disabling MD5 checksums
This was a significant breaking change that was landed despite explicit
disagreement by some community members (myself included). It has already
resulted in an accidental Ironic CI breakage, has broken Bifrost and has
a potential of breaking Metal3. In case of Metal3, MD5 support is a part
of its public API.

While MD5 is a potential security hazard, I don't see the need to hurry
this change without giving the community time to prepare. This change
reverts the new option md5_enabled to True.

Change-Id: I32b291ea162e8eb22429712c15cb5b225a6daafd
2023-05-04 09:26:10 +02:00
Harald Jensås
e7a048ecbe
Add support for CentOS SUM files
The CentOS Stream SUM files uses format:
  # FILENAME: <size> bytes
  ALGORITHM (FILENAME) = CHECKSUM

Compared to the more common format:
  CHECKSUM  *FILE_A
  CHECKSUM  FILE_B

Use regular expressions to check for filename both
in the middle with parentheses and at the end.
Similarly look for valid checksums at beginning or
end of line. Also look for know checsum patterns in
case file only contain the checksum iteself.

Change-Id: I9e49c1a6c66e51a7b884485f0bcaf7f1802bda33
2023-05-03 21:31:23 +02:00
Dmitry Tantsur
9ed232e77e Add network interface speed to the inventory
This is another fact that Metal3's baremetal-operator is currently
consuming from extra-hardware.

Change-Id: I2ec9d5e9369f5508e7583a4e13c2083f5c8b28ba
2023-05-03 12:20:35 +02:00
Julia Kreger
c05fdf790c Fix checksum validation logic
The checksum validation logic, which was updated early on in the
whole process of deprecating md5, didn't account for a URL *or* a
longer checksum (i.e. sha256/sha512) which was decided while the
overall approach was being decided.

Fixes the logic, and adds additional tests.

Change-Id: Ic4053776e131fc02ace295a1e69e9f9faab47f42
2023-05-02 17:24:57 -07:00
Zuul
f37ea85a27 Merge "Disable MD5 image checksums" 2023-05-02 06:41:25 +00:00
Zuul
3cd8c294fb Merge "Deprecate LLDP in inventory in favour of a new collector" 2023-04-27 12:05:11 +00:00
Zuul
33e3bae28b Merge "Fix UTF-16 result handling for efibootmgr" 2023-04-27 00:49:35 +00:00
Dmitry Tantsur
3e05a03f7c Deprecate LLDP in inventory in favour of a new collector
Binary LLDP data is bloating inventory causing us to disable its collection
by default. For other similar low-level information, such as PCI devices
or DMI data, we already use inspection collectors instead. Now that the
inventory format is shared with out-of-band inspection, having LLDP
there makes even less sense.

This change adds a new collector ``lldp`` to replace the now-deprecated
inventory field.

Change-Id: I56be06a7d1db28407e1128c198c12bea0809d3a3
2023-04-26 19:33:51 +00:00
Julia Kreger
32df26a22a Disable MD5 image checksums
MD5 image checksums have long been supersceeded by the use of a
``os_hash_algo`` and ``os_hash_value`` field as part of the
properties of an image.

In the process of doing this, we determined that checksum via
URL usage was non-trivial and determined that an appropriate
path was to allow the checksum type to be determined as needed.

Change-Id: I26ba8f8c37d663096f558e83028ff463d31bd4e6
2023-04-24 16:54:42 -07:00
Jay Faulkner
d7234c2be0 Upgrade to latest hacking - v6
No code changes needed to comply with newest flake.

Change-Id: I256397efe0fbb3e307d808b0eda2e4d72d83f9b0
2023-04-21 12:19:02 -07:00
Julia Kreger
76accfb880 Fix UTF-16 result handling for efibootmgr
The tl;dr is that UEFI NVRAM is in encoded
in UTF-16, and when we run the efibootmgr command,
we can get unicode characters back.

Except we previously were forcing everything to be
treated as UTF-8 due to the way oslo.concurrency's
processutils module works.

This could be observed with UTF character 0x00FF
which raises up a nice exception when we try to
decode it.

Anyhow! while fixing handling of this, we discovered
we could get basically the cruft out of the NVRAM,
by getting what was most likey a truncated string
out of our own test VMs. As such, we need to also
permit decoding to be tollerant of failures.
This could be binary data or as simple as flipped
bits which get interpretted invalid characters.
As such, we have introduced such data into one of our
tests involving UEFI record de-duplication.

Closes-Bug: 2015602
Change-Id: I006535bf124379ed65443c7b283bc99ecc95568b
2023-04-17 09:14:24 -07:00
Dmitry Tantsur
0304c73c0e Report system firmware information in the inventory
Change-Id: I5b6ceb9cdcf4baa97a6f0482d1030d14f3f2ecff
2023-03-31 14:28:32 +02:00
Dmitry Tantsur
2ddb693491 Trivial: formatting issue in the inventory docs
Double ticks don't work if followed by a symbol without space.

Change-Id: Ia455650b5e601dadb2b0ab91f71e1d9286d26071
2023-03-30 13:33:39 +02:00
Arne Wiebalck
b32f6c6d94 [Trivial] Fix typo in efi_utils
Change-Id: I692e045e6bc8683038a2e85a6a132687d2b30f18
2023-03-15 14:25:42 +01:00
9f09b885bd Update master for stable/2023.1
Add file to the reno documentation build to show release notes for
stable/2023.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.1.

Sem-Ver: feature
Change-Id: Id58fd751e6ed8ed3d478a78a6895cad75667c9b1
2023-03-09 14:09:43 +00:00
Zuul
088610844a Merge "update NVIDIA NIC firmware images and settings by ironic-python-agent" 9.4.0 2023-01-31 19:35:53 +00:00
Zuul
c12135911a Merge "Make logs collection a hardware manager call" 9.3.0 2023-01-26 16:26:42 +00:00
Zuul
7f687a1734 Merge "Readd usedevelop true to tox.ini" 2023-01-26 09:28:56 +00:00
Jay Faulkner
f8fc7e52f3 Make reno ignore bugfix eol tags
Reno was assuming all tags ending in -eol represented an old, EOL'd
stable branch. That's not true for Ironic projects which have bugfix
branches. Update the regexp to exclude those branches.

Co-Authored-By: Adam McArthur <adam@mcaq.me>
Change-Id: I265969ab40a98a02962c2fc8460b6519ab576f99
2023-01-25 13:18:01 -08:00
Dmitry Tantsur
c26f498f49 Make logs collection a hardware manager call
This allows hardware managers to collect additional logs.

Change-Id: If082b921d4bf71c4cc41a5a72db6995b08637374
2023-01-25 15:17:06 +01:00
Riccardo Pittau
f2b4ac8751 Readd usedevelop true to tox.ini
This avoid using sdist, which is the wanted behavior

Change-Id: Ibfb867ee0ff8274130a19538aa7ff9b9296dc4e1
2023-01-25 14:40:21 +01:00
waleed mousa
2c7f95e3ac update NVIDIA NIC firmware images and settings by ironic-python-agent
Add "update_nvidia_nic_firmware_image" and "update_nvidia_nic_firmware_settings"
clean steps to MellanoxDeviceHardwareManager.

By adding those two steps, we can update the firmware image and
firmware settings of NVIDIA NICs by ironic-python-agent using
manual cleaning command
The clean steps require mstflint package installed on the image.
The "update_nvidia_nic_firmware_image" clean step requires to pass
"images" parameter to the clean command
The "images" parameter is a json blob contains
a list of images, where each image contains a map of:
  * url: to firmware image (file://, http://)
  * checksum: checksum of the provided image
  * checksumType: md5/sha512/sha256
  * componentFlavor: PSID of the nic
  * version: version of the FW

The "update_nvidia_nic_firmware_settings" clean step requires to pass
"settings" parameter to the clean command
The "settings" parameter is a json blob contains
a list of settings, where each settings contains a map of:
  * deviceID: device ID
  * globalConfig: global config
  * function0Config: function 0 config
  * function1Config: function 1 config

Change-Id: Icfaffd7c58c3c73c3fa28cfc2a6c954d2c93c16e
Story: 2010228
Task: 46016
2023-01-11 14:00:07 +00:00
Zuul
06413e54a0 Merge "modify error word node" 2023-01-04 16:12:42 +00:00
Jay Faulkner
7f69144886 Remove old, unused proxy.sh file
This should've been removed with the rest of the contents of imagebuild/

Change-Id: I2ce3ad155d2963352daa5ea8d983d35e8f41eb19
2023-01-03 12:45:50 -08:00
Riccardo Pittau
39b6553161 Fix for tox4 and setuptools
- Disable auto discovery in setuptools
This is to avoid issues with latest setuptools after version 61.0
For more info see [1]
- add bash script to allowlist in tox.ini
- use default size for vm memory

[1] https://github.com/pypa/setuptools/issues/3197

Change-Id: Iae24917add1aae6b1ed46dba0d2c270b39c5698c
2023-01-02 14:40:35 +01:00
liuyuanfeng
1846d6f776 modify error word node
Change-Id: Ie5c9fa7489eb891ef1bbe57c7d51ecb64e1c0db8
2022-12-30 01:18:36 -08:00
Zuul
5c0eab351c Merge "Fix create configuration unit tests" 2022-12-16 15:34:59 +00:00
Jay Faulkner
3ded705e56 Fixes for tox 4.0
- passenv needed better structure
- usedevelop no longer allowed alongside skipsdist

Change-Id: I22dae3b1e4cca2dc328c21e889acf7fe98a80e0b
2022-12-15 10:20:43 -08:00
Riccardo Pittau
604c7081db Fix create configuration unit tests
The unit tests for create_configuration give different result if
ran on a bios or uefi booted machine because they get the
partition table type value based on the utils function
get_node_boot_mode.
Let's mock the boot_mode as we do in other tests to get an
independent result.

Change-Id: Ic0e7daea7ec4ce0806cd126c27166f84690c5d9e
2022-12-15 11:49:34 +01:00
Zuul
a1670753a2 Merge "Fix failure of bind mount in _install_grub2" 9.2.0 bugfix-9.2-eol 2022-10-17 23:46:05 +00:00
Zuul
929ae3dd28 Merge "prioritize lsblk as a source of device serials" 2022-10-14 16:02:51 +00:00
Zuul
29c03cadc3 Merge "Update release versions for yoga" 2022-10-14 01:13:07 +00:00
Rozzii
830fdfa4c6
prioritize lsblk as a source of device serials
The current way of prioritizing ID/DM_SERIAL_SHORT or ID/DM_SERIAL works
in most cases but the udev values seem to be unreliable.

Based on experience it looks like lsblk might be a better
source of truth than udev in regerards to serial number
information. This commit makes lsblk the default provider
of block device serial number information.

Story: 2010263
Task: 46161

Change-Id: I16039b46676f1a61b32ee7ca7e6d526e65829113
2022-10-10 19:31:47 +03:00
43748af791 Switch to 2023.1 Python3 unit tests and generic template name
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for antelope. Also,
updating the template name to generic one.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I91232b32f26842802fc42c1d9e28a6ea791ecb7b
2022-09-23 08:41:54 +00:00