802 Commits

Author SHA1 Message Date
Julia Kreger
7f281392c2 Change wholedisk image checksum to sha256
Change-Id: I0c90ac87ca88329e7fb315385345e8020a59fdd5
2023-05-02 13:03:13 -07:00
Riccardo Pittau
87a5f1add5 Bump cirros to version 0.6.1
Use latest cirros available

Change-Id: I916b96d1386d3b1f090df11a7a8d2ef70fe22559
2023-01-26 11:55:58 +01:00
Zuul
320b1f0ca7 Merge "Fix unbound variable in devstack plugin" 2023-01-07 16:30:49 +00:00
Zuul
c93641ff59 Merge "Use tinycore 13 for base ramdisk image" 2023-01-07 15:54:56 +00:00
Zuul
bed680951a Merge "Remove lib/neutron-legacy leftovers" 2023-01-05 12:25:48 +00:00
Zuul
7f6a737e98 Merge "Use centos grub artifacts with centos ramdisk for vmedia" 2023-01-04 20:31:36 +00:00
Julia Kreger
1d07be8237 Use centos grub artifacts with centos ramdisk for vmedia
It appears we are getting an opcode error when attempting to boot
Centos 9-stream utilizing the EFI artifacts from Ubuntu.

Technically this should work, however further aftifacts in the boot
chain may be signed with other key credentials that Ubuntu's
grub does not know about, because the chain of trust is
MSFT -> Vendor shim (slow change rate) -> Vendor GRUB -> Kernel

Where vendor differences should never work, is if Secure Boot
is enforcing.

Exception on launch:
 X64 Exception Type - 06(#UD - Invalid Opcode)  CPU Apic ID - 00000000 !!!!

A similar Debian bug is open for a very similar issue:

https://groups.google.com/g/linux.debian.bugs.dist/c/BOiLLeROrmo

However, no additional comments or information have been in follow
up to that reported issue. So in the mean time, we're going to try
and do what those smarter than I recommend, use the vendor's
binaries for their distribution.

There is one further, potentially far more depressing possibility,
that centos9's kernel doesn't support the type of hardware
we're getting. This is suggested by the precise opcode error, UD,
https://xem.github.io/minix86/manual/intel-x86-and-64-manual-vol3/o_fe12b1e2a880e0ce-212.html
But again, easiest possibility first.

Change-Id: Id9bd30bc3c2f1076555317e4a3f277725fa7c1f4
2023-01-03 17:05:04 -08:00
Riccardo Pittau
cab04afe6e Create IRONIC_VM_MACS_CSV_FILE if it does not exist
The IRONIC_VM_MACS_CSV_FILE is generated only if we execute the
ironic basic ops, so when IRONIC_BAREMETAL_BASIC_OPS is True.
In some jobs we set IRONIC_BAREMETAL_BASIC_OPS to False but we
still look for that file causing a "file not found" error which
does not trigger a trap until focal, but it does in jammy.
Let's create the file if it does not exist.

Change-Id: Ib938abe0723072419f336159cbffff33e46ea39b
2023-01-03 16:09:00 +01:00
Riccardo Pittau
7150478cd8 Fix unbound variable in devstack plugin
The RC_DIR does not existed (and it never existed, it was SRC_DIR)
Change that to TOP_DIR which is what we use commonly in other
sections.

Change-Id: I4a400fd434a20938cd38c0bb876da21fec7473a1
2023-01-02 10:10:50 +01:00
Riccardo Pittau
93158aadd0 Use tinycore 13 for base ramdisk image
We're builing tinyipa using tinycore 13.x since a while, we should use
the same version for the base ramdisk image.

Change-Id: I9d144f122c20f717ff946282ef7ffa16d82812f5
2023-01-02 09:44:26 +01:00
Sławek Kapłoński
facd1bca66 Remove lib/neutron-legacy leftovers
In [1] we finally got rid of the unfinished lib/neutron module and kept
only lib/neutron-legacy. It's renamed to lib/neutron now and it's the
only neutron related module in Devstack.
So this patch removes leftovers related to the old lib/neutron-legacy.

[1] https://review.opendev.org/c/openstack/devstack/+/865014

Change-Id: Id938deab7188743e754d028dee8e0b2591ab6f7b
2022-12-20 11:18:25 +01:00
Ghanshyam Mann
ceec890947 Use project scoped token for cinder, glance services
All services except Ironic (and keystone to support ironic
with system scope deployement), will not have system scope in
their API policy instead they are default to project scoped.

Change-Id: Id13a359086f9b24dbfcd2b565a42c50d0dab7736
2022-10-15 20:05:06 -05:00
Julia Kreger
9344eb22d1 Add upgrade check warning for allocations db
Adding an upgrade check to provide awareness to the state of
the database in regards if an unexpected engine is in use or
if the character set encoding is also not UTF8.

These will raise non-fatal warnings on the upgrade status
check.

Change-Id: Ide0eb4690a056be557e5ea7d5ba5f6be37b50d0a
Story: 2010384
2022-10-13 10:54:55 -07:00
Julia Kreger
d8fc96fd1f CI: Changes to support Anaconda CI jobs
Introduces additional job configuration to enable automated
integration testing via tempest of the anaconda deployment
interface.

Also, configures a private subnet with DNS, which is required
by anaconda executing, in order to facilitate processing of URLs.

Change-Id: I61b5205cf2c9f83dfcabf4314247c76fb6a56acd
2022-09-06 07:38:11 -07:00
Dmitry Tantsur
f0a1778766 Finally remove support for netboot and the boot_option capability
Instance network boot (not to be confused with ramdisk, iSCSI or
anaconda deploy methods) is insecure, underused and difficult to
maintain. This change removes a lot of related code from Ironic.

The so called "netboot fallback" is still supported for legacy boot when
boot device management is not available or is unreliable.

Change-Id: Ia8510e4acac6dec0a1e4f5cb0e07008548a00c52
2022-08-02 12:47:31 +02:00
Julia Kreger
f7471f07c3 CI: Only setup fake v6 interface if needed
In the case of CI test nodes natively supporting and using ipv6,
we don't need to actually setup a fake IPv6 network for ports
to bind to on the local system. So before doin gso, lets check
to see if we can ping the address first. If not, then set it up.

Change-Id: Ib68c706c1f9ef0ad0cf27e7a6acffd2c50ff37ea
2022-07-20 11:08:20 -07:00
Julia Kreger
8b99fcb0e4 CI: Default to TinyIPA when nested virt is not possible
Change-Id: I6bd52f61ff5e9f928b504b09a1ce6eb97cff57da
2022-06-23 14:24:41 -07:00
Julia Kreger
e0c758bb95 CI: Add iweb to the use tinyipa on list
Change-Id: Ib1d415928a6555298d42e8d525f04eb1028a4bb8
2022-06-23 14:13:58 -07:00
Zuul
1b6114934c Merge "Switch to q35 machine type for test nodes" 2022-06-22 14:22:17 +00:00
Steve Baker
832dc8bf94 Switch to q35 machine type for test nodes
q35 is recommended as emulating modern baremetal with better support
for UEFI in general, and Secure Boot in particular.

Old pc type usage is removed, like IDE controller, PS2 mouse, manual
PSI addresses.

Change-Id: Ic33e0f23c5c514a45541534ddd68329d7b4d0480
2022-05-31 04:37:28 +00:00
Dmitry Tantsur
81f583f69b devstack: use CentOS 9 for DIB IPA builds
Additionally bumps CPU model to host-model as centos9 builds now
require a subset of CPU processors which include advanced features.
Host-model also allows for the VM to still start when running with
pure qemu, as opposed to KVM passthrough.

https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level#architectural_considerations_for_rhel_9

Change-Id: Ic261efd4bf6f5929687df5e7b1b51b541554af18
2022-05-25 08:57:15 -07:00
Zuul
49113385e8 Merge "Fix v6 CI job - Return it to normal non-voting status" 2022-05-05 14:46:27 +00:00
Julia Kreger
a9f4acfdb0 Fix v6 CI job - Return it to normal non-voting status
* Fixes the IPv6 job by utilizing HOST_IPV6 instead of
  SERVICE_IPV6, as Devstack now automatically wraps
  SERVICE_IPV6 with brackets as if it is for a URL.
* Locks ipv6 job to bios mode. Ubuntu Focal OVMF/EDK2 does not
  support IPv6 PXE boot by default.
* Split from Devstack in terms of IP usage, since full explicit
  V6 usage is not a thing anymore. 4+6 is the default in devstack
  and regardless of what we set on the job we see both now used.
  So we delineate apart our usage for our own sanity.
* Reduce VM Interface count for IPv6 in an attempt to eliminate
  in-kernel routing confusion by two interfaces on the same physical
  network.
* Set IPv6 mode to dhcpv6-stateless due to fun issues in dhcp clients.
  When we move to UEFI, this will need to be changed to stateful as
  stateless is not supported in general by OVMF/E2DK.

Once the job has run in normal non-voting for a while, and we
ensure that it seems to be stable, we can make it voting again.

Change-Id: Ia833bfb64c6c3cc8e48cbe34ed200536652a8adf
2022-05-04 11:32:29 -07:00
Dmitry Tantsur
b94e52f55a Fix ironic-lib from source and branch detection in IPA builds
Prevents the ironic-lib CI from testing ironic-lib changes in IPA.

Change-Id: I936f6c1506c585826501ff3ac0bad0c755b4d360
2022-05-02 12:19:59 +02:00
Julia Kreger
fe3021fc37 Grenade: Turn up interfaces for vxlan
So... We can't do this in a single patch, and we *actually*
need to merge the vxlan fix before the subnode will ever pickup
the configuration

From the logs, I can observe the vxlan tunnel connects between
the nodes. Awesome.

Where things break is in the local setup of the local bridges
used to wire everything together.

setup_vxlan_network:3274 :   sudo ovs-vsctl add-port sub1brbm phy-brbm-infra
ovs-vsctl: Error detected while setting up 'phy-brbm-infra':
could not open network device phy-brbm-infra (No such device).
See ovs-vswitchd log for details.

Basically, with the same change on a separate patch, we're able to
observe the controller node work perfectly. It is the subnode
connectivity which is just broken.

So, activate the bridge interfaces seems ideal. This likely broke
at some point due to behavior changes in OpenVSwitch.

Change-Id: I11dbba1957d67187d859a1ef60563c0301da9812
2022-04-26 10:43:27 -07:00
Dmitry Tantsur
7ac4804126 Build the new cirros image even when netboot is the default
The standalone job changes boot_option in runtime, so local boot
can be used even when the default boot option is netboot.

Change-Id: Ia538907f3662e8cd84d988ea5d862c7f488558e1
2022-02-17 15:36:11 +01:00
Dmitry Tantsur
bbceca562e CI: use a custom cirros partition image instead of the default
Cirros partition images are not compatible with local boot since they
don't ship grub (nor a normal root partition). This change adds a script
that builds a partition image with UEFI artifacts present. It still
cannot be booted in legacy mode, but it's a progress.

Set the tempest plugin's partition_netboot option. We need it to inform
the tempest plugin about the ability to do local boot. This option
already exists but is never set.

Also set the new default_boot_option parameter, which will be introduced
and used in Iaba563a2ecbca029889bc6894b2a7f0754d27b88.

Remove netboot from most of the UEFI jobs.

Change-Id: I15189e7f5928126c6b336b1416ce6408a4950062
2022-02-16 10:12:06 +01:00
Dmitry Tantsur
720b42d538 Avoid non-Stream CentOS and temporary disable the standalone job
The standalone job is failing because of a bug in IPA. To fix it we need
to make DIB jobs operational, and they're failing because of CentOS repos.

Change-Id: I8bd051ea709d328cb5efa2c2cbd5a226bdb4cfd3
2022-01-31 16:32:08 +01:00
Aija Jauntēva
365a4545fe Fix DevStack plugin ipxe-snponly-x86_64.efi name
Change-Id: Ic25eb356d1bc86c1dc4b09df7fc0df42b3821cf3
2022-01-27 09:59:51 -05:00
Dmitry Tantsur
cfcea55cf6 Automatically configure enabled_***_interfaces
This change makes it easier to configure power and management interfaces
(and thus vendor drivers) by figuring out reasonable defaults.

Story: #2009316
Task: #43717
Change-Id: I8779603e566be5a84daf6f680c0bbe2f191923d9
2021-12-20 15:11:17 +01:00
Steve Baker
3f76724dfb Write initial grub config on startup
This change removes the documentation to copy master_grub_cfg.txt to
/tftpboot/grub/grub.cfg and instead writes it on conductor startup.
This grub config is a simple redirect config requested by grub network
boot. "master" has been renamed to "initial" as a more accurate label
of its function.

New configuration option [pxe]initial_grub_template allows the deployer
to specify a different initial grub template.

Change-Id: I71191dd399a6c49607f91d69b5b1673799a38624
2021-12-10 15:44:50 +13:00
Zuul
3b949b2086 Merge "Use test_with_retry to get the tap device name" 2021-12-08 19:27:53 +00:00
Dmitry Tantsur
b37ee7c911 devstack: provide a default for OS_CLOUD
Not having it breaks the inspector grenade job.

Change-Id: I7ee28a85cb2005dd69e6711b301cd029b8ca40cc
2021-12-08 09:49:26 +01:00
Dmitry Tantsur
f85f649136 Install isolinux on devstack
It is required for virtual media BIOS booting.

Clean up old bindep tags.

Change-Id: I345e5b5287594e62ac7a8abb4de3add242120dfd
2021-12-03 17:50:52 +01:00
Steve Baker
9b4631ae0d Use test_with_retry to get the tap device name
This change replaces the 10 second sleep with a retry that has a
timeout of 20 seconds to discover the name of the tap device.

There are gate failures when there is still not a tap device after the
10 second sleep, so this approach should be faster in the common case,
and the higher timeout should provide more reliability.

Change-Id: I5e59ade9f830182b483b9655aaaf6c93b0bfac44
2021-11-22 13:50:59 +13:00
Julia Kreger
350c2f7a50 CI: Fix devstack plugin with RBAC changes
Changes a neutron call to be project scoped as system
scoped can't create a resource and, and removes the unset
which no longer makes sense now that
I86ffa9cd52454f1c1c72d29b3a0e0caa3e44b829
has merged removing the legacy vars from devstack.

Also renames intenral use setting of OS_CLOUD to IRONIC_OS_CLOUD
as some services were still working with system scope or some sort
of mixed state occuring previously as some of the environment variables
were present still, however they have been removed from devstack.

This change *does* explicitly set an OS_CLOUD variable as well on
the base ironic job. This is because things like grenade for Xena
will expect the variable to be present.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/818449
Change-Id: I912527d7396a9c6d8ee7e90f0c3fd84461d443c1
2021-11-19 08:22:22 -08:00
Zuul
386c15836c Merge "Remove debian packages file for devstack" 2021-10-18 11:58:42 +00:00
Julia Kreger
f205e7d2be Remove debian packages file for devstack
This file duplicages devstack's load of bindep for the ironic
repository. As bindep is the authortative file, removing the
legacy way of installing packages from ironic.

Also revises the bindep file to explicitly remove the devstack
group, as it is all devstack.

Change-Id: Ida7ca230069fc0e4f54bde2fc6fffdc9eb0bdcc2
2021-10-08 10:40:07 -07:00
Julia Kreger
371313214a SRBAC - Prepare for additional services
In order to effectively handle cross-service integrations, we need to
evaluate two separate items which are not standardized in devstack.

Names, and common service references. Unfortunately, only a couple
services presently have support in devstack for these settings, and
cases where it was previously supported has been removed for unknown
reasons, but this seems to be the overall plan.

Sets the stage, so we can be early to the cross-service testing party
of secure rbac.

Change-Id: I8794374c02a24185b6e24a675ad9cb7b3dfd69df
2021-10-08 17:26:16 +00:00
Julia Kreger
044091c146 Retool devstack plugin to use pxe loaders configuration
Begins to peel back some of the override plugin/setting
nature in use in the ironic devstack plugin by trying to
place all of the files and letting the *defaults* take
the service lead while also putting in place the required
configuration for pxe loaders to be used.

Change-Id: I73ca82e0d123fd6efab06dbbdeef40c2d9972887
2021-10-08 17:25:54 +00:00
Zuul
0b36d9afa0 Merge "CI: Change CI ipxe file to snponly" 2021-10-08 11:10:36 +00:00
Zuul
af903094ea Merge "Yoga: Change default boot mode to uefi" 2021-10-07 10:04:24 +00:00
Julia Kreger
2b55c8d388 CI: Change CI ipxe file to snponly
Ironic's configured default is snponly.efi, and realistically
we should be using it for devstack as ipxe.efi lacks snp which
is included in the EFI standard.

Change-Id: I749420b127cc9954bfa02d9e4efaa0980a9242be
2021-10-04 18:59:52 +00:00
Julia Kreger
493b4f0caf Yoga: Change default boot mode to uefi
Change the default boot mode to UEFI, as discussed during the end
of the Wallaby release cycle and previously agreed a very long time
ago by the Ironic community.

Change-Id: I6d735604d56d1687f42d0573a2eed765cbb08aec
2021-10-04 11:57:55 -07:00
Zuul
28a06c1bd4 Merge "Remove legacy rpm install list and use bindep" 2021-10-04 17:08:07 +00:00
Julia Kreger
c1e355011c Remove legacy rpm install list and use bindep
The legacy rpm install list is out of date and includes
packages which no longer exist in newer distribution builds.

This functionality was entirely replaced by bindep, which is
*the* authortative file for things like this. The major difference
is the separate bindep file can't include mysql, or we break
devstack \o/.

Change-Id: Ic86f6efdf75fc2871c03e21b7f9166192b0f212c
2021-09-24 09:44:50 -07:00
Julia Kreger
4775fb3d92 Devstack: don't scan /opt, /etc looking for isolinux
/opt contains a mirror of an insane amount of stuff, and it chews
disk io to scan it looking for isolinux.bin which should be under
/usr on... well... every OS we support.

Also, don't scan /etc. That is just weird.

Change-Id: I52f4c1ba8808fea637df69a631eaa1c674dc8e69
2021-09-22 11:43:46 +00:00
Zuul
0affe4de8d Merge "Use packaged grub efi for network boot" 2021-09-09 17:19:17 +00:00
Steve Baker
fc8601cd02 Use packaged grub efi for network boot
Instead of using the efi written by grub-mknetdir, use the packaged
signed binary. The core.efi generated by grub-mknetdir is not signed
so it does not help with end-to-end secure-boot.

Also, the successful run of
ironic-tempest-ipa-partition-uefi-pxe-grub2[1] demonstrates that grub
continues to boot even when the grub-mknetdir generated
grub/x86_64-efi/*.lst are missing. Avoiding using grub-mknetdir makes
for a much simpler setup of /tftpboot for grub network boot.

[1] https://zuul.opendev.org/t/openstack/build/bab62f6bf032474cb80af3cb5a999117/log/tftpd-journal.txt

Change-Id: Ide0aa416391c20371bbb8d1a18288b262872e313
2021-09-08 13:35:45 +12:00
Zuul
5d79347f97 Merge "Fix upgrade logic to allow for bundled changes" 2021-09-03 13:20:09 +00:00