282 Commits

Author SHA1 Message Date
Riccardo Pittau
e5d35ba65c Add lower-constraints job to current development branch
The lower-constraints test was removed becuase of an issue where pip
could not correctly determine the required packages versions to install,
ending in an almost infinite loop that would end up in timeout, failure,
and general mayhem.
Recently the issue has been fixed and, if properly configured, the
lower-constraints test can provide good indication of which minimum
versions are required to support the current code.
This patch adds the test back to the current development branch.
The long term goal is to keep the lower-constraints file in the stable
branches, but remove the test job to avoid issues during backports.

Change-Id: I5fff32ec5dd1a045116bcf02349650b1f5e3a196
2021-07-30 14:20:28 +02:00
Zuul
c71583fc8a Merge "Scoped RBAC Devstack Plugin support" 2021-07-21 11:27:17 +00:00
Julia Kreger
b5872c9032 Set glance limit for baremetal friendly images
The devstack default limit enforcement for glance defaults
to 1GB, and unfortunately this is too small for many to use
larger images such as centos which includes hardware firmware
images for execution on baremetal where drivers need the vendor
blobs in order to load/run.

Sets ironic-base to 5GB, and updates examples accordingly.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/801309
Change-Id: I41294eb571d07a270a69e5b816cdbad530749a94
2021-07-19 10:32:43 -07:00
Julia Kreger
2cd6468346 Scoped RBAC Devstack Plugin support
Adds support to the ironic devstack plugin to configure
ironic to be used in a scope-enforcing mode in line with
the Secure RBAC effort. This change also defines two new
integration jobs *and* changes one of the existing
integration.

In these cases, we're testing functional crub interactions,
integration with nova, and integration with ironic-inspector.

As other services come online with their plugins and
devstack code being able to set the appropriate scope
enforcement configuration, we will be able to change the
overall operating default for all of ironic's jobs and
exclude the differences.

This effort identified issues in ironic-tempest-plugin,
tempest, devstack, and required plugin support in
ironic-inspector as well, and is ultimately required
to ensure we do not break the Secure RBAC.

Luckilly, it all works.

Change-Id: Ic40e47cb11a6b6e9915efcb12e7912861f25cae7
2021-07-15 21:58:31 +00:00
Dmitry Tantsur
5a8a77ec5b Don't run the inspector job on changes to inspector tests
The current irrelevant-files contains the ironic tests directory, but
not the inspector one.

Change-Id: I1654e1333f95d4c3ffd25daa2b80ae4cf1033d91
2021-06-17 19:01:24 +02:00
Julia Kreger
6b0fc9089f Add bifrost-benchmark-ironic job
Utilizes a simple bifrost job to stand up a simple ironic deployment
where fake nodes will be created added, and a simple benchmark will
then be executed.

Change-Id: I33e29ee303b2cf4987b36c7aad2482bc3673f669
2021-06-01 15:07:36 +00:00
Zuul
848ba44608 Merge "CI: Collect a snapshot of network connections" 2021-05-25 11:20:45 +00:00
Dmitry Tantsur
a7f4b4a21c Stop testing the iscsi deploy interface
Change-Id: If876d5bbb7e18f293673d56912ecab3170fe5dfb
2021-04-30 15:54:23 +02:00
94bd4c8e37 Add Python3 xena unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for xena.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I24a4b8fd3d7c086b79875a2d0dd5f2399357d988
2021-04-01 09:12:27 +00:00
Vanou Ishii
d6dd05ab12 Enable Reuse of Zuul Job in 3rd Party CI Environment
At current Zuul job in zuul.d/ironic-jobs.yaml, items of
required-project are like this (without leading hostname)

    required-projects:
      - openstack/ironic
      - openstack/ABCD

but not like this (with leading hostname)

    required-projects:
      - opendev.org/openstack/ironic
      - opendev.org/openstack/ABCD

With first format, if we have two openstack/ironic entries in
Zuul's tenant configuration file (Zuul tenant config file in 3rd
party CI environment usually has 2 entries: one to fetch upstream
code, another for Gerrit event stream to trigger Zuul job), we'll
have warning in zuul-scheduler's log

    Project name 'openstack/ironic' is ambiguous,
    please fully qualify the project with a hostname

With second format, that warning doesn't appear. And Zuul running at
3rd party CI environment can reuse Zuul jobs in zuul.d/ironic-jobs.yaml
in their Zuul jobs.

This commit modifies all Zuul jobs in zuul.d/ironic-jobs.yaml
to use second format.

Story: 2008724
Task: 42068
Change-Id: I85adf3c8b3deaf0d1b2d58dcd82724c7e412e2db
2021-03-17 19:01:07 +09:00
Zuul
fc7f34b2fe Merge "Prepare to use tinycore 12 for tinyipa" 2021-03-15 13:15:45 +00:00
Julia Kreger
ebaa359937 Mark multinode non-voting due to high failure rate
Change-Id: Iea8212ee69a8fe8c5f181c87271f46779e3a46b4
2021-03-11 17:05:50 -08:00
Riccardo Pittau
807a6d2bea Prepare to use tinycore 12 for tinyipa
Tinycore 12 requires some more RAM than its predecessor.

Change-Id: Ibced843f34c78af7780bfe1ade833208b458bb8b
2021-03-02 17:42:45 +01:00
Dmitry Tantsur
5533077c7d Enable swift temporary URLs in grenade and provide a good error message
The fixed configdrive_use_object_store requires them.

Change-Id: Ie7323ae107c7f801be010353c7c4f3b8a43c3a1a
2021-02-24 13:34:17 +01:00
Dmitry Tantsur
414f0ca24e Switch multinode jobs to 512M RAM
384M no longer works reliably with newer tinyIPA.

Change-Id: I7e48b2e682dc0d5e6109e17b0e73ee9763a29d23
2021-02-15 16:20:24 +01:00
Dmitry Tantsur
7c8d1e1e7f Move the IPv6 job to the experimental pipeline
It's broken for months and no effort is under way to fix it.

Change-Id: I88fb5733b3054c2ffa4660f3cb5bff3c852faa75
2021-02-12 17:08:47 +01:00
Julia Kreger
561ed90390 Swap Metalsmith job out for centos8-uefi
Depends-On: https://review.opendev.org/c/openstack/metalsmith/+/773701
Change-Id: Ide1a8988e12958e684670a340bf3c09d516ffa23
2021-02-02 07:01:43 -08:00
Julia Kreger
d44b3b6fec CI: Collect a snapshot of network connections
A recent magnum bug (https://storyboard.openstack.org/#!/story/2008494)
when running with uwsgi has yielded an interesting question if Ironic
is orphaning rpc clients or not. Ironic's code is slightly different
but also very similar, and similar bugs have been observed in the past
where the python garbage collection never cleans up the old connection
objects and effectively orphans the connection.

So we should likely at least try to collect some of the information
so we can determine if this is the case in our CI jobs. Hence this
change and attempt to collect that data after running CI.

Change-Id: I4e80c56449c0b3d07b160ae6c933a7a16c63c5c5
2021-01-12 13:50:14 -08:00
likui
378557b7f2 add openstack-python3-wallaby-jobs-arm64 job
This is a non-voting job to validate py3 unittests on ARM64

Change-Id: I7a3a783ddeb5e9b7aaad9ccfb8aeeb7fcc8a1593
Task: 41376
Story: 2007938
2020-12-31 09:06:10 +08:00
Zuul
a58b88c737 Merge "Remove lower-constraints job" 2020-12-16 15:46:46 +00:00
Riccardo Pittau
840488e595 Remove lower-constraints job
As discussed during the upstream ironic community meeting on
Monday Dec 14 2020, the lower-constraints job is being removed.

Change-Id: I116d99014a7bf77ca77b796ea3b759800dd808ce
2020-12-15 18:43:09 +01:00
Dmitry Tantsur
97ceb38a72 CI: switch the multinode job to tempest-multinode-full-base
The non-base job is designed for the integrated gate and may have
unnecessary side effects. It has recently overriding the OVS agent
bridge settings, breaking our job.

Make the job voting again.

Change-Id: Ied8cafd32c3e634d498467ebe878a411f0b24e6d
2020-12-14 10:16:12 +01:00
Iury Gregory Melo Ferreira
98732623b2 Fix lower-constraints with the new pip resolver
* move pep8 dependencies from test-requirements to tox.ini,
  they're not needed there and are hard to constraint properly.
* add oslo.cache to l-c to avoid bump of dependencies

Change-Id: Ia5330f3d5778ee62811da081c28a16965e512b55
2020-12-11 13:34:24 -08:00
Zuul
e425b6d663 Merge "CI: add a non-voting bifrost-vmedia-uefi job" 2020-12-02 09:01:44 +00:00
Riccardo Pittau
475af371dd Use openstack-tox for ironic-tox-unit-with-driver-libs
All the tox jobs are based on openstack-tox, we should convert
ironic-tox-unit-with-driver-libs too.

Change-Id: I20836d586edccfb8cd8fed1f3a89f1497ff96943
2020-11-27 12:20:53 +01:00
Dmitry Tantsur
9ea4142982 CI: add a non-voting bifrost-vmedia-uefi job
It provides useful coverage of e.g. fast-track with virtual media.

Change-Id: Ie09f4daced5ffd9d953b9add4d5484bbdd1ba1ac
2020-11-26 12:11:48 +01:00
Riccardo Pittau
89af9aef80 Make standalone jobs voting again
Also remove 2 non-voting jobs from gate.

Change-Id: I40574cad53de8b9f89e1ae0a033b75de39140769
2020-11-23 14:51:25 +01:00
Riccardo Pittau
1ab192beca Convert last bionic jobs to focal
And disable dstat in ironic-base for the time being.

Change-Id: Ib05a952260d027f9f1307a9948ac5691b57e96d3
2020-11-12 14:43:05 +00:00
Zuul
09e246294b Merge "CI: increase cleaning timeout and tie it to PXE boot timeout" 2020-10-30 19:19:20 +00:00
Zuul
485601be5b Merge "Mark standalone job non-voting/remove from gate" 2020-10-28 13:47:07 +00:00
Julia Kreger
9696ec9a5a Mark standalone job non-voting/remove from gate
The standalone job at present has a high chance of failure
due to two separate things occuring:

1) The deployed nodes from raid tests can be left in a dirty state
   as the raid configuration remains and is chosen as the root
   device for the next deployment. IF this is chosen by any job,
   such as rescue or a deployment test that attempts to login,
   then the job fails with unable to ssh. The fix for this is
   in the ironic-tempest-plugin but we need to get other fixes
   into stablilize the gate first.
   https://review.opendev.org/#/c/757141/
2) Long running scenarios run in cleaning such as deployment with
   RAID in the standalone suite can encounter conditions where
   the conductor tries to send the next command along before the
   present configuration command has completed. An example is
   downloading the image is still running, while a heartbeat
   has occured in the background and the conductor then seeks
   to perform a second action. This then causes the entire
   deployment to fail, even though it was transitory.
   This should be a relatively easy fix.
   https://review.opendev.org/759906

Change-Id: I6b02be0fa353daac90abf2b1576800c0710f651e
2020-10-27 17:16:44 +00:00
Dmitry Tantsur
2e2b07bb91 Move the multinode grenade job to the experimental pipeline
It's hopelessly broken, let's not waste resources on it until we
get back to making it work.

Change-Id: I171fa566e36ad5ac8659ecb0578029df270497d6
2020-10-27 12:32:46 +01:00
Dmitry Tantsur
87e634dee1 CI: increase cleaning timeout and tie it to PXE boot timeout
We're seeing cases where cleaning barely manages to finish after
a 2nd PXE retry, failing a job.

Also make the PXE retry timeout consistent between the CI and
local devstack installations.

Change-Id: I6dc7a91d1a482008cf4ec855a60a95ec0a1abe28
2020-10-27 12:21:43 +01:00
Riccardo Pittau
fc2964cb75 Run bifrost integration job on focal
Part of the migration from bionic to focal as community goal.

Change-Id: I100f799efb7be4a0413a38cd0e218dce43a44573
2020-10-14 17:50:31 +02:00
Riccardo Pittau
4775288b93 migrate testing to ubuntu focal
As per victoria cycle testing runtime and community goal
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

keeping few jobs running on bionic nodeset till
https://storyboard.openstack.org/#!/story/2008185 is fixed
otherwise base devstack jobs switching to Focal will block
the gate.

Change-Id: I1106c5c2b400e7db899959550eb1dc92577b319d
Story: #2007865
Task: #40188
2020-10-05 17:05:49 +02:00
1c49b62e2f Add Python3 wallaby unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for wallaby.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I3d01db0826babc1022a3a8aa3254ea164cd3265e
2020-10-01 19:15:32 +00:00
Zuul
28c0f38322 Merge "Deprecate the iscsi deploy interface" 2020-09-24 06:09:10 +00:00
Zuul
19d910db40 Merge "Limit inspector jobs to 1 testing VM" 2020-09-23 11:12:17 +00:00
Dmitry Tantsur
b0b71653c7 Make the standalone-redfish job voting
It was supposed to be made voting shortly after the split, but we
sort of forgot. It provides coverage for things (like ansible deploy)
that we used to have voting jobs for.

Change-Id: Id99586d5e01b940089d55c133d9181db05bfdc7e
2020-09-22 16:55:26 +02:00
Dmitry Tantsur
d8dccc8d06 Deprecate the iscsi deploy interface
This change marks the iscsi deploy interface as deprecated and
stops enabling it by default.

An online data migration is provided for iscsi->direct, provided that:
1) the direct deploy is enabled,
2) image_download_source!=swift.

The CI coverage for iscsi deploy is left only on standalone jobs.

Story: #2008114
Task: #40830
Change-Id: I4a66401b24c49c705861e0745867b7fc706a7509
2020-09-22 15:39:36 +02:00
Julia Kreger
ed0ef6cf59 Reduce VMs for multinode and standalone jobs
The minimum amount of disk space on CI test nodes
may be approximately 60GB on /opt with now only 1GB
of available swap space by default.

This means we're constrained on the number of VMs and
their disk storage capacity in some cases.

Change-Id: Ia6dac22081c92bbccc803f233dd53740f6b48abb
2020-09-21 15:31:07 -07:00
Julia Kreger
7d0661a1b4 Reduce grenade node count
Infra's disk/swap availability has been apparently
reduced with the new focal node sets such that we
have ~60GB of disk space and only 1GB of swap.

If we configure more swap, then naturally that means
we take away from available VMs as well.

And as such, we should be able to complete grenade
with only four instances, I hope.

Change-Id: I36f8fc8130ed914e8a2c2a11c9679144d931ad73
2020-09-21 15:20:03 -07:00
Dmitry Tantsur
a141b6f17c Limit inspector jobs to 1 testing VM
Currently ironic-base defaults to 2 and our tests try to introspect
all of them. This puts unnecessary strain on the CI systems, return
the number back to 1.

Change-Id: I820bba1347954b659fd7469ed542f98ef0a6eaf0
2020-09-21 19:33:26 +02:00
Iury Gregory Melo Ferreira
1f0174bb41 Native zuulv3 grenade multinode multitenant
Based on the native 'grenade-multinode' job

Change-Id: I4d0a23c371bc42c5bf18e79ea7920bd77b066154
2020-09-16 23:33:42 +02:00
Dmitry Tantsur
b5d5e5774c Change [agent]image_download_source=http
As part of the plan to deprecate the iSCSI deploy interface, changing
this option to a value that will work out-of-box for more deployments.

The standalone CI jobs are switched to http as well, the rest of jobs
are left with swift. The explicit indirect jobs are removed.

Change-Id: Idc56a70478dfe65e9b936006a5355d6b96e536e1
Story: #2008114
Task: #40831
2020-09-08 16:28:31 +02:00
Zuul
b6cf0432a7 Merge "Remove token-less agent support" 2020-09-07 15:07:17 +00:00
Zuul
3709cce11f Merge "ISO ramdisk virtual media test enablement" 2020-09-06 12:07:02 +00:00
Julia Kreger
5b272b0c46 Remove token-less agent support
Removes the deprecated support for token-less agents which
better secures the ironic-python-agent<->ironic interactions
to help ensure heartbeat operations are coming from the same
node which originally checked-in with the Ironic and that
commands coming to an agent are originating from the same
ironic deployment which the agent checked-in with to begin
with.

Story: 2007025
Task: 40814
Change-Id: Id7a3f402285c654bc4665dcd45bd0730128bf9b0
2020-09-04 17:09:39 +00:00
Riccardo Pittau
e9bda223f7 Increase memory of tinyipa vms
Tinyipa is not that tiny anymore and we need to increase the base
memory for VMs in jobs that use it.

Change-Id: Ibd7e87c0b5676eef94512285edaca416635a29ef
2020-08-21 14:41:15 +02:00
Julia Kreger
9e7f1cb570 ISO ramdisk virtual media test enablement
Sets the settings to enable the ramdisk iso booting tests
including a bootable ISO image that should boot the machine.

NB: The first depends-on is only for temporary testing of another
which changes the substrate ramdisk interface. Since this change pulls
in tempest testing for iso ramdisk and uses it, might as well
use it to test if the change works or not as the other two patches
below are known to be in a good state.

Change-Id: I5d4213b0ba4f7884fb542e7d6680f95fc94e112e
2020-08-17 10:14:38 -07:00