11005 Commits

Author SHA1 Message Date
Julia Kreger
5f6a51e178 Set default to prevent out of memory conditions
Change-Id: I00efd4784346adb89b137424ed35bdeafe9b6f24
2021-01-29 22:34:54 +00:00
Julia Kreger
d9913370de Guard conductor from consuming all of the ram
One of the biggest frustrations larger operators have is when they
trigger a massive number of concurrent deployments. As one would
expect, the memory utilization of the conductor goes up. Except,
even with the default number of worker threads, if we're requested
to convert 80 images at the same time, or to perform the write-out
to the remote node at the same time, we will consume a large amount
of system RAM. Or more specifically, qemu-img will consume a large
amount of memory.

If the amount of memory goes too low, the system can trigger
OOMKiller which will slay processes using ram. Ideally, we do not
want this to happen to our conductor process, much less the work
that is being performed, so we need to add some guard rails to help
keep us from entering into situations where we may compromise the
conductor by taking on too much work.

Adds a guard in the conductor to prevent multiple parallel
deployment operations from running the conductor out of memory.

With the defaults, the conductor will attempt to throttle back
automatically and hold worker threads which will slow down the
amount of work also proceeding through the conductor, as we are
in a memory condition where we should be careful about the work.

The defaults allow this to occur for a total of 15 seconds between
re-check of available RAM, for a total number of six retries.
The minimum default is 1024 (MB), as this is the amount of memory
qemu-img allocates when trying to write images. This quite literally
means no additional qemu-img process can spawn until the default
memory situation has resolved itself.

Change-Id: I69db0169c564c5b22abd0cb1b890f409c13b0ac2
2021-01-29 14:33:57 -08:00
Zuul
fd34d3c437 Merge "Add centralized secure boot documentation" 2021-01-27 13:36:39 +00:00
Dmitry Tantsur
4c4c7a869a Add a few words about UEFI user images
Change-Id: I37a686e6f48a422d38ac5921a188d894519b7530
2021-01-26 21:22:56 +01:00
Dmitry Tantsur
04400eea47 Add centralized secure boot documentation
Move the bits from iLO and iRMC, clean them up a bit.

Change-Id: I5b6da854ae0214141ae25a17b8ea3c7874636372
2021-01-26 17:00:50 +01:00
Zuul
2a7871d060 Merge "Common framework for configuring secure boot" 2021-01-26 12:25:26 +00:00
Zuul
8612c06d4a Merge "redfish-virtual-media: allow a link to raw configdrive image" 2021-01-25 21:22:50 +00:00
Zuul
4af9e49859 Merge "Update minversion of tox" 2021-01-24 15:02:18 +00:00
Dmitry Tantsur
bb318008b9 redfish-virtual-media: allow a link to raw configdrive image
For historical reasons we always base64+gzip configdrives, even
when accessing them via a URL. This change allows binary images
to work for the redfish-virtual-media case.

Change-Id: If19144de800b67275e3f8fb297f0a5c4a54b2981
2021-01-22 16:26:44 +01:00
Zuul
5640860c81 Merge "Follow-up for ramdisk deploy configdrive support" 2021-01-21 14:06:14 +00:00
Riccardo Pittau
5165edaf93 Update minversion of tox
The minimum version since when tox accepts having inline comments
in deps is 3.9.0

https://tox.readthedocs.io/en/latest/changelog.html#v3-9-0-2019-04-17

Change-Id: I4d2aaf97459344ca01e5ccd6e563474c094a4678
2021-01-21 14:57:07 +01:00
Zuul
d6661a2d4b Merge "Attempt to slim down protection test base class" 2021-01-21 11:33:13 +00:00
Zuul
ac70e24665 Merge "Write stub ACL test for every existing API call" 2021-01-21 11:29:03 +00:00
Zuul
cd89ea38e7 Merge "Update iDRAC doc with missing interfaces" 2021-01-21 08:53:21 +00:00
Lance Bragstad
c7f24e8ceb Attempt to slim down protection test base class
This commit builds on an attempt to introduce a base class that we can
use for protection testing. I'm removing the majority of the fake tokens
and context objects, ultimately removing anything that
keystonemiddleware_authtoken middleware does.

Doing this allows us to set authorization headers directly in the test,
which makes the test case really clear from an input and outcome
perspective because we can see who is make the request and the intended
outcome.

It also removes any need to mock or understand keystonemiddleware
implementation details to implement protection testing in ironic.

Change-Id: I9a3eb62bb41e0623df9aa5c34fde6f343053dd76
2021-01-20 06:41:47 -08:00
Steve Baker
36d819e2fb Write stub ACL test for every existing API call
This adds a skipped test for every documented path and method
to aid in getting test coverage of existing ACL behaviour, in
preparation for doing the same for secure-rbac.

When adding test coverage, the skip keys should be removed, and
specific test inputs and asserts should be added. The test can be
duplicated and renamed to get the required allow/deny test coverage.

Its possible we can delete some of these stubs as the path/method
shares a policy name with another path/method that has test coverage.

test_acl_existing.yaml was generated with the script
http://paste.openstack.org/show/801106/

Change-Id: Iee91d80cef3b9e6024507171352c6de9e89ce36e
2021-01-20 06:36:47 -08:00
Aija Jauntēva
b0df0960e2 Update iDRAC doc with missing interfaces
Change-Id: I691b76879ba00fb5535d7016c9d6fb53e9dde462
2021-01-20 09:25:19 -05:00
Lance Bragstad
6ea73bdfbb Bump oslo.log requirement to 4.3.0
This ensures we have the WALLABY marker available for deprecation
noticies.

Change-Id: Ic91f34a90fdd0c22c06c669551eaf6f9a398956b
2021-01-19 10:59:48 -08:00
Dmitry Tantsur
b6f4587f0b Common framework for configuring secure boot
Two drivers already support turning secore boot on and off,
Redfish will follow soon. This patch adds ManagementInterface
calls to get and set the secure boot state.

Story: #2008270
Task: #41561
Change-Id: I96b2697163def52618b4c051a5c85adf7d1818a5
2021-01-19 17:56:06 +01:00
Zuul
7b0487df2e Merge "redfish-virtual-media: make fewer calls when preparing boot" 2021-01-19 06:47:21 +00:00
Zuul
67c90e7e4f Merge "Policy json to yaml migration" 2021-01-19 02:11:28 +00:00
Zuul
75fe7cb143 Merge "Enable testing to dynamicly leverage ACL roles" 2021-01-18 17:44:15 +00:00
Dmitry Tantsur
3ca8671a2c redfish-virtual-media: make fewer calls when preparing boot
Currently we're loading a new System and ManagerCollection on each
operation. Cache the ManagerCollection instead.

Change-Id: I0ec9630003b31e869703e75ffcea1ece90ac3dc2
2021-01-18 15:18:42 +01:00
Zuul
d35eb8bd0e Merge "Add a delay/retry is vmedia insert fails" 2021-01-18 14:08:52 +00:00
Derek Higgins
1a0f1cd545 Add a delay/retry is vmedia insert fails
If media had been ejected beforehand then an immediate
insert may fail.

Change-Id: I6cf7169f83356c248d4df61e1d9bba157c8499db
Story: #2008504
Task: #41572
2021-01-15 16:40:04 +00:00
Dmitry Tantsur
ad044d9e0b Fix redfish-virtual-media boot mode handling ordering
Currently sync_boot_mode does not happen for nodes with local boot
because of an early return. This patches fixes it.

Change-Id: I37b91f93f8f1ca41b814199ec1c5733611b28299
2021-01-14 17:33:37 +01:00
Steve Baker
04a1f17ef8 Enable testing to dynamicly leverage ACL roles
The 'existing' tests will need users with credentials other than admin
and member, and the secure-rbac tests will need different credentials
again.

As this no longer uses a cache, and is dynamically generated and passed
the actual *token* vality is not checked, and all ACL testing uses
role values.

Change-Id: Ief00e16da32a5196c21920c317c309241f2ad1cb
2021-01-13 13:01:06 -08:00
Zuul
07bdccea58 Merge "Do not enter maintenance if cleaning fails before running the 1st step" 2021-01-12 07:10:42 +00:00
Dmitry Tantsur
fe380bbbab Follow-up for ramdisk deploy configdrive support
1) Do not issue a warning if the boot interface supports configdrive
2) Implement missing support for Swift URLs in configdrives

Change-Id: I4b06478a14ab514d785f8e3972e5afbd79f8d3b5
2021-01-11 20:02:27 +01:00
Zuul
e7a372b017 Merge "Rewrite existing ACL tests with ddt, yaml" 2021-01-11 18:33:32 +00:00
Zuul
4e93127986 Merge "Register all hardware_interfaces together" 2021-01-11 18:33:14 +00:00
Zuul
6af2e2d9d1 Merge "Support configdrive when doing ramdisk deploy with redfish-virtual-media" 2021-01-11 17:28:39 +00:00
Zuul
1c7b5f8259 Merge "docs: Add information on post-branch release tasks for bifrost" 2021-01-08 15:25:17 +00:00
Derek Higgins
7d85b35c84 Register all hardware_interfaces together
Prevent each driver comming online one at a time. So that
/driver returns nothign until all interfaces are registered

Story: #2008423
Task: #41368

Change-Id: I6ef3e6e36b96106faf4581509d9219e5c535a6d8
2021-01-08 15:16:53 +00:00
Dmitry Tantsur
ad696c9bac Do not enter maintenance if cleaning fails before running the 1st step
We use maintenance mode to signal that hardware needs additional
intervention, because of potential damage or stuck long-running
processes. This is not the case for PXE booting or invalid requested
manual clean steps, so don't set maintenance if no clean step is
running when the failure occurs.

Change-Id: I8a7ce072359660fc6640e5f20ec2d3c452033557
2021-01-08 14:57:07 +01:00
Zuul
d5f184ea16 Merge "Document using ramdisks with the ramdisk deploy interface" 2021-01-05 18:31:38 +00:00
Julia Kreger
2404d486ac Policy json to yaml migration
Adds the status upgrade check for the JSON to YAML migration
effort and updates the documentation where it seems appropriate
to move from "policy.json" to "policy.yaml"

Mostly shamelessly copied from https://review.opendev.org/#/c/748059/
however is in-line with ironic's configuration and patching methods.

Related Blueprint: policy-json-to-yaml

Change-Id: I1d5b3892451579ebfd4d75a0f7185e0ef3c984c8
2021-01-04 13:40:54 -08:00
Julia Kreger
1e96ecbdbc Add troubleshooting on changing ironic.conf default interfaces
Change-Id: If836d064ed7e8f6eaefbc0cfab8c404d2c3174fb
2021-01-04 09:40:41 -08:00
Zuul
fcf029a0ad Merge "Modify port group document for ironic" 2021-01-04 09:51:49 +00:00
Zuul
0112b33291 Merge "Mark the iSCSI deploy as deprecated in the docs" 2021-01-01 17:51:12 +00:00
Zuul
6886c5205c Merge "add openstack-python3-wallaby-jobs-arm64 job" 2021-01-01 17:51:04 +00:00
Zuul
3864483a76 Merge "update python packages to python3 in quickstart.rst" 2021-01-01 04:08:24 +00:00
huth
182a6fcff5 Modify port group document for ironic
Add a simple sample about configuring bonding via configdrive,
and it can make user to use port group more easily.

Story: 2008474
Task: 41514

Signed-off-by: huth <428437106@qq.com>
Change-Id: Ic425ecb35bfa173adf72b0ee104d28c6b79cb4b1
2020-12-31 10:34:26 +08:00
likui
378557b7f2 add openstack-python3-wallaby-jobs-arm64 job
This is a non-voting job to validate py3 unittests on ARM64

Change-Id: I7a3a783ddeb5e9b7aaad9ccfb8aeeb7fcc8a1593
Task: 41376
Story: 2007938
2020-12-31 09:06:10 +08:00
Dmitry Tantsur
382a43627e Mark the iSCSI deploy as deprecated in the docs
Also move it to the bottom of the user guide and fix some ancient
bits (more fixing required).

Change-Id: I118d3385110c85cb6e5f1beacb7c5d1887bda616
2020-12-29 17:34:35 +01:00
likui
d99a52f2cf update python packages to python3 in quickstart.rst
since 'train' cycle, we should test python3 by default.

Change-Id: Iadba4098e7ff5b9456fd0224353e55aad73a2b5b
2020-12-27 16:19:03 +08:00
Dmitry Tantsur
06a1d38fc1 Support configdrive when doing ramdisk deploy with redfish-virtual-media
When using Redfish virtual media, it's possible to connect a configdrive
via a free USB slot when the ramdisk deploy is used.

Using Swift as configdrive storage is not supported in this case yet.

Story: #2008380
Task: #41302
Change-Id: Ib847dbfe96072cfe4137388ba88ef133bd7ab186
2020-12-23 18:30:07 +01:00
Zuul
6c9e28dd50 Merge "Inject TLS certificate when using virtual media" 2020-12-19 22:14:12 +00:00
Zuul
f11f330d00 Merge "Support port name" 2020-12-19 20:46:10 +00:00
Zuul
8518f36014 Merge "Include HeartbeatMixin in the ramdisk deploy" 2020-12-18 18:47:27 +00:00