RBAC config options enforce_scope and enforce_new_defaults
were disabled by default in oslo.policy and Ironic had to override
the default value to enable those by default. Now oslo.policy
(4.4.0 onwards[1]) changed the default values[2][3] and enabled
by default for all the services. OpenStack service does not need
to override the default anymore.
NOTE: There is no change in behaviour here, oslo.policy provides the
same configuration that Ironic has overridden till now.
[1] https://review.opendev.org/c/openstack/releases/+/925032
[2] https://review.opendev.org/c/openstack/oslo.policy/+/924283
[3] https://review.opendev.org/c/openstack/requirements/+/925464
Change-Id: I280ae374048b16f1d27a55b09a4d7729de43f469
These are detected as errors since the clean up was done[1] in
the requirements repository.
[1] 314734e938f107cbd5ebcc7af4d9167c11347406
Also remove the note about old pip's behavior because the resolver
in recent pip no longer requires specific order.
Change-Id: I742ea0192398b9e9b78b969fa81f65621d9490de
DMTF now changes their Virtual Media URI to support Systems. Redfish driver now
support this resource to boot, so it is needed that Ironic have a way to use it.
Closes-Bug: #2039458
Change-Id: I66e8edb847e93f96374072525222f05e7561fb07
The VTEP switch support patch merged with a constraint of jsonschema
version 4.19 or above.
Except Debian only currently has 4.10, Centos 9 Stream only has 4.16,
and at present launchpad and the ubuntu mirror list is non-functional.
So in the interest of of packagers, we'll lower the version.
Note: I was able to successfully execute the unit tests with jsonschema
4.0.0 installed in the py3 virtualenv.
Change-Id: Ic3667a7663b7bd5dfad4665321d9c82cc08cc885
pytz will be removed from RHEL/CentOS 10 because of the built-in
zoneinfo[1].
Because the current usage of pytz can be very easily replaced, this
removes the dependency on pytz.
[1] https://issues.redhat.com/browse/RHEL-219
Change-Id: Ia72c528eadeccf6075894ff58477fecade65ad71
Adds basic support for passing OVN VTEP switch metadata to
neutron via Ironic's port.local_link_connection field.
Adds microversion 1.90 to Ironic's API, adding support for
new schema in port.local_link_connection
Bump version of the jsonschema library to ensure consistent
behavior with new schema configurations.
Add documentation warning: This has not been tested as no
Ironic developers have access to the hardware in question.
Closes-bug: #2034953
Co-Authored-By: Austin Cormier <acormier@juniper.net>
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Change-Id: Ie98dc4552ec2ea16db1e2d382aed54ce9dfef41b
After removing the iSCSI deploy and changing ISO parsing code to use
a corresponding library, Ironic no longer executes any commands as root
and it should stay this way.
Change-Id: I47d2bab9b94345fbcf89a2a80028853050a041ea
Adds a redfish-https boot interface, based upon the
redfish-virtual-media boot interface, however substantially copies
some base methods because of simplification offered to use by
putting "attach/detach" logic into how the sushy library handles
the application and reset of a URL as a boot setting.
This feature also increases the requirement for the Sushy library
to version 4.7.0 which includes support to set the HttpBootUri
field in the BMC and automatically unset it as well.
Closes-Bug: #2032380
Change-Id: I991611cd67cb91aea21fc30bbae7cd24409dbbfa
Object create/delete operations translate clearly from swiftclient to
the SDK. Switching the temp URL handling is a little more disruptive but
the result is slightly more centralized and enables key rotation.
Change-Id: I8df2f032224bd5e540139a798a7ab76a1aeebb06
Closes-Bug: #2042493
Adds the 'local-link-connection' and 'parse-lldp' inspection hooks in
the agent inspect interface for processing data received from the
ramdisk at the /v1/continue_inspection endpoint.
Change-Id: I540f03b961b858e8fc00cd4abbc905faa8f0c6c5
Story: #2010275
Eventlet expects to have green versions of every module loaded, including OS. The reasons we originally did not patch os are lost
to time, but there have been many releases of eventlet since, and we should return to a reasonable baseline.
Change-Id: Ia4113124b415bd647e3c984e587828eb5b612eee
Adds storage of the json-rpc port number to the conductor hostname
to enable rpc clients to understand which rpc servies they need to
connect to.
Depends-On: https://review.opendev.org/c/openstack/ironic-lib/+/879211
Change-Id: I6021152c83ab5025a9a9e6d8d24c64278c4c1053
... To not try instantly, but also not to wait forever to retry.
Also, the maximum delay is also now the proper setting to cause
the attempt to exit, and is only set to 10 seconds, with a fairly
tight interval for retries to occur within.
This change also doesn't abort retries for releasing a node lock
and updating a node, both actions if they halt due to the close
out of a task, can be catastrophic to the underlying operation
and state, because internal actions around locking can't be retried
with a long interval, otherwise things break in very bad ways.
Change-Id: I2041e90bb0f7f522bde4338eceda97f0ae8b2c35
This change adds the capability for the ironic-conductor
and standalone service process to transmit timer and counter
metrics to the message bus notifier which may be consumed by
a ceilometer, ironic-prometheus-exporter, or other consumer of
metrics event data on to the message bus.
This functionality is not presently supported on dedicated API
services such as those running as an ``ironic-api`` application
process, or Ironic WSGI application. This is due to the lack of
an internal trigger mechanism to transmit the data in a metrics
update to the message bus and/or notifier plugin.
This change requires ironic-lib 5.4.0 to collect and ship metrics via
the message bus.
Depends-On: https://review.opendev.org/c/openstack/ironic-lib/+/865311
Change-Id: If6941f970241a22d96e06d88365f76edc4683364
One of the major changes in SQLAlchemy 2.0 is the removal
of autocommit support. It turns out Ironic was using this quite
aggressively without even really being aware of it.
* Moved the declaritive_base to ORM, as noted in the SQLAlchemy 2.0
changes[0].
* Console testing caused us to become aware of issues around locking
where session synchronization, when autocommit was enabled, was
defaulted to False. The result of this is that you could have two
sessions have different results, which could results on different
threads, and where one could still attempt to lock based upon prior
information. Inherently, while this basically worked, it was
also sort of broken behavior. This resulted in locking being
rewritten to use the style mandated in SQLAlchemy 2.0 migration
documentation. This ultimately is due to locking, which is *heavily*
relied upon in Ironic, and in unit testing with sqlite, there are
no transactions, which means we can get some data inconsistency
in unit testing as well if we're reliant upon the database to
precisely and exactly return what we committed.[1]
* Begins changing the query.one()/query.all() style to use explicit
select statements as part of the new style mandated for migration
to SQLAlchemy 2.0.
* Instead of using field label strings for joined queries, use the
object format, which makes much more sense now, and is part of
the items required for eventual migration to 2.0.
* DB queries involving Traits are now loaded using SelectInLoad
as opposed to Joins. The now deprecated ORM queries were quietly
and silently de-duplicating rows and providing consistent sets
from the resulting joined table responses, however putting much
higher CPU load on the processing of results on the client.
Prior performance testing has informed us this should be a minimal
overhead impact, however these queries should no longer be in
transactions with the Database Servers which should offset the
shift in load pattern. The reason we cannot continue to deduplicate
locally in our code is because we carry Dict data sets which cannot
be hashed for deduplication. Most projects have handled this by
treating them as Text and then converting, but without a massive
rewrite, this seems to be the viable middle ground.
* Adds an explict mapping for traits and tags on the Node object
to point directly to the NodeTrait and NodeTag classes. This
superceeds the prior usage of a backref to make the association.
* Splits SQLAlchemy class model Node into Node and NodeBase, which
allows for high performance queries to skip querying for ``tags``
and ``traits``. Otherwise with the afrormentioned lookups would
always execute as they are now properties as well on the Node
class. This more common of a SQLAlchemy model, but Ironic's model
has been a bit more rigid to date.
* Adds a ``start_consoles`` and ``start_allocations`` option to the
conductor ``init_host`` method. This allows unit tests to be
executed and launched with the service context, while *not* also
creating race conditions which resulted in failed tests.
* The db API ``_paginate_query`` wrapper now contains additional
logic to handle traditional ORM query responses and the newer style
of unified query responses. Due to differences in queries and handling,
which also was part of the driver for the creation of ``NodeBase``,
as SQLAlchemy will only create an object if a base object is referenced.
Also, by default, everything returned is a tuple in 1.4 with the
unified interface.
* Also modified one unit test which counted time.sleep calls, which is
a known pattern which can create failures which are ultimately noise.
Ultimately, I have labelled the remaining places which SQLAlchemy
warnings are raised at for deprecation/removal of functionality,
which needs to be addressed.
[0] https://docs.sqlalchemy.org/en/14/changelog/migration_20.html
[1] https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#transaction-isolation-level-autocommit
Change-Id: Ie0f4b8a814eaef1e852088d12d33ce1eab408e23
Sushy 4 includes enhancements including support for hardware Ironic
should work with in Zed.
Story: #2009865
Task: #44548
Change-Id: Ib82bd4d1442bf7d9b135d1c1553c39cfef87548a
Ironic supports enabling the audit middleware. This change adds
the option for the middleware to the ironic.conf file generated by
oslo-config-generator.
Depends-on: https://review.opendev.org/804316
Change-Id: Ic7adb755f47ea65fe975dfbc7cca905a376d728e
Grenade, for some confusing reason, creates a separate network,
and uses that for upgrade testing as opposed to the original network
the VMs were bound to. If Julia's memory is correct, this was for
multinode upgrade testing.
Anyway, When in UEFI mode, it appears that the TFTP packets
don't get tracked nor cross the boundrary. We likley need to
explicitly address this, but first, lets get the job working as
it was and can then update it.
Also, update requirements because markupsafe removed soft_unicode
method taht was deprecated since a while. Jinja2 started using the
new soft_str method since version 3.0.0
Change-Id: Iaebe966569962b0d3d43774d57b570469479f159
Redfish has become an established technology with growing popularity.
By enabling it by default we make it easier for operators to use Ironic,
especially its advanced features, out of box.
Bump stevedore to match sushy.
Change-Id: I94dd4066ac598a7e5c2e0812e1ff286de5d164a2
Update minimum required versions of python packages.
The updates is based on crosscheck requirements from required
packages:
oslo-log 4.3.0 depends on oslo.context>=2.20.0
oslo-config 6.8.0 depends on rfc3986>=1.2.0
oslo-config 6.8.0 depends on requests>=2.18.0
oslo-log 4.3.0 depends on pbr>=3.1.1
oslo-log 4.3.0 depends on oslo.serialization>=2.25.0
oslo-policy 3.7.0 depends on oslo.context>=2.22.0
Also update minimum versions of ddt and stestr to avoid compatibility
issues with recent Python versions (3.8 and higher).
Change-Id: I34b558f4e7b93618130af1ba2250f6ab8575d4d6
Config drive is processed by the deploy interface and
written to the disk during post install of the deployment.
The config drive is written to /var/lib/cloud/seed/config_drive
on the disk. cloud-init should be able to process read config
drive contents and take actions on firstboot.
Change-Id: Id5a949e777e242d2f20966ec045ac319c61636ba
oslo.db 9.1.0 extends capabilities to handle changes
in duplicate key error information introduced since
MySQL 8.0.19 and fixes issue arose during inspection
of node created with idrac driver, inspect interface
as idrac-redfish.
Story: 2008901
Task: 42468
Change-Id: Ic18e485d4eb14566554d25379a719669ac362d70
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project.
Unit tests will be added/removed where applicable.
Tenacity [1] is a fork of retrying, but has improved the
interface and extensibility.
Our end goal here is removing the retrying package from our
requirements.
Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did.
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg.
- For retries that check a result, tenacity will raise if the retried
function raises, whereas retrying retried on all exceptions.
[1] https://github.com/jd/tenacity
Co-Authored-By: Dmitry Tantsur <dtantsur@protonmail.com>
Co-Authored-By: Riccardo Pittau <elfosardo@gmail.com>
Story: #1635390
Task: #10528
Change-Id: Ie5eb3ddc196505e8f58ed14de9952284598586fb
This version of oslo.policy includes fixes that ensure the Enforcer only
modifies copies of the rules, making it safer in environments that run
tests in parallel where the Enforcer is configured differently.
Change-Id: I8d7a06558cbf073487707fc33219e43beb5fc043
Adds the status upgrade check for the JSON to YAML migration
effort and updates the documentation where it seems appropriate
to move from "policy.json" to "policy.yaml"
Mostly shamelessly copied from https://review.opendev.org/#/c/748059/
however is in-line with ironic's configuration and patching methods.
Related Blueprint: policy-json-to-yaml
Change-Id: I1d5b3892451579ebfd4d75a0f7185e0ef3c984c8
This patch removes the dependency from ironicclient to
communicate with neutron in favor of openstacksdk.
Also:
* Use import keystoneauth1.loading as ks_loading accross
the project.
* Refactor to have one 'get_client' function, removing the
'_get_config_client' method. Setting config_client=True
when calling 'get_client' returns a client using auth
options values from conf parameters.
Depends-On: https://review.opendev.org/735601
Change-Id: Ib6c0fa2acfc33deb9c5b36ae724d5d8304d1dd29
It's very confusing that we use username/password everywhere, except
for [json_rpc]. Just use the standard options.
Also the version if keystoneauth is bumpted to one that supports
http_basic.
Change-Id: Icc834c3f8febd45c2548314ee00b85a7f9cebd2c