368 Commits

Author SHA1 Message Date
Riccardo Pittau
06124bae73 Update requirements
Update minimum required versions of python packages.
The updates is based on crosscheck requirements from required
packages:
oslo-log 4.3.0 depends on oslo.context>=2.20.0
oslo-config 6.8.0 depends on rfc3986>=1.2.0
oslo-config 6.8.0 depends on requests>=2.18.0
oslo-log 4.3.0 depends on pbr>=3.1.1
oslo-log 4.3.0 depends on oslo.serialization>=2.25.0
oslo-policy 3.7.0 depends on oslo.context>=2.22.0

Also update minimum versions of ddt and stestr to avoid compatibility
issues with recent Python versions (3.8 and higher).

Change-Id: I34b558f4e7b93618130af1ba2250f6ab8575d4d6
2021-07-13 14:37:28 +02:00
Zuul
7d6e936cfe Merge "Add support for configdrive in anaconda interface" 2021-07-06 17:21:06 +00:00
Ghanshyam Mann
34337f75fb Fix oslo policy DeprecatedRule warnings
Since 3.7.0, oslo policy started the DeprecationWarning[1] if
deprecated_reason and deprecated_since param are not passed
in DeprecatedRule or they are passed in RuleDefault object.

These warnings are logged for every test which increase the
log size and sometime can full the log buffer and fail the
job.

This fixes the 1431 warnings
- https://zuul.opendev.org/t/openstack/build/cace355f7c0d4cbcb1df7856a03194d7/log/job-output.txt#1029

[1] https://github.com/openstack/oslo.policy/blob/3.7.0/oslo_policy/policy.py#L1538

Change-Id: I98d20aea80d547e1f301c150564f732619c9e744
2021-07-05 10:57:21 -05:00
Arun S A G
4ac8340fb6 Add support for configdrive in anaconda interface
Config drive is processed by the deploy interface and
written to the disk during post install of the deployment.

The config drive is written to /var/lib/cloud/seed/config_drive
on the disk. cloud-init should be able to process read config
drive contents and take actions on firstboot.

Change-Id: Id5a949e777e242d2f20966ec045ac319c61636ba
2021-07-04 10:36:18 -07:00
Kamlesh Chauvhan
e3a4c7d77d Upgrade oslo.db version
oslo.db 9.1.0 extends capabilities to handle changes
in duplicate key error information introduced since
MySQL 8.0.19 and fixes issue arose during inspection
of node created with idrac driver, inspect interface
as idrac-redfish.

Story: 2008901
Task: 42468
Change-Id: Ic18e485d4eb14566554d25379a719669ac362d70
2021-06-28 13:24:22 +00:00
Dmitry Tantsur
f2e8c031cc Switch to JSON RPC from ironic-lib
Change-Id: I8b438861780c85faae7ff18646960723a1fd9876
2021-03-10 16:48:07 +01:00
Zuul
3f2431001e Merge "Update oslo.policy requirement to version 3.6.2" 2021-03-01 17:25:38 +00:00
Dmitry Tantsur
6e09a008ab Replace pysendfile with os.sendfile
Change-Id: I1ef33d41fd3784f55929fa6a086fca0c335212e5
2021-02-18 10:16:57 +01:00
Boden R
b0607a2668 Replace retrying with tenacity
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project.
Unit tests will be added/removed where applicable.

Tenacity [1] is a fork of retrying, but has improved the
interface and extensibility.
Our end goal here is removing the retrying package from our
requirements.

Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did.
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg.
- For retries that check a result, tenacity will raise if the retried
function raises, whereas retrying retried on all exceptions.

[1] https://github.com/jd/tenacity

Co-Authored-By: Dmitry Tantsur <dtantsur@protonmail.com>
Co-Authored-By: Riccardo Pittau <elfosardo@gmail.com>
Story: #1635390
Task: #10528
Change-Id: Ie5eb3ddc196505e8f58ed14de9952284598586fb
2021-02-15 17:11:33 +01:00
Lance Bragstad
94ba2641e3 Update oslo.policy requirement to version 3.6.2
This version of oslo.policy includes fixes that ensure the Enforcer only
modifies copies of the rules, making it safer in environments that run
tests in parallel where the Enforcer is configured differently.

Change-Id: I8d7a06558cbf073487707fc33219e43beb5fc043
2021-02-05 21:03:39 +00:00
Lance Bragstad
6ea73bdfbb Bump oslo.log requirement to 4.3.0
This ensures we have the WALLABY marker available for deprecation
noticies.

Change-Id: Ic91f34a90fdd0c22c06c669551eaf6f9a398956b
2021-01-19 10:59:48 -08:00
Julia Kreger
2404d486ac Policy json to yaml migration
Adds the status upgrade check for the JSON to YAML migration
effort and updates the documentation where it seems appropriate
to move from "policy.json" to "policy.yaml"

Mostly shamelessly copied from https://review.opendev.org/#/c/748059/
however is in-line with ironic's configuration and patching methods.

Related Blueprint: policy-json-to-yaml

Change-Id: I1d5b3892451579ebfd4d75a0f7185e0ef3c984c8
2021-01-04 13:40:54 -08:00
Iury Gregory Melo Ferreira
19d97e6562 Fix lower-constraints for Ubuntu Focal
Change-Id: Id3cc2d1b619790813b94b0cfd52ff6590aff060c
2020-09-11 04:23:12 +00:00
Harald Jensås
ff00047b4a Switch Ironic to openstacksdk for Neutron
This patch removes the dependency from ironicclient to
communicate with neutron in favor of openstacksdk.

Also:
 * Use import keystoneauth1.loading as ks_loading accross
   the project.
 * Refactor to have one 'get_client' function, removing the
   '_get_config_client' method. Setting config_client=True
   when calling 'get_client' returns a client using auth
   options values from conf parameters.

Depends-On: https://review.opendev.org/735601
Change-Id: Ib6c0fa2acfc33deb9c5b36ae724d5d8304d1dd29
2020-08-19 10:48:17 +02:00
Zuul
3670be1283 Merge "Deprecate http_basic_username and http_basic_password in [json_rpc]" 2020-07-28 19:14:00 +00:00
Dmitry Tantsur
74e9e1d82a Deprecate http_basic_username and http_basic_password in [json_rpc]
It's very confusing that we use username/password everywhere, except
for [json_rpc]. Just use the standard options.

Also the version if keystoneauth is bumpted to one that supports
http_basic.

Change-Id: Icc834c3f8febd45c2548314ee00b85a7f9cebd2c
2020-07-24 11:51:41 +02:00
Zuul
39a38ecf4e Merge "Add an option to choose the hash ring algorithm" 2020-07-23 17:31:00 +00:00
Dmitry Tantsur
9189b4bb26 Add an option to choose the hash ring algorithm
MD5 is not available in FIPS mode, we need a way to use something else.

Change-Id: Ie6e09ac66028cbe18717a7ea7a4c23730e3cb642
2020-07-22 18:27:19 +02:00
Zuul
b5ae75a406 Merge "Use native oslo.concurrency execution timeout in ipmitool" 2020-07-22 15:58:19 +00:00
Steve Baker
44cc6dd792 Add wsme core types, remove WSME
The header for the file types.py denotes its dual-licensed status as
MIT with copyright to the original WSME authors, plus apache licensed
as part of Ironic.

Story: 1651346
Task: 10551

Change-Id: I986cc4a936c8679e932463ff3c91d1876a713196
2020-07-14 10:34:13 +12:00
Dmitry Tantsur
737076fae2 Use native oslo.concurrency execution timeout in ipmitool
This change replaces custom Popen-based code with the new argument
(backed by the corresponding stdlib argument).

Story: #2004449
Task: #40283
Change-Id: I6840b1caffd272ef12ab2b259a02376ec185bc3f
2020-07-06 18:35:44 +02:00
Steve Baker
c6112b01c3 Enable Basic HTTP authentication middleware
When the config option ``auth_strategy`` is set to ``http_basic`` then
non-public API calls require a valid HTTP Basic authentication header
to be set. The config option ``http_basic_auth_user_file`` defaults to
``/etc/ironic/htpasswd`` and points to a file which supports the
Apache htpasswd syntax[1]. This file is read for every request, so no
service restart is required when changes are made.

The only password digest supported is bcrypt, and the ``bcrypt``
python library is used for password checks since it supports ``$2y$``
prefixed bcrypt passwords as generated by the Apache htpasswd utility.

To try HTTP basic authentication, the following can be done:

* Set ``/etc/ironic/ironic.conf`` ``DEFAULT`` ``auth_strategy`` to ``http_basic``
* Populate the htpasswd file with entries, for example:
  ``htpasswd -nbB myName myPassword >> /etc/ironic/htpasswd``
* Make basic authenticated HTTP requests, for example:
  ``curl --user myName:myPassword http://localhost:6385/v1/drivers``

[1] https://httpd.apache.org/docs/current/misc/password_encryptions.html

Change-Id: I7b89155d8bbd2f48e186c12adea9d6932cd0bfe2
Story: 2007656
Task: 39825
Depends-On: https://review.opendev.org/729070
2020-06-05 01:15:08 +12:00
Hervé Beraud
f266baffc3 Cap jsonschema 3.2.0 as the minimal version
Previous versions of jsonschema (<3.2.0) doesn't support python 3.8 [1].
Python 3.8 is part of the victoria supported runtimes [2] so we now force
to use jsonschema version 3.2.0 to avoid issues, remove ambiguity and ensure
that everything works with python 3 in general.

[1] https://github.com/Julian/jsonschema/pull/627
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: I929a573714e0ce8218e0820babc16f9862c4a36e
2020-05-26 21:50:19 +02:00
Iury Gregory Melo Ferreira
6397777ea8 Stop configuring install_command in tox.
Currently, we are overriding 'install_command' to use 'pip'. This is
considered poor behavior and 'python -m pip' should be used instead:

https://snarky.ca/why-you-should-use-python-m-pip/

It turns out that this is the the default value provided by tox:

https://tox.readthedocs.io/en/latest/config.html#conf-install_command

So we can remove the line and simply use the default value.

openstack-discuss thread
http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014237.html

Bumping  in requirements:
-alembic to 0.9.6
-keystoneauth1 to 3.18

Bumping libs in  in l-c:
-alembic to 0.9.6
-keystoneauth1 to 3.18
-pika to 0.10.0 (necessary for installation)

Change-Id: I059e841ab891ba0451d0423065e9727c8c77f337
2020-04-21 12:52:07 +02:00
Dmitry Tantsur
1d9c8283b8 Make oslo.i18n an optional dependency
In an ideal world we would have all messages translated to at least major
languages. In reality, ironic has never boasted a great set of translation,
and according to Zanata only 2.94% of messages are translated for ironic
in Train (1.33% for ironic-inspector). Given that ironic-python-agent is
not translated at all, using i18n by default seems an overkill.

Change-Id: I12287c6b7208af487cc9eee825ae7f38ec707d91
2020-04-02 14:10:21 +02:00
Dmitry Tantsur
9f93694b9a Make oslo.reports an optional dependency
It is only required for one specific feature, let people install it
if they need it.

This change is a part of the major effort to reduce the number of
ironic dependencies.

Change-Id: Ia45ce1d573c89f583d641be3d37d1c127e6345bc
2020-04-02 11:52:04 +02:00
Riccardo Pittau
78c121a5d7 Stop using six library
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.

Change-Id: Ib546f16965475c32b2f8caabd560e2c7d382ac5a
2019-12-23 09:38:25 +01:00
Vladyslav Drok
68a5e31a54 Mock out the correct greenthread sleep method
It seems at some point oslo_service loopingcall started using
eventletutils from oslo_utils to sleep during the loopingcall
retries, and some untittests started taking up to 40 seconds
to complete. This change mocks out the correct method offering
significant speedup to unittests' run time.

The EventletEvent class is introduced to eventletutils in version
3.38.0 so lower constraints are bumped as well.

Change-Id: Id7e6ff2a4748b5301e2259acdc760ac7f56b96c3
2019-11-20 17:52:33 +01:00
Steve Baker
7ebad2e344 Allow vendor_data to be included in a configdrive dict
configdrive can contain a vendor_data2.json file containing key/value
pairs injected by nova's vendordata mechanism[1].

This change lets Ironic accept a vendor_data key when configdrive is
provided as json, allowing parity with nova.

This change requires an openstacksdk release 0.37.0

[1] https://www.madebymikal.com/nova-vendordata-deployment-an-excessively-detailed-guide/

Change-Id: Id990b970619a113c5d5ead47fb550870d91b5e04
Task: 36756
Story: 2006597
Blueprint: nova-less-deploy
2019-10-31 08:55:39 +13:00
Riccardo Pittau
61c8364703 Raising minimum version of oslo.db
oslo.db was not compatible with Python 3.7 until version 4.40.0
because Python 3.7 makes "async" a keyword [1].
To prevent bad surprises, this patch increases the minumum version
required for oslo.db to 4.40.0.

[1] https://review.opendev.org/574833

Change-Id: I3271fd2d7d81261db6da079e55fb87656c833686
2019-10-07 14:50:58 +02:00
Zuul
e518bebe03 Merge "Support power state change callbacks to nova using ksa_adapter" 2019-08-24 02:20:13 +00:00
Surya Seetharaman
d693d4c06c Support power state change callbacks to nova using ksa_adapter
Add power state change callbacks of an instance to nova by
performing API requests. Whenever there is a change in the
power state of a physical instance (example a "power on"
or "power off" IPMI command is issued or the periodic
``_sync_power_states`` task detects a change in power state)
ironic will create and send a ``power-update`` external event
to nova using which nova will update the power state of the
instance in its database. By conveying the power state changes
to nova, ironic becomes the source of truth thus preventing
nova from forcing wrong power states on the instance during
the nova-ironic periodic sync. It also adds the possibility of
bringing up/down a physical instance through the ironic API
even if it was put down/up through the nova API.

Note that ironic only sends requests to nova if the target
power state is either "power on" or "power off". Other error
states will be ignored. In cases where the power state change
is originally coming from nova, the event will still be
created and sent to nova and on the nova side it will be a
no-op with a debug log saying the node is already powering on/off.

NOTE: Although an exclusive lock (task_manager.upgrade_lock()
method) is used when calling the nova API to send events,
there can still be a race condition if the nova-ironic power sync
happens to happen a nano-second before the power state change
event is received from ironic in which case the nova state will
be forced on the node.

Credit for introducing ksa adapter: Eric Fried <openstack@fried.cc>
Depends-On: https://review.opendev.org/#/c/645611/
Part of blueprint nova-support-instance-power-update
Story: 2004969
Task: 29424

Change-Id: I6d105524e1645d9a40dfeae2850c33cf2d110826
2019-08-12 19:29:57 +02:00
Dmitry Tantsur
d6774c294d Use openstacksdk for accessing ironic-inspector
Change-Id: Ibe4f6bf0b38364b5dd214e6c7e58d45a4d71ffdf
2019-07-31 15:23:39 +02:00
Riccardo Pittau
2d5f0c8b91 Bump keystonauth and warlock versions
openstacksdk 0.25.0 requires keystoneauth1>=3.11.0
python-glanceclient 2.8.0 requires warlock!=1.3.0,<2,>=1.0.1

Change-Id: Iaf80327538d2395fe61bd0e77743d1b6021a66f7
2019-07-15 18:08:42 +02:00
Riccardo Pittau
f5dbf8ba0c Switch to use exception from ironic-lib
The exception modules in ironic and ironic-lib contain the same
almost identical class IronicException.
With this patch we directly use the one in ironic-lib.

Updating requirements and lower-constraints to use compatible
version of ironic-lib.

Also deprecating duplicated fatal_exception_format_errors
option.

Change-Id: I1ce0d12d912020346425fd658d3b1807607455a4
Story: 1626578
Task: 10515
2019-06-11 12:03:44 +02:00
Zuul
aae98fbedc Merge "Publish baremetal endpoint via mdns" 2019-05-28 12:18:41 +00:00
pengyuesheng
2c42e3cdf3 Blacklist python-cinderclient 4.0.0
This release of the Cinder client broke support for the v3
volume-transfer APIs unless microversion 3.55 or higher was requested.
depend on https://review.opendev.org/#/c/587877/

Change-Id: I12f6d4f540a0a09c6609e14c615dcd11f191660a
2019-05-27 14:54:54 +08:00
Dmitry Tantsur
c36a01a439 Publish baremetal endpoint via mdns
This change adds an option to publish the endpoint via mDNS on start
up and clean it up on tear down.

Story: #2005393
Task: #30383
Change-Id: I55d2e7718a23cde111eaac4e431588184cb16bda
2019-05-23 17:11:50 +02:00
Dmitry Tantsur
84dfc151ea Uncap jsonschema in requirements
It's no longer capped in global-requirements, so the requirements
check no longer passes on ironic. This patch fixes it.

Change-Id: Iefbb2e935fda9b9aeb332d04715b98840a581c86
2019-04-10 11:36:58 +02:00
Dmitry Tantsur
3e1e0c9d5e Allow building configdrive from JSON in the API
Extend the API with the ability to build config drives from meta_data,
network_data and user_data, where meta_data and network_data are JSON
objects, and user_data is either a JSON object, a JSON array or
raw contents as a string.

This change uses openstacksdk (which is already an indirect dependency)
for building config drives.

Change-Id: Ie1f399a4cb6d4fe5afec79341d3bccc0f81204b2
Story: #2005083
Task: #29663
2019-03-05 19:14:56 +01:00
Mark Goddard
ec2f7f992e Deploy templates: API & notifications
Adds deploy_templates REST API endpoints for retrieving, creating,
updating and deleting deployment templates. Also adds notification
objects for deploy templates.

Bumps the minimum WSME requirement to 0.9.3, since the lower constraints
job was failing with a 500 error when sending data in an unexpected
format to the POST /deploy_templates API.

Change-Id: I0e8c97e600f9b1080c8bdec790e5710e7a92d016
Story: 1722275
Task: 28677
2019-03-04 10:30:16 +00:00
Mark Goddard
db94e35745 Fix pysendfile requirement marker
Currently, any patch that changes one of the requirements files is
failing the requirements-check job, with the following error:

Requirement(package='pysendfile', location='', specifiers='>=2.0.0',
markers='', comment='# MIT', extras=frozenset()) 'markers': '' does not
match "sys_platform!='win32'"

Could not find a global requirements entry to match package {}. If the
package is already included in the global list, the name or platform
markers there may not match the local settings.

The problem is that a "sys_platform!='win32'" marker was added [1] to
the pysendfile requirement in the requirements repo, which needs to be
propagated to the requirements of all repos using the package.

This change adds the required marker to requirements.txt.

[1] https://review.openstack.org/#/c/639084/

Change-Id: Ibd7656cbcfbf204930f9c8562f79ab92bd4aefb5
Story: 2005096
Task: 29697
2019-02-28 09:56:26 +00:00
whoami-rajat
1f1a2c3bd8 Add ironic-status upgrade check command framework
This adds basic framework for ironic-status upgrade
check commands. For now it has only "check_placeholder"
check implemented.
Real checks can be added to this tool in the future.

Change-Id: I7d5f018656322b92c663a2adaaf6330a55c00fb8
Story: 2003657
Task: 26133
2018-11-01 19:58:13 +00:00
Yolanda Robla
f532ec4f70 Enable configuration of conversion flags for iscsi
Add an option conv_flags, that will be passed at ironic.conf into
iscsi section. This will pass the conv_flags option to the iscsi
disk utils, and this will modify the behaviour of the image
copy to disk. It can be used for things as optimization (adding
an sparse flag)

Change-Id: Ia9e11dda35bb06e5b37b00e1b8fb42f9267a95d6
Story: #2004124
Task: #27573
2018-10-19 16:59:29 +02:00
Michael Turek
8f89954f9a Pass prep_boot_part_uuid to install_bootloader for ppc64* partition images
This patch gets the PReP Boot partition UUID when performing a local boot
partition image deployment on ppc64* hardware.

Depends-On: I8f9748dd58146bfb2411c229b02969e0faf18222
Change-Id: I2bc9f13ec605de7b7b96d96a1a4edebee0af76dc
Story: #1749057
Task: #22995
2018-07-31 19:52:29 +00:00
Dmitry Tantsur
31cc654fc1 Bump osprofiler minimum requirement to 1.5.0
Ironic cannot even start with 1.4.0. The lower-constraints job works
because it actually uses upper-constraints.

Change-Id: Id34909b9fb159cd2bb06f1512908db4836055c29
Story: #2002900
Task: #22864
2018-07-06 16:38:12 +02:00
Doug Hellmann
40a3fea8aa uncap eventlet
We will manage the eventlet version using constraints now. See the
thread starting at
http://lists.openstack.org/pipermail/openstack-dev/2018-April/129096.html
for more details.

Change-Id: I192ab9b96391be24eb0b51240fd6ddc9434b4283
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-11 11:52:24 -04:00
OpenStack Proposal Bot
2051f14b2b Updated from global requirements
Change-Id: I6931ca09643d7d671a37930f8cb68349447b919c
2018-03-26 08:04:33 +00:00
OpenStack Proposal Bot
af02064a8f Updated from global requirements
Change-Id: Ic3b45e088fdcff6aaeaf2ff2591c55dfa68ac3eb
2018-03-14 05:30:56 +00:00
OpenStack Proposal Bot
cd3c0112bd Updated from global requirements
Change-Id: I7a742deaccd68682421233c68bd87e77b7336aa6
2018-03-06 12:47:06 +00:00