39 Commits

Author SHA1 Message Date
Dmitry Tantsur
0d4ae976c2
Support several API and Inspector URLs
Allows nodes with a single IP stack to be deployed from a dual-stack
Ironic.

Detecting advertised address and usable Ironic URLs are done completely
independently which does open some space for a misconfiguration. I hope
it's not likely in the reality, especially since this feature is
targetting advanced standalone users.

Change-Id: Ifa506c58caebe00b37167d329b81c166cdb323f2
Closes-Bug: #2045548
2024-01-09 16:43:23 +01:00
Julia Kreger
5ed520df89 Handle the node being locked
If the node is locked, a lookup cannot be performed when an agent
token needs to be generated, which tends to error like this:

  ironic_python_agent.ironic_api_client [-] Failed looking up node
  with addresses '00:6f:bb:34:b3:4d,00:6f:bb:34:b3:4b' at
  https://172.22.0.2:6385. Error 409: Node
  c25e451b-d2fb-4168-b690-f15bc8365520 is locked by host 172.22.0.2,
  please retry after the current operation is completed..
  Check if inspection has completed.

Problem is, if we keep pounding on the door, we can actually worsen
the situation, and previously we would just just let tenacity
retry.

We will now hold for 30 seconds before proceeding, so we have
hopefully allowed the operation to complete.

Also fixes the error logging to help human's sanity.

Change-Id: I97d3e27e2adb731794a7746737d3788c6e7977a0
2023-08-22 16:47:28 -07:00
Riccardo Pittau
64ffd2ee80 Remove oslo.serialization dependency
Use pure json instead of jsonutils.

Borrow encode function from oslo.serialization to be used in the
utils module.

Change-Id: Ied9a2259a4329a86b4f0853bd1fb187563c0a036
2022-06-17 09:37:35 +02:00
Dmitry Tantsur
3251d7b641 Remove runtime dependency on pbr
Pbr is a very heavy package to depend on. It requires git-core, which is
16 MiB on my Fedora. We only use it to detect the version, which can be
done without pbr using a much lighter importlib_metadata.

Copied from https://review.opendev.org/c/openstack/osprofiler/+/739379

Change-Id: I5f434e6bfde6f645804941f3a36d5458a28270e7
2021-04-26 09:16:34 +02:00
Dmitry Tantsur
53dbc87a35 Correctly decode error messages from ironic API
Knowing a status code is simply not enough for debugging.

Change-Id: If1d3f182ab028948ff05aea7e8024d4e7bc3d53c
2020-12-07 18:59:08 +01:00
Jay Faulkner
80575566b1 Allow manual setting of Ironic API Version
Typically, the Ironic API client in IPA will autodetect the API version
based on the output of a GET of the root of the API. If for some reason
this API endpoint is restricted, or the operator wishes to limit the
Ironic API version IPA uses, they can now set CONF.ironic_api_version to
avoid autodetection and force a version.

Change-Id: Ib96a1057792f45f2e4554671e32c436140463ee8
2020-10-23 15:38:42 +00:00
Zuul
11a87365fb Merge "Generate a TLS certificate and send it to ironic" 2020-09-23 12:14:38 +00:00
Dmitry Tantsur
021e0a6a46 Generate a TLS certificate and send it to ironic
Adds a new flag (on by default) that enables generating a TLS
certificate and sending it to ironic via heartbeat. Whether
ironic supports auto-generated certificates is determined by
checking its API version.

Change-Id: I01f83dd04cfec2adc9e2a6b9c531391773ed36e5
Depends-On: https://review.opendev.org/747136
Depends-On: https://review.opendev.org/749975
Story: #2007214
Task: #40604
2020-09-11 17:46:52 +02:00
Dmitry Tantsur
471666905c Replace oslo's loopingcall with tenacity
The latter has a more natural API and does not have a hard requirement
of eventlet. It is already a dependency of ironic-lib.

Change-Id: I68de9e989af137b34c19bbaf9b7c0a5ba6e1d4e3
2020-09-01 11:53:25 +02:00
Julia Kreger
f670f704f3 Clarify connection error on heartbeats
Heartbeat connection errors are often a sign of a transitory
network failures which may resolve themselves. But an operator
looking at the screen doesn't necessarilly know that.

They don't understand that there could have been a network
failure, or a misconfiguration that caused the connectivity
failure and soft of kind of default to "well it failed"
without further clarification.

As such, this patch adds explicit catching of the requests
ConnectionError exception and rasies a new internal error
with a more verbose error message in that event to provide
operators with additional clarity.

Change-Id: I4cb2c0d1f577df1c4451308bd86efa8f94390b0c
Story: 2008046
Task: 40709
2020-08-20 13:45:47 -07:00
Vladyslav Drok
ba6ca246f5 Add possibility to pass global request ID
It can be done via ipa-global-request-id kernel commandline parameter.

Story: 2007681
Task: 39792
Change-Id: I6f544327d310c976a1625cfb411947591867882a
2020-08-12 15:21:08 +03:00
Zuul
3e938b6fcc Merge "Support changing the protocol part of callback_url to https" 2020-08-10 14:59:51 +00:00
Dmitry Tantsur
353d09c3b0 Support changing the protocol part of callback_url to https
Adds a new kernel parameter for manual configuration and also creates
foundation for automatic TLS support later.

Change-Id: If341c3a8a268fc8cab6bd6be04b12ca32b31c8d8
Story: #2007214
Task: #40619
2020-08-06 15:14:31 +02:00
Julia Kreger
5eab9bced6 Fix TypeError on agent lookup failure
Agent lookups can fail as we presently use logging.exception,
better known in our code as LOG.exception, which can also generate
other fun issues on journald based systems where additional errors
could be raised resulting in us being unable to troubleshoot the
the actual issue.

Because of the mis-use of LOG.exception and the default behavior
of the backoff retry handler, the retry logic was also not
functional as any error no matter how small caused IPA to
just exit.

Change-Id: Ic4608b7c6ff9773d1403926efb3d59869c71343b
Story: 2007968
Task: 40465
2020-08-04 20:43:02 -07:00
Riccardo Pittau
d5d62c8dbf Use unittest mock from standard library
Drop the third party mock library to use unittest mock from
standard library.

Change-Id: Ib64b661572e4869a24865c02a6c84a6603930394
2020-04-06 14:35:50 +02:00
Julia Kreger
af5f05a0ee Agent token support
Adds support to the agent to receive, store, and return
that token to ironic's API, when supported.

This feature allows ironic and ultimately the agent to
authenticate interactions, when supported, to prevent
malicious abuse of the API endpoint.

Sem-Ver: feature
Change-Id: I6db9117a38be946b785e6f5e75ada1bfdff560ba
2020-03-12 10:35:17 -07:00
Sam Betts
903ec3ff12 Include IPA Version during heartbeat
In order for Ironic to know what parameters can be sent to IPA commands,
Ironic needs to know which version of IPA it is talking to.  This patch
adds a new node heartbeat parameter agent_version which will carry the IPA
version information to Ironic.

Change-Id: I27e3311accf3a113a48a73df372ed46ff50c7e22
Partial-Bug: #1602265
Depends-On: I400adba5d908b657751a83971811e8586f46c673
2017-12-07 12:16:06 +00:00
Julian Edwards
f57cbccf8b Prevent tests' unmocked access to utils.execute()
This change introduces a new base test class that mocks out
utils.execute and forces an exception if it gets called.
This has rooted out many tests that were doing this as a side effect of
calling other functions, doing things like modprobe and running iscsi
on the host's actual machine.

The tests are all now appropriately patched in places where this was
happening, and the new base class permanently prevents this from
accidentally happening again.

If you really want to call utils.execute() then you need to re-mock it
in your unit test.

Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-05-15 10:48:43 +10:00
John L. Villalovos
1695cb18c2 Add missing 'autospec' argument to mock.patch
Add missing 'autospec' keyword argument to mock.patch and
mock.patch.object calls. Use 'autospec=True' except for a few cases
where it fails because the mocked function is a @classmethod and it
doesn't work. In that case explicity set it to 'autospec=False'

Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 10:04:19 -07:00
Derek Higgins
9f5f664080 Advertise the correct address when using IPv6
Parse the output of "ip route get $IP" taking
IPv6 into consideration. Also wrap the IP address
in square brackets if it is IPv6.

Change-Id: Ifc44e5aa3c5b814b6ceba04461bb68fe1d75c22b
Closes-Bug: #1650533
2017-01-11 11:00:56 +00:00
Luong Anh Tuan
ab41106cf6 Python 3 Compatible JSON
In order to be really python3 compatible, the json lib was replaced
with oslo.serialization(1.10 or newer) module jsontuils since it's
the recommended migration to python3 guide.

https://wiki.openstack.org/wiki/Python3#Serialization:_base64.2C_JSON.2C_etc.

Change-Id: I2d8b62e642aba4ccd1b70be7e9b3784a95a6743d
Closes-Bug: #1629068
2016-11-16 08:19:51 +00:00
Pavlo Shchelokovskyy
b033bfd933 Remove old lookup/heartbeat from IPA
Lookup/Heartbeat via vendor passthru was deprecated in Newton.

This patch removes the corresponding functionality from IPA,
and also removes handling of 'ipa-driver-name' kernel parameter,
as it was only used in code related to old passthru.

Change-Id: I2c7989063ab3e4c0bae33f05d6d2ed857a2d9944
Closes-Bug: #1640533
2016-11-09 16:34:44 +00:00
Moshe Levi
fe3b630360 Add vendor, product to interface information
This patch updates the interfaces JSON to include
vendor and  product
u'interfaces': [
    {
        u'mac_address': u'00:0c:29:8c:11:b1',
        u'name': u'eth0',
        u'ipv4_address': None,
        u'switch_chassis_descr': None,
        u'switch_port_descr': None,
        u'lldp': None,
        u'vendor': u'0x15b3',
        u'product': u'0x1014,
    }]

Co-Authored-By: yogananth subramanian <ysubrama@redhat.com>

Closes-Bug: #1611856

Change-Id: I8fa58dde29601abee959f74b69b692ed9eaffb94
2016-09-21 10:14:26 -04:00
Julia Kreger
fb1cbbdafc Fix IPA for stable/mitaka with noauth mode
In noauth mode when running stable/mitaka, Ironic responds to the
initial lookup request with code 406, NOT_ACCEPTABLE, due to the client
supplied microversion of 1.22 by IPA.

As such, we should fallback upon receiving the 406 error code.

Related-Bug: #1616461
Change-Id: Ief9c151cf27a3c38c33a837bb7d6581868a3dc22
2016-08-26 13:06:34 +00:00
Lucas Alvares Gomes
09d5d0c377 Fallback to the old /lookup endpoint on 401
Prior to this patch the ironic-python-agent service would only fallback
to use the old endpoints for /lookup and /heartbeat on a 404 error but,
since Ironic will check auth (if enabled) before it routes the requests
a 401 (Unauthorized) was returned.

Closes-Bug: #1612696
Change-Id: Idba5fed587e77aaa683d2c2b2126a520214712ce
2016-08-12 16:41:51 +01:00
Dmitry Tantsur
09265ba4b5 Use new agent API if available
Falls back to vendor passthru on receiving 404.

Also fixes logging around lookup: log traceback on unexpected
exceptions, log successful lookup and replace % with ,

Change-Id: I7160c99ca63585fc333482fa578fdf5e0962b2b6
Depends-On: I9080c07b03103cd7a323e2fc01be821733b07eea
Partial-Bug: #1570841
2016-08-05 12:02:41 +02:00
Sam Betts
a7f0af722f Support LLDP data as part of interfaces in inventory
To support multi-tenant networking in Ironic we need to be able to
discover not just the NICs a baremetal machine has but also the physical
connectivity to switches in the network.

This patch collects LLDP (Link Layer Discovery Protocol) data as part of
the list interfaces stage of the generic hardware manager. This
information can then be processed by the ironic inspector to populate
the local link information on each ironic port.

The processing done on this data in ironic python agent is limited, this
is to allow for server side processing hooks to process as much or as
little of the data as they want. This is to allow for multi-vendor
environments that might use different parts of the LLDP packet to use a
generic ramdisk and configure the processing server side using inspector
plugins.

Reserved fields switch_port_descr and switch_chassis_descr have been
deprecated for removal in Ocata in favor of passing the whole packet.

Change-Id: Idae9b1ede1797029da1bd521501b121957ca1f1a
Partial-Bug: #1526403
2016-06-22 18:26:04 +01:00
Dmitry Tantsur
6670da4ed1 Returns CPU flags in the CPU inventory
These flags will be processed in a new ironic-inspector plugin
to support setting capabilities like cpu_vt (virtualization enabled).

Change-Id: I5fe9310c316841eabdd2d5e2ef2ae30afa03d29a
Partial-Bug: #1571580
2016-06-01 16:12:32 +02:00
Dmitry Tantsur
3deb25a3ce Wait for the interfaces to get IP addresses before inspection
In the DIB build the DHCP code (provided by the dhcp-all-interfaces element)
races with the service starting IPA. It does not matter for deployment itself,
as we're waiting for the route to the Ironic API to appear. However, for
inspection it may result in reporting back all NIC's without IP addresses.
Inspection fails in this case.

This change makes inspection wait for *all* NIC's to get their IP addresses up
to a small timeout. The timeout is 60 seconds by default and can be changed
via the new ipa-inspection-dhcp-wait-timeout kernel option (0 to not wait).

After the wait inspection proceedes in any case, so the worst downside
is making inspection 60 seconds longer.

To avoid waiting for NIC's that are not even connected, this change extends the
NetworkInterface class with 'has_carrier' field.

Closes-Bug: #1564954
Change-Id: I5bf14de4c1c622f4bf6e3eadbe20c44759da5d66
2016-04-05 20:03:33 +02:00
Lucas Alvares Gomes
f09dce79af Fix programmatic error in heartbeat()
This patch is fixing a programmatic error in the heartbeat() method of
the APIClient() class. When 409 (Conflict) was returned from the the
Ironic API the code wasn't parsing the content of the response
correctly.

Change-Id: I01fbb8d866b2f94fe128d0bc40b69d05b5add1a3
Closes-Bug: #1556199
2016-03-11 16:58:28 +00:00
Zhenguo Niu
d25d94b316 Change to use WARNING level for heartbeat conflict errors
It's normal that ironic returns 409 Conflict from time to time, so
it's a bit confusing that we report this with Exception level and
traceback.

Change-Id: I1627c61facc3fadd0f5d9d324150e7d2833c7fbc
Closes-Bug: #1533113
2016-03-06 17:13:02 +08:00
Joshua Harlow
df701c979c Replace backoff looping call with oslo_service provided version
The code that comprises the backoff looping call moved to
oslo.service in 0.10.0 and IPA can use that version so remove
the local backoff code and just prefer the oslo.service maintained
and supported version instead.

Change-Id: Ifec3490f9e5c68859deff4a951dcdf59caa7ca3a
2016-02-04 12:22:21 -08:00
Kan
2b07976b94 Fix params order in assertEqual
Fix params order to correspond to real signature:
    assertEqual(expected, actual)

Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba
Closes-Bug: #1277104
2016-01-13 02:03:11 +00:00
Lucas Alvares Gomes
c21409e98b Follow up patch for da9c3b0adc67efa916fc534d975823c0a45948a1
This patch is a follow up patch fixing some nits left by the review
da9c3b0adc67efa916fc534d975823c0a45948a1, this patch adds the
wwn_with_extension and wwn_vendor_extension root device hints to the
"serializable_fields" list attribute of the BlockDevice class and fixes
some tests.

Change-Id: I6039be535988319276f9ac355c80997d34328ce8
2015-11-18 09:56:09 +00:00
John L. Villalovos
dcbba2b121 Enforce all flake8 rules except E129
Bring ironic-python-agent in line with the other ironic projects.

Stop ignoring all E12* errors except E129
Stop ignoring E711

Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
Dmitry Tantsur
b569e37d06 Expose serial, wwn and vendor on the BlockDevice object
Currently we only use these disk properties for root device hints.
However, they'll be really useful for inspector, especially for also
implementing root device hints.

Change-Id: I48aa6b6d2d198d16f2f8e387970f7230066cf8a2
2015-09-21 13:17:20 +02:00
Dmitry Tantsur
dab1ce9831 Allow passing node UUID to lookup
After successful inspection we will know node UUID already, so no need
to do a proper lookup. In the future we might get rid of lookup completely
in this case.

Change-Id: Ic00049bf69f9ecc59ac4e389a3e6afe20ba67aba
2015-08-21 16:25:04 +02:00
Dmitry Tantsur
17c7e05235 Extend hardware manager with data needed for inspector
* Added NetworkInterface.ip4_address
* Added HardwareManager.get_bmc_address()
* Added Memory.physical_mb

  This is total memory as reported by dmidecode, and yes,
  it's different from total, as it includes kernel reserved space.

* Added CPU.architecture

  As a side effect, get_cpus was switched to lscpu.
  Also fixes problem when get_cpus reported the current frequency
  instead of maximum one.

Change-Id: I4080d4d551eb0bb995a94ef9a300351910c09fb9
2015-08-21 16:25:04 +02:00
Josh Gachnang
fb2790774d Add base for IPA functional tests
Starts up an agent in a thread and then runs functional tests against
it.

Add the first functional test of the API: a simple test of the
commands API.

Co-Authored-By: Mario Villaplana <mario.villaplana@gmail.com>
Change-Id: If4ad611929d388a2a9454224646cb296bd0ba0ce
2015-08-18 16:19:47 +00:00