This change drops check on UUID validness from our API.
It also has a subtle effect of doing Ironic node fetching in
the introspection status and data fetch calls, which might make them
slightly longer (but only when name is used).
A new helper common.ironic.get_node is created to unify how we fetch nodes
from Ironic. It also provides nicer exceptions.
Change-Id: I20cf65e57910568b70a62c3f9269a962e78a07e2
Closes-Bug: #1523902
Returning False from is-empty condition on missing values seems extremely
confusing and rules out some potential use cases.
Closes-Bug: #1578184
Change-Id: I8f976516f89367512e2ffae2815085be1776b6f6
Previously the ramdisk logs were only stored if the ramdisk reported an error.
However, we are moving from ramdisk-side validation to server-side, so we need
ramdisk logs to be available if processing fails too.
This change moves storing ramdisk logs from a ramdisk_error plugin to core
processing code. As before, it can be disabled by setting ramdisk_logs_dir to
an empty value.
Change-Id: Ib3742ee1c1d4f2f96d29466626e1121694610dc3
Closes-Bug: #1564448
* Use fixtures instead of a self-invented decorator
* Give proper names to mocks
* Swift from inline mocks to @decorators
* Split TestProcess into several test cases
* Remove the old style of setting side_effect (via iter)
Change-Id: If74221642723b0a6bea439dbcbdd360a43e7172f
Otherwise on reintrospection the firewall code may not update
iptables rules, assuming they are already correct.
Change-Id: Icc05174854bd9ab51bfed9d1360873bf5db9ed54
Closes-Bug: #1570447
This patch contains tempest plugin interface which are generated by
tempest-plugin-cookiecutter[1].
Also empty test manager and basic test was added.
[1] https://github.com/openstack/tempest-plugin-cookiecutter
Change-Id: I333462987bb6cdd1933fbb3550f527364c74ba07
As requested in the related bug, this pull request allows to run the
introspection again on previously stored data. This should make it simple
to correct mistakes in the introspection rules.
For this purpose, new API entry point was introduced:
/v1/introspection/<UUID>/data/unprocessed that supports an empty POST
method to trigger the introspection over stored data. New function
`reapply` was introduced that takes care about the entry point and carries
out the introspection. The `process` function was modified to allow
reusing common parts in the new reapply function. The storage access
methods were updated to allow saving the "raw" memdisk data besides the
processed introspection data.
Following preconditions are checked the reapply function having been
triggered:
* no data is being sent along with the request
* Swift store is configured and enabled and the stored data is present
for the node UUID
* node_info object is cached for the UUID and it is possible to lock the
object
Should the preconditions fail, an immediate response is given to the user:
* 400 if the request contained data or in case Swift store is not enabled
in configuration
* 409 if it wasn't possible to acquire lock for the node_info object
* 404 in case Ironic didn't keep track of related BM node
If the preconditions are met, a background task is executed to carry out
the processing and a 202 Accepted response is returned to the endpoint
user.
As requested, these steps are performed in the background task:
* preprocessing hooks
* post processing hooks, storing result in Swift
* introspection rules
These steps are avoided, based on the RFE:
* not_found_hook is skipped
* power operations
Limitations:
* IMPI credentials are not updated --- ramdisk not running
* there's no way to update the raw data atm.
* the raw data is never cleaned from the store
* check for stored data presence is performed in background;
missing data situation still results in a 202 response
Change-Id: Ic027c9d15f7f5475fcc3f599d081d1e8d5e244d4
Closes-Bug: #1525237
This refoctor is needed for tempest test work as tempest tests
will placed in test dir. So move unit tests to separate directory
"unit" under test.
Change-Id: Ic99df6111ef30947148a9e38b9435a54f3d37064
This test essentially tests get_session(), which is covered by anything else
touching the database code. More importantly, as it does not use any mocks,
it fails on stable/liberty right now.
Change-Id: I175a38bd3675aabe1c518a0b7d4c2bd56d93cc1a
This patch does not change the options in config file yet to showcase
backward compatibility with old config options.
Change-Id: I1da93b59b2f4813c42008277bd6479dc6673e7f1
When attempting to convert some eDeploy data to integer, inspector will
handle the ValueError exception which works fine for strings that are
not interger-like. But, when this data is None a TypeError exception
will be raised which wasn't handlded before, this patch is fixing it.
Closes-Bug: #1560050
Change-Id: I830a1a88c765c6471c457e383c7e859fd7f93ef9
Currently we are using only the resulting MAC(s) when doing a node lookup.
In many cases it is the MAC of the PXE-booting NIC. However, it's not necessary
the MAC that people used for enrolling the Ironic node, which will lead to
lookup failures on the virtual environment. This change makes the lookup
procedure use all of the valid MAC's.
Similarly, the enroll node_not_found_hook now checks all MAC's before creating
a node.
Code in the validate_interfaces hook was reordered to ensure we only keep
interfaces with valid MAC's even in the "all_interfaces" list.
Change-Id: Ie7df05d9a7855716fb835c90cfb0ac7fc4cd66df
Currently we keep DHCP always open for new nodes. This is an overkill, as we
always know which nodes are on introspection. It also causes problems when not
all node NIC's are registered in Ironic, as these NIC's might get DHCP from our
server.
This change reduces probability of wrong nodes accessing our DHCP by REJECT'ing
all DHCP requests when no nodes are on introspection and node_not_found_hook is
not set. It does not solve the problem completely: conflicts are still possible
during the introspection.
Change-Id: I7a50c02023ef4364e14825cd80fa75565fac3dc8
Partial-Bug: #1557979
With enroll hook it's possible to process nodes in enroll state.
As nodes in this state are not expected to have valid power
credentials, we shouldn't fail if power off goes wrong.
Change-Id: I79e502052c4ae1b531b4f0a0bc314b4cf6d29aac
Ironic port list with provided mac address returns list with one
element, in case port exist. So always get uuid from 0 port.
Closes-Bug: #1555498
Change-Id: I9729e1a0fb158b80ba9b7260e43814bd9ad206d8
Fix calls from .dict() to to_dict() in
IntrospectionRule.check_conditions and corresponding unit test.
Co-Authored-By: Nathan Harrison <nathan.harrison@sap.com>
Related-Bug: #1552781
Change-Id: I9fe7d400edcd7573132081a196e74eb09bc88cb7
A green thread is now used instead of spawn_n for running asynchronous
operations during introspection, processing and aborting.
The existing periodic tasks are now run using Futurist PeriodicWorker.
Main shut down procedure was split into a separate function for convenience.
Also updated the example.conf to the latest versions (some pending updates from
3rdparty libraries included).
Change-Id: Id0efa31aee68a80ec55e4136c53189484b452559
Tests for database migrations. There are "opportunistic" tests here,
supported backends are sqlite (used in test environment by default),
mysql and postgresql, which are required properly configured unit
test environment.
Change-Id: I660fad241c71f2e73b41f317d00adf97551579c6
Add new node_not_found_hook - enroll_node_not_found hook,
which allows to enroll unknown nodes to Ironic automatically.
Change-Id: If1528688504e4be4b2369b985bc576544d96868d
Related-Bug: #1524753
Some conditions do not have a native counterparts, so we need a way
to invert them (aka NOT operation). This patch adds a new generic
parameter "invert", defaulting to False.
Change-Id: I50342689ba52346a5a4fbf362536b629fc688986
Now, if operators want to check some attribute isn't specified,
they should use 'eq' condition with None, '', etc values:
{'op': 'eq', 'field': 'inventory.bmc_addres', 'value': ''}
It would be useful to have condition plugin witch verify this case,
'is-empty' checks that field is empty string, list, dict or None value:
{'op': 'is-empty', 'field': 'inventory.bmc_addres'}
Change-Id: I6596a067e769530092c3db34405e0f0917d2f052
Conditions: ``field`` supports new format, it allow to comparison
data both from inspection and node info:
{'field': 'node://ironic/style/path', 'op': 'eq', 'value': 'val'}
{'field': 'data://introspection/path', 'op': 'eq', 'value': 'val'}
Actions: ``value`` supports fetching data from introspection, it's
using python string formatting notation:
{'action': 'set-attribute', 'path': '/driver_info/ipmi_address',
'value': '{data[inventory][bmc_address]}'}
Related-Bug: #1524753
Change-Id: Ie05f82e7a29fba2f743217f0893c085fd843cd5b
Rollback actions were designed to help with rerunning introspection
on the same node. However, rollback actions for these actions proved
to be confusing and were never properly documented at all.
Even worse, the rollback action for set-attribute actually makes
this command impossible to use with non-removable attributes
(e.g. /driver).
This change removes rollback actions from all rules.
We need to rethink how we handle rollback in rules later on.
Change-Id: I2260f4b463c5dc804edac642c86e0da153e163f6
The configuration values are not initialized at the moment of import,
so it has no effect. The only reason we didn't hit it is due to helper
functions overriding the arguments. But extra_hardware plugin does not work.
Change-Id: If23bbdc4162589707cd63e6446a92036436b7f2a
Closes-Bug: #1544613
Ramdisk logs are huge (and may potentially be VERY huge) and clutter the output
for 'introspection data save'. They might even potentially expose unwanted
information to a user.
Change-Id: I0e7f1f647e60711f84d7883caab5ce8b14d6184f
not doing so introduces implicit (ordering) dependency issues between test
cases. For instance, in python client functional test[1] renaming
`TestSimplePythonAPI.test_introspect_get_status` to
`TestSimplePythonAPI.test_01_introspect_get_status` makes
`TestSimplePythonAPI.test_api_version` fail.
Notes:
I use this command to check for the cases where fixed uuids are being used:
`git grep -En "\<[^\s,\.(]*uuid[^\s,\.]*\s*=\s*['\"]"`
Links:
[1] https://github.com/openstack/python-ironic-inspector-client/blob/
master/ironic_inspector_client/test/functional.py#L137
Change-Id: Icd4a2a25824c6372d10fc40d67d8f275016b201c
Dmitry Tantsur