35 Commits

Author SHA1 Message Date
ChangBo Guo(gcb)
30e0da15ea Remove usage of parameter enforce_type
Oslo.config deprecated parameter enforce_type and change its
default value to True in Ifa552de0a994e40388cbc9f7dbaa55700ca276b0.
Remove the usage of it to avoid DeprecationWarning: "Using the
'enforce_type' argument is deprecated in version '4.0' and will be
removed in version '5.0': The argument enforce_type has changed its
default value to True and then will be removed completely."

Change-Id: I0f0fb540c43edde64e489915c5199da40a0da9c1
Related--Bug: #1517839
2017-06-14 13:47:29 +08:00
Annie Lezil
fdcb0922a5 Collect NIC name given by BIOS
Adds an extra field ``biosdevname`` to network interface inventory
collected by ``default`` inspection collector (which collects the whole
inventory returned by hardware manager) of ironic-python-agent.

This feature requires biosdevname utility to collect the bios given NIC
names. The tooling module for tinyIPA is created for the same purpose.
For CoreOS IPA pxe images, biosdevname tooling module is limited,
because Docker repository is created and embedded into CoreOS pxe
images. The Docker repository uses debian to download the packages.
Debian does not have biosdevname package.

Adds an export variable TINYIPA_REQUIRE_BIOSDEVNAME. Set this
variable to ``true`` in your shell before building tinyIPA.

Closes-Bug: #1635351
Change-Id: Ia96af59e2a74868cac59e5a88cfbb3be60d85687
2017-05-18 14:44:11 -07:00
Julian Edwards
f57cbccf8b Prevent tests' unmocked access to utils.execute()
This change introduces a new base test class that mocks out
utils.execute and forces an exception if it gets called.
This has rooted out many tests that were doing this as a side effect of
calling other functions, doing things like modprobe and running iscsi
on the host's actual machine.

The tests are all now appropriately patched in places where this was
happening, and the new base class permanently prevents this from
accidentally happening again.

If you really want to call utils.execute() then you need to re-mock it
in your unit test.

Change-Id: Idf87d09a9c01a6bfe2767f8becabe65c02983518
2017-05-15 10:48:43 +10:00
John L. Villalovos
1695cb18c2 Add missing 'autospec' argument to mock.patch
Add missing 'autospec' keyword argument to mock.patch and
mock.patch.object calls. Use 'autospec=True' except for a few cases
where it fails because the mocked function is a @classmethod and it
doesn't work. In that case explicity set it to 'autospec=False'

Change-Id: I620dce91abaa4440e1803aeefb3e93c0b65d1419
2017-03-19 10:04:19 -07:00
John L. Villalovos
949f4f509e Use flake8-import-order
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.

Change-Id: I946457e9079ce0b54c7fe0ad554d024a1c61dce0
2017-02-16 09:46:21 -08:00
Derek Higgins
256b233b4b Add IPv6 unit test for _get_route_source
Follow up from previous patch

Change-Id: Ic0286c4eac57f3ae06b237b700d9dbe95ea8b2c0
2017-01-19 15:24:17 +00:00
Derek Higgins
9f5f664080 Advertise the correct address when using IPv6
Parse the output of "ip route get $IP" taking
IPv6 into consideration. Also wrap the IP address
in square brackets if it is IPv6.

Change-Id: Ifc44e5aa3c5b814b6ceba04461bb68fe1d75c22b
Closes-Bug: #1650533
2017-01-11 11:00:56 +00:00
Szymon Borkowski
ef47d62f43 Add a new Hardware Manager for CNA network card
This patch adds a new hardware manager, which will disable the embedded
LLDP agent on Intel CNA network cards in order to allow the gathering of
LLDP data during the inspection process.

Change-Id: I572756ac6a7bf67a7f446738ba9d145e1c1bdc48
Closes-Bug: #1623659
2016-12-12 17:17:23 +01:00
John L. Villalovos
9ebb70d2fd Update mock variable name in unit tests
In test_agent.py change variable names from 'mocked_' to 'mock_' to
follow convention we use in openstack/ironic.

Change the mock variable names 'wsgi_server_cls' and
'mocked_server_maker' in some of the unit tests to
'mock_make_server'. It is the convention to have 'mock' in the
variable names for mock objects.

Not changing the other unit test files at this time.

Change-Id: I844071c928f92778e8ce0bfbdb8fb89fc26ee43b
2016-12-07 13:37:34 -08:00
Jenkins
9f9b5b5c43 Merge "Skip API related work if no api url configured" 2016-12-07 17:05:39 +00:00
Yufei
dd9253f1b6 Skip API related work if no api url configured
Currently, if IPA is booted without an ironic api url, it will default
to localhost and fail to connect. Instead, we now explicitly fail and
print a log message if no api callback url is provided.

Change-Id: I0271be94ba7febc6abd5bf3343f6fa179bc1a6a4
Closes-Bug: #1643966
2016-12-07 17:04:05 +08:00
Jenkins
f9236682f7 Merge "Dispatched out network interface info to all hardware managers" 2016-11-20 13:05:01 +00:00
Moshe Levi
966db1c18c Dispatched out network interface info to all hardware managers
This patch dispatches out the network_interface_info
to allow vendor hardware managers to plug the spacific
implementation. It also move neworking releated methods
form hardware to netutils
Related-Bug: #1532534

Change-Id: Idcd25c4753c009b5ba70bea97ee4eb83391a77a9
2016-11-17 13:08:03 +02:00
Luong Anh Tuan
ab41106cf6 Python 3 Compatible JSON
In order to be really python3 compatible, the json lib was replaced
with oslo.serialization(1.10 or newer) module jsontuils since it's
the recommended migration to python3 guide.

https://wiki.openstack.org/wiki/Python3#Serialization:_base64.2C_JSON.2C_etc.

Change-Id: I2d8b62e642aba4ccd1b70be7e9b3784a95a6743d
Closes-Bug: #1629068
2016-11-16 08:19:51 +00:00
Pavlo Shchelokovskyy
b033bfd933 Remove old lookup/heartbeat from IPA
Lookup/Heartbeat via vendor passthru was deprecated in Newton.

This patch removes the corresponding functionality from IPA,
and also removes handling of 'ipa-driver-name' kernel parameter,
as it was only used in code related to old passthru.

Change-Id: I2c7989063ab3e4c0bae33f05d6d2ed857a2d9944
Closes-Bug: #1640533
2016-11-09 16:34:44 +00:00
John L. Villalovos
abf98ae84a Use namedtuple to improve code readability
Use the namedtuple class to improve code readability by creating a Host
class with namedtuple to store the 'hostname' and 'port'

Replace foo[0] with foo.hostname, and foo[1] with foo.port to make code
more readable.

Change-Id: Ie2b5f9cf89e7ccbbcf0a2573dab6f6c5d14c018b
2016-08-30 16:12:29 -07:00
Jenkins
87b2b82478 Merge "Use new agent API if available" 2016-08-07 02:44:48 +00:00
Dmitry Tantsur
09265ba4b5 Use new agent API if available
Falls back to vendor passthru on receiving 404.

Also fixes logging around lookup: log traceback on unexpected
exceptions, log successful lookup and replace % with ,

Change-Id: I7160c99ca63585fc333482fa578fdf5e0962b2b6
Depends-On: I9080c07b03103cd7a323e2fc01be821733b07eea
Partial-Bug: #1570841
2016-08-05 12:02:41 +02:00
Jim Rollenhagen
7ab53e34b0 Fix races in advertise_address unit tests
This was testing that _check_for_iscsi was called, which is part of
evaluate_hardware_support for the in-tree hardware manager.
evaluate_hardware_support is called at hardware manager instantiation,
which can happen when dispatch_to_managers is called. The hardware
manager instances are cached, so calling dispatch_to_managers won't
always invoke _check_for_iscsi. As such, these tests were intermittently
failing.

Mock _check_for_iscsi at the class level for these tests (since it's
unrelated), and don't check that it was called.

Change-Id: Iefca50c4c231c5eafd199bf8da6649bc3008a8fa
2016-07-26 10:22:41 -04:00
Yossi Ovadia
90424fad0a Handle diskless hardware connected to remote iscsi
It is required to issue iscsistart -b on those hardware in order for
/dev/sda /dev/sdb etc to be exposed to the OS.

On this type of hw, without the fix
introspection fails with -  missing: ['local_gb']' err.

Change-Id: If6667b123e9b890d97c03612596fb78ee2b0ae85
Closes-Bug: 1590606
2016-07-11 09:42:20 -07:00
vmud213
09db71d640 Wait for at least one interface before node lookup
During node look up sometimes IPA sends hardware inventory information before
the interfaces are up, resulting in empty interfaces information being sent
to the conductor. This causes conductor to throw exceptions and polluting
the log until the real interfaces information is populated. This change makes
IPA to wait for at least one interface to be up before trying for node lookup.

Change-Id: Ifdb91298eaa5c725f108fa722263ed925691ecda
Closes-Bug: #1562786
2016-06-14 12:36:52 +00:00
Jenkins
be548fe5e0 Merge "Add a configurable sleep before IPA starts working" 2016-05-25 21:34:44 +00:00
Dmitry Tantsur
c09753bb60 Add a configurable sleep before IPA starts working
Some kernel modules take substantial time to initialize. For example,
with mpt2sas RAID driver inspection and deployment randomly fail
due to IPA starting before the driver finishes initialization.

As much as I hate it, the only way to guarantee that the hardware is
truely initalized is to wait for it. Apparently all hardware in Linux
is treated as hotplugged, so there is no such thing as "hardware
initialization is finished". Operators can add a sleep based on their
knowledge of their hardware.

The default behaviour remains the same.

Change-Id: I0446ae81d760dacaf31eea6ad9f9eaa098cf5e93
Partial-Bug: #1582797
2016-05-24 10:39:52 +02:00
Dmitry Tantsur
c15ed6a48e Wait for at least one suitable disk to appear on start up
Some kernel modules take substantial time to initialize. For example,
with mpt2sas RAID driver inspection and deployment randomly fail
due to IPA starting before the driver finishes initialization.

This problem is probably impossible to solve in a generic case, as
modern Linux environment do not have a notion of "hardware is fully
initialized" moment. All hardware is essentially hotplug.

To solve it at least for the simplest case, this patch adds a wait loop
on start up waiting for at least one suitable disk to appear in inventory.
Note that root device hints are not considered, as the node might not
be known at that moment yet.

Change-Id: Id163ca28f7c140c302ea04947ded3f3c58b284de
Partial-Bug: #1582797
2016-05-24 10:36:45 +02:00
Jay Faulkner
5a1a1ca61c Revert "Add hardware manager interface for hardware initialization"
I would've voted -1 on the patch in question had I reviewed it, and per
standard OpenStack/Ironic procedure, I'm reverting it for re-review and
discussion.

In this case; I don't think the new method in the HWM interface is
needed, and that evaluate_hardware_support() is intended to handle the
cases handled.

This reverts commit 0962cae1da69a1a2981d5950ad741d91115dac06.

Change-Id: Ic08e44bdf116403444b257ee9f4e5b906f5eac53
2016-05-23 17:41:29 +00:00
Dmitry Tantsur
0962cae1da Add hardware manager interface for hardware initialization
Some kernel modules take substantial time to initialize. For example,
with mpt2sas RAID driver inspection and deployment randomly fail
due to IPA starting before the driver finishes initialization.

Add a new hardware manager method initialize_hardware, which gets
run on start up before other hardware manager method invocations.

The generic implementation is to call udev settle and wait for
at least one suitable disk device to appear with the hardcoded
timeout of 15 seconds. Also preload the IPMI modules instead of
calling modprobe every time the inventory is requested.

Change-Id: If7758bb6e3faac7d05451baa3a26adb8ab9953d5
Partial-Bug: #1582797
2016-05-20 15:38:53 +02:00
Jenkins
c5531af3b3 Merge "Bind to interface routable to the ironic host, not a random one" 2016-03-21 19:17:48 +00:00
Jenkins
c3e9aca2c0 Merge " make enforce_type=True in CONF.set_override" 2016-03-21 17:25:46 +00:00
Dmitry Tantsur
6829d34c15 Bind to interface routable to the ironic host, not a random one
Binding to the first interface that has an IP address is error-prone: there is
no guarantee that ironic can reach us via this inteface. It is much safer to
detect the interface facing ironic and bind to it.

Unused LookupAgentInterfaceError exception is deleted.

The TinyIPA build also requires iptables dependency at build time to insert the
required kernel modules.

Closes-Bug: #1558956
Change-Id: I9586805e6c7f52a50834bc03efeb72d1faa6cb65
2016-03-21 14:21:12 +00:00
Shuquan Huang
fac700c8e7 Change assertTrue(isinstance()) by optimal assert
Some of tests use different method of assertTrue(isinstance(A, B)) or
assertEqual(type(A), B). The correct way is to use assertIsInstance(A,
B) provided by testtools.

Change-Id: I8e95e10fac4748b115838b6b480f1085da73f28a
Closes-bug: #1268480
2016-01-21 14:20:11 +08:00
Kan
2b07976b94 Fix params order in assertEqual
Fix params order to correspond to real signature:
    assertEqual(expected, actual)

Change-Id: I129aad5d8bb7ca0541c07913cb5edd8eef9e90ba
Closes-Bug: #1277104
2016-01-13 02:03:11 +00:00
LiuNanke
b563196a37 make enforce_type=True in CONF.set_override
Method CONF.set_override to change config option's
value with designated value in unit test, but never check if the
designated vaule is valid. Each config option has a type like strOpt,
BoolOpt, etc. StrOpt with parameter choices only allows values in set
of choices. In short word, each config option has limitation for type
and value. In production code, oslo.conf can ensure user's input is
valid, but in unit test, test methods can pass if we use method
CONF.set_override without parameter enforce_type=True even we pass wrong
type or wrong value to config option. This commit makes sure calling
method CONF.set_override with enforce_type=True.
This commit also fixes violations.

Note: We can't set enforce_type=True by default in oslo.config now, it
may break all project's unit test. We can switch enforce_type=True by
default when all project fix violations like this commit.

Change-Id: Iba3e7fca01fc91e4396e698fc00cad35ba8f3543
Related-Bug: #1517839
2016-01-12 20:57:31 +08:00
Shuquan Huang
cfcef973e8 Replace assertEqual(None, *) with assertIsNone in tests
Replace assertEqual(None, *) with assertIsNone in tests to have
more clear messages in case of failure.

Change-Id: Iad3f8fbb23a8b0f9e5ae4f304799465724c1a433
Closes-bug: #1280522
2015-12-17 11:32:33 +08:00
Dmitry Tantsur
096830414b Add support for inspection using ironic-inspector
Adds a new module ironic_python_agent.inspector and new entry point
for extensions, which will allow vendor-specific inspection.

Inspection is run on service start up just before the lookup.
Due to this early start, and due to the fact we don't even know
MAC address of nodes on inspection (to say nothing about IP addresses),
exception handling is a bit different from other agent features:
we try hard not to error out until we send at least something to inspector.

Change-Id: I00932463d41819fd0a050782e2c88eddf6fc08c6
2015-09-07 18:22:54 +02:00
Josh Gachnang
fb2790774d Add base for IPA functional tests
Starts up an agent in a thread and then runs functional tests against
it.

Add the first functional test of the API: a simple test of the
commands API.

Co-Authored-By: Mario Villaplana <mario.villaplana@gmail.com>
Change-Id: If4ad611929d388a2a9454224646cb296bd0ba0ce
2015-08-18 16:19:47 +00:00