If two hardware managers have the same clean step, for example
'erase_devices' in the GenericHardwareManager and a custom manager,
IPA must determine which step should be kept and which should be run
in order to prevent running the step multiple times.
This patch uses the following filtering logic to decide which step
"wins":
- Keep the step that belongs to HardwareManager with highest
HardwareSupport (larger int) value.
- If equal support level, keep the step with the higher defined
priority (larger int).
- If equal support level and priority, keep the step associated with
the HardwareManager whose name comes earlier in the alphabet.
Other than individual step priority, picking which step to keep does
not actually impact the cleaning run. However, in order to make
testing easier, this change ensures deterministic, predictable
results.
Co-Authored-By: Mario Villaplana <mario.villaplana@gmail.com>
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Brad Morgan <brad@morgabra.com>
Change-Id: Iaeea4200c38ee22cab72ba81c1dbae3389e675e4
The iSCSI extension now tries to use Linux-IO first (via rtslib)
and falls back to tgtd if Linux-IO can't be used (e.g. in the CoreOS-based
image which uses containers).
Change-Id: I9cc7a30d9c93c445a66d183146e9260c2b096d33
Closes-Bug: #1504562
This patch adds support for streaming a raw image directly onto the disk,
that means no more time spent writing the image to a tmpfs partition prior
to copying it to the disk. Checksum computation is also done as the image
is being streamed. Streaming raw images is disabled by default, however
this behavior can be enabled by passing a key called "stream_raw_images"
with the value of True to the prepare_image() command of IPA.
For non-raw images this may not be possible, not sure about all image
file formats, but common types such as qcow2 requires random access to
the image file in order to be converted to raw.
Closes-Bug: #1505685
Change-Id: Iddf67907bc9b54bbd3065a97064cb5a3602cfe18
Prior to this patch downloading and computing the checksum of the image
were done in different stages, after the download the file would need
to be re-read and the checksum was computed. This patch is changing it
by creating a ImageDownload class which computes the checksum at the same
time the image is being downloaded, this saves time and also make the
code more portable.
Related-Bug: #1505685
Change-Id: I71f9f2bd9a62a6a6cc474d0ae519591cea6381d6
This change adds proxies parameter to requests.get method when
downloading images, and also sets no_proxy environment variable,
as requests does not support such parameter directly.
Implements: blueprint agent-image-proxy
Change-Id: I000ff9d0d4f5bde4766d085dc5a1c6cea1949bc5
The intention of the iscsi extension _wait_for_iscsi_daemon function
is to wait till the tgtd process boots completely before running any
iscsi commands through the tgtadm command line utility. The tgtd
process utilises a unix socket for communication with the user and the
_wait_for_iscsi_daemon function asserts the tgtd process status based
on the socket presence. The socket name/path to use is not specified
by iscsi extension configuration but rather falls-back to a
(system-wide) default value[1]. Since tgtd version 1.0.60, this path
changed from /var/run/tgtd.ipc_abstract_namespace to
/var/run/tgtd/socket[2] and now causes a false-negative while waiting
for the tgtd process to boot[3].
This patch follows implementation suggestion from the bug report[3] to
avoid relying on the socket presence but rather to check the tgtd
status by trying to communicate through the tgtadm tool.
[1] https://github.com/openstack/ironic-python-agent/blob/
5bbb9ded082f2cfde5e8877e9f33294eb93a5fb7/
ironic_python_agent/extensions/iscsi.py#L47 [2]
https://github.com/fujita/tgt/commit/
d1aa4dcfd691c9409dbad5db49ce6754ce9c1b5a
[3] https://bugs.launchpad.net/ironic-python-agent/+bug/1505923
Closes-Bug: #1505923
Change-Id: If3284e0b441fe6c2da507640c08a4bbefd9dad2b
This patch updates the _install_grub2 function in image.py
so that we use 'mount -t sysfs' instead of bind mounting
the hosts /sys into the chroot.
This resolves issues which can occur with unmounting
the chroot mounts where 'target is busy' errors
may occur when you are debugging things with a serial
console.
Change-Id: I5c9a6546d048cbf54695329d9744fda55127bd30
Closes-bug: #1503385
This fixes a ValueError exception that would occur if
log debug was enabled in IPA.
Change-Id: Id95f291dd77d69e7d4dd4f945c6c151db7a0646b
Closes-bug: #1503354
Bring ironic-python-agent in line with the other ironic projects.
Stop ignoring all E12* errors except E129
Stop ignoring E711
Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
If an image cannot be downloaded for some reason, it is helpful for
operators to have the image path, checksum, and calculated checksums
available easily from the API.
Change-Id: I6a2fb46726245cebd730b5c51d4f25f8465f1658
Create a SerializableComparable class derived from the Serializable
class.
Added the following functions to the SerializableComparable class:
'__eq__'
'__ne__'
Disable the '__hash__' function in the SerializableComparable class as
some derived classes are mutable.
Use the SerializableComparable class in hardware.py and
extensions/base.py
This should make unit testing users of the class easier when doing a
self.assertEqual() or self.assertNotEqual()
Added some initial unit testing for encoding.py
Change-Id: If0f14b3bfe7f1391f65dd730a16a534afed0da82
If a clean step returns a tuple (like the output of
processutils.execute), WSME raises an error about the type. Casting
the result as a list seems to make it work. This is almost certainly
a bug in WSME, but after digging a bit I couldn't find one.
Change-Id: I91cb7a989ab56c0942469e03937945d4a590ad95
Partial-bug: #1493554
Adding documentation for methods in these files that were missing
docstrings.
Change-Id: I2ad9f85605720ab2295dec3d0436ff0af07847c2
Partial-bug: 1367915
ordereddict, and a related workaround method, were removed as we no
longer support python 2.6. Collections contains ordereddict on newer
python versions.
Change-Id: I48cc55a5c56fbc1f71442fc858eb4905f0213988
The code that is covered in this exception block
does not actually retry three times, so it seems better
to have the message not reference that it does.
Change-Id: I0cc19aab65aa9ba4d73a8842390b712f7f63925c
This is a fixup patch for bde6ed557034bc2eac633ff84f2e9515a5ef14d6,
and should follow the OpenStack Logging Standards.
Change-Id: Ie277c5742ab736f4c5cee1ee1badedf055323515
Debugging the agent is a huge pain point. Tracebacks are rarely logged,
error messages are often only returned via the API, and lack of
info logging makes it hard to determine where some failures occur.
Some errors only return a 500 with no error message or logs.
Change-Id: I0a127de6e4abf62e20d5c5ad583ba46738604d2d
This patch add Python 3 support for IPA. We still need to enable it on
gate to avoid code breaking the support from sneaking in.
Partial-Bug: #1474896
Depends-On: I5a774eaa8d978f4d50faca4dc1b03a87fb194ce2
Change-Id: If47c0797b63d2914b3c47aba62ec5201301b6c33
This commit adds a new command power_off to
standby extension which runs shutdown -h now
on the system. This commit also adds mappings
for /proc and /sys in cloud-config.yml for the
agent service spawned.
Partial-Bug: #1451310
Change-Id: I2a5f984af26bbbe03002bb8c367c8c6af8d91434
This does a few things:
* Update hacking to the version in global-requirements. Old hacking was
installing a version of pbr that was breaking other packages.
* Fix all the hacking/pep8 rules that updating hacking raised.
* Do some general docstring cleanup, while already in there cleaning up
a bunch of docstrings due to H405 violations.
Change-Id: I1fc1e59d4c3d7b14631f8b576e3f3854bc452188
Closes-Bug: #1461717
The docstrings here were all giving WARNINGs or ERRORs during the docs
build, and were generally making unappealing looking developer
documentation. I corrected the syntax and did what was neccessary to
make the build come out clean.
Change-Id: I74b00a7f125770b0468cff3bdf26d0d52cd054d7
(cherry picked from commit c0921cdff372ce1fd6df1c4ab4eb5463e2cba0e4)
This commit adds running 'udevadm settle' after
running partx so that the partition table is re-read
properly by lsblk which follows it.
Closes-bug: 1432918
Change-Id: I1c7dddd3ed8cab175e608e09ecc4ae7f289cfc31
This will add support for in band cleaning operations to IPA and
replace the decom API that was unused.
Adds API support for get_clean_steps, which returns a list of
supported clean steps for the node, execute_clean_step, to execute
one of the steps returned by get_clean_steps.
Adds versioning and naming for hardware managers, so if a new hardware
manager version is deployed in the middle of cleaning/zapping, the
cleaning/zapping will be restarted to avoid incompatabilities.
blueprint implement-cleaning-states
blueprint inband-raid-configuration
blueprint implement-zaping-states
Depends-On: Ia2500ed5afb72058b4c5e8f41307169381cbce48
Change-Id: I750b80b9bf98b3ddc5643bb4c14a67d2052239af
This commit adds support for uefi systems in the image
extension so that grub can be installed onto efi system
partition for uefi machines.
Implements: blueprint local-boot-support-with-partition-images
Change-Id: I8fbb4b2ebdff991d41c7b618a4d654af26311a56
This commit changes ExecuteCommandMixin to store the
actual exception in SyncCommandResult instead of just
the message in the exception. This is required in the
API side where the result.command_error is trasformed
to base.ExceptionType.
Closes-bug: 1430896
Change-Id: I222bfc11868e73c7da03c84723f1450e38aa1a2f
Initially this extension supports installing a bootloader so the user
image can boot from the local disk.
Change-Id: Ia588aafc240b55119c02f1254addc0cf796f88c5
Some of the extensions in IPA currently do not return any values
from their respective command methods. This results in a lack of a
paper trail of the command_result value to the point where it is
not possible to know more information than a boolean result of if
a command succeeded or failed.
This change adds more logging to the extensions and provides return
values which are used in the command_result property on
(Async|Sync)CommandResult instances to provide useful debug info.
While creating this patch several unit tests for *CommandResult objects
were found to be using invalid data that would not pass validation
if returned in API results. Those have been corrected in this patch.
Change-Id: I23bae25a83881e7013c93d73f858c622ea941253
Closes-Bug: 1408080
Currently, we only log the image ID and attempted URL. Now, we log the
status code recieved and detailed information about how and when things
failed.
Change-Id: I718c7facbe1500d98be78b7b6137e92fdfb2fdf1
Closes-bug: 1420981
Depends-On: I69f6f6eef4ad573f406d64d579a9811c70ac5d28
It appears this comment was overlooked when implementing this
decorator. Simple correction to change async -> sync with appropriate
wording.
Change-Id: I2d53eac2b58772f90e435e704ddf309768a7fdbe
Currently we pick the most specific manager and use it. Instead, call
each method on each hardware manager in priority order, and consider the
call successful if the method exists and doesn't throw
IncompatibleHardwareMethodError.
This is an API breaking change for anyone with out-of-tree
HardwareManagers.
Closes-bug: 1408469
Change-Id: I30c65c9259acd4f200cb554e7d688344b7486a58
Currently extensions.standby._verify_image function loads the whole
image to memory to compute its hash. This behaviour is changed to
loading images by chunks and using hash update() method to compute
hash.
Change-Id: Ib3cd6cb565e35b0ed58f4526d6ad226676606705
Currently this logline contains the full, base64'd configdrive. This
changes it to log the file it came from instead.
Change-Id: I40fa4b8d889ce08fa20f7b0841b5abf122499676
Closes-bug: 1401244
Update openstack/common files and use oslo.utils, oslo.concurrency
instead of modules from oslo-incubator.
Additionally, sort and set requirements.txt to correct versions in
openstack/requirements in order to ensure pep8 passes.
Change-Id: I15f88a31a4c889bba27cd0cd1c7fc481c0c0b51e
Added missing docstrings and improved existing docstrings for every
method in the CommandResult classes.
Change-Id: I71ca54a6ee2e3df2705b60bfae7c5a29a2b4657d
Partial-bug: 1367915
This commit adds support for passing an HTTP(S) URL as the
`configdrive` parameter for the `prepare_image` command.
This will load the entire image in memory. Although the configdrive
is limited to 64MB, this is not ideal and should be made better
in the future (likely when imaging gets refactored).
Change-Id: I6f14b9059920d2cef108c30d027c5486fa0b7e9a
Implements: blueprint expose-configdrive
This change passes the agent object into extensions, such that
they may call agent methods as needed. It also causes async
commands to force a heartbeat immediately after completing the
command. This allows Ironic to get a heartbeat and continue
work as quickly as possible, while also allowing deployers to
configure Ironic (agent) to heartbeat less often.
Change-Id: Ib3c3a43dfd0ed4ed51b7d52ac099f01181ca822f
sync_command and async_command in ironic_python_agent/extensions/base.py
are implemented so that they change decorated method signature,
which is really bad practice. Example:
@sync_command
def orig_method(self, **kw):
...
sync_command decorator returns lambda which has one additional
argument 'command_name'
def sync_command(func):
def decorated(self, command_name, **kw):
...
func(self, **kw)
...
return decorated
Additional parameter 'command_name' is moved from decorated function to
decorator signature in both decorators. Every call of decorated functions
is fixed, BaseAgentExtension.execute is updated to not send this extra
argument. BaseAgentExtension.__init__ automatically generates command_map
from decorated methods.
Change-Id: Iff0f495bc8d9c731892feb0f4759a7f10ee76328
Closes-Bug: 1314148
