Currently we support streaming raw whole disk images, but not
partition ones. This change enables it.
Change-Id: Ie95102aa3f2054a6b429f3d3e0926e90923c5faf
Story: #2003809
Task: #26558
Adds enhanced checksum support to IPA, when os_hash_algo and os_hash_value
are passed in via image_info, it will be used to calculate image checksum
and verification.
In other cases, the old md5 checksum is used.
Change-Id: I1d2f33e7059910326b4ac3f7786543b333a93a5a
Story: 2003938
Task: 26846
Installs the grub bootloader to the PreP Boot partition when the
prep_boot_partition_uuid is provided. This is required when
booting a partition image locally on ppc64* systems.
This change also passes the cpu_arch along to work_on_disk so
that the PReP partition is created when partitioning disks for
local boot on ppc64* systems,
Change-Id: I70667d43af962b357e6eeccba258f4fa5a91a09e
Depends-On: I2bc9f13ec605de7b7b96d96a1a4edebee0af76dc
Story: #1749057
Task: #22999
Increases the amount of ram for CoreOS IPA to 2GB
as the base CoreOS image is now 310MB.
Bumped CPU count for CoreOS runs to 2 CPUs as the
concurrency helps boot times for the CoreOS ramdisk.
Adds netbase, udev, and open-iscsi to debian jessie container
as they are no longer present in the default container.
Explicitly set path variable for execution in the debian
container as udevadm is in /sbin, and we may not have
/sbin on the path that is passed through to the
chroot.
Also fixed new pep8 test failures.
Story: #1600228
Task: #16287
Change-Id: I488445dfd261b7bca322a0be7b4d8ca6105750a3
Change c5bf7b088f1ec776b788a81f2775e1b2577720e8 introduced
a new requirement via a pre-existing ironic-lib method being
called that utilizes hexdump. Hexdump is not always present
and since we did not explicitly call it out as a new
requirement, we should at least somewhat gracefully handle
the exception.
Change-Id: Id0223ef1417f6e419770ceb56b2a3b80c6118a85
Closes-Bug: #1732470
IPA does not return 'root_uuid' as part of command status when
provisioning of whole disk image is done using 'agent' deploy
interface from ironic. This commit fixes the issue.
Also updated Dockerfile to include package 'bsdmainutils' related
to 'hexdump' binary.
Change-Id: I89597fe4a704686fe31c064c3443fd8404a300e5
Partial-Bug: #1713916
Specify 'ironic_python_agent' as the name of the application for the
flake8-import-order plugin. That way it knows that imports of
ironic_python_agent should come after external libraries.
Change-Id: Id39d558a51aeb97d96633afea28676634547d0d7
This fixes unreadable output on download image failure.
Adding new instance variable to exception `ImageDownloadError` class
to avoid redundant logs.
Change-Id: I51782abd572588adfc62745eeab9c559eb8346dd
Closes-Bug: #1657691
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.
Change-Id: I946457e9079ce0b54c7fe0ad554d024a1c61dce0
The CoreOS IPA images do not support poweroff/reboot due to running in a
chroot. For this case, we fall back to forcing poweroff or reboot via
sysrq commands
Change-Id: I75d68b6308beba299d043e43a5fa1671b6ef3ada
Closes-Bug: #1628367
This patch adds standard SSL options to IPA config and makes use of them
when making HTTP requests.
For now, a single set of certificates is used when needed.
In the future configuration can be expanded to allow per-service
certificates.
Besides, the 'insecure' option (defaults to False) can be overridden
through kernel command line parameter 'ipa-insecure'.
This will allow running IPA in CI-like environments with self-signed SSL
certificates.
Change-Id: I259d9b3caa9ba1dc3d7382f375b8e086a5348d80
Closes-Bug: #1642515
Currently a reboot bash script file is used to call reboot and
poweroff operation. Deleting this file and moving the code to
python file using utils.execute()
Partial-Bug: #1557542
Change-Id: Iad9cd9d15417e9a954d108d2759e6303452fca27
Author: Bharath kumar <shettybharath4@gmail.com>
Co-Authored-By: Annie Lezil <annie.lezil@gmail.com>
Ironic-lib logs a message when configdrive is created
successfully. Remove duplicate message from IPA.
Change-Id: I2af81cdfda4cfc004288f44d14a5c127639cc1f1
Shell script to create config drive being replaced with python
code in ironic-lib.
Closes-Bug: #1493328
Change-Id: I31108f1173db3fb585386b2949ec880a95305fb6
In Python 2.6 it was required to use {0}, {1}...{n} when using the
string format function. In Python 2.7 and Python 3 it it not required.
Change {N} to {} in code.
This brings the code in style alignment with other projects like
ironic and ironic-lib.
Change-Id: I81c4bb67b0974f73905f14b589b3dd0a7131650d
Depends-On: I8f0e5405f3e2d6e35418c73f610ac6b779dd75e5
Collecting warning logs in the case of download failure
and write them to error logs in the end. This will help
a user to diagnose a problem when warning log was not
enabled.
Change-Id: I4198d7be08fc11b616b3f95c595ff53794436e24
Partial-Bug: 1512186
This utilizes the new metrics support in ironic-lib to allow the agent to
report timing metrics for agent API methods as configured in ironic-lib.
Additionally, this adds developer docs on how to use metrics in IPA,
including some caveats specific to ironic-lib.metrics use in IPA.
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Alex Weeks <alex.weeks@gmail.com>
Change-Id: Ic08d4ff78b6fb614b474b956a32eac352a14262a
Partial-bug: #1526219
This commit adds the disk_label support for partition
images. It also fixes the node_uuid info passed to the
ironic_lib.
Partial-Bug: 1560560
Change-Id: I8b8ef20787468c1b8dc6fbc0b8905abd285325e1
The agent returns "efi_system_partition_uuid=None" in the
status message for uefi netboot for partition images.
This commit fixes to remove this unwanted message
from the status message as efi partition is created only
for localboot.
Closes-bug: 1526289
Change-Id: I6376406cdde29493619f50b0a6cd8b6ce3784d6e
This patch is adding a new command called sync to the standby module of
IPA. The new command runs synchronously and it's responsible for
flushing file system buffers to the disks.
The initial intention for this command is to use it as part of the fix
for the bug #1512492 where some hardware/firmwares do have problems to
come back online after a soft ACPI power off, therefore we need to call
sync() to make sure all file system buffers have been synced and then
issue a hard power off (e.g via the BMC).
Partial-Bug: #1512492
Change-Id: I5cd1d1b821426e995dc584452494b93ab23917e0
It also adds the ironic-lib in the requirements
list of the IPA package.
Partial-bug: 1526289
Depends-On: I22bc29a39bf5c35f3eecb6d4e51cebd6aee0ce19
Change-Id: I37908470484744bb720f741d378106d1cb1227a3
This patch adds support for streaming a raw image directly onto the disk,
that means no more time spent writing the image to a tmpfs partition prior
to copying it to the disk. Checksum computation is also done as the image
is being streamed. Streaming raw images is disabled by default, however
this behavior can be enabled by passing a key called "stream_raw_images"
with the value of True to the prepare_image() command of IPA.
For non-raw images this may not be possible, not sure about all image
file formats, but common types such as qcow2 requires random access to
the image file in order to be converted to raw.
Closes-Bug: #1505685
Change-Id: Iddf67907bc9b54bbd3065a97064cb5a3602cfe18
Prior to this patch downloading and computing the checksum of the image
were done in different stages, after the download the file would need
to be re-read and the checksum was computed. This patch is changing it
by creating a ImageDownload class which computes the checksum at the same
time the image is being downloaded, this saves time and also make the
code more portable.
Related-Bug: #1505685
Change-Id: I71f9f2bd9a62a6a6cc474d0ae519591cea6381d6
This change adds proxies parameter to requests.get method when
downloading images, and also sets no_proxy environment variable,
as requests does not support such parameter directly.
Implements: blueprint agent-image-proxy
Change-Id: I000ff9d0d4f5bde4766d085dc5a1c6cea1949bc5
Bring ironic-python-agent in line with the other ironic projects.
Stop ignoring all E12* errors except E129
Stop ignoring E711
Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
If an image cannot be downloaded for some reason, it is helpful for
operators to have the image path, checksum, and calculated checksums
available easily from the API.
Change-Id: I6a2fb46726245cebd730b5c51d4f25f8465f1658
Debugging the agent is a huge pain point. Tracebacks are rarely logged,
error messages are often only returned via the API, and lack of
info logging makes it hard to determine where some failures occur.
Some errors only return a 500 with no error message or logs.
Change-Id: I0a127de6e4abf62e20d5c5ad583ba46738604d2d
This patch add Python 3 support for IPA. We still need to enable it on
gate to avoid code breaking the support from sneaking in.
Partial-Bug: #1474896
Depends-On: I5a774eaa8d978f4d50faca4dc1b03a87fb194ce2
Change-Id: If47c0797b63d2914b3c47aba62ec5201301b6c33
This commit adds a new command power_off to
standby extension which runs shutdown -h now
on the system. This commit also adds mappings
for /proc and /sys in cloud-config.yml for the
agent service spawned.
Partial-Bug: #1451310
Change-Id: I2a5f984af26bbbe03002bb8c367c8c6af8d91434
Some of the extensions in IPA currently do not return any values
from their respective command methods. This results in a lack of a
paper trail of the command_result value to the point where it is
not possible to know more information than a boolean result of if
a command succeeded or failed.
This change adds more logging to the extensions and provides return
values which are used in the command_result property on
(Async|Sync)CommandResult instances to provide useful debug info.
While creating this patch several unit tests for *CommandResult objects
were found to be using invalid data that would not pass validation
if returned in API results. Those have been corrected in this patch.
Change-Id: I23bae25a83881e7013c93d73f858c622ea941253
Closes-Bug: 1408080
Currently, we only log the image ID and attempted URL. Now, we log the
status code recieved and detailed information about how and when things
failed.
Change-Id: I718c7facbe1500d98be78b7b6137e92fdfb2fdf1
Closes-bug: 1420981
Depends-On: I69f6f6eef4ad573f406d64d579a9811c70ac5d28
Currently we pick the most specific manager and use it. Instead, call
each method on each hardware manager in priority order, and consider the
call successful if the method exists and doesn't throw
IncompatibleHardwareMethodError.
This is an API breaking change for anyone with out-of-tree
HardwareManagers.
Closes-bug: 1408469
Change-Id: I30c65c9259acd4f200cb554e7d688344b7486a58
Currently extensions.standby._verify_image function loads the whole
image to memory to compute its hash. This behaviour is changed to
loading images by chunks and using hash update() method to compute
hash.
Change-Id: Ib3cd6cb565e35b0ed58f4526d6ad226676606705
Currently this logline contains the full, base64'd configdrive. This
changes it to log the file it came from instead.
Change-Id: I40fa4b8d889ce08fa20f7b0841b5abf122499676
Closes-bug: 1401244
Update openstack/common files and use oslo.utils, oslo.concurrency
instead of modules from oslo-incubator.
Additionally, sort and set requirements.txt to correct versions in
openstack/requirements in order to ensure pep8 passes.
Change-Id: I15f88a31a4c889bba27cd0cd1c7fc481c0c0b51e
This commit adds support for passing an HTTP(S) URL as the
`configdrive` parameter for the `prepare_image` command.
This will load the entire image in memory. Although the configdrive
is limited to 64MB, this is not ideal and should be made better
in the future (likely when imaging gets refactored).
Change-Id: I6f14b9059920d2cef108c30d027c5486fa0b7e9a
Implements: blueprint expose-configdrive
This change passes the agent object into extensions, such that
they may call agent methods as needed. It also causes async
commands to force a heartbeat immediately after completing the
command. This allows Ironic to get a heartbeat and continue
work as quickly as possible, while also allowing deployers to
configure Ironic (agent) to heartbeat less often.
Change-Id: Ib3c3a43dfd0ed4ed51b7d52ac099f01181ca822f
sync_command and async_command in ironic_python_agent/extensions/base.py
are implemented so that they change decorated method signature,
which is really bad practice. Example:
@sync_command
def orig_method(self, **kw):
...
sync_command decorator returns lambda which has one additional
argument 'command_name'
def sync_command(func):
def decorated(self, command_name, **kw):
...
func(self, **kw)
...
return decorated
Additional parameter 'command_name' is moved from decorated function to
decorator signature in both decorators. Every call of decorated functions
is fixed, BaseAgentExtension.execute is updated to not send this extra
argument. BaseAgentExtension.__init__ automatically generates command_map
from decorated methods.
Change-Id: Iff0f495bc8d9c731892feb0f4759a7f10ee76328
Closes-Bug: 1314148
Previously this code was calling processutils.execute incorrectly,
expecting it to return the exit code. This change:
* fixes calls to processutils.execute
* fixes related tests
Change-Id: I3343f104a5a8200d53bd87a8799ff918e375b17f
This was broken when extensions/ subdirectory was created.
Also add test to ensure this doesn't break again.
Change-Id: I12524a927397c3d2a90d00198873e2e1b4786171
