48 Commits

Author SHA1 Message Date
Jenkins
8165e18949 Merge "Remove ordereddict as 2.6 is no longer supported" 2015-09-05 20:54:07 +00:00
Jay Faulkner
ac040f5152 Add docstrings for cmd/agent and extensions/clean
Adding documentation for methods in these files that were missing
docstrings.

Change-Id: I2ad9f85605720ab2295dec3d0436ff0af07847c2
Partial-bug: 1367915
2015-09-03 16:15:54 -07:00
Jay Faulkner
d29b9af748 Remove ordereddict as 2.6 is no longer supported
ordereddict, and a related workaround method, were removed as we no
longer support python 2.6. Collections contains ordereddict on newer
python versions.

Change-Id: I48cc55a5c56fbc1f71442fc858eb4905f0213988
2015-09-03 13:15:55 -07:00
Joshua Harlow
e9d2bfe580 Three attempts message is incorrect
The code that is covered in this exception block
does not actually retry three times, so it seems better
to have the message not reference that it does.

Change-Id: I0cc19aab65aa9ba4d73a8842390b712f7f63925c
2015-09-02 16:27:43 -07:00
Josh Gachnang
50875381d8 Fix logging levels
This is a fixup patch for bde6ed557034bc2eac633ff84f2e9515a5ef14d6,
and should follow the OpenStack Logging Standards.

Change-Id: Ie277c5742ab736f4c5cee1ee1badedf055323515
2015-08-04 09:29:44 -07:00
Josh Gachnang
59281ecda8 Improve IPA logging and exception handling
Debugging the agent is a huge pain point. Tracebacks are rarely logged,
error messages are often only returned via the API, and lack of
info logging makes it hard to determine where some failures occur.
Some errors only return a 500 with no error message or logs.

Change-Id: I0a127de6e4abf62e20d5c5ad583ba46738604d2d
2015-07-28 09:37:43 -07:00
Lucas Alvares Gomes
06eed473fb Add Python3 support
This patch add Python 3 support for IPA. We still need to enable it on
gate to avoid code breaking the support from sneaking in.

Partial-Bug: #1474896
Depends-On: I5a774eaa8d978f4d50faca4dc1b03a87fb194ce2
Change-Id: If47c0797b63d2914b3c47aba62ec5201301b6c33
2015-07-15 17:12:08 +01:00
Ramakrishnan G
be36ed6903 Add power_off command in standby extension
This commit adds a new command power_off to
standby extension which runs shutdown -h now
on the system. This commit also adds mappings
for /proc and /sys in cloud-config.yml for the
agent service spawned.

Partial-Bug: #1451310
Change-Id: I2a5f984af26bbbe03002bb8c367c8c6af8d91434
2015-06-09 23:44:08 +00:00
Jim Rollenhagen
601201d120 Update hacking and fix hacking violations
This does a few things:

* Update hacking to the version in global-requirements. Old hacking was
  installing a version of pbr that was breaking other packages.

* Fix all the hacking/pep8 rules that updating hacking raised.

* Do some general docstring cleanup, while already in there cleaning up
  a bunch of docstrings due to H405 violations.

Change-Id: I1fc1e59d4c3d7b14631f8b576e3f3854bc452188
Closes-Bug: #1461717
2015-06-03 16:58:57 -07:00
Jenkins
a1c87672ea Merge "Fix Sphinx Autodoc WARNING/ERROR in docs build" 2015-04-01 00:39:21 +00:00
Jay Faulkner
8bad5bbac3 Fix Sphinx Autodoc WARNING/ERROR in docs build
The docstrings here were all giving WARNINGs or ERRORs during the docs
build, and were generally making unappealing looking developer
documentation. I corrected the syntax and did what was neccessary to
make the build come out clean.

Change-Id: I74b00a7f125770b0468cff3bdf26d0d52cd054d7
(cherry picked from commit c0921cdff372ce1fd6df1c4ab4eb5463e2cba0e4)
2015-03-31 16:22:57 -07:00
Ghe Rivero
10a3a800e7 Use oslo_log lib
Module 'log' from oslo-incubator was removed after release of oslo_log
library.

Change-Id: Ia00e26071fc0cab4fd3f3334ca94860bc7b1c75c
2015-03-24 08:43:35 +01:00
Ramakrishnan G
ccda38a86e Do udevadm settle after running partx in image extn
This commit adds running 'udevadm settle' after
running partx so that the partition table is re-read
properly by lsblk which follows it.

Closes-bug: 1432918
Change-Id: I1c7dddd3ed8cab175e608e09ecc4ae7f289cfc31
2015-03-23 02:06:24 -07:00
Jenkins
173f86963a Merge "Add cleaning/zapping support to IPA" 2015-03-20 23:19:54 +00:00
Josh Gachnang
5f4fa7f27e Add cleaning/zapping support to IPA
This will add support for in band cleaning operations to IPA and
replace the decom API that was unused.

Adds API support for get_clean_steps, which returns a list of
supported clean steps for the node, execute_clean_step, to execute
one of the steps returned by get_clean_steps.

Adds versioning and naming for hardware managers, so if a new hardware
manager version is deployed in the middle of cleaning/zapping, the
cleaning/zapping will be restarted to avoid incompatabilities.

blueprint implement-cleaning-states
blueprint inband-raid-configuration
blueprint implement-zaping-states
Depends-On: Ia2500ed5afb72058b4c5e8f41307169381cbce48
Change-Id: I750b80b9bf98b3ddc5643bb4c14a67d2052239af
2015-03-17 17:07:04 -07:00
Ramakrishnan G
ac4e9901d6 Add uefi support in image extension
This commit adds support for uefi systems in the image
extension so that grub can be installed onto efi system
partition for uefi machines.

Implements: blueprint local-boot-support-with-partition-images
Change-Id: I8fbb4b2ebdff991d41c7b618a4d654af26311a56
2015-03-17 18:21:22 +00:00
Ramakrishnan G
1e04c8fffb SyncCommandResult should store actual exception
This commit changes ExecuteCommandMixin to store the
actual exception in SyncCommandResult instead of just
the message in the exception. This is required in the
API side where the result.command_error is trasformed
to base.ExceptionType.

Closes-bug: 1430896
Change-Id: I222bfc11868e73c7da03c84723f1450e38aa1a2f
2015-03-11 16:44:12 +00:00
Lucas Alvares Gomes
d23e0170de Add the image extension (for local boot)
Initially this extension supports installing a bootloader so the user
image can boot from the local disk.

Change-Id: Ia588aafc240b55119c02f1254addc0cf796f88c5
2015-03-04 16:34:17 +00:00
Lucas Alvares Gomes
d3aa7c93aa Add iscsi extension
This extension allows IPA to be used with the PXE/iSCSI methodology of
deployment in Ironic.

Change-Id: I32ec9fa74182c0d03c7ef1b698b1d0c0e3007773
2015-02-26 12:13:00 +00:00
Kyle Stevenson
f1dec5c2e2 Add additional logging and return results for IPA extensions
Some of the extensions in IPA currently do not return any values
from their respective command methods. This results in a lack of a
paper trail of the command_result value to the point where it is
not possible to know more information than a boolean result of if
a command succeeded or failed.

This change adds more logging to the extensions and provides return
values which are used in the command_result property on
(Async|Sync)CommandResult instances to provide useful debug info.

While creating this patch several unit tests for *CommandResult objects
were found to be using invalid data that would not pass validation
if returned in API results. Those have been corrected in this patch.

Change-Id: I23bae25a83881e7013c93d73f858c622ea941253
Closes-Bug: 1408080
2015-02-18 13:12:44 -08:00
Jay Faulkner
e5d88be8cb Log required troubleshooting info on image dl fail
Currently, we only log the image ID and attempted URL. Now, we log the
status code recieved and detailed information about how and when things
failed.

Change-Id: I718c7facbe1500d98be78b7b6137e92fdfb2fdf1
Closes-bug: 1420981
Depends-On: I69f6f6eef4ad573f406d64d579a9811c70ac5d28
2015-02-12 07:35:11 -08:00
Kyle Stevenson
571445438a Correct inline comment in sync_command decorator
It appears this comment was overlooked when implementing this
decorator. Simple correction to change async -> sync with appropriate
wording.

Change-Id: I2d53eac2b58772f90e435e704ddf309768a7fdbe
2015-02-04 17:44:14 -08:00
Jenkins
c35af55354 Merge "Reduce _verify_image memory consumption" 2015-01-15 18:23:38 +00:00
Jay Faulkner
2bbec5770c Allow use of multiple simultaneous HW managers
Currently we pick the most specific manager and use it. Instead, call
each method on each hardware manager in priority order, and consider the
call successful if the method exists and doesn't throw
IncompatibleHardwareMethodError.

This is an API breaking change for anyone with out-of-tree
HardwareManagers.

Closes-bug: 1408469
Change-Id: I30c65c9259acd4f200cb554e7d688344b7486a58
2015-01-08 15:15:13 -08:00
Vladyslav Drok
4fa6634ab9 Reduce _verify_image memory consumption
Currently extensions.standby._verify_image function loads the whole
image to memory to compute its hash. This behaviour is changed to
loading images by chunks and using hash update() method to compute
hash.

Change-Id: Ib3cd6cb565e35b0ed58f4526d6ad226676606705
2014-12-25 17:39:34 +02:00
Jenkins
3dda038707 Merge "Use oslo.utils and oslo.concurrency" 2014-12-17 01:44:47 +00:00
Jay Faulkner
86d4b41709 Do not log configdrive contents ever
Currently this logline contains the full, base64'd configdrive. This
changes it to log the file it came from instead.

Change-Id: I40fa4b8d889ce08fa20f7b0841b5abf122499676
Closes-bug: 1401244
2014-12-10 22:06:00 +00:00
Oleksii Chuprykov
86ef5d368d Use oslo.utils and oslo.concurrency
Update openstack/common files and use oslo.utils, oslo.concurrency
instead of modules from oslo-incubator.

Additionally, sort and set requirements.txt to correct versions in
openstack/requirements in order to ensure pep8 passes.

Change-Id: I15f88a31a4c889bba27cd0cd1c7fc481c0c0b51e
2014-12-09 00:06:23 +00:00
Jenkins
cf6386521b Merge "Improve/add docstrings for CommandResult classes" 2014-11-25 19:21:43 +00:00
Jay Faulkner
8c3595929f Improve/add docstrings for CommandResult classes
Added missing docstrings and improved existing docstrings for every
method in the CommandResult classes.

Change-Id: I71ca54a6ee2e3df2705b60bfae7c5a29a2b4657d
Partial-bug: 1367915
2014-11-25 10:18:35 -08:00
Jim Rollenhagen
c7c81b6374 Support configdrive from HTTP(S)
This commit adds support for passing an HTTP(S) URL as the
`configdrive` parameter for the `prepare_image` command.

This will load the entire image in memory. Although the configdrive
is limited to 64MB, this is not ideal and should be made better
in the future (likely when imaging gets refactored).

Change-Id: I6f14b9059920d2cef108c30d027c5486fa0b7e9a
Implements: blueprint expose-configdrive
2014-11-19 16:08:16 -08:00
Jim Rollenhagen
082cf29cec Force heartbeat immediately after async command completes
This change passes the agent object into extensions, such that
they may call agent methods as needed. It also causes async
commands to force a heartbeat immediately after completing the
command. This allows Ironic to get a heartbeat and continue
work as quickly as possible, while also allowing deployers to
configure Ironic (agent) to heartbeat less often.

Change-Id: Ib3c3a43dfd0ed4ed51b7d52ac099f01181ca822f
2014-10-13 11:09:09 -07:00
Szymon Wroblewski
324f037eb0 Enhance decorators in agent and automate creation of command_map
sync_command and async_command in ironic_python_agent/extensions/base.py
are implemented so that they change decorated method signature,
which is really bad practice. Example:

@sync_command
def orig_method(self, **kw):
...

sync_command decorator returns lambda which has one additional
argument 'command_name'

def sync_command(func):
    def decorated(self, command_name, **kw):
         ...
         func(self, **kw)
         ...
    return decorated

Additional parameter 'command_name' is moved from decorated function to
decorator signature in both decorators. Every call of decorated functions
is fixed, BaseAgentExtension.execute is updated to not send this extra
argument. BaseAgentExtension.__init__ automatically generates command_map
from decorated methods.

Change-Id: Iff0f495bc8d9c731892feb0f4759a7f10ee76328
Closes-Bug: 1314148
2014-09-11 10:49:33 +02:00
Kyle Stevenson
3c09b3975f Add support for passing no configdrive
Some people use metadata services, we should allow instances to be created without configdrive.

Change-Id: I5eb64b33291d6b975cf19357c04bf85066e4e3e8
2014-06-27 18:07:19 +00:00
Jim Rollenhagen
c5df7070af Better errors for execute() failures
Exceptions raised due to processutils.execute() failing now include
stdout and stderr.

Change-Id: Id5d1b5bc51d377f9f3c338cd7303ea800f76e5cd
2014-06-24 06:50:54 -07:00
Jim Rollenhagen
7b5575f660 Use processutils properly
Previously this code was calling processutils.execute incorrectly,
expecting it to return the exit code. This change:

* fixes calls to processutils.execute
* fixes related tests

Change-Id: I3343f104a5a8200d53bd87a8799ff918e375b17f
2014-06-12 06:51:40 -07:00
Russell Haering
263f97c0e0 Expose decom.erase_hardware command via API
Expose an async decom.erase_hardware command in the API. Executing
this command simply calls erase_devices on the loaded HardwareManager
implementation.

Change-Id: I238264a4d5903200ad6a30d82b61804c1e3f5916
2014-06-08 18:01:10 -07:00
Vladimir Kozhukalov
b306626e86 Flow extension uses extension manager from agent
Removed creating separate extension manager for flow extension.
Instead, have made flow extension using the same extension manager
instance which is initialized in agent. It fixes circular
extension loading in stevedore.

Closes-Bug: #1316145
Change-Id: Id339f1876168a41ca43ba7473f3ff6949a233ef3
2014-06-02 15:21:38 +04:00
Jenkins
d69bd2b054 Merge "Make encoding.serialize() more programmatical" 2014-05-29 21:58:50 +00:00
Alexander Gordeev
ed4460990e Make encoding.serialize() more programmatical
Introduce `serializable_fields` to express which class attributes
to be serialized.

Get rid of OrderedDict. Just replacing it with regular dict.

Change-Id: I3f7639dab171d3d62e92d0d1bb6d7b071cf963ad
2014-05-06 18:02:45 +04:00
Jim Rollenhagen
ea652bced0 Fix broken path to shell scripts
This was broken when extensions/ subdirectory was created.
Also add test to ensure this doesn't break again.

Change-Id: I12524a927397c3d2a90d00198873e2e1b4786171
2014-04-29 14:56:39 -07:00
Jim Rollenhagen
2e691d7971 Check configdrive size before writing to partition
Avoids writing a configdrive out to disk that is larger than
the intended partition.

Change-Id: I4e067ccb23ba528d96e4faad39219f67b4178e82
2014-04-25 13:42:47 -07:00
Jim Rollenhagen
620f05eb33 Accept new parameters for prepare_image
The parameters sent to `prepare_image` changed in
https://review.openstack.org/#/c/86490/

This patch brings `prepare_image` up to date with that change.
It also changes the way configdrive is written to disk, to match
that Ironic is now allowing Nova to build an ISO partition and
send the raw image to the agent.

This patch also swaps out subprocess.call for processutils.execute
in the standby module, since the commands were being changed anyway.

Lastly, this patch changes the expected `hashes` dict to be a
string parameter called `checksum`, to match what glance returns.

Change-Id: Id8af9be920ba51e7e1ce60f4ffd1477e413582c9
2014-04-24 13:00:24 -07:00
Jenkins
24c4382fed Merge "Deprecated extension name attribute" 2014-04-17 15:50:55 +00:00
Russell Haering
bf74673226 Decorated commands no longer take command_name
Prior to the (sync|async)_command decorators, extension methods had to
construct command results themselves, which required that they have
the name of the command being executed.

When the async_command decorator was introduced, we wanted to maintain
the current command interface, so that a command could be made
asynchronous simply by adding the decorator.

The introduction of the sync_command decorator allows us to satisfy
this objective, without requiring decorated methods to accept a
command_name. This patch modifies both decorators to no longer pass
the command_name argument, and modifies existing command methods not
to expect one.

Closes-Bug: #1308021
Change-Id: I1b33bb7519588cdcb79c6f4d2b10c1e5e8c0cca3
2014-04-15 10:25:12 -07:00
Vladimir Kozhukalov
9e153253e5 Deprecated extension name attribute
It seems not to be used, so it is not needed.

Change-Id: Ie46470d888ff11edaf16b94e5b2e716d13cf764f
Closes-Bug: #1308017
2014-04-15 15:56:09 +04:00
Russell Haering
dcd6e3e0dc Add a @sync_command() decorator
Instead of special-casing syncronous commands, add a decorator similar
to @async_command(), which can be used to decorate extension methods
for execution as a synchronous command.

Change-Id: I1b27f179f667cb065bcffd71ae0f303b05d33b82
2014-04-14 15:22:07 -07:00
Russell Haering
5ebd2e9797 Organize agent extensions
Move extensions under an ironic_python_agent.extensions module. This
change also moves the @async_command() decorator into the base
extension module.

Change-Id: I4021fcc33a30f3460a31bca44a4bf776cd53d488
2014-04-14 15:09:12 -07:00