3503 Commits

Author SHA1 Message Date
Nikolay Mahotkin
e0e3a58cf5 Fix result of HTTP action
* Fixed HTTPAction result: To use expressions we need to
   transform any action result to some expressions-readable
   structure, e.g. dict. In case of HTTPActions we expose
   status, content and headers. For example, if we recieved
   response - {"server": "info" {"id": "some_id"}} we can
   address to property 'id' using expression '$.content.server.id'
 * Unit test

Change-Id: I9ecf4dc01f1ae084e27b29f46507c65fba9a6692
2014-04-17 13:24:23 +04:00
Jenkins
ca851b494a Merge "Fixing http action and abstract engine" 2014-04-17 06:31:01 +00:00
Nikolay Mahotkin
60e2c064ff Fix returning ERROR task state
* Force raising ActionException in case if we can't convert result
   in action output (e.g. we recieved a wrong result)

Change-Id: I2d16e0c8ff3a9050674d6d8ba44b29da822aec7c
2014-04-16 19:38:19 +04:00
Renat Akhmerov
1d45c508dd Fixing http action and abstract engine
* Fixed defaults for parameter 'data' in HTTPAction and MistralHTTPAction
* Small fix in abstract_engine.py related to adhoc acitons

Change-Id: I25c1b0581b17ca5c3e419b260a5bcc0e3c938ec5
2014-04-16 16:21:26 +07:00
Renat Akhmerov
d5c253229f Moving expressions.py out of package 'engine'
Change-Id: Id512e753e6d82ccc329ffbdef04ff0f58a2305af
2014-04-16 16:21:26 +07:00
Manas Kelshikar
51e11b70ee Change repeater to retry on error
* Retry is only attempted if task responds with error. Successful tasks do
not have to be retried which is the primary difference repeat and retry.

* Once a retry is triggered delay and break-on are taken into considertaion.
Modified tests appropriately and renamed types and properties appropriately.

Change-Id: I660cac05c20e791c0620add99d9d348f4bd910d1
2014-04-15 11:19:28 -07:00
Renat Akhmerov
11f236a785 BP mistral-actions-design (raw action spec -> ActionSpec)
* Switched from raw action spec (dict) to using ActionSpec in
  ad-hoc actions

TODO:
* Fix all samples in mistral-extra
* Action plugin architecture

Change-Id: Id8da3642bcd9549bf2e64c07eb4d79fd856a01de
2014-04-14 19:24:37 +07:00
Renat Akhmerov
9970767a86 BP mistral-actions-design (removing old actions, addressing previous comments)
* Removed obsolete packages
* Fixed arguments order in assertions as pointed by Sergey Murashov in
  https://review.openstack.org/#/c/85653/6/

TODO:
* Fix all samples in mistral-extra
* Action plugin architecture

Change-Id: I120363da862786e081f6eb0df2ae9cd5a434659c
2014-04-14 19:24:37 +07:00
Renat Akhmerov
f72daa44e8 BP mistral-actions-design (add SSH action)
* Added standard SSH action (based on SSHAction in engine/actions/actions.py)
* Added tests for SSH action

TODO:
* Fix all samples in mistral-extra
* Action plugin architecture

Change-Id: Ie00630a4b43f289888c3d3ce705857c45d738ae6
2014-04-14 19:24:37 +07:00
Renat Akhmerov
b2a072f35c BP mistral-actions-design (switch to new design)
* Refactored action_factory.py (adhoc actions into a separate method)
* Fixed task db model (service_spec -> action_spec)
* Fixed workbook model (tasks, actions, Service -> Namespace)
* Fixed all tests related to DSL syntax changes

TODO:
* Fix all samples in mistral-extra
* Action plugin architecture

Change-Id: Iac6317dde894f17c81332a8c9a2397b70448f6b2
2014-04-14 19:24:37 +07:00
Renat Akhmerov
4c1e5a2a80 BP mistral-actions-design (ad-hoc actions in factory)
* Fixing action factory to create ad-hoc actions if needed
* More tests for action factory

TODO:
* Refactoring executor to use new action infrastructure
* Remove old action infrastructure
* Action plugin architecture

Partially implements: blueprint mistral-actions-design

Change-Id: I69946989226a1c05ed089acccb7e10a6516cf966
2014-04-14 19:24:37 +07:00
Renat Akhmerov
15ea7c06f6 BP mistral-actions-design (ad-hoc action)
* Created AdHocAction that works as a transforming wrapper
  for any other action (uses action specification dictionary)
* Added tests for AdHocAction
* Moved and refactored tests for email and echo actions from
  old package

TODO:
* Integrate AdHocAction with action factory (it is a special built-in
  action class and requires some special action on action factory side)
* Refactoring executor to use new action infrastructure
* Remove old action infrastructure
* Action plugin architecture

Partially implements: blueprint mistral-actions-design

Change-Id: Ieaf479047bfcaf3da61d40f28f108d198af6d16f
2014-04-14 19:24:37 +07:00
Renat Akhmerov
0d5a04d7cc BP mistral-actions-design (Mistral HTTP action)
* Created MistralHTTPAction
* More unit tests
* Refactoring

TODO:
* Ad-hoc per-workflow namespaces and actions
* Refactoring executor to use new action infrastructure
* Remove old action infrastructure
* Action plugin architecture

Partially implements: blueprint mistral-actions-design

Change-Id: I6856d6587243872a67fa42efff0bf3f88ec19d82
2014-04-14 19:24:37 +07:00
Renat Akhmerov
b0e94b00a8 BP mistral-actions-design (action creation)
* Method create_action in action_factory.py
* Method get_action_class in action_factory.py
* Additional tests for action_factory.py
* Removing namespace and action name params from action constructors

TODO:
* Ad-hoc per-workflow namespaces and actions
* Mistral HTTP standard action
* Refactoring executor to use new action infrastructure
* Remove old action infrastructure
* Action plugin architecture

Partially implements: blueprint mistral-actions-design
Partially implements: blueprint mistral-change-rest-api-to-http

Change-Id: Ia90c66e9e06a577d06d3db687098cdb6d741afaa
2014-04-14 19:24:37 +07:00
Renat Akhmerov
32720f2c72 BP mistral-actions-design (add new actions package)
* Base class for actions (Action)
* Class for representing action namespaces (NameSpace)
* Module action_factory.py for registering action classes, accessing
  them and creating action instances
* Unit tests for action_factory.py
* Standard actions (echo, http, email)

TODO:
* Ad-hoc per-workflow namespaces and actions
* Mistral HTTP standard action
* Refactoring executor to use new action infrastructure
* Remove old action infrastructure
* Action plugin architecture

Partially implements: blueprint mistral-actions-design
Partially implements: blueprint mistral-change-rest-api-to-http

Change-Id: I8e79b1f8173809584f0156e666e1675f5bd7bb37
2014-04-14 19:24:27 +07:00
Renat Akhmerov
ebeb68b6dc BP mistral-actions-design
* Renaming task 'input' property to 'parameters' according to BP
* Fixing auth token generation related test
* Small refactoring in expressions
* Formatting changes

Partially implements: blueprint mistral-actions-design

Change-Id: I8659145d022ac4a2895c9d06f85f0961bf0b4f50
2014-04-14 15:21:27 +07:00
Nikolay Mahotkin
75fc00796a Add SSH Action
* We need an another way to communicate with VM,
   so SSH is this way. Also it gives a possibility
   to check if VM operation system is up by sending
   a simple ssh command to it.
 * Added SSH Action
 * Only password access, forwarding public/private keys
   will be in the future
 * It is tested on real VM. However, it needs to be tested
   by somebody else

Implements blueprint mistral-ssh-action

Change-Id: I7c51453afb6a926bbce01f4a9065a3c313a79fff
2014-04-11 13:41:35 +04:00
Winson Chan
a736f03bf8 Remove local engine
The local engine is replaced in process executor using the oslo.messaging
fake transport.  The local engine module is removed here along with related
unit tests. Some unit tests are rewritten to use the ScalableEngine. An
EngineTestCase is included in the base module of tests to simplify engine
testing.

Change-Id: I1a49a53eac87a209660c493fd7c3fe3e914fd092
Implements: blueprint mistral-inproc-executor
2014-04-07 10:46:03 -07:00
Jenkins
7f8530aeda Merge "Fix repeatable task scheduling" 2014-04-07 17:07:26 +00:00
Nikolay Mahotkin
6342ffb6ab Fix repeatable task scheduling
* Fixed repeatable task scheduling. See bug description for
   more information.
 * Small fix in data flow test

 Fixes bug: #1301866

Change-Id: I40e653ac242df0f5deee8b72ac77a7c04e4fda2a
2014-04-04 19:04:17 +04:00
Nikolay Mahotkin
e1bf0a1479 Add resolving inline expressions
* We allow to resolve something like:
   - /{$.project_id}/servers
   - {$.auth_token}

Change-Id: Ic2a5a32b42c168bac021ff1e96d1dae8bac6647e
2014-04-04 13:37:17 +04:00
Jenkins
697f67d7e7 Merge "Refactor the local engine to use an in process executor" 2014-04-04 08:58:56 +00:00
Jenkins
f8e7327678 Merge "Fix minor issues" 2014-04-03 05:23:32 +00:00
Jenkins
1a24ebc4e0 Merge "Add extracting action output" 2014-04-03 05:23:32 +00:00
Jenkins
7905fa5be5 Merge "Cosmetic change." 2014-04-03 03:41:29 +00:00
Manas Kelshikar
8b0bfa5061 Cosmetic change.
Pick up cosmetic changes from https://review.openstack.org/#/c/80127/10
* rename exec_flow_context to task_runtime_context.
* Fix comments to start with cap letter and end with a period.

Change-Id: Ic74f045f8d401053a983b1f6356a8d3562b23f2e
2014-04-02 11:43:23 -07:00
Jenkins
a1754e9e24 Merge "Fix keystone trust client" 2014-04-02 17:09:50 +00:00
TimurNurlygayanov
0b743fdd74 Fixed issue with deprecated exception
Change-Id: Ic2f1fec236f193fa4b07ebfd96e878d86cc6ad37
Closes-Bug: #1298524
2014-04-02 13:06:35 +04:00
Nikolay Mahotkin
24f1c7d455 Fix minor issues
* Action factory creates rest_action with either
   parameters or body accordingly to request method
   (for GET - parameters, for POST/PUT - body)
 * Fix apply context method:
   * All data which is not resolved by YAQL is ignored
 * Rest action send body in form json.dumps(data)
   (it is better for nested dictionary)
 * Improved some unit tests

 * With this patch Mistral is able to connect to Nova
   and create instance

Change-Id: Ibc73a7dc2de4339acbb9310755c2c53028815b38
2014-04-02 12:11:01 +04:00
Nikolay Mahotkin
36a84e299e Fix keystone trust client
* The issue popped up after release of python-keystoneclient 0.7.
   The issue is caused by new keystoneclient not accepting
   project_id and trust_id simultaneously as constructor parameters.

Change-Id: Id0b7b582e9b794dace0f440df582ff9502b781c5
2014-04-02 12:04:05 +04:00
Nikolay Mahotkin
94fde896b5 Add extracting action output
* In case if we define the specific output
   in action, we should evaluate all given YAQL
   expressions and return calculated dict,
   otherwise - return raw result

Change-Id: I8f4a91c7fb0de743793c7cd66b892e038e25332f
2014-03-31 12:18:40 +04:00
Winson Chan
e9a0f14db2 Refactor the local engine to use an in process executor
Refactored launch script to start the API and executor on the same process
so the transport for a oslo.messaging fake driver can be shared. A transport
attribute is added to the abstract engine as a class attribute to reference
the transport object.

Change-Id: I4b8684ebded9eb993749f2503380e078087edd7d
Implements: blueprint mistral-inproc-executor
2014-03-30 18:05:54 -07:00
Manas Kelshikar
025343c050 Implements: blueprint mistral-std-repeat-action
In this implementation there is only a provision to repeat a specific action.
- Features like configurable iterations, delay, break-on are implemented.
- All final data is published to the context and there is no per iteration
  persistence.
- The iteration context which is specific to the scope is saved in the
  exec_flow_context property of a task.

Change-Id: I1e9ff7847f0046e11d3b32eec65f664c2b09d3b1
2014-03-28 10:33:28 -07:00
Jenkins
6424670a58 Merge "Add keystone auth_token in context" 2014-03-27 00:33:47 +00:00
Jenkins
5673db7b2f Merge "Correct fake action test name" 2014-03-27 00:32:29 +00:00
Jenkins
ebf9bb0bc0 Merge "Remove unneeded declarations in unit tests" 2014-03-27 00:31:30 +00:00
Nikolay Mahotkin
9fd4c69ac1 Correct fake action test name
Change-Id: Ibb2322139fd8d7f3365d3522afde622def910fe9
2014-03-26 17:26:24 +04:00
Nikolay Mahotkin
7709cb0f0b Remove unneeded declarations in unit tests
Change-Id: I75e3a60ef7720011aa2bd3610bbc60d14140bdaa
2014-03-26 17:23:54 +04:00
Nikolay Mahotkin
64ea892672 Add keystone auth_token in context
* Add creating keystone auth_token in context
   when task is scheduled for execution. Auth token
   is created from workbook trust_id in case Auth_enable
 * Added transforming values of task_spec and service_spec
   with YAQL and context.
 * This feature affects some unit tests, so they are fixed
 * Added unit tests

Partially implements: blueprint mistral-dataflow

Change-Id: I15794da427bfc7babbc4e3d58a90200c10a19a72
2014-03-24 14:24:18 +04:00
Nikolay Mahotkin
23073881aa Fix keystone config group name
* keystone_authtoken is renamed to keystone due to
   unexisting name. It isn't able to create workbook
   with auth_enable=True without this fix

Change-Id: I3d865e31b87dbb18e62ee3d6fa4ca5054ff5c8fd
2014-03-17 17:21:01 +04:00
Jenkins
da7b742aa7 Merge "Add script to allow update dependencies in all envs" 2014-03-17 05:04:58 +00:00
Nikolay Mahotkin
f00803f467 Add script to allow update dependencies in all envs
Change-Id: I75aa921607fb214d55205cc380749180e0991024
2014-03-14 14:41:48 +04:00
Renat Akhmerov
91df3876ed Fixing ordering bugs in local engine tests
Change-Id: I4429895f6a342ee796b5b21291bd9de1da38d09e
2014-03-14 17:14:21 +07:00
Renat Akhmerov
427db9d84b Fixing ordering bugs in workbook model and tests
* When parsing yaml we can't rely on order of properties
 because it's against yaml specification
* In tests we often make assumptions about list items order

Change-Id: I536bc31ba7bee68e2904a23af73004f2e0ba2261
2014-03-14 14:47:44 +07:00
Renat Akhmerov
d90a727c6b Fixing executor launch script
* Executor launch script should now start engine because
 it calls engine.convey_task_result() directly. It's a
 temporary solution until engine is a standalone process.
 When it's done executor will be sending a message back to
 engine instead of making a local call.

Change-Id: I4666ca5f1a9cb093e8b7c9e640048f0176955595
2014-03-13 22:19:20 +07:00
Jenkins
06aaa949fd Merge "Rename 'events' to 'triggers'" 2014-03-13 14:48:14 +00:00
Nikolay Mahotkin
383e7775f6 Fix getting task on-* properties
Fixes: Bug #1288227

Change-Id: I46680abe82c86e6b40b7f0274ad40f8c6cc250ef
2014-03-12 19:05:40 +04:00
Nikolay Mahotkin
7e01958c95 Rename 'events' to 'triggers'
* "events" are renamed to "triggers"
 * triggers moved out of workflow section

Implements: blueprint mistral-rename-event-to-trigger

Change-Id: I5b76e9c063a3ce9eec5d485bb5db5e22ac7981d4
2014-03-12 15:04:45 +04:00
Nikolay Mahotkin
0884cd0ff0 Implement new object-model specification
* Implemented new object model
 * Implemented spec validation
 * Old DSL parser was deleted
 * All code is replaced by new workbook spec
 * Also tests are fixed
 * events -> triggers
 * Unit tests

Implements: blueprint mistral-dsl-model
Partially implements: blueprint mistral-dsl-validation
Partially implements: blueprint mistral-rename-event-to-trigger

Change-Id: I782b7983ddc724375229587f467c55ad9341478b
2014-03-12 14:58:54 +04:00
Jenkins
c3ecd49d26 Merge "Working on Data Flow (step 5)" 2014-03-11 06:45:13 +00:00