3042 Commits

Author SHA1 Message Date
Renat Akhmerov
58407b6f94 Working on Data Flow (step 1)
* Refactored API layer so that we can work with 'context' as with a
 json object in underlying layers (DB, engine, etc.) rather than
 a string
* Added "context" parameter in all required places
* Added necessary Data Flow related properties to DB models
* Refactored and fixed a series of tests
* Minor formatting changes

TODO:
* Calculation of task incoming context
* Abstract interface for expression evaluator
* Data Flow related tests

Partially implements blueprint: mistral-dataflow

Change-Id: Ie7f94d79265e9861f7ad15c76ff6d788ec62b683
2014-02-25 12:27:43 +07:00
Nikolay Mahotkin
8a73da20fd Add scheduling specific task on sucess/error
* Fix get task properties, get task in DSL
 * Evaluating task on success/error/finish properties in engine:
   - evaluate YAQL conditions
   - schedule tasks
 * Unit tests for handling success/error/finish

 * For real work with context there is need to store
   it somewhere

Implements: blueprint schedule-tasks-after-completion

Change-Id: Ia9c34e755055ff0273dcd79a314858dbfbcc2fa9
2014-02-20 21:12:10 +04:00
Jenkins
77a70c36e2 Merge "Send email action, part 2" 2014-02-17 17:14:11 +00:00
Dmitri Zimine
e971459eb7 Send email action, part 2
Added send_email to action types and action factory.
Now it works end-to-end.

NOTE:this renames DSL from action.parameters.responce
to action.output. Other projects will need to adjust.

Tested manually with mistral-extra/examples/webhooks/cmd/run.py,
It works, almost:  except a bug causing exception on parsing out results.
In discussion, but a fix should go in separate commit.

Also, small fixes:
action_factory:31 - used to blow up when action has no 'parameters'
dsl.py:32 - return details on parsing errors (error, line, column)
api/config.py - deleted: not used

Change-Id: Ie7079eb9589c23cc6704f8e8739af67a635f1c77
2014-02-17 08:09:51 -08:00
Renat Akhmerov
dbfeedd9d7 Rename "target_task" to "task"
* Renamed all occurrences of "target_task" to "task"
* Fixed a typo in executors controller.

Change-Id: I6b412090de224282c7eef2abceb989446d8b8df9
2014-02-14 18:19:42 +07:00
dzimine
9bca04c79e Send email action, step 1
Basic send-email functionality and unit tests
See comments in test_send_email_action on how to run with real smtpd.

Implements: blueprint mistral-std-send-email-action
Change-Id: Ib1d460d26ce54f4ab85d34b18947d858ab471dfd
2014-02-11 01:11:42 -08:00
Jenkins
842d75713a Merge "Fixing access to task "parameters" property in DSL" 2014-01-31 00:23:10 +00:00
Jenkins
d8082084bc Merge "Add negative tests to api" 2014-01-30 08:42:28 +00:00
dzimine
1c15945f86 Add negative tests to api
1. Add couple of negative tests, find the way, more to go.

2. Add assertNotFound to the base test class for convenience.

3. MistralException - added default message to enable default instantiation - make mock.assertRaise fails when exception has no default constructor

Change-Id: I708637e3995597a819571a90f3292d6e34deab4e
2014-01-29 19:01:27 -08:00
Renat Akhmerov
430b5ec2e4 Fixing access to task "parameters" property in DSL
Change-Id: I046430ffb77abf2d462c89050e9e28e67fa72816
2014-01-29 16:48:00 -08:00
Nikolay Mahotkin
ca37071c5d Fix getting task on-* properties in DSL
* Unit test

Change-Id: I949db952920da0606993f3dc260fb84ef2c4fdaf
2014-01-28 20:35:01 +04:00
Jenkins
aae48e19ec Merge "Fix task keys properties in DSL parser" 2014-01-28 13:49:18 +00:00
Jenkins
96611e8576 Merge "Add YAQL expression evaluation" 2014-01-28 13:11:28 +00:00
Jenkins
3a9c3a499e Merge "Modified Rest action for process 'input' property" 2014-01-28 13:05:52 +00:00
Jenkins
3af480bb27 Merge "Add sync task execution" 2014-01-28 13:05:22 +00:00
Nikolay Mahotkin
d32552b9fa Fix task keys properties in DSL parser
Change-Id: I5624fc6e35d100c63e79ed96c832bdc8eda67cbb
2014-01-28 17:01:22 +04:00
Jenkins
7d55b2befb Merge "Fixing and refactoring authentication" 2014-01-28 13:00:48 +00:00
Jenkins
4637e46423 Merge "Deleting client and demo app from main Mistral repo" 2014-01-28 13:00:45 +00:00
Nikolay Mahotkin
4ebc881b0c Add YAQL expression evaluation
* YAQL evaluate
 * Unit tests

Change-Id: I0853a08234c75853875b42cd8e89819e9b1e431f
2014-01-28 15:21:58 +04:00
Nikolay Mahotkin
1020554f1c Modified Rest action for process 'input' property
* For HTTP GET task input is converted to request url parameters
 * For HTTP POST/PUT - into request body

Change-Id: Ibc863ffb99ffdd0099f9ba2f43178e02dc1dd0c2
2014-01-28 13:57:40 +04:00
Nikolay Mahotkin
380689f223 Add sync task execution
* Non-Mistral actions gets a JSON response
   from which result can be extracted with
   YAQL-expression
 * Unit test
 * Feature is added to both engines

Change-Id: Iae6aa031e6b5700b3a08ee184f5eae291458a8c4
2014-01-28 13:57:32 +04:00
Renat Akhmerov
44e38c97b7 Fixing and refactoring authentication
* Moved pecan configuration to oslo config
* Refactored functional base test
* Got rig of thread local related code in mistral/context.py
* Added new exception ApplicationContextNotFoundException
* Fixed example configuration file
* Made minor cosmetic changes (blank lines, naming)

Change-Id: I1899ce2562a34ebafa20c5735bdf4f0c80dd0175
2014-01-27 12:11:28 -05:00
Renat Akhmerov
378c33ec95 Deleting client and demo app from main Mistral repo
Change-Id: Ib4e1deb0bcad53b55478b00f7f8a203186af305f
2014-01-27 11:13:55 -05:00
Jenkins
05b0989327 Merge "Add integration tests" 2014-01-27 14:21:10 +00:00
TimurNurlygayanov
d25e1de5cc Fixed issue with tarballs.
Closes-Bug: #1271068

Change-Id: I6f3035b19655313bb481650af69d9681cae02cb7
2014-01-24 20:04:46 +04:00
Sergey Murashov
6aff5394dc Add integration tests
Change-Id: I0873da3f0ddc73722f3d8506fde463da9d39cea9
2014-01-24 12:20:53 +04:00
Jenkins
4f7ba4402d Merge "Divide RestAction on two separated actions in DSL" 2014-01-20 15:48:33 +00:00
Jenkins
c69cfbd4f5 Merge "Adding Data Flow related code to REST API" 2014-01-17 19:20:55 +00:00
Nikolay Mahotkin
050f6220f6 Divide RestAction on two separated actions in DSL
* in case MISTRAL_REST_ACTION we add Mistral-specific
   headers, in case REST_ACTION - we are not.
 * new class for this type of action is unnecessary,
   all of these are Rest, only properties can be edited
   in factories methods

Change-Id: I431e5009a531da6bea191e7253d43b4cd58c46fb
2014-01-17 17:01:42 +04:00
Nikolay Mahotkin
12d7056def Add new access methods in DSL parser
* Access to new task properties:
   input, onSuccess, OnError; any task property
 * 'dependsOn' is renamed to 'requires' and returns dict object
   which contains conditions of task transition.

  DSL can be defined either:
     task3:
       requires: [task1, task2]
   or:
     task3:
       requires:
         task1: condition1  (or empty)
         task2: condition2  (or empty)

Change-Id: Ic3ab5754443100e101f04260d9054b0a604fcd6d
2014-01-17 13:06:27 +04:00
Jenkins
08113db18b Merge "Refactoring local and scalable engines" 2014-01-16 21:14:12 +00:00
Renat Akhmerov
baba497d99 Refactoring local and scalable engines
* Moved iethods start_workflow_execution and convey_task_result into
 abstract engine
* Refactored existing local engine unit tests
* Added simple unit tests for scalable engine
* Minor cosmetic changes (imports, blank lines)

Change-Id: Ic781b134068f52ed5827b945fb57b202239fe9c4
2014-01-16 07:06:30 -08:00
Renat Akhmerov
b940d50c87 Adding Data Flow related code to REST API
* Added "context" property for executions
* Refactored all REST API test to use @mock.patch

Change-Id: Ia0939e7099ae4386c1c3a761ac3892533eb3aed4
2014-01-15 18:01:10 -08:00
Renat Akhmerov
385d123e5d Fixing typo in an exception message in Mistral client
Closes-Bug: #1264146
Change-Id: I69170e7529f7ec85c8dade71f212adb34085304e
2014-01-14 15:55:12 -08:00
Nikolay Mahotkin
ea1fbeeb6b Step 2 refactoring Mistral engines
* Combine common parts of single and scalable engines

Change-Id: Iaa8708a831ca22a79d3ecd59c960be09a904e57a
2014-01-14 16:11:47 +04:00
Nikolay Mahotkin
0934507ed2 Step 1 refactoring Mistral engines
* Action definition was combined in actions module
 * workflow from both single and scalable engines were deleted
 * task from single engine was deleted
 * unit test 'test_workflow' was deleted due to deleting workflow
 * minor fixes to single engine

 Next step will be extracting and combining common parts
 of scalable and single engine.

Change-Id: I15fbbb7f9d5e453c17625423eeb1bf841a19db18
2014-01-14 15:57:28 +04:00
Nikolay Mahotkin
348f3eea19 Fix exceptions output when creating object in DB
Fixes: bug #1268501

Change-Id: I0fe7f61b72afc9cd7a5c3b65b343c1d438e07632
2014-01-13 13:59:08 +04:00
Sergey Murashov
673b3976ad Add SQLAlchemy in requirements.txt
Change-Id: Ieb7849cadfa9a14423bea74fd280a52aeb406f0b
2014-01-10 14:06:11 +04:00
Jenkins
d8ece7f8c2 Merge "Fix DB API import in scheduler.py" 2014-01-09 10:28:00 +00:00
Renat Akhmerov
a9089beb52 Fix DB API import in scheduler.py
Change-Id: If0705fc7a0c415f3233a8de209570486e6527ef9
2014-01-09 15:51:25 +07:00
Jenkins
dca045314f Merge "Implement single (non-scalable) engine" 2014-01-09 08:50:19 +00:00
Nikolay Mahotkin
c4430e5dd1 Implement single (non-scalable) engine
* New engine for running workflow (w/o rabbit)
 * Engine constructs a workflow
 * REST API action
 * AMQP action (not finished)
 * Task and execution state update
 * Added unit tests for this engine

Partially implements blueprint: mistral-poc
Partially implements blueprint: mistral-workflow-execution

Change-Id: I0e2b1a25f4281f77dad07a6f5241025be0963ed7
2014-01-07 16:22:01 +04:00
Jenkins
9ac513a357 Merge "context creation in periodic task to execute workbook" 2014-01-02 17:03:19 +00:00
Jenkins
5a2064cb24 Merge "Fixing scheduler transactions" 2014-01-02 17:02:25 +00:00
Jenkins
17953f1366 Merge "Fixing flak8 excludes in tox.ini" 2014-01-02 05:40:51 +00:00
Jenkins
f56cff6777 Merge "Fixing workbook events creation" 2014-01-02 05:40:27 +00:00
Renat Akhmerov
6638ee35f5 Fixing scheduler transactions
* Multiple events are created within a single transaction

Change-Id: I1ad1122cf4080de41e704420ca89874e42148f3e
2014-01-02 12:35:37 +07:00
Renat Akhmerov
d17ea38b27 Fixing workbook events creation
* Workbook events creation is now called from a REST controller rather
 than from DB API. This is more proper place (level of abstraction).
* Fixed workbook definition controller and evets tests.

Change-Id: Ib17d0b0dae614c474c161cd12ecbb1b05aea74a8
2013-12-31 12:48:54 +07:00
Renat Akhmerov
1652d2e7fd Fixing flak8 excludes in tox.ini
Change-Id: I914b4dfd9155b1f813b951c1b41ca462f33c4e12
2013-12-30 21:19:01 +07:00
Jenkins
e6d4c593f8 Merge "Adding license and authors file" 2013-12-30 11:48:56 +00:00