operation engine design

Implements: blueprint operation-engine-design
Change-Id: Icc9acc0c10a4ae586c78a87426df0dcd13e544e9

doc/source/specs/operation-engine/create_scheduled_operation_seq_diagram.pu

@@ -0,0 +1,56 @@
+@startuml
+
+title create scheduled operation
+hide footbox
+
+actor User
+participant API_Service as AS
+participant MQ
+participant OperationEngine as OE
+participant Engine.Trigger as engine
+participant DB
+
+User -> AS:create scheduled operation
+activate AS
+
+AS --> MQ:create scheduled operation
+activate MQ
+
+MQ -> OE:create scheduled operation
+activate OE
+
+OE -> DB:create operation and set status to init
+activate DB
+
+DB --> OE:finish
+deactivate DB
+
+OE -> DB:get trigger info
+activate DB
+
+DB --> OE:trigger info
+deactivate DB
+
+OE -> engine:register operation
+activate engine
+
+engine --> OE:finish
+deactivate engine
+
+OE -> DB:update operation's status to registered
+activate DB
+
+DB --> OE:finish
+deactivate DB
+
+OE -> MQ:finish
+deactivate OE
+
+MQ -> AS:finish
+deactivate MQ
+
+AS --> User:finish
+deactivate AS
+
+@enduml
+

doc/source/specs/operation-engine/delete_scheduled_operation_seq_diagram.pu

@@ -0,0 +1,43 @@
+@startuml
+
+title delete scheduled operation
+hide footbox
+
+actor User
+participant API_Service as AS
+participant OperationEngine as OE
+participant Engine.Trigger as engine
+participant DB
+
+User -> AS:delete scheduled operation
+activate AS
+
+AS -> OE:delete scheduled operation
+activate OE
+
+OE -> DB:update operation's status to deleted
+activate DB
+
+DB --> OE:finish
+deactivate DB
+
+OE -> engine:unregister operation
+activate engine
+
+engine --> OE:finish
+deactivate engine
+
+OE -> DB:delete operation
+activate DB
+
+DB --> OE:finish
+deactivate DB
+
+OE -> AS:finish
+deactivate OE
+
+AS --> User:finish
+deactivate AS
+
+@enduml
+

doc/source/specs/operation-engine/operation_engine_class_diagram.pu

@@ -0,0 +1,79 @@
+@startuml
+
+title OperationEngine class diagram
+
+class OperationEngineManager
+abstract class BaseTrigger
+abstract class Executor
+class OperationManager
+class ScheduledOperation
+class TimeTrigger
+class EventTrigger
+interface Operation
+class OperationExeInfo
+class Trigger
+class ProtectOperation
+class DeleteCheckpointOperation
+
+OperationEngineManager *-- BaseTrigger
+OperationEngineManager *-- Executor
+OperationEngineManager *-- OperationManager
+BaseTrigger -- ScheduledOperation
+BaseTrigger "1" o-- "1" Executor
+Executor -- ScheduledOperation
+Executor -- OperationManager
+Executor -- OperationExeInfo
+OperationManager *-- Operation
+BaseTrigger <|-- TimeTrigger
+BaseTrigger <|-- EventTrigger
+ScheduledOperation -- Trigger
+Operation <|-- ProtectOperation
+Operation <|-- DeleteCheckpointOperation
+
+abstract class BaseTrigger {
+_executor: Executor
+
+register_operation(op_id, op_name, **kwargs)
+unregister_operation(op_id, op_name)
+}
+
+abstract class Executor {
+submit_op(op_id, info)
+}
+
+class ScheduledOperation {
+id: uuid
+name: string
+operation_type:string
+trigger_id:uuid
+operation_definition:dict
+}
+note left: class of DB table scheduled_operations
+
+class Trigger {
+id:uuid
+type:string
+properties:dict
+}
+note left: class of DB table triggers
+
+class OperationManager {
+operation_obj_map:dict
+
+check_operation_definition(op_type, operation_definition)
+execute_operation(op_type, operation_definition, operation_exe_info)
+}
+
+interface Operation {
+check_operation_definition(operation_definition)
+execute(operation_definition, operation_exe_info)
+}
+
+class OperationExeInfo {
+id:uuid
+extend_info:string
+}
+note "class of DB table operation_exe_infos" as N1
+OperationExeInfo -- N1
+
+@enduml

doc/source/specs/operation-engine/operation_engine_design.rst

@@ -0,0 +1,101 @@
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
+ License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+
+================
+Operation Engine
+================
+
+https://blueprints.launchpad.net/smaug/+spec/operation-engine-design
+
+Operation Engine is one of components in Smaug, which is responsable for triggering the operations
+to execute when the time is up or event happens.
+
+Problem description
+===================
+1.Define the operations and triggers(time or event)
+2.Bind the operation with trigger and activate the trigger
+3.Define the executor which will run the operations
+4.Ensure the high availablity of Operation Engin Service
+
+Use Cases
+---------
+1.CRUD operations and triggers
+
+Proposed Change
+===============
+
+Data model impact
+-----------------
+There are 5 relevant talbes in th DB.
+1. triggers
+2. services
+3. scheduled_operations
+   These three tables are defined at `https://blueprints.launchpad.net/openstack/?searchtext=api-service-design`.
+   Please see the bp to get the details.
+
+4. scheduled_operation_states
++--------------------+--------------+------+-----+---------+----------------+
+| Field              | Type         | Null | Key | Default |      Extra     |
++--------------------+--------------+------+-----+---------+----------------+
+| created_at         | datetime     | YES  |     | NULL    |                |
+| updated_at         | datetime     | YES  |     | NULL    |                |
+| deleted_at         | datetime     | YES  |     | NULL    |                |
+| id                 | init(11)     | NO   | PRI | NULL    | auto_increment |
+| operation_id       | varchar(36)  | NO   |     | NULL    |                |
+| service_id         | int(11)      | NO   |     | NULL    |                |
+| state              | varchar(32)  | NO   |     | NULL    | values: init   |
+|                    |              |      |     |         | registered     |
+|                    |              |      |     |         | triggered      |
+|                    |              |      |     |         | running        |
+|                    |              |      |     |         | deleted        |
+| deleted            | tinyint(1)   | NO   |     | NULL    |                |
++--------------------+--------------+------+-----+---------+----------------+
+FOREIGN KEY(operation_id) REFERENCES scheduled_operations(id)
+FOREIGN KEY(service_id)   REFERENCES Services(id)
+
+Please see the `images/operation-engine/operation_state_diagram.png` to get the state transition
+of scheduled_operation_states.state
+
+
+5. scheduled_operation_logs
++--------------------+--------------+------+-----+---------+------------------------+
+| Field              | Type         | Null | Key | Default |           Extra        |
++--------------------+--------------+------+-----+---------+------------------------+
+| created_at         | datetime     | YES  |     | NULL    |                        |
+| updated_at         | datetime     | YES  |     | NULL    |                        |
+| deleted_at         | datetime     | YES  |     | NULL    |                        |
+| id                 | int(11)      | NO   | PRI | NULL    | auto_increment         |
+| operation_id       | varchar(36)  | NO   |     | NULL    |                        |
+| expect_start_time  | datetime     | YES  |     | NULL    |                        |
+| triggered_time     | datetime     | YES  |     | NULL    |                        |
+| actual_start_time  | datetime     | YES  |     | NULL    |                        |
+| end_time           | datetime     | YES  |     | NULL    |                        |
+| state              | varchar(32)  | NO   |     | NULL    | values:                |
+|                    |              |      |     |         |  in_progress           |
+|                    |              |      |     |         |  sucdess               |
+|                    |              |      |     |         |  failed                |
+|                    |              |      |     |         |  dropped_out_of_window |
+| extend_info        | Text         | YES  |     | NULL    | execution info         |
+| deleted            | tinyint(1)   | NO   |     | NULL    |                        |
++----------------------+--------------+------+-----+---------+----------------------+
+FOREIGN KEY(operation_id) REFERENCES scheduled_operations(id)
+
+instance:
++---------------------+---------------------+---------------------+-----+----------------------------------+
++---------------------+---------------------+---------------------+---------------------+------------------+
++---------------------+---------------------+
+| created_at          | updated_at          | deleted_at          | id  | opration_id                      |
+| expect_start_time   | triggered_time      | actual_start_time   | end_time            | status           |
+| extend_info         | deleted             |
++---------------------+---------------------+---------------------+-----+----------------------------------+
++---------------------+---------------------+---------------------+---------------------+------------------+
++---------------------+---------------------+
+| 2016-01-01 01:00:02 | 2016-01-01 01:00:07 | NULL                | 0   | 0354ca9ddcd046b693340d78759fd274 |
+| 2016-01-01 01:00:00 | 2016-01-01 01:00:02 | 2016-01-01 01:00:05 | 2016-01-01 01:00:07 | success          |
+| NULL                | 0                   |
++---------------------+---------------------+---------------------+-----+----------------------------------+
++---------------------+---------------------+---------------------+---------------------+------------------+
++---------------------+---------------------+

doc/source/specs/operation-engine/operation_state_diagram.pu

@@ -0,0 +1,20 @@
+@startuml
+
+[*] --> init: create
+
+init --> registered: register to triggers
+
+registered --> triggered: trigger and submit to executor
+
+triggered --> running: executor runs it
+
+running --> registered: finish
+
+init --> deleted: delete
+registered --> deleted: delete
+triggered --> deleted: delete
+running --> deleted: delete
+
+deleted --> [*]
+
+@enduml