Proposed Smaug API v1.0

Easier link view  of the proposed API
http://editor.swagger.io/#/#%2Fc%2F244756%2F16%2Fdoc%2Fsource%2Fapi%2Fsmaug_api.v1.yaml

https://docs.google.com/presentation/d/1JYO1VIlTkGTF6lvKEMcsHkaST3mYFxuarpcNTJ3HBhk/edit?pref=2&pli=1#slide=id.p20

Implements: blueprint api-data-model
Change-Id: I1eb3c4c80da74cfedf589b9446c55c80955c40d3
This commit is contained in:
Saggi Mizrahi 2015-11-12 18:40:30 +02:00 committed by Eran Gampel
parent cc883a2e24
commit 5ee58942f1
5 changed files with 1093 additions and 0 deletions

140
doc/source/api/bank.md Normal file
View File

@ -0,0 +1,140 @@
# Bank basics
*** :exclamation: This is still a work in progress ***
This document will describe the layout and algorithms used by smaug using the
bank interface.
## Abstract
Since Smaug want's to be able to store metadata in many locations (swift, mongodb, etc.)
we defined a simplified object store interface that we believe most backends will be able
to support without much work.
But the simplified interface doesn't describe how Smaug will do its higher
level operations and how the higher level logic will be layed out in the object
store. This is why we need higher level logic defined explicitly so that later
we could use higher level bank functions knowing they are correct, safe and atomic.
## Layout
### Checkpoint directory
`/checkpoints/<checkpoint_id>/index.json`
#### Example content
```json
{
"trigger": {},
#ISO 8601 time UTC
"started_at": "2015-10-29T13:41:02Z",
"status": "in_progress",
"plan": {},
"plugins": {
"plugin_1": {
"name": "cinder volume",
"version": "1.2.3",
}
}
}
```
### Protection definition directory
`/checkpoints/<Checkpoint_id>/<protection_definition_id>/index.json`
#### Example content
```json
{
"plugin": "plugin_1",
"dependencies": {
"vol1": "<other_protection_defintion_id>"
}
}
```
### Protection definition plugin data directory
`/checkpoints/<checkpoint_id>/<protection_defintion_id>/plugin_data/*`
## Checkpoint Creation Process
Create new Checkpoint with id <CHECKPOINT-ID>;
1. Acquire checkpoint lock
* action acquire_lock
* id: `<CHECKPOINT-ID>`
2. Create checkpoint pointer
* action: write_object
* path: `/indices/unfinished_checkpoints/<CHECKPOINT-ID>`,
* buffer: `<CHECKPOINT-ID>`
3. Create checkpoint
* action: write_object
* path: `/checkpoints/<CHECKPOINT-ID>/index.json`,
* buffer:
```json
{
"smaug_version": "1.0.0",
"status": "in_progress",
"plugins": {}
}
```
4. Run plugins
5. Checkpoint finished but indices not yet created
* action: write_object
* path: `/checkpoints/<CHECKPOINT-ID>/index.json`,
* buffer:
```json
{
"smaug_version": "1.0.0",
"status": "creating_indices",
"plugins": {}
}
```
6. Create index 'plan' (Example, there could be any number of indexes)
* action: write_object
* path: `/indices/by_plan/<PLAN-ID>/<CHECKPOINT-ID>`
* buffer: `<CHECKPOINT-ID>`
7. Remove checkpoint pointer
* action: delete_object
* path: `/indices/unfinished_checkpoints/<CHECKPOINT-ID>`
8. Release checkpoint lock
* action: release_lock
* id: `<CHECKPOINT-ID>`
## Delete Checkpoint
1. Acquire checkpoint lock
* action acquire_lock
* id: `<CHECKPOINT-ID>`
2. Create checkpoint pointer
* action: write_object
* path: `/indices/unfinished_checkpoints/<CHECKPOINT-ID>`,
* buffer: `<CHECKPOINT-ID>`
3. Mark transaction as being deleted
* action: write_object
* path: `/checkpoints/<CHECKPOINT-ID>/index.json`,
* buffer:
```json
{
"smaug_version": "1.0.0",
"status": "deleting",
"plugins": {}
}
```
4. Remove indices
- Remove index 'plan' (Example, there could be any number of indexes)
* action: delete_object
* path: `/indices/by_plan/<PLAN-ID>/<CHECKPOINT-ID>`
6. Run plugins
7. Delete checkpoint file
* action: delete_object
* path: `/checkpoints/<CHECKPOINT-ID>/index.json`,
8. Remove checkpoints pointer
* action: delete_object
* path: `/indices/unfinished_checkpoints/<CHECKPOINT-ID>`
9. Release checkpoint lock
* action: release_lock
* id: `<CHECKPOINT-ID>`

View File

@ -0,0 +1,145 @@
@startuml
title "Smaug API model"
class Protectable {
name: string
instances: []Resource
is_root: bool
}
Protectable --> Resource: lists
class Resource {
id: UUID
type: ResourceType
schema: JSONSchema
dependent_resources: []Resource
}
class Trigger {
}
class TimedTrigger extends Trigger {
}
class EventTrigger extends Trigger {
}
class Checkpoint {
id: UUID
tenant_id: UUID
plan: ProtectionPlan
status: string
started_at: DateTime
}
Checkpoint *-> ProtectionPlan: stores a copy of
class AutomaticOperation {
id: UUID
name: string
comments: string
tenant_id: UUID
}
class PlanRetention extends AutomaticOperation {
daily: bool
weekly: bool
monthly: bool
yearly: bool
protection_plan: ProtectionPlan
}
class ScheduledOperation <<abstract>> extends AutomaticOperation {
trigger: Trigger
}
ScheduledOperation *- Trigger: when should the operation should trigger
class BackupPlan extends ScheduledOperation {
protection_plan: ProtectionPlan
}
BackupPlan *--> ProtectionPlan
class DeleteCheckpoints extends ScheduledOperation {
query: string
protection_provider: ProtectionProvider
}
class ProtectionProvider {
name: string
description: string
extended_info_schema: [ResourceType]JSONSchema
options_schema: [ResourceType]JSONSchema
checkpoints: []Checkpoint
}
ProtectionProvider o-> Checkpoint: lists
class ProtectionPlan {
id: UUID
is_enabled: boolean
name: string
comments: string
resources: []Resource
protection_provider: ProtectionProvider
parameters: dict
Start()
Suspend()
}
ProtectionPlan "1" *--> "N" Resource: aggragates
ProtectionPlan -> ProtectionProvider
class RestoreTarget {
keystone_uri: URI
}
class Restoration {
template: string
target: RestoreTarget
provider: ProtectionProvider
checkpoint: Checkpoint
started_at: string
}
Restoration *-> RestoreTarget: restores to
@enduml
@startuml
package "Plugin Examples" {
interface BackupPlugin <<action set>> {
<<protect>> create_backup(resource: Resource, bank: Bank)
<<delete>> delete_backup(backup: Backup)
}
interface ReplicationPlugin <<action set>> {
<<enable>> start_replication(resource: Resource, replicationType: eReplicationType)
<<protect>> create_snapshot_from_replicated_target(resource: Resource, replicationType: eReplicationType)
<<delete>> delete_snapshot(resource: Resource, replicationType: eReplicationType)
<<disable>> stop_replication(resource: Resource)
}
ReplicationPlugin -> eReplicationType
interface SnapshotPlugin <<action set>> {
<<protect>> create_snapshot(resource: Resource)
<<delete>> delete_snapshot(snapshot: Snapshot)
}
enum eReplicationType {
hypervisor
cinder
}
}
@enduml

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 32 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 7.0 KiB

View File

@ -0,0 +1,806 @@
swagger: '2.0'
info:
title: Smaug API
description: Protect all you hold dear
version: 0.99.0
host: api.smaug.nowhere.com
schemes:
- https
basePath: /v1
produces:
- application/json
paths:
/providers:
get:
summary: Providers
description: |
The Providers endpoint returns information about the providers
offered at a given service. All providers need to be configuered
first by the admin.
parameters:
- $ref: '#/parameters/nameFilterParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Protection Provider
responses:
'200':
description: An array of providers
schema:
type: array
items:
$ref: '#/definitions/Provider'
examples:
application/json: [
{
"id": "2220f8b1-975d-4621-a872-fa9afb43cb6c",
"name": "OS Infra Provider",
"description": "This provider uses OpenStack's own services (swift, cinder) as storage",
"extended_info_schema": {
"OS::Nova::Cinder": {
"type": "object",
"properties": {
"use_cbt": {
"type": "boolean",
"title": "Use CBT",
"description": "Use Changed Block Tracking when backin up this volume"
}
}
}
}
}
]
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/providers/{provider_id}/checkpoints:
get:
summary: List checkpoints
description: |
The checkpoints endpoint returns information about the checkpoints
offered at a given provider.
parameters:
- $ref: '#/parameters/provider_idParam'
- name: tenant_id
in: query
description: |
Specifies the ID of the tenant that owns this entity
type: string
format: uuid
required: false
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Protection Provider
- Checkpoint
responses:
'200':
description: An array of checkpoints
schema:
type: array
items:
$ref: '#/definitions/Checkpoint'
examples:
application/json: [
{
"id": "2220f8b1-975d-4621-a872-fa9afb43cb6c",
"status": "comitted",
"ptoection_plan": {
"id": "2a9ce1f3-cc1a-4516-9435-0ebb13caa398",
"name": "My 3 tier application",
"resources": [
{
"id": "64e51e85-4f31-441f-9a5d-6e93e3196628",
"type": "OS::Nova::Server"
},
{
"id": "61e51e85-4f31-441f-9a5d-6e93e3196628",
"type": "OS::Cinder::Volume"
},
{
"id": "62e51e85-4f31-441f-9a5d-6e93e3196628",
"type": "OS::Cinder::Volume"
}
]
}
}]
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
post:
summary: Checkpoints
description: |
The Checkpoints endpoint creates a checkpoint
at a given provider.
parameters:
- $ref: '#/parameters/provider_idParam'
- name: checkpoint
in: body
required: true
schema:
$ref: '#/definitions/Checkpoint'
tags:
- Protection Provider
- Checkpoint
responses:
'200':
description: Checkpoint created
schema:
$ref: '#/definitions/Checkpoint'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/providers/{provider_id}/checkpoints/{checkpoint_id}:
delete:
summary: Delete checkpoints
description: |
The checkpoint endpoint deletes a checkpoint
at a given provider.
parameters:
- $ref: '#/parameters/provider_idParam'
- $ref: '#/parameters/checkpoint_idParam'
tags:
- Protection Provider
- Checkpoint
responses:
'200':
description: Checkpoint deleted
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/{tenant_id}/plans:
get:
summary: Get protection plans
description: |
The Plans endpoint returns information about the protection plans
offered for the given tenant.
parameters:
- $ref: '#/parameters/tenantParam'
- $ref: '#/parameters/nameFilterParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Tenant API
- Protection Plan
responses:
'200':
description: An array of protection plans
schema:
type: array
items:
$ref: '#/definitions/ProtectionPlan'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
post:
summary: Create a plan
description: |
Create a new plan. The update will create a new revision for
the plan.
tags:
- Tenant API
- Protection Plan
parameters:
- $ref: '#/parameters/tenantParam'
- name: plan
in: body
required: true
schema:
$ref: '#/definitions/ProtectionPlan'
responses:
'200':
description: The new checkpoint
schema:
$ref: '#/definitions/ProtectionPlan'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/{tenant_id}/plans/{plan_id}:
get:
summary: Protection Plan
description: |
The Plan endpoint returns information about a specific plan.
parameters:
- $ref: '#/parameters/tenantParam'
- $ref: '#/parameters/plan_idParam'
tags:
- Tenant API
- Protection Plan
responses:
'200':
description: the protection plan
schema:
$ref: '#/definitions/ProtectionPlan'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
delete:
summary: Protection Plan
description: |
The Plan endpoint deletes a specific plan.
parameters:
- $ref: '#/parameters/tenantParam'
- $ref: '#/parameters/plan_idParam'
tags:
- Tenant API
- Protection Plan
responses:
'200':
description: the protection plan
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/protectables:
get:
summary: Protectables
description: |
Return all the available protectable types.
parameters:
- $ref: '#/parameters/nameFilterParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Protectable
responses:
'200':
description: The available protectable types
schema:
type: array
items:
$ref: '#/definitions/ProtectableType'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/protectables/{protectable_type}:
get:
summary: Protectables
description: |
Return the information of a given protectable type.
parameters:
- $ref: '#/parameters/protectable_typeParam'
tags:
- Protectable
responses:
'200':
description: The protectable information
schema:
$ref: '#/definitions/ProtectableInfo'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/protectables/{protectable_type}/instances:
get:
summary: Resource Instances
description: |
Return all the available instances for the given protectable type.
parameters:
- $ref: '#/parameters/protectable_typeParam'
- $ref: '#/parameters/nameFilterParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Protectable
- Resource
responses:
'200':
description: The available instances for the protectable type.
schema:
type: array
items:
$ref: '#/definitions/Resource'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/{tenant_id}/scheduled_operations:
get:
summary: Scheduled Operations
description: |
Scheduled operations are operations that will be executed when
a specific trigger is triggered.
parameters:
- $ref: '#/parameters/tenantParam'
- $ref: '#/parameters/nameFilterParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Tenant API
- Scheduled Operation
responses:
'200':
description: An array of scheduled operations
schema:
type: array
items:
$ref: '#/definitions/AutomaticOperation'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
post:
summary: Scheduled operation
description: |
Create a new scheduled operation.
tags:
- Tenant API
- Scheduled Operation
parameters:
- $ref: '#/parameters/tenantParam'
- name: scheduled_operation
in: body
required: true
schema:
$ref: '#/definitions/AutomaticOperation'
responses:
'200':
description: The new scheduled operation
schema:
$ref: '#/definitions/AutomaticOperation'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/{tenant_id}/operations:
get:
summary: Operations
description: |
List the history operations for a given tenant.
parameters:
- $ref: '#/parameters/tenantParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Tenant API
- Operation
responses:
'200':
description: An array of operations.
schema:
type: array
items:
$ref: '#/definitions/OperationDefinition'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
post:
summary: Operations
description: |
Trigger an operations to execute manually.
parameters:
- $ref: '#/parameters/tenantParam'
- name: operation
in: body
required: true
schema:
$ref: '#/definitions/OperationDefinition'
tags:
- Tenant API
- Operation
responses:
'200':
description: An operation is excuting.
schema:
$ref: '#/definitions/OperationDefinition'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/{tenant_id}/triggers:
get:
summary: Triggers
description: |
List all of the triggers created by a given tenant.
parameters:
- $ref: '#/parameters/tenantParam'
- $ref: '#/parameters/nameFilterParam'
- $ref: '#/parameters/sortParam'
- $ref: '#/parameters/limitParam'
- $ref: '#/parameters/markerParam'
tags:
- Tenant API
- Trigger
responses:
'200':
description: An array of triggers
schema:
type: array
items:
$ref: '#/definitions/Trigger'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
post:
summary: Trigger
description: |
Create a new scheduled operation.
tags:
- Tenant API
- Trigger
parameters:
- $ref: '#/parameters/tenantParam'
- name: trigger_info
in: body
required: true
schema:
$ref: '#/definitions/Trigger'
responses:
'200':
description: The new created trigger
schema:
$ref: '#/definitions/Trigger'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
/{tenant_id}/triggers/{trigger_id}:
delete:
summary: Trigger
description: |
Delete a trigger created by a given tenant.
parameters:
- $ref: '#/parameters/tenantParam'
- name: trigger_id
in: path
required: true
type: string
format: uuid
tags:
- Tenant API
- Trigger
responses:
'200':
description: Trigger deleted
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
definitions:
Provider:
type: object
required: [ name ]
properties:
id:
readOnly: true
type: string
format: UUID
description: |
Unique identifier representing a specific protection provider.
description:
type: string
description: Description of provider.
name:
type: string
description: Display name of provider.
Checkpoint:
type: object
properties:
id:
readOnly: true
type: string
format: UUID
description: |
Unique identifier representing a specific protection provider.
protection_plan:
readOnly: true
$ref: '#/definitions/ProtectionPlan'
status:
readOnly: true
type: string
Resource:
type: object
properties:
type:
$ref: '#/definitions/ProtectableType'
readOnly: true
id:
readOnly: true
type: string
ProtectionPlan:
type: object
required: [ name, protection_provider_id, resources ]
properties:
id:
readOnly: true
type: string
format: UUID
description: Unique identifier representing a specific protection protection plan.
name:
type: string
description: Display name of plan.
comments:
type: string
description: Comments about the plan.
resources:
type: array
items:
$ref: '#/definitions/Resource'
protection_provider_id:
type: string
format: UUID
description: |
Unique identifier representing a specific protection provider that
will store checkpoints for this protection plan.
parameters:
type: object
description: TODO
ProtectableType:
type: string
format: Heat Type String
example: "OS::Nova::Server"
description: |
Name of the resource type. When available the types that are defined by Heat
are used.
ProtectableInfo:
type: object
properties:
name:
$ref: '#/definitions/ProtectableType'
is_root:
type: boolean
description: |
Defines whether this type has any dependencies or not. Useful for
UIs.
dependent_types:
type: array
description: |
List of types that might depend on this type. For example an
"OS::Nova::Server" has "OS::Cinder::Volume" as a dependent type.
items:
$ref: '#/definitions/ProtectableType'
OperationDefinition:
type: object
discriminator: type
required: [ type ]
properties:
id:
type: string
format: UUID
description: |
Unique identifier representing a specific operation definition.
type:
type: string
description: |
Type of the operation. This defines what kind of operation this
object defines the arguments for.
ProtectOperationDefinition:
description: |
Operation definition for protect operation.
allOf:
- $ref: '#/definitions/OperationDefinition'
- type: object
properties:
protection_plan_id:
type: string
format: UUID
parameters:
type: object
format: dict
required: [ protection_plan_id ]
DeleteOperationDefinition:
description: |
Operation definition for delete operation.
allOf:
- $ref: '#/definitions/OperationDefinition'
- type: object
properties:
checkpoint_path:
type: string
provider_id:
type: string
format: UUID
required: [ checkpoint_path, provider_id ]
StartOperationDefinition:
description: |
Operation definition for start operation.
allOf:
- $ref: '#/definitions/OperationDefinition'
- type: object
properties:
protection_plan_id:
type: string
format: UUID
required: [ protection_plan_id ]
SuspendOperationDefinition:
description: |
Operation definition for suspend operation.
allOf:
- $ref: '#/definitions/OperationDefinition'
- type: object
properties:
protection_plan_id:
type: string
format: UUID
required: [ protection_plan_id ]
RestoreOperationDefinition:
description: |
Operation definition for restore operation.
allOf:
- $ref: '#/definitions/OperationDefinition'
- type: object
properties:
checkpoint_id:
type: string
format: UUID
provider_id:
type: string
format: UUID
restore_target:
type: string
format: UUID
parameters:
type: object
format: dict
required: [ checkpoint_id, provider_id ]
AutomaticOperation:
type: object
properties:
id:
type: string
format: UUID
description: |
Unique identifier representing a specific automatic operation.
name:
type: string
description: Display name of automatic operation.
comments:
type: string
description: Comments about the automatic operation.
ScheduledOperation:
type: object
allOf:
- $ref: '#/definitions/AutomaticOperation'
- type: object
properties:
trigger_id:
type: string
format: UUID
- $ref: '#/definitions/OperationDefinition'
OperationStatus:
type: object
properties:
status:
type: string
description:
type: string
Trigger:
type: object
discriminator: type
required: [ type ]
properties:
id:
type: string
format: UUID
description: |
Unique identifier representing a specific trigger.
name:
type: string
description: Display name of trigger.
description:
type: string
description: Description of trigger.
type:
type: string
description: |
Type of the trigger. This defines what kind of trigger this
object defines the arguments for.
TimeTrigger:
description: |
Trigger definition for time tigger.
allOf:
- $ref: '#/definitions/Trigger'
- type: object
properties:
start_time:
type: string
format: date-time
trigger_window:
type: integer
format: int32
recurrence:
type: string
required: [ start_time, trigger_window, recurrence ]
Error:
type: object
properties:
code:
type: integer
format: int32
message:
type: string
fields:
type: string
parameters:
sortParam:
name: sort
in: query
description: |
Comma-separated list of sort keys and optional sort directions in the
form of '&lt;key&gt;[&#58;&lt;direction&gt;]`. A valid direction is asc
(ascending) or desc (descending).
required: false
type: string
limitParam:
name: limit
in: query
description: |
Requests a specified page size of returned items from the query.
Returns a number of items up to the specified limit value.
Use the limit parameter to make an initial limited request and use the
ID of the last-seen item from the response as the marker parameter value
in a subsequent limited request.
type: integer
format: int64
markerParam:
name: marker
in: query
description: |
Specifies the ID of the last-seen item. Use the limit parameter to make
an initial limited request and use the ID of the last-seen item from the
response as the marker parameter value in a subsequent limited request.
type: string
tenantParam:
name: tenant_id
in: path
description: |
Specifies the ID of the tenant that owns this entity
type: string
format: uuid
required: true
nameFilterParam:
name: name
in: query
format: regex
description: name of the entity. Could be a regex pattern.
required: false
type: string
provider_idParam:
name: provider_id
type: string
format: uuid
required: true
in: path
description: id of the provider.
plan_idParam:
name: plan_id
type: string
format: uuid
required: true
in: path
description: id of the plan.
checkpoint_idParam:
name: checkpoint_id
type: string
format: uuid
required: true
in: path
description: id of the checkpoint.
protectable_typeParam:
name: protectable_type
type: string
format: Heat Type String
required: true
in: path
description: the resource type.