diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst new file mode 100644 index 00000000..55c76d92 --- /dev/null +++ b/doc/source/architecture.rst @@ -0,0 +1,70 @@ +Climate architecture +==================== + +Climate design can be described by following diagram: + +.. image:: images/climate-architecture.png + :width: 700 px + :scale: 99 % + :align: left + +**climate-client** - provides the opportunity to communicate with Climate via +*REST API* (climate-api service). + +**climate-api** - waits for the REST calls from the outside world to redirect +them to the manager. climate-api communicates with climate-manager via RPC. +Runs as a separated process. + +**climate-manager** - implements all logic and operations with leases, +reservations and events. Communicates with Climate DB and stores there data +structure of connected leases, reservations (both physical and virtual) and +events. climate-manager service is responsible for running events created for +lease and process all actions that should be done this moment. Manager uses +resource-plugins to work with concrete resources (instances, volumes, compute +hosts). climate-manager uses Keystone trusts to commit actions on behalf of +user who has created lease before. + +**resource-plugin** - responsible for exact actions to do with reserved +resources (VMs, volumes, etc.) When working knows only about resource ID and +token to use. All resource plugins work in the same process as climate-manager. + +Virtual instance reservation +---------------------------- + +Virtual instance reservation mostly looks like usual instance booting for user +- he/she only passes special hints to Nova containing information about future +lease - lease start and end dates, its name, etc. Special Nova API extensions +parse these parameter and use them to call Climate, passing to it ID of just +created instance. If there is a need to reserve all instances in cloud (like in +developer labs to automate process of resource reclaiming), default reservation +extension might be used. By default it starts lease at the moment of request +and gives it one month of lifetime. + +During the time lease has not started yet, instance will be shelved. + +Compute host reservation +------------------------ + +Now process of compute hosts reserving contains two steps: + +* admin marks hosts from common pool as possible to be reserved. That is + implemented by moving these hosts to special aggregate; +* user asks for reserving of host with specified characteristics like: + + * the region + * the availability zone + * the host capabilities extra specs (scoped and non-scoped format is + accepted) + * the number of CPU cores + * the amount of free RAM + * the amount of free disk space + * the number of hosts + +Technically speaking, resource ID here will be not host ID, because there might +be many of them wanted. Resource here will be new aggregate containing reserved +hosts. The time lease starts, user may use reserved compute capacity to run +his/her instances on it passing special scheduler hint to Nova. When host is +reserved, it’s not used for usual instance running, it might be used only when +lease starts and only by passing reservation ID to Nova. That is implemented +using special Nova Scheduler filter, that passes reservation ID to Climate and +checks if user really can use reserved compute capacity. \ No newline at end of file diff --git a/doc/source/images/climate-architecture.png b/doc/source/images/climate-architecture.png new file mode 100644 index 00000000..981f7ff3 Binary files /dev/null and b/doc/source/images/climate-architecture.png differ diff --git a/doc/source/index.rst b/doc/source/index.rst index 46ea298b..32942c3f 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -5,9 +5,22 @@ Climate is an OpenStack service to provide resource reservations in the OpenStack cloud for different resource types - both virtual (instances, volumes, stacks) and physical (hosts). +Overview +-------- + +.. toctree:: + :maxdepth: 1 + + introduction + architecture + Roadmap + User guide ---------- + +**APIs** + .. toctree:: :maxdepth: 1 - rest_api_v1.0 + restapi/index diff --git a/doc/source/introduction.rst b/doc/source/introduction.rst new file mode 100644 index 00000000..f165986b --- /dev/null +++ b/doc/source/introduction.rst @@ -0,0 +1,105 @@ +Introduction +============ + +Idea of creating Climate originated with two different use cases: + +* Compute host reservation (when user with admin privileges can reserve + hardware resources that are dedicated to the sole use of a tenant) +* Virtual machine (instance) reservation (when user may ask reservation service + to provide him working VM not necessary now, but also in the future) + +Now these ideas have been transformed to more general view: with Climate user +can request the resources of cloud environment to be provided (“leased”) to his +project for specific amount on time, immediately or in future. + +Both virtual (Instances, Volumes, Networks) and hardware (full hosts with +specific characteristics of RAM, CPU and etc) resources can be allocated via +“lease”. + +In terms of benefits added, Resource Reservation Service will: + +* improve visibility of cloud resources consumption (current and planned for + future); +* enable cloud resource planning based on current and future demand from end + users; +* automate the processes of resource allocation and reclaiming; +* provide energy efficiency for physical hosts (both compute and storage ones); +* potentially provide leases as billable items for which customers can be + charged a flat fee or a premium price depending on amount of reserved cloud + resources and their usage. + +Glossary of terms +----------------- + +**Reservation** is an allocation of certain cloud resource (Nova instance, Cinder +volume, compute host, etc.) to particular project. Speaking about virtual +reservations, we may have not only simple, solid ones (like already mentioned +instances and volumes), but also complex ones - like Heat stacks and Savanna +clusters. Reservation is characterized by status, resource type and identifier +and lease it belongs to. + +**Lease** is a negotiation agreement between the provider (Climate, using OpenStack +resources) and the consumer (user) where the former agrees to make some kind of +resources (both virtual and physical) available to latter, based on a set of +lease terms presented by the consumer. Here lease may be described as contract +between user and reservation service about cloud resources to be provided right +now or later. Technically speaking, lease is a group of reservations granted to +particular project upon request. Lease is characterized by start time, end +time, set of individual reservations and associated events. + +**Event** is simply something that may happen to lease. In most simple case event +might describe lease start and lease end. Also it might be notification to user +(e.g. about soon lease expiration) and some extra actions. + +Rationale +--------- + +Climate is created to: + +* manage cloud resources not only right now, but also in the future; +* have dedicated recourses on certain amount of time; +* prepare for the peak loads and perform capacity planning; +* optimise energy consumption. + +Lease types (concepts) +---------------------- + +* **Immediate reservation**. Resources are provisioned immediately (like VM + boot or moving host to reserved user aggregate) or not at all. If request can + be fulfilled, lease is created and **success** status is returned. Lease + should be marked as **active** or **to_be_started**. Otherwise (if + request resource cannot be provisioned right now) failure status for this + request should be returned. +* **Reservation with retries**. Mostly looks like previous variant, but in case + of failure, user may want to have several (configurable number) retries to + process lease creation action. In this case request will be processed till + that will be possible to create lease but not more than set in configuration + file number of times. +* **Best-effort reservation**. Also might have place if lease creation request + cannot be fulfilled immediately. Best-effort mechanism starts something like + scavenger hunt trying to find resources for reservations. For compute hosts + reservation that makes much sense, because in case there are instances + belonging to other tenant on eligible hosts, and without them there will be + possible to reserve these hosts, Climate may start instances migration. + This operation can be timely and fairly complex and so different strategies + may be applied depending on heuristic factors such as the number, type and + state of the instances to be migrated. Also Climate should assert that there + are at least enough potential candidates for the migration prior to starting + the actual migration. If Climate decides to start migration, it returns + **success** state and marks lease as **in_progress**, otherwise - + **failure**. If this 'hunting' ends successfully before configurable + timeout has passed, lease should be marked as **active**, otherwise its + status is set to **timedout**. +* **Delayed resource acquiring** or **scheduled reservation**. In this + reservation type lease is created successfully if Climate thinks there will + be enough resources to process provisioning later (otherwise this request + returns **failure** status). Lease is marked as **inactive** till all + resources will be actually provisioned. That works pretty nice and + predictable speaking about compute hosts reservation (because hosts as + resources are got not from common cloud pool, but from admin defined pool). + So Climate is possible to predict these physical resources usage and use that + information during lease creation. If we speak about virtual reservations, + here situation is more complicated, because all resources are got from common + cloud resources pool, and Climate cannot guarantee there will be enough + resources to provision them. In this failure case lease state will be marked + as **error** with appropriate explanation. \ No newline at end of file diff --git a/doc/source/restapi/index.rst b/doc/source/restapi/index.rst new file mode 100644 index 00000000..5071bcc0 --- /dev/null +++ b/doc/source/restapi/index.rst @@ -0,0 +1,9 @@ +Climate REST API docs +********************* + +This page includes documentation for Climate APIs. + +.. toctree:: + :maxdepth: 1 + + rest_api_v1.0 diff --git a/doc/source/rest_api_v1.0.rst b/doc/source/restapi/rest_api_v1.0.rst similarity index 100% rename from doc/source/rest_api_v1.0.rst rename to doc/source/restapi/rest_api_v1.0.rst