Start moving Climate docs from wiki to Sphinx docs

* add common introduction and architecture information
* create more appropriate structure for REST API docs

Change-Id: I27b95aa76814f5d6bf6737c56594c7e5ba9d667c
Partially implements: blueprint climate-docs

doc/source/architecture.rst

@@ -0,0 +1,70 @@
+Climate architecture
+====================
+
+Climate design can be described by following diagram:
+
+.. image:: images/climate-architecture.png
+    :width: 700 px
+    :scale: 99 %
+    :align: left
+
+**climate-client** - provides the opportunity to communicate with Climate via
+*REST API* (climate-api service).
+
+**climate-api** - waits for the REST calls from the outside world to redirect
+them to the manager. climate-api communicates with climate-manager via RPC.
+Runs as a separated process.
+
+**climate-manager** - implements all logic and operations with leases,
+reservations and events. Communicates with Climate DB and stores there data
+structure of connected leases, reservations (both physical and virtual) and
+events. climate-manager service is responsible for running events created for
+lease and process all actions that should be done this moment. Manager uses
+resource-plugins to work with concrete resources (instances, volumes, compute
+hosts). climate-manager uses Keystone trusts to commit actions on behalf of
+user who has created lease before.
+
+**resource-plugin** - responsible for exact actions to do with reserved
+resources (VMs, volumes, etc.) When working knows only about resource ID and
+token to use. All resource plugins work in the same process as climate-manager.
+
+Virtual instance reservation
+----------------------------
+
+Virtual instance reservation mostly looks like usual instance booting for user
+- he/she only passes special hints to Nova containing information about future
+lease - lease start and end dates, its name, etc. Special Nova API extensions
+parse these parameter and use them to call Climate, passing to it ID of just
+created instance. If there is a need to reserve all instances in cloud (like in
+developer labs to automate process of resource reclaiming), default reservation
+extension might be used. By default it starts lease at the moment of request
+and gives it one month of lifetime.
+
+During the time lease has not started yet, instance will be shelved.
+
+Compute host reservation
+------------------------
+
+Now process of compute hosts reserving contains two steps:
+
+* admin marks hosts from common pool as possible to be reserved. That is
+  implemented by moving these hosts to special aggregate;
+* user asks for reserving of host with specified characteristics like:
+
+  * the region
+  * the availability zone
+  * the host capabilities extra specs (scoped and non-scoped format is
+    accepted)
+  * the number of CPU cores
+  * the amount of free RAM
+  * the amount of free disk space
+  * the number of hosts
+
+Technically speaking, resource ID here will be not host ID, because there might
+be many of them wanted. Resource here will be new aggregate containing reserved
+hosts. The time lease starts, user may use reserved compute capacity to run
+his/her instances on it passing special scheduler hint to Nova. When host is
+reserved, it’s not used for usual instance running, it might be used only when
+lease starts and only by passing reservation ID to Nova. That is implemented
+using special Nova Scheduler filter, that passes reservation ID to Climate and
+checks if user really can use reserved compute capacity.

doc/source/index.rst

@@ -5,9 +5,22 @@ Climate is an OpenStack service to provide resource reservations in the
 OpenStack cloud for different resource types - both virtual (instances,
 volumes, stacks) and physical (hosts).
 
-User guide
-----------
+Overview
+--------
+
 .. toctree::
     :maxdepth: 1
 
-   rest_api_v1.0
+    introduction
+    architecture
+    Roadmap <https://wiki.openstack.org/wiki/Climate/Roadmap>
+
+User guide
+----------
+
+**APIs**
+
+.. toctree::
+   :maxdepth: 1
+
+   restapi/index

doc/source/introduction.rst

@@ -0,0 +1,105 @@
+Introduction
+============
+
+Idea of creating Climate originated with two different use cases:
+
+* Compute host reservation (when user with admin privileges can reserve
+  hardware resources that are dedicated to the sole use of a tenant)
+* Virtual machine (instance) reservation (when user may ask reservation service
+  to provide him working VM not necessary now, but also in the future)
+
+Now these ideas have been transformed to more general view: with Climate user
+can request the resources of cloud environment to be provided (“leased”) to his
+project for specific amount on time, immediately or in future.
+
+Both virtual (Instances, Volumes, Networks) and hardware (full hosts with
+specific characteristics of RAM, CPU and etc) resources can be allocated via
+“lease”.
+
+In terms of benefits added, Resource Reservation Service will:
+
+* improve visibility of cloud resources consumption (current and planned for
+  future);
+* enable cloud resource planning based on current and future demand from end
+  users;
+* automate the processes of resource allocation and reclaiming;
+* provide energy efficiency for physical hosts (both compute and storage ones);
+* potentially provide leases as billable items for which customers can be
+  charged a flat fee or a premium price depending on amount of reserved cloud
+  resources and their usage.
+
+Glossary of terms
+-----------------
+
+**Reservation** is an allocation of certain cloud resource (Nova instance, Cinder
+volume, compute host, etc.) to particular project. Speaking about virtual
+reservations, we may have not only simple, solid ones (like already mentioned
+instances and volumes), but also complex ones - like Heat stacks and Savanna
+clusters. Reservation is characterized by status, resource type and identifier
+and lease it belongs to.
+
+**Lease** is a negotiation agreement between the provider (Climate, using OpenStack
+resources) and the consumer (user) where the former agrees to make some kind of
+resources (both virtual and physical) available to latter, based on a set of
+lease terms presented by the consumer. Here lease may be described as contract
+between user and reservation service about cloud resources to be provided right
+now or later. Technically speaking, lease is a group of reservations granted to
+particular project upon request. Lease is characterized by start time, end
+time, set of individual reservations and associated events.
+
+**Event** is simply something that may happen to lease. In most simple case event
+might describe lease start and lease end. Also it might be notification to user
+(e.g. about soon lease expiration) and some extra actions.
+
+Rationale
+---------
+
+Climate is created to:
+
+* manage cloud resources not only right now, but also in the future;
+* have dedicated recourses on certain amount of time;
+* prepare for the peak loads and perform capacity planning;
+* optimise energy consumption.
+
+Lease types (concepts)
+----------------------
+
+* **Immediate reservation**. Resources are provisioned immediately (like VM
+  boot or moving host to reserved user aggregate) or not at all. If request can
+  be fulfilled, lease is created and **success** status is returned. Lease
+  should be marked as **active** or **to_be_started**. Otherwise (if
+  request resource cannot be provisioned right now) failure status for this
+  request should be returned.
+* **Reservation with retries**. Mostly looks like previous variant, but in case
+  of failure, user may want to have several (configurable number) retries to
+  process lease creation action. In this case request will be processed till
+  that will be possible to create lease but not more than set in configuration
+  file number of times.
+* **Best-effort reservation**. Also might have place if lease creation request
+  cannot be fulfilled immediately. Best-effort mechanism starts something like
+  scavenger hunt trying to find resources for reservations. For compute hosts
+  reservation that makes much sense, because in case there are instances
+  belonging to other tenant on eligible hosts, and without them there will be
+  possible to reserve these hosts, Climate may start instances migration.
+  This operation can be timely and fairly complex and so different strategies
+  may be applied depending on heuristic factors such as the number, type and
+  state of the instances to be migrated. Also Climate should assert that there
+  are at least enough potential candidates for the migration prior to starting
+  the actual migration. If Climate decides to start migration, it returns
+  **success** state and marks lease as **in_progress**, otherwise -
+  **failure**. If this 'hunting' ends successfully before configurable
+  timeout has passed, lease should be marked as **active**, otherwise its
+  status is set to **timedout**.
+* **Delayed resource acquiring** or **scheduled reservation**. In this
+  reservation type lease is created successfully if Climate thinks there will
+  be enough resources to process provisioning later (otherwise this request
+  returns **failure** status). Lease is marked as **inactive** till all
+  resources will be actually provisioned. That works pretty nice and
+  predictable speaking about compute hosts reservation (because hosts as
+  resources are got not from common cloud pool, but from admin defined pool).
+  So Climate is possible to predict these physical resources usage and use that
+  information during lease creation. If we speak about virtual reservations,
+  here situation is more complicated, because all resources are got from common
+  cloud resources pool, and Climate cannot guarantee there will be enough
+  resources to provision them. In this failure case lease state will be marked
+  as **error** with appropriate explanation.

doc/source/restapi/index.rst

@@ -0,0 +1,9 @@
+Climate REST API docs
+*********************
+
+This page includes documentation for Climate APIs.
+
+.. toctree::
+   :maxdepth: 1
+
+   rest_api_v1.0