diff --git a/doc/source/images/hadoop-cluster-example.jpg b/doc/source/images/hadoop-cluster-example.jpg new file mode 100644 index 00000000..70f9f675 Binary files /dev/null and b/doc/source/images/hadoop-cluster-example.jpg differ diff --git a/doc/source/index.rst b/doc/source/index.rst index 4914b0e4..eca0a0c2 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -20,9 +20,10 @@ User guide userdoc/installation.guide horizon/installation.guide - userdoc/rest_api_v1.0 + userdoc/overview userdoc/features userdoc/hadoop-swift.rst + userdoc/rest_api_v1.0 Developer Guide diff --git a/doc/source/userdoc/overview.rst b/doc/source/userdoc/overview.rst new file mode 100644 index 00000000..0ada9578 --- /dev/null +++ b/doc/source/userdoc/overview.rst @@ -0,0 +1,64 @@ +General Overview +================ + +Clusters +-------- + +A cluster deployed by Savanna consists of node groups. Node groups vary by +their role, parameters and number of machines. The picture below +illustrates example of Hadoop cluster consisting of 3 node groups each having +different role (set of processes). + +.. image:: ../images/hadoop-cluster-example.jpg + +Node group parameters include Hadoop parameters like `io.sort.mb` or +`mapred.child.java.opts`, and several infrastructure parameters like flavor +for VMs or storage location (ephemeral drive or Cinder volume). + +A cluster is characterized by its node groups and its parameters. Like a node +group, cluster has Hadoop and infrastructure parameters. An +example of cluster-wide Hadoop parameter is `dfs.replication`. For +infrastructure an example could be image which will be used to launch cluster +VMs. + +Templates +--------- + +In order to simplify cluster provisioning Savanna employs concept of templates. +There are two kind of templates: node group template and cluster template. The +former is used to create node groups, the later - clusters. Essentially +templates have the very same parameters as corresponding entities. Their aim +is to remove burden of specifying all the required parameters each time user +wants to launch a cluster. + +Provisioning Plugins +-------------------- + +A provisioning plugin is a component responsible for provisioning Hadoop +cluster. Generally each plugin is capable of provisioning a specific Hadoop +distribution. Also plugin can install management and/or monitoring tools for +a cluster. + +Since Hadoop parameters vary depending on distribution and Hadoop version, +templates are always plugin and Hadoop version specific. A template could not +be used with plugun/Hadoop version different than ones it was created for. + +Image Registry +-------------- + +OpenStack starts VMs based on pre-built image with installed OS. The image +requirements for Savanna depend on plugin and Hadoop version. Some plugins +require just basic cloud image and install Hadoop on VMs from scratch. Some +plugins might require images with pre-installed Hadoop. + +The Savanna Image Registry is a feature which helps filter out images during +cluster creation. Before creating a cluster, user should register an image in +Image Registry and tag it with specific tags. Each plugin has its own set of +tags which help it to determine if an image is suitable for a specific +Hadoop version. + +Features +-------- + +Savanna has several interesting features. The full list could be found there: +:doc:`features`