openstack/sahara
Code Issues Proposed changes

Merge "[IDH] Initial documentation for IDH plugin"

Browse Source
This commit is contained in:
Jenkins 2014-03-21 15:54:43 +00:00 committed by Gerrit Code Review
commit 48ba384974
2 changed files with 54 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/source/index.rst
View File

@ -45,6 +45,7 @@ User guide
userdoc/plugins
userdoc/vanilla_plugin
userdoc/hdp_plugin
userdoc/idh_plugin
**Elastic Data Processing**

53
doc/source/userdoc/idh_plugin.rst Normal file
View File

@ -0,0 +1,53 @@
Intel Distribution for Apache Hadoop Plugin
===========================================
The Intel Distribution for Apache Hadoop (IDH) Sahara plugin provides a way
to provision IDH clusters on OpenStack using templates in a single click and
in an easily repeatable fashion. The Sahara controller serves as the glue
between Hadoop and OpenStack. The IDH plugin mediates between the Sahara
controller and Intel Manager in order to deploy and configure Hadoop on
OpenStack. Intel Manager is used as the orchestrator for deploying the IDH
stack on OpenStack.
For cluster provisioning images supporting cloud init should be used. The only
supported operation system for now is Cent OS 6.4. Here you can find the image:
* http://sahara-files.mirantis.com/CentOS-6.4-cloud-init.qcow2
IDH plugin requires an image to be tagged in Sahara Image Registry with
two tags: 'idh' and '<IDH version>' (e.g. '2.5.1').
Also you should specify a default username of "cloud-user" to be used in the
Image.
Limitations
-----------
The IDH plugin currently has the following limitations:
* IDH plugin downloads the Intel Manager package from a URL provided in the
cluster configuration. A local HTTP mirror should be used in cases where the
VMs do not have access to the Internet or have port limitations.
* IDH plugin adds the Intel rpm repository to the yum configuration. The
repository URL can be chosen during Sahara cluster configuration. A local
mirror should be used in cases where the VMs have no access to the Internet
or have port limitations. Refer to the IDH documentation for instructions on
how to create a local mirror.
* Hadoop cluster scaling is supported only for datanode and tasktracker
(nodemanager for IDH 3.x) processes.
Cluster Validation
------------------
When a user creates or scales a Hadoop cluster using the IDH plugin, the
cluster topology requested by the user is verified for consistency.
Currently there are the following limitations in cluster topology for IDH plugin:
* Cluster should contain
* exactly one manager
* exactly one namenode
* at most one jobtracker for IDH 2.x or resourcemanager for IDH 3.x
* at most one oozie
* Cluster cannot be created if it contains worker processes without containing
corresponding master processes. E.g. it cannot contain tasktracker if there
is no jobtracker.