openstack/deb-sahara
Code Issues Proposed changes
7ca1794e87
deb-sahara/doc/source/userdoc/configs_recommendations.rst
Alok Jani 7ba05c5e30 Fix doc8 check failures 
Resolves D000 and D001 checks in sahara docs.

Change-Id: If6d577859170ae55d578d70885a9ed5a6b0db651
Closes-Bug: #1514887
2015-11-13 15:04:39 +00:00

1.5 KiB
Raw Blame History

Autoconfiguring templates

During the Liberty development cycle sahara implemented a tool that recommends and applies configuration values for cluster templates and node group templates. These recommendations are based on the number of specific instances and on flavors of the cluster node groups. Currently the following plugins support this feature:

  • Cloudera;
  • Spark;
  • the Vanilla Apache Hadoop plugin.

By default this feature is enabled for all cluster templates and node group templates. If you want to disable this feature for a particular cluster or node group template you should set the use_autoconfig field to false.

The following describes the settings for which sahara can recommend autoconfiguration:

The Cloudera, Spark and Vanilla Apache Hadoop plugin support configuring dfs.replication (dfs_replication for Cloudera plugin) which is calculated as a minimun from the amount of datanode (HDFS_DATANODE for Cloudera plugin) instances in the cluster and the default value for dfs.replication.

The Vanilla Apache Hadoop plugin and Cloudera plugin support autoconfiguration of basic YARN and MapReduce configs. These autoconfigurations are based on the following documentation: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.1/bk_installing_manually_book/content/rpm-chap1-11.html