openstack
/
sahara-plugin-spark
Code Issues Proposed changes
sahara-plugin-spark/sahara_plugin_spark/plugins/spark/resources
History
Telles Nobrega 3a1da9f208 Adding Spark to sahara-image-pack 
Adding the ability to create spark images using the new image
generation.

Change-Id: I220a05d782f749d3799fc37aa7d787d2fd8993b4
 		2019-02-15 16:53:53 -02:00
..
images Adding Spark to sahara-image-pack 2019-02-15 16:53:53 -02:00
README.rst Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00
core-default.xml Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00
hdfs-default.xml Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00
spark-cleanup.cron Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00
spark-env.sh.template Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00
tmp-cleanup.sh.template Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00
topology.sh Plugins splitted from sahara core 2019-01-03 14:54:33 -03:00

README.rst

Apache Spark and HDFS Configurations for Sahara

This directory contains default XML configuration files and Spark scripts:

  • core-default.xml,
  • hdfs-default.xml,
  • spark-env.sh.template,
  • topology.sh

These files are used by Sahara's plugin for Apache Spark and Cloudera HDFS. XML config files were taken from here: * https://github.com/apache/hadoop-common/blob/release-1.2.1/src/core/core-default.xml * https://github.com/apache/hadoop-common/blob/release-1.2.1/src/hdfs/hdfs-default.xml

Cloudera packages use the same configuration files as standard Apache Hadoop.

XML configs are used to expose default Hadoop configurations to the users through Sahara's REST API. It allows users to override some config values which will be pushed to the provisioned VMs running Hadoop services as part of appropriate xml config.