d4b2f376e4
This class takes the name of an XML file to add to the default Hadoop configuration for a Spark job before invoking the job's main class. This class is based on the MainWrapper used for Oozie Java actions but is simpler. There is no need to modify the SecurityManager, and the name of the file may be anything. This class is broken out separately so that the wrappers can vary independently in separate JARs. Change-Id: I5f34fcf525f4f16b095d52fab5059b9be59896f7 Partial-Implements: blueprint edp-spark-swift-integration |
||
---|---|---|
.. | ||
src/main/java/org/openstack/sahara/edp | ||
README.rst | ||
pom.xml |
README.rst
Sources for main function wrapper for Spark
The Hadoop configuration for a Spark job must be modified if the Spark job is going to access Swift paths. Specifically, the Hadoop configuration must contain values that allow the job to authenticate to the Swift service.
This wrapper adds a specified xml file to the default Hadoop Configuration resource list and then calls the specified main class. Any necessary Hadoop configuration values can be added to the xml file. This allows the main class to be run and access Swift paths without alteration.