S3 data source

* Create S3 data source type for EDP
* Support storing S3 secret key in Castellan
* Unit tests for new data source type
* Document new data source type and related ideas
* Add support of S3 configs into Spark and Oozie workflows
* Hide S3 credentials in job execution info, like for Swift
* Release note

Change-Id: I3ae5b9879b54f81d34bc7cd6a6f754347ce82f33
This commit is contained in:
Jeremy Freudberg 2018-06-28 15:20:29 -04:00
parent b6a2addb96
commit 819d9b5acf

View File

@ -59,6 +59,7 @@ sahara.data_source.types =
manila = sahara.service.edp.data_sources.manila.implementation:ManilaType
maprfs = sahara.service.edp.data_sources.maprfs.implementation:MapRFSType
swift = sahara.service.edp.data_sources.swift.implementation:SwiftType
s3 = sahara.service.edp.data_sources.s3.implementation:S3Type
sahara.job_binary.types =
internal-db = sahara.service.edp.job_binaries.internal_db.implementation:InternalDBType