Commit Graph

88 Commits (75e2ff46ea41f194fc459ed312fe45e88d956c5c)

Author SHA1 Message Date
Luigi Toscano 542e7e78ba Post-import fixes: name, license, doc, translations
- fix the name of the module in setup.cfg, according the correct
  pattern (otherwise the name of the directory is not correct);
- add LICENSE file, required by (at least some) distributions;
- setup the translations;
- fix the name of the manpage, use the name of the module;
- add the classifier for Python 3.6.

Change-Id: I37d205c2d900600674d3862eba61c3c017e5d538
4 years ago
Telles Nobrega 40fd2aa34e Plugins splitted from sahara core
Change-Id: I43e0beec6508f93a436a150749bfa23571986b9d
4 years ago
Luigi Toscano f6218ef321 Update devel info: mailing list, meeting time
- The new openstack-discuss mailing list is going to replace few
  mailing lists, including openstack-dev.
- There is only one meeting time now (1400UTC).

Change-Id: I14c5d37ae7f59095d00a7b26fa0a0fc23b01a33a
5 years ago
akhiljain23 fd0d8e9b69 Add framework for sahara-status upgrade check
This commit adds the functionality of sahara-status CLI for performing
upgrade checks as part of the Stein cycle upgrade-checkers goal.
It only includes a sample check which must be replaced by real checks in
future.

Change-Id: Idcb8d9eaf689800812cf6087e9c5937058c89ea6
Story: 2003657
Task: 26152
5 years ago
Jeremy Freudberg 819d9b5acf S3 data source
* Create S3 data source type for EDP
* Support storing S3 secret key in Castellan
* Unit tests for new data source type
* Document new data source type and related ideas
* Add support of S3 configs into Spark and Oozie workflows
* Hide S3 credentials in job execution info, like for Swift
* Release note

Change-Id: I3ae5b9879b54f81d34bc7cd6a6f754347ce82f33
5 years ago
Nguyen Hai c0f8f4a693 Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Remove the '[build_sphinx]' section as described in:
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128594.html

Change-Id: Ia8e8d6b4e23c2737a9e948e7c4b682dca697f6a4
5 years ago
Jeremy Freudberg f5435f8914 S3 job binary and binary retriever
* Create common module for managing S3 job binaries
* Add new dependency on botocore
* Use common S3 library to create S3 job binary type for EDP
* Use common S3 library to create S3 job binary retriever
* Support storing S3 secret key in Castellan
* Document new job binary type (and foreshadow the S3 data source type)
* Unit tests for new code

Change-Id: I6781203d802305446ba1418ed6999186db4dfe9b
Partially-Implements: bp sahara-support-s3
5 years ago
Telles Nobrega d5959b0760 Policy in code for Sahara
This patch is part of the community goal to move policies into code.

https://governance.openstack.org/tc/goals/queens/policy-in-code.html

Change-Id: I8f228ca40acc589b9aa41fc1e80a1c0308f179cf
6 years ago
Jesse Pretorius 8b21d1fc76 Add default configuration files to data_files
In order to make it simpler to use the default
configuration files when deploying services
from source, the files are added to pbr's
data_files section so that the files are
included in the built wheels and therefore
deployed with the code. Packaging and deployment
tools can then more easily use the default files
if they wish to.

This pattern is already established with similar
files for neutron, designate and glance as has
been mentioned in the related bug report.

Change-Id: Iae987bdae55b75143f192f9a642cf1ff564c62b6
Closes-Bug: #1718356
6 years ago
chenxing 216507f29b Update Documention link
Change-Id: I39f0bc1c7b5ef8114f6391058ab4378638b50907
6 years ago
pawnesh.kumar 37694d8377 Enable warnings as errors for doc building
Treat warnings as errors for doc building - adding Sphinx'
warning-is-errors variable.

Change-Id: I12f9c92e8b8f9245614797e65af1ae48a0714923
6 years ago
Marianne Linhares Monteiro 4827ec2430 Support Job binary pluggability
Changes to support the job binary abstraction.

Change-Id: If5133f2d35974f0b4ca1776bc9fe39f011575389
Implements: blueprint data-source-plugin
6 years ago
Jenkins 4a0ba2a117 Merge "Support Data Source pluggability" 6 years ago
Marianne Linhares Monteiro 50f9d2bdc3 Support Data Source pluggability
Changes to support the data source abstraction.

Change-Id: I62274c08619c8e720231eb6aa0563ac6e12497f1
Implements: blueprint data-source-plugin
6 years ago
Cao Xuan Hoang fffb54bff2 Remove support for py34
The gating on python 3.4 is restricted to <= Mitaka. This is due to
the change from Ubuntu Trusty to Xenial, where only python3.5 is
available. There is no need to continue to keep these settings.

Change-Id: Ifbe877705dbd87dea34d2b1974d2ae3ed7cac9a3
6 years ago
Shu Yingya fc9f11e176 Remove entry point of sahara tempest plugin
In patch [1], tempest-like test of sahara client has been removed.
But in setup.cfg, there still is entry point of it. Remove it!

[1]:https://review.openstack.org/#/c/367980/

Change-Id: I802e27b91486baa43286bc949e304448ad1610f1
7 years ago
Jenkins dab2a46420 Merge "Add Python 3.5 classifier and venv" 7 years ago
PanFengyun c30e03dca0 Add Python 3.5 classifier and venv
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: I15a761c0b6d5769b2dba9f521ef96cf43c99e552
7 years ago
Ethan Gafford 49e179260c CLI for Plugin-Declared Image Declaration
Introduces a new libfuestfs-driven CLI for packing of
Sahara images, using the same recipe definition scheme
introduced in the validate-image-spi blueprint.

Documentation and plugin image packing configurability
can and will be provided in separate patches for ease
of review.

Partially-implements: blueprint image-generation-cli
Change-Id: I6788108e3fb6232045fc56937639a6348768a7bc
7 years ago
Vitaly Gridnev 907e3236a7 Remove hdp 2.0.6 plugin
Newton release is opened, so we can remove this
plugin from sahara codebase

Implements blueprint: remove-hdp206
Change-Id: I1ebf90e716964151c88de53d127789251bef066b
7 years ago
Doug Hellmann 32f66271cf register the config generator default hook with the right name
No config generator hooks should ever be registered with a name that
belongs to another project. In this case, using oslo.middleware.cors
means that *every other project* that loads the middleware gets this
application's defaults when the generator is run on a system with
everything installed (such as a dev box with devstack). Use the name
of the app instead, to ensure that the defaults are only set when this
app's sample config and documentation are being generated.

Change-Id: I6a8c7d44b9db9325003ff2fdb667b0ced7739e96
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
7 years ago
Michael Krotscheck d2bf895258 Moved CORS middleware configuration into oslo-config-generator
The default values needed for sahara's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to the default initialization procedure. This ensures
that if a value remains unset in the configuration file, it will
fallback to using sane defaults. It also ensures that an operator
modifying the configuration will be presented with that same
set of defaults.

Change-Id: Iedcaa002ff3d40cf61168769bc3946f8c6e42b87
Closes-Bug: 1551836
7 years ago
Sergey Reshetnyak bfb557117d Add support running Sahara as wsgi app
This patch adds support of running sahara-api as wsgi application.
Tested with apache + mod_wsgi web server.

Change-Id: I7355b09ebedd44e51242a252e96ac3a51073b2e6
7 years ago
Sergey Lukjanov 7844d7dda2 Remove version from setup.cfg
Per [1] we now want to use the git repository for knowing the version instead
of just trying to modify the setup.cfg file.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-November/080692.html

Change-Id: I790506ea15e67f0cdeaed62b5b87a7d51c589e7b
Depends-On: I001a18895489a56eb55a5eb8c73ab6a3d4b1d4d0
8 years ago
Vitaly Gridnev cebf807ab1 Drop direct engine support
This change remove support of direct engine in Sahara.
Unit tests related to direct engine were removed.

Implements blueprint: remove-direct-engine

Change-Id: Iba292996dd052a069a98f1024441255a15c0f734
8 years ago
Thierry Carrez b813c022e3 Open Mitaka development
Bump preversion to mark the start of the Mitaka development branch.
The liberty release branch will be cut from the previous commit.

Change-Id: I2e08186efb1d4f0d280bce11f2c3ac710bfa0a5d
8 years ago
Luigi Toscano dcc878b56c Adapt python client tests to use Tempest plugin interface
Use the Tempest plugin interface for python client tests (which
are already tempest-based) instead of copying the tests inside
the tempest tree.

Inspired by the same type of change applied to Manila,
see Ie5ed64a6777ed1acf8dd56522c26705ae897596d

Depends-On: I06f1e13207cc6d661c078c4c4cf1ae7974ecf3da
Change-Id: I6073a528413aefd96882179a1eb8bbe715c6417b
8 years ago
Sergey Reshetnyak a151a8027d New version of HDP plugin
Current features:
* Set up Ambari server and Ambari agents from pre-built image
* Support Ambari 2.0 / 2.1
* Password generation for management console

partially implements bp: hdp-22-support

Change-Id: I53084a83cccde52654a42911f449cc8b7d769bea
8 years ago
Sergey Vilgelm 23d5d138a1 Remove openstack.common package
Clean up the unused modules from the openstack.common package

Closes-Bug: #1476407
Change-Id: I5697f761c310dd8e13b7219b2968c3e6f4cfb299
8 years ago
Doug Hellmann dd3f439561 Update version for Liberty
Update the version for Liberty, switching from date-based versioning
to pre-versioning using SemVer. See
http://lists.openstack.org/pipermail/openstack-dev/2015-May/065211.html
and
http://lists.openstack.org/pipermail/openstack-dev/2015-June/067082.html
for details.

Change-Id: I6a35fa0dda798fad93b804d00a46af80f08d475c
8 years ago
Jenkins 7de90913b0 Merge "Open Liberty development" 8 years ago
Thierry Carrez 733b80d066 Open Liberty development
Bump pre-version in setup.cfg to formally open Liberty development.
Kilo release branch will be cut from the previous commit.

Change-Id: I6fa1e9e45fbaae62f7feb64039565266cbc86788
8 years ago
Sergey Lukjanov cd8aec02b6 Migrate to oslo.policy lib instead of copy-pasted oslo-incubator
The first official oslo.policy was released and so we're ready to
migrate from the copy-pasted oslo-incubator code to the separated
lib.

Change-Id: I8e404d2b622876389d9b1019ae50f1e1a8cecdf1
8 years ago
Trevor McKay 34fba54e26 Add a CLI tool for managing default templates
This change adds a CLI called sahara-templates to manage default
templates.

Partial-implements: blueprint default-templates
Change-Id: I4f30bd6bc378d90fda41b512c04afe8023d2b4b2
8 years ago
Andrew Lazarev c62ce088b5 Reorganized heat template generation code
* Extracted heat template generation from
  'utils/openstack/heat.py' to 'service/heat/templates.py'.
  Now heat.py is for heat client only.
* Moved heat template resources from 'resources' to
  'service/heat/resources'
* Separated tests for heat templates and heat client.

Change-Id: I4c7a561f1648f34e71574a556cbc07ed2cf9b173
Closes-Bug: #1373075
8 years ago
Jenkins 5a6e4aa213 Merge "Remove log module from common modules" 8 years ago
Vitaly Gridnev d488295b05 Remove log module from common modules
Log module was removed from oslo-incubator,
so let's sync with latest oslo-incubator and remove
log module.

Change-Id: Id27ade473ed8597f2741ed0c746fa6b7ef6c7f06
Related-bug: #1412673
8 years ago
ChangBo Guo(gcb) b3ad434a9a Drop cli/sahara-rootwrap
It can be automatically generated from console_scripts entry in
setup.cfg. So we don't need maintain it for sahara.

Change-Id: Ibe2a170cc60eb69b44aec43fb61f367e53c47b9f
8 years ago
Andrew Lazarev 50603d13a5 Added ability to listen HTTPS port
* Added oslo-inclubator 'sslutils' module dependency
* Added config params from sslutils
* Added ability to make SSL port for sahara-api

Implements blueprint: sahara-support-https

Change-Id: I184e39b6a0eb6ad17b51aa408a65a455983bfb0c
9 years ago
Jenkins 200cc760d9 Merge "Cleaned up config generator settings" 9 years ago
Michael McCune 63dbea5569 Adding Storm entry point to setup.cfg
This change allows the stevedore package to find the Storm plugin when
enabling extensions.

Change-Id: I1748609caff74e073171d5e4a1b76fda9efef1d6
Closes-Bug: 1407120
9 years ago
Andrew Lazarev 0c5ed6a947 Cleaned up config generator settings
1. Sahara opts are listed once
2. Oslo incubator opts are listed separately
3. Groups are referenced by variables

Still things to improve:
1. opt-group mapping is listed twice now (in reg and in list)
2. registering spread across the project (should be in one place)
3. dependancies are not managed (it should be one place with CONF)

Change-Id: I4b8edf26187b3c23d08f9efbee42086652efea5a
Closes-Bug: #1407834
9 years ago
Vitaly Gridnev cbdf52fa2b Drop obsolete oslo-confing-generator
Change-Id: Ie25d7081b57dd4242d39c52a22ba65955d210691
Closes-Bug:#1380725
9 years ago
Artem Osadchiy 26e3244460 MapR plugin implementation
Change-Id: Id08a3c7a8a40fd826cd902cedbc2590ca284e548
Implements: blueprint mapr-plugin
9 years ago
Michael McCune 8217142785 Adding support for oslo.rootwrap to namespace access
Changes
* adding configuration options for rootwrap
* refactoring ssh connection to use rootwrap as a proxy when requested
* adding documentation for rootwrap configuration
* adding default rootwrap filters file
* adding default sudoers conf file for sahara user
* adding default rootwrap conf file for sahara-rootwrap
* adding sahara-rootwrap cli script
* adding requirement for oslo.rootwrap

Change-Id: I7871400b2342a4cd1a8910ae5121b1bfdc46078d
Closes-Bug: #1271349
9 years ago
Thierry Carrez c8a1173dcc Open Kilo development
Bump pre-version to 2015.1 to formally open master branch to Kilo
development.

Change-Id: Ibd929d8f826bab9b87e44b8de881f58768874294
9 years ago
Sergey Reshetnyak 6f9852cd1b Add CDH plugin to Sahara
Features:
* cluster provisioning
* scaling
* edp
* validation
* swift support

TODO (in other CRs):
* unit tests
* integration tests
* data locality

partially implement: blueprint cdh-plugin

Change-Id: Ie231c434d61ba9a379a6ee2fd0f0bf2af21ce44d
9 years ago
Daniele Venzano 8bb52c4067 Add Spark plugin to Sahara
Integrate the Spark plugin in Sahara. It adds the capability of
provisioning Spark clusters with (Cloudera) HDFS. The plugin
assumes the use of VM images generated with the Savanna
diskimage-builder and the '-p spark' option.

Implements: blueprint spark-plugin

Wiki: https://wiki.openstack.org/wiki/Sahara/SparkPlugin

This code is running on our Bigfoot research cluster.

Change-Id: Ic105f7de64248bdfb05879ededf35503bc04925b
9 years ago
Jenkins 719ed3e4a1 Merge "Split sahara into sahara-api and sahara-engine" 9 years ago
Jenkins 4ed8c6c993 Merge "Add sahara-all binary" 9 years ago