The external link helper makes sure that the generated link respects the current branch (latest for master, <release_name> when built on stable/<release_name>. Do not use it for sahara-tests (which is branchless). Change-Id: Ie37fe37858a41f54558642ea93a75365668ef573
8.3 KiB
Sahara Installation Guide
We recommend installing sahara in a way that will keep your system in a consistent state. We suggest the following options:
To install with Fuel
- Start by following the MOS Quickstart to install and setup OpenStack.
- Enable the sahara service during installation.
To install with Kolla
- Start by following the
Kolla Quickstart <user/quickstart.html>
to install and setup OpenStack. - Enable the sahara service during installation.
To install with RDO
- Start by following the RDO Quickstart to install and setup OpenStack.
- Install sahara:
# yum install openstack-sahara
- Configure sahara as needed. The configuration file is located in
/etc/sahara/sahara.conf
. For details seeSahara Configuration Guide <../admin/configuration-guide>
- Create the database schema:
# sahara-db-manage --config-file /etc/sahara/sahara.conf upgrade head
- Go through
common_installation_steps
and make any necessary changes. - Start the sahara-api and sahara-engine services:
# systemctl start openstack-sahara-api
# systemctl start openstack-sahara-engine
- (Optional) Enable sahara services to start on boot
# systemctl enable openstack-sahara-api
# systemctl enable openstack-sahara-engine
To install into a virtual environment
- First you need to install a number of packages with your OS package manager. The list of packages depends on the OS you use. For Ubuntu run:
$ sudo apt-get install python-setuptools python-virtualenv python-dev
For Fedora:
$ sudo yum install gcc python-setuptools python-virtualenv python-devel
For CentOS:
$ sudo yum install gcc python-setuptools python-devel
$ sudo easy_install pip
$ sudo pip install virtualenv
- Setup a virtual environment for sahara:
$ virtualenv sahara-venv
This will install a python virtual environment into
sahara-venv
directory in your current working directory.
This command does not require super user privileges and can be executed
in any directory where the current user has write permissions.
- You can get a sahara archive from http://tarballs.openstack.org/sahara/ and install it using pip:
$ sahara-venv/bin/pip install 'http://tarballs.openstack.org/sahara/sahara-master.tar.gz'
Note that sahara-master.tar.gz
contains the latest
changes and might not be stable at the moment. We recommend browsing http://tarballs.openstack.org/sahara/
and selecting the latest stable release. For installation just execute
(where replace the 'release' word with release name, e.g. 'mitaka'):
$ sahara-venv/bin/pip install 'http://tarballs.openstack.org/sahara/sahara-stable-release.tar.gz'
For example, you can get Sahara Mitaka release by executing:
$ sahara-venv/bin/pip install 'http://tarballs.openstack.org/sahara/sahara-stable-mitaka.tar.gz'
- After installation you should create a configuration file from the
sample file located in
sahara-venv/share/sahara/sahara.conf.sample-basic
:
$ mkdir sahara-venv/etc
$ cp sahara-venv/share/sahara/sahara.conf.sample-basic sahara-venv/etc/sahara.conf
Make any necessary changes to
sahara-venv/etc/sahara.conf
. For details see Sahara Configuration Guide <../admin/configuration-guide>
Common installation steps
The steps below are common to both the RDO and virtual environment installations of sahara.
- If you use sahara with a MySQL database, then for storing big job
binaries in the sahara internal database you must configure the size of
the maximum allowed packet. Edit the
my.cnf
file and change themax_allowed_packet
parameter as follows:
...
[mysqld]
...
max_allowed_packet = 256M
Then restart the mysql server to ensure these changes are active.
- Create the database schema:
$ sahara-venv/bin/sahara-db-manage --config-file sahara-venv/etc/sahara.conf upgrade head
- Start sahara services from different terminals:
# first terminal
$ sahara-venv/bin/sahara-api --config-file sahara-venv/etc/sahara.conf
# second terminal
$ sahara-venv/bin/sahara-engine --config-file sahara-venv/etc/sahara.conf
- For sahara to be accessible in the OpenStack Dashboard and for python-saharaclient to work properly you must register sahara in the Identity service catalog. For example:
$ openstack service create --name sahara --description \
"Sahara Data Processing" data-processing
$ openstack endpoint create --region RegionOne \
data-processing public http://10.0.0.2:8386/v1.1/%\(project_id\)s
$ openstack endpoint create --region RegionOne \
data-processing internal http://10.0.0.2:8386/v1.1/%\(project_id\)s
$ openstack endpoint create --region RegionOne \
data-processing admin http://10.0.0.2:8386/v1.1/%\(project_id\)s
Note
You have to install the openstack-client package in order to execute
openstack
command.
- For more information on configuring sahara with the OpenStack
Dashboard please see
dashboard-guide
.
Optional installation of default templates
Sahara bundles default templates that define simple clusters for the supported plugins. These templates may optionally be added to the sahara database using a simple CLI included with sahara.
The default template CLI is described in detail in a README
file included with the sahara sources at
<sahara_home>/db/templates/README.rst
but it is
summarized here.
Flavor id values must be specified for the default templates included
with sahara. The recommended configuration values below correspond to
the m1.medium and m1.large flavors in a default
OpenStack installation (if these flavors have been edited, their
corresponding values will be different). Values for flavor_id should be
added to /etc/sahara/sahara.conf
or another configuration
file in the sections shown here:
[DEFAULT]
# Use m1.medium for {flavor_id} unless specified in another section
flavor_id = 2
[cdh-5-default-namenode]
# Use m1.large for {flavor_id} in the cdh-5-default-namenode template
flavor_id = 4
[cdh-530-default-namenode]
# Use m1.large for {flavor_id} in the cdh-530-default-namenode template
flavor_id = 4
The above configuration values are included in a sample configuration
file at
<sahara_home>/plugins/default_templates/template.conf
The command to install all of the default templates is as follows,
where $PROJECT_ID
should be a valid project id and the
above configuration values have been set in myconfig
:
$ sahara-templates --config-file /etc/sahara/sahara.conf --config-file myconfig update -t $PROJECT_ID
Help is available from the sahara-templates
command:
$ sahara-templates --help
$ sahara-templates update --help
Notes:
Ensure that your operating system is not blocking the sahara port (default: 8386). You may need to configure iptables in CentOS and other Linux distributions to allow this access.
To get the list of all possible options run:
$ sahara-venv/bin/python sahara-venv/bin/sahara-api --help
$ sahara-venv/bin/python sahara-venv/bin/sahara-engine --help
Further, consider reading ../intro/overview
for general sahara concepts and
../user/plugins
for
specific plugin features/requirements.