openstack/storlets
Code Issues Proposed changes

Fixing doc to reflect changes in s2aio

Browse Source 
Change-Id: If8b60ac8c535cc23b74b866a9e00ab16c7d3f4a0
This commit is contained in:
Eran Rom
2015-10-28 15:01:01 +02:00
parent 4627be2263
commit 76a7d56784
26 changed files with 134 additions and 698 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
README.md
View File

@@ -2,6 +2,21 @@
Swift Storlets
==============
tl ; dr
=======
* Get a fresh 14.04 Ubuntu (A disposal VM is a good start)
* Create a sudoer that can sudo without a password
::
sudo apt-get install python-tox
sudo apt-get install python-nose
git clone https://github.com/openstack/storlets.git
cd storlets
./s2aio.sh
Browse through the StorletSamples directory so see storlet code examples,
and through the SystemTests to see samples of deploying and invoking a storlet
Introduction
============
Swift Storlets extend Swift with the capability to run computation near the data in a secure and isolated manner. With Swift Storlets a user can write code,
@@ -47,28 +62,15 @@ The documentation in this repo is organized according to the various roles invol
* <https://github.com/openstack/storlets/blob/master/doc/source/dev_and_test_guide.rst>
* <https://github.com/openstack/storlets/blob/master/doc/source/storlets_docker_gateway.rst>
Finally, these are a MUST:
* <https://github.com/openstack/storlets/blob/master/doc/source/storlets_installation_guide.rst>
* <https://github.com/openstack/storlets/blob/master/doc/source/storlet_all_in_one.rst>
Installation
============
<https://github.com/openstack/storlets/blob/master/doc/source/storlets_installation_guide.rst> describes how to install Storlets in an existing Swift cluster that uses Keystone.
For convenience we also provide a storlet all-in-one installation script that installs Swift with Keystone and Storlets in a single virtual machine.
See <https://github.com/openstack/storlets/blob/master/doc/source/storlet_all_in_one.rst>
The installation is based on Ansible and was tested on Ubuntu 14.10, and with Swift 1.13 and Swift 2.2.
<https://github.com/openstack/storlets/blob/master/doc/source/installation.rst> describes how to install Storlets in an existing Swift cluster that uses Keystone.
The installation is based on Ansible and was tested on Ubuntu 14.04, 14.10, and with Swift 1.13 and Swift 2.2.
Once installation is completed, you can try run the system tests as described in the <https://github.com/openstack/storlets/blob/master/doc/source/dev_and_test_guide.rst>
The system tests are a good reference for writing and deploying a Storlet.
Status
======
The purpose of this repository is to serve as a mostly read only reference for (1) the Swift storlets middleware, and (2) a storlets gateway
implementaton.
Having said that we will be doing fixing of major bugs, potentially add some improvements and adaptations required to stay tuned with
the Swift Storlets middleware as it evolves while getting upstream.
Given enough interest from the community this status may change to be a more active project.
Acknowledgements
================

125
doc/source/installation.rst
View File

@@ -1,12 +1,121 @@
============
Installation
============
=====================
Storlets Installation
=====================
At the command line::
Background
==========
$ pip install storlets
Storlets installation (and build) assume an existing Swift cluster that works with Keystone.
The installation consists of the following components:
Or, if you have virtualenvwrapper installed::
#. A set of scripts to assist the Storlets admin. This admin represents the provider, and is responsible for the Storlet enabled accounts and their image management.
Those scripts make use of a private Docker registry as well as a designated account and containers used to keep Storlet management related state.
#. A Docker engine that is installed on all proxy and object nodes.
#. The Swift storlet middleware and gateway that are installed on all proxy and object nodes.
#. A basic storlets enabled Docker image that is added to the private Docker registry.
#. A default account, enabled for Storlets with a default image deployed on all proxy and object nodes. This image is based on the basic storlets enabled image.
$ mkvirtualenv storlets
$ pip install storlets
The installation scripts take two input files:
#. A 'hosts' file describing the nodes on which the installation takes place. This is a standard Ansible hosts file that needs to have the following sections (an example is given below).
#. docker. The node to be installed with a private Docker registry
#. storlet-mgmt. The node to be installed with the Storlet management scripts
#. storlet-proxy. The list of the Swift cluster proxies
#. storlet-storage. The list of the Swift cluster object servers
#. root or a sudoer credentials Ansible can use to ssh the machines. In the below example we assume all nodes have the same credentials.
#. An Ansible var file with various inputs, such as the Keystone IP and credentials, the Storlet management account information, etc. The file is located in Deploy/playbook/common.yml, and we give below the entries of interest that may need editing.
At a high level the installation consists of the following steps:
#. Install Docker on all nodes.
#. Install the storlets middleware on each of the Swift nodes (proxies and storage nodes).
#. Create a tenant enabled for storlets (assumes Keystone).
#. Deploy a default Docker image for the above tenant.
#. Install a set of storlets management scripts. Done on a designated node having a storlet management role.
hosts file example
==================
::
192.168.33.2 ansible_ssh_user=root ansible_ssh_pass=passw0rd
192.168.33.3 ansible_ssh_user=root ansible_ssh_pass=passw0rd
192.168.33.4 ansible_ssh_user=root ansible_ssh_pass=passw0rd
192.168.33.5 ansible_ssh_user=root ansible_ssh_pass=passw0rd
[docker]
192.168.33.5
[storlet-mgmt]
192.168.33.5
[storlet-proxy]
192.168.33.2
[storlet-storage]
192.168.33.3
192.168.33.4
Few notes:
#. For an 'all-in-one' type of installation, one can specify 127.0.0.1 in all sections.
#. If all hosts have the same ssh user and password one can use ansible's group_vars/all
common.yml
==========
Below are the entries of interest of common.yml
Special attention should be given to the swift user and group IDs. Make sure they are identical on all hosts and match what is defined in the common.yml file.
The entry 'storlet_execute_on_proxy_only' controls whether storlets will be invoked only on proxy servers or also on object servers. This option allows to use
storlets with EC and encryption. Valid values are true / false
::
# A cross nodes directory for Storlets internal usage. Must exist with the same name in all proxy and storage nodes.
lxc_device: /home/lxc_device
# A pointer to this repo
storlet_source_dir: <need to point to the repo root>
# Swift Access information. The below IP should be an IP of one of the proxies.
swift_endpoint_host: 127.0.0.1
swift_public_url: http://{{ swift_endpoint_host }}:80/v1
# Keystone access information
keystone_endpoint_host: 127.0.0.1
keystone_admin_url: http://{{ keystone_endpoint_host }}:35357/v2.0
keystone_public_url: http://{{ keystone_endpoint_host }}:5000/v2.0
keystone_admin_token: ADMIN
keystone_admin_password: passw0rd
# Information for creating an account for the Storlet manager
storlet_management_account: storlet_management
storlet_management_admin_username: storlet_manager
storlet_manager_admin_password: storlet_manager
# Information for creating a Storlet enabled account
storlets_default_tenant_name: service
storlets_default_tenant_user_name: swift
storlets_default_tenant_user_password: passw0rd
swift_user_id: 1003
swift_group_id: 1003
# middleware and gateway config file constants
storlet_execute_on_proxy_only: false
Install
=======
to perform the installation follow these steps:
#. Create a hosts file as described above
#. Edit the file Deploy/playbook/common.yml according to the above
#. Under the root dir of the repo run 'ant build'
#. Under Deploy/playbook/ run 'ansible-playbook -i <hosts file> storlet.yml'
in case the hosts file has credentials of a sudoer user, you will need to run: 'ansible-playbook -s -i <hosts file> storlet.yml'
Tip: you might want to "export ANSIBLE_HOST_KEY_CHECKING=False" before running the playbook in case the hosts are not in known_hosts.
Note: The hosts file used for running the playbook is also used by the admin tool to deploy future images. Thus, the ssh information kept in
this file must also apply when used from the storlet-mgmt node.

50
doc/source/storlet_all_in_one.rst
View File

@@ -1,50 +0,0 @@
=====
S2AIO
=====
swift-storlets all in one is an Ansible based script that installs both swift and storlets in a virtual machine.
To run the script simply cd to the s2aio directory, edit the credentials for localhost in the hosts file, and run
the *storlets_aio.sh* script. The credentials for localhost must be of a 'sudoer user'
Note that the script takes a long time to run (10 minutes or more). More information below.
The script does the following:
1. Download the swift-install repo <https://github.com/Open-I-Beam/swift-install> and uses it
to install swift all in one inside a VM.
2. Download the swift-storlets repo into the VM, and executes the storlets installation within it.
One can edit s2aio/vars.yaml to control various parameters of the installation. Parameters of interest are:
* *installation_dir*. the directory where the swift-install repo is downloaded and where the VM data will reside
* *ip_prefix*. The prefix of the VMs IP
* *installation_user* and *installation_password*. Do not change these as they are aligned with the user created for the provisioned VM. These can be controlled via the swift-install tunables which are beyond the scope of this short doc.
The defaults for the storlets installation are found in:
* s2aio/roles/install_storlets/templates/common.yml
* s2aio/hosts
known issues
------------
1. Most issues are around ssh: The *storlets_aio.sh* script activates 2 major ansible playbooks:
* An ansible playbook that uses ssh to localhost with root or a 'sudoer user'
* An ansible palybook that is activated within the VM and uses ssh to localhost (within the VM) with the user vagrant
Although Ansible is configured not to check the host key, it sometimes fail. If this is the case just ssh manually where required and confirm the host key
2. The script takes a long time to run, and for most of the time it does not show progress. An alternative to executing *storlets_aio.sh* are the following steps. These would allow monitoring the progress of the ansible installation inside the VM.
1. cd s2aio
2. export ANSIBLE_HOST_KEY_CHECKING=False
3. ansible-playbook -s -i swift_install.yml
4. ansible-playbook -s -i prepare_storlets_install.yml
5. ssh vagrant 192.168.10.2 (using 'vagrant' as password)
Now, inside the VM:
1. cd /home/vagrant/swift-storlets
2. ant build
3. cd Deploy/playbook
4. ansible-playbook -s -i hosts storlet.yml
3. Sometimes the storlet installation part fails on a docker build operation. Usually, re-executing the ansible playbook is enough. This seems to have disappeared with Docker 1.6.2. Leaving the comment for a while.

121
doc/source/storlets_installation_guide.rst
View File

@@ -1,121 +0,0 @@
=====================
Storlets Installation
=====================
Background
==========
Storlets installation (and build) assume an existing Swift cluster that works with Keystone.
The installation consists of the following components:
#. A set of scripts to assist the Storlets admin. This admin represents the provider, and is responsible for the Storlet enabled accounts and their image management.
Those scripts make use of a private Docker registry as well as a designated account and containers used to keep Storlet management related state.
#. A Docker engine that is installed on all proxy and object nodes.
#. The Swift storlet middleware and gateway that are installed on all proxy and object nodes.
#. A basic storlets enabled Docker image that is added to the private Docker registry.
#. A default account, enabled for Storlets with a default image deployed on all proxy and object nodes. This image is based on the basic storlets enabled image.
The installation scripts take two input files:
#. A 'hosts' file describing the nodes on which the installation takes place. This is a standard Ansible hosts file that needs to have the following sections (an example is given below).
#. docker. The node to be installed with a private Docker registry
#. storlet-mgmt. The node to be installed with the Storlet management scripts
#. storlet-proxy. The list of the Swift cluster proxies
#. storlet-storage. The list of the Swift cluster object servers
#. root or a sudoer credentials Ansible can use to ssh the machines. In the below example we assume all nodes have the same credentials.
#. An Ansible var file with various inputs, such as the Keystone IP and credentials, the Storlet management account information, etc. The file is located in Deploy/playbook/common.yml, and we give below the entries of interest that may need editing.
At a high level the installation consists of the following steps:
#. Install Docker on all nodes.
#. Install the storlets middleware on each of the Swift nodes (proxies and storage nodes).
#. Create a tenant enabled for storlets (assumes Keystone).
#. Deploy a default Docker image for the above tenant.
#. Install a set of storlets management scripts. Done on a designated node having a storlet management role.
hosts file example
==================
::
192.168.33.2 ansible_ssh_user=root ansible_ssh_pass=passw0rd
192.168.33.3 ansible_ssh_user=root ansible_ssh_pass=passw0rd
192.168.33.4 ansible_ssh_user=root ansible_ssh_pass=passw0rd
192.168.33.5 ansible_ssh_user=root ansible_ssh_pass=passw0rd
[docker]
192.168.33.5
[storlet-mgmt]
192.168.33.5
[storlet-proxy]
192.168.33.2
[storlet-storage]
192.168.33.3
192.168.33.4
Few notes:
#. For an 'all-in-one' type of installation, one can specify 127.0.0.1 in all sections.
#. If all hosts have the same ssh user and password one can use ansible's group_vars/all
common.yml
==========
Below are the entries of interest of common.yml
Special attention should be given to the swift user and group IDs. Make sure they are identical on all hosts and match what is defined in the common.yml file.
The entry 'storlet_execute_on_proxy_only' controls whether storlets will be invoked only on proxy servers or also on object servers. This option allows to use
storlets with EC and encryption. Valid values are true / false
::
# A cross nodes directory for Storlets internal usage. Must exist with the same name in all proxy and storage nodes.
lxc_device: /home/lxc_device
# A pointer to this repo
storlet_source_dir: <need to point to the repo root>
# Swift Access information. The below IP should be an IP of one of the proxies.
swift_endpoint_host: 127.0.0.1
swift_public_url: http://{{ swift_endpoint_host }}:80/v1
# Keystone access information
keystone_endpoint_host: 127.0.0.1
keystone_admin_url: http://{{ keystone_endpoint_host }}:35357/v2.0
keystone_public_url: http://{{ keystone_endpoint_host }}:5000/v2.0
keystone_admin_token: ADMIN
keystone_admin_password: passw0rd
# Information for creating an account for the Storlet manager
storlet_management_account: storlet_management
storlet_management_admin_username: storlet_manager
storlet_manager_admin_password: storlet_manager
# Information for creating a Storlet enabled account
storlets_default_tenant_name: service
storlets_default_tenant_user_name: swift
storlets_default_tenant_user_password: passw0rd
swift_user_id: 1003
swift_group_id: 1003
# middleware and gateway config file constants
storlet_execute_on_proxy_only: false
Install
=======
to perform the installation follow these steps:
#. Create a hosts file as described above
#. Edit the file Deploy/playbook/common.yml according to the above
#. Under the root dir of the repo run 'ant build'
#. Under Deploy/playbook/ run 'ansible-playbook -i <hosts file> storlet.yml'
in case the hosts file has credentials of a sudoer user, you will need to run: 'ansible-playbook -s -i <hosts file> storlet.yml'
Tip: you might want to "export ANSIBLE_HOST_KEY_CHECKING=False" before running the playbook in case the hosts are not in known_hosts.
Note: The hosts file used for running the playbook is also used by the admin tool to deploy future images. Thus, the ssh information kept in
this file must also apply when used from the storlet-mgmt node.

28
s2aio/hosts
View File

@@ -1,28 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
[deploy]
localhost
[aio]
192.168.10.2
[deploy:vars]
ansible_ssh_user = root
ansible_ssh_pass = passw0rd
[aio:vars]
ansible_ssh_user = vagrant
ansible_ssh_pass = vagrant

21
s2aio/prepare_storlets_install.yml
View File

@@ -1,21 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- hosts: 192.168.10.2
vars_files:
- [ vars.yml ]
roles:
- role: storlets_aio_dependencies
- role: pull_storlets
- role: prepare_storlets_install

14
s2aio/roles/deploy_aio_dependencies/tasks/main.yml
View File

@@ -1,14 +0,0 @@
- name: ensure storlets aio dependencies are installed
shell: apt-cache showpkg ansible | grep 1.9.1-1ppa~trusty | wc -l
register: add_repo
- apt_repository: repo='ppa:ansible/ansible' state=present
when: "'0' in add_repo.stdout_lines[0]"
- apt:
pkg: "{{ item }}"
force: yes
update_cache: yes
cache_valid_time: 600
with_items: packages

6
s2aio/roles/deploy_aio_dependencies/vars/main.yml
View File

@@ -1,6 +0,0 @@
# Note that all packages carry a specific version.
# Thus, the ansible task would not use version: latest
packages:
- vagrant=1:1.5.4
- ansible=1.9.1-1ppa~trusty
- virtualbox=4.3.10-dfsg-1ubuntu4

18
s2aio/roles/install_storlets/tasks/main.yml
View File

@@ -1,18 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- name: Invoke Storlets Installation playbook
command: ansible-playbook -s -i hosts storlet.yml
args:
chdir: "{{ installation_user_home }}/{{swift_storlet_repo_name}}/Deploy/playbook"

34
s2aio/roles/install_swift/tasks/main.yml
View File

@@ -1,34 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- name: copy swift install template
template:
src: swift_config_json
dest: "{{ installation_dir }}{{ swift_install_repo_name }}/{{ swift_install_swift_config_file }}"
mode: 0755
- name: bring up aio VM
command: vagrant status
args:
chdir: "{{ installation_dir }}{{ swift_install_repo_name }}"
register: vagrant_status
- command: vagrant up
args:
chdir: "{{ installation_dir }}{{ swift_install_repo_name }}"
- command: ansible-playbook -s -i inventory/vagrant/swift_dynamic_inventory.py main-install.yml
args:
chdir: "{{ installation_dir }}{{ swift_install_repo_name }}"

43
s2aio/roles/install_swift/templates/swift_config_json
View File

@@ -1,43 +0,0 @@
{
"vagrant" : {
"machines" : {
"proxy" : {
"num":{{ proxy_num }}
},
"storage" : {
"num":{{ storage_num }},
"disk":{{ storage_disks}},
"disk_size":{{ storage_size }}
},
"client" : {
"num": {{ client_num }}
},
"dockerhub" : {
"num":{{ dockerhub_num }}
}
},
"boxes" : {
"centos7" : {
"box": "centbox" ,
"disk_controller" : "SATA Controller"
},
"ubuntu1404" : {
"box": "ubuntu/trusty64" ,
"box_url" : "https://vagrantcloud.com/ubuntu/boxes/trusty64",
"disk_controller" : "SATAController"
}
},
"box":"ubuntu1404",
"cpus":2,
"memory":2048,
"ip_prefix": "{{ ip_prefix }}"
},
"openstack_version":"juno",
"fstype":"xfs",
"groupvars":{
},
"credentials":{
"ansible_ssh_user":"vagrant",
"ansible_ssh_private_key_file":"~/.vagrant.d/insecure_private_key"
}
}

31
s2aio/roles/prepare_storlets_install/tasks/main.yml
View File

@@ -1,31 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- name: Copy templates
template:
src: common.yml
dest: "{{ installation_user_home }}{{swift_storlet_repo_name}}/Deploy/playbook/common.yml"
- template:
src: hosts
dest: "{{ installation_user_home }}{{swift_storlet_repo_name}}/Deploy/playbook/hosts"
- template:
src: ansible.cfg
dest: "{{ installation_user_home }}/.ansible.cfg"
- name: Build the Storlets code
shell: ant build
args:
chdir: "{{ installation_user_home }}{{swift_storlet_repo_name}}/"

2
s2aio/roles/prepare_storlets_install/templates/ansible.cfg
View File

@@ -1,2 +0,0 @@
[defaults]
host_key_checking = False

65
s2aio/roles/prepare_storlets_install/templates/common.yml
View File

@@ -1,65 +0,0 @@
ibm_container_install_dir: opt/ibm
lxc_device: "{{ installation_user_home }}/lxc_device"
storlet_source_dir: "{{ installation_user_home }}{{ swift_storlet_repo_name }}"
python_dist_packages_dir: usr/local/lib/python2.7/dist-packages
storlet_gateway_conf_file: /etc/swift/storlet_docker_gateway.conf
#----------------------------------------------------------------
# storlet_management / swift_storlet_management
# The below keystone vars MUST be synchronized with the swift_install variables.
keystone_endpoint_host: {{ aio_ip }}
keystone_admin_url: "http://{{ aio_ip }}:35357/v2.0"
keystone_public_url: "http://{{ aio_ip }}:5000/v2.0"
keystone_admin_token: ADMIN
keystone_admin_password: passw0rd
swift_endpoint_host: {{ aio_ip }}
swift_public_url: "http://{{ aio_ip }}:80/v1"
storlet_management_account: storlet_management
storlet_management_admin_username: storlet_manager
storlet_manager_admin_password: storlet_manager
storlet_management_swift_topology_container: swift_cluster
storlet_management_swift_topology_object : swift_cluster_nodes
storlet_management_ansible_dir: /opt/ibm/ansible/playbook
storlet_management_install_dir: /opt/ibm
# The below attribute can take true / false
storlets_enabled_attribute_name: storlet-enabled
#----------------------------------------------------------------
#----------------------------------------------------------------
# Storlets default tenant
storlets_default_tenant_name: service
storlets_default_tenant_user_name: swift
storlets_default_tenant_user_password: passw0rd
#----------------------------------------------------------------
#----------------------------------------------------------------
# docker_repository / docker_registry
docker_registry_random_string: ABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJABCDEFGHIJ1234
docker_registry_port: 5001
#----------------------------------------------------------------
#----------------------------------------------------------------
# docker_base_storlet_images / docekr_base_images
base_image_maintainer: root
base_os_image: ubuntu_14.04
storlets_image_name_suffix: ubuntu_14.04_jre8_storlets
swift_user_id: 1003
swift_group_id: 1003
#----------------------------------------------------------------
#----------------------------------------------------------------
# middleware and gateway config file constants
storlet_middleware: storlet_handler
storlet_container: storlet
storlet_dependency: dependency
storlet_log: storletlog
storlet_images: docker_images
storlet_timeout: 40
storlet_gateway_module: storlet_gateway.storlet_docker_gateway:StorletGatewayDocker
restart_linux_container_timeout: 3
#----------------------------------------------------------------

14
s2aio/roles/prepare_storlets_install/templates/hosts
View File

@@ -1,14 +0,0 @@
localhost ansible_ssh_user={{ installation_user }} ansible_ssh_pass={{ installation_password }}
[docker]
localhost
[storlet-proxy]
localhost
[storlet-storage]
localhost
[storlet-mgmt]
localhost

29
s2aio/roles/pull_storlets/tasks/main.yml
View File

@@ -1,29 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- stat: path="{{ installation_user_home }}"
register: inst_dir
- command: mkdir -p {{ installation_dir }}
when: not inst_dir.stat.exists
- name: clone repo
stat: path="{{ installation_user_home }}/{{swift_storlet_repo_name}}"
register: repo_dir
- git: repo={{ swift_storlet_repo_url }}
dest={{ installation_user_home }}{{ swift_storlet_repo_name }}
when: not repo_dir.stat.exists
- command: chmod 1777 "{{ installation_user_home }}/{{swift_storlet_repo_name}}"

34
s2aio/roles/pull_swift_ansible/tasks/main.yml
View File

@@ -1,34 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- stat: path="{{ installation_dir }}"
register: inst_dir
- command: mkdir -p {{ installation_dir }}
when: not inst_dir.stat.exists
- name: clone repo
stat: path="{{ installation_dir }}/{{swift_install_repo_name}}"
register: repo_dir
- git: repo={{ swift_install_repo_url }}
dest={{ installation_dir }}{{ swift_install_repo_name }}
version=1.0.0
when: not repo_dir.stat.exists
- command: chmod 1777 "{{ installation_dir }}/{{swift_install_repo_name}}"
- command: ansible-playbook -s -i "{{ installation_dir }}/{{swift_install_repo_name}}/{{ swift_install_swift_config_dir}}/{{swift_install_dynamic_inventory }}" pull_keystone_service_module.yml
args:
chdir: "{{ installation_dir }}{{ swift_install_repo_name }}"

20
s2aio/roles/storlets_aio_dependencies/tasks/main.yml
View File

@@ -1,20 +0,0 @@
- name: ensure storlets aio dependencies are installed
shell: apt-cache showpkg ansible | grep 1.9.1-1ppa~trusty | wc -l
register: add_repo
- apt_repository: repo='ppa:ansible/ansible' state=present
when: "'0' in add_repo.stdout_lines[0]"
- apt:
pkg: ansible=1.9.1-1ppa~trusty
force: yes
update_cache: yes
- apt:
pkg: "{{ item }}"
state: latest
force: yes
update_cache: yes
cache_valid_time: 600
with_items: packages

6
s2aio/roles/storlets_aio_dependencies/vars/main.yml
View File

@@ -1,6 +0,0 @@
packages:
- git
- sshpass
- ant
- oracle-java8-installer
- python-setuptools

2
s2aio/storlets_aio.sh
View File

@@ -1,2 +0,0 @@
export ANSIBLE_HOST_KEY_CHECKING=False
ansible-playbook -s -i hosts storlets_aio.yml

17
s2aio/storlets_aio.yml
View File

@@ -1,17 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- include: swift_install.yml
- include: prepare_storlets_install.yml
- include: storlets_install.yml

19
s2aio/storlets_install.yml
View File

@@ -1,19 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- hosts: 192.168.10.2
vars_files:
- [ vars.yml ]
roles:
- role: install_storlets

22
s2aio/swift_install.yml
View File

@@ -1,22 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- hosts: deploy
vars_files:
- [ vars.yml ]
roles:
- role: deploy_aio_dependencies
- role: pull_swift_ansible
- role: install_swift

2
s2aio/ustorlets_aio.sh
View File

@@ -1,2 +0,0 @@
export ANSIBLE_HOST_KEY_CHECKING=False
ansible-playbook -s -i hosts ustorlets_aio.yml

30
s2aio/ustorlets_aio.yml
View File

@@ -1,30 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
- hosts: deploy
vars_files:
- [ vars.yml ]
tasks:
- name: tear down aio VM
command: vagrant status
args:
chdir: "{{ installation_dir }}{{ swift_install_repo_name }}"
register: vagrant_status
- command: vagrant destroy -f
args:
chdir: "{{ installation_dir }}{{ swift_install_repo_name }}"
when: "vagrant_status is defined and 'running' in vagrant_status.stdout"
- command: rm -fr "{{ installation_dir }}"

47
s2aio/vars.yml
View File

@@ -1,47 +0,0 @@
#---------------------------------------------------------------------------
# Copyright IBM Corp. 2015, 2015 All Rights Reserved
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# Limitations under the License.
#---------------------------------------------------------------------------
installation_dir: /tmp/aio/
swift_install_repo_url : https://github.com/Open-I-Beam/swift-install.git
swift_install_repo_name: swift-install
swift_install_swift_config_dir: inventory/vagrant/
swift_install_swift_config_file: vagrant_config.json
swift_install_dynamic_inventory: swift_dynamic_inventory.py
swift_storlet_repo_url: https://github.com/openstack/storlets.git
swift_storlet_repo_name: swift-storlets
#--------------------------------------------------------------------------
# Swift cluster config - the below represents an 'all in one' configuration
proxy_num: 0
storage_num: 1
storage_disks: 1
# Storage size is in GBs
storage_size: 32
client_num: 0
dockerhub_num: 0
# The IP of the VM
ip_prefix: 192.168.10
# The IP used by the local installation
aio_ip: localhost
#
#--------------------------------------------------------------------------
#--------------------------------------------------------------------------
# Storlets installation vars
installation_user: vagrant
installation_password: vagrant
installation_user_home: /home/vagrant/
#
#--------------------------------------------------------------------------