openstack/mistral-extra
Code Issues Proposed changes

Add vagrant environment with devstack and mistral

Browse Source 
Also:
 - Fixed several typos

Change-Id: I23d187abf7a1984fc9021a34317852319d7e10a4
This commit is contained in:
Kirill Izotov 2014-06-27 12:29:04 +07:00
parent af0c1ddef7
commit 67185c62de
13 changed files with 223 additions and 103 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -40,3 +40,5 @@ nosetests.xml
etc/mistral.conf
tools/lintstack.head.py
tools/pylint_exceptions
/.vagrant/

22
README.md
View File

@ -1,3 +1,21 @@
Mistral Extras
================
Contains example applications additional tools for Mistral.
==============
Contains example applications and additional tools for Mistral.
Currently, there are three examples:
#### Create VM
Connects to OpenStack Nova and creates a VM with image and flavor id provided.
See `examples/create_vm/README.md` for more.
#### Run VM job
Spins up a VM, deploys web server, sends request, reports by email if an error occurred.
See `examples/vm_job/README.md` for more.
#### Webhooks scheduling
Starts local webserver and then assess it periodically using HTTP action.
See `examples/webhooks/README.md` for more.

21
Vagrantfile vendored Normal file
View File

@ -0,0 +1,21 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
require 'securerandom'
Vagrant.configure("2") do |config|
config.vm.box = "hashicorp/precise64"
hostname = "mistralextra"
config.vm.define "#{hostname}" do |box|
box.vm.hostname = "#{hostname}.book"
box.vm.network :private_network, ip: "172.16.80.100", :netmask => "255.255.0.0"
box.vm.network :forwarded_port, guest: 8000, host: 8000
box.vm.synced_folder ".", "/opt/mistral-extra"
box.vm.provision :shell, :path => "bootstrap.sh"
box.vm.provider :virtualbox do |vbox|
vbox.customize ["modifyvm", :id, "--memory", 3072]
vbox.customize ["modifyvm", :id, "--cpus", 2]
vbox.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
end
end
end

59
bootstrap.sh Normal file
View File

@ -0,0 +1,59 @@
#!/bin/bash
# Hack for eventlet case sensitivity problem
# (https://bitbucket.org/eventlet/eventlet/issue/81/stdlib-queue-not-found-from-within).
mkdir -p /opt/mistral-extra/.tox
mkdir -p /tmp/.tox
mount --bind /tmp/.tox /opt/mistral-extra/.tox
chown vagrant:vagrant /opt/mistral-extra/.tox
# Add README to motd
echo > /etc/motd.tail
cat /vagrant/README.md >> /etc/motd.tail
echo >> /etc/motd.tail
# Make user login directly into /opt/mistral-extra directory
su vagrant - -c "echo 'cd /opt/mistral-extra' >> /home/vagrant/.bashrc"
sudo apt-get -y install git
cd /opt/
git clone https://github.com/openstack-dev/devstack.git
git clone https://github.com/stackforge/mistral.git
git clone https://github.com/stackforge/python-mistralclient.git
# TODO(enykeev): make mistral-dashboard a devstack service
git clone https://github.com/stackforge/mistral-dashboard.git
cp /vagrant/local.conf /opt/devstack/
cp /opt/mistral/contrib/devstack/lib/* /opt/devstack/lib/
cp /opt/mistral/contrib/devstack/extras.d/* /opt/devstack/extras.d/
chown -R vagrant:vagrant /opt/
cd /opt/devstack
su vagrant - -c "./stack.sh"
cd /opt/python-mistralclient
sudo python setup.py install
cd /opt/mistral-dashboard
sudo python setup.py install
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://localhost:35357/v2.0
keystone user-role-add --user=mistral --tenant=admin --role=admin
keystone user-role-add --user=mistral --tenant=demo --role=admin
# Devstack's Horison service, for some reason, defines OPENSTACK_KEYSTONE_URL to v2 API instead of
# v3. Mistral, at the same time, requires v3 to work.
echo 'OPENSTACK_KEYSTONE_URL="http://localhost:5000/v3"' >> \
/opt/stack/horizon/openstack_dashboard/local/local_settings.py
echo 'OPENSTACK_API_VERSIONS = {"identity": 3}' >> \
/opt/stack/horizon/openstack_dashboard/local/local_settings.py
cd /opt/mistral-dashboard
sudo pip install -r requirements.txt
sudo cp _50_mistral.py.example /opt/stack/horizon/openstack_dashboard/local/enabled/_50_mistral.py
sudo service apache2 restart

46
examples/create_vm/README.md
View File

@ -2,41 +2,31 @@ Create VM example
==================
It demonstrates Mistral project features:
This example connects to OpenStack Nova and creates a VM with image id and flavor id you provided.
This example connects to OpenStack Nova and creates a VM with image and flavor id provided.
How to run
----------
- Make sure that you have installed python-mistralclient
- if not, install it:
1. Make sure that python-mistralclient have been installed. If not, install it:
```
git clone https://github.com/stackforge/python-mistralclient.git
cd python-mistralclient
python setup.py install
```
git clone https://github.com/stackforge/python-mistralclient.git
cd python-mistralclient
python setup.py install
- Make sure that Mistral API and at least one Mistral-executor are up and running
- Create workbook and upload the definition
2. Make sure that Mistral API and at least one Mistral-executor are up and running
3. Create workbook and upload the definition
```
mistral workbook-create myWorkbook description tag1,tag2 <path to create_vm_example.yaml>
```
mistral workbook-create myWorkbook description tag1,tag2 <path to create_vm_example.yaml>
- Create context file (simple json, which contains needed by workflow properties)
```
{
"server_name": "name_of_your_VM",
"nova_url": "url_to_nova_service",
"image_id": "id_of_your_image",
"network_id": "your_network_id_for_associate_with_VM",
"flavor_id": "id_of_flavor",
}
```
4. Create context file (simple json, which contains needed by workflow properties)
- Start execution
{
"server_name": "mistral-vm",
"nova_url": "http://localhost:8774/v2",
"image_id": "[copy from horizon or nova cli client]",
"flavor_id": "1"
}
```
mistral execution-create myWorkbook create-vm <path-to-the-context-file>
```
5. Start execution
mistral execution-create myWorkbook create-vm <path-to-the-context-file>

2
examples/create_vm/create_vm_example.yaml
View File

@ -1,6 +1,6 @@
# This example requires the following properties provided in execution context:
# - nova_url ## url to Nova service, e.g. http://0.0.0.0:8774/v3
# - server_name ## Name you want to give to new instance
# - server_name ## Name of the new instance
# - image_id ## image id from Glance service
# - flavor_id ## flavor id - type of instance hardware

95
examples/vm_job/README.md
View File

@ -7,67 +7,60 @@ It is needed for spinning up a VM and use it to do some useful work given calcul
What this example does
--------------------
1. Creates a VM
2. Waits till it is up and running
3. Runs small web server on VM
4. Sends request to the server
5. If an error occurred, it sends a email to admin with error message
1. Creates a VM
2. Waits till it is up and running
3. Runs small web server on VM
4. Sends request to the server
5. If an error occurred, it sends a email to admin with error message
How to run
----------
- Preparing
- Create your own image of virtual machine
- Make sure that VM has SSH server in running state
- Make sure that VM has password access via SSH
- Install packages (example for Debian/Ubuntu based systems)
1. Preparing
```
sudo apt-get install python-dev
sudo apt-get install python-pip
sudo pip install flask
```
1. Create an image of virtual machine
2. Make sure that VM has SSH server in running state
3. Make sure that VM has password access via SSH
4. Install packages (example for Debian/Ubuntu based systems)
- Put *web_app.py* file in your home user directory
> To check if it works, please type
```python ~/web_app.py```
> (this should run a small server on 5000 port)
- Save image
sudo apt-get install python-dev
sudo apt-get install python-pip
sudo pip install flask
- Make sure that you have installed python-mistralclient
- if not, install it:
5. Put *web_app.py* file into user's home directory. To check if it works, type
```
git clone https://github.com/stackforge/python-mistralclient.git
cd python-mistralclient
python setup.py install
```
python ~/web_app.py
- Make sure that Mistral API and at least one Mistral-executor are up and running
- Create workbook and upload the definition
This should run a small server on 5000 port.
```
mistral workbook-create myWorkbook description tag1,tag2 <path to run_vm_job.yaml>
```
6. Save image
- Create context file (simple json, which contains needed by workflow properties)
2. Make sure that python-mistralclient have been installed. If not, install it:
```
{
"server_name": "name_of_your_VM",
"nova_url": "url_to_nova_service",
"image_id": "id_of_your_image",
"flavor_id": "id_of_flavor",
"ssh_username": "your_VM_username",
"ssh_password": "your_VM_password",
"smtp_server": "address_to_smtp_server",
"from_email": "your_email_address",
"smtp_password": "password_of_your_email",
"admin_email": "address_on_which_you_wish_to_send_messages",
}
```
git clone https://github.com/stackforge/python-mistralclient.git
cd python-mistralclient
python setup.py install
- Start execution
```
mistral execution-create myWorkbook createVM <path-to-the-context-file>
```
3. Make sure that Mistral API and at least one Mistral-executor are up and running
4. Create workbook and upload the definition
mistral workbook-create myWorkbook description tag1,tag2 <path to run_vm_job.yaml>
5. Create context file (simple json, which contains needed by workflow properties)
{
"server_name": "mistral-vm",
"nova_url": "http://172.16.80.100:8774/v2",
"image_id": "[copy from horizon or nova cli client]",
"flavor_id": "1",
"ssh_username": "[VM username]",
"ssh_password": "[VM password]",
"smtp_server": "[address to smtp server]",
"from_email": "[email address]",
"smtp_password": "[email password]",
"admin_email": "[address the message should be sent to]",
}
6. Start execution
mistral execution-create myWorkbook createVM <path-to-the-context-file>

6
examples/vm_job/run_vm_job.yaml
View File

@ -1,10 +1,10 @@
# This example requires the following properties provided in execution context:
# - nova_url ## url to Nova service, e.g. http://0.0.0.0:8774/v3
# - server_name ## Name you want to give to new instance
# - server_name ## Name of the new instance
# - image_id ## image id from Glance service
# - flavor_id ## flavor id - type of instance hardware
# - ssh_username ## username of your VM
# - ssh_password ## password to your VM
# - ssh_username ## VM username
# - ssh_password ## VM password
# - admin_email ## email address to send notifications to
# - from_email ## email address to send notifications from
# - smtp_server ## SMTP server to use for sending emails (e.g. smtp.gmail.com:587)

17
examples/webhooks/README.md
View File

@ -1,8 +1,7 @@
Webhooks scheduling
===================
What does this example do?
--------------------------
### What does this example do?
It runs small web-server and exposes one URL - 0.0.0.0:8988/ (host and port by default)
We can make request to /<name>, where name is the task name which will run on the server.
@ -14,13 +13,11 @@ For this Mistral example an OpenStack installation is optional.
In case of running the example without OpenStack server side authentication
based on Keystone must be disabled by setting configuration option "auth_enable"
under group "pecan" to False like the following:
```
[pecan]
auth_enable = False
```
[pecan]
auth_enable = False
### Running the example
From the root folder ("mistral-extra" by default) run the following shell command:<br><br>
```
tox -evenv -- python examples/webhooks/cmd/api.py -v
```
From the root folder ("mistral-extra" by default) run the following shell command:
tox -evenv -- python examples/webhooks/cmd/run.py

13
examples/webhooks/api/client.py
View File

@ -17,11 +17,20 @@
import pkg_resources as pkg
from mistralclient.api import client
from mistralclient.openstack.common.cliutils import env
from examples.webhooks import version
MISTRAL_URL = "http://localhost:8989/v1"
CLIENT = client.Client(mistral_url=MISTRAL_URL)
from oslo.config import cfg
CLIENT = client.Client(
mistral_url=env('OS_MISTRAL_URL', default=cfg.CONF.client.mistral_url),
auth_url=env('OS_AUTH_URL', default=cfg.CONF.client.auth_url),
username=env('OS_USERNAME', default=cfg.CONF.client.username),
api_key=env('OS_PASSWORD', default=cfg.CONF.client.password),
project_name=env('OS_TENANT_NAME', default=cfg.CONF.client.tenant_name)
)
WB_NAME = "myWorkbook"

9
examples/webhooks/config.py
View File

@ -24,8 +24,17 @@ api_opts = [
cfg.IntOpt('port', default=8988, help='Simple-app API server port')
]
client_opts = [
cfg.StrOpt('mistral_url', default='http://localhost:8989/v1'),
cfg.StrOpt('auth_url', default='http://localhost:5000/v3'),
cfg.StrOpt('username', default='admin'),
cfg.StrOpt('password', default='openstack'),
cfg.StrOpt('tenant_name', default='admin')
]
CONF = cfg.CONF
CONF.register_opts(api_opts, group='api')
CONF.register_opts(client_opts, group='client')
def parse_args(args=None, usage=None, default_config_files=None):

12
examples/webhooks/demo.yaml
View File

@ -54,9 +54,9 @@ Workflow:
requires: [backup_user_data, backup_service_data, send_email]
action: MyRest.execute_backup
triggers:
execute_backup:
type: periodic
tasks: execute_backup
parameters:
cron-pattern: "*/1 * * * *"
# triggers:
# execute_backup:
# type: periodic
# tasks: execute_backup
# parameters:
# cron-pattern: "*/1 * * * *"

22
local.conf Normal file
View File

@ -0,0 +1,22 @@
[[local|localrc]]
SERVICE_TOKEN=ThisIsAServiceToken
ADMIN_PASSWORD=openstack
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
FLOATING_RANGE=172.16.80.224/27
FIXED_RANGE=10.11.12.0/24
FIXED_NETWORK_SIZE=256
FLAT_INTERFACE=br100
PUBLIC_INTERFACE=eth1
HOST_IP=172.16.80.100
LOGFILE=$DEST/logs/stack.sh.log
LOGDAYS=1
SCREEN_LOGDIR=$DEST/logs/screen
SYSLOG=True
SYSLOG_HOST=$HOST_IP
SYSLOG_PORT=516
enable_service h-eng h-api h-api-cfn h-api-cw
enable_service horizon
enable_service tempest
enable_service mistral