devstack plugin interface
adds interface files for freezer-api to be installed as a devstack plugin Change-Id: I8e7d2d8ee973b6c6b88208cdb38c9aa6d69ca13d
This commit is contained in:
Fabrizio Vanni
parent
d66754784d
commit
8c43204bfd
93
README.rst
93
README.rst
|
|
@ -2,6 +2,20 @@
|
|||
Freezer API
|
||||
===========
|
||||
|
||||
TOC
|
||||
===
|
||||
::
|
||||
|
||||
1. Installation
|
||||
2. Devstack Plugin
|
||||
3. Concepts and definitions
|
||||
4. API registration
|
||||
5. API routes
|
||||
6. Backup metadata structure
|
||||
7. Freezer Client document structure
|
||||
8. Jobs
|
||||
9. Actions
|
||||
10. Sessions
|
||||
|
||||
1. Installation
|
||||
===============
|
||||
|
|
@ -10,14 +24,14 @@ Freezer API
|
|||
-----------------------------
|
||||
::
|
||||
|
||||
# pip install -r requirements.txt
|
||||
# git clone https://github.com/stackforge/freezer-api.git
|
||||
# cd freezer-api && sudo python setup.py install
|
||||
|
||||
1.2 Install freezer_api
|
||||
-----------------------
|
||||
::
|
||||
|
||||
# git clone https://github.com/stackforge/freezer-api.git
|
||||
# cd freezer-api && sudo python setup.py install
|
||||
# pip install -r requirements.txt
|
||||
|
||||
1.3 edit config file
|
||||
--------------------
|
||||
|
|
@ -72,7 +86,46 @@ To get information about optional additional parameters:
|
|||
# uwsgi --https :9090,foobar.crt,foobar.key --need-app --master --module freezer_api.cmd.api:application
|
||||
|
||||
|
||||
2. Concepts and definitions
|
||||
2. Devstack Plugin
|
||||
==================
|
||||
|
||||
2.1 Edit local.conf
|
||||
-------------------
|
||||
|
||||
To configure the Freezer API with DevStack, you will need to enable the
|
||||
freezer-api plugin by adding one line to the [[local|localrc]] section
|
||||
of your local.conf file::
|
||||
|
||||
enable_plugin freezer-api <GITURL> [GITREF]
|
||||
|
||||
where::
|
||||
|
||||
<GITURL> is the URL of a freezer-api repository
|
||||
[GITREF] is an optional git ref (branch/ref/tag). The default is master.
|
||||
|
||||
For example::
|
||||
|
||||
enable_plugin freezer-api https://github.com/stackforge/freezer-api.git master
|
||||
|
||||
|
||||
2.2 Plugin Options
|
||||
------------------
|
||||
The plugin makes use of apache2 by default.
|
||||
To use the *uwsgi* server set the following environment variable::
|
||||
|
||||
export FREEZER_API_SERVER_TYPE=uwsgi
|
||||
|
||||
The default port is *9090*. To configure the api to listen on a different port
|
||||
set the variable FREEZER_API_PORT.
|
||||
For example to make use of port 19090 use::
|
||||
|
||||
export FREEZER_API_PORT=19090
|
||||
|
||||
For more information, see:
|
||||
http://docs.openstack.org/developer/devstack/plugins.html
|
||||
|
||||
|
||||
3. Concepts and definitions
|
||||
===========================
|
||||
|
||||
*hostname* is _probably_ going to be the host fqdn.
|
||||
|
|
@ -86,7 +139,7 @@ defined as "container_hostname_backupname" identifies a group of related
|
|||
backups which share the same container,hostname and backupname
|
||||
|
||||
|
||||
3. API registration
|
||||
4. API registration
|
||||
===================
|
||||
::
|
||||
|
||||
|
|
@ -104,7 +157,7 @@ backups which share the same container,hostname and backupname
|
|||
--region regionOne
|
||||
|
||||
|
||||
4. API routes
|
||||
5. API routes
|
||||
=============
|
||||
|
||||
General
|
||||
|
|
@ -177,7 +230,7 @@ Freezer sessions management
|
|||
PUT /v1/sessions/{sessions_id}/jobs/{job_id} adds the job to the session
|
||||
DELETE /v1/sessions/{sessions_id}/jobs/{job_id} adds the job to the session
|
||||
|
||||
5. Backup metadata structure
|
||||
6. Backup metadata structure
|
||||
============================
|
||||
NOTE: sizes are in MB
|
||||
::
|
||||
|
|
@ -228,7 +281,7 @@ It stores and returns the information provided in this form:
|
|||
}
|
||||
|
||||
|
||||
6. Freezer Client document structure
|
||||
7. Freezer Client document structure
|
||||
====================================
|
||||
|
||||
Identifies a freezer client for the purpose of sending action
|
||||
|
|
@ -253,7 +306,7 @@ client_type document embeds the client_info and adds user_id::
|
|||
}
|
||||
|
||||
|
||||
7. Jobs
|
||||
8. Jobs
|
||||
=======
|
||||
A job describes a single action to be executed by a freezer client, for example a backup, or a restore.
|
||||
It contains the necessary information as if they were provided on the command line.
|
||||
|
|
@ -330,7 +383,7 @@ job document structure::
|
|||
}
|
||||
|
||||
|
||||
7.1 Scheduling Time Information
|
||||
8.1 Scheduling Time Information
|
||||
-------------------------------
|
||||
|
||||
Three types of scheduling can be identified:
|
||||
|
|
@ -340,13 +393,13 @@ Three types of scheduling can be identified:
|
|||
|
||||
Each type has specific parameters which can be given.
|
||||
|
||||
7.1.1 date scheduling
|
||||
8.1.1 date scheduling
|
||||
---------------------
|
||||
::
|
||||
|
||||
"schedule_date": : datetime isoformat
|
||||
|
||||
7.1.2 interval scheduling
|
||||
8.1.2 interval scheduling
|
||||
-------------------------
|
||||
::
|
||||
|
||||
|
|
@ -355,7 +408,7 @@ Each type has specific parameters which can be given.
|
|||
"schedule_start_date" : datetime isoformat
|
||||
"schedule_end_date" : datetime isoformat
|
||||
|
||||
7.1.3 cron-like scheduling
|
||||
8.1.3 cron-like scheduling
|
||||
--------------------------
|
||||
::
|
||||
|
||||
|
|
@ -371,7 +424,7 @@ Each type has specific parameters which can be given.
|
|||
"schedule_start_date" : datetime isoformat
|
||||
"schedule_end_date" : datetime isoformat
|
||||
|
||||
7.2 Job examples
|
||||
8.2 Job examples
|
||||
----------------
|
||||
|
||||
example backup freezer_action::
|
||||
|
|
@ -514,7 +567,7 @@ Finished job with result::
|
|||
|
||||
|
||||
|
||||
8 Actions
|
||||
9 Actions
|
||||
=========
|
||||
Actions are stored only to facilitate the assembling of different actions into jobs in the web UI.
|
||||
They are not directly used by the scheduler.
|
||||
|
|
@ -548,7 +601,7 @@ When a job is added to a session, the scheduling time of the session is copied i
|
|||
job data structure, so that any job belonging to the same session will start at the same time.
|
||||
|
||||
|
||||
9.1 Session Data Structure
|
||||
10.1 Session Data Structure
|
||||
--------------------------
|
||||
::
|
||||
|
||||
|
|
@ -583,7 +636,7 @@ job data structure, so that any job belonging to the same session will start at
|
|||
"user_id": string
|
||||
}
|
||||
|
||||
9.2 Session actions
|
||||
10.2 Session actions
|
||||
-------------------
|
||||
When the freezer scheduler running on a node wants to start a session,
|
||||
it sends a POST request to the following endpoint: ::
|
||||
|
|
@ -592,7 +645,7 @@ it sends a POST request to the following endpoint: ::
|
|||
|
||||
The body of the request bears the action and parameters
|
||||
|
||||
9.2.1 Session START action
|
||||
109.2.1 Session START action
|
||||
--------------------------
|
||||
::
|
||||
|
||||
|
|
@ -610,7 +663,7 @@ Example of a succesfull response: ::
|
|||
'session_tag': 23
|
||||
}
|
||||
|
||||
8.2.2 Session STOP action
|
||||
10.2.2 Session STOP action
|
||||
-------------------------
|
||||
::
|
||||
|
||||
|
|
@ -622,7 +675,7 @@ Example of a succesfull response: ::
|
|||
}
|
||||
}
|
||||
|
||||
8.3 Session-Job association
|
||||
10.3 Session-Job association
|
||||
---------------------------
|
||||
|
||||
PUT /v1/sessions/{sessions_id}/jobs/{job_id} adds the job to the session
|
||||
|
|
|
|||
|
|
@ -0,0 +1,33 @@
|
|||
This directory contains the Freezer-API DevStack plugin.
|
||||
|
||||
To configure the Freezer API with DevStack, you will need to
|
||||
enable this plugin by adding one line to the [[local|localrc]]
|
||||
section of your local.conf file.
|
||||
|
||||
To enable the plugin, add a line of the form::
|
||||
|
||||
enable_plugin freezer-api <GITURL> [GITREF]
|
||||
|
||||
where::
|
||||
|
||||
<GITURL> is the URL of a freezer-api repository
|
||||
[GITREF] is an optional git ref (branch/ref/tag). The default is master.
|
||||
|
||||
For example::
|
||||
|
||||
enable_plugin freezer-api https://github.com/stackforge/freezer-api.git master
|
||||
|
||||
|
||||
The plugin makes use of apache2 by default.
|
||||
To use the uwsgi server set the following environment variable:
|
||||
|
||||
export FREEZER_API_SERVER_TYPE=uwsgi
|
||||
|
||||
The default port is 9090. To configure the api to listen on a different port
|
||||
set the variable FREEZER_API_PORT.
|
||||
For example to make use of port 19090 use:
|
||||
|
||||
export FREEZER_API_PORT=19090
|
||||
|
||||
For more information, see:
|
||||
http://docs.openstack.org/developer/devstack/plugins.html
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
Listen %SERVICE_HOST%:%FREEZER_API_PORT%
|
||||
Listen 127.0.0.1:%FREEZER_API_PORT%
|
||||
|
||||
<VirtualHost *:%FREEZER_API_PORT%>
|
||||
WSGIDaemonProcess freezer-api processes=1 threads=1 user=%USER%
|
||||
WSGIProcessGroup freezer-api
|
||||
WSGIApplicationGroup freezer-api
|
||||
WSGIScriptAlias / %FREEZER_API_DIR%/freezer_api/cmd/api.py
|
||||
|
||||
ErrorLog /var/log/%APACHE_NAME%/freezer-api.log
|
||||
CustomLog /var/log/%APACHE_NAME%/freezer-api_access.log combined
|
||||
LogLevel info
|
||||
|
||||
<Directory %FREEZER_API_DIR%/freezer_api>
|
||||
Options Indexes FollowSymLinks MultiViews
|
||||
Require all granted
|
||||
AllowOverride None
|
||||
Order allow,deny
|
||||
allow from all
|
||||
LimitRequestBody 102400
|
||||
</Directory>
|
||||
</VirtualHost>
|
||||
|
|
@ -0,0 +1,185 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# Install and start Freezer service
|
||||
|
||||
# add the following to localrc:
|
||||
# enable_service freezer-api
|
||||
#
|
||||
# Dependencies:
|
||||
# - functions
|
||||
# - OS_AUTH_URL for auth in api
|
||||
# - DEST set to the destination directory
|
||||
# - SERVICE_PASSWORD, SERVICE_TENANT_NAME for auth in api
|
||||
# - STACK_USER service user
|
||||
|
||||
# functions called by the plugin.sh script
|
||||
# source plugin.sh <mode> [phase]
|
||||
# ---------
|
||||
# - <stack>
|
||||
# - <stack> [pre-install]
|
||||
# - <stack> [install]
|
||||
# - install_freezer_api
|
||||
# - <stack> [post-config]
|
||||
# - configure_freezer_api
|
||||
# - create_freezer_api_accounts
|
||||
# - <stack> [extra]
|
||||
# - init_freezer_api
|
||||
# - start_freezer_api
|
||||
# - <unstack>
|
||||
# - stop_freezer_api
|
||||
# - <clean>
|
||||
# - cleanup_freezer_api
|
||||
|
||||
# Save trace setting
|
||||
XTRACE=$(set +o | grep xtrace)
|
||||
set +o xtrace
|
||||
|
||||
|
||||
# Functions
|
||||
# ---------
|
||||
|
||||
function is_freezer_api_enabled {
|
||||
[[ ,${ENABLED_SERVICES} =~ ,"freezer-api" ]] && return 0
|
||||
}
|
||||
|
||||
|
||||
# executed during: clean
|
||||
function cleanup_freezer_api {
|
||||
sudo rm -f $(apache_site_config_for keystone)
|
||||
${TOP_DIR}/pkg/elasticsearch.sh uninstall
|
||||
}
|
||||
|
||||
# executed during: stack install
|
||||
function install_freezer_api {
|
||||
|
||||
git_clone $FREEZER_API_REPO $FREEZER_API_DIR $FREEZER_API_BRANCH
|
||||
|
||||
${TOP_DIR}/pkg/elasticsearch.sh download
|
||||
${TOP_DIR}/pkg/elasticsearch.sh install
|
||||
|
||||
setup_develop $FREEZER_API_DIR
|
||||
|
||||
if [[ "${FREEZER_API_SERVER_TYPE}" == "uwsgi" ]]; then
|
||||
pip_install 'uwsgi'
|
||||
fi
|
||||
}
|
||||
|
||||
# executed during: stack post-config
|
||||
function configure_freezer_api {
|
||||
|
||||
[ ! -d $FREEZER_API_CONF_DIR ] && sudo mkdir -m 755 -p $FREEZER_API_CONF_DIR
|
||||
sudo chown $USER $FREEZER_API_CONF_DIR
|
||||
|
||||
[ ! -d $FREEZER_API_LOG_DIR ] && sudo mkdir -m 755 -p $FREEZER_API_LOG_DIR
|
||||
sudo chown $USER $FREEZER_API_LOG_DIR
|
||||
|
||||
sudo cp $FREEZER_API_DIR/etc/freezer-api.conf $FREEZER_API_CONF_DIR
|
||||
|
||||
iniset $FREEZER_API_CONF 'storage' db elasticsearch
|
||||
iniset $FREEZER_API_CONF 'storage' index freezer
|
||||
iniset $FREEZER_API_CONF 'storage' hosts http://$SERVICE_HOST:9200
|
||||
|
||||
iniset $FREEZER_API_CONF 'keystone_authtoken' auth_protocol $KEYSTONE_AUTH_PROTOCOL
|
||||
iniset $FREEZER_API_CONF 'keystone_authtoken' auth_host $KEYSTONE_AUTH_HOST
|
||||
iniset $FREEZER_API_CONF 'keystone_authtoken' auth_port $KEYSTONE_AUTH_PORT
|
||||
iniset $FREEZER_API_CONF 'keystone_authtoken' admin_user freezer
|
||||
iniset $FREEZER_API_CONF 'keystone_authtoken' admin_password $SERVICE_PASSWORD
|
||||
iniset $FREEZER_API_CONF 'keystone_authtoken' admin_tenant_name $SERVICE_TENANT_NAME
|
||||
|
||||
if [[ "${FREEZER_API_SERVER_TYPE}" == "apache2" ]]; then
|
||||
echo_summary "Configuring Freezer API wsgi app in Apache"
|
||||
configure_apache_freezer_app
|
||||
fi
|
||||
|
||||
# TODO certs and more recent options such as elasticsearch connection parameters
|
||||
}
|
||||
|
||||
|
||||
# executed during: stack extra
|
||||
function init_freezer_api {
|
||||
# this also waits for elasticsearch to start
|
||||
${TOP_DIR}/pkg/elasticsearch.sh start
|
||||
|
||||
# put elasticsearch mappings
|
||||
freezer-db-init -y -e
|
||||
}
|
||||
|
||||
|
||||
# executed during: stack extra
|
||||
function start_freezer_api {
|
||||
if [[ "${FREEZER_API_SERVER_TYPE}" == "uwsgi" ]]; then
|
||||
run_process freezer-api "uwsgi --http :$FREEZER_API_PORT --need-app --master --module freezer_api.cmd.api:application"
|
||||
sleep 1
|
||||
elif [[ "${FREEZER_API_SERVER_TYPE}" == "apache2" ]]; then
|
||||
enable_apache_site freezer-api
|
||||
restart_apache_server
|
||||
tail_log freezer /var/log/$APACHE_NAME/freezer-api.log
|
||||
tail_log freezer-access /var/log/$APACHE_NAME/freezer-api_access.log
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
# executed during: stop
|
||||
function stop_freezer_api {
|
||||
${TOP_DIR}/pkg/elasticsearch.sh stop
|
||||
if [[ "${FREEZER_API_SERVER_TYPE}" == "uwsgi" ]]; then
|
||||
killall -9 uwsgi
|
||||
fi
|
||||
stop_process freezer-api
|
||||
}
|
||||
|
||||
|
||||
# utility function
|
||||
function get_id {
|
||||
echo `"$@" | awk '/ id / { print $4 }'`
|
||||
}
|
||||
|
||||
|
||||
# executed during: stack post-config
|
||||
function create_freezer_api_accounts {
|
||||
#
|
||||
# Setup admin user
|
||||
#
|
||||
SERVICE_TENANT_NAME=${SERVICE_TENANT_NAME:-service}
|
||||
local admin_role=$(get_or_create_role "admin")
|
||||
|
||||
local freezer_user=$(openstack user create \
|
||||
--password "$SERVICE_PASSWORD" \
|
||||
--project "$SERVICE_TENANT_NAME" \
|
||||
--email "freezer@example.com" \
|
||||
freezer \
|
||||
| grep " id " | get_field 2)
|
||||
|
||||
get_or_add_user_project_role $admin_role $freezer_user $SERVICE_TENANT_NAME
|
||||
|
||||
#
|
||||
# freezer backup service registration
|
||||
#
|
||||
local freezer_service=$(get_or_create_service "freezer" "backup" "Freezer Backup Service")
|
||||
|
||||
#
|
||||
# freezer backup endpoint registration
|
||||
#
|
||||
get_or_create_endpoint "$freezer_service" \
|
||||
"$REGION_NAME" \
|
||||
"$FREEZER_API_SERVICE_PROTOCOL://$SERVICE_HOST:$FREEZER_API_PORT" \
|
||||
"$FREEZER_API_SERVICE_PROTOCOL://$SERVICE_HOST:$FREEZER_API_PORT" \
|
||||
"$FREEZER_API_SERVICE_PROTOCOL://$SERVICE_HOST:$FREEZER_API_PORT"
|
||||
}
|
||||
|
||||
|
||||
function configure_apache_freezer_app {
|
||||
local freezer_api_apache_conf=$(apache_site_config_for freezer-api)
|
||||
sudo cp $FREEZER_API_FILES/apache-freezer-api.template $freezer_api_apache_conf
|
||||
sudo sed -e "
|
||||
s|%FREEZER_API_PORT%|$FREEZER_API_PORT|g;
|
||||
s|%SERVICE_HOST%|$SERVICE_HOST|g;
|
||||
s|%USER%|$STACK_USER|g;
|
||||
s|%FREEZER_API_DIR%|$FREEZER_API_DIR|g;
|
||||
s|%APACHE_NAME%|$APACHE_NAME|g;
|
||||
s|%VIRTUALENV%|$venv_path|g
|
||||
" -i $freezer_api_apache_conf
|
||||
}
|
||||
|
||||
# Restore xtrace
|
||||
$XTRACE
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
[[local|localrc]]
|
||||
disable_all_services
|
||||
|
||||
enable_plugin freezer-api https://github.com/stackforge/freezer-api.git master
|
||||
|
||||
enable_service rabbit mysql key
|
||||
|
||||
# This is to keep the token small for testing
|
||||
KEYSTONE_TOKEN_FORMAT=UUID
|
||||
|
||||
# Modify passwords as needed
|
||||
DATABASE_PASSWORD=secretdatabase
|
||||
RABBIT_PASSWORD=secretrabbit
|
||||
ADMIN_PASSWORD=secretadmin
|
||||
SERVICE_PASSWORD=secretservice
|
||||
SERVICE_TOKEN=111222333444
|
||||
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# check for service enabled
|
||||
if is_service_enabled freezer-api; then
|
||||
if [[ "$1" == "source" || "`type -t install_freezer_api`" != 'function' ]]; then
|
||||
# Initial source
|
||||
source $FREEZER_API_DIR/devstack/lib/freezer-api
|
||||
fi
|
||||
|
||||
if [[ "$1" == "stack" && "$2" == "install" ]]; then
|
||||
echo_summary "Installing Freezer API"
|
||||
install_freezer_api
|
||||
elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
|
||||
echo_summary "Configuring Freezer API"
|
||||
configure_freezer_api
|
||||
echo_summary "Creating keystone accounts for Freezer API"
|
||||
create_freezer_api_accounts
|
||||
elif [[ "$1" == "stack" && "$2" == "extra" ]]; then
|
||||
echo_summary "Initializing Freezer API"
|
||||
init_freezer_api
|
||||
start_freezer_api
|
||||
fi
|
||||
|
||||
if [[ "$1" == "unstack" ]]; then
|
||||
stop_freezer_api
|
||||
fi
|
||||
|
||||
if [[ "$1" == "clean" ]]; then
|
||||
cleanup_freezer_api
|
||||
fi
|
||||
fi
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
# Defaults
|
||||
# --------
|
||||
|
||||
# Set up default directories
|
||||
FREEZER_API_DIR=$DEST/freezer-api
|
||||
FREEZER_API_FILES=${FREEZER_API_DIR}/devstack/files
|
||||
FREEZER_API_CONF_DIR=${FREEZER_API_CONF_DIR:-/etc}
|
||||
FREEZER_API_CONF=$FREEZER_API_CONF_DIR/freezer-api.conf
|
||||
FREEZER_API_LOG_DIR=$DEST/logs
|
||||
|
||||
# Freezer API repository
|
||||
FREEZER_API_REPO=${FREEZER_API_REPO:-${GIT_BASE}/stackforge/freezer-api.git}
|
||||
FREEZER_API_BRANCH=${FREEZER_API_BRANCH:-master}
|
||||
|
||||
# supported servers: "apache2", "uwsgi"
|
||||
FREEZER_API_SERVER_TYPE=${FREEZER_API_SERVER_TYPE:-apache2}
|
||||
|
||||
FREEZER_API_SERVICE_PROTOCOL=http
|
||||
FREEZER_API_PORT=9090
|
||||
|
||||
enable_service freezer-api
|
||||
|
|
@ -22,10 +22,11 @@ use_syslogd = false
|
|||
#syslog_log_facility = LOG_LOCAL0
|
||||
|
||||
[keystone_authtoken]
|
||||
identity_uri = http://keystone:35357/
|
||||
auth_uri = http://keystone:5000/
|
||||
auth_protocol = http
|
||||
auth_host = keystone_host
|
||||
auth_port = 35357
|
||||
admin_user = freezer
|
||||
admin_password = FREEZER_PWD
|
||||
admin_password = freezer
|
||||
admin_tenant_name = service
|
||||
include_service_catalog = False
|
||||
delay_auth_decision = False
|
||||
|
|
|
|||
|
|
@ -39,5 +39,5 @@ CONF.register_cli_opts(common_cli_opts)
|
|||
|
||||
def parse_args(args=[], usage=None, default_config_files=[]):
|
||||
CONF(args=args,
|
||||
project='freezer',
|
||||
project='freezer-api',
|
||||
default_config_files=default_config_files)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,7 @@
|
|||
oslo.config>=1.9.3,<1.10.0 # Apache-2.0
|
||||
oslo.i18n>=1.5.0,<1.6.0 # Apache-2.0
|
||||
keystonemiddleware>=1.5.0,<1.6.0
|
||||
jsonschema>=2.0.0,<3.0.0
|
||||
elasticsearch>=1.3.0
|
||||
falcon>=0.1.6,<0.2.0
|
||||
mock>=1.0,<1.1.0
|
||||
Loading…
Reference in New Issue