openstack/charm-glance
Code Issues Proposed changes

Deploy from source

Browse Source
This commit is contained in:
Corey Bryant 2015-04-01 16:48:59 +00:00
commit d1e4995977
36 changed files with 1310 additions and 181 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.bzrignore
View File

@ -1,2 +1,3 @@
.coverage
bin
tags

6
Makefile
View File

@ -3,7 +3,7 @@ PYTHON := /usr/bin/env python
lint:
@echo "Running flake8 tests: "
@flake8 --exclude hooks/charmhelpers hooks unit_tests tests
@flake8 --exclude hooks/charmhelpers actions hooks unit_tests tests
@echo "OK"
@echo "Running charm proof: "
@charm proof
@ -27,7 +27,9 @@ test:
# raise_status() messages to stderr:
# https://bugs.launchpad.net/amulet/+bug/1320357
@juju test -v -p AMULET_HTTP_PROXY --timeout 900 \
00-setup 14-basic-precise-icehouse 15-basic-trusty-icehouse
00-setup 14-basic-precise-icehouse 15-basic-trusty-icehouse \
16-basic-trusty-icehouse-git 17-basic-trusty-juno \
18-basic-trusty-juno-git
publish: lint unit_test
bzr push lp:charms/glance

82
README.md
View File

@ -81,6 +81,88 @@ as the endpoint for Glance).
Note that Glance in this configuration must be used with either Ceph or
Swift providing backing image storage.
Deploying from source
---------------------
The minimum openstack-origin-git config required to deploy from source is:
openstack-origin-git:
"repositories:
- {name: requirements,
repository: 'git://git.openstack.org/openstack/requirements',
branch: stable/juno}
- {name: glance,
repository: 'git://git.openstack.org/openstack/glance',
branch: stable/juno}"
Note that there are only two 'name' values the charm knows about: 'requirements'
and 'glance'. These repositories must correspond to these 'name' values.
Additionally, the requirements repository must be specified first and the
glance repository must be specified last. All other repostories are installed
in the order in which they are specified.
The following is a full list of current tip repos (may not be up-to-date):
openstack-origin-git:
"repositories:
- {name: requirements,
repository: 'git://git.openstack.org/openstack/requirements',
branch: master}
- {name: oslo-concurrency,
repository: 'git://git.openstack.org/openstack/oslo.concurrency',
branch: master}
- {name: oslo-config,
repository: 'git://git.openstack.org/openstack/oslo.config',
branch: master}
- {name: oslo-db,
repository: 'git://git.openstack.org/openstack/oslo.db',
branch: master}
- {name: oslo-i18n,
repository: 'git://git.openstack.org/openstack/oslo.i18n',
branch: master}
- {name: oslo-messaging,
repository: 'git://git.openstack.org/openstack/oslo.messaging',
branch: master}
- {name: oslo-serialization,
repository: 'git://git.openstack.org/openstack/oslo.serialization',
branch: master}
- {name: oslo-utils,
repository: 'git://git.openstack.org/openstack/oslo.utils',
branch: master}
- {name: oslo-vmware,
repository: 'git://git.openstack.org/openstack/oslo.vmware',
branch: master}
- {name: osprofiler,
repository: 'git://git.openstack.org/stackforge/osprofiler',
branch: master}
- {name: pbr,
repository: 'git://git.openstack.org/openstack-dev/pbr',
branch: master}
- {name: python-keystoneclient,
repository: 'git://git.openstack.org/openstack/python-keystoneclient',
branch: master}
- {name: python-swiftclient,
repository: 'git://git.openstack.org/openstack/python-swiftclient',
branch: master}
- {name: sqlalchemy-migrate,
repository: 'git://git.openstack.org/stackforge/sqlalchemy-migrate',
branch: master}
- {name: stevedore,
repository: 'git://git.openstack.org/openstack/stevedore',
branch: master}
- {name: wsme,
repository: 'git://git.openstack.org/stackforge/wsme',
branch: master}
- {name: keystonemiddleware,
repository: 'git://git.openstack.org/openstack/keystonemiddleware',
branch: master}
- {name: glance-store,
repository: 'git://git.openstack.org/openstack/glance_store',
branch: master}
- {name: glance,
repository: 'git://git.openstack.org/openstack/glance',
branch: master}"
Contact Information
-------------------

2
actions.yaml Normal file
View File

@ -0,0 +1,2 @@
git-reinstall:
description: Reinstall glance from the openstack-origin-git repositories.

1
actions/git-reinstall Symbolic link
View File

@ -0,0 +1 @@
git_reinstall.py

40
actions/git_reinstall.py Executable file
View File

@ -0,0 +1,40 @@
#!/usr/bin/python
import sys
import traceback
sys.path.append('hooks/')
from charmhelpers.contrib.openstack.utils import (
git_install_requested,
)
from charmhelpers.core.hookenv import (
action_set,
action_fail,
config,
)
from glance_utils import (
git_install,
)
def git_reinstall():
"""Reinstall from source and restart services.
If the openstack-origin-git config option was used to install openstack
from source git repositories, then this action can be used to reinstall
from updated git repositories, followed by a restart of services."""
if not git_install_requested():
action_fail('openstack-origin-git is not configured')
return
try:
git_install(config('openstack-origin-git'))
except:
action_set({'traceback': traceback.format_exc()})
action_fail('git-reinstall resulted in an unexpected error')
if __name__ == '__main__':
git_reinstall()

16
config.yaml
View File

@ -14,6 +14,22 @@ options:
Note that updating this setting to a source that is known to
provide a later version of OpenStack will trigger a software
upgrade.
Note that when openstack-origin-git is specified, openstack
specific packages will be installed from source rather than
from the openstack-origin repository.
openstack-origin-git:
default:
type: string
description: |
Specifies a YAML-formatted dictionary listing the git
repositories and branches from which to install OpenStack and
its dependencies.
Note that the installed config files will be determined based on
the OpenStack release of the openstack-origin option.
For more details see README.md.
database-user:
default: glance
type: string

27
hooks/charmhelpers/contrib/openstack/amulet/deployment.py
View File

@ -15,6 +15,7 @@
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
import six
from collections import OrderedDict
from charmhelpers.contrib.amulet.deployment import (
AmuletDeployment
)
@ -100,12 +101,34 @@ class OpenStackAmuletDeployment(AmuletDeployment):
"""
(self.precise_essex, self.precise_folsom, self.precise_grizzly,
self.precise_havana, self.precise_icehouse,
self.trusty_icehouse) = range(6)
self.trusty_icehouse, self.trusty_juno, self.trusty_kilo) = range(8)
releases = {
('precise', None): self.precise_essex,
('precise', 'cloud:precise-folsom'): self.precise_folsom,
('precise', 'cloud:precise-grizzly'): self.precise_grizzly,
('precise', 'cloud:precise-havana'): self.precise_havana,
('precise', 'cloud:precise-icehouse'): self.precise_icehouse,
('trusty', None): self.trusty_icehouse}
('trusty', None): self.trusty_icehouse,
('trusty', 'cloud:trusty-juno'): self.trusty_juno,
('trusty', 'cloud:trusty-kilo'): self.trusty_kilo}
return releases[(self.series, self.openstack)]
def _get_openstack_release_string(self):
"""Get openstack release string.
Return a string representing the openstack release.
"""
releases = OrderedDict([
('precise', 'essex'),
('quantal', 'folsom'),
('raring', 'grizzly'),
('saucy', 'havana'),
('trusty', 'icehouse'),
('utopic', 'juno'),
('vivid', 'kilo'),
])
if self.openstack:
os_origin = self.openstack.split(':')[1]
return os_origin.split('%s-' % self.series)[1].split('/')[0]
else:
return releases[self.series]

154
hooks/charmhelpers/contrib/openstack/context.py
View File

@ -47,6 +47,7 @@ from charmhelpers.core.hookenv import (
)
from charmhelpers.core.sysctl import create as sysctl_create
from charmhelpers.core.strutils import bool_from_string
from charmhelpers.core.host import (
list_nics,
@ -67,6 +68,7 @@ from charmhelpers.contrib.hahelpers.apache import (
)
from charmhelpers.contrib.openstack.neutron import (
neutron_plugin_attribute,
parse_data_port_mappings,
)
from charmhelpers.contrib.openstack.ip import (
resolve_address,
@ -82,7 +84,6 @@ from charmhelpers.contrib.network.ip import (
is_bridge_member,
)
from charmhelpers.contrib.openstack.utils import get_host_ip
CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
ADDRESS_TYPES = ['admin', 'internal', 'public']
@ -319,14 +320,15 @@ def db_ssl(rdata, ctxt, ssl_dir):
class IdentityServiceContext(OSContextGenerator):
interfaces = ['identity-service']
def __init__(self, service=None, service_user=None):
def __init__(self, service=None, service_user=None, rel_name='identity-service'):
self.service = service
self.service_user = service_user
self.rel_name = rel_name
self.interfaces = [self.rel_name]
def __call__(self):
log('Generating template context for identity-service', level=DEBUG)
log('Generating template context for ' + self.rel_name, level=DEBUG)
ctxt = {}
if self.service and self.service_user:
@ -340,7 +342,7 @@ class IdentityServiceContext(OSContextGenerator):
ctxt['signing_dir'] = cachedir
for rid in relation_ids('identity-service'):
for rid in relation_ids(self.rel_name):
for unit in related_units(rid):
rdata = relation_get(rid=rid, unit=unit)
serv_host = rdata.get('service_host')
@ -1162,3 +1164,145 @@ class SysctlContext(OSContextGenerator):
sysctl_create(sysctl_dict,
'/etc/sysctl.d/50-{0}.conf'.format(charm_name()))
return {'sysctl': sysctl_dict}
class NeutronAPIContext(OSContextGenerator):
'''
Inspects current neutron-plugin-api relation for neutron settings. Return
defaults if it is not present.
'''
interfaces = ['neutron-plugin-api']
def __call__(self):
self.neutron_defaults = {
'l2_population': {
'rel_key': 'l2-population',
'default': False,
},
'overlay_network_type': {
'rel_key': 'overlay-network-type',
'default': 'gre',
},
'neutron_security_groups': {
'rel_key': 'neutron-security-groups',
'default': False,
},
'network_device_mtu': {
'rel_key': 'network-device-mtu',
'default': None,
},
'enable_dvr': {
'rel_key': 'enable-dvr',
'default': False,
},
'enable_l3ha': {
'rel_key': 'enable-l3ha',
'default': False,
},
}
ctxt = self.get_neutron_options({})
for rid in relation_ids('neutron-plugin-api'):
for unit in related_units(rid):
rdata = relation_get(rid=rid, unit=unit)
if 'l2-population' in rdata:
ctxt.update(self.get_neutron_options(rdata))
return ctxt
def get_neutron_options(self, rdata):
settings = {}
for nkey in self.neutron_defaults.keys():
defv = self.neutron_defaults[nkey]['default']
rkey = self.neutron_defaults[nkey]['rel_key']
if rkey in rdata.keys():
if type(defv) is bool:
settings[nkey] = bool_from_string(rdata[rkey])
else:
settings[nkey] = rdata[rkey]
else:
settings[nkey] = defv
return settings
class ExternalPortContext(NeutronPortContext):
def __call__(self):
ctxt = {}
ports = config('ext-port')
if ports:
ports = [p.strip() for p in ports.split()]
ports = self.resolve_ports(ports)
if ports:
ctxt = {"ext_port": ports[0]}
napi_settings = NeutronAPIContext()()
mtu = napi_settings.get('network_device_mtu')
if mtu:
ctxt['ext_port_mtu'] = mtu
return ctxt
class DataPortContext(NeutronPortContext):
def __call__(self):
ports = config('data-port')
if ports:
portmap = parse_data_port_mappings(ports)
ports = portmap.values()
resolved = self.resolve_ports(ports)
normalized = {get_nic_hwaddr(port): port for port in resolved
if port not in ports}
normalized.update({port: port for port in resolved
if port in ports})
if resolved:
return {bridge: normalized[port] for bridge, port in
six.iteritems(portmap) if port in normalized.keys()}
return None
class PhyNICMTUContext(DataPortContext):
def __call__(self):
ctxt = {}
mappings = super(PhyNICMTUContext, self).__call__()
if mappings and mappings.values():
ports = mappings.values()
napi_settings = NeutronAPIContext()()
mtu = napi_settings.get('network_device_mtu')
if mtu:
ctxt["devs"] = '\\n'.join(ports)
ctxt['mtu'] = mtu
return ctxt
class NetworkServiceContext(OSContextGenerator):
def __init__(self, rel_name='quantum-network-service'):
self.rel_name = rel_name
self.interfaces = [rel_name]
def __call__(self):
for rid in relation_ids(self.rel_name):
for unit in related_units(rid):
rdata = relation_get(rid=rid, unit=unit)
ctxt = {
'keystone_host': rdata.get('keystone_host'),
'service_port': rdata.get('service_port'),
'auth_port': rdata.get('auth_port'),
'service_tenant': rdata.get('service_tenant'),
'service_username': rdata.get('service_username'),
'service_password': rdata.get('service_password'),
'quantum_host': rdata.get('quantum_host'),
'quantum_port': rdata.get('quantum_port'),
'quantum_url': rdata.get('quantum_url'),
'region': rdata.get('region'),
'service_protocol':
rdata.get('service_protocol') or 'http',
'auth_protocol':
rdata.get('auth_protocol') or 'http',
}
if context_complete(ctxt):
return ctxt
return {}

13
hooks/charmhelpers/contrib/openstack/templates/git.upstart Normal file
View File

@ -0,0 +1,13 @@
description "{{ service_description }}"
author "Juju {{ service_name }} Charm <juju@localhost>"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
exec start-stop-daemon --start --chuid {{ user_name }} \
--chdir {{ start_dir }} --name {{ process_name }} \
--exec {{ executable_name }} -- \
--config-file={{ config_file }} \
--log-file={{ log_file }}

9
hooks/charmhelpers/contrib/openstack/templates/section-keystone-authtoken Normal file
View File

@ -0,0 +1,9 @@
{% if auth_host -%}
[keystone_authtoken]
identity_uri = {{ auth_protocol }}://{{ auth_host }}:{{ auth_port }}/{{ auth_admin_prefix }}
auth_uri = {{ service_protocol }}://{{ service_host }}:{{ service_port }}/{{ service_admin_prefix }}
admin_tenant_name = {{ admin_tenant_name }}
admin_user = {{ admin_user }}
admin_password = {{ admin_password }}
signing_dir = {{ signing_dir }}
{% endif -%}

22
hooks/charmhelpers/contrib/openstack/templates/section-rabbitmq-oslo Normal file
View File

@ -0,0 +1,22 @@
{% if rabbitmq_host or rabbitmq_hosts -%}
[oslo_messaging_rabbit]
rabbit_userid = {{ rabbitmq_user }}
rabbit_virtual_host = {{ rabbitmq_virtual_host }}
rabbit_password = {{ rabbitmq_password }}
{% if rabbitmq_hosts -%}
rabbit_hosts = {{ rabbitmq_hosts }}
{% if rabbitmq_ha_queues -%}
rabbit_ha_queues = True
rabbit_durable_queues = False
{% endif -%}
{% else -%}
rabbit_host = {{ rabbitmq_host }}
{% endif -%}
{% if rabbit_ssl_port -%}
rabbit_use_ssl = True
rabbit_port = {{ rabbit_ssl_port }}
{% if rabbit_ssl_ca -%}
kombu_ssl_ca_certs = {{ rabbit_ssl_ca }}
{% endif -%}
{% endif -%}
{% endif -%}

4
hooks/charmhelpers/contrib/openstack/templates/zeromq → hooks/charmhelpers/contrib/openstack/templates/section-zeromq
View File

@ -3,12 +3,12 @@
rpc_backend = zmq
rpc_zmq_host = {{ zmq_host }}
{% if zmq_redis_address -%}
rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_redis.MatchMakerRedis
rpc_zmq_matchmaker = redis
matchmaker_heartbeat_freq = 15
matchmaker_heartbeat_ttl = 30
[matchmaker_redis]
host = {{ zmq_redis_address }}
{% else -%}
rpc_zmq_matchmaker = oslo.messaging._drivers.matchmaker_ring.MatchMakerRing
rpc_zmq_matchmaker = ring
{% endif -%}
{% endif -%}

185
hooks/charmhelpers/contrib/openstack/utils.py
View File

@ -30,6 +30,10 @@ import yaml
from charmhelpers.contrib.network import ip
from charmhelpers.core import (
unitdata,
)
from charmhelpers.core.hookenv import (
config,
log as juju_log,
@ -330,6 +334,21 @@ def configure_installation_source(rel):
error_out("Invalid openstack-release specified: %s" % rel)
def config_value_changed(option):
"""
Determine if config value changed since last call to this function.
"""
hook_data = unitdata.HookData()
with hook_data():
db = unitdata.kv()
current = config(option)
saved = db.get(option)
db.set(option, current)
if saved is None:
return False
return current != saved
def save_script_rc(script_path="scripts/scriptrc", **env_vars):
"""
Write an rc file in the charm-delivered directory containing
@ -469,82 +488,103 @@ def os_requires_version(ostack_release, pkg):
def git_install_requested():
"""Returns true if openstack-origin-git is specified."""
return config('openstack-origin-git') != "None"
"""
Returns true if openstack-origin-git is specified.
"""
return config('openstack-origin-git') is not None
requirements_dir = None
def git_clone_and_install(file_name, core_project):
"""Clone/install all OpenStack repos specified in yaml config file."""
global requirements_dir
def git_clone_and_install(projects_yaml, core_project):
"""
Clone/install all specified OpenStack repositories.
if file_name == "None":
The expected format of projects_yaml is:
repositories:
- {name: keystone,
repository: 'git://git.openstack.org/openstack/keystone.git',
branch: 'stable/icehouse'}
- {name: requirements,
repository: 'git://git.openstack.org/openstack/requirements.git',
branch: 'stable/icehouse'}
directory: /mnt/openstack-git
http_proxy: http://squid.internal:3128
https_proxy: https://squid.internal:3128
The directory, http_proxy, and https_proxy keys are optional.
"""
global requirements_dir
parent_dir = '/mnt/openstack-git'
if not projects_yaml:
return
yaml_file = os.path.join(charm_dir(), file_name)
projects = yaml.load(projects_yaml)
_git_validate_projects_yaml(projects, core_project)
# clone/install the requirements project first
installed = _git_clone_and_install_subset(yaml_file,
whitelist=['requirements'])
if 'requirements' not in installed:
error_out('requirements git repository must be specified')
if 'http_proxy' in projects.keys():
os.environ['http_proxy'] = projects['http_proxy']
# clone/install all other projects except requirements and the core project
blacklist = ['requirements', core_project]
_git_clone_and_install_subset(yaml_file, blacklist=blacklist,
update_requirements=True)
if 'https_proxy' in projects.keys():
os.environ['https_proxy'] = projects['https_proxy']
# clone/install the core project
whitelist = [core_project]
installed = _git_clone_and_install_subset(yaml_file, whitelist=whitelist,
update_requirements=True)
if core_project not in installed:
error_out('{} git repository must be specified'.format(core_project))
if 'directory' in projects.keys():
parent_dir = projects['directory']
for p in projects['repositories']:
repo = p['repository']
branch = p['branch']
if p['name'] == 'requirements':
repo_dir = _git_clone_and_install_single(repo, branch, parent_dir,
update_requirements=False)
requirements_dir = repo_dir
else:
repo_dir = _git_clone_and_install_single(repo, branch, parent_dir,
update_requirements=True)
def _git_clone_and_install_subset(yaml_file, whitelist=[], blacklist=[],
update_requirements=False):
"""Clone/install subset of OpenStack repos specified in yaml config file."""
global requirements_dir
installed = []
def _git_validate_projects_yaml(projects, core_project):
"""
Validate the projects yaml.
"""
_git_ensure_key_exists('repositories', projects)
with open(yaml_file, 'r') as fd:
projects = yaml.load(fd)
for proj, val in projects.items():
# The project subset is chosen based on the following 3 rules:
# 1) If project is in blacklist, we don't clone/install it, period.
# 2) If whitelist is empty, we clone/install everything else.
# 3) If whitelist is not empty, we clone/install everything in the
# whitelist.
if proj in blacklist:
continue
if whitelist and proj not in whitelist:
continue
repo = val['repository']
branch = val['branch']
repo_dir = _git_clone_and_install_single(repo, branch,
update_requirements)
if proj == 'requirements':
requirements_dir = repo_dir
installed.append(proj)
return installed
for project in projects['repositories']:
_git_ensure_key_exists('name', project.keys())
_git_ensure_key_exists('repository', project.keys())
_git_ensure_key_exists('branch', project.keys())
if projects['repositories'][0]['name'] != 'requirements':
error_out('{} git repo must be specified first'.format('requirements'))
if projects['repositories'][-1]['name'] != core_project:
error_out('{} git repo must be specified last'.format(core_project))
def _git_clone_and_install_single(repo, branch, update_requirements=False):
"""Clone and install a single git repository."""
dest_parent_dir = "/mnt/openstack-git/"
dest_dir = os.path.join(dest_parent_dir, os.path.basename(repo))
def _git_ensure_key_exists(key, keys):
"""
Ensure that key exists in keys.
"""
if key not in keys:
error_out('openstack-origin-git key \'{}\' is missing'.format(key))
if not os.path.exists(dest_parent_dir):
juju_log('Host dir not mounted at {}. '
'Creating directory there instead.'.format(dest_parent_dir))
os.mkdir(dest_parent_dir)
def _git_clone_and_install_single(repo, branch, parent_dir, update_requirements):
"""
Clone and install a single git repository.
"""
dest_dir = os.path.join(parent_dir, os.path.basename(repo))
if not os.path.exists(parent_dir):
juju_log('Directory already exists at {}. '
'No need to create directory.'.format(parent_dir))
os.mkdir(parent_dir)
if not os.path.exists(dest_dir):
juju_log('Cloning git repo: {}, branch: {}'.format(repo, branch))
repo_dir = install_remote(repo, dest=dest_parent_dir, branch=branch)
repo_dir = install_remote(repo, dest=parent_dir, branch=branch)
else:
repo_dir = dest_dir
@ -561,16 +601,39 @@ def _git_clone_and_install_single(repo, branch, update_requirements=False):
def _git_update_requirements(package_dir, reqs_dir):
"""Update from global requirements.
"""
Update from global requirements.
Update an OpenStack git directory's requirements.txt and
test-requirements.txt from global-requirements.txt."""
Update an OpenStack git directory's requirements.txt and
test-requirements.txt from global-requirements.txt.
"""
orig_dir = os.getcwd()
os.chdir(reqs_dir)
cmd = "python update.py {}".format(package_dir)
cmd = ['python', 'update.py', package_dir]
try:
subprocess.check_call(cmd.split(' '))
subprocess.check_call(cmd)
except subprocess.CalledProcessError:
package = os.path.basename(package_dir)
error_out("Error updating {} from global-requirements.txt".format(package))
os.chdir(orig_dir)
def git_src_dir(projects_yaml, project):
"""
Return the directory where the specified project's source is located.
"""
parent_dir = '/mnt/openstack-git'
if not projects_yaml:
return
projects = yaml.load(projects_yaml)
if 'directory' in projects.keys():
parent_dir = projects['directory']
for p in projects['repositories']:
if p['name'] == project:
return os.path.join(parent_dir, os.path.basename(p['repository']))
return None

4
hooks/charmhelpers/core/services/helpers.py
View File

@ -139,7 +139,7 @@ class MysqlRelation(RelationContext):
def __init__(self, *args, **kwargs):
self.required_keys = ['host', 'user', 'password', 'database']
super(HttpRelation).__init__(self, *args, **kwargs)
RelationContext.__init__(self, *args, **kwargs)
class HttpRelation(RelationContext):
@ -154,7 +154,7 @@ class HttpRelation(RelationContext):
def __init__(self, *args, **kwargs):
self.required_keys = ['host', 'port']
super(HttpRelation).__init__(self, *args, **kwargs)
RelationContext.__init__(self, *args, **kwargs)
def provide_data(self):
return {

2
hooks/charmhelpers/core/unitdata.py
View File

@ -443,7 +443,7 @@ class HookData(object):
data = hookenv.execution_environment()
self.conf = conf_delta = self.kv.delta(data['conf'], 'config')
self.rels = rels_delta = self.kv.delta(data['rels'], 'rels')
self.kv.set('env', data['env'])
self.kv.set('env', dict(data['env']))
self.kv.set('unit', data['unit'])
self.kv.set('relid', data.get('relid'))
return conf_delta, rels_delta

51
hooks/glance_relations.py
View File

@ -1,18 +1,20 @@
#!/usr/bin/python
from subprocess import (
check_call,
call
)
import sys
from subprocess import (
call,
check_call,
)
from glance_utils import (
do_openstack_upgrade,
git_install,
migrate_database,
register_configs,
restart_map,
services,
CLUSTER_RES,
PACKAGES,
determine_packages,
SERVICES,
CHARM,
GLANCE_REGISTRY_CONF,
@ -41,6 +43,7 @@ from charmhelpers.core.hookenv import (
)
from charmhelpers.core.host import (
restart_on_change,
service_reload,
service_stop,
)
from charmhelpers.fetch import (
@ -53,11 +56,13 @@ from charmhelpers.contrib.hahelpers.cluster import (
get_hacluster_config
)
from charmhelpers.contrib.openstack.utils import (
config_value_changed,
configure_installation_source,
get_os_codename_package,
openstack_upgrade_available,
git_install_requested,
lsb_release,
sync_db_with_multi_ipv6_addresses
openstack_upgrade_available,
os_release,
sync_db_with_multi_ipv6_addresses,
)
from charmhelpers.contrib.storage.linux.ceph import (
ensure_ceph_keyring,
@ -101,7 +106,9 @@ def install_hook():
configure_installation_source(src)
apt_update(fatal=True)
apt_install(PACKAGES, fatal=True)
apt_install(determine_packages(), fatal=True)
git_install(config('openstack-origin-git'))
for service in SERVICES:
service_stop(service)
@ -141,7 +148,7 @@ def pgsql_db_joined():
@hooks.hook('shared-db-relation-changed')
@restart_on_change(restart_map())
def db_changed():
rel = get_os_codename_package("glance-common")
rel = os_release('glance-common')
if 'shared-db' not in CONFIGS.complete_contexts():
juju_log('shared-db relation incomplete. Peer not ready?')
@ -164,7 +171,8 @@ def db_changed():
status = call(['glance-manage', 'db_version'])
if status != 0:
juju_log('Setting version_control to 0')
check_call(["glance-manage", "version_control", "0"])
cmd = ["glance-manage", "version_control", "0"]
check_call(cmd)
juju_log('Cluster leader, performing db sync')
migrate_database()
@ -173,7 +181,7 @@ def db_changed():
@hooks.hook('pgsql-db-relation-changed')
@restart_on_change(restart_map())
def pgsql_db_changed():
rel = get_os_codename_package("glance-common")
rel = os_release('glance-common')
if 'pgsql-db' not in CONFIGS.complete_contexts():
juju_log('pgsql-db relation incomplete. Peer not ready?')
@ -189,7 +197,8 @@ def pgsql_db_changed():
status = call(['glance-manage', 'db_version'])
if status != 0:
juju_log('Setting version_control to 0')
check_call(["glance-manage", "version_control", "0"])
cmd = ["glance-manage", "version_control", "0"]
check_call(cmd)
juju_log('Cluster leader, performing db sync')
migrate_database()
@ -316,9 +325,13 @@ def config_changed():
sync_db_with_multi_ipv6_addresses(config('database'),
config('database-user'))
if openstack_upgrade_available('glance-common'):
juju_log('Upgrading OpenStack release')
do_openstack_upgrade(CONFIGS)
if git_install_requested():
if config_value_changed('openstack-origin-git'):
git_install(config('openstack-origin-git'))
else:
if openstack_upgrade_available('glance-common'):
juju_log('Upgrading OpenStack release')
do_openstack_upgrade(CONFIGS)
open_port(9292)
configure_https()
@ -362,7 +375,7 @@ def cluster_changed():
@hooks.hook('upgrade-charm')
@restart_on_change(restart_map(), stopstart=True)
def upgrade_charm():
apt_install(filter_installed_packages(PACKAGES), fatal=True)
apt_install(filter_installed_packages(determine_packages()), fatal=True)
configure_https()
update_nrpe_config()
CONFIGS.write_all()
@ -462,6 +475,10 @@ def configure_https():
cmd = ['a2dissite', 'openstack_https_frontend']
check_call(cmd)
# TODO: improve this by checking if local CN certs are available
# first then checking reload status (see LP #1433114).
service_reload('apache2', restart_on_failure=True)
for r_id in relation_ids('identity-service'):
keystone_joined(relation_id=r_id)
for r_id in relation_ids('image-service'):

154
hooks/glance_utils.py
View File

@ -1,6 +1,7 @@
#!/usr/bin/python
import os
import shutil
import subprocess
import glance_contexts
@ -14,21 +15,27 @@ from charmhelpers.fetch import (
add_source)
from charmhelpers.core.hookenv import (
charm_dir,
config,
log,
relation_ids,
service_name)
from charmhelpers.core.host import (
adduser,
add_group,
add_user_to_group,
mkdir,
service_stop,
service_start,
lsb_release
service_restart,
lsb_release,
write_file,
)
from charmhelpers.contrib.openstack import (
templating,
context, )
context,)
from charmhelpers.contrib.hahelpers.cluster import (
eligible_leader,
@ -37,8 +44,14 @@ from charmhelpers.contrib.hahelpers.cluster import (
from charmhelpers.contrib.openstack.alternatives import install_alternative
from charmhelpers.contrib.openstack.utils import (
get_os_codename_install_source,
get_os_codename_package,
configure_installation_source)
git_install_requested,
git_clone_and_install,
git_src_dir,
configure_installation_source,
os_release,
)
from charmhelpers.core.templating import render
CLUSTER_RES = "grp_glance_vips"
@ -46,8 +59,27 @@ PACKAGES = [
"apache2", "glance", "python-mysqldb", "python-swiftclient",
"python-psycopg2", "python-keystone", "python-six", "uuid", "haproxy", ]
BASE_GIT_PACKAGES = [
'libxml2-dev',
'libxslt1-dev',
'python-dev',
'python-pip',
'python-setuptools',
'zlib1g-dev',
]
SERVICES = [
"glance-api", "glance-registry", ]
"glance-api",
"glance-registry",
]
# ubuntu packages that should not be installed when deploying from git
GIT_PACKAGE_BLACKLIST = [
'glance',
'python-swiftclient',
'python-keystone',
]
CHARM = "glance"
@ -140,7 +172,7 @@ def register_configs():
# Register config files with their respective contexts.
# Regstration of some configs may not be required depending on
# existing of certain relations.
release = get_os_codename_package('glance-common', fatal=False) or 'essex'
release = os_release('glance-common')
configs = templating.OSConfigRenderer(templates_dir=TEMPLATES,
openstack_release=release)
@ -177,6 +209,18 @@ def register_configs():
return configs
def determine_packages():
packages = [] + PACKAGES
if git_install_requested():
packages.extend(BASE_GIT_PACKAGES)
# don't include packages that will be installed from git
for p in GIT_PACKAGE_BLACKLIST:
packages.remove(p)
return list(set(packages))
def migrate_database():
'''Runs glance-manage to initialize a new database
or migrate existing
@ -205,7 +249,7 @@ def do_openstack_upgrade(configs):
]
apt_update()
apt_upgrade(options=dpkg_opts, fatal=True, dist=True)
apt_install(PACKAGES, fatal=True)
apt_install(determine_packages(), fatal=True)
# set CONFIGS to load templates from new release and regenerate config
configs.set_release(openstack_release=new_os_rel)
@ -256,3 +300,99 @@ def setup_ipv6():
' main')
apt_update()
apt_install('haproxy/trusty-backports', fatal=True)
def git_install(projects_yaml):
"""Perform setup, and install git repos specified in yaml parameter."""
if git_install_requested():
git_pre_install()
git_clone_and_install(projects_yaml, core_project='glance')
git_post_install(projects_yaml)
def git_pre_install():
"""Perform glance pre-install setup."""
dirs = [
'/var/lib/glance',
'/var/lib/glance/images',
'/var/lib/glance/image-cache',
'/var/lib/glance/image-cache/incomplete',
'/var/lib/glance/image-cache/invalid',
'/var/lib/glance/image-cache/queue',
'/var/log/glance',
'/etc/glance',
]
logs = [
'/var/log/glance/glance-api.log',
'/var/log/glance/glance-registry.log',
]
adduser('glance', shell='/bin/bash', system_user=True)
add_group('glance', system_group=True)
add_user_to_group('glance', 'glance')
for d in dirs:
mkdir(d, owner='glance', group='glance', perms=0700, force=False)
for l in logs:
write_file(l, '', owner='glance', group='glance', perms=0600)
def git_post_install(projects_yaml):
"""Perform glance post-install setup."""
src_etc = os.path.join(git_src_dir(projects_yaml, 'glance'), 'etc')
configs = {
'glance-cache': {
'src': os.path.join(src_etc, 'glance-cache.conf'),
'dest': '/etc/glance/glance-cache.conf',
},
'glance-scrubber': {
'src': os.path.join(src_etc, 'glance-scrubber.conf'),
'dest': '/etc/glance/glance-scrubber.conf',
},
'policy': {
'src': os.path.join(src_etc, 'policy.json'),
'dest': '/etc/glance/policy.json',
},
'schema-image': {
'src': os.path.join(src_etc, 'schema-image.json'),
'dest': '/etc/glance/schema-image.json',
},
}
for conf, files in configs.iteritems():
shutil.copyfile(files['src'], files['dest'])
glance_api_context = {
'service_description': 'Glance API server',
'service_name': 'Glance',
'user_name': 'glance',
'start_dir': '/var/lib/glance',
'process_name': 'glance-api',
'executable_name': '/usr/local/bin/glance-api',
'config_file': '/etc/glance/glance-api.conf',
'log_file': '/var/log/glance/api.log',
}
glance_registry_context = {
'service_description': 'Glance registry server',
'service_name': 'Glance',
'user_name': 'glance',
'start_dir': '/var/lib/glance',
'process_name': 'glance-registry',
'executable_name': '/usr/local/bin/glance-registry',
'config_file': '/etc/glance/glance-registry.conf',
'log_file': '/var/log/glance/registry.log',
}
# NOTE(coreycb): Needs systemd support
templates_dir = 'hooks/charmhelpers/contrib/openstack/templates'
templates_dir = os.path.join(charm_dir(), templates_dir)
render('git.upstart', '/etc/init/glance-api.conf',
glance_api_context, perms=0o644, templates_dir=templates_dir)
render('git.upstart', '/etc/init/glance-registry.conf',
glance_registry_context, perms=0o644, templates_dir=templates_dir)
service_restart('glance-api')
service_restart('glance-registry')

77
templates/kilo/glance-api-paste.ini Normal file
View File

@ -0,0 +1,77 @@
# Use this pipeline for no auth or image caching - DEFAULT
[pipeline:glance-api]
pipeline = versionnegotiation osprofiler unauthenticated-context rootapp
# Use this pipeline for image caching and no auth
[pipeline:glance-api-caching]
pipeline = versionnegotiation osprofiler unauthenticated-context cache rootapp
# Use this pipeline for caching w/ management interface but no auth
[pipeline:glance-api-cachemanagement]
pipeline = versionnegotiation osprofiler unauthenticated-context cache cachemanage rootapp
# Use this pipeline for keystone auth
[pipeline:glance-api-keystone]
pipeline = versionnegotiation osprofiler authtoken context rootapp
# Use this pipeline for keystone auth with image caching
[pipeline:glance-api-keystone+caching]
pipeline = versionnegotiation osprofiler authtoken context cache rootapp
# Use this pipeline for keystone auth with caching and cache management
[pipeline:glance-api-keystone+cachemanagement]
pipeline = versionnegotiation osprofiler authtoken context cache cachemanage rootapp
# Use this pipeline for authZ only. This means that the registry will treat a
# user as authenticated without making requests to keystone to reauthenticate
# the user.
[pipeline:glance-api-trusted-auth]
pipeline = versionnegotiation osprofiler context rootapp
# Use this pipeline for authZ only. This means that the registry will treat a
# user as authenticated without making requests to keystone to reauthenticate
# the user and uses cache management
[pipeline:glance-api-trusted-auth+cachemanagement]
pipeline = versionnegotiation osprofiler context cache cachemanage rootapp
[composite:rootapp]
paste.composite_factory = glance.api:root_app_factory
/: apiversions
/v1: apiv1app
/v2: apiv2app
[app:apiversions]
paste.app_factory = glance.api.versions:create_resource
[app:apiv1app]
paste.app_factory = glance.api.v1.router:API.factory
[app:apiv2app]
paste.app_factory = glance.api.v2.router:API.factory
[filter:versionnegotiation]
paste.filter_factory = glance.api.middleware.version_negotiation:VersionNegotiationFilter.factory
[filter:cache]
paste.filter_factory = glance.api.middleware.cache:CacheFilter.factory
[filter:cachemanage]
paste.filter_factory = glance.api.middleware.cache_manage:CacheManageFilter.factory
[filter:context]
paste.filter_factory = glance.api.middleware.context:ContextMiddleware.factory
[filter:unauthenticated-context]
paste.filter_factory = glance.api.middleware.context:UnauthenticatedContextMiddleware.factory
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
delay_auth_decision = true
[filter:gzip]
paste.filter_factory = glance.api.middleware.gzip:GzipMiddleware.factory
[filter:osprofiler]
paste.filter_factory = osprofiler.web:WsgiMiddleware.factory
hmac_keys = SECRET_KEY
enabled = yes

83
templates/kilo/glance-api.conf Normal file
View File

@ -0,0 +1,83 @@
[DEFAULT]
verbose = {{ verbose }}
use_syslog = {{ use_syslog }}
debug = {{ debug }}
workers = {{ workers }}
known_stores = {{ known_stores }}
{% if rbd_pool -%}
default_store = rbd
{% elif swift_store -%}
default_store = swift
{% else -%}
default_store = file
{% endif -%}
bind_host = {{ bind_host }}
{% if ext -%}
bind_port = {{ ext }}
{% elif bind_port -%}
bind_port = {{ bind_port }}
{% else -%}
bind_port = 9292
{% endif -%}
log_file = /var/log/glance/api.log
backlog = 4096
registry_host = {{ registry_host }}
registry_port = 9191
registry_client_protocol = http
{% if api_config_flags -%}
{% for key, value in api_config_flags.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
{% endif -%}
{% if rabbitmq_host or rabbitmq_hosts -%}
notification_driver = rabbit
{% endif -%}
{% if swift_store -%}
swift_store_auth_version = 2
swift_store_auth_address = {{ service_protocol }}://{{ service_host }}:{{ service_port }}/v2.0/
swift_store_user = {{ admin_tenant_name }}:{{ admin_user }}
swift_store_key = {{ admin_password }}
swift_store_create_container_on_put = True
swift_store_container = glance
swift_store_large_object_size = 5120
swift_store_large_object_chunk_size = 200
swift_enable_snet = False
{% endif -%}
{% if rbd_pool -%}
rbd_store_ceph_conf = /etc/ceph/ceph.conf
rbd_store_user = {{ rbd_user }}
rbd_store_pool = {{ rbd_pool }}
rbd_store_chunk_size = 8
{% endif -%}
delayed_delete = False
scrub_time = 43200
scrubber_datadir = /var/lib/glance/scrubber
image_cache_dir = /var/lib/glance/image-cache/
db_enforce_mysql_charset = False
[glance_store]
filesystem_store_datadir = /var/lib/glance/images/
[image_format]
disk_formats=ami,ari,aki,vhd,vmdk,raw,qcow2,vdi,iso,root-tar
{% include "section-keystone-authtoken" %}
{% if auth_host -%}
[paste_deploy]
flavor = keystone
{% endif %}
{% include "parts/section-database" %}
{% include "section-rabbitmq-oslo" %}

30
templates/kilo/glance-registry-paste.ini Normal file
View File

@ -0,0 +1,30 @@
# Use this pipeline for no auth - DEFAULT
[pipeline:glance-registry]
pipeline = osprofiler unauthenticated-context registryapp
# Use this pipeline for keystone auth
[pipeline:glance-registry-keystone]
pipeline = osprofiler authtoken context registryapp
# Use this pipeline for authZ only. This means that the registry will treat a
# user as authenticated without making requests to keystone to reauthenticate
# the user.
[pipeline:glance-registry-trusted-auth]
pipeline = osprofiler context registryapp
[app:registryapp]
paste.app_factory = glance.registry.api:API.factory
[filter:context]
paste.filter_factory = glance.api.middleware.context:ContextMiddleware.factory
[filter:unauthenticated-context]
paste.filter_factory = glance.api.middleware.context:UnauthenticatedContextMiddleware.factory
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
[filter:osprofiler]
paste.filter_factory = osprofiler.web:WsgiMiddleware.factory
hmac_keys = SECRET_KEY
enabled = yes

27
templates/kilo/glance-registry.conf Normal file
View File

@ -0,0 +1,27 @@
[DEFAULT]
verbose = {{ verbose }}
use_syslog = {{ use_syslog }}
debug = {{ debug }}
workers = {{ workers }}
bind_host = {{ bind_host }}
bind_port = 9191
log_file = /var/log/glance/registry.log
backlog = 4096
api_limit_max = 1000
limit_param_default = 25
{% if registry_config_flags -%}
{% for key, value in registry_config_flags.iteritems() -%}
{{ key }} = {{ value }}
{% endfor -%}
{% endif -%}
{% include "section-keystone-authtoken" %}
{% if auth_host -%}
[paste_deploy]
flavor = keystone
{% endif %}
{% include "parts/section-database" %}

1
templates/parts/section-database
View File

@ -1,4 +1,5 @@
{% if database_host -%}
[database]
connection = {{ database_type }}://{{ database_user }}:{{ database_password }}@{{ database_host }}/{{ database }}{% if database_ssl_ca %}?ssl_ca={{ database_ssl_ca }}{% if database_ssl_cert %}&ssl_cert={{ database_ssl_cert }}&ssl_key={{ database_ssl_key }}{% endif %}{% endif %}
idle_timeout = 3600
{% endif -%}

9
tests/10-basic-precise-essex
View File

@ -1,9 +0,0 @@
#!/usr/bin/python
"""Amulet tests on a basic glance deployment on precise-essex."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='precise')
deployment.run_tests()

11
tests/11-basic-precise-folsom
View File

@ -1,11 +0,0 @@
#!/usr/bin/python
"""Amulet tests on a basic glance deployment on precise-folsom."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='precise',
openstack='cloud:precise-folsom',
source='cloud:precise-updates/folsom')
deployment.run_tests()

11
tests/12-basic-precise-grizzly
View File

@ -1,11 +0,0 @@
#!/usr/bin/python
"""Amulet tests on a basic glance deployment on precise-grizzly."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='precise',
openstack='cloud:precise-grizzly',
source='cloud:precise-updates/grizzly')
deployment.run_tests()

11
tests/13-basic-precise-havana
View File

@ -1,11 +0,0 @@
#!/usr/bin/python
"""Amulet tests on a basic glance deployment on precise-havana."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='precise',
openstack='cloud:precise-havana',
source='cloud:precise-updates/havana')
deployment.run_tests()

9
tests/16-basic-trusty-icehouse-git Executable file
View File

@ -0,0 +1,9 @@
#!/usr/bin/python
"""Amulet tests on a basic Glance git deployment on trusty-icehouse."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='trusty', git=True)
deployment.run_tests()

11
tests/17-basic-trusty-juno Executable file
View File

@ -0,0 +1,11 @@
#!/usr/bin/python
"""Amulet tests on a basic Glance deployment on trusty-juno."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='trusty',
openstack='cloud:trusty-juno',
source='cloud:trusty-updates/juno')
deployment.run_tests()

12
tests/18-basic-trusty-juno-git Executable file
View File

@ -0,0 +1,12 @@
#!/usr/bin/python
"""Amulet tests on a basic Glance git deployment on trusty-juno."""
from basic_deployment import GlanceBasicDeployment
if __name__ == '__main__':
deployment = GlanceBasicDeployment(series='trusty',
openstack='cloud:trusty-juno',
source='cloud:trusty-updates/juno',
git=True)
deployment.run_tests()

28
tests/basic_deployment.py
View File

@ -1,6 +1,7 @@
#!/usr/bin/python
import amulet
import yaml
from charmhelpers.contrib.openstack.amulet.deployment import (
OpenStackAmuletDeployment
@ -23,9 +24,11 @@ class GlanceBasicDeployment(OpenStackAmuletDeployment):
# * Add tests with different storage back ends
# * Resolve Essex->Havana juju set charm bug
def __init__(self, series=None, openstack=None, source=None, stable=False):
def __init__(self, series=None, openstack=None, source=None, git=False,
stable=False):
'''Deploy the entire test environment.'''
super(GlanceBasicDeployment, self).__init__(series, openstack, source, stable)
self.git = git
self._add_services()
self._add_relations()
self._configure_services()
@ -55,11 +58,29 @@ class GlanceBasicDeployment(OpenStackAmuletDeployment):
def _configure_services(self):
'''Configure all of the services.'''
glance_config = {}
if self.git:
branch = 'stable/' + self._get_openstack_release_string()
openstack_origin_git = {
'repositories': [
{'name': 'requirements',
'repository': 'git://git.openstack.org/openstack/requirements',
'branch': branch},
{'name': 'glance',
'repository': 'git://git.openstack.org/openstack/glance',
'branch': branch},
],
'directory': '/mnt/openstack-git',
'http_proxy': 'http://squid.internal:3128',
'https_proxy': 'https://squid.internal:3128',
}
glance_config['openstack-origin-git'] = yaml.dump(openstack_origin_git)
keystone_config = {'admin-password': 'openstack',
'admin-token': 'ubuntutesting'}
mysql_config = {'dataset-size': '50%'}
configs = {'keystone': keystone_config,
configs = {'glance': glance_config,
'keystone': keystone_config,
'mysql': mysql_config}
super(GlanceBasicDeployment, self)._configure_services(configs)
@ -160,7 +181,6 @@ class GlanceBasicDeployment(OpenStackAmuletDeployment):
'auth_port': '35357',
'auth_protocol': 'http',
'private-address': u.valid_ip,
'https_keystone': 'False',
'auth_host': u.valid_ip,
'service_username': 'glance',
'service_tenant_id': u.not_null,

27
tests/charmhelpers/contrib/openstack/amulet/deployment.py
View File

@ -15,6 +15,7 @@
# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
import six
from collections import OrderedDict
from charmhelpers.contrib.amulet.deployment import (
AmuletDeployment
)
@ -100,12 +101,34 @@ class OpenStackAmuletDeployment(AmuletDeployment):
"""
(self.precise_essex, self.precise_folsom, self.precise_grizzly,
self.precise_havana, self.precise_icehouse,
self.trusty_icehouse) = range(6)
self.trusty_icehouse, self.trusty_juno, self.trusty_kilo) = range(8)
releases = {
('precise', None): self.precise_essex,
('precise', 'cloud:precise-folsom'): self.precise_folsom,
('precise', 'cloud:precise-grizzly'): self.precise_grizzly,
('precise', 'cloud:precise-havana'): self.precise_havana,
('precise', 'cloud:precise-icehouse'): self.precise_icehouse,
('trusty', None): self.trusty_icehouse}
('trusty', None): self.trusty_icehouse,
('trusty', 'cloud:trusty-juno'): self.trusty_juno,
('trusty', 'cloud:trusty-kilo'): self.trusty_kilo}
return releases[(self.series, self.openstack)]
def _get_openstack_release_string(self):
"""Get openstack release string.
Return a string representing the openstack release.
"""
releases = OrderedDict([
('precise', 'essex'),
('quantal', 'folsom'),
('raring', 'grizzly'),
('saucy', 'havana'),
('trusty', 'icehouse'),
('utopic', 'juno'),
('vivid', 'kilo'),
])
if self.openstack:
os_origin = self.openstack.split(':')[1]
return os_origin.split('%s-' % self.series)[1].split('/')[0]
else:
return releases[self.series]

1
unit_tests/__init__.py
View File

@ -1,3 +1,4 @@
import sys
sys.path.append('actions/')
sys.path.append('hooks/')

87
unit_tests/test_actions_git_reinstall.py Normal file
View File

@ -0,0 +1,87 @@
from mock import patch
import os
os.environ['JUJU_UNIT_NAME'] = 'glance'
import git_reinstall
from test_utils import (
CharmTestCase
)
TO_PATCH = [
'config',
]
openstack_origin_git = \
"""repositories:
- {name: requirements,
repository: 'git://git.openstack.org/openstack/requirements',
branch: stable/juno}
- {name: glance,
repository: 'git://git.openstack.org/openstack/glance',
branch: stable/juno}"""
class TestGlanceActions(CharmTestCase):
def setUp(self):
super(TestGlanceActions, self).setUp(git_reinstall, TO_PATCH)
self.config.side_effect = self.test_config.get
@patch.object(git_reinstall, 'action_set')
@patch.object(git_reinstall, 'action_fail')
@patch.object(git_reinstall, 'git_install')
@patch('charmhelpers.contrib.openstack.utils.config')
def test_git_reinstall(self, _config, git_install, action_fail,
action_set):
_config.return_value = openstack_origin_git
self.test_config.set('openstack-origin-git', openstack_origin_git)
git_reinstall.git_reinstall()
git_install.assert_called_with(openstack_origin_git)
self.assertTrue(git_install.called)
self.assertFalse(action_set.called)
self.assertFalse(action_fail.called)
@patch.object(git_reinstall, 'action_set')
@patch.object(git_reinstall, 'action_fail')
@patch.object(git_reinstall, 'git_install')
@patch('charmhelpers.contrib.openstack.utils.config')
def test_git_reinstall_not_configured(self, _config, git_install,
action_fail, action_set):
_config.return_value = None
git_reinstall.git_reinstall()
msg = 'openstack-origin-git is not configured'
action_fail.assert_called_with(msg)
self.assertFalse(git_install.called)
self.assertFalse(action_set.called)
@patch.object(git_reinstall, 'action_set')
@patch.object(git_reinstall, 'action_fail')
@patch.object(git_reinstall, 'git_install')
@patch('charmhelpers.contrib.openstack.utils.config')
def test_git_reinstall_exception(self, _config, git_install, action_fail,
action_set):
_config.return_value = openstack_origin_git
e = OSError('something bad happened')
git_install.side_effect = e
traceback = (
"Traceback (most recent call last):\n"
" File \"actions/git_reinstall.py\", line 33, in git_reinstall\n"
" git_install(config(\'openstack-origin-git\'))\n"
" File \"/usr/lib/python2.7/dist-packages/mock.py\", line 964, in __call__\n" # noqa
" return _mock_self._mock_call(*args, **kwargs)\n"
" File \"/usr/lib/python2.7/dist-packages/mock.py\", line 1019, in _mock_call\n" # noqa
" raise effect\n"
"OSError: something bad happened\n")
git_reinstall.git_reinstall()
msg = 'git-reinstall resulted in an unexpected error'
action_fail.assert_called_with(msg)
action_set.assert_called_with({'traceback': traceback})

131
unit_tests/test_glance_relations.py
View File

@ -1,6 +1,7 @@
from mock import call, patch, MagicMock
import json
import os
import yaml
from test_utils import CharmTestCase
@ -38,10 +39,11 @@ TO_PATCH = [
'apt_install',
'apt_update',
'restart_on_change',
'service_reload',
'service_stop',
# charmhelpers.contrib.openstack.utils
'configure_installation_source',
'get_os_codename_package',
'os_release',
'openstack_upgrade_available',
# charmhelpers.contrib.hahelpers.cluster_utils
'eligible_leader',
@ -52,6 +54,7 @@ TO_PATCH = [
'migrate_database',
'ensure_ceph_keyring',
'ceph_config_file',
'git_install',
'update_nrpe_config',
# other
'call',
@ -74,23 +77,26 @@ class GlanceRelationTests(CharmTestCase):
super(GlanceRelationTests, self).setUp(relations, TO_PATCH)
self.config.side_effect = self.test_config.get
def test_install_hook(self):
@patch.object(utils, 'git_install_requested')
def test_install_hook(self, git_requested):
git_requested.return_value = False
repo = 'cloud:precise-grizzly'
self.test_config.set('openstack-origin', repo)
self.service_stop.return_value = True
relations.install_hook()
self.configure_installation_source.assert_called_with(repo)
self.apt_update.assert_called_with(fatal=True)
self.apt_install.assert_called_with(['apache2', 'glance',
'python-mysqldb',
'python-swiftclient',
'python-psycopg2',
self.apt_install.assert_called_with(['haproxy', 'python-six', 'uuid',
'python-mysqldb', 'apache2',
'python-psycopg2', 'glance',
'python-keystone',
'python-six',
'uuid', 'haproxy'], fatal=True)
'python-swiftclient'], fatal=True)
self.assertTrue(self.execd_preinstall.called)
self.git_install.assert_called_with(None)
def test_install_hook_precise_distro(self):
@patch.object(utils, 'git_install_requested')
def test_install_hook_precise_distro(self, git_requested):
git_requested.return_value = False
self.test_config.set('openstack-origin', 'distro')
self.lsb_release.return_value = {'DISTRIB_RELEASE': 12.04,
'DISTRIB_CODENAME': 'precise'}
@ -100,6 +106,37 @@ class GlanceRelationTests(CharmTestCase):
"cloud:precise-folsom"
)
@patch.object(utils, 'git_install_requested')
def test_install_hook_git(self, git_requested):
git_requested.return_value = True
repo = 'cloud:trusty-juno'
openstack_origin_git = {
'repositories': [
{'name': 'requirements',
'repository': 'git://git.openstack.org/openstack/requirements', # noqa
'branch': 'stable/juno'},
{'name': 'glance',
'repository': 'git://git.openstack.org/openstack/glance',
'branch': 'stable/juno'}
],
'directory': '/mnt/openstack-git',
}
projects_yaml = yaml.dump(openstack_origin_git)
self.test_config.set('openstack-origin', repo)
self.test_config.set('openstack-origin-git', projects_yaml)
relations.install_hook()
self.assertTrue(self.execd_preinstall.called)
self.configure_installation_source.assert_called_with(repo)
self.apt_update.assert_called_with(fatal=True)
self.apt_install.assert_called_with(['haproxy', 'python-setuptools',
'python-six', 'uuid',
'python-mysqldb', 'python-pip',
'apache2', 'libxslt1-dev',
'python-psycopg2', 'zlib1g-dev',
'python-dev', 'libxml2-dev'],
fatal=True)
self.git_install.assert_called_with(projects_yaml)
def test_db_joined(self):
self.unit_get.return_value = 'glance.foohost.com'
self.is_relation_made.return_value = False
@ -216,7 +253,7 @@ class GlanceRelationTests(CharmTestCase):
@patch.object(relations, 'CONFIGS')
def test_db_changed_with_essex_not_setting_version_control(self, configs):
self.get_os_codename_package.return_value = "essex"
self.os_release.return_value = "essex"
self.call.return_value = 0
self._shared_db_test(configs, 'glance/0')
self.assertEquals([call('/etc/glance/glance-registry.conf')],
@ -229,7 +266,7 @@ class GlanceRelationTests(CharmTestCase):
@patch.object(relations, 'CONFIGS')
def test_postgresql_db_changed_with_essex_not_setting_version_control(
self, configs):
self.get_os_codename_package.return_value = "essex"
self.os_release.return_value = "essex"
self.call.return_value = 0
self._postgresql_db_test(configs)
self.assertEquals([call('/etc/glance/glance-registry.conf')],
@ -241,7 +278,7 @@ class GlanceRelationTests(CharmTestCase):
@patch.object(relations, 'CONFIGS')
def test_db_changed_with_essex_setting_version_control(self, configs):
self.get_os_codename_package.return_value = "essex"
self.os_release.return_value = "essex"
self.call.return_value = 1
self._shared_db_test(configs, 'glance/0')
self.assertEquals([call('/etc/glance/glance-registry.conf')],
@ -257,7 +294,7 @@ class GlanceRelationTests(CharmTestCase):
@patch.object(relations, 'CONFIGS')
def test_postgresql_db_changed_with_essex_setting_version_control(
self, configs):
self.get_os_codename_package.return_value = "essex"
self.os_release.return_value = "essex"
self.call.return_value = 1
self._postgresql_db_test(configs)
self.assertEquals([call('/etc/glance/glance-registry.conf')],
@ -441,8 +478,12 @@ class GlanceRelationTests(CharmTestCase):
@patch.object(relations, 'configure_https')
@patch.object(relations, 'object_store_joined')
@patch.object(relations, 'CONFIGS')
def test_keystone_changed_with_object_store_relation(
self, configs, object_store_joined, configure_https):
@patch.object(utils, 'git_install_requested')
def test_keystone_changed_with_object_store_relation(self, git_requested,
configs,
object_store_joined,
configure_https):
git_requested.return_value = False
configs.complete_contexts = MagicMock()
configs.complete_contexts.return_value = ['identity-service']
configs.write = MagicMock()
@ -457,14 +498,20 @@ class GlanceRelationTests(CharmTestCase):
self.assertTrue(configure_https.called)
@patch.object(relations, 'configure_https')
def test_config_changed_no_openstack_upgrade(self, configure_https):
@patch.object(relations, 'git_install_requested')
def test_config_changed_no_openstack_upgrade(self, git_requested,
configure_https):
git_requested.return_value = False
self.openstack_upgrade_available.return_value = False
relations.config_changed()
self.open_port.assert_called_with(9292)
self.assertTrue(configure_https.called)
@patch.object(relations, 'configure_https')
def test_config_changed_with_openstack_upgrade(self, configure_https):
@patch.object(relations, 'git_install_requested')
def test_config_changed_with_openstack_upgrade(self, git_requested,
configure_https):
git_requested.return_value = False
self.openstack_upgrade_available.return_value = True
relations.config_changed()
self.juju_log.assert_called_with(
@ -473,6 +520,32 @@ class GlanceRelationTests(CharmTestCase):
self.assertTrue(self.do_openstack_upgrade.called)
self.assertTrue(configure_https.called)
@patch.object(relations, 'configure_https')
@patch.object(relations, 'git_install_requested')
@patch.object(relations, 'config_value_changed')
def test_config_changed_git_updated(self, config_val_changed,
git_requested, configure_https):
git_requested.return_value = True
repo = 'cloud:trusty-juno'
openstack_origin_git = {
'repositories': [
{'name': 'requirements',
'repository': 'git://git.openstack.org/openstack/requirements', # noqa
'branch': 'stable/juno'},
{'name': 'glance',
'repository': 'git://git.openstack.org/openstack/glance',
'branch': 'stable/juno'}
],
'directory': '/mnt/openstack-git',
}
projects_yaml = yaml.dump(openstack_origin_git)
self.test_config.set('openstack-origin', repo)
self.test_config.set('openstack-origin-git', projects_yaml)
relations.config_changed()
self.git_install.assert_called_with(projects_yaml)
self.assertFalse(self.do_openstack_upgrade.called)
self.assertTrue(configure_https.called)
@patch.object(relations, 'CONFIGS')
def test_cluster_changed(self, configs):
self.test_config.set('prefer-ipv6', False)
@ -499,7 +572,9 @@ class GlanceRelationTests(CharmTestCase):
configs.write.call_args_list)
@patch.object(relations, 'CONFIGS')
def test_upgrade_charm(self, configs):
@patch.object(utils, 'git_install_requested')
def test_upgrade_charm(self, git_requested, configs):
git_requested.return_value = False
self.filter_installed_packages.return_value = ['test']
relations.upgrade_charm()
self.apt_install.assert_called_with(['test'], fatal=True)
@ -604,8 +679,9 @@ class GlanceRelationTests(CharmTestCase):
configs.write = MagicMock()
self.relation_ids.return_value = ['identity-service:0']
relations.configure_https()
cmd = ['a2ensite', 'openstack_https_frontend']
self.check_call.assert_called_with(cmd)
calls = [call('a2dissite', 'openstack_https_frontend'),
call('service', 'apache2', 'reload')]
self.check_call.assert_called_has_calls(calls)
keystone_joined.assert_called_with(relation_id='identity-service:0')
@patch.object(relations, 'keystone_joined')
@ -617,8 +693,9 @@ class GlanceRelationTests(CharmTestCase):
configs.write = MagicMock()
self.relation_ids.return_value = ['identity-service:0']
relations.configure_https()
cmd = ['a2dissite', 'openstack_https_frontend']
self.check_call.assert_called_with(cmd)
calls = [call('a2dissite', 'openstack_https_frontend'),
call('service', 'apache2', 'reload')]
self.check_call.assert_called_has_calls(calls)
keystone_joined.assert_called_with(relation_id='identity-service:0')
@patch.object(relations, 'image_service_joined')
@ -630,8 +707,9 @@ class GlanceRelationTests(CharmTestCase):
configs.write = MagicMock()
self.relation_ids.return_value = ['image-service:0']
relations.configure_https()
cmd = ['a2ensite', 'openstack_https_frontend']
self.check_call.assert_called_with(cmd)
calls = [call('a2dissite', 'openstack_https_frontend'),
call('service', 'apache2', 'reload')]
self.check_call.assert_called_has_calls(calls)
image_service_joined.assert_called_with(relation_id='image-service:0')
@patch.object(relations, 'image_service_joined')
@ -643,8 +721,9 @@ class GlanceRelationTests(CharmTestCase):
configs.write = MagicMock()
self.relation_ids.return_value = ['image-service:0']
relations.configure_https()
cmd = ['a2dissite', 'openstack_https_frontend']
self.check_call.assert_called_with(cmd)
calls = [call('a2dissite', 'openstack_https_frontend'),
call('service', 'apache2', 'reload')]
self.check_call.assert_called_has_calls(calls)
image_service_joined.assert_called_with(relation_id='image-service:0')
def test_amqp_joined(self):

152
unit_tests/test_glance_utils.py
View File

@ -14,7 +14,6 @@ TO_PATCH = [
'config',
'log',
'relation_ids',
'get_os_codename_package',
'get_os_codename_install_source',
'configure_installation_source',
'eligible_leader',
@ -23,6 +22,7 @@ TO_PATCH = [
'apt_upgrade',
'apt_install',
'mkdir',
'os_release',
'service_start',
'service_stop',
'service_name',
@ -34,6 +34,15 @@ DPKG_OPTS = [
'--option', 'Dpkg::Options::=--force-confdef',
]
openstack_origin_git = \
"""repositories:
- {name: requirements,
repository: 'git://git.openstack.org/openstack/requirements',
branch: stable/juno}
- {name: glance,
repository: 'git://git.openstack.org/openstack/glance',
branch: stable/juno}"""
class TestGlanceUtils(CharmTestCase):
@ -50,7 +59,7 @@ class TestGlanceUtils(CharmTestCase):
@patch('os.path.exists')
def test_register_configs_apache(self, exists):
exists.return_value = False
self.get_os_codename_package.return_value = 'grizzly'
self.os_release.return_value = 'grizzly'
self.relation_ids.return_value = False
configs = utils.register_configs()
calls = []
@ -69,7 +78,7 @@ class TestGlanceUtils(CharmTestCase):
@patch('os.path.exists')
def test_register_configs_apache24(self, exists):
exists.return_value = True
self.get_os_codename_package.return_value = 'grizzly'
self.os_release.return_value = 'grizzly'
self.relation_ids.return_value = False
configs = utils.register_configs()
calls = []
@ -88,7 +97,7 @@ class TestGlanceUtils(CharmTestCase):
@patch('os.path.exists')
def test_register_configs_ceph(self, exists):
exists.return_value = True
self.get_os_codename_package.return_value = 'grizzly'
self.os_release.return_value = 'grizzly'
self.relation_ids.return_value = ['ceph:0']
self.service_name.return_value = 'glance'
configs = utils.register_configs()
@ -121,8 +130,26 @@ class TestGlanceUtils(CharmTestCase):
])
self.assertEquals(ex_map, utils.restart_map())
@patch('charmhelpers.contrib.openstack.utils.config')
def test_determine_packages(self, _config):
_config.return_value = None
result = utils.determine_packages()
ex = utils.PACKAGES
self.assertEquals(set(ex), set(result))
@patch('charmhelpers.contrib.openstack.utils.config')
def test_determine_packages_git(self, _config):
_config.return_value = openstack_origin_git
result = utils.determine_packages()
ex = utils.PACKAGES + utils.BASE_GIT_PACKAGES
for p in utils.GIT_PACKAGE_BLACKLIST:
ex.remove(p)
self.assertEquals(set(ex), set(result))
@patch.object(utils, 'migrate_database')
def test_openstack_upgrade_leader(self, migrate):
@patch.object(utils, 'git_install_requested')
def test_openstack_upgrade_leader(self, git_requested, migrate):
git_requested.return_value = True
self.config.side_effect = None
self.config.return_value = 'cloud:precise-havana'
self.eligible_leader.return_value = True
@ -130,14 +157,17 @@ class TestGlanceUtils(CharmTestCase):
configs = MagicMock()
utils.do_openstack_upgrade(configs)
self.assertTrue(configs.write_all.called)
self.apt_install.assert_called_with(utils.PACKAGES, fatal=True)
self.apt_install.assert_called_with(utils.determine_packages(),
fatal=True)
self.apt_upgrade.assert_called_with(options=DPKG_OPTS,
fatal=True, dist=True)
configs.set_release.assert_called_with(openstack_release='havana')
self.assertTrue(migrate.called)
@patch.object(utils, 'migrate_database')
def test_openstack_upgrade_not_leader(self, migrate):
@patch.object(utils, 'git_install_requested')
def test_openstack_upgrade_not_leader(self, git_requested, migrate):
git_requested.return_value = True
self.config.side_effect = None
self.config.return_value = 'cloud:precise-havana'
self.eligible_leader.return_value = False
@ -145,8 +175,114 @@ class TestGlanceUtils(CharmTestCase):
configs = MagicMock()
utils.do_openstack_upgrade(configs)
self.assertTrue(configs.write_all.called)
self.apt_install.assert_called_with(utils.PACKAGES, fatal=True)
self.apt_install.assert_called_with(utils.determine_packages(),
fatal=True)
self.apt_upgrade.assert_called_with(options=DPKG_OPTS,
fatal=True, dist=True)
configs.set_release.assert_called_with(openstack_release='havana')
self.assertFalse(migrate.called)
@patch.object(utils, 'git_install_requested')
@patch.object(utils, 'git_clone_and_install')
@patch.object(utils, 'git_post_install')
@patch.object(utils, 'git_pre_install')
def test_git_install(self, git_pre, git_post, git_clone_and_install,
git_requested):
projects_yaml = openstack_origin_git
git_requested.return_value = True
utils.git_install(projects_yaml)
self.assertTrue(git_pre.called)
git_clone_and_install.assert_called_with(openstack_origin_git,
core_project='glance')
self.assertTrue(git_post.called)
@patch.object(utils, 'mkdir')
@patch.object(utils, 'write_file')
@patch.object(utils, 'add_user_to_group')
@patch.object(utils, 'add_group')
@patch.object(utils, 'adduser')
def test_git_pre_install(self, adduser, add_group, add_user_to_group,
write_file, mkdir):
utils.git_pre_install()
adduser.assert_called_with('glance', shell='/bin/bash',
system_user=True)
add_group.assert_called_with('glance', system_group=True)
add_user_to_group.assert_called_with('glance', 'glance')
expected = [
call('/var/lib/glance', owner='glance',
group='glance', perms=0700, force=False),
call('/var/lib/glance/images', owner='glance',
group='glance', perms=0700, force=False),
call('/var/lib/glance/image-cache', owner='glance',
group='glance', perms=0700, force=False),
call('/var/lib/glance/image-cache/incomplete', owner='glance',
group='glance', perms=0700, force=False),
call('/var/lib/glance/image-cache/invalid', owner='glance',
group='glance', perms=0700, force=False),
call('/var/lib/glance/image-cache/queue', owner='glance',
group='glance', perms=0700, force=False),
call('/var/log/glance', owner='glance',
group='glance', perms=0700, force=False),
call('/etc/glance', owner='glance',
group='glance', perms=0700, force=False),
]
self.assertEquals(mkdir.call_args_list, expected)
expected = [
call('/var/log/glance/glance-api.log', '', owner='glance',
group='glance', perms=0600),
call('/var/log/glance/glance-registry.log', '', owner='glance',
group='glance', perms=0600),
]
self.assertEquals(write_file.call_args_list, expected)
@patch.object(utils, 'git_src_dir')
@patch.object(utils, 'service_restart')
@patch.object(utils, 'render')
@patch('os.path.join')
@patch('shutil.copyfile')
def test_git_post_install(self, copyfile, join, render, service_restart,
git_src_dir):
projects_yaml = openstack_origin_git
join.return_value = 'joined-string'
utils.git_post_install(projects_yaml)
expected = [
call('joined-string', '/etc/glance/glance-cache.conf'),
call('joined-string', '/etc/glance/glance-scrubber.conf'),
call('joined-string', '/etc/glance/policy.json'),
call('joined-string', '/etc/glance/schema-image.json'),
]
copyfile.assert_has_calls(expected, any_order=True)
glance_api_context = {
'service_description': 'Glance API server',
'service_name': 'Glance',
'user_name': 'glance',
'start_dir': '/var/lib/glance',
'process_name': 'glance-api',
'executable_name': '/usr/local/bin/glance-api',
'config_file': '/etc/glance/glance-api.conf',
'log_file': '/var/log/glance/api.log',
}
glance_registry_context = {
'service_description': 'Glance registry server',
'service_name': 'Glance',
'user_name': 'glance',
'start_dir': '/var/lib/glance',
'process_name': 'glance-registry',
'executable_name': '/usr/local/bin/glance-registry',
'config_file': '/etc/glance/glance-registry.conf',
'log_file': '/var/log/glance/registry.log',
}
expected = [
call('git.upstart', '/etc/init/glance-api.conf',
glance_api_context, perms=0o644,
templates_dir='joined-string'),
call('git.upstart', '/etc/init/glance-registry.conf',
glance_registry_context, perms=0o644,
templates_dir='joined-string'),
]
self.assertEquals(render.call_args_list, expected)
expected = [
call('glance-api'),
call('glance-registry'),
]
self.assertEquals(service_restart.call_args_list, expected)