Files
deb-python-taskflow/taskflow/utils/kazoo_utils.py
Joshua Harlow b473d9ac6b Some zookeeper persistence improvements/adjustments
- Only use zake in unit-tests by providing a client
  to the backend as a fake client. This avoids cases
  where a real client should be provided; and if not
  provided then a error should be thrown. This avoids
  potential misuse of zake (which really is only designed
  for testing usage and not for actual usage in production).
- Add a kazoo_utils file that will be shared to make
  kazoo clients and to parse hosts into a kazoo friendly
  format for other usages of kazoo clients that will be
  appearing soon.
- Ensure basic assertions when creating the zk backend,
  don't allow empty paths, require absolute paths.
- Update exc_wrapper to have slightly more descriptive
  messages that identify more of what the original errors
  category was.
- Create initial paths on upgrade() to mirror what other
  backends are doing in their upgrade() function. Other
  backends do their initial steps to upgrade there backends
  in this function (ie sqlalchemy does its migrations here
  so it seems to be consistent place to make sure zookeeper
  paths are created correctly).

Change-Id: Iabafe73c059b4719617f01bd1ee35795f71ca21d
2014-02-02 18:20:50 -08:00

55 lines
2.0 KiB
Python

# -*- coding: utf-8 -*-
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (C) 2014 Yahoo! Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from kazoo import client
import six
def _parse_hosts(hosts):
if isinstance(hosts, six.string_types):
return hosts.strip()
if isinstance(hosts, (dict)):
host_ports = []
for (k, v) in six.iteritems(hosts):
host_ports.append("%s:%s" % (k, v))
hosts = host_ports
if isinstance(hosts, (list, set, tuple)):
return ",".join([str(h) for h in hosts])
return hosts
def make_client(conf):
"""Creates a kazoo client given a configuration dictionary."""
client_kwargs = {
'read_only': bool(conf.get('read_only')),
'randomize_hosts': bool(conf.get('randomize_hosts')),
}
hosts = _parse_hosts(conf.get("hosts", "localhost:2181"))
if not hosts or not isinstance(hosts, six.string_types):
raise TypeError("Invalid hosts format, expected "
"non-empty string/list, not %s" % type(hosts))
client_kwargs['hosts'] = hosts
if 'timeout' in conf:
client_kwargs['timeout'] = float(conf['timeout'])
# Kazoo supports various handlers, gevent, threading, eventlet...
# allow the user of this client object to optionally specify one to be
# used.
if 'handler' in conf:
client_kwargs['handler'] = conf['handler']
return client.KazooClient(**client_kwargs)