eb66cfe767
This patch implements the register/deregister/lookup operations for the sharding manager. As a result of this patch, the sharding driver now takes the control driver as a paramater. This gives it the access needed to make decisions about queue placement (and lookups) based on the entries in the control plane. In order to get the sharding manager working correctly, several methods had to be overridden to properly handle the QueueNotMapped error. When a QueueNotMapped error is encountered under normal conditions, it is equivalent to the target resource not being found. The overrides mimic the default behavior of those controllers' methods in the case where the queue isn't found. An inconsistency was found between the base storage driver and the concrete implementations. message_controller.list did not provide an include_claimed parameter. This has been corrected. To avoid DuplicateOptErrors in the mongodb driver in a sharded context, a filtering algorithm is applied. To make it clear to data drivers that they need to apply this filtering algorithm, conf.dynamic is set by the sharding driver. Unit tests have been added for the weighted select algorithm. Up next: sharded queue listing support, caching Change-Id: Ic36cd657ba13fa4c0b624dc5c71bfe99f9a70139 Partitally-implements: blueprint storage-sharding
59 lines
1.7 KiB
Python
59 lines
1.7 KiB
Python
# Copyright (c) 2013 Rackspace Hosting, Inc.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
# implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
"""select: a collection of algorithms for choosing an entry from a
|
|
collection."""
|
|
|
|
import random
|
|
|
|
|
|
def weighted(objs, key='weight', generator=random.randint):
|
|
"""Perform a weighted select given a list of objects.
|
|
|
|
:param objs: a list of objects containing at least the field `key`
|
|
:type objs: [dict]
|
|
:param key: the field in each obj that corresponds to weight
|
|
:type key: six.text_type
|
|
:param generator: a number generator taking two ints
|
|
:type generator: function(int, int) -> int
|
|
:return: an object
|
|
:rtype: dict
|
|
"""
|
|
acc = 0
|
|
lookup = []
|
|
|
|
# construct weighted spectrum
|
|
for o in objs:
|
|
# NOTE(cpp-cabrera): skip objs with 0 weight
|
|
if o[key] <= 0:
|
|
continue
|
|
acc += o[key]
|
|
lookup.append((o, acc))
|
|
|
|
# no objects were found
|
|
if not lookup:
|
|
return None
|
|
|
|
# NOTE(cpp-cabrera): select an object from the lookup table. If
|
|
# the selector lands in the interval [lower, upper), then choose
|
|
# it.
|
|
gen = generator
|
|
selector = gen(0, acc - 1)
|
|
lower = 0
|
|
for obj, upper in lookup:
|
|
if lower <= selector < upper:
|
|
return obj
|
|
lower = upper
|