From 59e476e6b52827b4cc395d588dcacb416acc5490 Mon Sep 17 00:00:00 2001 From: Brian Curtin Date: Tue, 16 Feb 2016 09:07:58 -0500 Subject: [PATCH] Refactor Resource to better serve Proxy Through a series of attempts we haven't been able to sufficiently solve the issue of what were called "path_args" in the past, somewhat of a relic of the time before we had the Proxy layer. "path_args" made a lot of sense back when there was only the Resource layer, when the SDK's usage was almost entirely through the classmethods on Resource, back when command-line example scripts were all we had to go on. As things have grown in the year-plus since that time, we have bolted a great higher-level interface on a lower-level interface that was stuck in an older use-case. It was most recently a problem when we had resource.prop types that accepeted Resource types, which both there and elsewhere meant it became significantly harder - if not impossible - to have nested Resources. An example of this is how ServerInterface is a Resource that depends on Server. This new approach strips Resource down to a public API of only create, get, update, delete, head, find, and list methods. Resource subclasses are still constructed in the same manner, though the naming of what were "props" is now slightly different, but more clear. Additionally, through that change comes a more clear user experience for URI parameters, formerly known as, "path_args". The changes here pave the way for more clarity on query parameters, although they haven't been fully explored. As the changing of this class has pretty far reaching effects, it will exist temporarily in the "resource2" module. The first change will be to introduce the module itself as a standalone, and then we will change each service independent of the others. Subsequent changes should not be too disruptive, mostly in the form of renaming things like resource.prop to resource.Body, and the like. There is a fairly major side effect of the implementation of this class that is behaviorally different from the initial Resource implementation. Previously, any-and-all data sent into a Resource initializer would be tracked internally on the Resource instance and then sent to the server *even if it did not correspond to a resource.prop on the class*. Ultimately this would have become a problem as servers are becoming more strict about what they receive, but this implementation makes no allowance for so-called loose data to be entered. If there is no component attribute for an argument to be sent to the server, or one received from the server, it will not exist as an attribute of the Resource. Currently, said data is just dropped entirely, but it could be logged or perhaps a strict/loose mode could be engaged to warn or otherwise fail when unexpected data is entered. Ultimately, this behavior had to happen. In the near term it may mean that the SDK doesn't work exactly as it had in previous versions, but that will just mean updating Resource definitions to match what is truly expected or given by the server. A relatively major change that comes from this refactoring is a deconstruction of the unit tests that previously existed. While resource.prop now lives on as resource2._BaseComponent, the unit tests were started over from scratch as they were too tightly coupled to the Resource class rather than being tested as an independent unit. All of the code that is being refactored as a part of this change is now being tested as independent of any other code as is possible. Additional changes: 1. Removed support for aliases in _BaseComponent. Aliased property names were added for samples coming out of the telemetry service as they used to be supported and documented having multiple names. However, they are no longer documented this way. Additionally, aliases are abused in object_store due to a misunderstanding which causes actual breakage during use. One resource.prop was in some cases serving to retreieve both header and body values with similar names. Ultimately, aliases for property names are not needed and their support is removed as we have separated out the storage of body and header components. 2. Query parameters are now more officially and easily supported through the _query_mapping attribute on Resource2. The resource2.QueryParameters class creates the appropriate mapping of client-side names to server-side names, which is then used when calling Resource2.list in order to transpose the names and make a proper request. 3. As all tests were rewritten from scratch, tests for the wait_for* functions end up being 8-10 times faster as they mock out time.sleep rather than actually waiting, thus making test runs a lot quicker. Change-Id: I634ac93b913e4fe9a3a812b384c8539d9266f2c0 --- openstack/exceptions.py | 17 +- openstack/resource2.py | 790 +++++++++++++++ openstack/tests/unit/test_resource2.py | 1286 ++++++++++++++++++++++++ 3 files changed, 2091 insertions(+), 2 deletions(-) create mode 100644 openstack/resource2.py create mode 100644 openstack/tests/unit/test_resource2.py diff --git a/openstack/exceptions.py b/openstack/exceptions.py index f0048e03..ad2c81ac 100644 --- a/openstack/exceptions.py +++ b/openstack/exceptions.py @@ -35,6 +35,13 @@ class InvalidResponse(SDKException): self.response = response +class InvalidRequest(SDKException): + """The request to the server is not valid.""" + + def __init__(self, message=None): + super(InvalidRequest, self).__init__(message) + + class HttpException(SDKException): def __init__(self, message=None, details=None, response=None, @@ -65,9 +72,15 @@ class NotFoundException(HttpException): class MethodNotSupported(SDKException): """The resource does not support this operation type.""" - def __init__(self, cls, method): + def __init__(self, resource, method): + # This needs to work with both classes and instances. + try: + name = resource.__name__ + except AttributeError: + name = resource.__class__.__name__ + message = ('The %s method is not supported for %s.%s' % - (method, cls.__module__, cls.__name__)) + (method, resource.__module__, name)) super(MethodNotSupported, self).__init__(message=message) diff --git a/openstack/resource2.py b/openstack/resource2.py new file mode 100644 index 00000000..e0dfa26b --- /dev/null +++ b/openstack/resource2.py @@ -0,0 +1,790 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +The :class:`~openstack.resource.Resource` class is a base +class that represent a remote resource. The attributes that +comprise a request or response for this resource are specified +as class members on the Resource subclass where their values +are of a component type, including :class:`~openstack.resource2.Body`, +:class:`~openstack.resource2.Header`, and :class:`~openstack.resource2.URI`. + +For update management, :class:`~openstack.resource2.Resource` employs +a series of :class:`~openstack.resource2._ComponentManager` instances +to look after the attributes of that particular component type. This is +particularly useful for Body and Header types, so that only the values +necessary are sent in requests to the server. + +When making requests, each of the managers are looked at to gather the +necessary URI, body, and header data to build a request to be sent +via keystoneauth's sessions. Responses from keystoneauth are then +converted into this Resource class' appropriate components and types +and then returned to the caller. +""" + +import collections +import itertools +import time + +from openstack import exceptions +from openstack import format +from openstack import utils + + +class _BaseComponent(object): + + # The name this component is being tracked as in the Resource + key = None + + def __init__(self, name, type=None, default=None, alternate_id=False): + """A typed descriptor for a component that makes up a Resource + + :param name: The name this component exists as on the server + :param type: The type this component is expected to be by the server. + By default this is None, meaning any value you specify + will work. If you specify type=dict and then set a + component to a string, __set__ will fail, for example. + :param default: Typically None, but any other default can be set. + :param alternate_id: When `True`, this property is known + internally as a value that can be sent + with requests that require an ID but + when `id` is not a name the Resource has. + This is a relatively uncommon case, and this + setting should only be used once per Resource. + """ + self.name = name + self.type = type + self.default = default + self.alternate_id = alternate_id + + def __get__(self, instance, owner): + if instance is None: + return None + + attributes = getattr(instance, self.key) + + try: + value = attributes[self.name] + except KeyError: + return self.default + + # self.type() should not be called on None objects. + if value is None: + return None + + if self.type and not isinstance(value, self.type): + if issubclass(self.type, format.Formatter): + value = self.type.deserialize(value) + else: + value = self.type(value) + + return value + + def __set__(self, instance, value): + if (self.type and not isinstance(value, self.type) and + value != self.default): + if issubclass(self.type, format.Formatter): + value = self.type.serialize(value) + else: + value = str(self.type(value)) # validate to fail fast + + attributes = getattr(instance, self.key) + attributes[self.name] = value + + def __delete__(self, instance): + try: + attributes = getattr(instance, self.key) + del attributes[self.name] + except KeyError: + pass + + +class Body(_BaseComponent): + """Body attributes""" + + key = "_body" + + +class Header(_BaseComponent): + """Header attributes""" + + key = "_header" + + +class URI(_BaseComponent): + """URI attributes""" + + key = "_uri" + + +class _ComponentManager(collections.MutableMapping): + """Storage of a component type""" + + def __init__(self, attributes=None, synchronized=False): + self.attributes = dict() if attributes is None else attributes.copy() + self._dirty = set() if synchronized else set(self.attributes.keys()) + + def __getitem__(self, key): + return self.attributes[key] + + def __setitem__(self, key, value): + try: + orig = self.attributes[key] + except KeyError: + changed = True + else: + changed = orig != value + + if changed: + self.attributes[key] = value + self._dirty.add(key) + + def __delitem__(self, key): + del self.attributes[key] + self._dirty.add(key) + + def __iter__(self): + return iter(self.attributes) + + def __len__(self): + return len(self.attributes) + + @property + def dirty(self): + """Return a dict of modified attributes""" + return dict((key, self.attributes.get(key, None)) + for key in self._dirty) + + def clean(self): + """Signal that the resource no longer has modified attributes""" + self._dirty = set() + + +class _Request(object): + """Prepared components that go into a KSA request""" + + def __init__(self, uri, body, headers): + self.uri = uri + self.body = body + self.headers = headers + + +class QueryParameters(object): + + def __init__(self, *names, **mappings): + """Create a dict of accepted query parameters + + names are strings where the client-side name matches + what the server expects, e.g., server=server. + + mappings are key-value pairs where the key is the + client-side name we'll accept here and the value is + the name the server expects, e.g, changes_since=changes-since + """ + self._mapping = dict({name: name for name in names}, **mappings) + + def _transpose(self, query): + """Transpose the keys in query based on the mapping + + This method converts the keys in `query` from their + client-side names to have the appropriate keys as + expected by the server for query parameters. + """ + result = {} + for key, value in self._mapping.items(): + if key in query: + result[value] = query[key] + return result + + +class Resource(object): + + #: Singular form of key for resource. + resource_key = None + #: Plural form of key for resource. + resources_key = None + + #: The ID of this resource. + id = Body("id") + #: The name of this resource. + name = Body("name") + #: The location of this resource. + location = Header("location") + + #: Mapping of accepted query parameter names. + _query_mapping = QueryParameters() + + #: The base part of the URI for this resource. + base_path = "" + + #: The service associated with this resource to find the service URL. + service = None + + #: Allow create operation for this resource. + allow_create = False + #: Allow get operation for this resource. + allow_get = False + #: Allow update operation for this resource. + allow_update = False + #: Allow delete operation for this resource. + allow_delete = False + #: Allow list operation for this resource. + allow_list = False + #: Allow head operation for this resource. + allow_head = False + #: Use PATCH for update operations on this resource. + patch_update = False + + def __init__(self, synchronized=False, **attrs): + # NOTE: _collect_attrs modifies **attrs in place, removing + # items as they match up with any of the body, header, + # or uri mappings. + body, header, uri = self._collect_attrs(attrs) + # TODO(briancurtin): at this point if attrs has anything left + # they're not being set anywhere. Log this? Raise exception? + # How strict should we be here? Should strict be an option? + + self._body = _ComponentManager(attributes=body, + synchronized=synchronized) + self._header = _ComponentManager(attributes=header, + synchronized=synchronized) + self._uri = _ComponentManager(attributes=uri, + synchronized=synchronized) + + def __repr__(self): + pairs = ["%s=%s" % (k, v) for k, v in dict(itertools.chain( + self._body.attributes.items(), + self._header.attributes.items(), + self._uri.attributes.items())).items()] + args = ", ".join(pairs) + + return "%s.%s(%s)" % ( + self.__module__, self.__class__.__name__, args) + + def _update(self, **attrs): + """Given attributes, update them on this instance + + This is intended to be used from within the proxy + layer when updating instances that may have already + been created. + """ + body, header, uri = self._collect_attrs(attrs) + + self._body.update(body) + self._header.update(header) + self._uri.update(uri) + + def _collect_attrs(self, attrs): + """Given attributes, return a dict per type of attribute + + This method splits up **attrs into separate dictionaries + that correspond to the relevant body, header, and uri + attributes that exist on this class. + """ + body = self._consume_attrs(self._body_mapping(), attrs) + header = self._consume_attrs(self._header_mapping(), attrs) + uri = self._consume_attrs(self._uri_mapping(), attrs) + + return body, header, uri + + def _consume_attrs(self, mapping, attrs): + """Given a mapping and attributes, return relevant matches + + This method finds keys in attrs that exist in the mapping, then + both transposes them to their server-side equivalent key name + to be returned, and finally pops them out of attrs. This allows + us to only calculate their place and existence in a particular + type of Resource component one time, rather than looking at the + same source dict several times. + """ + relevant_attrs = {} + consumed_keys = [] + for key in attrs: + if key in mapping: + # Convert client-side key names into server-side. + relevant_attrs[mapping[key]] = attrs[key] + consumed_keys.append(key) + elif key in mapping.values(): + # Server-side names can be stored directly. + relevant_attrs[key] = attrs[key] + consumed_keys.append(key) + + for key in consumed_keys: + attrs.pop(key) + + return relevant_attrs + + @classmethod + def _get_mapping(cls, component): + """Return a dict of attributes of a given component on the class""" + mapping = {} + # Since we're looking at class definitions we need to include + # subclasses, so check the whole MRO. + for klass in cls.__mro__: + for key, value in klass.__dict__.items(): + if isinstance(value, component): + mapping[key] = value.name + return mapping + + @classmethod + def _body_mapping(cls): + """Return all Body members of this class""" + return cls._get_mapping(Body) + + @classmethod + def _header_mapping(cls): + """Return all Header members of this class""" + return cls._get_mapping(Header) + + @classmethod + def _uri_mapping(cls): + """Return all URI members of this class""" + return cls._get_mapping(URI) + + @classmethod + def _alternate_id(cls): + """Return the name of any value known as an alternate_id + + NOTE: This will only ever return the first such alternate_id. + Only one alternate_id should be specified. + + Returns an empty string if no name exists, as this method is + consumed by _get_id and passed to getattr. + """ + for key, value in cls.__dict__.items(): + if isinstance(value, Body): + if value.alternate_id: + return key + return "" + + @staticmethod + def _get_id(value): + """If a value is a Resource, return the canonical ID + + This will return either the value specified by `id` or + `alternate_id` in that order if `value` is a Resource. + If `value` is anything other than a Resource, likely to + be a string already representing an ID, it is returned. + """ + if isinstance(value, Resource): + # Don't check _alternate_id unless we need to. It's an uncommon + # case and it involves looping through the class' dict. + id = value.id or getattr(value, value._alternate_id(), None) + return id + else: + return value + + @classmethod + def new(cls, **kwargs): + """Create a new instance of this resource. + + Internally set flags such that it is marked as not present on the + server. + + :param dict kwargs: Each of the named arguments will be set as + attributes on the resulting Resource object. + """ + return cls(synchronized=False, **kwargs) + + @classmethod + def existing(cls, **kwargs): + """Create an instance of an existing remote resource. + + It is marked as an exact replication of a resource present on a server. + + :param dict kwargs: Each of the named arguments will be set as + attributes on the resulting Resource object. + """ + return cls(synchronized=True, **kwargs) + + def _prepare_request(self, requires_id=True, prepend_key=False): + """Prepare a request to be sent to the server + + Create operations don't require an ID, but all others do, + so only try to append an ID when it's needed with + requires_id. Create and update operations sometimes require + their bodies to be contained within an dict -- if the + instance contains a resource_key and prepend_key=True, + the body will be wrapped in a dict with that key. + + Return a _Request object that contains the constructed URI + as well a body and headers that are ready to send. + Only dirty body and header contents will be returned. + """ + body = self._body.dirty + if prepend_key and self.resource_key is not None: + body = {self.resource_key: body} + + headers = self._header.dirty + + uri = self.base_path % self._uri.attributes + if requires_id: + id = self._get_id(self) + if id is None: + raise exceptions.InvalidRequest( + "Request requires an ID but none was found") + + uri = utils.urljoin(uri, id) + + return _Request(uri, body, headers) + + def _transpose_component(self, component, mapping): + """Transpose the keys in component based on a mapping + + This method converts a dict of server-side data to have + the appropriate keys for attributes on this instance. + """ + result = {} + for key, value in mapping.items(): + if value in component: + result[key] = component[value] + + return result + + def _translate_response(self, response, has_body=True): + """Given a KSA response, inflate this instance with its data + + DELETE operations don't return a body, so only try to work + with a body when has_body is True. + + This method updates attributes that correspond to headers + and body on this instance and clears the dirty set. + """ + if has_body: + body = response.json() + if self.resource_key and self.resource_key in body: + body = body[self.resource_key] + + body = self._transpose_component(body, self._body_mapping()) + self._body.attributes.update(body) + self._body.clean() + + headers = self._transpose_component(response.headers, + self._header_mapping()) + self._header.attributes.update(headers) + self._header.clean() + + def create(self, session): + """Create a remote resource based on this instance. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + + :return: This :class:`Resource` instance. + :raises: :exc:`~openstack.exceptions.MethodNotSupported` if + :data:`Resource.allow_create` is not set to ``True``. + """ + if not self.allow_create: + raise exceptions.MethodNotSupported(self, "create") + + if self.id is None: + request = self._prepare_request(requires_id=False, + prepend_key=True) + response = session.post(request.uri, endpoint_filter=self.service, + json=request.body, headers=request.headers) + else: + request = self._prepare_request(requires_id=True, + prepend_key=True) + response = session.put(request.uri, endpoint_filter=self.service, + json=request.body, headers=request.headers) + + self._translate_response(response) + return self + + def get(self, session): + """Get a remote resource based on this instance. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + + :return: This :class:`Resource` instance. + :raises: :exc:`~openstack.exceptions.MethodNotSupported` if + :data:`Resource.allow_get` is not set to ``True``. + """ + if not self.allow_get: + raise exceptions.MethodNotSupported(self, "get") + + request = self._prepare_request() + + response = session.get(request.uri, endpoint_filter=self.service) + + self._translate_response(response) + return self + + def head(self, session): + """Get headers from a remote resource based on this instance. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + + :return: This :class:`Resource` instance. + :raises: :exc:`~openstack.exceptions.MethodNotSupported` if + :data:`Resource.allow_head` is not set to ``True``. + """ + if not self.allow_head: + raise exceptions.MethodNotSupported(self, "head") + + request = self._prepare_request() + + response = session.head(request.uri, endpoint_filter=self.service, + headers={"Accept": ""}) + + self._translate_response(response) + return self + + def update(self, session): + """Update the remote resource based on this instance. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + + :return: This :class:`Resource` instance. + :raises: :exc:`~openstack.exceptions.MethodNotSupported` if + :data:`Resource.allow_update` is not set to ``True``. + """ + # Only try to update if we actually have anything to update. + if not any([self._body.dirty, self._header.dirty]): + return self + + if not self.allow_update: + raise exceptions.MethodNotSupported(self, "update") + + request = self._prepare_request(prepend_key=True) + + if self.patch_update: + response = session.patch(request.uri, endpoint_filter=self.service, + json=request.body, + headers=request.headers) + else: + response = session.put(request.uri, endpoint_filter=self.service, + json=request.body, headers=request.headers) + + self._translate_response(response) + return self + + def delete(self, session): + """Delete the remote resource based on this instance. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + + :return: This :class:`Resource` instance. + :raises: :exc:`~openstack.exceptions.MethodNotSupported` if + :data:`Resource.allow_update` is not set to ``True``. + """ + if not self.allow_delete: + raise exceptions.MethodNotSupported(self, "delete") + + request = self._prepare_request() + + response = session.delete(request.uri, endpoint_filter=self.service, + headers={"Accept": ""}) + + self._translate_response(response, has_body=False) + return self + + @classmethod + def list(cls, session, paginated=False, **params): + """This method is a generator which yields resource objects. + + This resource object list generator handles pagination and takes query + params for response filtering. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + :param bool paginated: ``True`` if a GET to this resource returns + a paginated series of responses, or ``False`` + if a GET returns only one page of data. + **When paginated is False only one + page of data will be returned regardless + of the API's support of pagination.** + :param dict params: These keyword arguments are passed through the + :meth:`~openstack.resource2.QueryParamter._transpose` method + to find if any of them match expected query parameters to be + sent in the *params* argument to + :meth:`~openstack.session.Session.get`. They are additionally + checked against the + :data:`~openstack.resource2.Resource.base_path` format string + to see if any path fragments need to be filled in by the contents + of this argument. + + :return: A generator of :class:`Resource` objects. + :raises: :exc:`~openstack.exceptions.MethodNotSupported` if + :data:`Resource.allow_list` is not set to ``True``. + """ + if not cls.allow_list: + raise exceptions.MethodNotSupported(cls, "list") + + more_data = True + query_params = cls._query_mapping._transpose(params) + uri = cls.base_path % params + + while more_data: + resp = session.get(uri, endpoint_filter=cls.service, + headers={"Accept": "application/json"}, + params=query_params) + resp = resp.json() + if cls.resources_key: + resp = resp[cls.resources_key] + + if not resp: + more_data = False + + # Keep track of how many items we've yielded. If we yielded + # less than our limit, we don't need to do an extra request + # to get back an empty data set, which acts as a sentinel. + yielded = 0 + new_marker = None + for data in resp: + value = cls.existing(**data) + new_marker = value.id + yielded += 1 + yield value + + if not paginated: + return + if "limit" in params and yielded < params["limit"]: + return + params["limit"] = yielded + params["marker"] = new_marker + + @classmethod + def _get_one_match(cls, name_or_id, results): + """Given a list of results, return the match""" + the_result = None + for maybe_result in results: + id_value = cls._get_id(maybe_result) + name_value = maybe_result.name + + if (id_value == name_or_id) or (name_value == name_or_id): + # Only allow one resource to be found. If we already + # found a match, raise an exception to show it. + if the_result is None: + the_result = maybe_result + else: + msg = "More than one %s exists with the name '%s'." + msg = (msg % (cls.__name__, name_or_id)) + raise exceptions.DuplicateResource(msg) + + return the_result + + @classmethod + def find(cls, session, name_or_id, ignore_missing=True, **params): + """Find a resource by its name or id. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + :param name_or_id: This resource's identifier, if needed by + the request. The default is ``None``. + :param bool ignore_missing: When set to ``False`` + :class:`~openstack.exceptions.ResourceNotFound` will be + raised when the resource does not exist. + When set to ``True``, None will be returned when + attempting to find a nonexistent resource. + :param dict params: Any additional parameters to be passed into + underlying methods, such as to + :meth:`~openstack.resource2.Resource.existing` + in order to pass on URI parameters. + + :return: The :class:`Resource` object matching the given name or id + or None if nothing matches. + :raises: :class:`openstack.exceptions.DuplicateResource` if more + than one resource is found for this request. + :raises: :class:`openstack.exceptions.ResourceNotFound` if nothing + is found and ignore_missing is ``False``. + """ + # Try to short-circuit by looking directly for a matching ID. + try: + match = cls.existing(id=name_or_id, **params) + return match.get(session) + except exceptions.NotFoundException: + pass + + data = cls.list(session, **params) + + result = cls._get_one_match(name_or_id, data) + if result is not None: + return result + + if ignore_missing: + return None + raise exceptions.ResourceNotFound( + "No %s found for %s" % (cls.__name__, name_or_id)) + + +def wait_for_status(session, resource, status, failures, interval, wait): + """Wait for the resource to be in a particular status. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + :param resource: The resource to wait on to reach the status. The resource + must have a status attribute. + :type resource: :class:`~openstack.resource.Resource` + :param status: Desired status of the resource. + :param list failures: Statuses that would indicate the transition + failed such as 'ERROR'. + :param interval: Number of seconds to wait between checks. + :param wait: Maximum number of seconds to wait for transition. + + :return: Method returns self on success. + :raises: :class:`~openstack.exceptions.ResourceTimeout` transition + to status failed to occur in wait seconds. + :raises: :class:`~openstack.exceptions.ResourceFailure` resource + transitioned to one of the failure states. + :raises: :class:`~AttributeError` if the resource does not have a status + attribute + """ + if resource.status == status: + return resource + + total_sleep = 0 + if failures is None: + failures = [] + + while total_sleep < wait: + resource.get(session) + if resource.status == status: + return resource + if resource.status in failures: + msg = ("Resource %s transitioned to failure state %s" % + (resource.id, resource.status)) + raise exceptions.ResourceFailure(msg) + time.sleep(interval) + total_sleep += interval + msg = "Timeout waiting for %s to transition to %s" % (resource.id, status) + raise exceptions.ResourceTimeout(msg) + + +def wait_for_delete(session, resource, interval, wait): + """Wait for the resource to be deleted. + + :param session: The session to use for making this request. + :type session: :class:`~openstack.session.Session` + :param resource: The resource to wait on to be deleted. + :type resource: :class:`~openstack.resource.Resource` + :param interval: Number of seconds to wait between checks. + :param wait: Maximum number of seconds to wait for the delete. + + :return: Method returns self on success. + :raises: :class:`~openstack.exceptions.ResourceTimeout` transition + to status failed to occur in wait seconds. + """ + total_sleep = 0 + while total_sleep < wait: + try: + resource.get(session) + except exceptions.NotFoundException: + return resource + time.sleep(interval) + total_sleep += interval + msg = "Timeout waiting for %s delete" % (resource.id) + raise exceptions.ResourceTimeout(msg) diff --git a/openstack/tests/unit/test_resource2.py b/openstack/tests/unit/test_resource2.py new file mode 100644 index 00000000..58439ce5 --- /dev/null +++ b/openstack/tests/unit/test_resource2.py @@ -0,0 +1,1286 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import itertools + +import mock +import six + +from openstack import exceptions +from openstack import format +from openstack import resource2 +from openstack import session +from openstack.tests.unit import base + + +class TestComponent(base.TestCase): + + class ExampleComponent(resource2._BaseComponent): + key = "_example" + + # Since we're testing ExampleComponent, which is as isolated as we + # can test _BaseComponent due to it's needing to be a data member + # of a class that has an attribute on the parent class named `key`, + # each test has to implement a class with a name that is the same + # as ExampleComponent.key, which should be a dict containing the + # keys and values to test against. + + def test_implementations(self): + self.assertEqual("_body", resource2.Body.key) + self.assertEqual("_header", resource2.Header.key) + self.assertEqual("_uri", resource2.URI.key) + + def test_creation(self): + sot = resource2._BaseComponent("name", type=int, default=1, + alternate_id=True) + + self.assertEqual("name", sot.name) + self.assertEqual(int, sot.type) + self.assertEqual(1, sot.default) + self.assertTrue(sot.alternate_id) + + def test_get_no_instance(self): + sot = resource2._BaseComponent("test") + + # Test that we short-circuit everything when given no instance. + result = sot.__get__(None, None) + self.assertIsNone(result) + + # NOTE: Some tests will use a default=1 setting when testing result + # values that should be None because the default-for-default is also None. + def test_get_name_None(self): + name = "name" + + class Parent(object): + _example = {name: None} + + instance = Parent() + sot = TestComponent.ExampleComponent(name, default=1) + + # Test that we short-circuit any typing of a None value. + result = sot.__get__(instance, None) + self.assertIsNone(result) + + def test_get_default(self): + expected_result = 123 + + class Parent(object): + _example = {} + + instance = Parent() + # NOTE: type=dict but the default value is an int. If we didn't + # short-circuit the typing part of __get__ it would fail. + sot = TestComponent.ExampleComponent("name", type=dict, + default=expected_result) + + # Test that we directly return any default value. + result = sot.__get__(instance, None) + self.assertEqual(expected_result, result) + + def test_get_name_untyped(self): + name = "name" + expected_result = 123 + + class Parent(object): + _example = {name: expected_result} + + instance = Parent() + sot = TestComponent.ExampleComponent("name") + + # Test that we return any the value as it is set. + result = sot.__get__(instance, None) + self.assertEqual(expected_result, result) + + # The code path for typing after a raw value has been found is the same. + def test_get_name_typed(self): + name = "name" + value = "123" + + class Parent(object): + _example = {name: value} + + instance = Parent() + sot = TestComponent.ExampleComponent("name", type=int) + + # Test that we run the underlying value through type conversion. + result = sot.__get__(instance, None) + self.assertEqual(int(value), result) + + def test_get_name_formatter(self): + name = "name" + value = "123" + expected_result = "one hundred twenty three" + + class Parent(object): + _example = {name: value} + + class FakeFormatter(object): + @classmethod + def deserialize(cls, value): + return expected_result + + instance = Parent() + sot = TestComponent.ExampleComponent("name", type=FakeFormatter) + + # Mock out issubclass rather than having an actual format.Formatter + # This can't be mocked via decorator, isolate it to wrapping the call. + mock_issubclass = mock.Mock(return_value=True) + module = six.moves.builtins.__name__ + with mock.patch("%s.issubclass" % module, mock_issubclass): + result = sot.__get__(instance, None) + self.assertEqual(expected_result, result) + + def test_set_name_untyped(self): + name = "name" + expected_value = "123" + + class Parent(object): + _example = {} + + instance = Parent() + sot = TestComponent.ExampleComponent("name") + + # Test that we don't run the value through type conversion. + sot.__set__(instance, expected_value) + self.assertEqual(expected_value, instance._example[name]) + + def test_set_name_typed(self): + expected_value = "123" + + class Parent(object): + _example = {} + + instance = Parent() + + # The type we give to ExampleComponent has to be an actual type, + # not an instance, so we can't get the niceties of a mock.Mock + # instance that would allow us to call `assert_called_once_with` to + # ensure that we're sending the value through the type. + # Instead, we use this tiny version of a similar thing. + class FakeType(object): + calls = [] + + def __init__(self, arg): + FakeType.calls.append(arg) + + sot = TestComponent.ExampleComponent("name", type=FakeType) + + # Test that we run the value through type conversion. + sot.__set__(instance, expected_value) + self.assertEqual([expected_value], FakeType.calls) + + def test_set_name_formatter(self): + expected_value = "123" + + class Parent(object): + _example = {} + + instance = Parent() + + # As with test_set_name_typed, create a pseudo-Mock to track what + # gets called on the type. + class FakeFormatter(format.Formatter): + calls = [] + + @classmethod + def serialize(cls, arg): + FakeFormatter.calls.append(arg) + + sot = TestComponent.ExampleComponent("name", type=FakeFormatter) + + # Test that we run the value through type conversion. + sot.__set__(instance, expected_value) + self.assertEqual([expected_value], FakeFormatter.calls) + + def test_delete_name(self): + name = "name" + expected_value = "123" + + class Parent(object): + _example = {name: expected_value} + + instance = Parent() + + sot = TestComponent.ExampleComponent("name") + + sot.__delete__(instance) + + self.assertNotIn(name, instance._example) + + def test_delete_name_doesnt_exist(self): + name = "name" + expected_value = "123" + + class Parent(object): + _example = {"what": expected_value} + + instance = Parent() + + sot = TestComponent.ExampleComponent(name) + + sot.__delete__(instance) + + self.assertNotIn(name, instance._example) + + +class TestComponentManager(base.TestCase): + + def test_create_basic(self): + sot = resource2._ComponentManager() + self.assertEqual(dict(), sot.attributes) + self.assertEqual(set(), sot._dirty) + + def test_create_unsynced(self): + attrs = {"hey": 1, "hi": 2, "hello": 3} + sync = False + + sot = resource2._ComponentManager(attributes=attrs, synchronized=sync) + self.assertEqual(attrs, sot.attributes) + self.assertEqual(set(attrs.keys()), sot._dirty) + + def test_create_synced(self): + attrs = {"hey": 1, "hi": 2, "hello": 3} + sync = True + + sot = resource2._ComponentManager(attributes=attrs, synchronized=sync) + self.assertEqual(attrs, sot.attributes) + self.assertEqual(set(), sot._dirty) + + def test_getitem(self): + key = "key" + value = "value" + attrs = {key: value} + + sot = resource2._ComponentManager(attributes=attrs) + self.assertEqual(value, sot.__getitem__(key)) + + def test_setitem_new(self): + key = "key" + value = "value" + + sot = resource2._ComponentManager() + sot.__setitem__(key, value) + + self.assertIn(key, sot.attributes) + self.assertIn(key, sot.dirty) + + def test_setitem_unchanged(self): + key = "key" + value = "value" + attrs = {key: value} + + sot = resource2._ComponentManager(attributes=attrs, synchronized=True) + # This shouldn't end up in the dirty list since we're just re-setting. + sot.__setitem__(key, value) + + self.assertEqual(value, sot.attributes[key]) + self.assertNotIn(key, sot.dirty) + + def test_delitem(self): + key = "key" + value = "value" + attrs = {key: value} + + sot = resource2._ComponentManager(attributes=attrs, synchronized=True) + sot.__delitem__(key) + + self.assertIsNone(sot.dirty[key]) + + def test_iter(self): + attrs = {"key": "value"} + sot = resource2._ComponentManager(attributes=attrs) + self.assertItemsEqual(iter(attrs), sot.__iter__()) + + def test_len(self): + attrs = {"key": "value"} + sot = resource2._ComponentManager(attributes=attrs) + self.assertEqual(len(attrs), sot.__len__()) + + def test_dirty(self): + key = "key" + key2 = "key2" + value = "value" + attrs = {key: value} + sot = resource2._ComponentManager(attributes=attrs, synchronized=False) + self.assertEqual({key: value}, sot.dirty) + + sot.__setitem__(key2, value) + self.assertEqual({key: value, key2: value}, sot.dirty) + + def test_clean(self): + key = "key" + value = "value" + attrs = {key: value} + sot = resource2._ComponentManager(attributes=attrs, synchronized=False) + self.assertEqual(attrs, sot.dirty) + + sot.clean() + + self.assertEqual(dict(), sot.dirty) + + +class Test_Request(base.TestCase): + + def test_create(self): + uri = 1 + body = 2 + headers = 3 + + sot = resource2._Request(uri, body, headers) + + self.assertEqual(uri, sot.uri) + self.assertEqual(body, sot.body) + self.assertEqual(headers, sot.headers) + + +class TestQueryParameters(base.TestCase): + + def test_create(self): + location = "location" + mapping = {"first_name": "first-name"} + + sot = resource2.QueryParameters(location, **mapping) + + self.assertEqual({"location": "location", "first_name": "first-name"}, + sot._mapping) + + def test_transpose_unmapped(self): + location = "location" + mapping = {"first_name": "first-name"} + + sot = resource2.QueryParameters(location, **mapping) + result = sot._transpose({"location": "Brooklyn", + "first_name": "Brian", + "last_name": "Curtin"}) + + # last_name isn't mapped and shouldn't be included + self.assertEqual({"location": "Brooklyn", "first-name": "Brian"}, + result) + + def test_transpose_not_in_query(self): + location = "location" + mapping = {"first_name": "first-name"} + + sot = resource2.QueryParameters(location, **mapping) + result = sot._transpose({"location": "Brooklyn"}) + + # first_name not being in the query shouldn't affect results + self.assertEqual({"location": "Brooklyn"}, + result) + + +class TestResource(base.TestCase): + + def test_initialize_basic(self): + body = {"body": 1} + header = {"header": 2} + uri = {"uri": 3} + everything = dict(itertools.chain(body.items(), header.items(), + uri.items())) + + mock_collect = mock.Mock() + mock_collect.return_value = body, header, uri + + with mock.patch.object(resource2.Resource, + "_collect_attrs", mock_collect): + sot = resource2.Resource(synchronized=False, **everything) + mock_collect.assert_called_once_with(everything) + + self.assertIsInstance(sot._body, resource2._ComponentManager) + self.assertEqual(body, sot._body.dirty) + self.assertIsInstance(sot._header, resource2._ComponentManager) + self.assertEqual(header, sot._header.dirty) + self.assertIsInstance(sot._uri, resource2._ComponentManager) + self.assertEqual(uri, sot._uri.dirty) + + self.assertFalse(sot.allow_create) + self.assertFalse(sot.allow_get) + self.assertFalse(sot.allow_update) + self.assertFalse(sot.allow_delete) + self.assertFalse(sot.allow_list) + self.assertFalse(sot.allow_head) + self.assertFalse(sot.patch_update) + + def test_repr(self): + a = {"a": 1} + b = {"b": 2} + c = {"c": 3} + + class Test(resource2.Resource): + def __init__(self): + self._body = mock.Mock() + self._body.attributes.items = mock.Mock( + return_value=a.items()) + + self._header = mock.Mock() + self._header.attributes.items = mock.Mock( + return_value=b.items()) + + self._uri = mock.Mock() + self._uri.attributes.items = mock.Mock( + return_value=c.items()) + + the_repr = repr(Test()) + + # Don't test the arguments all together since the dictionary order + # they're rendered in can't be depended on, nor does it matter. + self.assertIn("openstack.tests.unit.test_resource2.Test", the_repr) + self.assertIn("a=1", the_repr) + self.assertIn("b=2", the_repr) + self.assertIn("c=3", the_repr) + + def test__update(self): + sot = resource2.Resource() + + body = "body" + header = "header" + uri = "uri" + + sot._collect_attrs = mock.Mock(return_value=(body, header, uri)) + sot._body.update = mock.Mock() + sot._header.update = mock.Mock() + sot._uri.update = mock.Mock() + + args = {"arg": 1} + sot._update(**args) + + sot._collect_attrs.assert_called_once_with(args) + sot._body.update.assert_called_once_with(body) + sot._header.update.assert_called_once_with(header) + sot._uri.update.assert_called_once_with(uri) + + def test__collect_attrs(self): + sot = resource2.Resource() + + expected_attrs = ["body", "header", "uri"] + + sot._consume_attrs = mock.Mock() + sot._consume_attrs.side_effect = expected_attrs + + # It'll get passed an empty dict at the least. + actual_attrs = sot._collect_attrs(dict()) + + self.assertItemsEqual(expected_attrs, actual_attrs) + + def test__consume_attrs(self): + serverside_key1 = "someKey1" + clientside_key1 = "some_key1" + serverside_key2 = "someKey2" + clientside_key2 = "some_key2" + value1 = "value1" + value2 = "value2" + mapping = {clientside_key1: serverside_key1, + clientside_key2: serverside_key2} + + other_key = "otherKey" + other_value = "other" + attrs = {clientside_key1: value1, + serverside_key2: value2, + other_key: other_value} + + sot = resource2.Resource() + + result = sot._consume_attrs(mapping, attrs) + + # Make sure that the expected key was consumed and we're only + # left with the other stuff. + self.assertDictEqual({other_key: other_value}, attrs) + + # Make sure that after we've popped our relevant client-side + # key off that we are returning it keyed off of its server-side + # name. + self.assertDictEqual({serverside_key1: value1, + serverside_key2: value2}, result) + + def test__mapping_defaults(self): + # Check that even on an empty class, we get the expected + # built-in attributes. + + self.assertIn("location", resource2.Resource._header_mapping()) + self.assertIn("name", resource2.Resource._body_mapping()) + self.assertIn("id", resource2.Resource._body_mapping()) + + def test__body_mapping(self): + class Test(resource2.Resource): + x = resource2.Body("x") + y = resource2.Body("y") + z = resource2.Body("z") + + self.assertIn("x", Test._body_mapping()) + self.assertIn("y", Test._body_mapping()) + self.assertIn("z", Test._body_mapping()) + + def test__header_mapping(self): + class Test(resource2.Resource): + x = resource2.Header("x") + y = resource2.Header("y") + z = resource2.Header("z") + + self.assertIn("x", Test._header_mapping()) + self.assertIn("y", Test._header_mapping()) + self.assertIn("z", Test._header_mapping()) + + def test__uri_mapping(self): + class Test(resource2.Resource): + x = resource2.URI("x") + y = resource2.URI("y") + z = resource2.URI("z") + + self.assertIn("x", Test._uri_mapping()) + self.assertIn("y", Test._uri_mapping()) + self.assertIn("z", Test._uri_mapping()) + + def test__alternate_id_None(self): + self.assertEqual("", resource2.Resource._alternate_id()) + + def test__alternate_id(self): + class Test(resource2.Resource): + alt = resource2.Body("alt", alternate_id=True) + + self.assertTrue("alt", Test._alternate_id()) + + def test__get_id_instance(self): + class Test(resource2.Resource): + id = resource2.Body("id") + + value = "id" + sot = Test(id=value) + + self.assertEqual(value, sot._get_id(sot)) + + def test__get_id_instance_alternate(self): + class Test(resource2.Resource): + attr = resource2.Body("attr", alternate_id=True) + + value = "id" + sot = Test(attr=value) + + self.assertEqual(value, sot._get_id(sot)) + + def test__get_id_value(self): + value = "id" + self.assertEqual(value, resource2.Resource._get_id(value)) + + def test_new(self): + class Test(resource2.Resource): + attr = resource2.Body("attr") + + value = "value" + sot = Test.new(attr=value) + + self.assertIn("attr", sot._body.dirty) + self.assertEqual(value, sot.attr) + + def test_existing(self): + class Test(resource2.Resource): + attr = resource2.Body("attr") + + value = "value" + sot = Test.existing(attr=value) + + self.assertNotIn("attr", sot._body.dirty) + self.assertEqual(value, sot.attr) + + def test__prepare_request_with_id(self): + class Test(resource2.Resource): + base_path = "/something" + body_attr = resource2.Body("x") + header_attr = resource2.Header("y") + + the_id = "id" + body_value = "body" + header_value = "header" + sot = Test(id=the_id, body_attr=body_value, header_attr=header_value, + synchronized=False) + + result = sot._prepare_request(requires_id=True) + + self.assertEqual("something/id", result.uri) + self.assertEqual({"x": body_value, "id": the_id}, result.body) + self.assertEqual({"y": header_value}, result.headers) + + def test__prepare_request_missing_id(self): + sot = resource2.Resource(id=None) + + self.assertRaises(exceptions.InvalidRequest, + sot._prepare_request, requires_id=True) + + def test__prepare_request_with_key(self): + key = "key" + + class Test(resource2.Resource): + base_path = "/something" + resource_key = key + body_attr = resource2.Body("x") + header_attr = resource2.Header("y") + + body_value = "body" + header_value = "header" + sot = Test(body_attr=body_value, header_attr=header_value, + synchronized=False) + + result = sot._prepare_request(requires_id=False, prepend_key=True) + + self.assertEqual("/something", result.uri) + self.assertEqual({key: {"x": body_value}}, result.body) + self.assertEqual({"y": header_value}, result.headers) + + def test__transpose_component(self): + client_name = "client_name" + server_name = "serverName" + value = "value" + # Include something in the mapping that we don't receive + # so the branch that looks at existence in the compoment is checked. + mapping = {client_name: server_name, "other": "blah"} + component = {server_name: value} + + sot = resource2.Resource() + result = sot._transpose_component(component, mapping) + + self.assertEqual({client_name: value}, result) + + def test__translate_response_no_body(self): + class Test(resource2.Resource): + attr = resource2.Header("attr") + + response = mock.Mock() + response.headers = dict() + + sot = Test() + sot._transpose_component = mock.Mock(return_value={"attr": "value"}) + + sot._translate_response(response, has_body=False) + + self.assertEqual(dict(), sot._header.dirty) + self.assertEqual("value", sot.attr) + + def test__translate_response_with_body_no_resource_key(self): + class Test(resource2.Resource): + attr = resource2.Body("attr") + + body = {"attr": "value"} + response = mock.Mock() + response.headers = dict() + response.json.return_value = body + + sot = Test() + sot._transpose_component = mock.Mock(side_effect=[body, dict()]) + + sot._translate_response(response, has_body=True) + + self.assertEqual("value", sot.attr) + self.assertEqual(dict(), sot._body.dirty) + self.assertEqual(dict(), sot._header.dirty) + + def test__translate_response_with_body_with_resource_key(self): + key = "key" + + class Test(resource2.Resource): + resource_key = key + attr = resource2.Body("attr") + + body = {"attr": "value"} + response = mock.Mock() + response.headers = dict() + response.json.return_value = {key: body} + + sot = Test() + sot._transpose_component = mock.Mock(side_effect=[body, dict()]) + + sot._translate_response(response, has_body=True) + + self.assertEqual("value", sot.attr) + self.assertEqual(dict(), sot._body.dirty) + self.assertEqual(dict(), sot._header.dirty) + + def test_cant_do_anything(self): + class Test(resource2.Resource): + allow_create = False + allow_get = False + allow_update = False + allow_delete = False + allow_head = False + allow_list = False + + sot = Test() + + # The first argument to all of these operations is the session, + # but we raise before we get to it so just pass anything in. + self.assertRaises(exceptions.MethodNotSupported, sot.create, "") + self.assertRaises(exceptions.MethodNotSupported, sot.get, "") + self.assertRaises(exceptions.MethodNotSupported, sot.delete, "") + self.assertRaises(exceptions.MethodNotSupported, sot.head, "") + + # list is a generator so you need to begin consuming + # it in order to exercise the failure. + the_list = sot.list("") + self.assertRaises(exceptions.MethodNotSupported, next, the_list) + + # Update checks the dirty list first before even trying to see + # if the call can be made, so fake a dirty list. + sot._body = mock.Mock() + sot._body.dirty = mock.Mock(return_value={"x": "y"}) + self.assertRaises(exceptions.MethodNotSupported, sot.update, "") + + +class TestResourceActions(base.TestCase): + + def setUp(self): + super(TestResourceActions, self).setUp() + + self.service_name = "service" + self.base_path = "base_path" + + class Test(resource2.Resource): + service = self.service_name + base_path = self.base_path + allow_create = True + allow_get = True + allow_head = True + allow_update = True + allow_delete = True + allow_list = True + + self.test_class = Test + + self.request = mock.Mock(spec=resource2._Request) + self.request.uri = "uri" + self.request.body = "body" + self.request.headers = "headers" + + self.response = mock.Mock() + + self.sot = Test(id="id") + self.sot._prepare_request = mock.Mock(return_value=self.request) + self.sot._translate_response = mock.Mock() + + self.session = mock.Mock(spec=session.Session) + self.session.create = mock.Mock(return_value=self.response) + self.session.get = mock.Mock(return_value=self.response) + self.session.put = mock.Mock(return_value=self.response) + self.session.patch = mock.Mock(return_value=self.response) + self.session.post = mock.Mock(return_value=self.response) + self.session.delete = mock.Mock(return_value=self.response) + self.session.head = mock.Mock(return_value=self.response) + + def _test_create(self, requires_id=False, prepend_key=False): + if not requires_id: + self.sot.id = None + + result = self.sot.create(self.session) + + self.sot._prepare_request.assert_called_once_with( + requires_id=requires_id, prepend_key=prepend_key) + if requires_id: + self.session.put.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name, + json=self.request.body, headers=self.request.headers) + else: + self.session.post.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name, + json=self.request.body, headers=self.request.headers) + + self.sot._translate_response.assert_called_once_with(self.response) + self.assertEqual(result, self.sot) + + def test_create_with_id(self): + self._test_create(requires_id=True, prepend_key=True) + + def test_create_without_id(self): + self._test_create(requires_id=False, prepend_key=True) + + def test_get(self): + result = self.sot.get(self.session) + + self.sot._prepare_request.assert_called_once_with() + self.session.get.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name) + + self.sot._translate_response.assert_called_once_with(self.response) + self.assertEqual(result, self.sot) + + def test_head(self): + result = self.sot.head(self.session) + + self.sot._prepare_request.assert_called_once_with() + self.session.head.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name, + headers={"Accept": ""}) + + self.sot._translate_response.assert_called_once_with(self.response) + self.assertEqual(result, self.sot) + + def _test_update(self, patch_update=False): + self.sot.patch_update = patch_update + + # Need to make sot look dirty so we can attempt an update + self.sot._body = mock.Mock() + self.sot._body.dirty = mock.Mock(return_value={"x": "y"}) + + result = self.sot.update(self.session) + + self.sot._prepare_request.assert_called_once_with(prepend_key=True) + + if patch_update: + self.session.patch.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name, + json=self.request.body, headers=self.request.headers) + else: + self.session.put.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name, + json=self.request.body, headers=self.request.headers) + + self.sot._translate_response.assert_called_once_with(self.response) + self.assertEqual(result, self.sot) + + def test_update_put(self): + self._test_update(patch_update=False) + + def test_update_patch(self): + self._test_update(patch_update=True) + + def test_update_not_dirty(self): + self.sot._body = mock.Mock() + self.sot._body.dirty = dict() + self.sot._header = mock.Mock() + self.sot._header.dirty = dict() + + result = self.sot.update(self.session) + self.assertEqual(result, self.sot) + + self.session.put.assert_not_called() + + def test_delete(self): + result = self.sot.delete(self.session) + + self.sot._prepare_request.assert_called_once_with() + self.session.delete.assert_called_once_with( + self.request.uri, + endpoint_filter=self.service_name, + headers={"Accept": ""}) + + self.sot._translate_response.assert_called_once_with( + self.response, has_body=False) + self.assertEqual(result, self.sot) + + # NOTE: As list returns a generator, testing it requires consuming + # the generator. Wrap calls to self.sot.list in a `list` + # and then test the results as a list of responses. + def test_list_empty_response(self): + mock_response = mock.Mock() + mock_response.json.return_value = [] + + self.session.get.return_value = mock_response + + result = list(self.sot.list(self.session)) + + self.session.get.assert_called_once_with( + self.base_path, + endpoint_filter=self.service_name, + headers={"Accept": "application/json"}, + params={}) + + self.assertEqual([], result) + + def test_list_one_page_response_paginated(self): + id_value = 1 + mock_response = mock.Mock() + mock_response.json.side_effect = [[{"id": id_value}], + []] + + self.session.get.return_value = mock_response + + # Ensure that we break out of the loop on a paginated call + # that still only results in one page of data. + results = list(self.sot.list(self.session, paginated=True)) + + self.assertEqual(1, len(results)) + + # Look at the `params` argument to each of the get calls that + # were made. + self.session.get.call_args_list[0][1]["params"] = {} + self.session.get.call_args_list[1][1]["params"] = {"marker": id_value} + self.assertEqual(id_value, results[0].id) + self.assertIsInstance(results[0], self.test_class) + + def test_list_one_page_response_not_paginated(self): + id_value = 1 + mock_response = mock.Mock() + mock_response.json.return_value = [{"id": id_value}] + + self.session.get.return_value = mock_response + + results = list(self.sot.list(self.session, paginated=False)) + + self.session.get.assert_called_once_with( + self.base_path, + endpoint_filter=self.service_name, + headers={"Accept": "application/json"}, + params={}) + + self.assertEqual(1, len(results)) + self.assertEqual(id_value, results[0].id) + self.assertIsInstance(results[0], self.test_class) + + def test_list_one_page_response_resources_key(self): + key = "resources" + + class Test(self.test_class): + resources_key = key + + id_value = 1 + mock_response = mock.Mock() + mock_response.json.return_value = {key: [{"id": id_value}]} + + self.session.get.return_value = mock_response + + sot = Test() + + results = list(sot.list(self.session)) + + self.session.get.assert_called_once_with( + self.base_path, + endpoint_filter=self.service_name, + headers={"Accept": "application/json"}, + params={}) + + self.assertEqual(1, len(results)) + self.assertEqual(id_value, results[0].id) + self.assertIsInstance(results[0], self.test_class) + + def test_list_multi_page_response_not_paginated(self): + ids = [1, 2] + mock_response = mock.Mock() + mock_response.json.side_effect = [[{"id": ids[0]}], + [{"id": ids[1]}]] + + self.session.get.return_value = mock_response + + results = list(self.sot.list(self.session, paginated=False)) + + self.assertEqual(1, len(results)) + self.assertEqual(ids[0], results[0].id) + self.assertIsInstance(results[0], self.test_class) + + def test_list_query_params(self): + id = 1 + qp = "query param!" + qp_name = "query-param" + uri_param = "uri param!" + + mock_response = mock.Mock() + mock_response.json.side_effect = [[{"id": id}], + []] + + self.session.get.return_value = mock_response + + class Test(self.test_class): + _query_mapping = resource2.QueryParameters(query_param=qp_name) + base_path = "/%(something)s/blah" + something = resource2.URI("something") + + results = list(Test.list(self.session, paginated=True, + query_param=qp, something=uri_param)) + + self.assertEqual(1, len(results)) + + # Look at the `params` argument to each of the get calls that + # were made. + self.session.get.call_args_list[0][1]["params"] = {qp_name: qp} + + self.assertEqual(self.session.get.call_args_list[0][0][0], + Test.base_path % {"something": uri_param}) + + def test_list_multi_page_response_paginated(self): + ids = [1, 2, 3] + mock_response = mock.Mock() + mock_response.json.side_effect = [[{"id": ids[0]}], + [{"id": ids[1]}], + [{"id": ids[2]}], + []] + + self.session.get.return_value = mock_response + + results = list(self.sot.list(self.session, paginated=True)) + + self.assertEqual(3, len(results)) + + # Look at the `params` argument to each of the get calls that + # were made. + self.session.get.call_args_list[0][1]["params"] = {} + self.session.get.call_args_list[1][1]["params"] = {"marker": ids[0]} + self.session.get.call_args_list[2][1]["params"] = {"marker": ids[1]} + # We determine the limit based on the first response's size + self.session.get.call_args_list[1][1]["params"] = {"limit": 1} + self.session.get.call_args_list[2][1]["params"] = {"limit": 1} + + for call, result in enumerate(results): + # Look at the first item in the tuple of *args that is in the + # first item in the list of call_args for this call in the + # call_args_list. This represents any positional arguments + # to the session.get call. + self.assertEqual(self.session.get.call_args_list[call][0][0], + self.base_path) + self.assertEqual( + self.session.get.call_args_list[call][1]["endpoint_filter"], + self.service_name) + self.assertEqual( + self.session.get.call_args_list[call][1]["headers"], + {"Accept": "application/json"}) + + self.assertEqual(ids[call], results[call].id) + self.assertIsInstance(results[call], self.test_class) + + def test_list_multi_page_early_termination(self): + ids = [1, 2, 3] + mock_response = mock.Mock() + mock_response.json.side_effect = [[{"id": ids[0]}, {"id": ids[1]}], + [{"id": ids[2]}]] + + self.session.get.return_value = mock_response + + results = list(self.sot.list(self.session, paginated=True)) + + self.assertEqual(3, len(results)) + + # We should know that we received less than a full page of + # results from the second request, so ensure that we don't + # make a third call. + self.assertEqual(2, len(self.session.get.call_args_list)) + + # Look at the `params` argument to each of the get calls that + # were made. + self.session.get.call_args_list[0][1]["params"] = {} + self.session.get.call_args_list[1][1]["params"] = {"marker": ids[1]} + + for call in self.session.get.call_args_list: + # Look at the first item in the tuple of *args that is in the + # first item in the list of call_args for this call in the + # call_args_list. This represents any positional arguments + # to the session.get call. + self.assertEqual(call[0][0], self.base_path) + self.assertEqual(call[1]["endpoint_filter"], self.service_name) + self.assertEqual(call[1]["headers"], + {"Accept": "application/json"}) + + for call, result in enumerate(results): + self.assertEqual(ids[call], results[call].id) + self.assertIsInstance(results[call], self.test_class) + + +class TestResourceFind(base.TestCase): + + def setUp(self): + super(TestResourceFind, self).setUp() + + self.result = 1 + + class Base(resource2.Resource): + + @classmethod + def existing(cls, **kwargs): + raise exceptions.NotFoundException + + @classmethod + def list(cls, session): + return None + + class OneResult(Base): + + @classmethod + def _get_one_match(cls, *args): + return self.result + + class NoResults(Base): + + @classmethod + def _get_one_match(cls, *args): + return None + + self.no_results = NoResults + self.one_result = OneResult + + def test_find_short_circuit(self): + value = 1 + + class Test(resource2.Resource): + + @classmethod + def existing(cls, **kwargs): + mock_match = mock.Mock() + mock_match.get.return_value = value + return mock_match + + result = Test.find("session", "name") + + self.assertEqual(result, value) + + def test_no_match_raise(self): + self.assertRaises(exceptions.ResourceNotFound, self.no_results.find, + "session", "name", ignore_missing=False) + + def test_no_match_return(self): + self.assertIsNone( + self.no_results.find("session", "name", ignore_missing=True)) + + def test_find_result(self): + self.assertEqual(self.result, self.one_result.find("session", "name")) + + def test_match_empty_results(self): + self.assertIsNone(resource2.Resource._get_one_match("name", [])) + + def test_no_match_by_name(self): + the_name = "Brian" + + match = mock.Mock(spec=resource2.Resource) + match.name = the_name + + result = resource2.Resource._get_one_match("Richard", [match]) + + self.assertIsNone(result, match) + + def test_single_match_by_name(self): + the_name = "Brian" + + match = mock.Mock(spec=resource2.Resource) + match.name = the_name + + result = resource2.Resource._get_one_match(the_name, [match]) + + self.assertIs(result, match) + + def test_single_match_by_id(self): + the_id = "Brian" + + match = mock.Mock(spec=resource2.Resource) + match.id = the_id + + result = resource2.Resource._get_one_match(the_id, [match]) + + self.assertIs(result, match) + + def test_single_match_by_alternate_id(self): + the_id = "Richard" + + class Test(resource2.Resource): + other_id = resource2.Body("other_id", alternate_id=True) + + match = Test(other_id=the_id) + result = Test._get_one_match(the_id, [match]) + + self.assertIs(result, match) + + def test_multiple_matches(self): + the_id = "Brian" + + match = mock.Mock(spec=resource2.Resource) + match.id = the_id + + self.assertRaises( + exceptions.DuplicateResource, + resource2.Resource._get_one_match, the_id, [match, match]) + + +class TestWaitForStatus(base.TestCase): + + def test_immediate_status(self): + status = "loling" + resource = mock.Mock() + resource.status = status + + result = resource2.wait_for_status("session", resource, status, + "failures", "interval", "wait") + + self.assertEqual(result, resource) + + @mock.patch("time.sleep", return_value=None) + def test_status_match(self, mock_sleep): + status = "loling" + resource = mock.Mock() + + # other gets past the first check, two anothers gets through + # the sleep loop, and the third matches + statuses = ["other", "another", "another", status] + type(resource).status = mock.PropertyMock(side_effect=statuses) + + result = resource2.wait_for_status("session", resource, status, + None, 1, 5) + + self.assertEqual(result, resource) + + @mock.patch("time.sleep", return_value=None) + def test_status_fails(self, mock_sleep): + status = "loling" + failure = "crying" + resource = mock.Mock() + + # other gets past the first check, the first failure doesn't + # match the expected, the third matches the failure, + # the fourth is used in creating the exception message + statuses = ["other", failure, failure, failure] + type(resource).status = mock.PropertyMock(side_effect=statuses) + + self.assertRaises(exceptions.ResourceFailure, + resource2.wait_for_status, + "session", resource, status, [failure], 1, 5) + + @mock.patch("time.sleep", return_value=None) + def test_timeout(self, mock_sleep): + status = "loling" + resource = mock.Mock() + + # The first "other" gets past the first check, and then three + # pairs of "other" statuses run through the sleep counter loop, + # after which time should be up. This is because we have a + # one second interval and three second waiting period. + statuses = ["other"] * 7 + type(resource).status = mock.PropertyMock(side_effect=statuses) + + self.assertRaises(exceptions.ResourceTimeout, + resource2.wait_for_status, + "session", resource, status, None, 1, 3) + + def test_no_sleep(self): + resource = mock.Mock() + statuses = ["other"] + type(resource).status = mock.PropertyMock(side_effect=statuses) + + self.assertRaises(exceptions.ResourceTimeout, + resource2.wait_for_status, + "session", resource, "status", None, 0, -1) + + +class TestWaitForDelete(base.TestCase): + + @mock.patch("time.sleep", return_value=None) + def test_success(self, mock_sleep): + resource = mock.Mock() + resource.get.side_effect = [None, None, exceptions.NotFoundException] + + result = resource2.wait_for_delete("session", resource, 1, 3) + + self.assertEqual(result, resource) + + @mock.patch("time.sleep", return_value=None) + def test_timeout(self, mock_sleep): + resource = mock.Mock() + resource.get.side_effect = [None, None, None] + + self.assertRaises(exceptions.ResourceTimeout, + resource2.wait_for_delete, + "session", resource, 1, 3)