openstack/trove
Code Issues Proposed changes

Image tags support in datastore version

Browse Source 
Change-Id: I0e51d08515c121e3a7b0e82b6e4c4161bb4fbc4a
This commit is contained in:
Lingxian Kong 2020-10-07 18:50:53 +13:00
parent 680c7b8361
commit 1d24b65052
34 changed files with 721 additions and 785 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
api-ref/source/datastore-versions.inc
View File

@ -286,6 +286,13 @@ Request
.. rest_parameters:: parameters.yaml
- project_id: project_id
- name: datastore_version_name
- datastore_name: datastore_name_required
- datastore_manager: datastore_type1
- image: image_id
- image_tags: image_tags
- active: active
- default: default
Request Example
---------------
@ -363,6 +370,11 @@ Request
- project_id: project_id
- datastore_version_id: datastore_version_id
- datastore_manager: datastore_type
- image: image_id
- image_tags: image_tags
- active: active_optional
- default: default
Request Example
---------------

53
api-ref/source/parameters.yaml
View File

@ -99,6 +99,18 @@ access_is_public:
in: body
required: false
type: boolean
active:
description: |
Whether the database version is enabled.
in: body
required: true
type: boolean
active_optional:
description: |
Whether the database version is enabled.
in: body
required: false
type: boolean
availability_zone:
description: |
The availability zone of the instance.
@ -302,6 +314,12 @@ datastore2:
in: body
required: true
type: object
datastore_name_required:
description: |
The name of a datastore.
in: body
required: true
type: string
datastore_type:
description: |
The type of a datastore.
@ -339,6 +357,14 @@ datastore_version_name:
in: body
required: true
type: string
default:
description: |
When true this datastore version is created as the default in the
datastore. If not specified, for creating, default is false, for updating,
it's ignored.
in: body
required: false
type: boolean
description:
description: |
New description of the configuration group.
@ -392,6 +418,33 @@ flavorRef:
in: body
required: true
type: string
image_id:
description: |
The ID of an image.
Either ``image`` or ``image_tags`` needs to be specified when creating
datastore version.
in: body
required: false
type: string
image_tags:
description: |
List of image tags.
Either ``image`` or ``image_tags`` needs to be specified when creating
datastore version.
If the image ID is not provided, the image can be retrieved by the image
tags. The tags are used for filtering as a whole rather than separately.
Using image tags is more flexible than ID especially when a new guest image
is uploaded to Glance, Trove can pick up the latest image automatically for
creating instances.
When updating, only specifying ``image_tags`` could remove ``image``
from the datastore version.
in: body
required: false
type: array
instance:
description: |
An ``instance`` object.

2
devstack/plugin.sh
View File

@ -481,7 +481,7 @@ function create_guest_image {
echo "Register the image in datastore"
$TROVE_MANAGE datastore_update $TROVE_DATASTORE_TYPE ""
$TROVE_MANAGE datastore_version_update $TROVE_DATASTORE_TYPE $TROVE_DATASTORE_VERSION $TROVE_DATASTORE_TYPE $glance_image_id "" 1
$TROVE_MANAGE datastore_version_update $TROVE_DATASTORE_TYPE $TROVE_DATASTORE_VERSION $TROVE_DATASTORE_TYPE $glance_image_id "trove" "" 1
$TROVE_MANAGE datastore_update $TROVE_DATASTORE_TYPE $TROVE_DATASTORE_VERSION
echo "Add parameter validation rules if available"

18
doc/source/admin/building_guest_images.rst
View File

@ -3,15 +3,9 @@
.. role:: bash(code)
:language: bash
=========================================
Building Guest Images for OpenStack Trove
=========================================
.. If section numbers are desired, unindent this
.. sectnum::
.. If a TOC is desired, unindent this
.. contents::
====================
Building guest image
====================
Overview
========
@ -199,14 +193,16 @@ image in Glance and register a new datastore or version in Trove using
--private \
--disk-format qcow2 \
--container-format bare \
--tag trove --tag mysql \
--file ~/images/trove-guest-ubuntu-bionic-dev.qcow2
$ trove-manage datastore_version_update mysql 5.7.29 mysql $image_id "" 1
$ openstack datastore version create 5.7.29 mysql mysql "" \
--image-tags trove,mysql \
--active --default
$ trove-manage db_load_datastore_config_parameters mysql 5.7.29 ${trove_repo_dir}/trove/templates/mysql/validation-rules.json
.. note::
The command ``trove-manage`` needs to run on Trove controller node.
Otherwise, you can use ``openstack datastore version create`` CLI.
If you see anything error or need help for the image creation, please ask help
either in ``#openstack-trove`` IRC channel or sending emails to

212
doc/source/admin/datastore.rst
View File

@ -4,44 +4,44 @@
Datastore
=========
The Database service provides database management features.
Introduction
~~~~~~~~~~~~
The Database service provides scalable and reliable cloud
provisioning functionality for both relational and non-relational
database engines. Users can quickly and easily use database features
without the burden of handling complex administrative tasks. Cloud
users and database administrators can provision and manage multiple
database instances as needed.
A datastore is typically created as a type of database. For each datastore,
there could be multiple datastore versions. For example, for MySQL database,
Trove could support 5.7.29, 5.7.30 or 5.8.
The Database service provides resource isolation at high performance
levels, and automates complex administrative tasks such as deployment,
configuration, patching, backups, restores, and monitoring.
Admin user needs to create datastore and its versions as required.
Create datastore
~~~~~~~~~~~~~~~~
A datastore version is always associated with a Glance image, either by image
ID or image tags. If the image ID is not provided, the image can be retrieved
by the image tags. The tags are used for filtering as a whole rather than
separately. Using image tags is more flexible than ID especially when a new
guest image is uploaded to Glance, Trove can pick up the latest image
automatically for creating instances.
An administrative user can create datastores for a variety of databases.
Create datastore version
~~~~~~~~~~~~~~~~~~~~~~~~
This section assumes you do not yet have a MySQL data store, and shows
you how to create a MySQL data store and populate it with a MySQL 5.5
data store version.
When creating a datastore version, Trove will create the datastore first if it
doesn't exist.
When using image tags, make sure the image with the tags exists before creating
the datastore version.
.. note::
From Victoria release, all the datastores can be configured with a same
Glance image but with different datastore name and version number.
Glance image but with different datastore name and version name.
**To create a data store**
To create a datastore version:
#. **Create a trove image**
#. Create a trove guest image
Refer to `Build images using trovestack
<https://docs.openstack.org/trove/latest/admin/building_guest_images.html#build-images-using-trovestack>`_
#. **Register image with Image service**
#. Register image with Image service
You need to register your guest image with the Image service as cloud admin.
@ -53,133 +53,17 @@ data store version.
--disk-format qcow2 --container-format bare \
--file $image_file \
--property hw_rng_model='virtio' \
--tag trove
--tag trove --tag mysql
#. **Create the datastore**
Create the data store that configured with the new image. To do this, use
the :command:`trove-manage` :command:`datastore_update` command.
This example uses the following arguments:
.. list-table::
:header-rows: 1
:widths: 20 20 20
* - Argument
- Description
- In this example:
* - config file
- The configuration file to use.
- ``--config-file=/etc/trove/trove.conf``
* - name
- Name you want to use for this data store.
- ``mysql``
* - default version
- You can attach multiple versions/images to a data store. For
example, you might have a MySQL 5.5 version and a MySQL 5.6
version. You can designate one version as the default, which
the system uses if a user does not explicitly request a
specific version.
- ``""``
At this point, you do not yet have a default version, so pass
in an empty string.
|
Example:
#. Create the datastore version
.. code-block:: console
$ trove-manage --config-file=/etc/trove/trove.conf datastore_update mysql ""
openstack datastore version create 5.7.29 mysql mysql "" \
--image-tags trove,mysql \
--active --default
#. **Add a version to the new data store**
Now that you have a MySQL data store, you can add a version to it,
using the :command:`trove-manage` :command:`datastore_version_update`
command. The version indicates which guest image to use.
This example uses the following arguments:
.. list-table::
:header-rows: 1
:widths: 20 20 20
* - Argument
- Description
- In this example:
* - config file
- The configuration file to use.
- ``--config-file=/etc/trove/trove.conf``
* - data store
- The name of the data store you just created via
``trove-manage`` :command:`datastore_update`.
- ``mysql``
* - version name
- The name of the version you are adding to the data store.
- ``mysql-5.5``
* - data store manager
- Which data store manager to use for this version. Typically,
the data store manager is identified by one of the following
strings, depending on the database:
* cassandra
* couchbase
* couchdb
* db2
* mariadb
* mongodb
* mysql
* percona
* postgresql
* pxc
* redis
* vertica
- ``mysql``
* - glance ID
- The ID of the guest image you just added to the Image
service. You can get this ID by using the glance
:command:`image-show` IMAGE_NAME command.
- bb75f870-0c33-4907-8467-1367f8cb15b6
* - packages
- If you want to put additional packages on each guest that
you create with this data store version, you can list the
package names here.
- ``""``
In this example, the guest image already contains all the
required packages, so leave this argument empty.
* - active
- Set this to either 1 or 0:
* ``1`` = active
* ``0`` = disabled
- 1
|
Example:
.. code-block:: console
$ trove-manage --config-file=/etc/trove/trove.conf datastore_version_update mysql mysql-5.5 mysql GLANCE_ID "" 1
**Optional.** Set your new version as the default version. To do
this, use the :command:`trove-manage` :command:`datastore_update`
command again, this time specifying the version you just created.
.. code-block:: console
$ trove-manage --config-file=/etc/trove/trove.conf datastore_update mysql mysql-5.5
#. **Load validation rules for configuration groups**
#. Load validation rules for configuration groups
**Background.** You can manage database configuration tasks by using
configuration groups. Configuration groups let you set configuration
@ -200,21 +84,20 @@ data store version.
* - Ubuntu 18.04
- :file:`/usr/lib/python3/dist-packages/trove/templates/DATASTORE_NAME`
- DATASTORE_NAME is the name of either the MySQL data store or
the Percona data store. This is typically either ``mysql``
or ``percona``.
- DATASTORE_NAME is the name of the datastore, e.g. ``mysql``
or ``postgresql``.
* - RHEL 7, CentOS 7, Fedora 20, and Fedora 21
- :file:`/usr/lib/python3/site-packages/trove/templates/DATASTORE_NAME`
- DATASTORE_NAME is the name of either the MySQL data store or
the Percona data store. This is typically either ``mysql`` or ``percona``.
- DATASTORE_NAME is the name of the datastore, e.g. ``mysql``
or ``postgresql``.
|
Therefore, as part of creating a data store, you need to load the
``validation-rules.json`` file, using the :command:`trove-manage`
:command:`db_load_datastore_config_parameters` command. This command
takes the following arguments:
:command:`db_load_datastore_config_parameters` command on trove controller
node. This command takes the following arguments:
* Data store name
* Data store version
@ -227,30 +110,15 @@ data store version.
.. code-block:: console
$ trove-manage db_load_datastore_config_parameters mysql mysql-5.5 /usr/lib/python3/dist-packages/trove/templates/mysql/validation-rules.json
$ trove-manage db_load_datastore_config_parameters mysql 5.7.29 /usr/lib/python3/dist-packages/trove/templates/mysql/validation-rules.json
#. **Validate data store**
Hide a datastore version
~~~~~~~~~~~~~~~~~~~~~~~~
To validate your new data store and version, start by listing the
data stores on your system:
Sometimes, it's needed to make a datastore version invisible to the cloud
users, e.g when a datastore version is deprecated or creating a datastore
version for testing purpose, to do that:
.. code-block:: console
$ openstack datastore list
+--------------------------------------+--------------+
| id | name |
+--------------------------------------+--------------+
| 10000000-0000-0000-0000-000000000001 | Legacy MySQL |
| e5dc1da3-f080-4589-a4c2-eff7928f969a | mysql |
+--------------------------------------+--------------+
Show the versions of a specific datastore:
.. code-block:: console
$ openstack datastore version list mysql
+--------------------------------------+-----------+
| id | name |
+--------------------------------------+-----------+
| 36a6306b-efd8-4d83-9b75-8b30dd756381 | mysql-5.5 |
+--------------------------------------+-----------+
$ openstack datastore version <version-id> --disable

19
doc/source/admin/run_trove_in_production.rst
View File

@ -313,7 +313,8 @@ for more information. Make sure to use ``dev_mode=false`` for production
environment.
After image is created successfully, the cloud administrator needs to upload
the image to Glance and make it only accessible to service users.
the image to Glance and make it only accessible to service users. It's
recommended to use tags when creating Glance image.
Preparing the Datastore
@ -323,18 +324,18 @@ datastore versions and the configuration parameters for the particular version.
It's recommended to config a default version for each datastore.
``trove-manage`` can be only used on trove controller node.
Command examples:
.. code-block:: console
# Create a new datastore 'mysql'
trove-manage datastore_update mysql ""
# Create a new datastore version 5.7.29 for 'mysql'
trove-manage datastore_version_update mysql 5.7.29 mysql $imageid "" 1
# Use 5.7.29 as the default datastore version for 'mysql'
trove-manage datastore_update mysql 5.7.29
# Register configuration parameters for 5.7.29 version of datastore 'mysql'
trove-manage db_load_datastore_config_parameters mysql 5.7.29 ${trove_repo_dir}}/trove/templates/mysql/validation-rules.json
$ # Creating datastore 'mysql' and datastore version 5.7.29.
$ openstack datastore version create 5.7.29 mysql mysql "" \
--image-tags trove,mysql \
--active --default
$ # Register configuration parameters for the datastore version
$ trove-manage db_load_datastore_config_parameters mysql 5.7.29 ${trove_repo_dir}}/trove/templates/mysql/validation-rules.json
Quota Management

2
doc/source/admin/upgrade.rst
View File

@ -157,7 +157,7 @@ Upgrade Trove services
--property hw_rng_model='virtio' \
--tag trove \
-c id -f value)
$ trove-manage datastore_version_update mysql 5.7.29 mysql $imageid "" 1
$ trove-manage datastore_version_update mysql 5.7.29 mysql $imageid "" "" 1
$ trove-manage db_load_datastore_config_parameters mysql 5.7.29 $stackdir/trove/trove/templates/mysql/validation-rules.json
Upgrade Trove guest agent

9
doc/source/cli/trove-manage.rst
View File

@ -200,7 +200,7 @@ trove-manage datastore_version_update
usage: trove-manage datastore_version_update [-h]
datastore version_name manager
image_id packages active
image_id image_tags packages active
Add or update a datastore version. If the datastore version already exists,
all values except the datastore name and version will be updated.
@ -219,6 +219,13 @@ all values except the datastore name and version will be updated.
``image_id``
ID of the image used to create an instance of the datastore version.
``image_tags``
List of image tags separated by comma. If the image ID is not provided
explicitly, the image can be retrieved by the image tags. Multiple image tags
are separated by comma, e.g. trove,mysql. Using image tags is more flexible
than ID especially when new guest image is uploaded to Glance, Trove can pick
up the latest image automatically for creating instances.
``packages``
Packages required by the datastore version that are installed on
the guest image.

10
doc/source/install/verify.rst
View File

@ -27,15 +27,13 @@ Verify operation of the Database service.
Create an image for the type of database you want to use, for example,
MySQL, MariaDB, etc.
* Create a datastore. You need to create a separate datastore for
each type of database you want to use, for example, MySQL, MongoDB,
Cassandra. This example shows you how to create a datastore for a
MySQL database:
* Create a datastore. You need to create at least one datastore version for
each type of database supported. This example creates a datastore version
for MySQL 5.7.29:
.. code-block:: console
$ trove-manage datastore_update mysql ""
$ trove-manage datastore_version_update mysql 5.7 mysql $imageid "" 1
$ openstack datastore version create 5.7.29 mysql mysql "" trove,mysql --active --default
#. Create a database `instance
<http://docs.openstack.org/user-guide/create_db.html>`_.

4
integration/scripts/trovestack
View File

@ -524,8 +524,8 @@ function cmd_set_datastore() {
local IMAGEID=$1
rd_manage datastore_update "$datastore" ""
# trove-manage datastore_version_update <datastore_name> <version_name> <datastore_manager> <image_id> <packages> <active>
rd_manage datastore_version_update "${DATASTORE_TYPE}" "${DATASTORE_VERSION}" "${DATASTORE_TYPE}" $IMAGEID "" 1
# trove-manage datastore_version_update <datastore_name> <version_name> <datastore_manager> <image_id> <image_tags> <packages> <active>
rd_manage datastore_version_update "${DATASTORE_TYPE}" "${DATASTORE_VERSION}" "${DATASTORE_TYPE}" $IMAGEID "" "" 1
rd_manage datastore_update "${DATASTORE_TYPE}" "${DATASTORE_VERSION}"
if [[ -f "$PATH_TROVE"/trove/templates/${DATASTORE_TYPE}/validation-rules.json ]]; then

5
releasenotes/notes/wallaby-datastore-version-image-tags.yaml Normal file
View File

@ -0,0 +1,5 @@
---
features:
- Support image tags for the datastore version. When using image tags, Trove
is able to get the image dynamically from Glance for creating instances. If
both are specified, image ID takes precedence over image tags.

10
trove/cmd/manage.py
View File

@ -62,12 +62,13 @@ class Commands(object):
print(e)
def datastore_version_update(self, datastore, version_name, manager,
image_id, packages, active):
image_id, image_tags, packages, active):
try:
datastore_models.update_datastore_version(datastore,
version_name,
manager,
image_id,
image_tags,
packages, active)
print("Datastore version '%s' updated." % version_name)
except exception.DatastoreNotFound as e:
@ -208,8 +209,13 @@ def main():
'manager', help='Name of the manager that will administer the '
'datastore version.')
parser.add_argument(
'image_id', help='ID of the image used to create an instance of '
'image_id',
help='ID of the image used to create an instance of '
'the datastore version.')
parser.add_argument(
'image_tags',
help='List of image tags separated by comma used for getting '
'guest image.')
parser.add_argument(
'packages', help='Packages required by the datastore version that '
'are installed on the guest image.')

22
trove/common/apischema.py
View File

@ -947,6 +947,19 @@ package_list = {
}
}
image_tags = {
"type": "array",
"minItems": 0,
"maxItems": 5,
"uniqueItems": True,
"items": {
"type": "string",
"minLength": 1,
"maxLength": 20,
"pattern": "^.*[0-9a-zA-Z]+.*$"
}
}
mgmt_datastore_version = {
"create": {
"name": "mgmt_datastore_version:create",
@ -955,14 +968,16 @@ mgmt_datastore_version = {
"properties": {
"version": {
"type": "object",
"required": ["name", "datastore_name", "image", "active"],
"additionalProperties": True,
"required": ["name", "datastore_name", "datastore_manager",
"active"],
"additionalProperties": False,
"properties": {
"name": non_empty_string,
"datastore_name": non_empty_string,
"datastore_manager": non_empty_string,
"packages": package_list,
"image": uuid,
"image_tags": image_tags,
"active": {"enum": [True, False]},
"default": {"enum": [True, False]}
}
@ -973,11 +988,12 @@ mgmt_datastore_version = {
"name": "mgmt_datastore_version:edit",
"type": "object",
"required": [],
"additionalProperties": True,
"additionalProperties": False,
"properties": {
"datastore_manager": non_empty_string,
"packages": package_list,
"image": uuid,
"image_tags": image_tags,
"active": {"enum": [True, False]},
"default": {"enum": [True, False]},
}

5
trove/common/exception.py
View File

@ -699,6 +699,11 @@ class ImageNotFound(NotFound):
message = _("Image %(uuid)s cannot be found.")
class ImageNotFoundByTags(NotFound):
message = _("Failed to retrieve image with tags: %(tags)s.")
class LogAccessForbidden(Forbidden):
message = _("You must be admin to %(action)s log '%(log)s'.")

36
trove/common/glance.py Normal file
View File

@ -0,0 +1,36 @@
# Copyright 2020 Catalyst Cloud
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from glanceclient import exc as glance_exceptions
from trove.common import exception
def get_image_id(client, image_id, image_tags):
"""Get and check image ID."""
if image_id:
try:
client.images.get(image_id)
except glance_exceptions.HTTPNotFound:
raise exception.ImageNotFound(uuid=image_id)
return image_id
elif image_tags:
filters = {'tag': image_tags, 'status': 'active'}
images = list(client.images.list(
filters=filters, sort='created_at:desc', limit=1))
if not images:
raise exception.ImageNotFoundByTags(tags=image_tags)
image_id = images[0]['id']
return image_id

15
trove/datastore/models.py
View File

@ -62,9 +62,8 @@ class DBCapabilityOverrides(dbmodels.DatabaseModelBase):
class DBDatastoreVersion(dbmodels.DatabaseModelBase):
_data_fields = ['datastore_id', 'name', 'image_id', 'packages',
'active', 'manager']
_data_fields = ['datastore_id', 'name', 'image_id', 'image_tags',
'packages', 'active', 'manager']
_table_name = 'datastore_versions'
@ -447,6 +446,10 @@ class DatastoreVersion(object):
def image_id(self):
return self.db_info.image_id
@property
def image_tags(self):
return self.db_info.image_tags
@property
def packages(self):
return self.db_info.packages
@ -577,8 +580,8 @@ def update_datastore(name, default_version):
db_api.save(datastore)
def update_datastore_version(datastore, name, manager, image_id, packages,
active):
def update_datastore_version(datastore, name, manager, image_id, image_tags,
packages, active):
db_api.configure_db(CONF)
datastore = Datastore.load(datastore)
try:
@ -592,6 +595,8 @@ def update_datastore_version(datastore, name, manager, image_id, packages,
version.datastore_id = datastore.id
version.manager = manager
version.image_id = image_id
version.image_tags = (",".join(image_tags)
if type(image_tags) is list else image_tags)
version.packages = packages
version.active = active

5
trove/datastore/views.py
View File

@ -90,6 +90,11 @@ class DatastoreVersionView(object):
datastore_version_dict['packages'] = (self.datastore_version.
packages)
datastore_version_dict['image'] = self.datastore_version.image_id
image_tags = []
if self.datastore_version.image_tags:
image_tags = self.datastore_version.image_tags.split(',')
datastore_version_dict['image_tags'] = image_tags
return {"version": datastore_version_dict}
def _build_links(self):

29
trove/db/sqlalchemy/migrate_repo/versions/047_image_tag_in_datastore_version.py Normal file
View File

@ -0,0 +1,29 @@
# Copyright 2020 Catalyst Cloud
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from sqlalchemy.schema import Column
from sqlalchemy.schema import MetaData
from trove.db.sqlalchemy.migrate_repo.schema import Table
from trove.db.sqlalchemy.migrate_repo.schema import String
def upgrade(migrate_engine):
meta = MetaData()
meta.bind = migrate_engine
ds_version = Table('datastore_versions', meta, autoload=True)
ds_version.create_column(Column('image_tags', String(255), nullable=True))
ds_version.c.image_id.alter(nullable=True)

51
trove/extensions/mgmt/datastores/service.py
View File

@ -12,17 +12,16 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from glanceclient import exc as glance_exceptions
from oslo_log import log as logging
from trove.backup import models as backup_model
from trove.common import apischema
from trove.common.auth import admin_context
from trove.common import clients
from trove.common import exception
from trove.common import glance as common_glance
from trove.common import utils
from trove.common import wsgi
from trove.common.auth import admin_context
from trove.configuration import models as config_model
from trove.datastore import models
from trove.extensions.mgmt.datastores import views
@ -43,23 +42,24 @@ class DatastoreVersionController(wsgi.Controller):
datastore_name = body['version']['datastore_name']
version_name = body['version']['name']
manager = body['version']['datastore_manager']
image_id = body['version']['image']
packages = body['version']['packages']
image_id = body['version'].get('image')
image_tags = body['version'].get('image_tags')
packages = body['version'].get('packages')
if type(packages) is list:
packages = ','.join(packages)
active = body['version']['active']
default = body['version']['default']
default = body['version'].get('default', False)
LOG.info("Tenant: '%(tenant)s' is adding the datastore "
"version: '%(version)s' to datastore: '%(datastore)s'",
{'tenant': tenant_id, 'version': version_name,
'datastore': datastore_name})
if not image_id and not image_tags:
raise exception.BadRequest("Image must be specified.")
client = clients.create_glance_client(context)
try:
client.images.get(image_id)
except glance_exceptions.HTTPNotFound:
raise exception.ImageNotFound(uuid=image_id)
common_glance.get_image_id(client, image_id, image_tags)
try:
datastore = models.Datastore.load(datastore_name)
@ -76,8 +76,8 @@ class DatastoreVersionController(wsgi.Controller):
raise exception.DatastoreVersionAlreadyExists(name=version_name)
except exception.DatastoreVersionNotFound:
models.update_datastore_version(datastore.name, version_name,
manager, image_id, packages,
active)
manager, image_id, image_tags,
packages, active)
if default:
models.update_datastore(datastore.name, version_name)
@ -114,23 +114,36 @@ class DatastoreVersionController(wsgi.Controller):
'datastore': datastore_version.datastore_name})
manager = body.get('datastore_manager', datastore_version.manager)
image_id = body.get('image', datastore_version.image_id)
image_id = body.get('image')
image_tags = body.get('image_tags')
active = body.get('active', datastore_version.active)
default = body.get('default', None)
packages = body.get('packages', datastore_version.packages)
if type(packages) is list:
packages = ','.join(packages)
if image_id or image_tags:
client = clients.create_glance_client(context)
try:
client.images.get(image_id)
except glance_exceptions.HTTPNotFound:
raise exception.ImageNotFound(uuid=image_id)
common_glance.get_image_id(client, image_id, image_tags)
if not image_id and image_tags:
# Remove the image ID from the datastore version.
image_id = ""
if image_id is None:
image_id = datastore_version.image_id
if image_tags is None:
image_tags = datastore_version.image_tags
if type(image_tags) is str:
image_tags = image_tags.split(',')
if not image_id and not image_tags:
raise exception.BadRequest("Image must be specified.")
models.update_datastore_version(datastore_version.datastore_name,
datastore_version.name,
manager, image_id, packages,
active)
manager, image_id, image_tags,
packages, active)
if default:
models.update_datastore(datastore_version.datastore_name,

2
trove/extensions/mgmt/datastores/views.py
View File

@ -26,6 +26,8 @@ class DatastoreVersionView(object):
"datastore_name": self.datastore_version.datastore_name,
"datastore_manager": self.datastore_version.manager,
"image": self.datastore_version.image_id,
"image_tags": (self.datastore_version.image_tags.split(',')
if self.datastore_version.image_tags else ['']),
"packages": (self.datastore_version.packages.split(
',') if self.datastore_version.packages else ['']),
"active": self.datastore_version.active,

8
trove/tests/api/datastores.py
View File

@ -92,18 +92,12 @@ class Datastores(object):
if version['name'] == 'inactive_version':
return
# Get a valid image ID from a datastore version
datastore = self.rd_client.datastores.get(test_config.dbaas_datastore)
ds_version = self.rd_client.datastore_versions.list(datastore.id)[0]
ds_version_info = self.rd_admin.datastore_versions.get_by_uuid(
ds_version.id)
# Create datastore version for testing
# 'Test_Datastore_1' is also used in other test cases.
# Will be deleted in test_delete_datastore_version
self.rd_admin.mgmt_datastore_versions.create(
"inactive_version", test_config.dbaas_datastore_name_no_versions,
"test_manager", ds_version_info.image,
"test_manager", None, image_tags=['trove'],
active='false', default='false'
)

80
trove/tests/unittests/datastore/base.py
View File

@ -11,7 +11,6 @@
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import uuid
from trove.datastore import models as datastore_models
from trove.datastore.models import Capability
@ -24,63 +23,66 @@ from trove.tests.unittests.util import util
class TestDatastoreBase(trove_testtools.TestCase):
def setUp(self):
# Basic setup and mock/fake structures for testing only
super(TestDatastoreBase, self).setUp()
@classmethod
def setUpClass(cls):
util.init_db()
self.rand_id = str(uuid.uuid4())
self.ds_name = "my-test-datastore" + self.rand_id
self.ds_version = "my-test-version" + self.rand_id
self.capability_name = "root_on_create" + self.rand_id
self.capability_desc = "Enables root on create"
self.capability_enabled = True
self.datastore_version_id = str(uuid.uuid4())
self.flavor_id = 1
self.volume_type = 'some-valid-volume-type'
datastore_models.update_datastore(self.ds_name, False)
self.datastore = Datastore.load(self.ds_name)
cls.ds_name = cls.random_name(name='test-datastore')
cls.ds_version_name = cls.random_name(name='test-version')
cls.capability_name = cls.random_name(name='root_on_create',
prefix='TestDatastoreBase')
cls.capability_desc = "Enables root on create"
cls.capability_enabled = True
cls.flavor_id = 1
cls.volume_type = 'some-valid-volume-type'
datastore_models.update_datastore(cls.ds_name, False)
cls.datastore = Datastore.load(cls.ds_name)
datastore_models.update_datastore_version(
self.ds_name, self.ds_version, "mysql", "", "", True)
cls.ds_name, cls.ds_version_name, "mysql", "", "", "", True)
DatastoreVersionMetadata.add_datastore_version_flavor_association(
self.ds_name, self.ds_version, [self.flavor_id])
cls.ds_name, cls.ds_version_name, [cls.flavor_id])
DatastoreVersionMetadata.add_datastore_version_volume_type_association(
self.ds_name, self.ds_version, [self.volume_type])
cls.ds_name, cls.ds_version_name, [cls.volume_type])
self.datastore_version = DatastoreVersion.load(self.datastore,
self.ds_version)
self.test_id = self.datastore_version.id
cls.datastore_version = DatastoreVersion.load(cls.datastore,
cls.ds_version_name)
cls.test_id = cls.datastore_version.id
self.cap1 = Capability.create(self.capability_name,
self.capability_desc, True)
self.cap2 = Capability.create("require_volume" + self.rand_id,
cls.cap1 = Capability.create(cls.capability_name,
cls.capability_desc, True)
cls.cap2 = Capability.create(
cls.random_name(name='require_volume', prefix='TestDatastoreBase'),
"Require external volume", True)
self.cap3 = Capability.create("test_capability" + self.rand_id,
cls.cap3 = Capability.create(
cls.random_name(name='test_capability',
prefix='TestDatastoreBase'),
"Test capability", False)
def tearDown(self):
super(TestDatastoreBase, self).tearDown()
capabilities_overridden = DBCapabilityOverrides.find_all(
datastore_version_id=self.datastore_version.id).all()
super(TestDatastoreBase, cls).setUpClass()
@classmethod
def tearDownClass(cls):
capabilities_overridden = DBCapabilityOverrides.find_all(
datastore_version_id=cls.test_id).all()
for ce in capabilities_overridden:
ce.delete()
self.cap1.delete()
self.cap2.delete()
self.cap3.delete()
datastore = datastore_models.Datastore.load(self.ds_name)
ds_version = datastore_models.DatastoreVersion.load(datastore,
self.ds_version)
cls.cap1.delete()
cls.cap2.delete()
cls.cap3.delete()
datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id).delete()
Datastore.load(self.ds_name).delete()
datastore_version_id=cls.test_id).delete()
cls.datastore_version.delete()
cls.datastore.delete()
super(TestDatastoreBase, cls).tearDownClass()
def capability_name_filter(self, capabilities):
new_capabilities = []
for capability in capabilities:
if self.rand_id in capability.name:
if 'TestDatastoreBase' in capability.name:
new_capabilities.append(capability)
return new_capabilities

9
trove/tests/unittests/datastore/test_capability.py
View File

@ -19,12 +19,6 @@ from trove.tests.unittests.datastore.base import TestDatastoreBase
class TestCapabilities(TestDatastoreBase):
def setUp(self):
super(TestCapabilities, self).setUp()
def tearDown(self):
super(TestCapabilities, self).tearDown()
def test_capability(self):
cap = Capability.load(self.capability_name)
self.assertEqual(self.capability_name, cap.name)
@ -38,9 +32,6 @@ class TestCapabilities(TestDatastoreBase):
self.ds_cap.delete()
def test_capability_enabled(self):
self.assertTrue(Capability.load(self.capability_name).enabled)
def test_capability_disabled(self):
capability = Capability.load(self.capability_name)
capability.disable()

58
trove/tests/unittests/datastore/test_datastore_version_metadata.py
View File

@ -35,8 +35,8 @@ class TestDatastoreVersionMetadata(TestDatastoreBase):
def test_map_flavors_to_datastore(self):
datastore = datastore_models.Datastore.load(self.ds_name)
ds_version = datastore_models.DatastoreVersion.load(datastore,
self.ds_version)
ds_version = datastore_models.DatastoreVersion.load(
datastore, self.ds_version_name)
mapping = datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id,
value=self.flavor_id, deleted=False, key='flavor')
@ -46,8 +46,8 @@ class TestDatastoreVersionMetadata(TestDatastoreBase):
def test_map_volume_types_to_datastores(self):
datastore = datastore_models.Datastore.load(self.ds_name)
ds_version = datastore_models.DatastoreVersion.load(datastore,
self.ds_version)
ds_version = datastore_models.DatastoreVersion.load(
datastore, self.ds_version_name)
mapping = datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id,
value=self.volume_type, deleted=False, key='volume_type')
@ -60,82 +60,86 @@ class TestDatastoreVersionMetadata(TestDatastoreBase):
self.assertRaisesRegex(
exception.DatastoreFlavorAssociationAlreadyExists,
"Flavor %s is already associated with datastore %s version %s"
% (self.flavor_id, self.ds_name, self.ds_version),
% (self.flavor_id, self.ds_name, self.ds_version_name),
dsmetadata.add_datastore_version_flavor_association,
self.ds_name, self.ds_version, [self.flavor_id])
self.ds_name, self.ds_version_name, [self.flavor_id])
def test_add_existing_volume_type_associations(self):
dsmetadata = datastore_models.DatastoreVersionMetadata
self.assertRaises(
exception.DatastoreVolumeTypeAssociationAlreadyExists,
dsmetadata.add_datastore_version_volume_type_association,
self.ds_name, self.ds_version, [self.volume_type])
self.ds_name, self.ds_version_name, [self.volume_type])
def test_delete_nonexistent_flavor_mapping(self):
dsmeta = datastore_models.DatastoreVersionMetadata
self.assertRaisesRegex(
exception.DatastoreFlavorAssociationNotFound,
"Flavor 2 is not supported for datastore %s version %s"
% (self.ds_name, self.ds_version),
% (self.ds_name, self.ds_version_name),
dsmeta.delete_datastore_version_flavor_association,
self.ds_name, self.ds_version, flavor_id=2)
self.ds_name, self.ds_version_name, flavor_id=2)
def test_delete_nonexistent_volume_type_mapping(self):
dsmeta = datastore_models.DatastoreVersionMetadata
self.assertRaises(
exception.DatastoreVolumeTypeAssociationNotFound,
dsmeta.delete_datastore_version_volume_type_association,
self.ds_name, self.ds_version,
self.ds_name, self.ds_version_name,
volume_type_name='some random thing')
def test_delete_flavor_mapping(self):
flavor_id = 2
dsmetadata = datastore_models. DatastoreVersionMetadata
dsmetadata.add_datastore_version_flavor_association(self.ds_name,
self.ds_version,
dsmetadata = datastore_models.DatastoreVersionMetadata
dsmetadata.add_datastore_version_flavor_association(
self.ds_name,
self.ds_version_name,
[flavor_id])
dsmetadata.delete_datastore_version_flavor_association(self.ds_name,
self.ds_version,
dsmetadata.delete_datastore_version_flavor_association(
self.ds_name,
self.ds_version_name,
flavor_id)
datastore = datastore_models.Datastore.load(self.ds_name)
ds_version = datastore_models.DatastoreVersion.load(datastore,
self.ds_version)
ds_version = datastore_models.DatastoreVersion.load(
datastore,
self.ds_version_name)
mapping = datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id, value=flavor_id, key='flavor')
self.assertTrue(mapping.deleted)
# check update
dsmetadata.add_datastore_version_flavor_association(
self.ds_name, self.ds_version, [flavor_id])
self.ds_name, self.ds_version_name, [flavor_id])
mapping = datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id, value=flavor_id, key='flavor')
self.assertFalse(mapping.deleted)
# clear the mapping
datastore_models.DatastoreVersionMetadata.\
datastore_models.DatastoreVersionMetadata. \
delete_datastore_version_flavor_association(self.ds_name,
self.ds_version,
self.ds_version_name,
flavor_id)
def test_delete_volume_type_mapping(self):
volume_type = 'this is bogus'
dsmetadata = datastore_models. DatastoreVersionMetadata
dsmetadata = datastore_models.DatastoreVersionMetadata
dsmetadata.add_datastore_version_volume_type_association(
self.ds_name,
self.ds_version,
self.ds_version_name,
[volume_type])
dsmetadata.delete_datastore_version_volume_type_association(
self.ds_name,
self.ds_version,
self.ds_version_name,
volume_type)
datastore = datastore_models.Datastore.load(self.ds_name)
ds_version = datastore_models.DatastoreVersion.load(datastore,
self.ds_version)
ds_version = datastore_models.DatastoreVersion.load(
datastore,
self.ds_version_name)
mapping = datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id, value=volume_type,
key='volume_type')
self.assertTrue(mapping.deleted)
# check update
dsmetadata.add_datastore_version_volume_type_association(
self.ds_name, self.ds_version, [volume_type])
self.ds_name, self.ds_version_name, [volume_type])
mapping = datastore_models.DBDatastoreVersionMetadata.find_by(
datastore_version_id=ds_version.id, value=volume_type,
key='volume_type')
@ -143,7 +147,7 @@ class TestDatastoreVersionMetadata(TestDatastoreBase):
# clear the mapping
dsmetadata.delete_datastore_version_volume_type_association(
self.ds_name,
self.ds_version,
self.ds_version_name,
volume_type)
@mock.patch.object(datastore_models.DatastoreVersionMetadata,

6
trove/tests/unittests/datastore/test_datastore_versions.py
View File

@ -20,8 +20,8 @@ class TestDatastoreVersions(TestDatastoreBase):
def test_load_datastore_version(self):
datastore_version = DatastoreVersion.load(self.datastore,
self.ds_version)
self.assertEqual(self.ds_version, datastore_version.name)
self.ds_version_name)
self.assertEqual(self.ds_version_name, datastore_version.name)
def test_datastore_version_capabilities(self):
self.datastore_version.capabilities.add(self.cap1, enabled=False)
@ -35,7 +35,7 @@ class TestDatastoreVersions(TestDatastoreBase):
# Test a fresh reloading of the datastore
self.datastore_version = DatastoreVersion.load(self.datastore,
self.ds_version)
self.ds_version_name)
test_filtered_capabilities = self.capability_name_filter(
self.datastore_version.capabilities)
self.assertEqual(3, len(test_filtered_capabilities),

0
trove/tests/unittests/mgmt/__init__.py → trove/tests/unittests/extensions/mgmt/__init__.py
View File

0
trove/tests/unittests/extensions/mgmt/datastores/__init__.py Normal file
View File

324
trove/tests/unittests/extensions/mgmt/datastores/test_service.py Normal file
View File

@ -0,0 +1,324 @@
# Copyright [2015] Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from unittest import mock
from unittest.mock import MagicMock
from unittest.mock import Mock
from unittest.mock import patch
from glanceclient import exc as glance_exceptions
import jsonschema
from trove.common import clients
from trove.common import exception
from trove.datastore import models
from trove.extensions.mgmt.datastores.service import DatastoreVersionController
from trove.tests.unittests import trove_testtools
from trove.tests.unittests.util import util
class TestDatastoreVersionController(trove_testtools.TestCase):
@classmethod
def setUpClass(cls):
util.init_db()
cls.ds_name = cls.random_name('datastore')
models.update_datastore(name=cls.ds_name, default_version=None)
models.update_datastore_version(
cls.ds_name, 'test_vr1', 'mysql', cls.random_uuid(), '', 'pkg-1',
1)
models.update_datastore_version(
cls.ds_name, 'test_vr2', 'mysql', cls.random_uuid(), '', 'pkg-1',
1)
cls.ds = models.Datastore.load(cls.ds_name)
cls.ds_version1 = models.DatastoreVersion.load(cls.ds, 'test_vr1')
cls.ds_version2 = models.DatastoreVersion.load(cls.ds, 'test_vr2')
cls.version_controller = DatastoreVersionController()
super(TestDatastoreVersionController, cls).setUpClass()
@classmethod
def tearDownClass(cls):
versions = models.DatastoreVersions.load_all(only_active=False)
for ver in versions:
ver.delete()
cls.ds.delete()
super(TestDatastoreVersionController, cls).tearDownClass()
def test_create_schema(self):
image_id = self.random_uuid()
ver_name = self.random_name('dsversion')
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"image": image_id,
"image_tags": [],
"active": True,
"default": False
}
}
schema = self.version_controller.get_schema('create', body)
validator = jsonschema.Draft4Validator(schema)
self.assertTrue(validator.is_valid(body))
def test_create_schema_too_many_image_tags(self):
ver_name = self.random_name('dsversion')
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"image_tags": ['a', 'b', 'c', 'd', 'e', 'f'],
"active": True,
"default": False
}
}
schema = self.version_controller.get_schema('create', body)
validator = jsonschema.Draft4Validator(schema)
self.assertFalse(validator.is_valid(body))
def test_create_schema_emptyname(self):
image_id = self.random_uuid()
body = {
"version": {
"datastore_name": self.ds_name,
"name": " ",
"datastore_manager": "mysql",
"image": image_id,
"image_tags": [],
"active": True,
"default": False
}
}
schema = self.version_controller.get_schema('create', body)
validator = jsonschema.Draft4Validator(schema)
self.assertFalse(validator.is_valid(body))
errors = sorted(validator.iter_errors(body), key=lambda e: e.path)
self.assertEqual(1, len(errors))
self.assertEqual("' ' does not match '^.*[0-9a-zA-Z]+.*$'",
errors[0].message)
@patch.object(clients, 'create_glance_client')
def test_create(self, mock_glance_client):
image_id = self.random_uuid()
ver_name = self.random_name('dsversion')
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"image": image_id,
"image_tags": [],
"packages": "test-pkg",
"active": True,
"default": True
}
}
output = self.version_controller.create(MagicMock(), body, mock.ANY)
self.assertEqual(202, output.status)
new_ver = models.DatastoreVersion.load(self.ds, ver_name)
self.assertEqual(image_id, new_ver.image_id)
@patch.object(clients, 'create_glance_client')
def test_create_by_image_tags(self, mock_create_client):
ver_name = self.random_name('dsversion')
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"image_tags": ["trove", "mysql"],
"active": True,
"default": True
}
}
mock_client = MagicMock()
mock_client.images.list.return_value = [{"id": self.random_uuid()}]
mock_create_client.return_value = mock_client
output = self.version_controller.create(MagicMock(), body, mock.ANY)
self.assertEqual(202, output.status)
mock_client.images.list.assert_called_once_with(
filters={'tag': ["trove", "mysql"], 'status': 'active'},
sort='created_at:desc',
limit=1
)
new_ver = models.DatastoreVersion.load(self.ds, ver_name)
self.assertIsNone(new_ver.image_id)
self.assertEqual('trove,mysql', new_ver.image_tags)
@patch.object(clients, 'create_glance_client')
def test_create_exist(self, mock_glance_client):
image_id = self.random_uuid()
ver_name = 'test_vr1'
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"image": image_id,
"image_tags": [],
"packages": "test-pkg",
"active": True,
"default": True
}
}
self.assertRaises(
exception.DatastoreVersionAlreadyExists,
self.version_controller.create, MagicMock(), body, mock.ANY)
def test_create_no_image(self):
ver_name = self.random_name('dsversion')
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"active": True,
"default": False
}
}
self.assertRaises(
exception.BadRequest,
self.version_controller.create, MagicMock(), body, mock.ANY)
@patch.object(clients, 'create_glance_client')
def test_create_image_notfound(self, mock_create_client):
image_id = self.random_uuid()
ver_name = self.random_name('dsversion')
body = {
"version": {
"datastore_name": self.ds_name,
"name": ver_name,
"datastore_manager": "mysql",
"image": image_id,
"active": True,
"default": False
}
}
mock_client = Mock()
mock_client.images.get.side_effect = [glance_exceptions.HTTPNotFound()]
mock_create_client.return_value = mock_client
self.assertRaises(
exception.ImageNotFound,
self.version_controller.create, MagicMock(), body, mock.ANY)
@patch.object(clients, 'create_glance_client')
def test_update_image(self, mock_create_client):
new_image = self.random_uuid()
body = {
"image": new_image
}
output = self.version_controller.edit(MagicMock(), body, mock.ANY,
self.ds_version1.id)
self.assertEqual(202, output.status)
updated_ver = models.DatastoreVersion.load(self.ds,
self.ds_version1.id)
self.assertEqual(new_image, updated_ver.image_id)
@patch.object(clients, 'create_glance_client')
def test_update_image_tags(self, mock_create_client):
name = self.random_name('dsversion')
models.update_datastore_version(
self.ds_name, name, 'mysql', self.random_uuid(), '', '', 1)
ver = models.DatastoreVersion.load(self.ds, name)
mock_client = MagicMock()
mock_client.images.list.return_value = [{"id": self.random_uuid()}]
mock_create_client.return_value = mock_client
body = {
"image_tags": ['trove', 'mysql']
}
output = self.version_controller.edit(MagicMock(), body, mock.ANY,
ver.id)
self.assertEqual(202, output.status)
updated_ver = models.DatastoreVersion.load(self.ds, ver.id)
self.assertEqual("", updated_ver.image_id)
self.assertEqual("trove,mysql", updated_ver.image_tags)
def test_delete(self):
name = self.random_name('dsversion')
models.update_datastore_version(
self.ds_name, name, 'mysql', self.random_uuid(), '', '', 1)
ver = models.DatastoreVersion.load(self.ds, name)
output = self.version_controller.delete(MagicMock(),
mock.ANY,
ver.id)
self.assertEqual(202, output.status)
self.assertRaises(
exception.DatastoreVersionNotFound,
models.DatastoreVersion.load_by_uuid, ver.id)
def test_index(self):
output = self.version_controller.index(MagicMock(), mock.ANY)
self.assertEqual(200, output.status)
data = output.data(None)
self.assertGreater(len(data['versions']), 0)
def test_show(self):
output = self.version_controller.show(
MagicMock(), mock.ANY, self.ds_version2.id)
self.assertEqual(200, output.status)
self.assertEqual(self.ds_version2.id,
output._data['version']['id'])
self.assertEqual(self.ds_version2.name,
output._data['version']['name'])
self.assertEqual(self.ds_version2.datastore_id,
output._data['version']['datastore_id'])
self.assertEqual(self.ds_version2.datastore_name,
output._data['version']['datastore_name'])
self.assertEqual(self.ds_version2.manager,
output._data['version']['datastore_manager'])
self.assertEqual(self.ds_version2.image_id,
output._data['version']['image'])
self.assertEqual(self.ds_version2.packages.split(','),
output._data['version']['packages'])
self.assertEqual(self.ds_version2.active,
output._data['version']['active'])
def test_show_image_tags(self):
ver_name = self.random_name('dsversion')
tags = ['trove', 'mysql']
models.update_datastore_version(self.ds_name, ver_name, 'mysql', '',
tags, '', 1)
ver = models.DatastoreVersion.load(self.ds, ver_name)
output = self.version_controller.show(
MagicMock(), mock.ANY, ver.id)
self.assertEqual(200, output.status)
data = output.data(None)
self.assertEqual(tags, data['version']['image_tags'])

0
trove/tests/unittests/extensions/mgmt/instances/__init__.py Normal file
View File

0
trove/tests/unittests/mgmt/test_models.py → trove/tests/unittests/extensions/mgmt/instances/test_models.py
View File

99
trove/tests/unittests/mgmt/test_clusters.py
View File

@ -1,99 +0,0 @@
# Copyright [2015] Hewlett-Packard Development Company, L.P.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from unittest.mock import Mock, patch
from trove.common import exception
from trove.extensions.mgmt.clusters.models import MgmtCluster
from trove.extensions.mgmt.clusters.service import MgmtClusterController
from trove.tests.unittests import trove_testtools
class TestClusterController(trove_testtools.TestCase):
def setUp(self):
super(TestClusterController, self).setUp()
self.context = trove_testtools.TroveTestContext(self)
self.req = Mock()
self.req.environ = Mock()
self.req.environ.__getitem__ = Mock(return_value=self.context)
mock_cluster1 = Mock()
mock_cluster1.datastore_version.manager = 'vertica'
mock_cluster1.instances = []
mock_cluster1.instances_without_server = []
mock_cluster2 = Mock()
mock_cluster2.datastore_version.manager = 'vertica'
mock_cluster2.instances = []
mock_cluster2.instances_without_server = []
self.mock_clusters = [mock_cluster1, mock_cluster2]
self.controller = MgmtClusterController()
def tearDown(self):
super(TestClusterController, self).tearDown()
def test_get_action_schema(self):
body = {'do_stuff': {}}
action_schema = Mock()
action_schema.get = Mock()
self.controller.get_action_schema(body, action_schema)
action_schema.get.assert_called_with('do_stuff', {})
@patch.object(MgmtCluster, 'load')
def test_show_cluster(self, mock_cluster_load):
tenant_id = Mock()
id = Mock()
mock_cluster_load.return_value = self.mock_clusters[0]
self.controller.show(self.req, tenant_id, id)
mock_cluster_load.assert_called_with(self.context, id)
@patch.object(MgmtCluster, 'load_all')
def test_index_cluster(self, mock_cluster_load_all):
tenant_id = Mock()
mock_cluster_load_all.return_value = self.mock_clusters
self.controller.index(self.req, tenant_id)
mock_cluster_load_all.assert_called_with(self.context, deleted=None)
@patch.object(MgmtCluster, 'load')
def test_controller_action_found(self, mock_cluster_load):
body = {'reset-task': {}}
tenant_id = Mock()
id = Mock()
mock_cluster_load.return_value = self.mock_clusters[0]
result = self.controller.action(self.req, body, tenant_id, id)
self.assertEqual(202, result.status)
self.assertIsNotNone(result.data)
def test_controller_no_body_action_found(self):
tenant_id = Mock()
id = Mock()
self.assertRaisesRegex(
exception.BadRequest, 'Invalid request body.',
self.controller.action, self.req, None, tenant_id, id)
@patch.object(MgmtCluster, 'load')
def test_controller_invalid_action_found(self, mock_cluster_load):
body = {'do_stuff': {}}
tenant_id = Mock()
id = Mock()
mock_cluster_load.return_value = self.mock_clusters[0]
self.assertRaisesRegex(
exception.BadRequest, 'Invalid cluster action requested.',
self.controller.action, self.req, body, tenant_id, id)

198
trove/tests/unittests/mgmt/test_datastore_controller.py
View File

@ -1,198 +0,0 @@
# Copyright [2015] Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import jsonschema
from unittest.mock import Mock, patch, MagicMock, PropertyMock
from testtools.matchers import Is, Equals
from trove.common import clients
from trove.common import exception
from trove.datastore import models as datastore_models
from trove.extensions.mgmt.datastores.service import DatastoreVersionController
from trove.tests.unittests import trove_testtools
class TestDatastoreVersionController(trove_testtools.TestCase):
def setUp(self):
super(TestDatastoreVersionController, self).setUp()
self.controller = DatastoreVersionController()
self.version = {
"version": {
"datastore_name": "test_dsx",
"name": "test_vr1",
"datastore_manager": "mysql",
"image": "154b350d-4d86-4214-9067-9c54b230c0da",
"packages": ["mysql-server-5.7"],
"active": True,
"default": False
}
}
self.tenant_id = Mock()
context = trove_testtools.TroveTestContext(self)
self.req = Mock()
self.req.environ = Mock()
self.req.environ.__getitem__ = Mock(return_value=context)
def test_get_schema_create(self):
schema = self.controller.get_schema('create', self.version)
self.assertIsNotNone(schema)
self.assertIn('version', schema['properties'])
def test_validate_create(self):
body = self.version
schema = self.controller.get_schema('create', body)
validator = jsonschema.Draft4Validator(schema)
self.assertTrue(validator.is_valid(body))
def test_validate_create_blankname(self):
body = self.version
body['version']['name'] = " "
schema = self.controller.get_schema('create', body)
validator = jsonschema.Draft4Validator(schema)
self.assertFalse(validator.is_valid(body))
errors = sorted(validator.iter_errors(body), key=lambda e: e.path)
self.assertThat(len(errors), Is(1))
self.assertThat(errors[0].message,
Equals("' ' does not match '^.*[0-9a-zA-Z]+.*$'"))
def test_validate_create_blank_datastore(self):
body = self.version
body['version']['datastore_name'] = ""
schema = self.controller.get_schema('create', body)
validator = jsonschema.Draft4Validator(schema)
self.assertFalse(validator.is_valid(body))
errors = sorted(validator.iter_errors(body), key=lambda e: e.path)
error_messages = [error.message for error in errors]
self.assertThat(len(errors), Is(2))
self.assertIn("'' is too short", error_messages)
self.assertIn("'' does not match '^.*[0-9a-zA-Z]+.*$'", error_messages)
@patch.object(clients, 'create_glance_client')
@patch.object(datastore_models.Datastore, 'load')
@patch.object(datastore_models.DatastoreVersion, 'load',
side_effect=exception.DatastoreVersionNotFound)
@patch.object(datastore_models, 'update_datastore_version')
def test_create_datastore_versions(self, mock_ds_version_create,
mock_ds_version_load,
mock_ds_load, mock_glance_client):
body = self.version
mock_ds_load.return_value.name = 'test_dsx'
self.controller.create(self.req, body, self.tenant_id)
mock_ds_version_create.assert_called_with(
'test_dsx', 'test_vr1', 'mysql',
'154b350d-4d86-4214-9067-9c54b230c0da',
'mysql-server-5.7', True)
@patch.object(datastore_models.DatastoreVersion, 'load_by_uuid')
def test_show_ds_version(self, mock_ds_version_load):
id = Mock()
self.controller.show(self.req, self.tenant_id, id)
mock_ds_version_load.assert_called_with(id)
@patch('trove.configuration.models.DBConfiguration.find_all')
@patch('trove.backup.models.DBBackup.find_all')
@patch('trove.instance.models.DBInstance.find_all')
@patch.object(datastore_models.Datastore, 'load')
@patch.object(datastore_models.DatastoreVersion, 'load_by_uuid')
def test_delete_ds_version(self, mock_ds_version_load, mock_ds_load,
mock_instance_find, mock_backup_find,
mock_config_find):
ds_version_id = Mock()
ds_version = Mock()
mock_ds_version_load.return_value = ds_version
self.controller.delete(self.req, self.tenant_id, ds_version_id)
ds_version.delete.assert_called_with()
@patch('trove.instance.models.DBInstance.find_all')
def test_delete_ds_version_instance_in_use(self, mock_instance_find):
mock_instance_find.return_value.all.return_value = [Mock()]
self.assertRaises(
exception.DatastoreVersionsInUse,
self.controller.delete,
self.req, self.tenant_id, 'fake_version_id'
)
@patch('trove.backup.models.DBBackup.find_all')
@patch('trove.instance.models.DBInstance.find_all')
def test_delete_ds_version_backup_in_use(self, mock_instance_find,
mock_backup_find):
mock_backup_find.return_value.all.return_value = [Mock()]
self.assertRaises(
exception.DatastoreVersionsInUse,
self.controller.delete,
self.req, self.tenant_id, 'fake_version_id'
)
@patch('trove.configuration.models.DBConfiguration.find_all')
@patch('trove.backup.models.DBBackup.find_all')
@patch('trove.instance.models.DBInstance.find_all')
def test_delete_ds_version_config_in_use(self, mock_instance_find,
mock_backup_find,
mock_config_find):
mock_config_find.return_value.all.return_value = [Mock()]
self.assertRaises(
exception.DatastoreVersionsInUse,
self.controller.delete,
self.req, self.tenant_id, 'fake_version_id'
)
@patch.object(datastore_models.DatastoreVersion, 'load_by_uuid')
@patch.object(datastore_models.DatastoreVersions, 'load_all')
def test_index_ds_version(self, mock_ds_version_load_all,
mock_ds_version_load_by_uuid):
mock_id = Mock()
mock_ds_version = Mock()
mock_ds_version.id = mock_id
mock_ds_version_load_all.return_value = [mock_ds_version]
self.controller.index(self.req, self.tenant_id)
mock_ds_version_load_all.assert_called_with(only_active=False)
mock_ds_version_load_by_uuid.assert_called_with(mock_id)
@patch.object(clients, 'create_glance_client')
@patch.object(datastore_models.DatastoreVersion, 'load_by_uuid')
@patch.object(datastore_models, 'update_datastore_version')
def test_edit_datastore_versions(self, mock_ds_version_update,
mock_ds_version_load,
mock_glance_client):
body = {'image': '21c8805a-a800-4bca-a192-3a5a2519044d'}
mock_ds_version = MagicMock()
type(mock_ds_version).datastore_name = PropertyMock(
return_value=self.version['version']['datastore_name'])
type(mock_ds_version).name = PropertyMock(
return_value=self.version['version']['name'])
type(mock_ds_version).image_id = PropertyMock(
return_value=self.version['version']['image'])
type(mock_ds_version).packages = PropertyMock(
return_value=self.version['version']['packages'])
type(mock_ds_version).active = PropertyMock(
return_value=self.version['version']['active'])
type(mock_ds_version).manager = PropertyMock(
return_value=self.version['version']['datastore_manager'])
mock_ds_version_load.return_value = mock_ds_version
self.controller.edit(self.req, body, self.tenant_id, Mock())
mock_ds_version_update.assert_called_with(
'test_dsx', 'test_vr1', 'mysql',
'21c8805a-a800-4bca-a192-3a5a2519044d',
'mysql-server-5.7', True)

165
trove/tests/unittests/mgmt/test_datastores.py
View File

@ -1,165 +0,0 @@
# Copyright [2015] Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from unittest.mock import Mock, patch
from glanceclient import exc as glance_exceptions
from trove.common import clients
from trove.common import exception
from trove.datastore import models
from trove.extensions.mgmt.datastores.service import DatastoreVersionController
from trove.tests.unittests import trove_testtools
from trove.tests.unittests.util import util
class TestDatastoreVersion(trove_testtools.TestCase):
def setUp(self):
super(TestDatastoreVersion, self).setUp()
util.init_db()
models.update_datastore(name='test_ds', default_version=None)
models.update_datastore_version(
'test_ds', 'test_vr1', 'mysql',
'154b350d-4d86-4214-9067-9c54b230c0da', 'pkg-1', 1)
models.update_datastore_version(
'test_ds', 'test_vr2', 'mysql',
'154b350d-4d86-4214-9067-9c54b230c0da', 'pkg-1', 1)
self.ds = models.Datastore.load('test_ds')
self.ds_version2 = models.DatastoreVersion.load(self.ds, 'test_vr2')
self.context = trove_testtools.TroveTestContext(self)
self.req = Mock()
self.req.environ = Mock()
self.req.environ.__getitem__ = Mock(return_value=self.context)
self.tenant_id = Mock()
self.version_controller = DatastoreVersionController()
def tearDown(self):
super(TestDatastoreVersion, self).tearDown()
@patch.object(clients, 'create_glance_client')
def test_version_create(self, mock_glance_client):
body = {"version": {
"datastore_name": "test_ds",
"name": "test_version",
"datastore_manager": "mysql",
"image": "image-id",
"packages": "test-pkg",
"active": True,
"default": True}}
output = self.version_controller.create(
self.req, body, self.tenant_id)
self.assertEqual(202, output.status)
@patch.object(clients, 'create_glance_client')
@patch.object(models.DatastoreVersion, 'load')
def test_fail_already_exists_version_create(self, mock_load,
mock_glance_client):
body = {"version": {
"datastore_name": "test_ds",
"name": "test_new_vr",
"datastore_manager": "mysql",
"image": "image-id",
"packages": "test-pkg",
"active": True,
"default": True}}
self.assertRaisesRegex(
exception.DatastoreVersionAlreadyExists,
"A datastore version with the name 'test_new_vr' already exists",
self.version_controller.create, self.req, body, self.tenant_id)
@patch.object(clients, 'create_glance_client')
def test_fail_image_not_found_version_create(self, mock_glance_client):
mock_glance_client.return_value.images.get = Mock(
side_effect=glance_exceptions.HTTPNotFound())
body = {"version": {
"datastore_name": "test_ds",
"name": "test_vr",
"datastore_manager": "mysql",
"image": "image-id",
"packages": "test-pkg",
"active": True,
"default": True}}
self.assertRaisesRegex(
exception.ImageNotFound,
"Image image-id cannot be found.",
self.version_controller.create, self.req, body, self.tenant_id)
def test_version_delete(self):
ds_version1 = models.DatastoreVersion.load(self.ds, 'test_vr1')
output = self.version_controller.delete(self.req,
self.tenant_id,
ds_version1.id)
err_msg = ("Datastore version '%s' cannot be found." %
ds_version1.id)
self.assertEqual(202, output.status)
# Try to find deleted version, this should raise exception.
self.assertRaisesRegex(
exception.DatastoreVersionNotFound,
err_msg, models.DatastoreVersion.load_by_uuid, ds_version1.id)
@patch.object(clients, 'create_glance_client')
def test_version_update(self, mock_client):
body = {"image": "c022f4dc-76ed-4e3f-a25e-33e031f43f8b"}
output = self.version_controller.edit(self.req, body,
self.tenant_id,
self.ds_version2.id)
self.assertEqual(202, output.status)
# Find the details of version updated and match the updated attribute.
test_ds_version = models.DatastoreVersion.load_by_uuid(
self.ds_version2.id)
self.assertEqual(body['image'], test_ds_version.image_id)
@patch.object(clients, 'create_glance_client')
def test_version_update_fail_image_not_found(self, mock_glance_client):
mock_glance_client.return_value.images.get = Mock(
side_effect=glance_exceptions.HTTPNotFound())
body = {"image": "non-existent-image-id"}
self.assertRaisesRegex(
exception.ImageNotFound,
"Image non-existent-image-id cannot be found.",
self.version_controller.edit, self.req, body,
self.tenant_id, self.ds_version2.id)
@patch.object(models.DatastoreVersion, 'load_by_uuid')
def test_version_index(self, mock_load):
output = self.version_controller.index(
self.req, self.tenant_id)
self.assertEqual(200, output.status)
def test_version_show(self):
output = self.version_controller.show(
self.req, self.tenant_id, self.ds_version2.id)
self.assertEqual(200, output.status)
self.assertEqual(self.ds_version2.id,
output._data['version']['id'])
self.assertEqual(self.ds_version2.name,
output._data['version']['name'])
self.assertEqual(self.ds_version2.datastore_id,
output._data['version']['datastore_id'])
self.assertEqual(self.ds_version2.datastore_name,
output._data['version']['datastore_name'])
self.assertEqual(self.ds_version2.manager,
output._data['version']['datastore_manager'])
self.assertEqual(self.ds_version2.image_id,
output._data['version']['image'])
self.assertEqual(self.ds_version2.packages.split(','),
output._data['version']['packages'])
self.assertEqual(self.ds_version2.active,
output._data['version']['active'])

26
trove/tests/unittests/trove_testtools.py
View File

@ -14,8 +14,10 @@
# under the License.
import abc
import random
import testtools
from unittest import mock
import uuid
from trove.common import cfg
from trove.common.context import TroveContext
@ -96,3 +98,27 @@ class TestCase(testtools.TestCase):
new_callable=mock.PropertyMock(return_value=value))
self.addCleanup(conf_patcher.stop)
return conf_patcher.start()
@classmethod
def random_name(cls, name='', prefix=None):
"""Generate a random name that inclues a random number.
:param str name: The name that you want to include
:param str prefix: The prefix that you want to include
:return: a random name. The format is
'<prefix>-<name>-<random number>'.
(e.g. 'prefixfoo-namebar-154876201')
:rtype: string
"""
randbits = str(random.randint(1, 0x7fffffff))
rand_name = randbits
if name:
rand_name = name + '-' + rand_name
if prefix:
rand_name = prefix + '-' + rand_name
return rand_name
@classmethod
def random_uuid(cls):
return str(uuid.uuid4())