openstack/openstacksdk
Code Issues Proposed changes

Finish Magnum rework

Browse Source 
- add coe service resource and proxy methods
- add coe cluster certificates resource and methods
- switch remaining cloud methods to use proxy
- add coe docs

Change-Id: I7532d03ad26785dccdcc37b19165c19246ebd6e1
This commit is contained in:
Artem Goncharov 2023-01-27 12:34:08 +01:00
parent b66c6cc847
commit a27619cbf4
17 changed files with 367 additions and 189 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/source/user/index.rst
View File

@ -96,6 +96,7 @@ control which services can be used.
Block Storage v3 <proxies/block_storage_v3>
Clustering <proxies/clustering>
Compute <proxies/compute>
Container Infrastructure Management <proxies/container_infrastructure_management>
Database <proxies/database>
DNS <proxies/dns>
Identity v2 <proxies/identity_v2>
@ -133,6 +134,7 @@ The following services have exposed *Resource* classes.
Block Storage <resources/block_storage/index>
Clustering <resources/clustering/index>
Compute <resources/compute/index>
Container Infrastructure Management <resources/container_infrastructure_management/index>
Database <resources/database/index>
DNS <resources/dns/index>
Identity <resources/identity/index>

35
doc/source/user/proxies/container_infrastructure_management.rst Normal file
View File

@ -0,0 +1,35 @@
Container Infrastructure Management
===================================
.. automodule:: openstack.container_infrastructure_management.v1._proxy
Cluster Operations
^^^^^^^^^^^^^^^^^^
.. autoclass:: openstack.container_infrastructure_management.v1._proxy.Proxy
:noindex:
:members: create_cluster, delete_cluster, update_cluster, get_cluster,
find_cluster, clusters
Cluster Certificates Operations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. autoclass:: openstack.container_infrastructure_management.v1._proxy.Proxy
:noindex:
:members: create_cluster_certificate, get_cluster_certificate
Cluster Templates Operations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. autoclass:: openstack.container_infrastructure_management.v1._proxy.Proxy
:noindex:
:members: create_cluster_template, delete_cluster_template,
find_cluster_template,
get_cluster_template, cluster_templates, update_cluster_template
Service Operations
^^^^^^^^^^^^^^^^^^
.. autoclass:: openstack.container_infrastructure_management.v1._proxy.Proxy
:noindex:
:members: services

12
doc/source/user/resources/container_infrastructure_management/cluster.rst Normal file
View File

@ -0,0 +1,12 @@
openstack.container_infrastructure_management.v1.cluster
========================================================
.. automodule:: openstack.container_infrastructure_management.v1.cluster
The Cluster Class
------------------
The ``Cluster`` class inherits from :class:`~openstack.resource.Resource`.
.. autoclass:: openstack.container_infrastructure_management.v1.cluster.Cluster
:members:

13
doc/source/user/resources/container_infrastructure_management/cluster_certificate.rst Normal file
View File

@ -0,0 +1,13 @@
openstack.container_infrastructure_management.v1.cluster_certificate
====================================================================
.. automodule:: openstack.container_infrastructure_management.v1.cluster_certificate
The Cluster Certificate Class
-----------------------------
The ``ClusterCertificate`` class inherits from
:class:`~openstack.resource.Resource`.
.. autoclass:: openstack.container_infrastructure_management.v1.cluster_certificate.ClusterCertificate
:members:

13
doc/source/user/resources/container_infrastructure_management/cluster_template.rst Normal file
View File

@ -0,0 +1,13 @@
openstack.container_infrastructure_management.v1.cluster_template
=================================================================
.. automodule:: openstack.container_infrastructure_management.v1.cluster_template
The Cluster Template Class
--------------------------
The ``ClusterTemplate`` class inherits from
:class:`~openstack.resource.Resource`.
.. autoclass:: openstack.container_infrastructure_management.v1.cluster_template.ClusterTemplate
:members:

10
doc/source/user/resources/container_infrastructure_management/index.rst Normal file
View File

@ -0,0 +1,10 @@
Container Infrastructure Management Resources
=============================================
.. toctree::
:maxdepth: 1
cluster
cluster_certificate
cluster_template
service

12
doc/source/user/resources/container_infrastructure_management/service.rst Normal file
View File

@ -0,0 +1,12 @@
openstack.container_infrastructure_management.v1.service
========================================================
.. automodule:: openstack.container_infrastructure_management.v1.service
The Service Class
-----------------
The ``Service`` class inherits from :class:`~openstack.resource.Resource`.
.. autoclass:: openstack.container_infrastructure_management.v1.service.Service
:members:

182
openstack/cloud/_coe.py
View File

@ -21,13 +21,6 @@ from openstack.cloud import exc
class CoeCloudMixin:
@property
def _container_infra_client(self):
if 'container-infra' not in self._raw_clients:
self._raw_clients['container-infra'] = self._get_raw_client(
'container-infra')
return self._raw_clients['container-infra']
@_utils.cache_on_arguments()
def list_coe_clusters(self):
"""List COE (Container Orchestration Engine) cluster.
@ -157,13 +150,8 @@ class CoeCloudMixin:
:returns: Details about the CA certificate for the given cluster.
"""
msg = ("Error fetching CA cert for the cluster {cluster_id}".format(
cluster_id=cluster_id))
url = "/certificates/{cluster_id}".format(cluster_id=cluster_id)
data = self._container_infra_client.get(url,
error_message=msg)
return self._get_and_munchify(key=None, data=data)
return self.container_infrastructure_management\
.get_cluster_certificate(cluster_id)
def sign_coe_cluster_certificate(self, cluster_id, csr):
"""Sign client key and generate the CA certificate for a cluster
@ -177,17 +165,10 @@ class CoeCloudMixin:
:raises: OpenStackCloudException on operation error.
"""
error_message = ("Error signing certs for cluster"
" {cluster_id}".format(cluster_id=cluster_id))
with _utils.shade_exceptions(error_message):
body = {}
body['cluster_uuid'] = cluster_id
body['csr'] = csr
certs = self._container_infra_client.post(
'/certificates', json=body)
return self._get_and_munchify(key=None, data=certs)
return self.container_infrastructure_management\
.create_cluster_certificate(
cluster_uuid=cluster_id,
csr=csr)
@_utils.cache_on_arguments()
def list_cluster_templates(self, detail=False):
@ -325,153 +306,4 @@ class CoeCloudMixin:
:raises: OpenStackCloudException on operation error.
"""
with _utils.shade_exceptions("Error fetching Magnum services list"):
data = self._container_infra_client.get('/mservices')
return self._normalize_magnum_services(
self._get_and_munchify('mservices', data))
def _normalize_coe_clusters(self, coe_clusters):
ret = []
for coe_cluster in coe_clusters:
ret.append(self._normalize_coe_cluster(coe_cluster))
return ret
def _normalize_coe_cluster(self, coe_cluster):
"""Normalize Magnum COE cluster."""
# Only import munch when really necessary
import munch
coe_cluster = coe_cluster.copy()
# Discard noise
coe_cluster.pop('links', None)
c_id = coe_cluster.pop('uuid')
ret = munch.Munch(
id=c_id,
location=self._get_current_location(),
)
if not self.strict_mode:
ret['uuid'] = c_id
for key in (
'status',
'cluster_template_id',
'stack_id',
'keypair',
'master_count',
'create_timeout',
'node_count',
'name'):
if key in coe_cluster:
ret[key] = coe_cluster.pop(key)
ret['properties'] = coe_cluster
return ret
def _normalize_cluster_templates(self, cluster_templates):
ret = []
for cluster_template in cluster_templates:
ret.append(self._normalize_cluster_template(cluster_template))
return ret
def _normalize_cluster_template(self, cluster_template):
"""Normalize Magnum cluster_templates."""
import munch
cluster_template = cluster_template.copy()
# Discard noise
cluster_template.pop('links', None)
cluster_template.pop('human_id', None)
# model_name is a magnumclient-ism
cluster_template.pop('model_name', None)
ct_id = cluster_template.pop('uuid')
ret = munch.Munch(
id=ct_id,
location=self._get_current_location(),
)
ret['is_public'] = cluster_template.pop('public')
ret['is_registry_enabled'] = cluster_template.pop('registry_enabled')
ret['is_tls_disabled'] = cluster_template.pop('tls_disabled')
# pop floating_ip_enabled since we want to hide it in a future patch
fip_enabled = cluster_template.pop('floating_ip_enabled', None)
if not self.strict_mode:
ret['uuid'] = ct_id
if fip_enabled is not None:
ret['floating_ip_enabled'] = fip_enabled
ret['public'] = ret['is_public']
ret['registry_enabled'] = ret['is_registry_enabled']
ret['tls_disabled'] = ret['is_tls_disabled']
# Optional keys
for (key, default) in (
('fixed_network', None),
('fixed_subnet', None),
('http_proxy', None),
('https_proxy', None),
('labels', {}),
('master_flavor_id', None),
('no_proxy', None)):
if key in cluster_template:
ret[key] = cluster_template.pop(key, default)
for key in (
'apiserver_port',
'cluster_distro',
'coe',
'created_at',
'dns_nameserver',
'docker_volume_size',
'external_network_id',
'flavor_id',
'image_id',
'insecure_registry',
'keypair_id',
'name',
'network_driver',
'server_type',
'updated_at',
'volume_driver'):
ret[key] = cluster_template.pop(key)
ret['properties'] = cluster_template
return ret
def _normalize_magnum_services(self, magnum_services):
ret = []
for magnum_service in magnum_services:
ret.append(self._normalize_magnum_service(magnum_service))
return ret
def _normalize_magnum_service(self, magnum_service):
"""Normalize Magnum magnum_services."""
import munch
magnum_service = magnum_service.copy()
# Discard noise
magnum_service.pop('links', None)
magnum_service.pop('human_id', None)
# model_name is a magnumclient-ism
magnum_service.pop('model_name', None)
ret = munch.Munch(location=self._get_current_location())
for key in (
'binary',
'created_at',
'disabled_reason',
'host',
'id',
'report_count',
'state',
'updated_at'):
ret[key] = magnum_service.pop(key)
ret['properties'] = magnum_service
return ret
return list(self.container_infrastructure_management.services())

45
openstack/container_infrastructure_management/v1/_proxy.py
View File

@ -13,9 +13,15 @@
from openstack.container_infrastructure_management.v1 import (
cluster as _cluster
)
from openstack.container_infrastructure_management.v1 import (
cluster_certificate as _cluster_cert
)
from openstack.container_infrastructure_management.v1 import (
cluster_template as _cluster_template
)
from openstack.container_infrastructure_management.v1 import (
service as _service
)
from openstack import proxy
@ -24,6 +30,7 @@ class Proxy(proxy.Proxy):
_resource_registry = {
"cluster": _cluster.Cluster,
"cluster_template": _cluster_template.ClusterTemplate,
"service": _service.Service
}
def create_cluster(self, **attrs):
@ -207,3 +214,41 @@ class Proxy(proxy.Proxy):
return self._update(
_cluster_template.ClusterTemplate, cluster_template, **attrs
)
# ============== Cluster Certificates ==============
def create_cluster_certificate(self, **attrs):
"""Create a new cluster_certificate from CSR
:param dict attrs: Keyword arguments which will be used to create a
:class:`~openstack.container_infrastructure_management.v1.cluster_certificate.ClusterCertificate`,
comprised of the properties on the ClusterCertificate class.
:returns: The results of cluster_certificate creation
:rtype:
:class:`~openstack.container_infrastructure_management.v1.cluster_certificate.ClusterCertificate`
"""
return self._create(_cluster_cert.ClusterCertificate, **attrs)
def get_cluster_certificate(self, cluster_certificate):
"""Get a single cluster_certificate
:param cluster_certificate: The value can be the ID of a
cluster_certificate or a
:class:`~openstack.container_infrastructure_management.v1.cluster_certificate.ClusterCertificate`
instance.
:returns: One
:class:`~openstack.container_infrastructure_management.v1.cluster_certificate.ClusterCertificate`
:raises: :class:`~openstack.exceptions.ResourceNotFound`
when no resource can be found.
"""
return self._get(_cluster_cert.ClusterCertificate, cluster_certificate)
# ============== Services ==============
def services(self):
"""Return a generator of services
:returns: A generator of service objects
:rtype:
:class:`~openstack.container_infrastructure_management.v1.service.Service`
"""
return self._list(_service.Service)

32
openstack/container_infrastructure_management/v1/cluster_certificate.py Normal file
View File

@ -0,0 +1,32 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from openstack import resource
class ClusterCertificate(resource.Resource):
base_path = '/certificates'
# capabilities
allow_create = True
allow_list = False
allow_fetch = True
#: The UUID of the bay.
bay_uuid = resource.Body('bay_uuid')
#: The UUID of the cluster.
cluster_uuid = resource.Body('cluster_uuid', alternate_id=True)
#: Certificate Signing Request (CSR) for authenticating client key.
csr = resource.Body('csr')
#: CA certificate for the bay/cluster.
pem = resource.Body('pem')

38
openstack/container_infrastructure_management/v1/service.py Normal file
View File

@ -0,0 +1,38 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from openstack import resource
class Service(resource.Resource):
resources_key = 'mservices'
base_path = '/mservices'
# capabilities
allow_list = True
#: The name of the binary form of the Magnum service.
binary = resource.Body('binary')
#: The date and time when the resource was created.
created_at = resource.Body('created_at')
#: The disable reason of the service, null if the service is enabled or
#: disabled without reason provided.
disabled_reason = resource.Body('disabled_reason')
#: The host for the service.
host = resource.Body('host')
#: The total number of report.
report_count = resource.Body('report_count')
#: The current state of Magnum services.
state = resource.Body('state')
#: The date and time when the resource was updated.
updated_at = resource.Body('updated_at')

29
openstack/tests/unit/cloud/test_coe_clusters_certificate.py
View File

@ -11,18 +11,19 @@
# under the License.
import munch
from openstack.container_infrastructure_management.v1 import (
cluster_certificate
)
from openstack.tests.unit import base
coe_cluster_ca_obj = munch.Munch(
coe_cluster_ca_obj = dict(
cluster_uuid="43e305ce-3a5f-412a-8a14-087834c34c8c",
pem="-----BEGIN CERTIFICATE-----\nMIIDAO\n-----END CERTIFICATE-----\n",
bay_uuid="43e305ce-3a5f-412a-8a14-087834c34c8c",
links=[]
)
coe_cluster_signed_cert_obj = munch.Munch(
coe_cluster_signed_cert_obj = dict(
cluster_uuid='43e305ce-3a5f-412a-8a14-087834c34c8c',
pem='-----BEGIN CERTIFICATE-----\nMIIDAO\n-----END CERTIFICATE-----',
bay_uuid='43e305ce-3a5f-412a-8a14-087834c34c8c',
@ -33,6 +34,12 @@ coe_cluster_signed_cert_obj = munch.Munch(
class TestCOEClusters(base.TestCase):
def _compare_cluster_certs(self, exp, real):
self.assertDictEqual(
cluster_certificate.ClusterCertificate(
**exp).to_dict(computed=False),
real.to_dict(computed=False),
)
def get_mock_url(
self,
@ -47,12 +54,12 @@ class TestCOEClusters(base.TestCase):
method='GET',
uri=self.get_mock_url(
resource='certificates',
append=[coe_cluster_ca_obj.cluster_uuid]),
append=[coe_cluster_ca_obj['cluster_uuid']]),
json=coe_cluster_ca_obj)
])
ca_cert = self.cloud.get_coe_cluster_certificate(
coe_cluster_ca_obj.cluster_uuid)
self.assertEqual(
coe_cluster_ca_obj['cluster_uuid'])
self._compare_cluster_certs(
coe_cluster_ca_obj,
ca_cert)
self.assert_calls()
@ -61,10 +68,10 @@ class TestCOEClusters(base.TestCase):
self.register_uris([dict(
method='POST',
uri=self.get_mock_url(resource='certificates'),
json={"cluster_uuid": coe_cluster_signed_cert_obj.cluster_uuid,
"csr": coe_cluster_signed_cert_obj.csr}
json={"cluster_uuid": coe_cluster_signed_cert_obj['cluster_uuid'],
"csr": coe_cluster_signed_cert_obj['csr']}
)])
self.cloud.sign_coe_cluster_certificate(
coe_cluster_signed_cert_obj.cluster_uuid,
coe_cluster_signed_cert_obj.csr)
coe_cluster_signed_cert_obj['cluster_uuid'],
coe_cluster_signed_cert_obj['csr'])
self.assert_calls()

7
openstack/tests/unit/cloud/test_magnum_services.py
View File

@ -10,6 +10,7 @@
# License for the specific language governing permissions and limitations
# under the License.
from openstack.container_infrastructure_management.v1 import service
from openstack.tests.unit import base
@ -18,7 +19,6 @@ magnum_service_obj = dict(
created_at='2015-08-27T09:49:58-05:00',
disabled_reason=None,
host='fake-host',
human_id=None,
id=1,
report_count=1,
state='up',
@ -37,6 +37,7 @@ class TestMagnumServices(base.TestCase):
json=dict(mservices=[magnum_service_obj]))])
mservices_list = self.cloud.list_magnum_services()
self.assertEqual(
mservices_list[0],
self.cloud._normalize_magnum_service(magnum_service_obj))
mservices_list[0].to_dict(computed=False),
service.Service(**magnum_service_obj).to_dict(computed=False),
)
self.assert_calls()

43
openstack/tests/unit/container_infrastructure_management/v1/test_cluster_certificate.py Normal file
View File

@ -0,0 +1,43 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from openstack.container_infrastructure_management.v1 import (
cluster_certificate
)
from openstack.tests.unit import base
EXAMPLE = {
"cluster_uuid": "0b4b766f-1500-44b3-9804-5a6e12fe6df4",
"pem": "-----BEGIN CERTIFICATE-----\nMIICzDCCAbSgAwIBAgIQOOkVcEN7TNa9E80G",
"bay_uuid": "0b4b766f-1500-44b3-9804-5a6e12fe6df4",
"csr": "-----BEGIN CERTIFICATE REQUEST-----\nMIIEfzCCAmcCAQAwFDESMBAGA1UE"
}
class TestClusterCertificate(base.TestCase):
def test_basic(self):
sot = cluster_certificate.ClusterCertificate()
self.assertIsNone(sot.resource_key)
self.assertEqual('/certificates', sot.base_path)
self.assertTrue(sot.allow_create)
self.assertTrue(sot.allow_fetch)
self.assertFalse(sot.allow_commit)
self.assertFalse(sot.allow_delete)
self.assertFalse(sot.allow_list)
def test_make_it(self):
sot = cluster_certificate.ClusterCertificate(**EXAMPLE)
self.assertEqual(EXAMPLE['cluster_uuid'], sot.cluster_uuid)
self.assertEqual(EXAMPLE['bay_uuid'], sot.bay_uuid)
self.assertEqual(EXAMPLE['csr'], sot.csr)
self.assertEqual(EXAMPLE['pem'], sot.pem)

29
openstack/tests/unit/container_infrastructure_management/v1/test_proxy.py
View File

@ -10,9 +10,13 @@
# License for the specific language governing permissions and limitations
# under the License.
from openstack.container_infrastructure_management.v1 import (
cluster_certificate
)
from openstack.container_infrastructure_management.v1 import _proxy
from openstack.container_infrastructure_management.v1 import cluster
from openstack.container_infrastructure_management.v1 import cluster_template
from openstack.container_infrastructure_management.v1 import service
from openstack.tests.unit import test_proxy_base
@ -52,6 +56,20 @@ class TestCluster(TestMagnumProxy):
self.verify_delete(self.proxy.delete_cluster, cluster.Cluster, True)
class TestClusterCertificate(TestMagnumProxy):
def test_cluster_certificate_get(self):
self.verify_get(
self.proxy.get_cluster_certificate,
cluster_certificate.ClusterCertificate
)
def test_cluster_certificate_create_attrs(self):
self.verify_create(
self.proxy.create_cluster_certificate,
cluster_certificate.ClusterCertificate,
)
class TestClusterTemplate(TestMagnumProxy):
def test_cluster_template_get(self):
self.verify_get(
@ -93,3 +111,14 @@ class TestClusterTemplate(TestMagnumProxy):
cluster_template.ClusterTemplate,
True,
)
class TestService(TestMagnumProxy):
def test_services(self):
self.verify_list(
self.proxy.services,
service.Service,
method_kwargs={},
expected_kwargs={},
)

49
openstack/tests/unit/container_infrastructure_management/v1/test_service.py Normal file
View File

@ -0,0 +1,49 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from openstack.container_infrastructure_management.v1 import service
from openstack.tests.unit import base
EXAMPLE = {
"binary": "magnum-conductor",
"created_at": "2016-08-23T10:52:13+00:00",
"state": "up",
"report_count": 2179,
"updated_at": "2016-08-25T01:13:16+00:00",
"host": "magnum-manager",
"disabled_reason": None,
"id": 1
}
class TestService(base.TestCase):
def test_basic(self):
sot = service.Service()
self.assertIsNone(sot.resource_key)
self.assertEqual('mservices', sot.resources_key)
self.assertEqual('/mservices', sot.base_path)
self.assertFalse(sot.allow_create)
self.assertFalse(sot.allow_fetch)
self.assertFalse(sot.allow_commit)
self.assertFalse(sot.allow_delete)
self.assertTrue(sot.allow_list)
def test_make_it(self):
sot = service.Service(**EXAMPLE)
self.assertEqual(EXAMPLE['binary'], sot.binary)
self.assertEqual(EXAMPLE['created_at'], sot.created_at)
self.assertEqual(EXAMPLE['disabled_reason'], sot.disabled_reason)
self.assertEqual(EXAMPLE['host'], sot.host)
self.assertEqual(EXAMPLE['report_count'], sot.report_count)
self.assertEqual(EXAMPLE['state'], sot.state)
self.assertEqual(EXAMPLE['updated_at'], sot.updated_at)

5
releasenotes/notes/switch-coe-to-proxy-c18789ed27cc1d95.yaml Normal file
View File

@ -0,0 +1,5 @@
---
features:
- |
Convert container_infrastructure_management cloud operations to rely fully
on service proxy with all resource classes created.