python-cephclient: use configured restful api plugin

ceph-mgr restful plugin is using self-signed certificate when providing
HTTPS access to Ceph REST API.

Instead of retrieving and using this certificate python-cephclient is
currently a shortcut and disables verifying HTTPS requests for the
entire requests/urllib3 library. This was meant to be temporary shortcut
until proper handling of ceph-mgr restful plugin HTTPS certificates is
implemented.

This commit implements automatic python-cephclient restful plugin
certificate retrieval such that it is no longer necessary to disable
requests/urllib3 certificates verification.

Two options were available:

1. provide path to certificate file when creating an instance of
   CephClient() or CephWrapper() then use that value when creating
   a request session ('verify' attribute).

   This delegates the responsibility of providing a valid certificate to
   the caller/user of python-cephclient library. Because it implies an
   API update all StarlingX components using python-cephclient need to
   be updated.

   The certificate file itself is created when mgr-restful-plugin
   is started before ceph-mgr restful plugin service is configured
   to use it.

2. add support for retrieving the certificate by using 'ceph' commands
   similar to how user credentials and restful plugin endpoint are
   discovered.

   If there is an error in getting the certificate then the session
   certificate verification is temporarily disabled until the next
   request is made. This means that if the corresponding Ceph config-key
   'mgr/restful/{hostname}/crt' is removed then python-cephclient will
   incur the overhead of running 'ceph config-key get' before each
   request but this is an unlikely scenario in our case.

Option #2 was selected because it doesn't change existing API.

Change-Id: I68acb3e1d2fb8e2bb07c8d67e65b02d55a6716ca
Depends-on: I6e8ca93c7b51546d134a6eb221c282961ba50afa
Closes-bug: 1828470
Signed-off-by: Daniel Badea <daniel.badea@windriver.com>

ceph/python-cephclient/python-cephclient/cephclient/client.py

@@ -4,13 +4,17 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
+import atexit
 import ipaddress
 import json
 import logging
+import os
+import urlparse
 import re
 import requests
 import six
 import subprocess
+import tempfile
 import time
 
 from cephclient.exception import CephMonRestfulListKeysError
@@ -58,22 +62,22 @@ class CephClient(object):
                  retry_timeout=CEPH_CLIENT_RETRY_TIMEOUT_SEC):
         self.username = username
         self.password = password
-        self.check_certificate = True
+        self.cert_file = None
         self.service_url = None
-        # TODO: fix certificates
-        self._disable_certificate_checks()
         self.session = None
         self.retry_count = retry_count
         self.retry_timeout = retry_timeout
+        atexit.register(
+            self._cleanup_certificate)
 
     def _refresh_session(self):
         self.session = requests.Session()
         self.session.auth = (self.username, self.password)
-
-    def _disable_certificate_checks(self):
-        self.check_certificate = False
-        requests.packages.urllib3.disable_warnings()
-        LOG.warning('skip checking server certificate')
+        if not self.cert_file:
+            self._get_certificate()
+            self.session.verify = self.cert_file.name
+        else:
+            self.session.verify = False
 
     def _get_password(self):
         try:
@@ -112,6 +116,30 @@ class CephClient(object):
             raise CephMgrMissingRestfulService(
                 status.get('services', ''))
 
+    def _get_certificate(self):
+        self._cleanup_certificate()
+        url = urlparse.urlparse(self.service_url)
+        try:
+            certificate = subprocess.check_output(
+                ('ceph config-key get '
+                 '--connect-timeout {} '
+                 'mgr/restful/{}/crt').format(
+                     CEPH_CLI_TIMEOUT_SEC,
+                     url.hostname),
+                shell=True)
+        except subprocess.CalledProcessError:
+            return
+        with tempfile.NamedTemporaryFile(delete=False) as self.cert_file:
+            self.cert_file.write(certificate)
+
+    def _cleanup_certificate(self):
+        if self.cert_file:
+            try:
+                os.unlink(self.cert_file.name)
+            except OSError:
+                pass
+            self.cert_file = None
+
     def _make_text_result(self, prefix, result):
         if result.get('has_failed'):
             assert(len(result['failed']) == 1)
@@ -187,7 +215,6 @@ class CephClient(object):
                 result = self.session.post(
                     self.service_url + 'request?wait=1',
                     json=req_json,
-                    verify=self.check_certificate,
                     timeout=timeout).json()
                 LOG.info('Result: {}'.format(result))
                 if 'is_finished' in result: