openstack/ironic-python-agent
Code Issues Proposed changes

Configure and use SSL-related requests options

Browse Source 
This patch adds standard SSL options to IPA config and makes use of them
when making HTTP requests.

For now, a single set of certificates is used when needed.
In the future configuration can be expanded to allow per-service
certificates.

Besides, the 'insecure' option (defaults to False) can be overridden
through kernel command line parameter 'ipa-insecure'.
This will allow running IPA in CI-like environments with self-signed SSL
certificates.

Change-Id: I259d9b3caa9ba1dc3d7382f375b8e086a5348d80
Closes-Bug: #1642515
This commit is contained in:
Pavlo Shchelokovskyy 2016-11-17 13:26:28 +02:00
parent 51ab461af8
commit fdd11b54a5
12 changed files with 198 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
doc/source/index.rst
View File

@ -226,6 +226,74 @@ ironic-python-agent.service unit in cloud-config.yaml [5]_.
* ``--debug``: Enables debug logging.
IPA and SSL
===========
During its operation IPA makes HTTP requests to a number of other services,
currently including
- ironic for lookup/heartbeats
- ironic-inspector to publish results of introspection
- HTTP image storage to fetch the user image to be written to the node's disk
(Object storage service or other service storing user images
when ironic is running in a standalone mode)
When these services are configured to require SSL-encrypted connections,
IPA can be configured to either properly use such secure connections or
ignore verifying such SSL connections.
Configuration mostly happens in the IPA config file
(default is ``/etc/ironic_python_agent/ironic_python_agent.conf``)
or command line arguments passed to ``ironic-python-agent``,
and it is possible to provide some options via kernel command line arguments
instead.
Available options in the ``[DEFAULT]`` config file section are:
insecure
Whether to verify server SSL certificates.
When not specified explicitly, defaults to the value of ``ipa-insecure``
kernel command line argument (converted to boolean).
The default for this kernel command line argument is taken to be ``False``.
Overriding it to ``True`` by adding ``ipa-insecure=1`` to the value of
``[pxe]pxe_append_params`` in ironic configuration file will allow running
the same IPA-based deploy ramdisk in a CI-like environment when services
are using secure HTTPS endpoints with self-signed certificates without
adding a custom CA file to the deploy ramdisk (see below).
cafile
Path to the PEM encoded Certificate Authority file.
When not specified, available system-wide list of CAs will be used to
verify server certificates.
Thus in order to use IPA with HTTPS endpoints of other services in
a secure fashion (with ``insecure`` option being ``False``, see above),
operators should either ensure that certificates of those services
are verifiable by root CAs present in the deploy ramdisk,
or add a custom CA file to the ramdisk and set this IPA option to point
to this file at ramdisk build time.
certfile
Path to PEM encoded client certificate cert file.
This option must be used when services are configured to require client
certificates on SSL-secured connections.
This cert file must be added to the deploy ramdisk and path
to it specified for IPA via this option at ramdisk build time.
This option has an effect only when the ``keyfile`` option is also set.
keyfile
Path to PEM encoded client certificate key file.
This option must be used when services are configured to require client
certificates on SSL-secured connections.
This key file must be added to the deploy ramdisk and path
to it specified for IPA via this option at ramdisk build time.
This option has an effect only when the ``certfile`` option is also set.
Currently a single set of cafile/certfile/keyfile options is used for all
HTTP requests to the other services.
Securing IPA's HTTP server itself with SSL is not yet supported in default
ramdisk builds.
Hardware Managers
=================

28
etc/ironic_python_agent/ironic_python_agent.conf.sample
View File

@ -5,9 +5,10 @@
#
# URL of the Ironic API. Can be supplied as "ipa-api-url"
# kernel parameter. (string value)
# kernel parameter.The value must start with either http:// or
# https://. (string value)
# Deprecated group/name - [DEFAULT]/api_url
#api_url = http://127.0.0.1:6385
#api_url = <None>
# The IP address to listen on. Can be supplied as "ipa-listen-
# host" kernel parameter. (string value)
@ -43,7 +44,7 @@
# Deprecated group/name - [DEFAULT]/ip_lookup_sleep
#ip_lookup_sleep = 10
# The interface to use when looking for an IPaddress. Can be
# The interface to use when looking for an IP address. Can be
# supplied as "ipa-network-interface" kernel parameter.
# (string value)
# Deprecated group/name - [DEFAULT]/network_interface
@ -122,6 +123,27 @@
# (integer value)
#disk_wait_delay = 3
# Verify HTTPS connections. Can be supplied as "ipa-insecure"
# kernel parameter. (boolean value)
#insecure = false
# Path to PEM encoded Certificate Authority file to use when
# verifying HTTPS connections. Default is to use available
# system-wide configured CAs. (string value)
#cafile = <None>
# Path to PEM encoded client certificate cert file. Must be
# provided together with "keyfile" option. Default is to not
# present any client certificates to the server. (string
# value)
#certfile = <None>
# Path to PEM encoded client certificate key file. Must be
# provided together with "certfile" option. Default is to not
# present any client certificates to the server. (string
# value)
#keyfile = <None>
#
# From oslo.log
#

3
ironic_python_agent/agent.py
View File

@ -162,6 +162,9 @@ class IronicPythonAgent(base.ExecuteCommandMixin):
lookup_timeout, lookup_interval, standalone,
hardware_initialization_delay=0):
super(IronicPythonAgent, self).__init__()
if bool(cfg.CONF.keyfile) != bool(cfg.CONF.certfile):
LOG.warning("Only one of 'keyfile' and 'certfile' options is "
"defined in config file. Its value will be ignored.")
self.ext_mgr = extension.ExtensionManager(
namespace='ironic_python_agent.extensions',
invoke_on_load=True,

18
ironic_python_agent/config.py
View File

@ -180,6 +180,24 @@ cli_opts = [
'in inventory. '
'Can be supplied as "ipa-disk-wait-delay" '
'kernel parameter.'),
cfg.BoolOpt('insecure',
default=APARAMS.get('ipa-insecure', False),
help='Verify HTTPS connections. Can be supplied as '
'"ipa-insecure" kernel parameter.'),
cfg.StrOpt('cafile',
help='Path to PEM encoded Certificate Authority file '
'to use when verifying HTTPS connections. '
'Default is to use available system-wide configured CAs.'),
cfg.StrOpt('certfile',
help='Path to PEM encoded client certificate cert file. '
'Must be provided together with "keyfile" option. '
'Default is to not present any client certificates to '
'the server.'),
cfg.StrOpt('keyfile',
help='Path to PEM encoded client certificate key file. '
'Must be provided together with "certfile" option. '
'Default is to not present any client certificates to '
'the server.'),
]
CONF.register_cli_opts(cli_opts)

6
ironic_python_agent/extensions/standby.py
View File

@ -19,6 +19,7 @@ import six
import time
from oslo_concurrency import processutils
from oslo_config import cfg
from oslo_log import log
from ironic_lib import disk_utils
@ -27,6 +28,7 @@ from ironic_python_agent.extensions import base
from ironic_python_agent import hardware
from ironic_python_agent import utils
CONF = cfg.CONF
LOG = log.getLogger(__name__)
IMAGE_CHUNK_SIZE = 1024 * 1024 # 1MB
@ -227,7 +229,9 @@ class ImageDownload(object):
if no_proxy:
os.environ['no_proxy'] = no_proxy
proxies = image_info.get('proxies', {})
resp = requests.get(url, stream=True, proxies=proxies)
verify, cert = utils.get_ssl_client_options(CONF)
resp = requests.get(url, stream=True, proxies=proxies,
verify=verify, cert=cert)
if resp.status_code != 200:
msg = ('Received status code {} from {}, expected 200. Response '
'body: {}').format(resp.status_code, url, resp.text)

4
ironic_python_agent/inspector.py
View File

@ -118,7 +118,9 @@ def call_inspector(data, failures):
encoder = encoding.RESTJSONEncoder()
data = encoder.encode(data)
resp = requests.post(CONF.inspection_callback_url, data=data)
verify, cert = utils.get_ssl_client_options(CONF)
resp = requests.post(CONF.inspection_callback_url, data=data,
verify=verify, cert=cert)
if resp.status_code >= 400:
LOG.error('inspector error %d: %s, proceeding with lookup',
resp.status_code, resp.content.decode('utf-8'))

6
ironic_python_agent/ironic_api_client.py
View File

@ -13,6 +13,7 @@
# limitations under the License.
from oslo_config import cfg
from oslo_log import log
from oslo_serialization import jsonutils
from oslo_service import loopingcall
@ -21,8 +22,10 @@ import requests
from ironic_python_agent import encoding
from ironic_python_agent import errors
from ironic_python_agent import netutils
from ironic_python_agent import utils
CONF = cfg.CONF
LOG = log.getLogger(__name__)
@ -57,10 +60,13 @@ class APIClient(object):
'Accept': 'application/json',
})
verify, cert = utils.get_ssl_client_options(CONF)
return self.session.request(method,
request_url,
headers=headers,
data=data,
verify=verify,
cert=cert,
**kwargs)
def heartbeat(self, uuid, advertise_address):

5
ironic_python_agent/tests/unit/extensions/test_standby.py
View File

@ -299,6 +299,7 @@ class TestStandbyExtension(test_base.BaseTestCase):
standby._download_image(image_info)
requests_mock.assert_called_once_with(image_info['urls'][0],
cert=None, verify=True,
stream=True, proxies={})
write = file_mock.write
write.assert_any_call('some')
@ -329,6 +330,7 @@ class TestStandbyExtension(test_base.BaseTestCase):
standby._download_image(image_info)
self.assertEqual(no_proxy, os.environ['no_proxy'])
requests_mock.assert_called_once_with(image_info['urls'][0],
cert=None, verify=True,
stream=True, proxies=proxies)
write = file_mock.write
write.assert_any_call('some')
@ -767,6 +769,7 @@ class TestStandbyExtension(test_base.BaseTestCase):
self.agent_extension._stream_raw_image_onto_device(image_info,
'/dev/foo')
requests_mock.assert_called_once_with(image_info['urls'][0],
cert=None, verify=True,
stream=True, proxies={})
expected_calls = [mock.call('some'), mock.call('content')]
file_mock.write.assert_has_calls(expected_calls)
@ -790,6 +793,7 @@ class TestStandbyExtension(test_base.BaseTestCase):
self.agent_extension._stream_raw_image_onto_device,
image_info, '/dev/foo')
requests_mock.assert_called_once_with(image_info['urls'][0],
cert=None, verify=True,
stream=True, proxies={})
# Assert write was only called once and failed!
file_mock.write.assert_called_once_with('some')
@ -863,5 +867,6 @@ class TestImageDownload(test_base.BaseTestCase):
self.assertEqual(content, list(image_download))
requests_mock.assert_called_once_with(image_info['urls'][0],
cert=None, verify=True,
stream=True, proxies={})
self.assertEqual(image_info['checksum'], image_download.md5sum())

3
ironic_python_agent/tests/unit/test_inspector.py
View File

@ -145,6 +145,7 @@ class TestCallInspector(test_base.BaseTestCase):
res = inspector.call_inspector(data, failures)
mock_post.assert_called_once_with('url',
cert=None, verify=True,
data='{"data": 42, "error": null}')
self.assertEqual(mock_post.return_value.json.return_value, res)
@ -157,6 +158,7 @@ class TestCallInspector(test_base.BaseTestCase):
res = inspector.call_inspector(data, failures)
mock_post.assert_called_once_with('url',
cert=None, verify=True,
data='{"data": 42, "error": "boom"}')
self.assertEqual(mock_post.return_value.json.return_value, res)
@ -168,6 +170,7 @@ class TestCallInspector(test_base.BaseTestCase):
res = inspector.call_inspector(data, failures)
mock_post.assert_called_once_with('url',
cert=None, verify=True,
data='{"data": 42, "error": null}')
self.assertIsNone(res)

30
ironic_python_agent/tests/unit/test_utils.py
View File

@ -455,3 +455,33 @@ class TestUtils(testtools.TestCase):
file_list=['/var/log'],
io_dict={'iptables': mock.ANY, 'ip_addr': mock.ANY, 'ps': mock.ANY,
'dmesg': mock.ANY, 'df': mock.ANY})
def test_get_ssl_client_options(self):
# defaults
conf = mock.Mock(insecure=False, cafile=None,
keyfile=None, certfile=None)
self.assertEqual((True, None), utils.get_ssl_client_options(conf))
# insecure=True overrides cafile
conf = mock.Mock(insecure=True, cafile='spam',
keyfile=None, certfile=None)
self.assertEqual((False, None), utils.get_ssl_client_options(conf))
# cafile returned as verify when not insecure
conf = mock.Mock(insecure=False, cafile='spam',
keyfile=None, certfile=None)
self.assertEqual(('spam', None), utils.get_ssl_client_options(conf))
# only both certfile and keyfile produce non-None result
conf = mock.Mock(insecure=False, cafile=None,
keyfile=None, certfile='ham')
self.assertEqual((True, None), utils.get_ssl_client_options(conf))
conf = mock.Mock(insecure=False, cafile=None,
keyfile='ham', certfile=None)
self.assertEqual((True, None), utils.get_ssl_client_options(conf))
conf = mock.Mock(insecure=False, cafile=None,
keyfile='spam', certfile='ham')
self.assertEqual((True, ('ham', 'spam')),
utils.get_ssl_client_options(conf))

17
ironic_python_agent/utils.py
View File

@ -416,3 +416,20 @@ def collect_system_logs(journald_max_lines=None):
try_get_command_output(io_dict, name, cmd)
return gzip_and_b64encode(io_dict=io_dict, file_list=file_list)
def get_ssl_client_options(conf):
"""Format SSL-related requests options.
:param conf: oslo_config CONF object
:returns: tuple of 'verify' and 'cert' values to pass to requests
"""
if conf.insecure:
verify = False
else:
verify = conf.cafile or True
if conf.certfile and conf.keyfile:
cert = (conf.certfile, conf.keyfile)
else:
cert = None
return verify, cert

15
releasenotes/notes/handle-ssl-063a91fb7bdcf9b9.yaml Normal file
View File

@ -0,0 +1,15 @@
---
features:
- Ironic Python Agent now can access other services
(Ironic, Inspector, image backend) when those are configured with
HTTPS endpoints and use custom server certificates or require
client certificates.
It is possible to add a custom Certificate Authority (CA) file and client
certificate files to the deploy ramdisk during build,
and provide paths to those as corresponding new options
in ``ironic_python_agent.conf`` configuration file.
Validation of server certificate can be turned off with ``insecure``
config option or via ``ipa-insecure`` kernel boot parameter.
This should make it possible to run IPA in CI-like environments that use
HTTPS endpoints with self-signed sertificates.