add bp:ssh-auth-strategy
Blueprint to support multiple ssh auth strategies. Change-Id: Ib882fd2c9354b91c5069a35ec74003e0259fec3f
This commit is contained in:
parent
6465dfdcd4
commit
9a87a0407c
|
@ -0,0 +1,251 @@
|
|||
::
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
..
|
||||
|
||||
=========================================
|
||||
Multiple strategies for ssh access to VMs
|
||||
=========================================
|
||||
|
||||
https://blueprints.launchpad.net/tempest/+spec/ssh-auth-strategy
|
||||
|
||||
Different strategies for ssh access to VMs in tests.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
Ssh access to created servers is in several cases key to properly validate the
|
||||
result of an API call or a scenario (use case) test. This is true for compute
|
||||
but not limited to it. Network and volume verification must often rely on test
|
||||
servers, and ssh access to the VM helps significantly for the verification.
|
||||
|
||||
Support for ssh access to VMs in tempest tests is both heterogeneous as well
|
||||
as incomplete. Not all tests honour the same config options. The existing
|
||||
``run_ssh`` option is only taken into account by some of the tests, the compute
|
||||
API ones. Not all tests use the same strategy for ssh access, and several tests
|
||||
do not perform any ssh verification at all. The reason often is that ssh
|
||||
verification is a common source of "flakiness" and timeouts in tests, and
|
||||
allocation of the resources required for ssh verification can be expensive.
|
||||
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
Consolidate the available configuration options and make sure they are
|
||||
honoured everywhere. Configuration shall be declaritive, i.e. tempest users
|
||||
shall configure how they expect ssh to work, and if that's not compatible
|
||||
with the deployed cloud tempest shall raise an ``InvalidConfiguration``.
|
||||
Improve the configuration help text to guide configuration for instance
|
||||
validation.
|
||||
|
||||
Current configuration options relevant to instance validation are:
|
||||
|
||||
- ``CONF.auth.allow_tenant_isolation``: affects the fixed network name
|
||||
- ``CONF.compute.[image|image_alt]_ssh_user``
|
||||
- ``CONF.compute.image_ssh_password``: not image specific, and it's used
|
||||
by only two tests, without checking against the ssh_auth_method
|
||||
- ``CONF.compute.image_alt_ssh_password``: unused
|
||||
- ``CONF.compute.run_ssh``
|
||||
- ``CONF.compute.ssh_auth_method``: used for resource setup by API compute
|
||||
tests, but not honoured by the tests. The image[_alt]_ssh_[user|password]
|
||||
settings are meant to be used when this is set to "configured".
|
||||
At the moment it is not enforced nor documented
|
||||
- ``CONF.compute.ssh_connect_method``: used for resource setup by API
|
||||
compute tests, not honoured by the tests. When set to floating, it
|
||||
should be verified that a floating IP range is configured
|
||||
- ``CONF.compute.ssh_user``: currently used for ssh verification by most
|
||||
API and scenario tests, which is a problem because configuration supports
|
||||
different images, each with an own ssh user
|
||||
- ``CONF.compute.ping_timeout``: used by scenario test only
|
||||
- ``CONF.compute.ssh_timeout``: used by RemoteClient
|
||||
- ``CONF.compute.ssh_channel_timeout``: used by RemoteClient
|
||||
- ``CONF.compute.fixed_network_name``: used by API and scenario tests.
|
||||
It's the name of the network for the primary IP with nova networking;
|
||||
or with neutron networking when tenant isolation is disabled.
|
||||
The logic, as implemented by test_list_server_filters shall be moved
|
||||
to an helper and reused everywhere. It may be used for ssh validation
|
||||
only if floating IPs are disabled
|
||||
- ``CONF.compute.network_for_ssh``: used by RemoteClient and some scenario
|
||||
tests to discover an IP for ssh validation. It can be used if floating
|
||||
IP for ssh is disabled, in which case the fixed_network_name could be
|
||||
used as well; except for the case of multi-nic testing, which would
|
||||
require more logic anyways to enable the 2nd nic
|
||||
- ``CONF.compute.ip_version_for_ssh``: used by ``RemoteClient``.
|
||||
It should be overridable via parameter instead of one config for all
|
||||
tests.
|
||||
- ``CONF.compute.use_floatingip_for_ssh``: used by some scenario tests,
|
||||
duplicate of ssh_connect_method, which is not used at the moment
|
||||
- ``CONF.compute.path_to_private_key``: unused
|
||||
- ``CONF.network.tenant_network_reachable``: used by scenario tests. In
|
||||
some cases it's used for tests that want to verify both tenant and
|
||||
public network connectivity. In other cases it's used to find out which
|
||||
IP to be used for instance validation, which overlaps with the
|
||||
ssh_connect_method
|
||||
- ``CONF.network.public_network_id``: used for allocation of floating
|
||||
IPs when neutron is enabled.
|
||||
|
||||
Target configuration shall include a new group "validation" used for all
|
||||
option related to validation of API call results, and the following options:
|
||||
|
||||
- ``CONF.validation.connect_method``: default ssh method. Tests may
|
||||
still use different method if they want to do so (fixed or floating)
|
||||
- ``CONF.validation.auth_method``: default auth method. Tests may
|
||||
still use a different method if they want to do so (only ssh key
|
||||
supported for now). Additional methods will be handled in a
|
||||
separate spec
|
||||
- ``CONF.validation.ip_version_for_ssh``: default IP version for ssh
|
||||
- ``CONF.validation.*timeout`` (for ping, connect and ssh)
|
||||
- ``CONF.*.*ssh_user`` (for the various images available)
|
||||
- ``CONF.network.fixed_network_name``: default fixed network name; this
|
||||
parameter is only valid in case of nova network (with flat networking),
|
||||
and for now with pre-provisioned accounts. Once the bp
|
||||
test-accounts-continued is implemented this may still be used as
|
||||
default fixed network name if not specified in accounts.yaml.
|
||||
- ``CONF.network.floating_network_name``: default floating network name,
|
||||
used to allocate floating IPs when neutron is enabled. Deprecates
|
||||
``CONF.network.public_network_id``
|
||||
- ``CONF.network.tenant_network_reachable``: used when the configured
|
||||
ssh_connect_method is "fixed". If this is set to false raise an
|
||||
``InvalidConfiguration`` exception
|
||||
|
||||
Configuration options that are renamed or that planned for removal
|
||||
should go through the deprecation process.
|
||||
|
||||
A few options are image specific: image name, ssh user / password,
|
||||
typical time to boot / ssh.
|
||||
Such options would be better handled in a dedicated images.yaml file
|
||||
rather than in tempest.conf. This will be handled in a separate spec.
|
||||
|
||||
Define an helper functions that read, validate and process the
|
||||
configuration, which in future will help decoupling
|
||||
``create_test_server`` from CONF, for migration to tempest-lib.
|
||||
|
||||
Extend the existing ``RemoteClient`` to provide tools for:
|
||||
|
||||
- ping: attempts a single ping to a target to server
|
||||
- connect: attempts a single TCP connect on a generic port to a target server
|
||||
- ssh: attempts a single ssh connection to a target server
|
||||
- validaton: validates a server by using a configurable sequence of the above;
|
||||
cares about retries and timeouts
|
||||
|
||||
Bits of implementation for that are already available in scenario
|
||||
tests. They should be consolidated in ``RemoteClient``.
|
||||
|
||||
Define a ``validation_resources`` function, similar to the existing
|
||||
``network_resources``, to be used in the class level ``resource_setup``,
|
||||
which allocates required reusable resources, such as: a key pair, a
|
||||
security group with rules in it, and a floating ip. It returns all the
|
||||
resources in form of a dict, ready to be used in ``create_test_server``.
|
||||
Tests which use more than one server will allocated additional floating
|
||||
IPs on demand. Once bp test-accounts-continued is implemented as well
|
||||
we may consider consolidating ``validation_resources`` and
|
||||
``network_resources``.
|
||||
|
||||
Centralize ``create_test_server``, and make sure all tests use
|
||||
this central implementation. Add the following features:
|
||||
|
||||
- it includes an ``sshable`` boolean parameter in the ``create_test_server``
|
||||
helper function, defaults to ``False``. If set to ``True`` it ensures the
|
||||
server is created with all the required resources associated, e.g. that it
|
||||
has a public key injected, and IP address on a public network, a security
|
||||
group that allows for ICMP and ssh communication. The default to false
|
||||
ensures that resources are used only when required.
|
||||
|
||||
- it accepts a resources dict with reusable items, which can be: a key_name,
|
||||
a security_group with rules for ssh and icmp in, a floating_ip. These are
|
||||
passed in as parameters in preparation for the migration to tempest-lib.
|
||||
|
||||
- it extends the valid value for ``wait_until`` with new types of wait
|
||||
abilities: ``PINGABLE`` and ``SSHABLE``. For instance if an ``SSHABLE``
|
||||
server is requested the create method takes care of performing basic ssh
|
||||
validation as well.
|
||||
|
||||
- it returns a tuple ``(created_server, remote_client)``, where the remote
|
||||
client is already initialized with access resources such as public key,
|
||||
admin password, IP address, ssh account name.
|
||||
|
||||
::
|
||||
|
||||
def create_test_server(self, client, wait_until=None, sshable=False,
|
||||
resources=None, **kwargs):
|
||||
if sshable == True and run_ssh == True:
|
||||
read config via helpers
|
||||
process result, extend kwargs, but do not override
|
||||
public_key: if key_name not defined use from resources or create
|
||||
sg rules: use from resources, or create sg with rules and append
|
||||
network name: append to network dict
|
||||
floating ip: use from resources or allocate one
|
||||
validation == True
|
||||
(...)
|
||||
server = servers_client.create_server(**kwargs)
|
||||
wait for status
|
||||
if ip_type == 'floating':
|
||||
attach an IP
|
||||
if validation:
|
||||
build params based on helpers above
|
||||
remote = RemoteClient(**params)
|
||||
wait for status (extended: ping / connect / ssh)
|
||||
return remote
|
||||
|
||||
def test_foo(self):
|
||||
myvm = servers.create_test_server(
|
||||
sshable=True, wait_until='SSHABLE')
|
||||
myvm['remote_client'].write_to_console("I could do something more useful")
|
||||
|
||||
..
|
||||
|
||||
A server can still be made ssh-able "by-hand" for more complex scenarios, such
|
||||
as hot-plug tests, where the server may only be connected at a later stage to
|
||||
a public network.
|
||||
|
||||
In case a test class contains tests which make use of ssh-able servers, network
|
||||
resources must be prepared for the tenant (if not yet available), so that it
|
||||
is possible to have network access to the VM.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
As run_ssh is currently disabled, an alternative could be to completely
|
||||
drop ssh verification from API tests. However a number of cases cannot really
|
||||
be verified unless ssh verification is on (e.g. reboot, rebuild, config drive).
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
Primary assignee:
|
||||
Andrea Frittoli <andrea.frittoli@hp.com>
|
||||
|
||||
Other assignees:
|
||||
Nithya Ganesan <nithya.ganesan@hp.com>,
|
||||
Joseph Lanoux <joseph.lanoux@hp.com>
|
||||
|
||||
|
||||
Milestones
|
||||
----------
|
||||
Target Milestone for completion:
|
||||
Kilo-2
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
- Introduce new configuration options, and helpers to read them
|
||||
- Create a validation_resources function
|
||||
- Create shared create_test_server function
|
||||
- Create shared ssh verification function / extend RemoteClient
|
||||
- Migrate tests to the new format (multiple patches)
|
||||
- Deprecate un-used / removed configuration options
|
||||
- Setup experimental / periodic jobs that run with validation
|
||||
enabled - the aim is to promote both run_ssh and sshable to
|
||||
be ``True`` by default, as well maintain the code path healthy
|
||||
until that happens
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
Loading…
Reference in New Issue