zuul
/
nodepool
Code Issues Proposed changes

Add shell-type config 

Ansible needs to know which shell type the node uses to operate
correctly, especially for ssh connections for windows nodes because
otherwise ansible defaults to trying bash.

Change-Id: I71abfefa57aaafd88f199be19ee7caa64efda538
This commit is contained in:
Albin Vass 2021-02-12 13:22:14 +01:00 committed by Daniel Blixt
parent 7a9d83603e
commit 0c84b7fa4e
35 changed files with 242 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
doc/source/aws.rst
View File

@ -189,6 +189,17 @@ section of the configuration.
most diskimages this is not necessary. This defaults to 22 for ssh and
5986 for winrm.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: pools
:type: list

11
doc/source/configuration.rst
View File

@ -401,6 +401,17 @@ Options
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g.
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: dib-cmd
:type: string
:default: disk-image-create

11
doc/source/gce.rst
View File

@ -153,6 +153,17 @@ section of the configuration.
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g.
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: connection-type
:type: str

11
doc/source/kubernetes.rst
View File

@ -130,6 +130,17 @@ Selecting the kubernetes driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g.
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: cpu
:type: int

11
doc/source/openshift-pods.rst
View File

@ -111,6 +111,17 @@ Selecting the openshift pods driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows node with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: env
:type: list
:default: []

11
doc/source/openshift.rst
View File

@ -138,6 +138,17 @@ Selecting the openshift driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: cpu
:type: int

11
doc/source/openstack.rst
View File

@ -347,6 +347,17 @@ Selecting the OpenStack driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: connection-type
:type: str

11
doc/source/static.rst
View File

@ -145,6 +145,17 @@ Selecting the static driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows node with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: max-parallel-jobs
:type: int
:default: 1

9
nodepool/builder.py
View File

@ -938,6 +938,7 @@ class BuildWorker(BaseWorker):
build_data.builder = self._hostname
build_data.username = diskimage.username
build_data.python_path = diskimage.python_path
build_data.shell_type = diskimage.shell_type
if self._statsd:
pipeline = self._statsd.pipeline()
@ -1045,7 +1046,7 @@ class UploadWorker(BaseWorker):
self._config = new_config
def _uploadImage(self, build_id, upload_id, image_name, images, provider,
username, python_path):
username, python_path, shell_type):
'''
Upload a local DIB image build to a provider.
@ -1057,6 +1058,7 @@ class UploadWorker(BaseWorker):
:param provider: The provider from the parsed config file.
:param username:
:param python_path:
:param shell_type:
'''
start_time = time.time()
timestamp = int(start_time)
@ -1166,6 +1168,7 @@ class UploadWorker(BaseWorker):
data.format = image.extension
data.username = username
data.python_path = python_path
data.shell_type = shell_type
return data
@ -1264,13 +1267,15 @@ class UploadWorker(BaseWorker):
data.state = zk.UPLOADING
data.username = build.username
data.python_path = build.python_path
data.shell_type = build.shell_type
upnum = self._zk.storeImageUpload(
image.name, build.id, provider.name, data)
data = self._uploadImage(build.id, upnum, image.name,
local_images, provider,
build.username, build.python_path)
build.username, build.python_path,
build.shell_type)
# Set final state
self._zk.storeImageUpload(image.name, build.id,

5
nodepool/config.py
View File

@ -210,6 +210,7 @@ class DiskImage(ConfigValue):
self.image_types = set([])
self.pause = False
self.python_path = 'auto'
self.shell_type = None
self.rebuild_age = self.REBUILD_AGE
self.release = ''
self.username = 'zuul'
@ -249,6 +250,9 @@ class DiskImage(ConfigValue):
python_path = config.get('python-path', None)
if python_path:
self.python_path = python_path
shell_type = config.get('shell-type', None)
if shell_type:
self.shell_type = shell_type
rebuild_age = config.get('rebuild-age', None)
if rebuild_age:
self.rebuild_age = rebuild_age
@ -269,6 +273,7 @@ class DiskImage(ConfigValue):
other.image_types == self.image_types and
other.pause == self.pause and
other.python_path == self.python_path and
other.shell_type == self.shell_type and
other.rebuild_age == self.rebuild_age and
other.release == self.release and
other.username == self.username)

3
nodepool/driver/aws/config.py
View File

@ -35,6 +35,7 @@ class ProviderCloudImage(ConfigValue):
and self.image_id == other.image_id
and self.username == other.username
and self.python_path == other.python_path
and self.shell_type == other.shell_type
and self.connection_type == other.connection_type
and self.connection_port == other.connection_port)
return False
@ -225,6 +226,7 @@ class AwsProviderConfig(ProviderConfig):
i.username = image.get('username', None)
i.python_path = image.get('python-path', 'auto')
i.shell_type = image.get('shell-type', None)
i.connection_type = image.get('connection-type', 'ssh')
i.connection_port = image.get(
'connection-port',
@ -272,6 +274,7 @@ class AwsProviderConfig(ProviderConfig):
'name': str,
'connection-type': str,
'connection-port': int,
'shell-type': str,
'image-id': str,
"image-filters": [image_filters],
'username': str,

1
nodepool/driver/aws/handler.py
View File

@ -111,6 +111,7 @@ class AwsInstanceLauncher(NodeLauncher):
self.node.host_keys = keys
self.node.username = self.label.cloud_image.username
self.node.python_path = self.label.cloud_image.python_path
self.node.shell_type = self.label.cloud_image.shell_type
self.zk.storeNode(self.node)
self.log.info("Instance %s is ready", instance_id)

6
nodepool/driver/gce/config.py
View File

@ -30,6 +30,7 @@ class ProviderCloudImage(ConfigValue):
self.python_path = None
self.connection_type = None
self.connection_port = None
self.shell_type = None
def __eq__(self, other):
if isinstance(other, ProviderCloudImage):
@ -39,7 +40,8 @@ class ProviderCloudImage(ConfigValue):
and self.key == other.key
and self.python_path == other.python_path
and self.connection_type == other.connection_type
and self.connection_port == other.connection_port)
and self.connection_port == other.connection_port
and self.shell_type == other.shell_type)
return False
def __repr__(self):
@ -202,6 +204,7 @@ class GCEProviderConfig(ProviderConfig):
i.connection_port = image.get(
'connection-port',
default_port_mapping.get(i.connection_type, 22))
i.shell_type = image.get('shell-type', None)
self.cloud_images[i.name] = i
for pool in self.provider.get('pools', []):
@ -229,6 +232,7 @@ class GCEProviderConfig(ProviderConfig):
'name': str,
'connection-type': str,
'connection-port': int,
'shell-type': str,
'image-id': str,
'image-project': str,
'image-family': str,

3
nodepool/driver/kubernetes/config.py
View File

@ -28,6 +28,7 @@ class KubernetesLabel(ConfigValue):
other.type == self.type and
other.image_pull == self.image_pull and
other.python_path == self.python_path and
other.shell_type == self.shell_type and
other.image == self.image and
other.cpu == self.cpu and
other.memory == self.memory and
@ -61,6 +62,7 @@ class KubernetesPool(ConfigPool):
pl.image = label.get('image')
pl.image_pull = label.get('image-pull', 'IfNotPresent')
pl.python_path = label.get('python-path', 'auto')
pl.shell_type = label.get('shell-type')
pl.cpu = label.get('cpu')
pl.memory = label.get('memory')
pl.env = label.get('env', [])
@ -112,6 +114,7 @@ class KubernetesProviderConfig(ProviderConfig):
'image': str,
'image-pull': str,
'python-path': str,
'shell-type': str,
'cpu': int,
'memory': int,
'env': [env_var],

1
nodepool/driver/kubernetes/handler.py
View File

@ -38,6 +38,7 @@ class K8SLauncher(NodeLauncher):
self.node.state = zk.READY
self.node.python_path = self.label.python_path
self.node.shell_type = self.label.shell_type
# NOTE: resource access token may be encrypted here
self.node.connection_port = resource
if self.label.type == "namespace":

3
nodepool/driver/openshift/config.py
View File

@ -32,6 +32,7 @@ class OpenshiftLabel(ConfigValue):
other.cpu == self.cpu and
other.memory == self.memory and
other.python_path == self.python_path and
other.shell_type == self.shell_type and
other.env == self.env and
other.node_selector == self.node_selector)
return False
@ -64,6 +65,7 @@ class OpenshiftPool(ConfigPool):
pl.cpu = label.get('cpu')
pl.memory = label.get('memory')
pl.python_path = label.get('python-path', 'auto')
pl.shell_type = label.get('shell-type')
pl.env = label.get('env', [])
pl.node_selector = label.get('node-selector')
pl.pool = self
@ -116,6 +118,7 @@ class OpenshiftProviderConfig(ProviderConfig):
'cpu': int,
'memory': int,
'python-path': str,
'shell-type': str,
'env': [env_var],
'node-selector': dict,
}

1
nodepool/driver/openshift/handler.py
View File

@ -47,6 +47,7 @@ class OpenshiftLauncher(NodeLauncher):
self.node.state = zk.READY
self.node.python_path = self.label.python_path
self.node.shell_type = self.label.shell_type
# NOTE: resource access token may be encrypted here
self.node.connection_port = resource
self.zk.storeNode(self.node)

1
nodepool/driver/openshiftpods/config.py
View File

@ -56,6 +56,7 @@ class OpenshiftPodsProviderConfig(OpenshiftProviderConfig):
'cpu': int,
'memory': int,
'python-path': str,
'shell-type': str,
'env': [env_var],
'node-selector': dict
}

1
nodepool/driver/openshiftpods/handler.py
View File

@ -34,6 +34,7 @@ class OpenshiftPodLauncher(OpenshiftLauncher):
self.node.state = zk.READY
self.node.python_path = self.label.python_path
self.node.shell_type = self.label.shell_type
# NOTE: resource access token may be encrypted here
k8s = self.handler.manager.k8s_client
self.node.connection_port = {

4
nodepool/driver/openstack/config.py
View File

@ -53,6 +53,7 @@ class ProviderCloudImage(ConfigValue):
self.image_name = None
self.username = None
self.python_path = None
self.shell_type = None
self.connection_type = None
self.connection_port = None
@ -64,6 +65,7 @@ class ProviderCloudImage(ConfigValue):
self.image_name == other.image_name and
self.username == other.username and
self.python_path == other.python_path and
self.shell_type == other.shell_type and
self.connection_type == other.connection_type and
self.connection_port == other.connection_port)
return False
@ -336,6 +338,7 @@ class OpenStackProviderConfig(ProviderConfig):
i.image_name = image.get('image-name', None)
i.username = image.get('username', None)
i.python_path = image.get('python-path', 'auto')
i.shell_type = image.get('shell-type', None)
i.connection_type = image.get('connection-type', 'ssh')
i.connection_port = image.get(
'connection-port',
@ -366,6 +369,7 @@ class OpenStackProviderConfig(ProviderConfig):
v.Exclusive('image-name', 'cloud-image-name-or-id'): str,
'username': str,
'python-path': str,
'shell-type': str,
}
pool_label_main = {

3
nodepool/driver/openstack/handler.py
View File

@ -87,6 +87,7 @@ class OpenStackNodeLauncher(NodeLauncher):
image_name = diskimage.name
username = cloud_image.username
python_path = cloud_image.python_path
shell_type = cloud_image.shell_type
connection_type = diskimage.connection_type
connection_port = diskimage.connection_port
@ -105,6 +106,7 @@ class OpenStackNodeLauncher(NodeLauncher):
image_name = self.label.cloud_image.name
username = self.label.cloud_image.username
python_path = self.label.cloud_image.python_path
shell_type = self.label.cloud_image.shell_type
connection_type = self.label.cloud_image.connection_type
connection_port = self.label.cloud_image.connection_port
@ -159,6 +161,7 @@ class OpenStackNodeLauncher(NodeLauncher):
self.node.username = username
self.node.python_path = python_path
self.node.shell_type = shell_type
self.node.connection_type = connection_type
self.node.connection_port = connection_port

1
nodepool/driver/simple.py
View File

@ -153,6 +153,7 @@ class SimpleTaskManagerLauncher(NodeLauncher):
self.node.host_keys = keys
self.node.username = self.label.cloud_image.username
self.node.python_path = self.label.cloud_image.python_path
self.node.shell_type = self.label.cloud_image.shell_type
self.zk.storeNode(self.node)
self.log.info("Instance %s is ready", hostname)

2
nodepool/driver/static/config.py
View File

@ -57,6 +57,7 @@ class StaticPool(ConfigPool):
'connection-port': int(
node.get('connection-port', node.get('ssh-port', 22))),
'connection-type': node.get('connection-type', 'ssh'),
'shell-type': node.get('shell-type', None),
'username': node.get('username', 'zuul'),
'max-parallel-jobs': int(node.get('max-parallel-jobs', 1)),
'python-path': node.get('python-path', 'auto'),
@ -108,6 +109,7 @@ class StaticProviderConfig(ProviderConfig):
'host-key': v.Any(str, [str]),
'connection-port': int,
'connection-type': str,
'shell-type': str,
'max-parallel-jobs': int,
'python-path': str,
}

7
nodepool/driver/static/provider.py
View File

@ -194,6 +194,7 @@ class StaticNodeProvider(Provider):
node.connection_port = static_node["connection-port"]
node.connection_type = static_node["connection-type"]
node.python_path = static_node["python-path"]
node.shell_type = static_node["shell-type"]
nodeutils.set_node_ip(node)
node.host_keys = host_keys
node.attributes = pool.node_attributes
@ -218,14 +219,15 @@ class StaticNodeProvider(Provider):
static_node["username"],
static_node["connection-port"],
static_node["connection-type"],
static_node["shell-type"],
static_node["python-path"],
host_keys,
)
for node in nodes:
original_attrs = (node.type, node.username, node.connection_port,
node.connection_type, node.python_path,
node.host_keys)
node.shell_type, node.connection_type,
node.python_path, node.host_keys)
if original_attrs == new_attrs:
continue
@ -236,6 +238,7 @@ class StaticNodeProvider(Provider):
node.username = static_node["username"]
node.connection_port = static_node["connection-port"]
node.connection_type = static_node["connection-type"]
node.shell_type = static_node["shell-type"]
node.python_path = static_node["python-path"]
nodeutils.set_node_ip(node)
node.host_keys = host_keys

17
nodepool/tests/fixtures/aws.yaml vendored
View File

@ -17,6 +17,7 @@ labels:
- name: ubuntu1404-iam-instance-profile-arn
- name: ubuntu1404-with-tags
- name: ubuntu1404-with-name-tag
- name: ubuntu1404-with-shell-type
providers:
- name: ec2-us-west-2
@ -48,6 +49,10 @@ providers:
values:
- ubuntu*
username: ubuntu
- name: ubuntu1404-with-shell-type
image-id: ami-1e749f67
username: ubuntu
shell-type: csh
pools:
- name: ebs-optimized
max-servers: 1
@ -161,3 +166,15 @@ providers:
key-name: zuul
tags:
Name: different-name
- name: shell-type
max-servers: 1
subnet-id: null
security-group-id: null
node-attributes:
key1: value1
key2: value2
labels:
- name: ubuntu1404-with-shell-type
cloud-image: ubuntu1404-with-shell-type
instance-type: t3.medium
key-name: zuul

12
nodepool/tests/fixtures/config_validate/good.yaml vendored
View File

@ -24,6 +24,9 @@ labels:
- name: openshift-project
- name: openshift-pod
- name: centos-ami
- name: winrm
- name: winssh
providers:
- name: cloud1
@ -124,6 +127,15 @@ providers:
connection-port: 22022
username: zuul
max-parallel-jobs: 1
- name: windows.example.com
labels: winrm
username: zuul
connection-type: winrm
connection-port: 5986
- name: windows-ssh.example.com
labels: winssh
username: zuul
shell-type: cmd
- name: kubespray
driver: kubernetes

1
nodepool/tests/fixtures/node_unmanaged_image.yaml vendored
View File

@ -29,6 +29,7 @@ providers:
cloud-images:
- name: fake-image
python-path: /usr/bin/python3
shell-type: csh
- name: fake-image-windows
username: zuul
connection-type: winrm

1
nodepool/tests/fixtures/openshift.yaml vendored
View File

@ -28,3 +28,4 @@ providers:
type: pod
image: docker.io/fedora:28
python-path: '/usr/bin/python3'
shell-type: csh

26
nodepool/tests/fixtures/static-shell-type.yaml vendored Normal file
View File

@ -0,0 +1,26 @@
zookeeper-servers:
- host: {zookeeper_host}
port: {zookeeper_port}
chroot: {zookeeper_chroot}
zookeeper-tls:
ca: {zookeeper_ca}
cert: {zookeeper_cert}
key: {zookeeper_key}
labels:
- name: fake-label
providers:
- name: static-provider
driver: static
pools:
- name: main
nodes:
- name: fake-host-1
labels: fake-label
host-key: ssh-rsa FAKEKEY
timeout: 13
connection-port: 22022
username: zuul
shell-type: cmd

10
nodepool/tests/unit/test_driver_aws.py
View File

@ -49,7 +49,8 @@ class TestDriverAws(tests.DBTestCase):
host_key_checking=True,
userdata=None,
public_ip=True,
tags=[]):
tags=[],
shell_type=None):
aws_id = 'AK000000000000000000'
aws_key = '0123456789abcdef0123456789abcdef0123456789abcdef'
self.useFixture(
@ -97,6 +98,8 @@ class TestDriverAws(tests.DBTestCase):
raw_config['providers'][0]['pools'][4]['security-group-id'] = sg_id
raw_config['providers'][0]['pools'][5]['subnet-id'] = subnet_id
raw_config['providers'][0]['pools'][5]['security-group-id'] = sg_id
raw_config['providers'][0]['pools'][6]['subnet-id'] = subnet_id
raw_config['providers'][0]['pools'][6]['security-group-id'] = sg_id
with tempfile.NamedTemporaryFile() as tf:
tf.write(yaml.safe_dump(
@ -152,6 +155,7 @@ class TestDriverAws(tests.DBTestCase):
self.assertEqual(node.connection_type, 'ssh')
self.assertEqual(node.attributes,
{'key1': 'value1', 'key2': 'value2'})
self.assertEqual(node.shell_type, shell_type)
if host_key_checking:
nodescan.assert_called_with(
node.interface_ip,
@ -251,3 +255,7 @@ class TestDriverAws(tests.DBTestCase):
tags=[
{"Key": "Name", "Value": "different-name"}
])
def test_ec2_machine_shell_type(self):
self._test_ec2_machine('ubuntu1404-with-shell-type',
shell_type="csh")

1
nodepool/tests/unit/test_driver_azure.py
View File

@ -195,3 +195,4 @@ class TestDriverAzure(tests.DBTestCase):
self.assertEqual(node.state, zk.READY)
self.assertIsNotNone(node.launcher)
self.assertEqual(node.connection_type, 'ssh')
self.assertIsNone(node.shell_type)

3
nodepool/tests/unit/test_driver_openshift.py
View File

@ -155,6 +155,7 @@ class TestDriverOpenshift(tests.DBTestCase):
self.assertEqual(node.connection_type, 'kubectl')
self.assertEqual(node.connection_port.get('token'), 'fake-token')
self.assertEqual(node.python_path, '/usr/bin/python3')
self.assertEqual(node.shell_type, 'csh')
self.assertEqual(node.attributes,
{'key1': 'value1', 'key2': 'value2'})
@ -183,6 +184,8 @@ class TestDriverOpenshift(tests.DBTestCase):
self.assertIsNotNone(node.launcher)
self.assertEqual(node.connection_type, 'project')
self.assertEqual(node.connection_port.get('token'), 'fake-token')
self.assertEqual(node.python_path, 'auto')
self.assertIsNone(node.shell_type)
node.state = zk.DELETING
self.zk.storeNode(node)

21
nodepool/tests/unit/test_driver_static.py
View File

@ -64,6 +64,8 @@ class TestDriverStatic(tests.DBTestCase):
self.assertEqual(nodes[0].host_keys, ['ssh-rsa FAKEKEY'])
self.assertEqual(nodes[0].attributes,
{'key1': 'value1', 'key2': 'value2'})
self.assertEqual(nodes[0].python_path, 'auto')
self.assertIsNone(nodes[0].shell_type)
def test_static_python_path(self):
'''
@ -528,6 +530,25 @@ class TestDriverStatic(tests.DBTestCase):
self.assertEqual(len(nodes), 1)
nodescan_mock.assert_not_called()
def test_static_shell_type(self):
'''
Test that static python-path works.
'''
configfile = self.setup_config('static-shell-type.yaml')
pool = self.useNodepool(configfile, watermark_sleep=1)
pool.start()
self.log.debug("Waiting for node pre-registration")
nodes = self.waitForNodes('fake-label')
self.assertEqual(nodes[0].shell_type, "cmd")
nodes[0].state = zk.USED
self.zk.storeNode(nodes[0])
self.log.debug("Waiting for node to be re-available")
nodes = self.waitForNodes('fake-label')
self.assertEqual(nodes[0].shell_type, "cmd")
def test_missing_static_node(self):
"""Test that a missing static node is added"""
configfile = self.setup_config('static-2-nodes.yaml')

4
nodepool/tests/unit/test_launcher.py
View File

@ -1383,6 +1383,7 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual(len(nodes), 1)
self.assertIsNone(nodes[0].username)
self.assertEqual(nodes[0].python_path, '/usr/bin/python3')
self.assertEqual(nodes[0].shell_type, 'csh')
nodes = self.waitForNodes('fake-label-windows')
self.assertEqual(len(nodes), 1)
@ -1391,6 +1392,7 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual(5986, nodes[0].connection_port)
self.assertEqual(nodes[0].host_keys, [])
self.assertEqual(nodes[0].python_path, 'auto')
self.assertIsNone(nodes[0].shell_type)
nodes = self.waitForNodes('fake-label-arbitrary-port')
self.assertEqual(len(nodes), 1)
@ -1398,6 +1400,8 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual('winrm', nodes[0].connection_type)
self.assertEqual(1234, nodes[0].connection_port)
self.assertEqual(nodes[0].host_keys, [])
self.assertEqual(nodes[0].python_path, 'auto')
self.assertIsNone(nodes[0].shell_type)
def test_unmanaged_image_provider_name(self):
"""

15
nodepool/zk.py
View File

@ -299,6 +299,7 @@ class ImageBuild(BaseModel):
self.builder_id = None # Unique ID
self.username = None
self.python_path = None
self.shell_type = None
def __repr__(self):
d = self.toDict()
@ -332,6 +333,7 @@ class ImageBuild(BaseModel):
d['formats'] = ','.join(self.formats)
d['username'] = self.username
d['python_path'] = self.python_path
d['shell_type'] = self.shell_type
return d
@staticmethod
@ -350,6 +352,7 @@ class ImageBuild(BaseModel):
o.builder_id = d.get('builder_id')
o.username = d.get('username', 'zuul')
o.python_path = d.get('python_path', '/usr/bin/python2')
o.shell_type = d.get('shell_type')
# Only attempt the split on non-empty string
if d.get('formats', ''):
o.formats = d.get('formats', '').split(',')
@ -363,7 +366,8 @@ class ImageUpload(BaseModel):
VALID_STATES = set([UPLOADING, READY, DELETING, FAILED])
def __init__(self, build_id=None, provider_name=None, image_name=None,
upload_id=None, username=None, python_path=None):
upload_id=None, username=None, python_path=None,
shell_type=None):
super(ImageUpload, self).__init__(upload_id)
self.build_id = build_id
self.provider_name = provider_name
@ -371,6 +375,7 @@ class ImageUpload(BaseModel):
self.format = None
self.username = username
self.python_path = python_path
self.shell_type = shell_type
self.external_id = None # Provider ID of the image
self.external_name = None # Provider name of the image
@ -403,6 +408,7 @@ class ImageUpload(BaseModel):
d['format'] = self.format
d['username'] = self.username
d['python_path'] = self.python_path
d['shell_type'] = self.shell_type
return d
@staticmethod
@ -425,6 +431,7 @@ class ImageUpload(BaseModel):
o.format = d.get('format')
o.username = d.get('username', 'zuul')
o.python_path = d.get('python_path', '/usr/bin/python2')
o.shell_type = d.get('shell_type')
return o
@ -551,7 +558,9 @@ class Node(BaseModel):
self.public_ipv6 = None
self.host_id = None
self.interface_ip = None
self.connection_type = None
self.connection_port = 22
self.shell_type = None
self.image_id = None
self.launcher = None
self.created_time = None
@ -560,7 +569,6 @@ class Node(BaseModel):
self.comment = None
self.hold_job = None
self.username = None
self.connection_type = None
self.host_keys = []
self.hold_expiration = None
self.resources = None
@ -600,6 +608,7 @@ class Node(BaseModel):
self.username == other.username and
self.connection_type == other.connection_type and
self.connection_port == other.connection_port and
self.shell_type == other.shell_type and
self.host_keys == other.host_keys and
self.hold_expiration == other.hold_expiration and
self.resources == other.resources and
@ -650,6 +659,7 @@ class Node(BaseModel):
d['username'] = self.username
d['connection_type'] = self.connection_type
d['connection_port'] = self.connection_port
d['shell_type'] = self.shell_type
d['hold_expiration'] = self.hold_expiration
d['resources'] = self.resources
d['attributes'] = self.attributes
@ -718,6 +728,7 @@ class Node(BaseModel):
self.resources = d.get('resources')
self.attributes = d.get('attributes')
self.python_path = d.get('python_path')
self.shell_type = d.get('shell_type')
class ZooKeeper(object):