Manage a pool of nodes for a distributed test infrastructure
Go to file
James E. Blair 559b01cfa0 Add timeout for ssh negotiation on keyscan
We had a launch thread stuck here:

Thread: NodeLauncher-0000341123 (140201917658880)
  File "/usr/lib/python3.5/threading.py", line 882, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.5/dist-packages/nodepool/driver/openstack/handler.py", line 245, in run
    self._run()
  File "/usr/local/lib/python3.5/dist-packages/nodepool/driver/openstack/handler.py", line 216, in _run
    self._launchNode()
  File "/usr/local/lib/python3.5/dist-packages/nodepool/driver/openstack/handler.py", line 201, in _launchNode
    interface_ip, timeout=self._provider.boot_timeout)
  File "/usr/local/lib/python3.5/dist-packages/nodepool/nodeutils.py", line 74, in keyscan
    t.start_client()
  File "/usr/local/lib/python3.5/dist-packages/paramiko/transport.py", line 489, in start_client
    event.wait(0.1)
  File "/usr/lib/python3.5/threading.py", line 549, in wait
    signaled = self._cond.wait(timeout)
  File "/usr/lib/python3.5/threading.py", line 297, in wait
    gotit = waiter.acquire(True, timeout)

This adds a timeout to that method so paramiko won't get stuck there.

Change-Id: I038d88cb141f57b93d8572c067e714f4a3af9c2d
2017-10-20 16:58:11 -04:00
devstack Add support to test fedora-26 2017-07-12 07:06:45 -04:00
doc Fix nodepool's docs about config-drive 2017-09-03 18:55:25 -05:00
nodepool Add timeout for ssh negotiation on keyscan 2017-10-20 16:58:11 -04:00
tools Move the fakeprovider module to the fake driver 2017-07-28 11:35:07 +00:00
.gitignore Add files for zuul-nodepool integration test 2017-01-24 09:46:08 -08:00
.gitreview Set default branch to feature/zuulv3 2016-08-18 09:08:23 -07:00
.testr.conf Add a test for subnodes 2014-03-31 09:22:00 -07:00
.zuul.yaml Add zuulv3 jobs for nodepool 2017-07-26 14:11:40 -04:00
LICENSE Add LICENSE file 2015-12-10 12:54:23 -05:00
README.rst Rename nodepoold to nodepool-launcher 2017-03-29 09:28:33 -04:00
bindep.txt Add libffi development headers to bindep 2017-03-21 08:53:12 -04:00
requirements.txt Add boot-from-volume support for nodes 2017-05-30 14:23:24 -04:00
setup.cfg Fix internal doc refs to renamed section 2017-04-05 11:07:03 -07:00
setup.py Bump pbr requirements to >=1.3 2015-09-14 16:19:13 -04:00
test-requirements.txt Block sphinx 1.6 2017-05-16 16:35:51 -05:00
tox.ini Set base environment as python3 2017-08-28 15:06:35 -04:00

README.rst

Nodepool

Nodepool is a service used by the OpenStack CI team to deploy and manage a pool of devstack images on a cloud server for use in OpenStack project testing.

Developer setup

Make sure you have pip installed:

wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py

Install dependencies:

sudo pip install bindep
sudo apt-get install $(bindep -b nodepool)

mkdir src
cd ~/src
git clone git://git.openstack.org/openstack-infra/system-config
git clone git://git.openstack.org/openstack-infra/nodepool
cd nodepool
sudo pip install -U -r requirements.txt
sudo pip install -e .

If you're testing a specific patch that is already in gerrit, you will also want to install git-review and apply that patch while in the nodepool directory, ie:

git review -x XXXXX

Create or adapt a nodepool yaml file. You can adapt an infra/system-config one, or fake.yaml as desired. Note that fake.yaml's settings won't Just Work - consult ./modules/openstack_project/templates/nodepool/nodepool.yaml.erb in the infra/system-config tree to see a production config.

If the cloud being used has no default_floating_pool defined in nova.conf, you will need to define a pool name using the nodepool yaml file to use floating ips.

Export variable for your ssh key so you can log into the created instances:

export NODEPOOL_SSH_KEY=`cat ~/.ssh/id_rsa.pub | awk '{print $2}'`

Start nodepool with a demo config file (copy or edit fake.yaml to contain your data):

export STATSD_HOST=127.0.0.1
export STATSD_PORT=8125
nodepool-launcher -d -c tools/fake.yaml

All logging ends up in stdout.

Use the following tool to check on progress:

nodepool image-list