Update nodepool to use secure.conf file

Depends on a new secure.conf file to grab mysql
connection and jenkins credentials. By this way,
nodepool.yaml is just a flat file without credentials,
that can be passed externally.

Co-Authored-By: Ramy Asselin <ramy.asselin@hpe.com>
Depends-On: Ie9381740e3644feaee1f1b201499e3a253677f39
Change-Id: Ifa4d500c52974b9fd3a0b7fd9c28cf8f52899ca9

devstack/plugin.sh

@@ -89,6 +89,7 @@ EOF
 
 function nodepool_write_config {
     sudo mkdir -p $(dirname $NODEPOOL_CONFIG)
+    sudo mkdir -p $(dirname $NODEPOOL_SECURE)
     local dburi=$(database_connection_url nodepool)
 
     cat > /tmp/logging.conf <<EOF
@@ -123,6 +124,16 @@ EOF
 
     sudo mv /tmp/logging.conf $NODEPOOL_LOGGING
 
+    cat > /tmp/secure.conf << EOF
+[database]
+# The mysql password here may be different depending on your
+# devstack install, you should double check it (the devstack var
+# is MYSQL_PASSWORD and if unset devstack should prompt you for
+# the value).
+dburi: $dburi
+EOF
+    sudo mv /tmp/secure.conf $NODEPOOL_SECURE
+
     cat > /tmp/nodepool.yaml <<EOF
 # You will need to make and populate these two paths as necessary,
 # cloning nodepool does not do this. Further in this doc we have an
@@ -255,7 +266,7 @@ function start_nodepool {
     # start gearman server
     run_process geard "geard -p 8991 -d"
 
-    run_process nodepool "nodepoold -c $NODEPOOL_CONFIG -l $NODEPOOL_LOGGING -d"
+    run_process nodepool "nodepoold -c $NODEPOOL_CONFIG -s $NODEPOOL_SECURE -l $NODEPOOL_LOGGING -d"
     :
 }
 

devstack/settings

@@ -1,5 +1,6 @@
 NODEPOOL_CONFIG=/etc/nodepool/nodepool.yaml
 NODEPOOL_LOGGING=/etc/nodepool/logging.conf
+NODEPOOL_SECURE=/etc/nodepool/secure.conf
 NODEPOOL_IMAGE_URL=https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
 NODEPOOL_IMAGE=$(basename "$NODEPOOL_IMAGE_URL" ".img")
 NODEPOOL_DIB_BASE_PATH=/opt/dib

doc/source/configuration.rst

@@ -3,6 +3,57 @@
 Configuration
 =============
 
+Nodepool reads its secure configuration from ``/etc/nodepool/secure.conf``
+by default. The secure file is a standard ini config file, with
+one section for database, and another section for the jenkins
+secrets for each target::
+
+  [database]
+  dburi={dburi}
+
+  [jenkins "{target_name}"]
+  user={user}
+  apikey={apikey}
+  credentials={credentials}
+  url={url}
+
+Following settings are available::
+
+**required**
+
+  ``dburi``
+  Indicates the URI for the database connection.  See the `SQLAlchemy
+  documentation
+  <http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls>`_
+  for the syntax.  Example::
+
+    dburi='mysql+pymysql://nodepool@localhost/nodepool'
+
+**optional**
+
+  While it is possible to run Nodepool without any Jenkins targets,
+  if Jenkins is used, the `target_name` and `url` are required. The
+  `user`, `apikey` and `credentials` also may be needed depending on
+  the Jenkins security settings.
+
+  ``target_name``
+  Name of the jenkins target. It needs to match with a target
+  specified in nodepool.yaml, in order to retrieve its settings.
+
+  ``url``
+  Url to the Jenkins REST API.
+
+  ``user``
+  Jenkins username.
+
+  ``apikey``
+  API key generated by Jenkins (not the user password).
+
+  ``credentials``
+  If provided, Nodepool will configure the Jenkins slave to use the Jenkins
+  credential identified by that ID, otherwise it will use the username and
+  ssh keys configured in the image.
+
 Nodepool reads its configuration from ``/etc/nodepool/nodepool.yaml``
 by default.  The configuration file follows the standard YAML syntax
 with a number of sections defined with top level keys.  For example, a
@@ -60,15 +111,6 @@ Example::
 
   images-dir: /path/to/images/dir
 
-dburi
------
-Indicates the URI for the database connection.  See the `SQLAlchemy
-documentation
-<http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls>`_
-for the syntax.  Example::
-
-  dburi: 'mysql+pymysql://nodepool@localhost/nodepool'
-
 cron
 ----
 This section is optional.
@@ -469,19 +511,9 @@ across all of the targets which are on-line::
 
   targets:
     - name: jenkins1
-      jenkins:
-        url: https://jenkins1.example.org/
-        user: username
-        apikey: key
-        credentials-id: id
       hostname: '{label.name}-{provider.name}-{node_id}'
       subnode-hostname: '{label.name}-{provider.name}-{node_id}-{subnode_id}'
     - name: jenkins2
-      jenkins:
-        url: https://jenkins2.example.org/
-        user: username
-        apikey: key
-        credentials-id: id
       hostname: '{label.name}-{provider.name}-{node_id}'
       subnode-hostname: '{label.name}-{provider.name}-{node_id}-{subnode_id}'
 
@@ -502,24 +534,6 @@ across all of the targets which are on-line::
     In seconds. Default 1.0
 
   ``jenkins`` (dict)
-    ``url``
-      Url to the Jenkins REST API.
-
-    ``user``
-      Jenkins username.
-
-    ``apikey``
-      API key generated by Jenkins (not the user password).
-
-    ``credentials-id`` (optional)
-      This credential is used by the Jenkins Master to login to the configured
-      Slaves.
-
-      If provided, it is used by the Jenkins Master to log in to the Jenkins
-      Slaves. This id is a Jenkins credential identified by that ID. If not
-      provided it will use the username and ssh keys configured in the image
-      instead. In this case ensure that the path to the ssh keys exist on the
-      Jenkins Master system as well as your Nodepool system.
 
     ``test-job`` (optional)
       Setting this would cause a newly created instance to be in a TEST state.

doc/source/installation.rst

@@ -83,8 +83,12 @@ pyzmq used by nodepool.
 Configuration
 -------------
 
-Nodepool has a single required configuration file and an optional
+Nodepool has two required configuration files: secure.conf and
-logging configuration file.
+nodepool.yaml, and an optional logging configuration file logging.conf.
+The secure.conf file is used to store nodepool configurations that contain
+sensitive data, such as the Nodepool database password and Jenkins
+api key. The nodepool.yaml files is used to store all other
+configurations.
 
 The logging configuration file is in the standard python logging
 `configuration file format

nodepool/cmd/config_validator.py

@@ -86,10 +86,7 @@ class ConfigValidator:
             'hostname': str,
             'subnode-hostname': str,
             'jenkins': {
-                'url': str,
+                'test-job': str
-                'user': str,
-                'apikey': str,
-                'credentials-id': str,
             }
         }
 
@@ -104,7 +101,6 @@ class ConfigValidator:
             'script-dir': str,
             'elements-dir': str,
             'images-dir': str,
-            'dburi': str,
             'zmq-publishers': [str],
             'gearman-servers': [{
                 'host': str,

nodepool/cmd/nodepoolcmd.py

@@ -46,6 +46,9 @@ class NodePoolCmd(object):
         parser.add_argument('-c', dest='config',
                             default='/etc/nodepool/nodepool.yaml',
                             help='path to config file')
+        parser.add_argument('-s', dest='secure',
+                            default='/etc/nodepool/secure.conf',
+                            help='path to secure file')
         parser.add_argument('--version', action='version',
                             version=npc_version_info.version_string(),
                             help='show version')
@@ -343,13 +346,14 @@ class NodePoolCmd(object):
         validator = ConfigValidator(self.args.config)
         validator.validate()
         log.info("Configuation validation complete")
+        #TODO(asselin,yolanda): add validation of secure.conf
 
     def main(self):
         # commands which do not need to start-up or parse config
         if self.args.command in ('config-validate'):
             return self.args.func()
 
-        self.pool = nodepool.NodePool(self.args.config)
+        self.pool = nodepool.NodePool(self.args.secure, self.args.config)
         config = self.pool.loadConfig()
         self.pool.reconfigureDatabase(config)
         self.pool.setConfig(config)

nodepool/cmd/nodepoold.py

@@ -86,6 +86,9 @@ class NodePoolDaemon(object):
         parser.add_argument('-c', dest='config',
                             default='/etc/nodepool/nodepool.yaml',
                             help='path to config file')
+        parser.add_argument('-s', dest='secure',
+                            default='/etc/nodepool/secure.conf',
+                            help='path to secure file')
         parser.add_argument('-d', dest='nodaemon', action='store_true',
                             help='do not run as a daemon')
         parser.add_argument('-l', dest='logconfig',
@@ -118,7 +121,8 @@ class NodePoolDaemon(object):
     def main(self):
         import nodepool.nodepool
         self.setup_logging()
-        self.pool = nodepool.nodepool.NodePool(self.args.config)
+        self.pool = nodepool.nodepool.NodePool(self.args.secure,
+                                               self.args.config)
 
         signal.signal(signal.SIGINT, self.exit_handler)
         # For back compatibility:

nodepool/nodepool.py

@@ -16,6 +16,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from six.moves import configparser as ConfigParser
 import apscheduler.scheduler
 import gear
 import json
@@ -1265,8 +1266,10 @@ class DiskImage(ConfigValue):
 class NodePool(threading.Thread):
     log = logging.getLogger("nodepool.NodePool")
 
-    def __init__(self, configfile, watermark_sleep=WATERMARK_SLEEP):
+    def __init__(self, securefile, configfile,
+                 watermark_sleep=WATERMARK_SLEEP):
         threading.Thread.__init__(self, name='NodePool')
+        self.securefile = securefile
         self.configfile = configfile
         self.watermark_sleep = watermark_sleep
         self._stopped = False
@@ -1300,6 +1303,8 @@ class NodePool(threading.Thread):
 
     def loadConfig(self):
         self.log.debug("Loading configuration")
+        secure = ConfigParser.ConfigParser()
+        secure.readfp(open(self.securefile))
         config = yaml.load(open(self.configfile))
         cloud_config = os_client_config.OpenStackConfig()
 
@@ -1312,7 +1317,7 @@ class NodePool(threading.Thread):
         newconfig.scriptdir = config.get('script-dir')
         newconfig.elementsdir = config.get('elements-dir')
         newconfig.imagesdir = config.get('images-dir')
-        newconfig.dburi = config.get('dburi')
+        newconfig.dburi = secure.get('database', 'dburi')
         newconfig.provider_managers = {}
         newconfig.jenkins_managers = {}
         newconfig.zmq_publishers = {}
@@ -1443,24 +1448,35 @@ class NodePool(threading.Thread):
                 l.providers[p.name] = p
 
         for target in config['targets']:
+            # look at secure file for that section
+            section_name = 'jenkins "%s"' % target['name']
             t = Target()
             t.name = target['name']
             newconfig.targets[t.name] = t
-            jenkins = target.get('jenkins')
             t.online = True
-            if jenkins:
+
-                t.jenkins_url = jenkins['url']
+            if secure.has_section(section_name):
-                t.jenkins_user = jenkins['user']
+                t.jenkins_url = secure.get(section_name, 'url')
-                t.jenkins_apikey = jenkins['apikey']
+                t.jenkins_user = secure.get(section_name, 'user')
-                t.jenkins_credentials_id = jenkins.get('credentials-id')
+                t.jenkins_apikey = secure.get(section_name, 'apikey')
-                t.jenkins_test_job = jenkins.get('test-job')
             else:
                 t.jenkins_url = None
                 t.jenkins_user = None
                 t.jenkins_apikey = None
-                t.jenkins_credentials_id = None
+
-                t.jenkins_test_job = None
             t.rate = target.get('rate', 1.0)
+            try:
+                t.jenkins_credentials_id = secure.get(
+                    section_name, 'credentials')
+            except:
+                t.jenkins_credentials_id = None
+
+            jenkins = target.get('jenkins')
+            if jenkins:
+                t.jenkins_test_job = jenkins.get('test-job', None)
+            else:
+                t.jenkins_test_job = None
+
             t.hostname = target.get(
                 'hostname',
                 '{label.name}-{provider.name}-{node_id}'

nodepool/tests/__init__.py

@@ -181,11 +181,6 @@ class BaseTestCase(testtools.TestCase, testresources.ResourcedTestCase):
                 return
             time.sleep(0.1)
 
-    def useNodepool(self, *args, **kwargs):
-        pool = nodepool.NodePool(*args, **kwargs)
-        self.addCleanup(pool.stop)
-        return pool
-
 
 class AllocatorTestCase(object):
     def setUp(self):
@@ -253,6 +248,7 @@ class DBTestCase(BaseTestCase):
         f = MySQLSchemaFixture()
         self.useFixture(f)
         self.dburi = f.dburi
+        self.secure_conf = self._setup_secure()
 
         gearman_fixture = GearmanServerFixture()
         self.useFixture(gearman_fixture)
@@ -265,12 +261,21 @@ class DBTestCase(BaseTestCase):
                                   'fixtures', filename)
         config = open(configfile).read()
         (fd, path) = tempfile.mkstemp()
-        os.write(fd, config.format(dburi=self.dburi,
+        os.write(fd, config.format(images_dir=images_dir.path,
-                                   images_dir=images_dir.path,
                                    gearman_port=self.gearman_server.port))
         os.close(fd)
         return path
 
+    def _setup_secure(self):
+        # replace entries in secure.conf
+        configfile = os.path.join(os.path.dirname(__file__),
+                                  'fixtures', 'secure.conf')
+        config = open(configfile).read()
+        (fd, path) = tempfile.mkstemp()
+        os.write(fd, config.format(dburi=self.dburi))
+        os.close(fd)
+        return path
+
     def wait_for_config(self, pool):
         for x in range(300):
             if pool.config is not None:
@@ -303,6 +308,12 @@ class DBTestCase(BaseTestCase):
             time.sleep(1)
         self.wait_for_threads()
 
+    def useNodepool(self, *args, **kwargs):
+        args = (self.secure_conf,) + args
+        pool = nodepool.NodePool(*args, **kwargs)
+        self.addCleanup(pool.stop)
+        return pool
+
 
 class IntegrationTestCase(DBTestCase):
     def setUpFakes(self):

nodepool/tests/fixtures/config_validate/good.yaml

@@ -1,7 +1,6 @@
 script-dir: /etc/nodepool/scripts
 elements-dir: /etc/nodepool/elements
 images-dir: /opt/nodepool_dib
-dburi: 'mysql+pymysql://nodepool:<%= mysql_password %>@localhost/nodepool'
 
 cron:
   cleanup: '*/1 * * * *'
@@ -943,47 +942,12 @@ providers:
 
 targets:
   - name: jenkins01
-    jenkins:
-      url: 'https://jenkins01.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins02
-    jenkins:
-      url: 'https://jenkins02.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins03
-    jenkins:
-      url: 'https://jenkins03.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins04
-    jenkins:
-      url: 'https://jenkins04.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins05
-    jenkins:
-      url: 'https://jenkins05.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins06
-    jenkins:
-      url: 'https://jenkins06.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins07
-    jenkins:
-      url: 'https://jenkins07.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
 
 diskimages:
   - name: devstack-trusty

nodepool/tests/fixtures/config_validate/schema_error.yaml

@@ -1,6 +1,5 @@
 script-dir: /etc/nodepool/scripts
 elements-dir: /etc/nodepool/elements
 images-dir: /opt/nodepool_dib
-dburi: 'mysql+pymysql://nodepool:<%= mysql_password %>@localhost/nodepool'
 
 nothandled: element

nodepool/tests/fixtures/config_validate/yaml_error.yaml

@@ -1,7 +1,6 @@
 script-dir: /etc/nodepool/scripts
 elements-dir: /etc/nodepool/elements
 images-dir: /opt/nodepool_dib
-dburi: 'mysql+pymysql://nodepool:<%= mysql_password %>@localhost/nodepool'
 
 cron:
   cleanup: '*/1 * * * *'
@@ -948,47 +947,12 @@ providers:
 
 targets:
   - name: jenkins01
-    jenkins:
-      url: 'https://jenkins01.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins02
-    jenkins:
-      url: 'https://jenkins02.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins03
-    jenkins:
-      url: 'https://jenkins03.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins04
-    jenkins:
-      url: 'https://jenkins04.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins05
-    jenkins:
-      url: 'https://jenkins05.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins06
-    jenkins:
-      url: 'https://jenkins06.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
   - name: jenkins07
-    jenkins:
-      url: 'https://jenkins07.openstack.org/'
-      user: '<%= jenkins_api_user %>'
-      apikey: '<%= jenkins_api_key %>'
-      credentials-id: '<%= jenkins_credentials_id %>'
 
 diskimages:
   - name: devstack-trusty

nodepool/tests/fixtures/leaked_node.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 images-dir: '{images_dir}'
 
 cron:
@@ -46,7 +45,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/fixtures/node.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 images-dir: '{images_dir}'
 
 cron:
@@ -46,7 +45,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/fixtures/node_az.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 images-dir: '{images_dir}'
 
 cron:
@@ -44,7 +43,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/fixtures/node_cmd.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 images-dir: '{images_dir}'
 
 cron:
@@ -70,7 +69,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/fixtures/node_dib.yaml

@@ -2,8 +2,6 @@ script-dir: .
 elements-dir: .
 images-dir: '{images_dir}'
 
-dburi: '{dburi}'
-
 cron:
   check: '*/15 * * * *'
   cleanup: '*/1 * * * *'
@@ -43,10 +41,6 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake
 
 diskimages:
   - name: fake-dib-diskimage

nodepool/tests/fixtures/node_dib_and_snap.yaml

@@ -2,8 +2,6 @@ script-dir: .
 elements-dir: .
 images-dir: '{images_dir}'
 
-dburi: '{dburi}'
-
 cron:
   check: '*/15 * * * *'
   cleanup: '*/1 * * * *'
@@ -60,10 +58,6 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake
 
 diskimages:
   - name: fake-dib-image

nodepool/tests/fixtures/node_dib_and_snap_fail.yaml

@@ -2,8 +2,6 @@ script-dir: .
 elements-dir: .
 images-dir: '{images_dir}'
 
-dburi: '{dburi}'
-
 cron:
   check: '*/15 * * * *'
   cleanup: '*/1 * * * *'
@@ -60,10 +58,6 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake
 
 diskimages:
   - name: fake-dib-image

nodepool/tests/fixtures/node_dib_and_snap_upload_fail.yaml

@@ -2,8 +2,6 @@ script-dir: .
 elements-dir: .
 images-dir: '{images_dir}'
 
-dburi: '{dburi}'
-
 cron:
   check: '*/15 * * * *'
   cleanup: '*/1 * * * *'
@@ -62,10 +60,6 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake
 
 diskimages:
   - name: fake-dib-image

nodepool/tests/fixtures/node_dib_vhd.yaml

@@ -2,8 +2,6 @@ script-dir: .
 elements-dir: .
 images-dir: '{images_dir}'
 
-dburi: '{dburi}'
-
 cron:
   check: '*/15 * * * *'
   cleanup: '*/1 * * * *'
@@ -44,10 +42,6 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake
 
 diskimages:
   - name: fake-dib-image

nodepool/tests/fixtures/node_dib_vhd_and_qcow2.yaml

@@ -2,8 +2,6 @@ script-dir: .
 elements-dir: .
 images-dir: '{images_dir}'
 
-dburi: '{dburi}'
-
 cron:
   check: '*/15 * * * *'
   cleanup: '*/1 * * * *'
@@ -62,10 +60,6 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake
 
 diskimages:
   - name: fake-dib-image

nodepool/tests/fixtures/node_ipv6.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 
 cron:
   check: '*/15 * * * *'
@@ -91,7 +90,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/fixtures/node_osc.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 images-dir: '{images_dir}'
 
 cron:
@@ -42,7 +41,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/fixtures/secure.conf

@@ -0,0 +1,8 @@
+[database]
+dburi={dburi}
+
+[jenkins "fake-target"]
+user=fake
+apikey=fake
+credentials=fake
+url=http://fake-url

nodepool/tests/fixtures/subnodes.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: '{dburi}'
 images-dir: '{images_dir}'
 
 cron:
@@ -49,7 +48,3 @@ providers:
 
 targets:
   - name: fake-target
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

nodepool/tests/test_commands.py

@@ -25,7 +25,7 @@ from nodepool import tests
 
 class TestNodepoolCMD(tests.DBTestCase):
     def patch_argv(self, *args):
-        argv = ["nodepool"]
+        argv = ["nodepool", "-s", self.secure_conf]
         argv.extend(args)
         self.useFixture(fixtures.MonkeyPatch('sys.argv', argv))
 

nodepool/tests/test_nodepool.py

@@ -124,7 +124,7 @@ class TestNodepool(tests.DBTestCase):
     def test_dib_and_snap_fail(self):
         """Test that snap based nodes build when dib fails."""
         configfile = self.setup_config('node_dib_and_snap_fail.yaml')
-        pool = nodepool.nodepool.NodePool(configfile, watermark_sleep=1)
+        pool = self.useNodepool(configfile, watermark_sleep=1)
         pool.start()
         self.addCleanup(pool.stop)
         # fake-provider1 will fail to build fake-dib-image

tools/check_devstack_plugin.sh

@@ -1,7 +1,8 @@
 #!/bin/bash -x
 
 NODEPOOL_CONFIG=${NODEPOOL_CONFIG:-/etc/nodepool/nodepool.yaml}
-NODEPOOL="nodepool -c $NODEPOOL_CONFIG"
+NODEPOOL_SECURE=${NODEPOOL_SECURE:-/etc/nodepool/secure.conf}
+NODEPOOL="nodepool -c $NODEPOOL_CONFIG -s $NODEPOOL_SECURE"
 
 function waitforimage {
     name=$1

tools/fake-dib.yaml

@@ -1,7 +1,6 @@
 script-dir: .
 elements-dir: .
 images-dir: /tmp/nodepool_dib
-dburi: 'mysql+pymysql://nodepool@localhost/nodepool'
 
 cron:
   check: '*/15 * * * *'
@@ -48,7 +47,3 @@ providers:
 
 targets:
   - name: fake-jenkins
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake

tools/fake.yaml

@@ -1,5 +1,4 @@
 script-dir: .
-dburi: 'mysql+pymysql://nodepool@localhost/nodepool'
 
 cron:
   check: '*/15 * * * *'
@@ -50,7 +49,3 @@ providers:
 
 targets:
   - name: fake-jenkins
-    jenkins:
-      url: https://jenkins.example.org/
-      user: fake
-      apikey: fake