.. _configuration: Configuration ============= Nodepool reads its configuration from ``/etc/nodepool/nodepool.yaml`` by default. The configuration file follows the standard YAML syntax with a number of sections defined with top level keys. For example, a full configuration file may have the ``labels``, ``providers``, and ``targets`` sections:: labels: ... providers: ... targets: ... The following sections are available. All are required unless otherwise indicated. script-dir ---------- When creating an image to use when launching new nodes, Nodepool will run a script that is expected to prepare the machine before the snapshot image is created. The ``script-dir`` parameter indicates a directory that holds all of the scripts needed to accomplish this. Nodepool will copy the entire directory to the machine before invoking the appropriate script for the image being created. Example:: script-dir: /path/to/script/dir dburi ----- Indicates the URI for the database connection. See the `SQLAlchemy documentation `_ for the syntax. Example:: dburi: 'mysql://nodepool@localhost/nodepool' cron ---- This section is optional. Nodepool runs several periodic tasks. The ``image-update`` task creates a new image for each of the defined images, typically used to keep the data cached on the images up to date. The ``cleanup`` task deletes old images and servers which may have encountered errors during their initial deletion. The ``check`` task attempts to log into each node that is waiting to be used to make sure that it is still operational. The following illustrates how to change the schedule for these tasks and also indicates their default values:: cron: image-update: '14 2 * * *' cleanup: '27 */6 * * *' check: '*/15 * * * *' zmq-publishers -------------- Lists the ZeroMQ endpoints for the Jenkins masters. Nodepool uses this to receive real-time notification that jobs are running on nodes or are complete and nodes may be deleted. Example:: zmq-publishers: - tcp://jenkins1.example.com:8888 - tcp://jenkins2.example.com:8888 gearman-servers --------------- Lists the Zuul Gearman servers that should be consulted for real-time demand. Nodepool will use information from these servers to determine if additional nodes should be created to satisfy current demand. Example:: gearman-servers: - host: zuul.example.com port: 4730 The ``port`` key is optional. labels ------ Defines the types of nodes that should be created. Maps node types to the images that are used to back them and the providers that are used to supply them. Jobs should be written to run on nodes of a certain label (so targets such as Jenkins don't need to know about what providers or images are used to create them). Example:: labels: - name: my-precise image: precise min-ready: 2 providers: - name: provider1 - name: provider2 - name: multi-precise image: precise subnodes: 2 min-ready: 2 ready-script: setup_multinode.sh providers: - name: provider1 The `name`, `image`, and `min-ready` keys are required. The `providers` list is also required if any nodes should actually be created (e.g., the label is not currently disabled). The `subnodes` key is used to configure multi-node support. If a `subnodes` key is supplied to an image, it indicates that the specified number of additional nodes of the same image type should be created and associated with each node for that image. Only one node from each such group will be added to the target, the subnodes are expected to communicate directly with each other. In the example above, for each Precise node added to the target system, two additional nodes will be created and associated with it. The script specified by `ready-script` (which is expected to be in `/opt/nodepool-scripts` along with the setup script) can be used to perform any last minute changes to a node after it has been launched but before it is put in the READY state to receive jobs. In particular, it can read the files in /etc/nodepool to perform multi-node related setup. Those files include: **/etc/nodepool/role** Either the string ``primary`` or ``sub`` indicating whether this node is the primary (the node added to the target and which will run the job), or a sub-node. **/etc/nodepool/primary_node** The IP address of the primary node. **/etc/nodepool/sub_nodes** The IP addresses of the sub nodes, one on each line. **/etc/nodepool/id_rsa** An OpenSSH private key generated specifically for this node group. **/etc/nodepool/id_rsa.pub** The corresponding public key. providers --------- Lists the OpenStack cloud providers Nodepool should use. Within each provider, the Nodepool image types are also defined. If the resulting images from different providers should be equivalent, give them the same name. Example:: providers: - name: provider1 username: 'username' password: 'password' auth-url: 'http://auth.provider1.example.com/' project-id: 'project' service-type: 'compute' service-name: 'compute' region-name: 'region1' max-servers: 96 rate: 1.0 images: - name: precise base-image: 'Precise' min-ram: 8192 setup: prepare_node.sh reset: reset_node.sh username: jenkins private-key: /var/lib/jenkins/.ssh/id_rsa - name: quantal base-image: 'Quantal' min-ram: 8192 setup: prepare_node.sh reset: reset_node.sh username: jenkins private-key: /var/lib/jenkins/.ssh/id_rsa - name: provider2 username: 'username' password: 'password' auth-url: 'http://auth.provider2.example.com/' project-id: 'project' service-type: 'compute' service-name: 'compute' region-name: 'region1' max-servers: 96 rate: 1.0 images: - name: precise base-image: 'Fake Precise' min-ram: 8192 setup: prepare_node.sh reset: reset_node.sh username: jenkins private-key: /var/lib/jenkins/.ssh/id_rsa For providers, the `name`, `username`, `password`, `auth-url`, `project-id`, and `max-servers` keys are required. For images, the `name`, `base-image`, and `min-ram` keys are required. The `username` and `private-key` values default to the values indicated. Nodepool expects that user to exist after running the script indicated by `setup`. targets ------- Lists the Jenkins masters to which Nodepool should attach nodes after they are created. Nodes of each label will be evenly distributed across all of the targets which are on-line:: targets: - name: jenkins1 jenkins: url: https://jenkins1.example.org/ user: username apikey: key credentials-id: id - name: jenkins2 jenkins: url: https://jenkins2.example.org/ user: username apikey: key credentials-id: id For targets, the `name` is required. If using Jenkins, the `url`, `user`, and `apikey` keys are required. If the `credentials-id` key is provided, Nodepool will configure the Jenkins slave to use the Jenkins credential identified by that ID, otherwise it will use the username and ssh keys configured in the image.