127 lines
5.0 KiB
ReStructuredText
127 lines
5.0 KiB
ReStructuredText
:title: Bridge
|
|
|
|
.. _bridge:
|
|
|
|
Bridge
|
|
######
|
|
|
|
Bridge is a bastion host that is the starting point for ops operations in
|
|
OpenDev. It is the server from which Ansible is run, and contains a
|
|
centralized database that contains secure information such as passwords.
|
|
The bridge server contains all of the ansible playbooks as well as the
|
|
scripts to create new servers.
|
|
|
|
Many of the systems in OpenDev are still configured using puppet, although
|
|
the trend is away from Puppet to Ansible. For the hosts still using puppet,
|
|
the process is still driven by Ansible.
|
|
|
|
At a Glance
|
|
===========
|
|
|
|
:Projects:
|
|
* https://ansible.com/
|
|
* https://puppetlabs.com/
|
|
:Bugs:
|
|
* https://storyboard.openstack.org/#!/project/748
|
|
* https://tickets.puppetlabs.com/
|
|
:Resources:
|
|
* `Puppet Language Reference <https://docs.puppetlabs.com/references/latest/type.html>`_
|
|
|
|
Ansible Hosts
|
|
-------------
|
|
In OpenDev, all host configuration is done via ansible playbooks.
|
|
|
|
Puppet Hosts
|
|
------------
|
|
|
|
For hosts still using puppet, ansible drives the running of ``puppet apply``
|
|
on hosts in the inventory in the ``puppet`` group. That process first
|
|
copies appropriate ``hiera`` data files to each host.
|
|
|
|
Hiera uses a systemwide configuration file in ``/etc/puppet/hiera.yaml``.
|
|
|
|
The hiera configuration is placed by ansible into common.yaml in
|
|
``/etc/puppet/hieradata/production`` on each puppet host.
|
|
The values are simple key-value pairs in yaml format. The keys needed are the
|
|
keys referenced in ``site.pp``, their values are typically obvious
|
|
(strings, lists of strings). ``/etc/puppet/hieradata/`` and below should be
|
|
owned by ``puppet:puppet`` and have mode ``0711``.
|
|
|
|
Below the ``hieradata`` directory, there should be a ``common.yaml`` file where
|
|
settings that should be available to all servers in the infrastructure go,
|
|
and then two directories full of files. The first is ``fqdn`` which should
|
|
contain a yaml file for every server in the infrastructure named
|
|
``${fqdn_of_server}.yaml``. That file has secrets that are only for that
|
|
server. Additionally, some servers can have a ``$group`` defined in
|
|
``manifests/site.pp``. There can be a correspondingly named yaml file in the
|
|
``group`` directory that contains secrets to be made available to each
|
|
server in the group.
|
|
|
|
All of the actual yaml files should have mode 0600 and be owned by root.
|
|
|
|
Adding a node
|
|
-------------
|
|
|
|
Adding a new node should be done using the
|
|
``/opt/system-config/launch/launch-node.py`` script
|
|
(see :git_file:`launch/README.rst` for full details). If the host is put into
|
|
the puppet group in the Ansible inventory, puppet will be run on on the host.
|
|
|
|
.. _running-ansible-on-nodes:
|
|
|
|
Running Ansible on Nodes
|
|
------------------------
|
|
|
|
Each service that has been migrated fully to Ansible has its own playbook in
|
|
:git_file:`playbooks` named ``service_{ service_name }.yaml``.
|
|
|
|
Because the playbooks are normally run by zuul, to run them manually, first
|
|
run the utility ``disable-ansible`` as root. That will touch the file
|
|
``/home/zuul/DISABLE-ANSIBLE``. We use the utility to avoid mistyping the
|
|
lockfile name. Then make sure no jobs are
|
|
currently executing ansible. Ensure that ``/home/zuul/src/opendev.org/opendev/system-config``
|
|
and ``/home/zuul/src/opendev.org/openstack/project-config`` are in the
|
|
appropriate states, then run:
|
|
|
|
.. code-block:: bash
|
|
|
|
cd /home/zuul/src/opendev.org/opendev/system-config
|
|
ansible-playbook --limit="$HOST:localhost" playbooks/service-$SERVICE.yaml
|
|
|
|
as root, where `$HOST` is the host you want to run puppet on.
|
|
The `:localhost` is important as some of the plays depend on performing a task
|
|
on the localhost before continuing to the host in question, and without it in
|
|
the limit section, the tasks for the host will have undefined values.
|
|
|
|
When done, don't forget to remove ``/home/zuul/DISABLE-ANSIBLE``
|
|
|
|
Running Puppet on Nodes
|
|
-----------------------
|
|
|
|
In OpenDev, puppet is run by ansible from a Zuul job running on bridge
|
|
which in turn runs a single run of puppet apply on each host using puppet.
|
|
|
|
The entry point for this process is :git_file:`playbooks/remote_puppet_else.yaml`
|
|
|
|
If an admin needs to run puppet by hand, it's a simple matter of following the
|
|
instructions in :ref:`running-ansible-on-nodes` but using ``playbooks/remote_puppet_else.yaml``
|
|
as the playbook.
|
|
|
|
Alternately, if local iteration is desired, it's possible to log in to the
|
|
server in question and running `puppet apply /opt/system-config/manifests/site.pp`.
|
|
|
|
There is also a script, `tools/kick.sh` that takes the host as an argument
|
|
and runs the above command.
|
|
|
|
Testing new puppet code can be done via `puppet apply --noop` or by
|
|
constructing a VM with a puppet install in it and just running `puppet apply`
|
|
on the code in question. This should actually make it fairly easy to test
|
|
how production works in a more self-contained manner.
|
|
|
|
Disabling Ansible on Nodes
|
|
--------------------------
|
|
|
|
In the case of needing to disable the running of ansible on a node, it's a
|
|
simple matter of adding an entry to the ansible inventory "disabled" group.
|
|
See the :ref:`disable-enable-ansible` section for more details.
|