system-config/doc/source/bridge.rst

127 lines
5.0 KiB
ReStructuredText

:title: Bridge
.. _bridge:
Bridge
######
Bridge is a bastion host that is the starting point for ops operations in
OpenDev. It is the server from which Ansible is run, and contains a
centralized database that contains secure information such as passwords.
The bridge server contains all of the ansible playbooks as well as the
scripts to create new servers.
Many of the systems in OpenDev are still configured using puppet, although
the trend is away from Puppet to Ansible. For the hosts still using puppet,
the process is still driven by Ansible.
At a Glance
===========
:Projects:
* https://ansible.com/
* https://puppetlabs.com/
:Bugs:
* https://storyboard.openstack.org/#!/project/748
* https://tickets.puppetlabs.com/
:Resources:
* `Puppet Language Reference <https://docs.puppetlabs.com/references/latest/type.html>`_
Ansible Hosts
-------------
In OpenDev, all host configuration is done via ansible playbooks.
Puppet Hosts
------------
For hosts still using puppet, ansible drives the running of ``puppet apply``
on hosts in the inventory in the ``puppet`` group. That process first
copies appropriate ``hiera`` data files to each host.
Hiera uses a systemwide configuration file in ``/etc/puppet/hiera.yaml``.
The hiera configuration is placed by ansible into common.yaml in
``/etc/puppet/hieradata/production`` on each puppet host.
The values are simple key-value pairs in yaml format. The keys needed are the
keys referenced in ``site.pp``, their values are typically obvious
(strings, lists of strings). ``/etc/puppet/hieradata/`` and below should be
owned by ``puppet:puppet`` and have mode ``0711``.
Below the ``hieradata`` directory, there should be a ``common.yaml`` file where
settings that should be available to all servers in the infrastructure go,
and then two directories full of files. The first is ``fqdn`` which should
contain a yaml file for every server in the infrastructure named
``${fqdn_of_server}.yaml``. That file has secrets that are only for that
server. Additionally, some servers can have a ``$group`` defined in
``manifests/site.pp``. There can be a correspondingly named yaml file in the
``group`` directory that contains secrets to be made available to each
server in the group.
All of the actual yaml files should have mode 0600 and be owned by root.
Adding a node
-------------
Adding a new node should be done using the
``/opt/system-config/launch/launch-node.py`` script
(see :git_file:`launch/README.rst` for full details). If the host is put into
the puppet group in the Ansible inventory, puppet will be run on on the host.
.. _running-ansible-on-nodes:
Running Ansible on Nodes
------------------------
Each service that has been migrated fully to Ansible has its own playbook in
:git_file:`playbooks` named ``service_{ service_name }.yaml``.
Because the playbooks are normally run by zuul, to run them manually, first
run the utility ``disable-ansible`` as root. That will touch the file
``/home/zuul/DISABLE-ANSIBLE``. We use the utility to avoid mistyping the
lockfile name. Then make sure no jobs are
currently executing ansible. Ensure that ``/home/zuul/src/opendev.org/opendev/system-config``
and ``/home/zuul/src/opendev.org/openstack/project-config`` are in the
appropriate states, then run:
.. code-block:: bash
cd /home/zuul/src/opendev.org/opendev/system-config
ansible-playbook --limit="$HOST:localhost" playbooks/service-$SERVICE.yaml
as root, where `$HOST` is the host you want to run puppet on.
The `:localhost` is important as some of the plays depend on performing a task
on the localhost before continuing to the host in question, and without it in
the limit section, the tasks for the host will have undefined values.
When done, don't forget to remove ``/home/zuul/DISABLE-ANSIBLE``
Running Puppet on Nodes
-----------------------
In OpenDev, puppet is run by ansible from a Zuul job running on bridge
which in turn runs a single run of puppet apply on each host using puppet.
The entry point for this process is :git_file:`playbooks/remote_puppet_else.yaml`
If an admin needs to run puppet by hand, it's a simple matter of following the
instructions in :ref:`running-ansible-on-nodes` but using ``playbooks/remote_puppet_else.yaml``
as the playbook.
Alternately, if local iteration is desired, it's possible to log in to the
server in question and running `puppet apply /opt/system-config/manifests/site.pp`.
There is also a script, `tools/kick.sh` that takes the host as an argument
and runs the above command.
Testing new puppet code can be done via `puppet apply --noop` or by
constructing a VM with a puppet install in it and just running `puppet apply`
on the code in question. This should actually make it fairly easy to test
how production works in a more self-contained manner.
Disabling Ansible on Nodes
--------------------------
In the case of needing to disable the running of ansible on a node, it's a
simple matter of adding an entry to the ansible inventory "disabled" group.
See the :ref:`disable-enable-ansible` section for more details.