diff --git a/doc/source/index.rst b/doc/source/index.rst index 890a9b9d9..c19ed5b35 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -345,4 +345,4 @@ Indices and tables * :ref:`genindex` * :ref:`modindex` * :ref:`search` - +* :ref:`troubleshooting` diff --git a/doc/source/troubleshooting.rst b/doc/source/troubleshooting.rst new file mode 100644 index 000000000..493d92109 --- /dev/null +++ b/doc/source/troubleshooting.rst @@ -0,0 +1,185 @@ +.. _troubleshooting: + +========================================= +Troubleshooting Ironic-Python-Agent (IPA) +========================================= + +This document contains basic trouble shooting information for IPA. + +Gaining Access to IPA on a node +=============================== +In order to access a running IPA instance a user must be added or enabled on +the image. Below we will cover several ways to do this. + +Access via ssh +-------------- +CoreOS +~~~~~~ +To enable the ``core`` user on pre-built or CoreOS images a ssh public key +will need to added. To do this you will need to: + +- Add ``sshkey="ssh-rsa AAAA..."`` to pxe_append_params setting in ironic.conf + file +- Restart the ironic-conductor with the command + ``service ironic-conductor restart`` +- ``ssh core@<ip-address-of-node>`` + +diskimage-builder (DIB) +~~~~~~~~~~~~~~~~~~~~~~~ +SSH access can be added to DIB built IPA images with the dynamic-login [0]_ +or the devuser element [1]_ + +The dynamic-login element allows the operator to inject a SSH key when the +image boots. Kernel command line parameters are used to do this. + +dynamic-login element example: + +- Add ``sshkey="ssh-rsa BBA1..."`` to pxe_append_params setting in + the ``ironic.conf`` file +- Restart the ironic-conductor with the command + ``service ironic-conductor restart`` + +devuser element example:: + + export DIB_DEV_USER_USERNAME=username + export DIB_DEV_USER_PWDLESS_SUDO=yes + export DIB_DEV_USER_AUTHORIZED_KEYS=$HOME/.ssh/id_rsa.pub + disk-image-create -o /path/to/custom-ipa debian ironic-agent devuser + +Access via console +------------------ +If you need to use console access, passwords must be enabled there are a +couple ways to enable this depending on how the IPA image was created: + +CoreOS +~~~~~~ +CoreOS has support for auto login on the console [4]_. This can be enabled by: + +- Adding ``coreos.autologin`` to pxe_append_params setting + in the ``ironic.conf`` file. See [4]_ for more information on using + autologin. + +If you do not wish to enable auto login users can be added to CoreOS by editing +the cloud-config.yml file and adding the following [2]_:: + + users: + - name: username + passwd: $6$5s2u6/jR$un0AvWnqilcgaNB3Mkxd5... <example password hash> + groups: + - sudo + +If using a pre-built image the cloud-config.yml must first be extracted:: + + mkdir tmp_folder + cd tmp_folder + zcat ../coreos_production_pxe_image-oem-stable-mitaka.cpio | cpio --extract --make-directories + +To create a password hash the mkpasswd command can be used:: + + mkpasswd --method=SHA-512 --rounds=4096 + +After adding the user block with your favorite editor recompress the image:: + + find . | cpio --create --format='newc' |gzip -c -9 > ../coreos_production_pxe_image-oem-stable-mitaka.cpio.NEW.gz + +An alternative to editing the embedded cloud-config.yml [4]_ file is to pass a +new one on the kernel command line by: + +- adding ``cloud-config-url=http://example.com/cloud-config.yml`` + to pxe_append_params setting in the ``ironic.conf`` file + +diskimage-builder (DIB) +~~~~~~~~~~~~~~~~~~~~~~~ +Users wishing to use password access can be add the dynamic-login [0]_ or the +devuser element [1]_ + +The dynamic-login element allows the operator to change the root password +dynamically when the image boots. Kernel command line parameters +are used to do this. + +dynamic-login element example:: + + Generate a ENCRYPTED_PASSWORD with the openssl passwd -1 command + Add rootpwd="$ENCRYPTED_PASSWORD" value on the pxe_append_params setting in /etc/ironic/ironic.conf + Restart the ironic-conductor with the command service ironic-conductor restart + +Users can also be added to DIB built IPA images with the devuser element [1]_ + +Example:: + + export DIB_DEV_USER_USERNAME=username + export DIB_DEV_USER_PWDLESS_SUDO=yes + export DIB_DEV_USER_PASSWORD=PASSWORD + disk-image-create -o /path/to/custom-ipa debian ironic-agent devuser + + +Set IPA to debug logging +======================== +Debug logging can be enabled a several different ways. The easiest way is to +add ``ipa-debug=1`` to the kernel command line. To do this: + +- Append ``ipa-debug=1`` to the pxe_append_params setting in the + ``ironic.conf`` file +- Restart the ironic-conductor with the command + ``service ironic-conductor restart`` + +Another method is to edit the cloud-config.yml file. IPA's instructions on +building a custom image can be found at [3]_. + +This essentially boils down to the following steps: + +#. ``git clone https://git.openstack.org/openstack/ironic-python-agent`` +#. ``cd ironic-python-agent`` +#. ``pip install -r ./requirements.txt`` +#. If not installed, please install the docker container engine. [5]_ +#. ``cd imagebuild/coreos`` +#. Edit ``oem/cloud-config.yml`` and add ``--debug`` to the end of the + ExecStart setting for the ironic-python-agent.service unit. +#. Execute ``make`` to complete the build process. + +If the the system is ruuning and uses systemd then editing the services file +will be required. + +- ``systemctl edit ironic-python-agent.service`` +- Append ``--debug`` to end of the ExecStart command +- Restart IPA. See the `Manually restart IPA`_ section below. + +Where can I find the IPA logs +============================= + +Retrieving the IPA logs will differ depending on which base image was used. + + +* Operating system that do not use ``systemd`` (ie Ubuntu 14.04) + + - logs will be found in the /var/log/ folder. + +* Operating system that do use ``systemd`` (ie Fedora 22, CoreOS) + + - logs may be viewed with ``sudo journalctl -u ironic-python-agent`` + + .. note:: + sudo is not required with the CoreOS images. + + +Manually restart IPA +==================== + +In some cases it is helpful to enable debug mode on a running node. +If the system does not use systemd then IPA can be restarted directly:: + + sudo /usr/local/bin/ironic-python-agent [--debug] + +If the system uses systemd then systemctl can be used to restart the service:: + + sudo systemctl restart ironic-python-agent.service + + +References +========== +.. [0] `Dynamic-login DIB element`: https://github.com/openstack/diskimage-builder/tree/master/elements/dynamic-login +.. [1] `DevUser DIB element`: https://github.com/openstack/diskimage-builder/tree/master/elements/devuser +.. [2] `Add User to CoreOS`: https://coreos.com/os/docs/latest/adding-users.html +.. [3] `IPA image build refference`: https://github.com/openstack/ironic-python-agent/tree/master/imagebuild/coreos/README.rst +.. [4] `Booting CoreOS via PXE`: https://coreos.com/os/docs/latest/booting-with-pxe.html +.. [5] `Install docker engine`: https://docs.docker.com/engine/installation/