Logging features for Fuel and OpenStack

user-guide/config -- debug logging, configure syslog

op-guide/troubleshooting -- using logs

terminology/logging -- brief overview of logging facilities with
links to our docs and community docs for additional information

Change-Id: I06a5ed5f2e23be6c677177dff03cb8ceb0bb5215
Partial-Bugs: 1321200

pages/operations/9000-troubleshoot.rst

@@ -4,6 +4,7 @@
 Troubleshooting
 ===============
 
+.. include:: /pages/operations/troubleshoot/1000-logs.rst
 .. include:: /pages/operations/troubleshoot/9100-tshoot-corosync-pacemaker.rst
 .. include:: /pages/operations/troubleshoot/9110-verify-neutron-ha-crm.rst
 .. include:: /pages/operations/troubleshoot/9120-corosync-crashes.rst

pages/operations/troubleshoot/1000-logs.rst

@@ -0,0 +1,173 @@
+
+
+.. _logs-top-tshoot:
+
+Logs and messages
+-----------------
+
+A number of logs are available
+to help you understand and troubleshoot
+what is going on in your environment.
+
+.. _screen-messages-tshoot:
+
+Screen messages
++++++++++++++++
+
+Clicking on the icon in the upper right corner
+of the Fuel UI displays a drop-down menu
+of notifications posted as the environment was deployed.
+If notifications have been posted
+that you have not yet viewed,
+this icon is an orange square with a number
+that indicates the number of unread notifications.
+If there are no unread notifications,
+the icon looks as it does here:
+
+.. image:: /_images/user_screen_shots/messages.png
+   :width: 80%
+
+These notifications tell you about the nodes that were deployed
+and when nodes go on and offline.
+Click on any notification
+to get a summary configuration view of the node.
+
+.. _view-logs-fuel:
+
+Viewing Logs through Fuel
++++++++++++++++++++++++++
+
+The Logs tab on the Fuel UI allows you to view
+key logs for any node in the environment.
+Use the drop-down list on the left to select
+whether to view Fuel Master logs
+or the logs of other servers in the environment.
+
+When you choose the "Fuel Master" logs,
+you get this display:
+
+.. image:: /_images/user_screen_shots/master-logs.png
+   :width: 80%
+
+Select the log to view by setting the fields:
+
+:Logs:    Choose between the Fuel Master or "Other servers".
+          When you choose "Other servers", the display changes
+          to provide a drop-down list of all nodes in the environment.
+
+:Source:  Log you want to view.  See the lists below.
+
+:Min Level:    By default, this shows "INFO" messages only.
+               If you are running your environment in debug mode,
+               you can use this field to filter out some of the messages.
+
+When you have set all the fields, click on "SHOW" at the right
+to display the requested log for the specified server.
+
+Viewing Fuel Master logs
+++++++++++++++++++++++++
+
+The following logs can be chosen from the "Source" list
+for the "Fuel Master" node:
+
+:puppet:    Logs activity of the :ref:`puppet-term`
+            configuration management system.
+
+:anaconda:  Logs activities of the Anaconda installation agent
+            used for provisioning.
+
+:syslog:  Shows the **syslog** entries
+          that will be sent to the **rsyslog** server
+          (Fuel Master by default).
+
+
+
+**Other install logs**
+
+:storage:    Log entries for disk partitioning.
+
+:kickstart-pre:   Shows activities before the :ref:`Cobbler<cobbler-term>`
+                  kickstart mechanism runs.
+
+:kickstart-post:  Shows activities after the Cobbler kickstart mechanism runs.
+
+
+**Bootstrap logs**
+
+:Web backend:  Logs each connection from the Master Node
+               to the Internet.
+
+:REST API:    :ref:`Nailgun<nailgun-term>` API activities.
+
+:RPC consumer:    Logs messaging between :ref:`Nailgun<nailgun-term>`
+                  and the :ref:`orchestration service<orchestration-service-term>`
+
+:Astute:  Records the activity of the :ref:`astute-term` agents
+          that implements the :ref:`nailgun-term` configuration tasks.
+
+:Health Check:  Displays the results of the most recent run
+                of the tests run from the "Health Check" tab.
+
+Viewing logs for target nodes ("Other servers")
++++++++++++++++++++++++++++++++++++++++++++++++
+
+When you choose "Other Servers",
+the display includes an extra field
+that you use to choose the node whose logs you want to view:
+
+.. image:: /_images/user_screen_shots/other-nodes-logs.png
+   :width: 80%
+
+Many of the same logs shown for the Fuel Master node
+are also shown for the target nodes.
+The difference is in the nodes given for "Bootstrap logs",
+plus the controller node includes a set of OpenStack logs
+that shows logs for services that run on the Controller node.
+
+The "Bootstrap logs" for "Other servers" are:
+
+**Bootstrap logs**
+
+:dmesg:  Standard Linux dmesg log that displays log messages
+         from the most recent system startup.
+
+:messages:  Logs all kernel messages for the node.
+
+:mcollective:    Logs activities of Mcollective
+
+:agent:    Logs activities of the
+           :ref:`Nailgun<nailgun-term>` agent.
+
+syslog
+++++++
+
+OpenStack uses the standard Linux **syslog**/**rsyslog** facilities
+to manage logging in the environment.
+Fuel puts the appropriate templates
+into the */etc/rsyslog.d* directory
+on each target node.
+
+By default, Fuel sets up the Fuel Master node
+to be the remote syslog server.
+See :ref:`syslog-ug` for instructions about
+how to configure the environment
+to use a different server as the **rsyslog** server.
+Note that Fuel configures all the files required for **rsyslog**
+when you use the Fuel Master node as the remote server;
+if you specify another server,
+you must configure that server to handle messages
+from the OpenStack environment.
+
+/var/logs
++++++++++
+
+Logs for each node are written to the node's */var/logs* directory
+and can be viewed there.
+Under this directory, you will find subdirectories
+for the major services that run on that node
+such as nova, cinder, glance, and heat.
+
+On the Fuel Master node,
+*/var/log/remote* is a symbolic link to the
+*/var/log/docker-logs/remote* directory.
+

pages/terminology/allterms.rst

@@ -38,6 +38,7 @@ Terminology Reference
 .. include:: /pages/terminology/i/iser.rst
 .. include:: /pages/terminology/k/keystone.rst
 .. include:: /pages/terminology/k/kvm.rst
+.. include:: /pages/terminology/l/logging.rst
 .. include:: /pages/terminology/l/lxc.rst
 .. include:: /pages/terminology/m/mirantis-openstack.rst
 .. include:: /pages/terminology/m/ml2.rst

pages/terminology/l/logging.rst

@@ -0,0 +1,34 @@
+
+
+.. _logging-term:
+
+Logging
+-------
+
+OpenStack and Fuel use the standard
+Linux **syslog** and **rsyslog** facilities
+for logging events on the Fuel Master Node
+and on the nodes in the OpenStack environment.
+
+See the following:
+
+- :ref:`syslog-ug` tells how to configure your environment
+  to use a remote server as the **rsyslog** server
+  rather than use the Fuel Master Node as the **rsyslog** server
+  for the environment.
+
+- :ref:`debug-level-ug` tells how to configure debug logging
+  for the environment.
+
+- :ref:`logs-top-tshoot` discusses
+
+For more information about OpenStack logs,
+see the community documentation:
+
+- `Manage Logs <http://docs.openstack.org/admin-guide-cloud/content/section_manage-logs.html>`_
+
+- `Logging and Monitoring <http://docs.openstack.org/openstack-ops/content/logging_monitoring.html>`_
+
+- `Compute service logging <http://docs.openstack.org/admin-guide-cloud/content/log-files-for-openstack-compute.html>`_
+
+

pages/user-guide/config-environment/5000-settings.rst

@@ -45,9 +45,11 @@ you can click on the "Load Defaults" button at the bottom of the screen.
 .. include:: /pages/user-guide/config-environment/settings/1500-modify-services.rst
 .. include:: /pages/user-guide/config-environment/settings/2000-vcenter-config.rst
 .. include:: /pages/user-guide/config-environment/settings/2500-common-settings.rst
+.. include:: /pages/user-guide/config-environment/settings/3250-debug-set.rst
 .. include:: /pages/user-guide/config-environment/settings/3300-scheduler.rst
 .. include:: /pages/user-guide/config-environment/settings/2590-public-key.rst
 .. include:: /pages/user-guide/config-environment/settings/3500-vlan-splinters.rst
+.. include:: /pages/user-guide/config-environment/settings/3550-syslog.rst
 .. include:: /pages/user-guide/config-environment/settings/3600-mellanox-neutron.rst
 .. include:: /pages/user-guide/config-environment/settings/4000-settings-storage.rst
 .. include:: /pages/user-guide/config-environment/settings/8000-zabbix-access.rst

pages/user-guide/config-environment/settings/3250-debug-set.rst

@@ -0,0 +1,18 @@
+
+.. _debug-level-ug:
+
+Setting debug level for the environment
++++++++++++++++++++++++++++++++++++++++
+
+Use this field to set DEBUG level logging
+for all services in the environment:
+
+.. image:: /_images/user_screen_shots/debug-logging.png
+   :width: 80%
+
+Debug logging consumes massive amounts of disk space
+as well as memory and CPU resources
+on the Fuel Master node and all OpenStack nodes in the environment.
+It should not normally be run unless you are attempting to diagnose a problem,
+and you may want to offload or delete the logs generated
+when you are finished with them.

pages/user-guide/config-environment/settings/3550-syslog.rst

@@ -0,0 +1,21 @@
+
+.. _syslog-ug:
+
+Configuring syslog
+++++++++++++++++++
+
+Fuel deploys the OpenStack environment to use
+the standard Linux **syslog** facility
+to log the activity of all services.
+By default, **rsyslog** is configured
+to use the Fuel Master node as the remote syslog server
+that contains all logs generated on all nodes in the environment.
+
+If you prefer to use an external server for rsyslog,
+specify the IP address and port number in this field:
+
+.. image:: /_images/user_screen_shots/syslog.png
+   :width: 80%
+
+See :ref:`logs-top-tshoot` for more details.
+