Merge "[Ops Guide] Rework logrotate configuration"

pages/operations/troubleshoot/1000-logs.rst

@@ -265,21 +265,62 @@ the value of `X` should be the same in both cases:
 - Modify the value of the **mtime +X** argument
   to the **lastaction** setting.
 
-Fuel Master node log rotation template
-++++++++++++++++++++++++++++++++++++++
+Fuel Master node log rotation
++++++++++++++++++++++++++++++
 
-The Fuel Master node configures two log rotation templates:
+The Fuel Master node provides an ability to configure log rotation
+using the template file.
 
-* The /etc/logrotate.d/10-fuel-docker.conf file is
-  called by the /etc/cron.daily/logrotate script. It provides daily
-  rotation of logs.
+* By default, the logrotate script calls `/etc/logrotate.d/fuel.nodaily`
+  file every 15 minutes to verify whether one of the following conditions is met:
+
+  * age - if the log has not been rotated in more than the specified period of
+    time (weekly by default), and the file is larger than 10MB on the Fuel
+    Master node or 5MB on slave nodes;
+
+  * size - if a log file exceeds 100MB on the Fuel Master node or 20MB on slave nodes.
+
+  .. warning::
+    Fuel enforces the following global changes to the `logrotate` configuration:
+    ``delaycompress`` and ``copytruncate``.
+
+* You can run a quick test to check if the logrotate script works.
+
+  Find the biggest file in ``/var/log/``, and check date time stamps in the first
+  and the last line:
+
+  .. code-block :: sh
+
+     biggest_file=$(find /var/log/ -type f | xargs du -h | sort -h | tail -n 1 | cut -f2);
+     ls -lah $biggest_file;
+     head -n1 $biggest_file | head -c35; echo "";
+     tail -n1 $biggest_file | head -c35; echo "";
+
+  If it is older than your rotation schedule and bigger than maxsize,
+  then logrotate is not working correctly.
+
+  To debug, type:
+
+  .. code-block :: sh
+
+     logrotate -v -d /etc/logrotate.d/fuel.nodaily
+
+  The output of this command is a list of files examined by `logrotate`,
+  including whether they should be rotated or not.
+
+* You can find an example of a writing rate evaluation for the ``neutron-server`` log file:
+  `LP1382515 <https://bugs.launchpad.net/fuel/+bug/1382515/comments/35>`_.
+
+* When backporting the reworked logrotate configuration
+  to the older Fuel releases, purge old template files:
+
+  .. code-block :: sh
+
+     rm /etc/logrotate.d/{1,2}0-fuel*
+
+  The `/usr/bin/fuel-logrotate` script is needed as well as a new cron job to perform the
+  rotation with it.
 
-* The /etc/logrotate.d/20-fuel-docker.conf file is
-  called by the /etc/cron.hourly/logrotate script. It provides
-  an additional layer of rotation, based on the maximum log
-  file size criteria. This ensures that the free space
-  of /var/log is never exhausted with an appropriate consideration
-  for free space planning that matches the given log rotation settings.
 
 Enabling debug logging for OpenStack services
 +++++++++++++++++++++++++++++++++++++++++++++