Create a separate doc for audit watcher

We have a separate doc page for middlewares that pulls the docstring
from each middleware's docstring[0]. This makes it easy to look up the docs
in our documentation and easy to find the middleware doc by looking in
the code of the middleware itself.

This patch does the same with the audit watchers. There is now a page
that generates a list of audit watchers, even though currently it's only
one, and pulls the docs from their docstrings. Giving us an easy way to
maintain each audit watcher doc along with it's code.

[0] - https://docs.openstack.org/swift/latest/middleware.html

Change-Id: I1456aba0158d29fa0a879dcc2dfb13245c45ad16
This commit is contained in:
Matthew Oliver 2021-01-06 16:17:30 +11:00 committed by Pete Zaitcev
parent 587a4bef33
commit 46a6a09d5b
4 changed files with 38 additions and 19 deletions

View File

@ -0,0 +1,12 @@
.. _common_audit_watchers:
*********************
Object Audit Watchers
*********************
.. _dark_data:
Dark Data
=========
.. automodule:: swift.obj.watchers.dark_data

View File

@ -104,23 +104,9 @@ Dark Data Watcher
The watcher API is assumed to be under development. Operators who The watcher API is assumed to be under development. Operators who
need extensions are welcome to report any needs for more arguments need extensions are welcome to report any needs for more arguments
to see_object(). For now, start by copying the provided template watcher to see_object().
swift/obj/watchers/dark_data.py and see if it is sufficient.
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter, The :ref:`dark_data` watcher has been provided as an example. If an
which supposes that the universe contains a lot of matter than we cannot operator wants to create their own watcher, start by copying
observe. The Dark Data in Swift is the name of objects that are not the provided example template ``swift/obj/watchers/dark_data.py`` and see
accounted in the containers. if it is sufficient.
The experience of running large scale clusters suggests that Swift does
not have any particular bugs that trigger creation of dark data. So,
this is an excercise in writing watchers, with a plausible function.
When enabled, Dark Data watcher definitely drags down the cluster's overall
performance, as mentioned above. Of course, the load increase can be
mitigated as usual, but at the expense of the total time taken by
the pass of auditor.
Finally, keep in mind that Dark Data watcher needs the container
ring to operate, but runs on an object node. This can come up if
cluster has nodes separated by function.

View File

@ -166,6 +166,7 @@ Source Documentation
object object
misc misc
middleware middleware
audit_watchers
Indices and tables Indices and tables

View File

@ -23,6 +23,26 @@
# Which is for the best, because of a large performance impact of this. # Which is for the best, because of a large performance impact of this.
# #
"""
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter,
which supposes that the universe contains a lot of matter than we cannot
observe. The Dark Data in Swift is the name of objects that are not
accounted in the containers.
The experience of running large scale clusters suggests that Swift does
not have any particular bugs that trigger creation of dark data. So,
this is an excercise in writing watchers, with a plausible function.
When enabled, Dark Data watcher definitely drags down the cluster's overall
performance. Of course, the load increase can be
mitigated as usual, but at the expense of the total time taken by
the pass of auditor.
Finally, keep in mind that Dark Data watcher needs the container
ring to operate, but runs on an object node. This can come up if
cluster has nodes separated by function.
"""
import os import os
import random import random
import shutil import shutil