From 46a6a09d5b15dbcd7ce7244a6c18a8edbb96013e Mon Sep 17 00:00:00 2001 From: Matthew Oliver Date: Wed, 6 Jan 2021 16:17:30 +1100 Subject: [PATCH] Create a separate doc for audit watcher We have a separate doc page for middlewares that pulls the docstring from each middleware's docstring[0]. This makes it easy to look up the docs in our documentation and easy to find the middleware doc by looking in the code of the middleware itself. This patch does the same with the audit watchers. There is now a page that generates a list of audit watchers, even though currently it's only one, and pulls the docs from their docstrings. Giving us an easy way to maintain each audit watcher doc along with it's code. [0] - https://docs.openstack.org/swift/latest/middleware.html Change-Id: I1456aba0158d29fa0a879dcc2dfb13245c45ad16 --- doc/source/audit_watchers.rst | 12 ++++++++++++ doc/source/development_watchers.rst | 24 +++++------------------- doc/source/index.rst | 1 + swift/obj/watchers/dark_data.py | 20 ++++++++++++++++++++ 4 files changed, 38 insertions(+), 19 deletions(-) create mode 100644 doc/source/audit_watchers.rst diff --git a/doc/source/audit_watchers.rst b/doc/source/audit_watchers.rst new file mode 100644 index 0000000000..51ca39537e --- /dev/null +++ b/doc/source/audit_watchers.rst @@ -0,0 +1,12 @@ +.. _common_audit_watchers: + +********************* +Object Audit Watchers +********************* + +.. _dark_data: + +Dark Data +========= + +.. automodule:: swift.obj.watchers.dark_data diff --git a/doc/source/development_watchers.rst b/doc/source/development_watchers.rst index 48e9310b17..6aee254394 100644 --- a/doc/source/development_watchers.rst +++ b/doc/source/development_watchers.rst @@ -104,23 +104,9 @@ Dark Data Watcher The watcher API is assumed to be under development. Operators who need extensions are welcome to report any needs for more arguments -to see_object(). For now, start by copying the provided template watcher -swift/obj/watchers/dark_data.py and see if it is sufficient. +to see_object(). -The name of "Dark Data" refers to the scientific hypothesis of Dark Matter, -which supposes that the universe contains a lot of matter than we cannot -observe. The Dark Data in Swift is the name of objects that are not -accounted in the containers. - -The experience of running large scale clusters suggests that Swift does -not have any particular bugs that trigger creation of dark data. So, -this is an excercise in writing watchers, with a plausible function. - -When enabled, Dark Data watcher definitely drags down the cluster's overall -performance, as mentioned above. Of course, the load increase can be -mitigated as usual, but at the expense of the total time taken by -the pass of auditor. - -Finally, keep in mind that Dark Data watcher needs the container -ring to operate, but runs on an object node. This can come up if -cluster has nodes separated by function. +The :ref:`dark_data` watcher has been provided as an example. If an +operator wants to create their own watcher, start by copying +the provided example template ``swift/obj/watchers/dark_data.py`` and see +if it is sufficient. diff --git a/doc/source/index.rst b/doc/source/index.rst index f9f23e464f..f1b6bee6f7 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -166,6 +166,7 @@ Source Documentation object misc middleware + audit_watchers Indices and tables diff --git a/swift/obj/watchers/dark_data.py b/swift/obj/watchers/dark_data.py index 524fccc3a6..92172bc884 100644 --- a/swift/obj/watchers/dark_data.py +++ b/swift/obj/watchers/dark_data.py @@ -23,6 +23,26 @@ # Which is for the best, because of a large performance impact of this. # +""" +The name of "Dark Data" refers to the scientific hypothesis of Dark Matter, +which supposes that the universe contains a lot of matter than we cannot +observe. The Dark Data in Swift is the name of objects that are not +accounted in the containers. + +The experience of running large scale clusters suggests that Swift does +not have any particular bugs that trigger creation of dark data. So, +this is an excercise in writing watchers, with a plausible function. + +When enabled, Dark Data watcher definitely drags down the cluster's overall +performance. Of course, the load increase can be +mitigated as usual, but at the expense of the total time taken by +the pass of auditor. + +Finally, keep in mind that Dark Data watcher needs the container +ring to operate, but runs on an object node. This can come up if +cluster has nodes separated by function. +""" + import os import random import shutil