Merge "Create a separate doc for audit watcher"

This commit is contained in:
Zuul 2021-04-02 00:27:31 +00:00 committed by Gerrit Code Review
commit b79ca57fa9
4 changed files with 38 additions and 19 deletions

View File

@ -0,0 +1,12 @@
.. _common_audit_watchers:
*********************
Object Audit Watchers
*********************
.. _dark_data:
Dark Data
=========
.. automodule:: swift.obj.watchers.dark_data

View File

@ -104,23 +104,9 @@ Dark Data Watcher
The watcher API is assumed to be under development. Operators who The watcher API is assumed to be under development. Operators who
need extensions are welcome to report any needs for more arguments need extensions are welcome to report any needs for more arguments
to see_object(). For now, start by copying the provided template watcher to see_object().
swift/obj/watchers/dark_data.py and see if it is sufficient.
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter, The :ref:`dark_data` watcher has been provided as an example. If an
which supposes that the universe contains a lot of matter than we cannot operator wants to create their own watcher, start by copying
observe. The Dark Data in Swift is the name of objects that are not the provided example template ``swift/obj/watchers/dark_data.py`` and see
accounted in the containers. if it is sufficient.
The experience of running large scale clusters suggests that Swift does
not have any particular bugs that trigger creation of dark data. So,
this is an excercise in writing watchers, with a plausible function.
When enabled, Dark Data watcher definitely drags down the cluster's overall
performance, as mentioned above. Of course, the load increase can be
mitigated as usual, but at the expense of the total time taken by
the pass of auditor.
Finally, keep in mind that Dark Data watcher needs the container
ring to operate, but runs on an object node. This can come up if
cluster has nodes separated by function.

View File

@ -166,6 +166,7 @@ Source Documentation
object object
misc misc
middleware middleware
audit_watchers
Indices and tables Indices and tables

View File

@ -23,6 +23,26 @@
# Which is for the best, because of a large performance impact of this. # Which is for the best, because of a large performance impact of this.
# #
"""
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter,
which supposes that the universe contains a lot of matter than we cannot
observe. The Dark Data in Swift is the name of objects that are not
accounted in the containers.
The experience of running large scale clusters suggests that Swift does
not have any particular bugs that trigger creation of dark data. So,
this is an excercise in writing watchers, with a plausible function.
When enabled, Dark Data watcher definitely drags down the cluster's overall
performance. Of course, the load increase can be
mitigated as usual, but at the expense of the total time taken by
the pass of auditor.
Finally, keep in mind that Dark Data watcher needs the container
ring to operate, but runs on an object node. This can come up if
cluster has nodes separated by function.
"""
import os import os
import random import random
import shutil import shutil