Merge "Create a separate doc for audit watcher"
This commit is contained in:
commit
b79ca57fa9
|
@ -0,0 +1,12 @@
|
||||||
|
.. _common_audit_watchers:
|
||||||
|
|
||||||
|
*********************
|
||||||
|
Object Audit Watchers
|
||||||
|
*********************
|
||||||
|
|
||||||
|
.. _dark_data:
|
||||||
|
|
||||||
|
Dark Data
|
||||||
|
=========
|
||||||
|
|
||||||
|
.. automodule:: swift.obj.watchers.dark_data
|
|
@ -104,23 +104,9 @@ Dark Data Watcher
|
||||||
|
|
||||||
The watcher API is assumed to be under development. Operators who
|
The watcher API is assumed to be under development. Operators who
|
||||||
need extensions are welcome to report any needs for more arguments
|
need extensions are welcome to report any needs for more arguments
|
||||||
to see_object(). For now, start by copying the provided template watcher
|
to see_object().
|
||||||
swift/obj/watchers/dark_data.py and see if it is sufficient.
|
|
||||||
|
|
||||||
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter,
|
The :ref:`dark_data` watcher has been provided as an example. If an
|
||||||
which supposes that the universe contains a lot of matter than we cannot
|
operator wants to create their own watcher, start by copying
|
||||||
observe. The Dark Data in Swift is the name of objects that are not
|
the provided example template ``swift/obj/watchers/dark_data.py`` and see
|
||||||
accounted in the containers.
|
if it is sufficient.
|
||||||
|
|
||||||
The experience of running large scale clusters suggests that Swift does
|
|
||||||
not have any particular bugs that trigger creation of dark data. So,
|
|
||||||
this is an excercise in writing watchers, with a plausible function.
|
|
||||||
|
|
||||||
When enabled, Dark Data watcher definitely drags down the cluster's overall
|
|
||||||
performance, as mentioned above. Of course, the load increase can be
|
|
||||||
mitigated as usual, but at the expense of the total time taken by
|
|
||||||
the pass of auditor.
|
|
||||||
|
|
||||||
Finally, keep in mind that Dark Data watcher needs the container
|
|
||||||
ring to operate, but runs on an object node. This can come up if
|
|
||||||
cluster has nodes separated by function.
|
|
||||||
|
|
|
@ -166,6 +166,7 @@ Source Documentation
|
||||||
object
|
object
|
||||||
misc
|
misc
|
||||||
middleware
|
middleware
|
||||||
|
audit_watchers
|
||||||
|
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
|
|
|
@ -23,6 +23,26 @@
|
||||||
# Which is for the best, because of a large performance impact of this.
|
# Which is for the best, because of a large performance impact of this.
|
||||||
#
|
#
|
||||||
|
|
||||||
|
"""
|
||||||
|
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter,
|
||||||
|
which supposes that the universe contains a lot of matter than we cannot
|
||||||
|
observe. The Dark Data in Swift is the name of objects that are not
|
||||||
|
accounted in the containers.
|
||||||
|
|
||||||
|
The experience of running large scale clusters suggests that Swift does
|
||||||
|
not have any particular bugs that trigger creation of dark data. So,
|
||||||
|
this is an excercise in writing watchers, with a plausible function.
|
||||||
|
|
||||||
|
When enabled, Dark Data watcher definitely drags down the cluster's overall
|
||||||
|
performance. Of course, the load increase can be
|
||||||
|
mitigated as usual, but at the expense of the total time taken by
|
||||||
|
the pass of auditor.
|
||||||
|
|
||||||
|
Finally, keep in mind that Dark Data watcher needs the container
|
||||||
|
ring to operate, but runs on an object node. This can come up if
|
||||||
|
cluster has nodes separated by function.
|
||||||
|
"""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import random
|
import random
|
||||||
import shutil
|
import shutil
|
||||||
|
|
Loading…
Reference in New Issue