Browse Source

Merge "Create a separate doc for audit watcher"

changes/33/769633/11
Zuul 3 weeks ago
committed by Gerrit Code Review
parent
commit
b79ca57fa9
4 changed files with 39 additions and 20 deletions
  1. +12
    -0
      doc/source/audit_watchers.rst
  2. +6
    -20
      doc/source/development_watchers.rst
  3. +1
    -0
      doc/source/index.rst
  4. +20
    -0
      swift/obj/watchers/dark_data.py

+ 12
- 0
doc/source/audit_watchers.rst View File

@ -0,0 +1,12 @@
.. _common_audit_watchers:
*********************
Object Audit Watchers
*********************
.. _dark_data:
Dark Data
=========
.. automodule:: swift.obj.watchers.dark_data

+ 6
- 20
doc/source/development_watchers.rst View File

@ -104,23 +104,9 @@ Dark Data Watcher
The watcher API is assumed to be under development. Operators who
need extensions are welcome to report any needs for more arguments
to see_object(). For now, start by copying the provided template watcher
swift/obj/watchers/dark_data.py and see if it is sufficient.
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter,
which supposes that the universe contains a lot of matter than we cannot
observe. The Dark Data in Swift is the name of objects that are not
accounted in the containers.
The experience of running large scale clusters suggests that Swift does
not have any particular bugs that trigger creation of dark data. So,
this is an excercise in writing watchers, with a plausible function.
When enabled, Dark Data watcher definitely drags down the cluster's overall
performance, as mentioned above. Of course, the load increase can be
mitigated as usual, but at the expense of the total time taken by
the pass of auditor.
Finally, keep in mind that Dark Data watcher needs the container
ring to operate, but runs on an object node. This can come up if
cluster has nodes separated by function.
to see_object().
The :ref:`dark_data` watcher has been provided as an example. If an
operator wants to create their own watcher, start by copying
the provided example template ``swift/obj/watchers/dark_data.py`` and see
if it is sufficient.

+ 1
- 0
doc/source/index.rst View File

@ -166,6 +166,7 @@ Source Documentation
object
misc
middleware
audit_watchers
Indices and tables


+ 20
- 0
swift/obj/watchers/dark_data.py View File

@ -23,6 +23,26 @@
# Which is for the best, because of a large performance impact of this.
#
"""
The name of "Dark Data" refers to the scientific hypothesis of Dark Matter,
which supposes that the universe contains a lot of matter than we cannot
observe. The Dark Data in Swift is the name of objects that are not
accounted in the containers.
The experience of running large scale clusters suggests that Swift does
not have any particular bugs that trigger creation of dark data. So,
this is an excercise in writing watchers, with a plausible function.
When enabled, Dark Data watcher definitely drags down the cluster's overall
performance. Of course, the load increase can be
mitigated as usual, but at the expense of the total time taken by
the pass of auditor.
Finally, keep in mind that Dark Data watcher needs the container
ring to operate, but runs on an object node. This can come up if
cluster has nodes separated by function.
"""
import os
import random
import shutil


Loading…
Cancel
Save