Browse Source

Configurable introspection data backends

Currently introspection data can only be stored to Swift, this spec
proposes a mechanism to read/write introspection data from the storage
backend according to configuration option, and provides support to
access introspection data in the ironic inspector database.

Change-Id: I24b2f11be2e8e538a88e41532eacf0fc25836680
Story: 1726713
Task: 11373
master
Kaifeng Wang 9 months ago
parent
commit
524db1e980
1 changed files with 168 additions and 0 deletions
  1. 168
    0
      specs/configurable-introspection-data-backends.rst

+ 168
- 0
specs/configurable-introspection-data-backends.rst View File

@@ -0,0 +1,168 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+===============================================
8
+Configurable introspection data storage backend
9
+===============================================
10
+
11
+https://storyboard.openstack.org/#!/story/1726713
12
+
13
+This spec proposes a mechanism to read and write introspection data from
14
+configured storage backend, and additionally provides support to read and
15
+write introspection data in ironic inspector database.
16
+
17
+Problem description
18
+===================
19
+
20
+Currently, introspection data can only be stored in Swift, there are no
21
+alternatives in an environment where Swift is not adopted. Also there is lack
22
+of mechanisms to support other storage backends by extending ironic-inspector
23
+with extensions.
24
+
25
+Proposed change
26
+===============
27
+
28
+* Adds a table named ``introspection_data`` to store introspection data.
29
+
30
+* Adds an inspector plugin named ``introspection_data`` which supports three
31
+  types of storage backend: ``none``, ``swift``, and ``database``.
32
+
33
+  Each type of storage backend exposes two interfaces::
34
+
35
+    def get(self, node_id, suffix=None):
36
+        pass
37
+
38
+    def save(self, node_info, data, suffix=None):
39
+        pass
40
+
41
+* Adds a plugin manager to dynamically load driver extensions according to
42
+  the configuration option ``[processing]storage_backend``, in a way similar
43
+  to rule manager.
44
+
45
+* Adds a new entry point to the setup.cfg::
46
+
47
+    ironic_inspector.introspection.storage_backend =
48
+        none = ironic_inspector.plugins.introspection_data:NoStore
49
+        swift = ironic_inspector.plugins.introspection_data:SwiftStore
50
+        database = ironic_inspector.plugins.introspection_data:DatabaseStore
51
+
52
+Alternatives
53
+------------
54
+
55
+None
56
+
57
+Data model impact
58
+-----------------
59
+
60
+A new table named ``introspection_data`` will be created, with three fields:
61
+
62
+* ``uuid``: String(36), foreign key to ``node.uuid``.
63
+* ``processed``: Boolean, used to determine whether the introspection data is
64
+  processed or not. Currently, inspector uses suffix ``UNPROCESSED`` for
65
+  unprocessed data, ``None`` for processed data, This will be mapped to the
66
+  boolean field.
67
+
68
+  .. note::
69
+     The Swift storage backend uses suffix as part of the object name:
70
+     ``inspector_data-<uuid>[-suffix]``.
71
+
72
+* ``data``: JsonEncodedDict(), LONGTEXT for MySQL. This field is used as
73
+  the storage of introspected data.
74
+
75
+When a node is removed from the cache, the associated introspection data will
76
+be removed as well.
77
+
78
+HTTP API impact
79
+---------------
80
+
81
+None
82
+
83
+Client (CLI) impact
84
+-------------------
85
+
86
+None
87
+
88
+Ironic python agent impact
89
+--------------------------
90
+
91
+None
92
+
93
+Performance and scalability impact
94
+----------------------------------
95
+
96
+None
97
+
98
+Security impact
99
+---------------
100
+
101
+None
102
+
103
+Deployer impact
104
+---------------
105
+
106
+Configuration option ``[processing]storage_backend`` will have ``database`` as
107
+a valid value. When set, the inspected data will be stored to ironic inspector
108
+database.
109
+
110
+Swift is not a mandatory requirement for storing introspection data in this
111
+case.
112
+
113
+Developer impact
114
+----------------
115
+
116
+After this feature is implemented, additional plugins can be implemented to
117
+support other type of stores.
118
+
119
+Upgrades and Backwards Compatibility
120
+------------------------------------
121
+
122
+Database upgrade will be handled by ironic-inspector-dbsync.
123
+
124
+Provides a tool to assist with the migration of existing introspection data
125
+between Swift and database, e.g.:
126
+
127
+.. code-block:: console
128
+
129
+   $ ironic-inspector-migrate-data --from swift --to database
130
+   $ ironic-inspector-migrate-data --from database --to swift
131
+
132
+Implementation
133
+==============
134
+
135
+Assignee(s)
136
+-----------
137
+
138
+Primary assignee:
139
+  kaifeng
140
+
141
+
142
+Work Items
143
+----------
144
+
145
+* Implements db layer support.
146
+* Implements inspection data plugin, migrates swift support into plugin as
147
+  a storage backend.
148
+* Implements database storage backend.
149
+* Creates introspection data plugin manager to load driver instance according
150
+  to configuration option, rework introspection data read/write access based
151
+  on interfaces provided by introspection data plugin.
152
+* Implements the tool to help with introspection data migration.
153
+
154
+Dependencies
155
+============
156
+
157
+None
158
+
159
+Testing
160
+=======
161
+
162
+This will be covered by unit tests and functional tests.
163
+
164
+
165
+References
166
+==========
167
+
168
+None

Loading…
Cancel
Save