Browse Source

Merge Monasca APIs into a single unified API

This commit serves as a straw man for merging the Monasca APIs into a
single unified API.

APIImpact
Story: 2003881
Task: 26742
Change-Id: Id08a46bb5b54db5baa5c3595100b73bab85ea8ff
Doug Szumski 7 months ago
parent
commit
b285ca6486
1 changed files with 251 additions and 0 deletions
  1. 251
    0
      specs/stein/approved/merge_apis.rst

+ 251
- 0
specs/stein/approved/merge_apis.rst View File

@@ -0,0 +1,251 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+==================
8
+Merge Monasca APIs
9
+==================
10
+
11
+https://storyboard.openstack.org/#!/story/2003881
12
+
13
+Monasca currently has 3 APIs; one for metrics, one for logs and one for
14
+events. Historically, this has created additional overhead for developers
15
+upgrading or adding new features to the APIs, for system administrators
16
+configuring the APIs, and for packagers generating Monasca distributions.
17
+The purpose of this spec is to consider whether the Monasca APIs should
18
+be merged into a single, unified API.
19
+
20
+Problem description
21
+===================
22
+
23
+The Monasca APIs are written in Python and share a lot of functionality,
24
+some, but not all of which has been factored out to the Monasca common
25
+library. Here lies the first problem:
26
+
27
+1. There exists significant technical debt.
28
+
29
+There is a significant amount of common functionality that has not been
30
+factored out to Monasca common. For example, the Monasca API repo contains
31
+code for validating metrics and so does the Monasca common library. The
32
+same is true for query authorisation, where validation code lives in
33
+both repos. This technical debt could of course be addressed, but how
34
+did it come about?
35
+
36
+2. Updating the APIs to support common features incurs significant
37
+   developer and reviewer overhead compared to updating a single API.
38
+
39
+The APIs are all written slightly differently. Adding support for
40
+Oslo Policy or uWSGI to one API is not the same as adding it to another
41
+API. Furthermore, changes frequently have to be made across multiple
42
+repos. Whilst Zuul is well suited to coordinating this task, it requires
43
+careful synchronisation of versions by developers and meticulous
44
+attention from reviewers. If one modifies code which is common to all
45
+three APIs, they must systematically verify that in each case the changes
46
+work as intended. Of course, automated testing can provide relief here, but
47
+it doesn't make the burden disappear entirely.
48
+
49
+3. Historically it has been difficult to maintain a standard experience
50
+   across multiple APIs.
51
+
52
+At times, APIs have diverged when in an ideal world they should not have.
53
+For example, when adding support for a common feature, work has traditionally
54
+been focused on one API to keep the task simple, with the view to adding
55
+support for the other APIs at a later date. If works stops, for example due
56
+to a release, or due to a developer moving on, the APIs can be left in a
57
+diverged state for a significant amount of time. Of course, one solution
58
+is to block merging of a common feature until the work is complete
59
+in all three APIs, and all common code is added to the Monasca
60
+common library. In practice, this has been prevented by the number of man
61
+hours available. For example, a contributor may only be interested in
62
+metrics. They may not be able to justify spending time working on the other
63
+APIs.
64
+
65
+4. Packaging, deploying, and configuring 3 APIs is more complex than
66
+   deploying a single, unified API.
67
+
68
+This is an obvious point, but it can be made worse by 3). For example,
69
+historically, it has not always been possible to run all APIs in the same
70
+way. Configuration of common functionality such as Kafka has not
71
+always been uniform. There is extra build system configuration,
72
+additional keystone endpoints that need to be registered and monitoring
73
+the availability of 3 APIs is more difficult than one.
74
+
75
+Use Cases
76
+---------
77
+
78
+As a developer I would save time by implementing a feature common
79
+to all APIs in a single repo, rather than across four repos.
80
+
81
+As a reviewer I will save time by not having to think about how
82
+a change may impact multiple repos.
83
+
84
+As a deployer I will save time by having two fewer services to deploy
85
+and configure.
86
+
87
+As a security analyst I can focus my efforts reviewing a single API,
88
+rather than three.
89
+
90
+As a user of Monasca I would like a consistent experience, including usability
91
+and documentation.
92
+
93
+As a packager I would like to have a single Monasca API, rather than
94
+three to save time configuring and maintaining my build system.
95
+
96
+Proposed change
97
+===============
98
+
99
+Merge all APIs into a single unified API. Merge all common API code from the
100
+Monasca common repo into the unified API repository. Specifically, it is
101
+proposed to merge all relevant code into the Monasca API.
102
+
103
+Alternatives
104
+------------
105
+
106
+1. Refactor all APIs so that the code is standard. Prevent merging of common
107
+   features until the work is complete across all APIs, and all common code
108
+   has been factored out into the Monasca common library. From historical
109
+   experience this will be difficult without additional developers.
110
+
111
+2. Don't do anything and carry on working around the technical debt. In the
112
+   long term this is likely to make it more difficult to add new features, and
113
+   require more time for maintenance.
114
+
115
+Data model impact
116
+-----------------
117
+
118
+None
119
+
120
+REST API impact
121
+---------------
122
+
123
+Aside from the fact that a single service will implement the combined schema
124
+from all APIs, the calls should not change. We should be careful when merging,
125
+for example dimension validation code, that we do not break things which were
126
+accepted in one API, but not another. An ideal result would be that we use
127
+the same code for parsing common fields such as dimensions for all API calls.
128
+
129
+Security impact
130
+---------------
131
+
132
+If a single API was previously exposed publicly, deploying the unified API
133
+will increase the surface area for attack. However, in general, it will be
134
+easier to review changes to make security improvements due to the reduced
135
+developer and reviewer overhead.
136
+
137
+During the deprecation period of the Events and Log API, security fixes made
138
+to the unified API may need to be backported.
139
+
140
+Other end user impact
141
+---------------------
142
+
143
+All services talking to the Monasca Events and Log APIs will need
144
+to be directed at the unified API. For services which use Keystone
145
+to lookup the Monasca endpoints this should occur automatically. Other
146
+services such as the Fluentd Monasca output plugin will need to be
147
+reconfigured manually. A grace period where the Monasca Events and Log
148
+APIs are still supported is one possibility to make this transition
149
+easier.
150
+
151
+Performance Impact
152
+------------------
153
+
154
+No direct impact.
155
+
156
+In general, it will be less effort to profile and optimise a single API than
157
+it would be to do the same for all three APIs.
158
+
159
+In clustered deployments with specialised nodes (for example a dedicated
160
+logging node, hosting the Monasca Log API) the unified API can be deployed
161
+as a direct replacement, and the additional functionality can simply be
162
+ignored.
163
+
164
+Other deployer impact
165
+---------------------
166
+
167
+For Monasca users supporting legacy releases, any security or bug fixes
168
+made to the unified API may need to be backported to the individual APIs.
169
+
170
+All actively maintained deployment solutions will need to be updated to
171
+deploy the unified API. For example, Monasca-Docker, DevStack, Kolla,
172
+OpenStack Ansible, and Helm.
173
+
174
+In the case of DevStack we should merge the three existing plugins into
175
+one. The resulting plugin should have options like `log_pipeline_enabled`
176
+and `metrics_pipeline_enabled` to support enabling those pipelines
177
+separately. This is useful, for example, when DevStack is used in OpenStack
178
+CI to allow testing changes localised to specific areas more efficiently.
179
+
180
+Developer impact
181
+----------------
182
+
183
+The motiviation behind this change is to reduce the burden placed on
184
+developers and reviewers when making improvements to the Monasca APIs. It
185
+is hoped that this will lead to an increase in developer productivity.
186
+
187
+Implementation
188
+==============
189
+
190
+Assignee(s)
191
+-----------
192
+
193
+Primary assignee:
194
+  <launchpad-id or None>
195
+
196
+Other contributors:
197
+  <launchpad-id or None>
198
+
199
+Work Items
200
+----------
201
+
202
+* Review test coverage of APIs, and add coverage for any missing areas.
203
+* Review test coverage of Monasca common and add coverage for any missing areas.
204
+* Merge Monasca common Python code into the Monasca API. Common functionality
205
+  should include:
206
+
207
+  * authorisation
208
+  * validation
209
+  * OpenStack policy
210
+  * WSGI deployment
211
+
212
+* Implement Log API schema in the Monasca API and port tests.
213
+* Implement Events API schema and port tests.
214
+* Merge DevStack plugins into a single plugin and add support for enabling
215
+  pipelines individually.
216
+* Deprecate Monasca Log and Event APIs.
217
+* Merge and update documentation
218
+
219
+Dependencies
220
+============
221
+
222
+No additional dependencies are added. The dependency on Monasca common can
223
+be removed.
224
+
225
+Testing
226
+=======
227
+
228
+The Monasca API and Log API Tempest test plugins have already been merged into
229
+one plugin. Any Tempest tests which exist for the Events API should also be
230
+merged into the unified plugin. The Tempest plugin will need to be updated to
231
+use the unified API.
232
+
233
+Unit tests from the Log API and Events API repo will need to be ported to the
234
+Monasca API (unified) repo. Some of these tests may be redundant.
235
+
236
+Documentation Impact
237
+====================
238
+
239
+Three sets of documentation will be reduced to one. Whilst it will take
240
+some effort to merge the documentation, it should hopefully be more
241
+consistent.
242
+
243
+References
244
+==========
245
+
246
+None
247
+
248
+History
249
+=======
250
+
251
+None

Loading…
Cancel
Save