Browse Source

Merge "Monasca Events Publishing Spec"

changes/62/626362/1
Zuul 5 months ago
parent
commit
abb3854784

+ 6
- 3
README.rst View File

@@ -25,11 +25,14 @@ The layout of this repository is::
25 25
   priorities/<release>/
26 26
   specs/<release>/
27 27
 
28
-Where there are two sub-directories in ``specs``:
28
+Where there are three sub-directories in ``specs``:
29 29
 
30 30
 specs/<release>/approved
31
-  specifications approved, but not yet implemented
31
+  Specifications approved, but not yet implemented
32 32
 
33 33
 specs/<release>/implemented
34
-  implemented specifications
34
+  Implemented specifications
35 35
 
36
+specs/<release>/not-implemented
37
+  Specifications that were approved but are not expected to be implemented.
38
+  These are kept for historical reference.

+ 8
- 0
doc/source/specs/rocky/index.rst View File

@@ -24,3 +24,11 @@ Rocky approved (but not implemented) specs:
24 24
    :maxdepth: 1
25 25
 
26 26
    approved/*
27
+
28
+Rocky deprecated specs:
29
+
30
+.. toctree::
31
+   :glob:
32
+   :maxdepth: 1
33
+
34
+   not-implemented/*

+ 1
- 0
doc/source/specs/rocky/not-implemented View File

@@ -0,0 +1 @@
1
+../../../../specs/rocky/not-implemented

+ 705
- 0
specs/rocky/not-implemented/monasca-events-publishing.rst View File

@@ -0,0 +1,705 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+================================================
8
+Monasca Events Publishing - from Ceilometer
9
+================================================
10
+
11
+https://storyboard.openstack.org/#!/story/2003023
12
+
13
+Monasca Events API [1] was developed to store Openstack Notification data in Elasticsearch. There is
14
+still a need to collect and publish Openstack Notifications to Monasca Events API. Monasca
15
+Ceilometer project[2] currently publishes ceilometer samples[3] to Monasca API. We are proposing to
16
+extend Monasca Ceilometer project and add a new events publisher which will publish Openstack
17
+notifications (or events)[3] to Monasca Events API.
18
+
19
+UPDATE: This spec is being superceded by the ../../stein/approved/monasca-events-listener.rst spec,
20
+but is kept here for reference.
21
+
22
+
23
+Problem description
24
+===================
25
+
26
+All Openstack services generate a lot of notifications or events which contain large amounts of
27
+operational and state information about the service and its resources. This notification data is not
28
+currently available in Monasca.
29
+
30
+Ceilometer data processing pipeline[3] provides an extensible mechanism of publishing samples and
31
+events using a custom publisher.  Ceilometer samples represent a quantity that can be measured (for
32
+e.g. the size of a volume) and events represent an occurrence of an event and do not have any
33
+associated quantity (e.g. volume was created).
34
+
35
+Monasca Ceilometer project currently provides a samples publisher. Monasca Ceilometer samples
36
+publisher converts Ceilometer samples to Monasca Metrics format which are then published to Monasca
37
+API. There is no corresponding events publisher to Monasca yet.
38
+
39
+By adding an event publisher to Monasca Ceilometer project we could take advantage of Ceilometer's
40
+event publishing mechanism to publish events to Monasca Events API.
41
+
42
+Ceilometer consists of different data collection components - namely Polling Agent, Notification
43
+Agent and Compute Agent. (Please see [7] for System Architecture diagram) Ceilometer also has a data
44
+storage and retrieval component, which would be Monasca in our case.
45
+
46
+Samples publisher and new proposed events publisher run within Ceilometer's Notification Agent
47
+component and are part of Notifcation Agent's data processing pipeline. Monasca
48
+Ceilometer presumes the need to install and deploy Ceilometer Notification Agent component (doesn't
49
+need Polling Agent or Compute Agent deployed) on all the control nodes. Ceilometer Notification
50
+Agent is Highly Available (HA) and can run on multiple nodes. We will have to evaluate its
51
+performance in terms of scaling for events, but we haven't run into performance/scale problems
52
+with current samples publisher.
53
+
54
+
55
+Use Cases
56
+---------
57
+
58
+#. Openstack notification data would be stored in Elasticsearch
59
+   via the Monasca Events API
60
+
61
+   Example sequence from Nova notification to Monasca API
62
+   #. Nova completes the creation of a VM
63
+   #. Nova generates a Notification message to oslo.messaging
64
+   #. Ceilometer Notification agent receives the Notification message
65
+   #. Ceilometer translates the Notification to a Monasca API format according to the configuration
66
+   #. Ceilometer Event Publisher publishes formatted Notification to Monasca Events API
67
+   #. Monasca Events API receives and validates formatted Notification
68
+   #. Monasca Events stores event Notification in configured Elasticsearch instance
69
+
70
+
71
+Proposed change
72
+===============
73
+
74
+#. Openstack Notifications consist of envelope and payload fields
75
+
76
+   Example Openstack Notification data format:
77
+
78
+   .. code-block:: javascript
79
+
80
+        {
81
+            "_context_auth_token": "42630b3ea13242fcad20e0a92d0207f1",
82
+            "_context_domain": null,
83
+            "_context_instance_lock_checked": false,
84
+            "_context_is_admin": true,
85
+            "_context_project_domain": null,
86
+            "_context_project_id": "a4f77",
87
+            "_context_project_name": "admin",
88
+            "_context_quota_class": null,
89
+            "_context_read_deleted": "no",
90
+            "_context_read_only": false,
91
+            "_context_remote_address": "192.168.245.4",
92
+            "_context_request_id": "req-5948338c-f223-4fd8-9249-8769f7a3e460",
93
+            "_context_resource_uuid": null,
94
+            "_context_roles": [
95
+                "monasca-user",
96
+                "admin",
97
+                "KeystoneAdmin"
98
+            ],
99
+            "_context_service_catalog": [
100
+                {
101
+                    "endpoints": [
102
+                        {
103
+                            "adminURL": "http://192.168.245.8:8776/v2/a4f77",
104
+                            "internalURL": "http://192.168.245.8:8776/v2/a4f77",
105
+                            "publicURL": "http://192.168.245.9:8776/v2/a4f77",
106
+                            "region": "region1"
107
+                        }
108
+                    ],
109
+                    "name": "cinderv2",
110
+                    "type": "volumev2"
111
+                },
112
+                {
113
+                    "endpoints": [
114
+                        {
115
+                            "adminURL": "http://192.168.245.8:8776/v1/a4f77",
116
+                            "internalURL": "http://192.168.245.8:8776/v1/a4f77",
117
+                            "publicURL": "http://192.168.245.9:8776/v1/a4f77",
118
+                            "region": "region1"
119
+                        }
120
+                    ],
121
+                    "name": "cinder",
122
+                    "type": "volume"
123
+                }
124
+            ],
125
+            "_context_show_deleted": false,
126
+            "_context_tenant": "a4f77",
127
+            "_context_timestamp": "2015-09-18T20:54:23.468522",
128
+            "_context_user": "be396488c7034811a200a3cb1d103a28",
129
+            "_context_user_domain": null,
130
+            "_context_user_id": "be396488c7034811a200a3cb1d103a28",
131
+            "_context_user_identity": "be396488c7034811a200a3cb1d103a28 a4f77 - - -",
132
+            "_context_user_name": "admin",
133
+            "_unique_id": "ff9699d587bf4283a3c367ab88be1541",
134
+            "event_type": "compute.instance.create.start",
135
+            "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
136
+            "payload": {
137
+                "access_ip_v4": null,
138
+                "access_ip_v6": null,
139
+                "architecture": null,
140
+                "availability_zone": null,
141
+                "cell_name": "",
142
+                "created_at": "2015-09-18 20:55:25+00:00",
143
+                "deleted_at": "",
144
+                "disk_gb": 1,
145
+                "display_name": "testeee",
146
+                "ephemeral_gb": 0,
147
+                "host": null,
148
+                "hostname": "testeee",
149
+                "image_meta": {
150
+                    "base_image_ref": "df0c8",
151
+                    "container_format": "bare",
152
+                    "disk_format": "qcow2",
153
+                    "min_disk": "1",
154
+                    "min_ram": "0"
155
+                },
156
+                "image_name": "glanceaaa3",
157
+                "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
158
+                "instance_flavor_id": "1",
159
+                "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
160
+                "instance_type": "m1.tiny",
161
+                "instance_type_id": 4,
162
+                "kernel_id": "",
163
+                "launched_at": "",
164
+                "memory_mb": 512,
165
+                "metadata": {},
166
+                "node": null,
167
+                "os_type": null,
168
+                "progress": "",
169
+                "ramdisk_id": "",
170
+                "reservation_id": "r-1ghilddw",
171
+                "root_gb": 1,
172
+                "state": "building",
173
+                "state_description": "",
174
+                "tenant_id": "a4f77",
175
+                "terminated_at": "",
176
+                "user_id": "be396488c7034811a200a3cb1d103a28",
177
+                "vcpus": 1
178
+            },
179
+            "priority": "INFO",
180
+            "publisher_id": "compute.ccp-compute0001-mgmt",
181
+            "timestamp": "2015-09-18 20:55:37.639023"
182
+        }
183
+
184
+#. All the fields with the prefix of '_context" are the envelope fields, the
185
+   other interesting fields are
186
+
187
+   #. 'message_id' - notification identifier
188
+   #. 'payload' - contains most of the relevant and useful information in JSON format
189
+   #. 'priority' - notification priority
190
+   #. 'publisher_id' - notification publisher
191
+   #. 'timestamp' - notification timestamp
192
+
193
+#. Ceilometer event publishing framework converts the Openstack notifications to events format[4].
194
+   Event publishing framework also has the ability to extract only some of the 'payload' data into
195
+   a flat set of key-value pairs called 'traits' and publish the normalized 'event' with 'traits'
196
+   extracted from the payload using a custom publisher.
197
+
198
+   Extraction of certain fields into traits from the payload is
199
+   driven by configuration file, but by default "publisher_id",
200
+   'request_id', 'tenant_id', 'user_id' and 'project_id'
201
+   fields are always extracted and added as 'traits'.
202
+
203
+   The event can also have an optional field called 'raw' which has original
204
+   notification, provided 'store_raw' option is set in ceilometer.conf
205
+
206
+   Questions/TODO:
207
+
208
+   * Q1: Does the store_raw field only apply to events, or to all notifications processed by
209
+     Ceilometer?
210
+   * We will have to find it out if it has any adverse impact on sample publisher. Though in the
211
+     case of samples, monasca sample publisher definitely does not submit raw payload, so it must
212
+     be getting dropped.
213
+
214
+   Example Ceilometer Event data format:
215
+
216
+   .. code-block:: javascript
217
+
218
+      {
219
+        "event_type": "compute.instance.create.start",
220
+        "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
221
+        "generated": "2015-09-18 20:55:37.639023",
222
+        "traits": {
223
+           "publisher_id": "compute.ccp-compute0001-mgmt",
224
+           "request_id": "req-5948338c-f223-4fd8-9249-8769f7a3e460",
225
+           "tenant_id": "a4f77",
226
+           "project_id": "a4f77",
227
+           "user_id": "be396488c7034811a200a3cb1d103a28"
228
+         },
229
+         "raw": {  "_context_auth_token": "42630b3ea13242fcad20e0a92d0207f1",
230
+                   "_context_domain": null,
231
+                   ...
232
+                   ...
233
+                   "event_type": "compute.instance.create.start",
234
+                   "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
235
+                   "payload": {
236
+                       "access_ip_v4": null,
237
+                       "access_ip_v6": null,
238
+                       "architecture": null,
239
+                       "availability_zone": null,
240
+                       "cell_name": "",
241
+                       "created_at": "2015-09-18 20:55:25+00:00",
242
+                       "deleted_at": "",
243
+                       "disk_gb": 1,
244
+                       "display_name": "testeee",
245
+                       "ephemeral_gb": 0,
246
+                       "host": null,
247
+                       "hostname": "testeee",
248
+                       "image_meta": {
249
+                           "base_image_ref": "df0c8",
250
+                           "container_format": "bare",
251
+                           "disk_format": "qcow2",
252
+                           "min_disk": "1",
253
+                           "min_ram": "0"
254
+                       },
255
+                      "image_name": "glanceaaa3",
256
+                      "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
257
+                      "instance_flavor_id": "1",
258
+                      "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
259
+                      "instance_type": "m1.tiny",
260
+                      "instance_type_id": 4,
261
+                      "kernel_id": "",
262
+                      "launched_at": "",
263
+                      "memory_mb": 512,
264
+                      "metadata": {},
265
+                      "node": null,
266
+                      "os_type": null,
267
+                      "progress": "",
268
+                      "ramdisk_id": "",
269
+                      "reservation_id": "r-1ghilddw",
270
+                      "root_gb": 1,
271
+                      "state": "building",
272
+                      "state_description": "",
273
+                      "tenant_id": "a4f77",
274
+                      "terminated_at": "",
275
+                      "user_id": "be396488c7034811a200a3cb1d103a28",
276
+                      "vcpus": 1
277
+                      }
278
+                }
279
+      }
280
+
281
+
282
+#. Key-Value pairs that can be extracted from 'payload' in form of traits
283
+   can be defined in events definitions file.
284
+
285
+   For example the following events definitions yaml specifies that for
286
+   all events which have a prefix of "compute.instance.*" then
287
+   add  "user_id", "instance_id", and "instance_type_id" as traits,
288
+   after extracting values from "payload.user_id", "payload.instance_id",
289
+   and "payload.instance_type_id" respectively.
290
+
291
+   .. code-block:: yaml
292
+
293
+   ---
294
+   - event_type: compute.instance.*
295
+
296
+     traits: &instance_traits
297
+      user_id:
298
+        fields: payload.user_id
299
+      instance_id:
300
+        fields: payload.instance_id
301
+      instance_type_id:
302
+        type: int
303
+        fields: payload.instance_type_id
304
+
305
+   We are for now proposing not to use this feature, of defining traits for each event
306
+   extracting since we have the ability to store entire payload, via
307
+   Monasca Events API.
308
+
309
+   We can certainly look at enabling this feature in the future if we run into trouble storing
310
+   entire JSON "payload" in Elasticsearch. This is certainly a nifty way to trim the amount
311
+   of data that will be stored.
312
+
313
+#. The proposed new Custom Monasca Ceilometer event publisher will run within Ceilometer's
314
+   Notification Agent component. It will leverage Ceilometer's data processing pipeline[3] which
315
+   converts notifications to Ceilometer's event format.  At the end of its processing, Monasca
316
+   Ceilometer event publisher will convert Ceilometer Event data into Monasca Event format[6] and
317
+   publish the monasca event to Monasca Events API.
318
+
319
+#. Monasca Events API allows a field called 'payload' which can be in an arbitrary
320
+   nested JSON format. Monasca-Ceilometer event publisher will extract JSON field called
321
+   'payload' from 'raw' (JSON path notation: 'raw.payload'), publish the payload from the
322
+   original notification to Monasca Events API.
323
+
324
+   Example Monasca Event Format:
325
+
326
+   .. code-block:: javascript
327
+
328
+        events: [
329
+        {
330
+          dimensions": {
331
+                "service": "compute.ccp-compute0001-mgmt",
332
+                "topic": "notification.sample",
333
+                "hostname": "nova-compute:compute
334
+          },
335
+          event: {
336
+
337
+                  "event_type": "compute.instance.create.start",
338
+
339
+                  "payload": {
340
+                       "access_ip_v4": null,
341
+                       "access_ip_v6": null,
342
+                       "architecture": null,
343
+                       "availability_zone": null,
344
+                       "cell_name": "",
345
+                       "created_at": "2015-09-18 20:55:25+00:00",
346
+                       "deleted_at": "",
347
+                       "disk_gb": 1,
348
+                       "display_name": "testeee",
349
+                       "ephemeral_gb": 0,
350
+                       "host": null,
351
+                       "hostname": "testeee",
352
+                       "image_meta": {
353
+                           "base_image_ref": "df0c8",
354
+                           "container_format": "bare",
355
+                           "disk_format": "qcow2",
356
+                           "min_disk": "1",
357
+                           "min_ram": "0"
358
+                       },
359
+                      "image_name": "glanceaaa3",
360
+                      "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
361
+                      "instance_flavor_id": "1",
362
+                      "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
363
+                      "instance_type": "m1.tiny",
364
+                      "instance_type_id": 4,
365
+                      "kernel_id": "",
366
+                      "launched_at": "",
367
+                      "memory_mb": 512,
368
+                      "metadata": {},
369
+                      "node": null,
370
+                      "os_type": null,
371
+                      "progress": "",
372
+                      "ramdisk_id": "",
373
+                      "reservation_id": "r-1ghilddw",
374
+                      "root_gb": 1,
375
+                      "state": "building",
376
+                      "state_description": "",
377
+                      "tenant_id": "a4f77",
378
+                      "terminated_at": "",
379
+                      "user_id": "be396488c7034811a200a3cb1d103a28",
380
+                      "vcpus": 1
381
+                      }
382
+                 },
383
+            publisher_id: "compute.ccp-compute0001-mgmt",
384
+            priority: "INFO"
385
+         }
386
+        ]
387
+
388
+#. If no traits are specified in events pipeline yaml configuration file for an event
389
+   Ceilometer's data processing pipeline will add the following default traits:
390
+
391
+   * service: (All notifications should have this) notification’s publisher
392
+   * tenant_id
393
+   * request_id
394
+   * project_id
395
+   * user_id
396
+
397
+   Note: "service" is not the service that produced the event as in say "compute", "glance",
398
+   "cinder" but rather notification RabbitMQ publisher that produced the event
399
+   e.g. "compute.ccp-compute0001-mgmt" so is not very useful.
400
+
401
+#. Ceilometer event data is converted to Monasca event data format before being published to Monasca
402
+   Event API. Following fields in Monasca Event data are not available in current Ceilometer Event
403
+   data format:
404
+
405
+   * "service"
406
+   * "dimensions.topic"
407
+   * "event.priority"
408
+
409
+   We are proposing removing these fields from Monasca Event format (will be done as a separate
410
+   spec/implementation process) for the following reasons:
411
+
412
+   "service": Currently Openstack notifications do not specify a service, that
413
+   generated the notification in a consistent way. It might be possible to create an external
414
+   mapping file which maps event name to a service but its hard to maintain such mapping over a
415
+   period of time.
416
+
417
+   "dimensions.topic": This field is not available in the source Openstack notification
418
+
419
+   "event.priority": This field is not currently available in Ceilometer Event format. It is
420
+   available in the source Openstack notification. Note: If we think this field can be useful we can
421
+   propose adding it to the Ceilometer Event format.
422
+
423
+#. Following new fields will be added to Monasca Event data as dimensions:
424
+
425
+   * "dimensions.publisher_id": Identifier for the publisher that generated the event. Maps to
426
+     "traits.publisher_id" in Ceilometer event data.
427
+   * "dimensions.user_id": Identifier for user that generated the event. Maps to "traits.user_id" in
428
+     Ceilometer event data.
429
+   * "dimensions.project_id": Identifier of the project that generated the event. Maps to
430
+     "traits.project_id" or "traits.tenant_id" in Ceilometer event data.
431
+
432
+#. hostname is available in the event payload, but its location might differ from event to event. We
433
+   can use Ceilometer's event definitions config to always add a trait called "hostname" to all
434
+   events. e.g. for compute.instance.* will have a trait called "hostname", which grabs data from
435
+   "payload.hostname"
436
+
437
+   .. code-block:: yaml
438
+
439
+   ---
440
+   - event_type: compute.instance.*
441
+
442
+     traits: &instance_traits
443
+      user_id:
444
+        fields: payload.hostname
445
+
446
+#. The proposed new Monasca Ceilometer event publisher will have the ability to submit event
447
+   data in a batch and at a configurable frequency (similar to current samples publisher). The
448
+   event data will be published if the items in the current batch reach their maximum size
449
+   (config setting) or if certain time interval has elapsed since the last publish
450
+   (config setting). This will make sure that the batch does not get huge at the same time
451
+   there is no significant delay in publishing of the events to Monasca Events API.
452
+
453
+#. Monasca Ceilometer event publisher will use service credentials from ceilometer configuration
454
+   file (in "[monasca]" section) to get keystone token.
455
+
456
+   Example "[monasca]" section in ceilometer config file
457
+   .. code-block:: text
458
+
459
+   [monasca]
460
+   service_auth_url = https://localhost:5000/v3
461
+   service_password = secretpassword
462
+   service_username = ceilometer
463
+   service_interface = internalURL
464
+   service_auth_type = password
465
+   # service_project_id may also be used
466
+   service_project_name = admin
467
+   service_domain_name = Default
468
+   service_region_name = RegionOne
469
+
470
+   The publisher will then make a POST request to Monasca Events /v1.0/events REST api[8] to publish
471
+   events to  Monasca Events API.  The URL for the instance of Monasca Events API will be configured
472
+   in the Ceilometer 'events-pipeline.yaml' file.  This has the added advantage of allowing
473
+   different events to be published differently (see Ceilometer pipeline documentation [10]).
474
+
475
+#. "tenant_id" and "user_id" that the notification relates to are available in "payload" section
476
+   of the notification, and these notifications are generated by each service itself.
477
+
478
+   There is no additional "Openstack-operator-agent" like component or functionality required to
479
+   fetch that data from the service and publish to monasca event api on behalf of the original
480
+   tenant.
481
+   Ceilometer publishing pipeline simply extracts these "tenant_id" and "user_id" fields from the
482
+   "payload" and makes those fields available as "tenant_id" and "user_id" traits, which would then
483
+   be mapped to "dimensions.project_id" and "dimensions.user_id" fields in monasca events format.
484
+
485
+   In other words, original "tenant_id" and "user_id" values are available in
486
+   the payload of the notification, and will make its way to "dimensions.tenant_id"
487
+   and "dimensions.user_id" in Monasca Event.
488
+
489
+   Questions/TODO:
490
+   * Q: Do we need to do anything special to handle multi-tenancy in monasca-events api like being
491
+   done for metrics[9] ? Would original user_id and tenant_id in "dimensions.user_id" and
492
+   "dimensions.tenant_id" fields in dimensions serve this purpose?
493
+   * Q: In Ceilometer V2 API (which has been deprecated and removed), when querying data the role
494
+   "admin" could access data for all tenants, whereas a user with "ceilometer-admin" role could
495
+   access only data for a particular tenant. Can we implement something like this for
496
+   monasca-events api when querying for data?
497
+
498
+#. Monasca Ceilometer event publisher will also retry submitting a batch, in case Monasca
499
+   Events API is temporarily unavailable or down. The retry frequency, the number of retries
500
+   and the number of items that can be in the retry batch will also be set via configuration.
501
+
502
+
503
+Alternative Solutions
504
+---------------------
505
+
506
+#. Standalone monasca event agent which reads Openstack notifications published to RabbitMQ
507
+   (on "notification" topic) and publishes them to Monasca Events API.
508
+   Pro:
509
+   * No dependency on Telemetry project.
510
+   * May be simple to develop if leverage the oslo.messaging functionality.
511
+   * Ceilometer has *deprecated* the events functionality in the Stein release. [13]
512
+   Con:
513
+   * Another agent to convince users to install on their systems.
514
+   * Reinventing work already done in the Ceilometer agent.  The OpenStack community already uses Ceilometer and contributes updates when something fails.
515
+   This alternate solution will be detailed in a separate spec, as it is likely
516
+   the long term solution Monasca will need.
517
+
518
+#. Openstack Panko [5] is a event storage and REST API for Ceilometer.
519
+   Pro:
520
+   * An 'official' subproject within Telemetry, so there is some community recognition.
521
+   Con:
522
+   * Its primary storage is in a relational database which has problems with scale.
523
+   * It is not maintained actively and not ready for production. [11]
524
+   * It will be deprecated eventually. [12]
525
+
526
+Data model impact
527
+-----------------
528
+
529
+None
530
+
531
+REST API impact
532
+---------------
533
+
534
+#. We are proposing to tweak the Monasca Event data format by removing and adding following
535
+   fields as mentioned in "Proposed change" section above.
536
+
537
+   Remove fields (JSON path notation): "service", "dimensions.topic",
538
+   "dimensions.hostname" and "event.priority"
539
+
540
+   Add fields (JSON path notation): "dimensions.publisher_id", "dimensions.user_id" and
541
+   "dimensions.project_id"
542
+
543
+   This change will have an impact on Monasca Events API.
544
+
545
+Security impact
546
+---------------
547
+
548
+The proposed Monasca Ceilometer events publisher will collect and publish
549
+Openstack event (notification) data to Monasca API. Openstack notification
550
+data does not have any sensitive data like 'tokens'.
551
+Notifications do contain 'user_id' and 'project_id' fields but do not
552
+contain any Personally Identifiable Information (PII) for the user or
553
+the project.
554
+
555
+
556
+Other end user impact
557
+---------------------
558
+
559
+None.
560
+
561
+Performance Impact
562
+------------------
563
+
564
+#. The number of notifications(events) generated by different services will depend on the capacity
565
+   of the cloud along with the number of resources being created by the users.
566
+
567
+   For example, if there was a large number of compute VM's being created or destroyed it could
568
+   lead to a surge in number of notifications (events) that would have to be published.  Optimum
569
+   configuration options related to say event batch size and event batch interval would have to be
570
+   documented, to reduce any adverse affect on performance.
571
+
572
+#. Monasca Ceilometer publisher runs within Ceilometer Notification Agent component and invoked as a
573
+   last step in its data processing pipeline. It is an additional component that will have to to be
574
+   deployed on all the controller nodes.  We will have to evaluate the performance impact of
575
+   Ceilometer Notification Agent when publishing events to Monasca Events API.
576
+
577
+
578
+Other deployer impact
579
+---------------------
580
+
581
+#. The proposed new Monasca-Ceilometer events publisher will introduce
582
+   few new configuration options like
583
+   * events api endpoint
584
+   * events batch interval
585
+   * events batch size
586
+   * events retry interval
587
+
588
+#. Monasca Ceilometer Events publisher will have to to be added to Ceilometer's
589
+   "[ceilometer.event.publisher]" section  entry_points.txt
590
+
591
+   For example:
592
+
593
+   [ceilometer.event.publisher]
594
+   monasca = ceilometer.publisher.monclient:MonascaEventsPublisher
595
+
596
+#. As part of developing new Monasca Ceilometer Events publisher devstack plugin would be updated to
597
+   add the above configuration changes.
598
+
599
+
600
+Developer impact
601
+----------------
602
+
603
+#. The proposed change to Monasca Event Format will have an impact on existing Monasca Event API,
604
+   since Monasca Event Format will have to be tweaked.  (See REST API Impact section above)
605
+
606
+
607
+Implementation
608
+==============
609
+
610
+Assignee(s)
611
+-----------
612
+
613
+Primary assignee:
614
+  joadavis, aagate
615
+
616
+Other contributors:
617
+  <launchpad-id or None>
618
+
619
+
620
+Work Items
621
+----------
622
+
623
+#. Implement new Monasca Ceilometer Events publisher.
624
+
625
+#. Implement monasca-ceilometer devstack plugin changes to deploy
626
+   new events publisher.
627
+
628
+#. Implement unit tests for Events publisher.
629
+
630
+#. Implement change to Monasca Event format in Monasca Events API.
631
+
632
+
633
+Dependencies
634
+============
635
+
636
+#. Monasca Events API 1.0: https://storyboard.openstack.org/#!/story/2001654
637
+
638
+#. Monasca Ceilometer project: https://github.com/openstack/monasca-ceilometer
639
+
640
+#. Ceilometer Data processing and pipelines:
641
+https://docs.openstack.org/ceilometer/pike/admin/telemetry-data-pipelines.html
642
+
643
+Testing
644
+=======
645
+
646
+#. New Monasca Ceilometer Event publisher unit tests will be added, which can test publishing with
647
+   various config options events batch size, events batch interval, handling retry when Monasca
648
+   Event API is not available.
649
+
650
+#. Adding tempest tests for Monasca Ceilometer events publisher could be looked at as part of
651
+   separate effort.
652
+
653
+   Please note that current Monasca Ceilometer samples publisher does not have tempest tests either
654
+   so having tempest tests for both events and samples publisher could be considered in the future.
655
+
656
+Documentation Impact
657
+====================
658
+
659
+#. New Monasca Events Publisher config options will be documented
660
+
661
+   * events api endpoint
662
+   * events batch interval
663
+   * events batch size
664
+   * events retry interval
665
+
666
+#. Recommended values for each of the config options will also be documented based on the size of
667
+   the cloud and resources for Cloud Operators.
668
+
669
+References
670
+==========
671
+
672
+[1] Monasca Events API 1.0: https://storyboard.openstack.org/#!/story/2001654
673
+
674
+[2] Monasca Ceilometer project: https://github.com/openstack/monasca-ceilometer
675
+
676
+[3] Ceilometer Data processing and pipelines:
677
+https://docs.openstack.org/ceilometer/pike/admin/telemetry-data-pipelines.html
678
+
679
+[4] Ceilometer Events: https://docs.openstack.org/ceilometer/latest/admin/telemetry-events.html
680
+
681
+[5] Openstack Panko: https://github.com/openstack/panko
682
+
683
+[6] Monasca Event Format:
684
+https://github.com/openstack/monasca-events-api/blob/master/doc/api-samples/v1/req_simple_event.json
685
+
686
+[7] Ceilometer System Architecture Diagram:
687
+https://docs.openstack.org/ceilometer/ocata/architecture.html
688
+
689
+[8] Monasca Events POST v1.0 API:
690
+https://github.com/openstack/monasca-events-api/blob/master/api-ref/source/events.inc
691
+
692
+[9] Cross-Tenant Metric Submission:
693
+https://github.com/openstack/monasca-agent/blob/master/docs/MonascaMetrics.md#cross-tenant-metric-submission
694
+
695
+[10] Ceilometer pipeline yaml documentation:
696
+https://docs.openstack.org/ceilometer/latest/admin/telemetry-data-pipelines.html
697
+
698
+[11] No future for Panko or Aodh:
699
+https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/
700
+
701
+[12] Ceilometer Events deprecated means Panko also deprecated:
702
+http://eavesdrop.openstack.org/irclogs/%23openstack-telemetry/%23openstack-telemetry.2018-10-10.log.html
703
+
704
+[13] Ceilometer Events marked as deprecated in Stein:
705
+https://review.openstack.org/#/c/603336/

+ 586
- 0
specs/stein/approved/monasca-events-listener.rst View File

@@ -0,0 +1,586 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+=======================
8
+Monasca Events Listener
9
+=======================
10
+
11
+https://storyboard.openstack.org/#!/story/2003023
12
+
13
+Monasca Events API [1]_ was developed to store event data in Elasticsearch.
14
+A new application could use the Monasca Events API to post an event directly for processing
15
+and storage.
16
+However, a general collection service is needed to capture the existing OpenStack Notifications
17
+already generated by OpenStack Services and pass them to Monasca for storage and processing.
18
+This specification proposes creating a new Monasca Events Listener to capture events and pass
19
+them to Monasca services.
20
+
21
+
22
+Problem description
23
+===================
24
+
25
+All Openstack services generate a lot of notifications or events which contain large amounts of
26
+operational and state information about the service and its resources. Services such as Nova [14]_
27
+already publish to a RabbitMQ topic.
28
+This notification data is not currently available in Monasca.
29
+
30
+Ceilometer data processing pipeline [3]_ provides an extensible mechanism of publishing samples and
31
+events using a custom publisher.  Ceilometer samples represent a quantity that can be measured
32
+(e.g. the size of a volume) and events represent an occurrence of an event and do not have any
33
+associated quantity (e.g. volume was created).  The Telemetry project also has the Panko [5]_
34
+service for indexing and storing these events.
35
+
36
+A previous `spec <../../rocky/not-implemented/monasca-events-publishing.html>`_ was created to
37
+specify an enhancement to Ceilometer to allow collected events to be published to the Monasca
38
+Events API.
39
+However, with the recent deprecation of Ceilometer's Event functionality [13]_, this is no longer
40
+a long-term option.
41
+
42
+This spec is for creating a new Monasca service which would listen for OpenStack Event messages
43
+(often called notifications) and process them through Monasca by consuming the RabbitMQ message
44
+and producing a post to the Monasca Events API with that event.
45
+
46
+This service could use Ceilometer, Vitrage, Watcher, or another service as an example of how to
47
+listen to notifications from OpenStack services such as Nova [14]_.
48
+
49
+It is being proposed to make the Monasca Events Listener service a part of the Monasca Agent
50
+code base and reuse code, including the existing monasca_setup script and config.
51
+
52
+
53
+Use Cases
54
+---------
55
+
56
+#. Openstack notification data would be stored in Elasticsearch via the Monasca services
57
+
58
+   Example sequence:
59
+
60
+   #. Nova completes the creation of a VM
61
+   #. Nova generates a Notification message to oslo.messaging
62
+   #. oslo.messaging posts the message to RabbitMQ
63
+   #. Monasca Event Listener receives the Notification message from RabbitMQ
64
+   #. Monasca Event Listener validates and translates the Notification to a Monasca Events
65
+      API format according to the configuration
66
+   #. Monasca Event Listener publishes formatted Notification to Monasca Events API
67
+   #. Monasca Events API receives and validates formatted Notification
68
+   #. Monasca Events API stores event Notification in configured Elasticsearch instance
69
+
70
+
71
+Proposed change
72
+===============
73
+
74
+#. Monasca Event Listener will be a new service which can be run in a Highly
75
+   Available configuration by running an instance of Monasca Event Listener
76
+   on each Controller node in a cloud.  Each node will listen to OpenStack
77
+   notifications in RabbitMQ and convert notifications to a post to the Monasca
78
+   Events API.  The Monasca Events API then passes the notification on as Kafka
79
+   messages on the 'monevents' topic, where the Monasca Persister will receive
80
+   and store them. By the nature of RabbitMQ clients, the load will be
81
+   distributed between the Monasca Events Listener instances (only one listener
82
+   will process each RabbitMQ message).
83
+
84
+#. Monasca Event Listener will filter messages from OpenStack services based on
85
+   specification of event_types to be collected.  This will reduce 'noise' and
86
+   focus event collection on those events that are deemed valuable.
87
+   This filtering specification can be from a configuration file, or optionally
88
+   could be controlled through a new API implemented as part of the Monasca
89
+   Events API service.
90
+
91
+#. OpenStack Notifications consist of envelope and payload fields.  See [15]_ and [16]_ for
92
+   examples.
93
+
94
+   Example OpenStack Notification data format:
95
+
96
+   .. code-block:: javascript
97
+
98
+        {
99
+            "_context_auth_token": "42630b3ea13242fcad20e0a92d0207f1",
100
+            "_context_domain": null,
101
+            "_context_instance_lock_checked": false,
102
+            "_context_is_admin": true,
103
+            "_context_project_domain": null,
104
+            "_context_project_id": "a4f77",
105
+            "_context_project_name": "admin",
106
+            "_context_quota_class": null,
107
+            "_context_read_deleted": "no",
108
+            "_context_read_only": false,
109
+            "_context_remote_address": "192.168.245.4",
110
+            "_context_request_id": "req-5948338c-f223-4fd8-9249-8769f7a3e460",
111
+            "_context_resource_uuid": null,
112
+            "_context_roles": [
113
+                "monasca-user",
114
+                "admin",
115
+                "KeystoneAdmin"
116
+            ],
117
+            "_context_service_catalog": [
118
+                {
119
+                    "endpoints": [
120
+                        {
121
+                            "adminURL": "http://192.168.245.8:8776/v2/a4f77",
122
+                            "internalURL": "http://192.168.245.8:8776/v2/a4f77",
123
+                            "publicURL": "http://192.168.245.9:8776/v2/a4f77",
124
+                            "region": "region1"
125
+                        }
126
+                    ],
127
+                    "name": "cinderv2",
128
+                    "type": "volumev2"
129
+                },
130
+                {
131
+                    "endpoints": [
132
+                        {
133
+                            "adminURL": "http://192.168.245.8:8776/v1/a4f77",
134
+                            "internalURL": "http://192.168.245.8:8776/v1/a4f77",
135
+                            "publicURL": "http://192.168.245.9:8776/v1/a4f77",
136
+                            "region": "region1"
137
+                        }
138
+                    ],
139
+                    "name": "cinder",
140
+                    "type": "volume"
141
+                }
142
+            ],
143
+            "_context_show_deleted": false,
144
+            "_context_tenant": "a4f77",
145
+            "_context_timestamp": "2015-09-18T20:54:23.468522",
146
+            "_context_user": "be396488c7034811a200a3cb1d103a28",
147
+            "_context_user_domain": null,
148
+            "_context_user_id": "be396488c7034811a200a3cb1d103a28",
149
+            "_context_user_identity": "be396488c7034811a200a3cb1d103a28 a4f77 - - -",
150
+            "_context_user_name": "admin",
151
+            "_unique_id": "ff9699d587bf4283a3c367ab88be1541",
152
+            "event_type": "compute.instance.create.start",
153
+            "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
154
+            "payload": {
155
+                "access_ip_v4": null,
156
+                "access_ip_v6": null,
157
+                "architecture": null,
158
+                "availability_zone": null,
159
+                "cell_name": "",
160
+                "created_at": "2015-09-18 20:55:25+00:00",
161
+                "deleted_at": "",
162
+                "disk_gb": 1,
163
+                "display_name": "testeee",
164
+                "ephemeral_gb": 0,
165
+                "host": null,
166
+                "hostname": "testeee",
167
+                "image_meta": {
168
+                    "base_image_ref": "df0c8",
169
+                    "container_format": "bare",
170
+                    "disk_format": "qcow2",
171
+                    "min_disk": "1",
172
+                    "min_ram": "0"
173
+                },
174
+                "image_name": "glanceaaa3",
175
+                "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
176
+                "instance_flavor_id": "1",
177
+                "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
178
+                "instance_type": "m1.tiny",
179
+                "instance_type_id": 4,
180
+                "kernel_id": "",
181
+                "launched_at": "",
182
+                "memory_mb": 512,
183
+                "metadata": {},
184
+                "node": null,
185
+                "os_type": null,
186
+                "progress": "",
187
+                "ramdisk_id": "",
188
+                "reservation_id": "r-1ghilddw",
189
+                "root_gb": 1,
190
+                "state": "building",
191
+                "state_description": "",
192
+                "tenant_id": "a4f77",
193
+                "terminated_at": "",
194
+                "user_id": "be396488c7034811a200a3cb1d103a28",
195
+                "vcpus": 1
196
+            },
197
+            "priority": "INFO",
198
+            "publisher_id": "compute.ccp-compute0001-mgmt",
199
+            "timestamp": "2015-09-18 20:55:37.639023"
200
+        }
201
+
202
+#. All the fields with the prefix of '_context' are the envelope fields, the
203
+   other interesting fields are
204
+
205
+   #. 'message_id' - notification identifier
206
+   #. 'payload' - contains most of the relevant and useful information in JSON format
207
+   #. 'priority' - notification priority
208
+   #. 'publisher_id' - notification publisher
209
+   #. 'timestamp' - notification timestamp
210
+
211
+#. Monasca Event Listener converts the OpenStack notifications to Monasca events format.
212
+   This format will be suitable for Kafka messaging and will match the expected data fields
213
+   of the Monasca Persister.  This conversion and validation should be common between the
214
+   Monasca Event Listener and Monasca Event API.
215
+
216
+#. The Kafka client connection will handle communication issues such as reconnections and
217
+   resending as needed.
218
+
219
+#. Monasca Events API allows a field called 'payload' which can be in an arbitrary
220
+   nested JSON format.
221
+   TODO: mappings
222
+
223
+   Example Monasca Event Format:
224
+
225
+   .. code-block:: javascript
226
+
227
+        events: [
228
+        {
229
+          dimensions": {
230
+                "service": "compute.ccp-compute0001-mgmt",
231
+                "topic": "notification.sample",
232
+                "hostname": "nova-compute:compute
233
+          },
234
+          event: {
235
+
236
+                  "event_type": "compute.instance.create.start",
237
+
238
+                  "payload": {
239
+                       "access_ip_v4": null,
240
+                       "access_ip_v6": null,
241
+                       "architecture": null,
242
+                       "availability_zone": null,
243
+                       "cell_name": "",
244
+                       "created_at": "2015-09-18 20:55:25+00:00",
245
+                       "deleted_at": "",
246
+                       "disk_gb": 1,
247
+                       "display_name": "testeee",
248
+                       "ephemeral_gb": 0,
249
+                       "host": null,
250
+                       "hostname": "testeee",
251
+                       "image_meta": {
252
+                           "base_image_ref": "df0c8",
253
+                           "container_format": "bare",
254
+                           "disk_format": "qcow2",
255
+                           "min_disk": "1",
256
+                           "min_ram": "0"
257
+                       },
258
+                      "image_name": "glanceaaa3",
259
+                      "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
260
+                      "instance_flavor_id": "1",
261
+                      "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
262
+                      "instance_type": "m1.tiny",
263
+                      "instance_type_id": 4,
264
+                      "kernel_id": "",
265
+                      "launched_at": "",
266
+                      "memory_mb": 512,
267
+                      "metadata": {},
268
+                      "node": null,
269
+                      "os_type": null,
270
+                      "progress": "",
271
+                      "ramdisk_id": "",
272
+                      "reservation_id": "r-1ghilddw",
273
+                      "root_gb": 1,
274
+                      "state": "building",
275
+                      "state_description": "",
276
+                      "tenant_id": "a4f77",
277
+                      "terminated_at": "",
278
+                      "user_id": "be396488c7034811a200a3cb1d103a28",
279
+                      "vcpus": 1
280
+                      }
281
+                 },
282
+            publisher_id: "compute.ccp-compute0001-mgmt",
283
+            priority: "INFO"
284
+         }
285
+        ]
286
+
287
+#. Following fields in Monasca Event data may not be available in the OpenStack notification
288
+   data format:
289
+
290
+   * "service"
291
+   * "dimensions.topic"
292
+   * "event.priority"
293
+
294
+   We are proposing removing these fields from Monasca Event format (will be done as a separate
295
+   spec/implementation process) for the following reasons:
296
+
297
+   "service": Currently OpenStack notifications do not specify a service, that
298
+   generated the notification in a consistent way. It might be possible to create an external
299
+   mapping file which maps event name to a service but its hard to maintain such mapping over a
300
+   period of time.
301
+
302
+   "dimensions.topic": This field is not available in the source OpenStack notification.
303
+   However, the Monasca Event Listener may be able to save the RabbitMQ topic that the notification
304
+   was collected from.  In that case, this field should be used.
305
+
306
+   "event.priority": This field is not currently available in Ceilometer Event format. It is
307
+   available in the source OpenStack notification. Note: If we think this field can be useful we can
308
+   propose adding it to the Monasca Event Listener format.
309
+
310
+#. Following new fields will be added to Monasca Event data as dimensions:
311
+
312
+   * "dimensions.publisher_id": Identifier for the publisher that generated the event.
313
+   * "dimensions.user_id": Identifier for user that generated the event.
314
+   * "dimensions.project_id": Identifier of the project that generated the event.
315
+
316
+#. 'hostname' is available in the event payload, but its location might differ from event to event.
317
+
318
+#. The proposed new Monasca Event Listener will have the ability to submit event
319
+   data in a batch and at a configurable frequency (similar to current samples publisher). The
320
+   event data will be published if the items in the current batch reach their maximum size
321
+   (config setting) or if certain time interval has elapsed since the last publish
322
+   (config setting). This will make sure that the batch does not get huge at the same time
323
+   there is no significant delay in publishing of the events to Monasca Events API.
324
+
325
+#. Monasca Event Listener will have a configuration file to configure connection information for
326
+   both RabbitMQ and Monasca Events API.
327
+
328
+#. The "tenant_id" and "user_id" that the notification relates to are available in "payload"
329
+   section of the notification, and these notifications are generated by each service itself.
330
+
331
+   There is no additional "OpenStack-operator-agent" like component or functionality required to
332
+   fetch that data from the service and publish to monasca event api on behalf of the original
333
+   tenant.
334
+   (Ceilometer publishing pipeline simply extracts these "tenant_id" and "user_id" fields from the
335
+   "payload" and makes those fields available as "tenant_id" and "user_id" traits, which would then
336
+   be mapped to "dimensions.project_id" and "dimensions.user_id" fields in monasca events format.)
337
+
338
+   In other words, original "tenant_id" and "user_id" values are available in
339
+   the payload of the notification, and will make its way to "dimensions.tenant_id"
340
+   and "dimensions.user_id" in Monasca Event.
341
+
342
+   Questions/TODO:
343
+
344
+   * Q: Do we need to do anything special to handle multi-tenancy in monasca-events api like being
345
+     done for metrics [9]_ ? Would original user_id and tenant_id in "dimensions.user_id" and
346
+     "dimensions.tenant_id" fields in dimensions serve this purpose?
347
+
348
+     * A: Monasca Events Listener can start with sending all events to a single "admin" tenant
349
+       and if required in the future some other process could copy select metrics back to tenant
350
+       projects.
351
+
352
+   * Q: In Ceilometer V2 API (which has been deprecated and removed), when querying data the role
353
+     "admin" could access data for all tenants, whereas a user with "ceilometer-admin" role could
354
+     access only data for a particular tenant. Can we implement something like this for
355
+     monasca-events api when querying for data?
356
+
357
+     * A: In Monasca API every request is scoped to the project, so there is no equivalent of
358
+       Ceilometer's "admin" role to query data for all projects.  So placing all events in to
359
+       an "admin" project may be the best approach.
360
+
361
+   * Q: How should services which generate notifications but do not include a tenant_id be
362
+     handled?  For example Keystone [16]_.
363
+     How does Ceilometer handle such events?
364
+
365
+     * A: If all events are in an "admin" project then admin metrics like shared ceph cluster
366
+       load or provider network load can be copied back to tenants so they may understand how
367
+       infrastructure is affecting their workload.
368
+
369
+   * Note: Configuration of Elasticsearch cluster is out of scope for this spec. If needed
370
+     could assign a separate Elasticsearch cluster to the events API to avoid overloads.
371
+
372
+
373
+Alternative Solutions
374
+---------------------
375
+
376
+#. Reuse the Ceilometer functionality to collect and publish events to the
377
+   Monasca Events API.  While this may be less work initially, Ceilometer has
378
+   deprecated the Events functionality as of Stein. [13]_
379
+
380
+#. An alternate Events Listener was proposed that would listen for RabbitMQ events
381
+   then publish them directly to the 'monevents' topic in Kafka.  Discussion on this
382
+   can be seen in the git history for this document and in IRC logs [18]_.
383
+
384
+   Pro:
385
+
386
+   * A much simpler approach, more efficient that HTTP hop through another service.
387
+   * No need for batching in service, as RabbitMQ and Kafka clients would handle fast
388
+     throughput and short network interruptions.
389
+
390
+   Con:
391
+
392
+   * Nova Cells v2 each have their own RabbitMQ instances. While most deployments
393
+     would likely have a centralized RabbitMQ, it is not required in the documented
394
+     architecture.
395
+   * Regions may also cause separation of RabbitMQ instances that need to be monitored.
396
+   * While it might be possible to have a service/agent in each Cell publish back to
397
+     a centralized to Kafka directly, our authentication and networking for Kafka was
398
+     not designed to support that.
399
+
400
+#. OpenStack Panko [5]_ is a event storage and REST API for Ceilometer and could
401
+   be used instead of a Monasca solution.
402
+
403
+   Pro:
404
+
405
+   * An 'official' subproject within Telemetry, so there is some community recognition.
406
+
407
+   Con:
408
+
409
+   * Its primary storage is in a relational database which has problems with scale.
410
+   * It is not maintained actively and not ready for production. [11]_
411
+   * It will be deprecated eventually. [12]_
412
+
413
+
414
+Data model impact
415
+-----------------
416
+
417
+None
418
+
419
+REST API impact
420
+---------------
421
+
422
+#. We are proposing to tweak the Monasca Event data format by removing and adding following
423
+   fields as mentioned in "Proposed change" section above.
424
+
425
+   Remove fields or make them optional (JSON path notation): "service", "dimensions.topic",
426
+   "dimensions.hostname" and "event.priority"
427
+
428
+   Add fields (JSON path notation): "dimensions.publisher_id", "dimensions.user_id" and
429
+   "dimensions.project_id"
430
+
431
+   This change will have an impact on Monasca Events API.
432
+
433
+Security impact
434
+---------------
435
+
436
+The proposed Monasca Event Listener will collect and publish OpenStack event
437
+(notification) data to Monasca Events API. OpenStack notification data does not
438
+have any sensitive data like 'tokens'.
439
+Notifications do contain 'user_id' and 'project_id' fields but do not
440
+contain any Personally Identifiable Information (PII) for the user or
441
+the project.
442
+
443
+
444
+Other end user impact
445
+---------------------
446
+
447
+None.
448
+
449
+Performance Impact
450
+------------------
451
+
452
+#. The number of notifications(events) generated by different services will depend on the capacity
453
+   of the cloud along with the number of resources being created by the users.
454
+
455
+   For example, if there was a large number of compute VM's being created or destroyed it could
456
+   lead to a surge in number of notifications (events) that would have to be published.  Optimum
457
+   configuration options related to say event batch size and event batch interval would have to be
458
+   documented, to reduce any adverse affect on performance.
459
+
460
+#. The proposed Monasca Event Listener is a new service, so performance is unknown.  However, the
461
+   Monasca API has been shown to have a high performance throughput.
462
+
463
+
464
+Other deployer impact
465
+---------------------
466
+
467
+#. The proposed Monasca Event Listener will introduce a
468
+   few new configuration options like
469
+
470
+   * RabbitMQ connection information
471
+   * Monasca Events API endpoint URL
472
+   * events batch interval
473
+   * events batch size
474
+   * events retry interval
475
+   * Keystone credentials for obtaining a token
476
+   * Conversion options for OpenStack notifications to the Monasca Event format.  This may
477
+     be stored in separate pipeline configuration files, similar to how transform specs
478
+     are configured in Monasca Transform.
479
+
480
+#. As part of developing new Monasca Event Listener devstack plugin would be
481
+   updated to add the above configuration changes.
482
+
483
+
484
+Developer impact
485
+----------------
486
+
487
+#. The proposed change to Monasca Event Format will have an impact on existing Monasca Event API,
488
+   since Monasca Event Format will have to be tweaked.  (See REST API Impact section above)
489
+
490
+Implementation
491
+==============
492
+
493
+Assignee(s)
494
+-----------
495
+
496
+Primary assignee:
497
+  joadavis, aagate
498
+
499
+Other contributors:
500
+  <launchpad-id or None>
501
+
502
+
503
+Work Items
504
+----------
505
+
506
+#. Implement new Monasca Event Listener.
507
+
508
+   * Connection to RabbitMQ for OpenStack Notifications
509
+   * Add filtering of notifications for configured event_types
510
+
511
+     * Specification in configuration file
512
+     * (Optional) Creation of a new API to configure event_type subscriptions
513
+
514
+   * Validation of OpenStack Notification data and format
515
+   * Conversion of data format to meet Monasca Events requirements
516
+   * Publishing to Monasca Events API
517
+   * Configuration of conversion specifications per-event type
518
+
519
+#. Implement monasca devstack plugin changes to deploy
520
+   new Events Listener service.
521
+
522
+#. Implement unit tests for Monasca Event Listener.
523
+
524
+
525
+Dependencies
526
+============
527
+
528
+None
529
+
530
+Testing
531
+=======
532
+
533
+#. New Monasca Event Listener unit tests will be added, which can test publishing with
534
+   various config options events batch size, events batch interval, handling retry when Monasca
535
+   Event API is not available.
536
+
537
+#. Adding tempest tests for Monasca Event Listener could be looked at as part of
538
+   separate effort.
539
+
540
+
541
+Documentation Impact
542
+====================
543
+
544
+#. New Monasca Event Listener config options will be documented
545
+
546
+#. Recommended values for each of the config options will also be documented based on the size of
547
+   the cloud and resources for Cloud Operators.
548
+
549
+References
550
+==========
551
+
552
+.. [1] Monasca Events API 1.0: https://git.openstack.org/cgit/openstack/monasca-events-api/
553
+
554
+[2] Monasca Ceilometer project: https://github.com/openstack/monasca-ceilometer
555
+
556
+.. [3] Ceilometer Data processing and pipelines: https://docs.openstack.org/ceilometer/pike/admin/telemetry-data-pipelines.html
557
+
558
+[4] Ceilometer Events: https://docs.openstack.org/ceilometer/latest/admin/telemetry-events.html
559
+
560
+.. [5] Openstack Panko: https://github.com/openstack/panko
561
+
562
+[6] Monasca Event Format: https://github.com/openstack/monasca-events-api/blob/master/doc/api-samples/v1/req_simple_event.json
563
+
564
+[7] Ceilometer System Architecture Diagram: https://docs.openstack.org/ceilometer/ocata/architecture.html
565
+
566
+[8] Monasca Events POST v1.0 API: https://github.com/openstack/monasca-events-api/blob/master/api-ref/source/events.inc
567
+
568
+.. [9] Cross-Tenant Metric Submission: https://github.com/openstack/monasca-agent/blob/master/docs/MonascaMetrics.md#cross-tenant-metric-submission
569
+
570
+[10] Ceilometer pipeline yaml documentation: https://docs.openstack.org/ceilometer/latest/admin/telemetry-data-pipelines.html
571
+
572
+.. [11] No future for Panko or Aodh: https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/
573
+
574
+.. [12] Ceilometer Events deprecated means Panko also deprecated: http://eavesdrop.openstack.org/irclogs/%23openstack-telemetry/%23openstack-telemetry.2018-10-10.log.html
575
+
576
+.. [13] Ceilometer Events marked as deprecated in Stein: https://review.openstack.org/#/c/603336/
577
+
578
+.. [14] Nova notification version update lists services effected (see "Deprecating legacy notifications"): https://etherpad.openstack.org/p/nova-ptg-stein
579
+
580
+.. [15] Nova notification reference: https://docs.openstack.org/nova/latest/reference/notifications.html#existing-versioned-notifications
581
+
582
+.. [16] Keystone notification reference: https://docs.openstack.org/keystone/latest/advanced-topics/event_notifications.html#example-notification-project-create
583
+
584
+[17] Monasca Events API publisher to Kafka: https://github.com/openstack/monasca-events-api/blob/master/monasca_events_api/app/common/events_publisher.py
585
+
586
+.. [18] Monasca IRC meeting Dec 15, 2018: http://eavesdrop.openstack.org/meetings/monasca/2018/monasca.2018-12-12-15.00.log.html

Loading…
Cancel
Save