4463 Commits

Author SHA1 Message Date
Chandan Kumar
92f275ffc5 Merge panko tempest config into ceilometer repo
* https://review.openstack.org/525072 removes the dummy panko
  tempest plugin from panko repo but panko tempest config is used
  in ceilometer integration tests. It merges the required config
  in order to avoid turbulence in integration tests.


Change-Id: I97a5abed3486f63363782f52e7746e87bd88ed4a
2017-12-05 07:30:08 +00:00
Zuul
163f2d756c Merge "remove unused pollster exception resources" 2017-12-04 04:19:48 +00:00
Zuul
d9bacb06f0 Merge "simplify test data generation" 2017-12-04 04:19:48 +00:00
Zuul
16910778a2 Merge "remove duplicate polling.yaml setup" 2017-12-04 04:19:38 +00:00
Zuul
8bc1b02134 Merge "remove test_manager_exception_persistency" 2017-12-04 04:18:34 +00:00
Zuul
6d940c5e56 Merge "minor polling cleaning" 2017-12-04 04:18:33 +00:00
OpenStack Proposal Bot
b9e134914e Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I2770cf36fbcf42fbbfec685fa0375bfa1b4cac47
2017-12-01 07:21:52 +00:00
gord chung
134743c292 remove unused pollster exception resources
this stuff isn't used in any of the tests. there is an existing
pollster exception test here in
test_manager.TestRunTasks.test_polling_exception

Change-Id: I868698dcad765880d30d5b2703250aad72a33338
2017-11-30 16:08:12 -05:00
gord chung
5f98f16c54 simplify test data generation
Change-Id: I611ce1974b62719030b5ff41e694eade5fda8abc
2017-11-30 15:50:56 -05:00
gord chung
8f7694d7a5 remove duplicate polling.yaml setup
Change-Id: Id4640cd5054140e3f175b71a3a0339dba7b9011d
2017-11-30 14:52:30 -05:00
gord chung
4ba8febea3 remove test_manager_exception_persistency
i'm not sure exactly what this tests... but the original test[1]
does not exist anymore

[1] I808dcfae18d23240f8e095d6c97c8dede7dede8f

Change-Id: I86a96557f7e316850adae32b9976cb1d7c7b12b3
2017-11-30 13:41:53 -05:00
gord chung
fcf78b1e96 minor polling cleaning
- remove custom Sample object from test
- remove mock of threadgroup that does not exist.

Change-Id: I1a55df5533b978c4c56dee37502fdb9f961cf91d
2017-11-30 10:44:00 -05:00
Zuul
2455ddda1e Merge "Change oslo.messaging prefetch default" 2017-11-28 23:59:27 +00:00
Zuul
6f946db775 Merge "Remove unused ceilometer.utils.update_nested" 2017-11-27 23:25:54 +00:00
Zuul
8492777e89 Merge "utils: remove unused create_periodic function" 2017-11-27 23:16:05 +00:00
Julien Danjou
a034d2d8f5 polling: run polling tasks immediately on start
When Ceilometer polling agent start, one has to wait N seconds for the first
polling to happen. This makes testing extremely difficult.

I can't see any good reason to not poll at (re)start. Since the last run time
is lost anyway, the interval will never be perfect. So at least let's make it
convenient by polling on startup.

Also set a default random 0-10 seconds delay before the first poll so if a lot
of daemons are started at the same time they don't all hit the same endpoint at
the same time.

Change-Id: I0741a586cec499c259f0e90977f185c4e68a99d3
2017-11-23 10:29:10 +01:00
Julien Danjou
2829f005ef Set shuffle_time_before_polling_task to float and set a minimum
Change-Id: I34eb3cc54680f4fe6c3a79c288b749b9dfa6e968
2017-11-23 10:27:51 +01:00
Zuul
bd464f1f57 Merge "Imported Translations from Zanata" 2017-11-21 21:03:53 +00:00
Zuul
3ba3b7fe7c Merge "fix ceilometer-compute invoke libvirt exception error" 2017-11-21 15:46:52 +00:00
xiexianbin
c477678e57 fix ceilometer-compute invoke libvirt exception error
when nova compute start vm not create by nova-compute,
the ceilometer-compute will occur libvirtError: "metadata
not found: Requested metadata element is not present".
and cause all vm meter not report.

Change-Id: Id71788606bc0da9a7959831fb90d13c25c0b8dcb
2017-11-21 12:48:31 +01:00
OpenStack Proposal Bot
1b09ed0618 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Ibb389c83463accdb847f3fbd651541901c5e4dbd
2017-11-21 06:35:52 +00:00
Zuul
77506641c4 Merge "always declare partitioning variables" 2017-11-21 04:12:55 +00:00
Zuul
6d27648c29 Merge "queues per manager rather than per pipeline" 2017-11-21 03:57:10 +00:00
Zuul
bd583fe13b Merge "pluggable notification agent" 2017-11-21 01:33:36 +00:00
gord chung
367a0de7a6 always declare partitioning variables
also, just use partition_coordinator to figure out if we need to handle
stuff as that's what we use everywhere else.

Change-Id: I8724a41408b89f29b600a03fbf1c7febb55fb5e5
2017-11-20 22:18:41 +00:00
Zuul
979f9b2595 Merge "Remove unused ceilometer.utils.EPOCH_TIME" 2017-11-20 09:49:31 +00:00
Zuul
6f1ae9d1b5 Merge "Remove unused ceilometer.utils.sanitize_timestamp" 2017-11-20 08:35:08 +00:00
Julien Danjou
5cb3260990 Remove unused ceilometer.utils.update_nested
Change-Id: I2fd6b649cf6faf4089d01eb410dfb44f1587ae42
2017-11-17 16:24:08 +01:00
Julien Danjou
73f84350ab Remove unused ceilometer.utils.EPOCH_TIME
Change-Id: I13f9787e7ce289c428ce87771f432902ec305e01
2017-11-17 16:23:07 +01:00
Julien Danjou
9362f5f1af Remove unused ceilometer.utils.sanitize_timestamp
Change-Id: I471ff71aae9644ea38830b99eafbca22c48adcab
2017-11-17 16:22:22 +01:00
Julien Danjou
4c664c7b2f utils: remove unused create_periodic function
Change-Id: Ia1e6e45466a4237f81d67885d1a59bed6b06d9d1
2017-11-17 15:32:56 +01:00
gord chung
911b973d70 queues per manager rather than per pipeline
currently we create queue per pipeline which is not necessary. it
creates more memory usage and doesn't necessarily distribute
work more effectively. this hashes data to queues based on
manager but still internally, the data is destined to specific
pipeline based on event_type. this will minimise queue usage while
keeping internal code path the same.

Change-Id: I0ccd51f13457f208fe2ccedb6e680c91e132f78f
2017-11-16 14:43:46 -05:00
gord chung
60d9b87a80 pluggable notification agent
event, meter (and any other custom pipeline) can be enabled/disabled
by setting `pipelines` option under [notification] agent

Change-Id: Ia21256d0308457d077836e27b45d2acb8bb697e4
Closes-Bug: #1720021
2017-11-16 14:43:46 -05:00
gord chung
c82d28d9db remove redundant filter check
Change-Id: I2d1b476b2d867cda942ae095380c836c91872286
2017-11-16 14:43:46 -05:00
gord chung
9e58f1a6f4 separate polling code
polling is different from pipeline.

Change-Id: I5ec3ece1295181acd10c3b598523e796d23264bf
2017-11-16 14:43:46 -05:00
gord chung
9b648cd54e move pipeline out of __init__
see title

Change-Id: I9ff4403bffc39dafccc7d2fe407a4498e2f4cae0
2017-11-16 14:43:46 -05:00
gord chung
50415c0d08 separate base manager from pipeline
common agent for all

Change-Id: I19a83d3d0e5c91ab5cb6e792ab7389e36f8ede55
2017-11-16 14:43:46 -05:00
gord chung
2d67bd21dd nearly pluggable notification agent
notification agent now just asks for pipelinemanagers and gets
endpoints it should broadcast to from there. it only sets up a
listener for main queue and a listener for internal queue
(if applicable)

- pass in publishing/processing context into endpoints instead of
manager. context is based on partitioning or not
- move all endpoint/notifier setup to respective pipeline managers
- change interim broadcast filtering to use event_type rather than
publisher_id so all filtering uses event_type.
- add namespace to load supported pipeline managers
- remove some notification tests as they are redundant and only
different that it mocks stuff other tests don't mock
- change relevant_endpoint test to verify endpoints cover all pipelines

Related-Bug: #1720021
Change-Id: I9f9073e3b15c4e3a502976c2e3e0306bc99282d9
2017-11-16 14:43:46 -05:00
gord chung
48f35a35a4 set models as part of pipeline manager class
we need pipeline/source/sink specific classes for each pipeline
so just make it required rather than passing in as dict

Change-Id: Ia861cf460d5937346229176ca10fb18c239639db
2017-11-16 14:43:46 -05:00
gord chung
5a0579dd93 remove agent setup_* helper functions
- they are only used essentially for testing.
- cleanup stray pipeline references in polling tests
- remove random mocks that aren't mocking anything for a reason

Change-Id: I5881c0926dde2247c4606fed26e60bc5e197cf48
2017-11-16 14:43:46 -05:00
gord chung
cfbc3e00c2 move sample/event specifc pipeline models to own module
- move sample/event specifc pipeline models to own module
- make grouping key computation part of pipeline
- remove pipeline mocks from polling tests

Change-Id: I20349e48751090210f8a0074c4a735f1b7e74bc1
2017-11-16 14:43:46 -05:00
Mehdi Abaakouk
d208a7117a Change oslo.messaging prefetch default
In some case, Ceilometer can consume To of RAM. If batch is not enabled
the default behavior is to fetch all messages waiting in the queue.

Since I fail to change/expose this bad oslo.messaging default for us.
This change set a correct default on our side.

Change-Id: I3f4b0ef5fa90afb965e31584b34fdc30a5f4f9f1
2017-11-15 13:51:00 +01:00
gord chung
a638ceb8c4 stop double refreshing on start
just let the periodic job decide if it needs to refresh.

Change-Id: I300967d926ea4b8b415aac4744fc7bd183b4cca4
Closes-Bug: #1730849
2017-11-08 22:47:19 +00:00
Zuul
6a97e62c54 Merge "move listener targets to notification agent" 2017-11-08 11:09:55 +00:00
Zuul
b9835bc808 Merge "rename sample handler" 2017-11-08 11:00:57 +00:00
Zuul
6f1471fd4d Merge "common notification endpoint" 2017-11-08 11:00:56 +00:00
Zuul
4be508c5d7 Merge "libvirt: share disk device listing" 2017-11-08 01:24:37 +00:00
gord chung
d0339588ce move listener targets to notification agent
processing endpoints shouldn't dictate what targets are being listened
to. they should just process what it is given based on their filter.

move this logic to notification agent so every processing endpoint
isn't defining the same set of targets to listen to.

ensure duplicate targets aren't created

Change-Id: I9ffe28b6406dcef88ef6861eb8a81e1a3ad786d2
2017-11-08 01:05:28 +00:00
gord chung
2768334d01 rename sample handler
process_notification and process_notifications are very similar.
renamed process_notification to build_sample so we don't accidentally
call wrong thing. also, it's a bit more descriptive.

Change-Id: Id838ae552e822479208337b9ece415981fb5b25a
2017-11-08 01:05:28 +00:00
gord chung
000c5d89a3 common notification endpoint
make samples and events use a common endpoint class

Change-Id: I1d15783721f91ee90adfbac88cef2a44e0b23868
2017-11-08 01:05:28 +00:00