4319 Commits

Author SHA1 Message Date
gordon chung
f9255d20c3 deprecate db2 nosql driver
the db2 nosql product is not in development anymore as IBM has moved
on to an alternate solution. the existing driver code is untested
in the community and IBM products are moving off of it. this patch
marks db2 nosql driver for deprecation.

Closes-Bug: #1491819
Change-Id: Ic4e906615931143744f25d70935bdb8138a6b93a
2015-09-03 08:33:41 -04:00
Chris Dent
f24d1f43cc Sync devstack plugin with devstack:lib/ceilometer
This removes the cp of the meters.yaml file to /etc/ceilometer. It
is not needed.

This is part of a regular process of keeping the plugin and the
devstack version functionally equivalent until the devstack version
finally goes away.

Change-Id: I1261f054f077b73d3d64529ca9c59678ecbf15de
2015-09-03 09:25:20 +01:00
OpenStack Proposal Bot
d3a0af416b Updated from global requirements
Change-Id: I9c7097e617324966368c17b1e8526a1212b02d3b
2015-09-02 23:44:19 +00:00
gordon chung
962e5843bb remove old nova_notifier processing code
the nova_notifier plugin was removed in Kilo. this removes the
corresponding processing code.

Change-Id: Ibd39b392293247ec088ceea4b592383b4b92f3f0
Closes-Bug: #1491603
2015-09-02 17:47:15 -04:00
Jenkins
6c3c5a16cd Merge "add mandatory limit value to complex query list" 5.0.0.0b3 2015-09-02 18:15:47 +00:00
Jenkins
40be122a8b Merge "Event filtering for non-admin users" 2015-09-02 04:41:16 +00:00
gordon chung
a7bc4463a8 restrict admin event access
this patch restricts the scope of returned events. if admin role,
user is allowed to query all events which have traits.project_id
value that match it's own project OR any event without any
project_id trait.

Implements: blueprint events-rbac
Change-Id: I41d30b2fc11c226e109c499e7a35fdb2fa057e0b
2015-09-01 23:10:49 -04:00
Lianhao Lu
8b1ba63d29 Migrate the old snmp pollsters to new declarative pollster
Delete all the old snmp pollsters and to register the new declarative
snmp pollster to be used.

Change-Id: I8ebb2a53bca52eed77229d4cc56cf1f42aaa1a7b
Implements: blueprint declarative-snmp-metrics
2015-09-02 10:21:22 +08:00
Lianhao Lu
f9981f3e3e Support to load pollsters extensions at runtime
Make AgentManager to load pollster extensions at runtime. So that a
pollster builder may build multiple pollster extensions to be used by
the AgentManager.

In this way, we don't need to change the existing pollsters interface in
order to support declarative pollster, which means a single pollster
class might gives more than one meters.

All pollsters which implements classmethod interface build_pollsters()
should be registered in the namesapce of ceilometer.builder.poll.xxx, so
that AgentManger can correctly find it and use to dynamically generate
multiple pollsters through the 'builder' logic.

Change-Id: If7f8f80301474e2684ea0f1bbdc69cc202b9ef7e
Implements: blueprint declarative-snmp-metrics
2015-09-02 09:22:04 +08:00
Lianhao Lu
1520f65f8b Added snmp declarative hardware pollster
The snmp declarative hardware pollster loads the metric definition from
a special yaml file and can poll those metrics. This enables the
developers to modify the yaml file only to add new snmp metrics without
writing any new python code.

Added a new classmethod interface build_pollsters for PollsterBase. The
pollster plugin which implements this interface should return a list of
tuple (name, pollster), where name is the meter name it will return, and
pollster is the pollster object instances. All the pollsters returned by
this would be wrapped in stevedore extensions in
PollsterBase.get_pollster_extensions().

This pollster doesn't get registered temporarily to avoid duplicating
samples. Will be registered in the following patches when old snmp
pollsters are deleted.

Implements: blueprint declarative-snmp-metrics

Change-Id: I5abbe704cc08ea84aacf742eda7cee215141c3bb
2015-09-02 09:20:20 +08:00
Rohit Jaiswal
967d927278 Requeuing event with workload_partitioning on publish failure
when workload_partitioning is enabled,
publishing of samples occurs in the
pipeline listeners. If publishing fails
when single publisher is configured, event
will not be requeued or ack'ed.

This fix requeues or acks the event based on
ack_on_event_error.

Change-Id: I8f2f889736c8897e5b15952ab32308cf33205c3f
Closes-Bug: 1488202
2015-09-02 00:14:09 +00:00
Divya
4732fd516d Event filtering for non-admin users
Non-admin users should be restricted to see only those
events that were generated by them and not the events
associated with other users.The current behavior doesn't
have any restrictions on the events that a non-admin user
can view.As part of this patch, a non-admin user will see
only the events that are associated with the user_id and
project_id that the user requesting events belongs to.
Events stored in the ceilometer database has information
from all other services and in a cloud environment, there
must be restrictions on the data a non-admin user can access.

Change-Id: I2ed7425c14a70d1db4621a277ba5f4f575a15eba
blueprint: events-rbac
2015-09-01 08:20:23 -04:00
Jenkins
68bf02d9d7 Merge "Updated from global requirements" 2015-09-01 12:19:34 +00:00
Jenkins
a2973bfab6 Merge "Add index to metadata_hash column of resource table" 2015-09-01 12:15:39 +00:00
Mehdi Abaakouk
91ce9e1f56 integration: fix typo
Descriptions in autoscaling yaml talk about three servers
when only two are used.

This change fixes that.

Change-Id: Iac981b51fb3e380aa0a0a7971f5bb4fe83708530
2015-09-01 08:13:51 +02:00
Mehdi Abaakouk
ecc6ad8854 gnocchi: cleanup instance resource definition
Change-Id: I5f953217f616e1c2d62d884fd808746c65a66436
2015-09-01 08:10:33 +02:00
OpenStack Proposal Bot
f0a35485fd Updated from global requirements
Change-Id: I7678a03637b04ebd87b4163ebca60b42276db912
2015-09-01 04:22:25 +00:00
Jenkins
92fb620127 Merge "make telemetry sample payloads dictionaries" 2015-09-01 01:46:26 +00:00
Jenkins
5ff9e47451 Merge "Incorrect Links are updated" 2015-08-31 18:57:07 +00:00
Julien Danjou
e2423262d6 Adding pradk to MAINTAINERS
Pradeep Kilambi did a good job this last months on Ceilometer, let's
bring him in.

Change-Id: Ib16109919b5bc33da45a816bce210037330e6fcf
2015-08-31 13:59:28 +02:00
Julien Danjou
09fbc1f913 Adding liusheng to MAINTAINERS
Liusheng did a good job this last months working on Ceilometer. Let's
bring him in.

Change-Id: Ie28391498301972f2574b30ca9a12132c17e1029
2015-08-31 13:55:48 +02:00
Jenkins
80f35d8480 Merge "Removing unused dependency: discover" 2015-08-31 11:25:22 +00:00
Jenkins
ecd917d508 Merge "Control Events RBAC from policy.json" 2015-08-30 02:42:48 +00:00
Jenkins
3f6a01bf85 Merge "Events RBAC needs scoped token" 2015-08-30 02:42:43 +00:00
Jenkins
9dfb45058e Merge "Fixed identity trust event types" 2015-08-30 02:30:17 +00:00
Jenkins
5e62fb4f43 Merge "Fix requeue process on event handling error" 2015-08-29 18:58:13 +00:00
Jenkins
4107aa7257 Merge "gnocchi: quote the resource_id in url" 2015-08-29 16:28:00 +00:00
Jenkins
d9620cfbde Merge "Use new location of subunit2html" 2015-08-29 16:16:44 +00:00
Jenkins
afd7acdf39 Merge "integration: add some new tests" 2015-08-29 05:36:05 +00:00
Ashwin Agate
4b739eeb66 Add index to metadata_hash column of resource table
Resource table's metadata_hash column is not indexed.

Adding index on metadata_hash column reduces number
of rows scanned to one. That is if a resource
(same resource_id) has say n rows in resource table
(due to difference in metadata), with additional index
only one row will have to be scanned.

Change-Id: I2a99bdac88f5fa14ac5f82497d1a12196419ae68
Closes-Bug: #1488544
2015-08-29 01:29:44 +00:00
Jenkins
9466bda1cd Merge "Move profiler meters to yaml" 2015-08-28 23:11:38 +00:00
venkatamahesh
160058b199 Incorrect Links are updated
Pecan deployment and measurements page links are
corrected

Change-Id: Iaee9bfce9d86598277b5c2ba3788789960531472
Closes-Bug: #1488792
2015-08-28 22:42:57 +05:30
Chandan Kumar
68fc8d1df1 Removing unused dependency: discover
Change-Id: I30ba5f147e1738f8232ecd0dd1f8185993676117
2015-08-28 19:57:08 +05:30
Chris Dent
848bb2d1ae Use new location of subunit2html
The previous location is going away, this location is preferred.

Change-Id: I3558cc0cbd483cb42f2094e180d74fb976d06839
Closes-Bug: #1489404
2015-08-28 09:11:36 +00:00
Chris Dent
28d60c2534 Change tox default targets for quick use
This change makes it so that if you call `tox` or `tox -r` you will
get a useful result as a developer running tests:

    * py27 and py34 unit tests
    * a single functional test run (using mongodb)
    * pep8

This is slightly different from what was there before which was
as above but _two_ functional test runs, with mysql and postgresql.

The goal here is to make it dead simple for someone to (as quickly
as possible) make sure that there code is okay before committing.

Change-Id: I27dcf5265dea18be541b8b2857b0a135ff22f499
2015-08-28 08:56:12 +00:00
Rohit Jaiswal
cc3f6892ff Fixed identity trust event types
Identity trust event types have resource_type
prefixed with OS-TRUST:, this patch fixes
the declarative meter definition file to
include that change.

Closes-Bug: 1488707
DocImpact

Change-Id: I1f82049a78b7693f8d03803ce500f8f4c268b0a1
2015-08-28 04:42:49 +00:00
Jenkins
a45c333658 Merge "Keep the instance_type meta from polling and notification consistent" 2015-08-28 02:14:57 +00:00
Jenkins
7caa98748a Merge "Fix disable_non_metric_meters referencing" 2015-08-28 00:50:48 +00:00
Mehdi Abaakouk
33355eaf68 gnocchi: quote the resource_id in url
Some resource_id looks like "812c9916-83f7-460b-8b27-ab5a2ba98275/foobar".

This change uses urllib.quote to encode them when they are used into
a url.

Related-bug: #1480346
Change-Id: If19b0aec5f89e031b886b7f1e2233c0a88d1b4b1
2015-08-28 00:14:18 +02:00
Jenkins
89e427d9e0 Merge "Fix description for "Incorrect spelling of a word"" 2015-08-27 21:01:24 +00:00
gordon chung
12c9668c67 fix metadata for compute cpu notifications
currently,the cpu notifications has extremely verbose metadata which
is list of all metrics in message. ie, each meter has metadata like:

  u'metrics': [
       {'timestamp': u'2013-07-29T06:51:34.472416',
        'name': 'cpu.frequency', 'value': 1600,
        'source': 'libvirt.LibvirtDriver'},

       ...

       {'timestamp': u'2013-07-29T06:51:34.472416',
        'name': 'cpu.user.time', 'value': 17421440000000,
        'source': 'libvirt.LibvirtDriver'}],

    ...
    }

the original metadata pre-declarative notifications is:

    {'timestamp': u'2013-07-29T06:51:34.472416',
     'name': 'cpu.frequency',
     'value': 1600,
     'source': 'libvirt.LibvirtDriver',
     'event_type': 'compute.metrics.update',
     'host': <publisher_id>}

the proposed metadata removes all data already included in meter:

    {'source': 'libvirt.LibvirtDriver',
     'event_type': 'compute.metrics.update',
     'host': <publisher_id>}

Change-Id: I2c8d4ba7768cba378b931acbfe84a29549da7f8b
2015-08-27 16:52:29 -04:00
gordon chung
d836f8f54a support custom metadata
this patch allows us to define metadata rather than blindly capturing
entire payload.

Change-Id: Id445f7f6d1231f8fdfddb43389783142190db662
2015-08-27 16:12:51 -04:00
Pradeep Kilambi
e8929f3e18 Move profiler meters to yaml
Partially-Implements: blueprint declarative-notifications

Change-Id: Ife5aabce76083b98494a64a4987b2e77718909b3
2015-08-27 16:00:40 -04:00
Jenkins
8d5e86e188 Merge "Create conf directory during devstack install phase" 2015-08-27 19:45:03 +00:00
Divya
52235d0748 Control Events RBAC from policy.json
The ceilometer events RBAC is currently hard-coded so
that only an admin user can view events.The end-user
should be able to customize who should be able to view
events rather than hard-coding the control to admins.
This changeset adds two new rules into the policy.json
so that RBAC for events index and show methods can be
configured using the policy.json file.

DocImpact

Change-Id: I7bf4d385b9ee8fa8f1097b6400cbbc4135f2a9b6
blueprint: events-rbac
2015-08-27 20:19:23 +02:00
Divya
d79593c943 Events RBAC needs scoped token
The role based access control of events api needs project-
scoped tokens to process the request. The project information
has to be available in the request header to be able to filter
events based on the project.In addition to the project, the user
information is also mandated as part of this change. The request
fails with a 403 if this information is not available in the
request.

Change-Id: I55e3d8eaf278024e89e43e36a9dbb92b1c432646
blueprint: events-rbac
2015-08-27 20:18:22 +02:00
gordon chung
988a573f7f make telemetry sample payloads dictionaries
this patch makes the payload we generate via polling agents and
api a dictionary. this gives us more flexibility to include metadata
on how the data was generated that we don't necessary want to
include in sample metadata.

Change-Id: I1350c78c52fad8111241b3f77698fef6c4aa77a9
Closes-Bug: #1484937
2015-08-27 12:19:56 -04:00
Jenkins
f4df5cb3f5 Merge "remove jsonpath-rw requirement" 2015-08-27 16:17:59 +00:00
liu-sheng
c197f4793d Fix requeue process on event handling error
The NotificationDispatcher in oslo_messaging is able to requeue
notifications which cannot be successfully handled by endpoint. That need
endpoint return a 'requeue' if exception happened on handling. In
EventsNotificationEndpoint of ceilometer, the return value was always
None.

Change-Id: Iab10ca06766e1454d1d2338e72087b6bca7590b6
Closes-Bug: #1489431
2015-08-27 20:07:14 +08:00
gordon chung
2e4b807131 allow configurable pipeline partitioning
existing code will create queues from the cartesian product of the
set of pipelines and the set of agents. these queues are identified
by pipe_type-pipe_name-agent_id. the problem arises where if an agent
dies, any messages left on it's queue will be lost as agent_id's tend
to be unique.

this patch changes the logic so rather than using the number of agents
as the set we build our cartesian product from, we use a set of numbers
of a configurable size. by default, it will be a set of 10 so that queues
are created from the product set of the set of pipelines and the set
{0, 1, 2, 3, ... 9}. similarly, the pipelines queues will be identified
by pipe_type-pipe_name-x, where x is value from numerical set. this
way, if an agent dies, the queues are still known and can be
redistributed and processed among the active agents.

Change-Id: I35e3afdd2c9f97ab5bc6bb89531bd20308c5f8c0
Closes-Bug: #1486754
2015-08-27 10:56:47 +00:00