52 Commits

Author SHA1 Message Date
Ryan Brandt
19592e7953 Add monasca-setup options max_buffer_size and backlog_send_rate
Change-Id: I4391cf7b9e67d8195191694f3317b8cf3c326acc
2016-09-15 16:06:43 -06:00
Jenkins
bd0abce582 Merge "Allow limiting the agent buffering on measurements instead of messages" 2016-09-15 21:12:04 +00:00
Ryan Brandt
443f2b1111 Allow limiting the agent buffering on measurements instead of messages
Size of messages may vary based on the number of metrics the
agent is generating, limiting by measurements will be more stable.

Store messages as json strings when queuing to save space

Change-Id: If6dc0a272a26ac951a5d8823dafd32cab69a3981
2016-09-14 13:24:45 -06:00
Kaiyan Sheng
224c56bbcf HttpCheck can't detect changes if only change search_pattern
http check is not detecting changes when search_pattern was changed
but url keeps the same.
Add unit test for modify_config

Change-Id: I0762e963429ec9e5c7df0d9aef7e9ef1b9b57626
2016-09-13 10:55:44 -06:00
Craig Bryant
68d6a7a0f2 Make plugin collection time warning value configurable
Allow the value of sub_collection_warn to be configurable using
monasca_setup. However, use the name plugin_collect_time_warn
as the configuration option since that is easier to understand

Change-Id: I769ec9dd1dcfb1fba5a6da9b59cf03376707c4a6
2016-06-28 16:56:43 -06:00
Craig Bryant
0a3701fffd Forgot to change one place to pool_full_max_retries
This failure broke the devstack tests

Change-Id: If54bc5438c1419f51da82029f75fb8766181c26d
2016-06-27 14:55:18 -06:00
Jenkins
0e21ce3e16 Merge "Use multiple threads to run the plugins" 2016-06-27 19:29:28 +00:00
Craig Bryant
3248ea503e Use multiple threads to run the plugins
This is to handle the case of long running plugins that
added together can exceed the collection frequency. Running
the plugins on multiple threads should bring the collection
time down close to the longest running plugin

A Thread pool is created and used if the new parameter
collector_threads is greater than 1

Due to the Global Interpreter Lock, this will only work well
if the checks are I/O bound which seems to be the case so far

Changed the logging messages to reference plugins instead of
checks to match the usage in docs/Agent.md

Plugins are run in reverse order of last collection time.
Plugins that are still running at the end of a collection cycle
are skipped in the next cycle. If all of the pool threads are
blocked too long by plugins, the collector will exit

Plugins are always run in the thread pool even if pool_size is
1 so that the other changes above still work

Change-Id: Ib50f7fe50a26c949a0066baa3d73ff4a25a8de84
2016-06-27 09:00:47 -06:00
Jenkins
031f9c198f Merge "add --skip_detection_plugins arg. for skipping listed detection plugins" 2016-06-24 20:30:00 +00:00
Joachim Barheine
e9d3766f9b add --skip_detection_plugins arg. for skipping listed detection plugins
Change-Id: I813b8e3aba1277b321a11db97ce2aef6a9e960b4
2016-06-14 13:23:07 +00:00
Tomasz Trębski
ebad10d738 Use both user and group for files agent creates
Use user that monasca-agent run as for creating
plugins configuration files as well as agent.yaml
instead of 'root' user.

Change-Id: I34691607a983b5b84a825f2ba2a6ce595a8d2c4d
2016-06-03 12:59:27 +02:00
Kaiyan Sheng
7ae485a7d1 Remove one check based on args in monasca-setup
When using monasca-setup -r -d <plugin name>, if there is no -a with
arguments, all the checks with built_by=<plugin name> will be removed.

If there are input arguments passed with -a <args>, remove this one specific
check if it 100% matches the input args.

Change-Id: I7c78d44770a1619d0302e698a8c06493bfd296b4
2016-04-05 12:15:25 -06:00
Kaiyan Sheng
fc66b0bc24 Fix Agent Restarting Issue with Changing is to ==
== is for value equality
"is" is for reference equality
During testing in padawan, agent was always restarting
even there were no changes and turned out misusing "is"
was the problem.

Change-Id: Ibb63c6675974259981bb92c3aa17945b25e631c3
2016-03-29 09:16:15 -06:00
Kaiyan Sheng
36e64968c1 Add collector_restart_interval in agent.yaml
Add collector_restart_interval in agent.yaml
Fix a table format in agent documentation
Add boundary check for the value of collector_restart_interval
Delete --collector_restart_interval and --sub_collection_warn
from monasca-setup

Change-Id: I909e392ed04bc5b2aebc5875860dd6725a00711f
2016-02-25 17:35:43 -07:00
Kaiyan Sheng
589bdb9551 Detect Endpoint Changes in Http Check
In monasca agent, detect protocol changes on urls in http check.
If the protocal changed, remove the old instance in http_check.yaml and
merge in the new one.

Change-Id: I16a1ee10cfad0bcdd624328cf4a54a34a8038f36
2016-02-18 13:45:58 -07:00
Kaiyan Sheng
85c24f10c3 Restart collector in agent every 24 hours
Restart the collector every 24 hours by default will
prevent the collection time growing over 15 seconds

Change-Id: I8e26ba703d13adaa685e397e9dbec749b2ed843d
2016-02-05 14:24:53 -07:00
Michael James Hoppal
b2211aa0eb Add HPE copyright
Change-Id: I46efbc9be90a8415e60f3db76656e7db3d7bcb54
2016-02-02 10:34:10 -07:00
Kaiyan Sheng
dc16d7b0ef Display collection time for each check
Add collection time in millisecond for each check in log file (DEBUG).
If the collection time for a single check is larger than the threshold value
for warning (5second as default), a warning message will be created in log file.

Change-Id: Ieb4949fbdd6b3db79de13b52abe2a367afceaa9e
2016-01-14 09:25:49 -07:00
Ben Motz
ebe05d564d Add setup option to skip all service/user config
In addition to configuring monitoring plugins, the monasca-setup code
currently attempts to automatically configure and start users and services,
which may not be desirable if other mechanisms are used for this, so add
"--install_plugins_only" option to disable all changes to user and service
configuration.

Change-Id: I79d2f9dc8ac17895918620e7a062441bdde93477
2015-10-08 10:15:05 +01:00
Shinya Kawabata
c29232a263 Fix typo
Change-Id: I420f443efe9aa6d3b09f261def20df3209c321c6
2015-09-24 16:00:56 +09:00
Joe Keen
1ded0b7b1c Fixing flake8 errors now that tox is working again
Change-Id: Id34991711c57f20922f003928008143032723af3
2015-07-30 14:47:02 -06:00
Tim Kuhlman
afb1d280e5 Added the ability to remove configuration and did various cleanup.
The ability to remove config is useful when calling monasca-setup via
Ansible.

Various refactoring for the main method to make things cleaner.

Fixed overwrite.

Change-Id: I37805e3ef68d794f7cba8fc40e7156e67118b556
2015-07-17 15:00:12 -06:00
Jenkins
462ee7a94a Merge "Added dry run mode to monasca-setup" 2015-07-16 20:48:14 +00:00
Tim Kuhlman
4514d629f3 Added dry run mode to monasca-setup
Change-Id: I5eb1c1ada8051365de0a88207230bd36c4073d3c
2015-07-14 12:52:56 -06:00
Joachim Barheine
36f3582da8 add support for (user-) domain_id/name
Closes-Bug: 1471882
Change-Id: I8574fa3d393dc7a2c9d0470cbf7b9e819a3be0cc
2015-07-08 16:09:43 +00:00
Michael James Hoppal
34386f531f Making sure ntp does not point to localhost
Change-Id: I85a9d26509dff3298024d79d388eabb110530129
2015-06-26 14:24:56 -06:00
Michael James Hoppal
cd4b36deb9 Lower default collection time from 60 seconds to 30
Change-Id: Ia75f871bdebc520995bbd73f53b921d2e42454ff
2015-06-25 13:12:15 -06:00
Tim Kuhlman
fbed9a2177 Misc doc fixes
Change-Id: Iec6fdb792fccd2e92456eba3ecc7e0b9679bc06b
2015-05-29 09:26:36 -06:00
Tim Kuhlman
7cee317a17 Change the default log level to WARN
Change-Id: I9ffe7bcece3d416b3420447bdc464e4d1cb5b5c3
2015-05-27 10:49:53 -06:00
Tim Kuhlman
7cb4c717d4 Add dimensions support to the argument based detection plugins
Change-Id: Icf930ee00c32814494911c14851c44b47648c6c0
2015-05-21 16:40:36 -06:00
Tim Kuhlman
7bdd702aad Warn when detection fails for a specificly called out plugin
Change-Id: I48f1f881a2de10046df7838d9d95d40b977e3182
2015-05-19 16:08:22 +00:00
Tim Kuhlman
24ef0598f9 Added host alive and http_check plugins and detection_args.
Change-Id: I02f525b51aeddd540472c6a8087bc2e9d6620e1a
2015-05-14 09:52:04 -06:00
Craig Bryant
b0cc0ccb14 Change the default agent user to mon-agent
Changed some places to templates to make it possible to change agent
user using monasca_setup. There is one place that it is still
hard coded

The deb creation now uses a hardcoded mon-agent user instead of
monasca-agent

Change-Id: I8f2ee8903289d74c0397ccf911701c89e7a1b691
2015-05-06 11:41:28 -06:00
Tim Kuhlman
9e6169522a Make sure the gid is detected after it is created.
Change-Id: Iafe76b17b29d33af121bbc430ceffd7ce278d8a3
2015-05-05 09:04:28 -06:00
Tim Kuhlman
1a73f3425b Modify the agent and setup to accept custom plugins
For the agent I just modified to the config to use the new standard location for custom checks.
For monasca-setup I added automatic plugin detection.
Fixed config compare so order changes don't register as a config change.

Change-Id: I7ab17c894bb0496e30d7f5aa5a1b1cf9684bdf87
2015-05-04 11:37:54 -06:00
Tim Kuhlman
a3a77ba227 Updated merge that avoid duplicating named instances in a config.
Skip config writing and restart if there are no changes to the config.
Added names to various config instances to aid in repeat runs of setup.

Change-Id: Idd29f8abe29b4889868a9044116cbdcdfdba6262
2015-05-01 11:31:16 -06:00
Dexter Fryar
3d1dc2c6d0 Docs to describe customizations. Added readthedocs config
Built on changes from Dexter
Updated toc, added internal dimensions, general formatting

Change-Id: Ica56c43fbf98fcd99792ab91e47dccf46ec1b9aa
2015-04-28 15:03:40 -06:00
Tim Kuhlman
d1a71e06b1 Added the ability to just detect/setup a subset of plugins
Change-Id: Ied595c19fb00ce944f4757db3f4b30a9bd21f7d4
2015-04-24 14:59:56 -06:00
Tim Kuhlman
5f5c77847d Added a systemd service provider
Change-Id: I2e98f120bd937131b739778f5737f068976a0a14
2015-04-21 15:05:46 -06:00
Tim Kuhlman
2b5fc32f2b Default the agent to a 1 minute collection time.
Change-Id: I1be628ad6076069784b58224b3044ce3a23973fc
2015-03-30 10:41:43 -06:00
Dexter Fryar
1c4f93f7f1 Incorrect platform check fix
bugs.launchpad.net/monasca/+bug/1436449

Closes-Bug: #1436449

Change-Id: Ia90e6a2e51c085afcbf9c678f675a0e7306516e3
2015-03-25 16:30:49 -05:00
Tim Kuhlman
b5613c5e70 Add the --system_only option to monasca-setup.
Also stop rabbitmq from doing a partial setup with guest/guest auth

Change-Id: I15802cb17dc53d1ae1ff90a4f861dcc275406ad9
2015-03-20 14:05:29 -06:00
Tim Kuhlman
5e2236f7e7 Changes to restore python 2.6 support
Change-Id: Ibccdb49e1d922c5855df2a0864eff35984f399ef
2015-03-06 08:20:20 -07:00
Gary Hessler
f9fd9c5a30 Converted the agent config file to yaml to match the plugin config files
Converted the config file to yaml and updated the config processing logic to match.
Cleaned up some deprecated config item processing.

Change-Id: I8c5d9ad841b22d20825608ac9f2c82d8011b04f0
2015-03-02 18:36:15 -07:00
Gary Hessler
b1de7db1f5 Removing subprocess call from system metrics classes
Changed the IO, Disk and Network classes to not use a subprocess.
They now use psutil to get the metrics.  Also, changed the linux
system metrics classes to subclass the AgentCheck class instead
of the old-style Check class.  Added additional configuration
and changed monasca-setup to support that. Fixed some Python
2.6 incompatible string formatting issues.

Change-Id: I1f8b65bf48e48e2c598aa4950c194fbae2f9e337
2015-02-25 10:58:43 -07:00
Tim Kuhlman
721b013d11 Various fixes to make it easier to run in diverse environments
Removed all the lsb helper utils from the sysv init script
Added a better error message for template missing.
Added the ability to skip the definition of the monasca_api_url
Made check frequency configurable and added an option to monasca setup
Also set the forwarder flush interval to be calculated based on check
frequency.
Modified monasca-setup so it can work with non-standard paths.

Change-Id: If8916e17bf42ffb804fa6c79166149a9fd33c553
2015-02-20 16:47:31 -07:00
gary-hessler
51b4f9b221 Update all plugins to handle dimensions correctly
Updated the plugin code to handle the dimensions from the agent config,
the plugin config and dimensions defined in the code correctly.
Also removed the old-style plugin configuration code from several of
the plugins and added a command-line option to monasca-setup to pass
the log_level if desired instead of hard-coding it in the config.
This will make it easier to test in debug mode. Fixed an issue
with the Swift detection plugin.

Change-Id: I679457e5e1f302b0cdc87b0f0f799a42a9fa0ba4
2015-02-13 16:36:46 -07:00
Tim Kuhlman
2a4e1e0080 Added swift and ntp checks and more automatic detection.
Added a bunch of swift diagnostic commands.

Added postfix checking back in but warned when root access is not
granted.

Ported over ntp check from the datadog agent and added detection.

Moved detection plugin list out of the main of monasca-setup just so it
looks tidy.

Change-Id: I9a76d0cc009545d30df60c17f65a1db6e8329b63
2015-02-12 12:07:25 -07:00
Michael James Hoppal
a8929debda Modifications to accommodate for python 2.6
Change-Id: If959b8a9c7073cbbc97a9712945754116f026c5c
2015-02-06 11:18:09 -07:00
gary-hessler
42f7c5ac16 Adding apache web server monitoring for OpenStack
Change-Id: I0c50e1d2825dd6324ef8ebd4b3a1bc32e1b2a3d5
2015-02-03 10:07:58 -07:00