326 Commits

Author SHA1 Message Date
Andreas Jaeger
fd530b4294 Remove -U from pip install
'pip install -U' ugrades specified packages, this is not necessary
since we use constraints, remove the parameter '-U' from the line.

With tools/tox_install.sh - which a previous change of mine removed -
the -U was not harmful, but with the current set up, it might cause
upgrades, so remove it.

Change-Id: Iea38327f48caf8c060479dfa4f42f3124a87fee8
8.1.0
2017-12-02 19:25:24 +01:00
Andreas Jaeger
81d355ea32 Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: Id5b1b0d8815c8324c53eca0fb9a32b1495617eba
2017-12-01 14:52:17 +01:00
OpenStack Proposal Bot
701438b9dd Updated from global requirements
Depends-On: Ifb85d41324ac997e30535f04e291244d6d02ecc6
Change-Id: Ia9d311c7e30d51888f2e22cb950b39913b300f90
2017-11-29 17:55:44 +00:00
Emilien Macchi
7ad5c4dd05 Migrate os-collect-config to zuul v3 jobs
Define the zuul v3 layout for jobs that we want to execute.

Change-Id: I9d1c80f8f0acfcbd030e1fabf75941515c2f80f8
2017-11-17 20:11:36 +00:00
OpenStack Proposal Bot
f7f49448a5 Updated from global requirements
Change-Id: I6404391dc149238d3f082d5e9add192be94cf41b
8.0.0
2017-09-11 21:46:25 +00:00
OpenStack Proposal Bot
e24c7de257 Updated from global requirements
Change-Id: I4e60acc1dcbaef39d56a8070d82ec141aeb1c206
2017-08-24 05:58:01 +00:00
OpenStack Proposal Bot
315be525be Updated from global requirements
Change-Id: I94606b6b043449a0640b01c55f427922df694597
2017-08-18 04:49:31 +00:00
OpenStack Proposal Bot
5be5fe782f Updated from global requirements
Change-Id: Ia690b022f7ad7850fd8ba80767608d090021dfef
7.1.0
2017-06-27 12:19:34 +00:00
OpenStack Proposal Bot
9d7ae9439b Updated from global requirements
Change-Id: I637b2bfe17842ab5fbcbe8369cd04cfbd38b8ecb
2017-06-15 16:32:17 +00:00
OpenStack Proposal Bot
5870ed691d Updated from global requirements
Change-Id: Ief06adcb7379a2b5e9fc3f2c60710e6e98f6403d
2017-06-10 21:46:13 +00:00
OpenStack Proposal Bot
53b551bf8a Updated from global requirements
Change-Id: Ife897862b799e983c7c401ffd7e2b6c5f477dc93
2017-06-02 02:34:25 +00:00
OpenStack Proposal Bot
bf91feb198 Updated from global requirements
Change-Id: Iac0df7f86abaf3e4eaaa2c3d21a439c57ca0036b
7.0.0
2017-05-17 13:47:21 +00:00
OpenStack Proposal Bot
b5723900fd Updated from global requirements
Change-Id: I9dde7726afc877c4918406ba210982b0e17e3fbb
2017-05-15 00:52:19 +00:00
Jenkins
7b8d598d1c Merge "Use universal_newlines with subprocess.check_output" 2017-04-13 05:34:40 +00:00
OpenStack Proposal Bot
66a2dd54e3 Updated from global requirements
Change-Id: I1160555be0f8c84ee4776ca3dafe6dc3b1800f23
2017-04-12 04:19:34 +00:00
Lukas Bezdicka
8566ad549c Don't fail on non-valid json in cache files
Resolves: rhbz#1438096
Closes-Bug: #1678328

Change-Id: Iee1bb4a7769568cfd3bad8b23c9dff440679ac16
Co-Authored-By: Alex Schultz <aschultz@redhat.com>
2017-04-11 07:58:07 -06:00
Corey O'Brien
cb171db770 Use universal_newlines with subprocess.check_output
Python 3 will complain when trying to split a byte-string using a string
so this change enables universal_newlines to make the split work in both
python 2 and 3. Also add a py3 tox environment.

Change-Id: I9af74b6c6cc9cfd0c973c85be04791aa6dc0acd4
2017-04-10 09:00:31 -04:00
Alex Schultz
0762fcb0b5 Add splay option to offset polling intervals
When the os-collect-config process is started on multiple systems at the
same time, the polling intervals can line up to cause performance
problems against the configuration source. To reduce the impact, this
change adds a splay option to allow the operator to configure a random
delay prior to the polling to attempt to offset the polling
syncronization.

Change-Id: I1a8be3345d783da9014eca7ea26da19d57e767c0
Closes-Bug: #1677314
7.0.0.0b1
2017-03-30 12:44:21 -06:00
OpenStack Proposal Bot
6ee0c59066 Updated from global requirements
Change-Id: I453410624fd05840d0f9dd0b9b461045b1f29c8f
2017-03-13 19:39:27 +00:00
Jenkins
0e2a8ab43a Merge "Use pop for checking messages with websocket" 2017-03-11 02:10:01 +00:00
Jenkins
9d23b907fa Merge "Add Constraints support" 2017-03-09 22:25:44 +00:00
OpenStack Proposal Bot
c3d54ff823 Updated from global requirements
Change-Id: I13724b3cb5733013516e7f16a3c28ec9f669acc0
2017-03-06 01:16:53 +00:00
Jenkins
34624661aa Merge "Allow setting minimum polling interval for collect" 2017-03-06 01:12:21 +00:00
ricolin
d677da173b Allow setting minimum polling interval for collect
Add `min_polling_interval` in `[default]` config.
This config option allow setting minimum polling interval,
default is 1 second.
Closes-Bug: #1656893
Depends-On: If863d2e190a4e1701ebf1fcb99a783075cda2556
Change-Id: I213afabeeb63f143ce359a3495a6d88b9b74ef4d
2017-03-02 03:17:07 +00:00
Tony Breeds
ec8b2576da Update to a newer hacking library
The older hacking library has a cap on pbr <2.0, with the recent 2.0.0
release of PBR it's causing failures in the pep8 job.

hacking isn't kept in sync via the typical proposal-bot updates.

Do it manually to clear the gate issue.

pep8 runtests: commands[0] | flake8
./os_collect_config/merger.py:45:22: W292 no newline at end of file
    return final_list

Change-Id: If863d2e190a4e1701ebf1fcb99a783075cda2556
Related-Bug: #1668848
2017-03-02 12:43:37 +11:00
gecong1973
8d038af700 Delete H803 from ignore list
The H803 rule have been removed from hacking since v0.10.0.
we can remove H803 from the ignore list.

Change-Id: Ib63f8a57db170bac08fecd2924e5a042d909b84e
2017-02-03 16:13:06 +08:00
Jenkins
6ddcd55394 Merge "Only re-exec on config change or HUP" 2017-01-31 05:21:13 +00:00
Thomas Herve
432abd47ec Use pop for checking messages with websocket
I1c94ad82c3b20cc331045c19d4a9987a701b081b fixed the issue in Zaqar where
the pop argument was wrongly named. We can now use it in the zaqar
collector to retrieve messages over websocket.

Change-Id: I0ce894b0eef7018a37db9ef9d1b43d8d20629bf0
2017-01-09 10:58:23 +01:00
Tony Breeds
37662029e2 Add Constraints support
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.

This change adds constraints support by also adding a helper script to
edit the constraints to remove os-collect-config.

Change-Id: I2a0983f41372b52e6d4e6a85cca1c51f980af36c
2016-12-21 12:07:17 +11:00
OpenStack Proposal Bot
cb846967f7 Updated from global requirements
Change-Id: Ia944bd48e00f174d979dbae2d8aec3727a365c87
6.0.0 6.0.0.0rc1
2016-12-02 17:16:19 +00:00
Jenkins
bf7cc362d3 Merge "Show team and repo badges on README" 6.0.0.0b2 2016-12-01 23:17:17 +00:00
Dan Prince
b6364df3e5 Add support for Zaqar websockets
This patch adds a new zaqar.use_websockets option which if set to True
in the config file will configure the Zaqar collector to use
the websocket transport instead of wsgi. This can be more efficient
where you want to avoid the continuous polling of o-c-c and instead
just listen on the websocket that is subscribed to a queue.

Like other collectors each iteration creates a new socket object. This
allows us to use the normal re-exec logic in o-c-c and thus gives
the option to re-configure the agent in the future to use other types
of collectors. We could (optionally) look into a higher level option in
the future that would allow o-c-c to avoid re-exec'ing and thus re-use
the same websocket for multiple sets of metadata.

Depends-On: Ia2a8deb599252d8308e44d595eb2bf443999aaad
Change-Id: Id5c7ed590df776844b6c7961eb40f89206cd24e0
2016-11-30 21:12:47 -05:00
Jenkins
358b73e239 Merge "Remove discover from test-requirements" 2016-11-30 17:12:39 +00:00
Flavio Percoco
e6fbb4cd98 Show team and repo badges on README
This patch adds the team's and repository's badges to the README file.
The motivation behind this is to communicate the project status and
features at first glance.

For more information about this effort, please read this email thread:

http://lists.openstack.org/pipermail/openstack-dev/2016-October/105562.html

To see an example of how this would look like check:

b'https://gist.github.com/e08d834d7d0be70f74da177780f25f4a\n'

Change-Id: I5b7c82f942547245a830db0e80c1e12069f21cb9
2016-11-25 14:19:52 +01:00
Jenkins
47bd8426fa Merge "ec2 collector attempt config-drive before metadata server" 2016-11-21 14:22:51 +00:00
Swapnil Kulkarni (coolsvap)
b7c5aedac4 Remove discover from test-requirements
It's only needed for python < 2.7 which is not supported

Change-Id: I654868000a614bc6a939c99cbca1e1998a6fa9ee
2016-11-16 10:41:07 +05:30
Jenkins
e8e2f3d5f2 Merge "Treat ec2 collector data as immutable" 6.0.0.0b1 2016-11-14 19:39:39 +00:00
OpenStack Proposal Bot
c18d93de7b Updated from global requirements
Change-Id: I7aa15e52fb726115d06371f54beffad019b3ddec
2016-11-09 04:22:23 +00:00
Jenkins
dc48fed4ed Merge "Use TEST-NET-1 for unit tests, not 127.0.0.1" 2016-10-25 12:43:45 +00:00
OpenStack Proposal Bot
51ecf22970 Updated from global requirements
Change-Id: I4a0d55b82b379ff28d63884bd77bd929fd02f078
2016-10-19 03:57:01 +00:00
Steve Baker
76975f04e5 ec2 collector attempt config-drive before metadata server
This change will first attempt to get ec2 metadata from a v2 config
drive before falling back to the nova metadata server if no config
drive is detected.

Config drive is enabled in overcloud nodes, so using this for
os-collect-config will allow the undercloud nova metadata server to be
disabled.

V2 of the config-drive was released in OpenStack Folsom, so a decision
was made to not support V1 discovery at all. This makes the block
device candidate selection much simpler than the cloud-init one[1].

[1] https://git.launchpad.net/cloud-init/tree/cloudinit/sources/DataSourceConfigDrive.py#n219

Change-Id: I2aec0544fe40c2e901aee38c79bb838a627d20da
Closes-Bug: #1619074
2016-10-13 09:38:29 +13:00
Steve Baker
a950edb320 Treat ec2 collector data as immutable
Currently the ec2 collector polls the nova metadata service every
$polling_period even though the data is not expected to change and no
known config actions have been written to respond to changes in these
values.

With larger overclouds, this metadata polling will cause noticeable
load on the undercloud, especially nova-api and neutron (for port
lookups).

This change will check for data in /var/lib/os-collect-config/ec2.json
and simply return that if it exists.

Change-Id: I84156f8b005b319fccf8ac17af8943aefb0a8f6d
Closes-Bug: #1619072
2016-10-12 12:16:59 +13:00
Steve Baker
61ff6aaec7 Use TEST-NET-1 for unit tests, not 127.0.0.1
Currently if http requests are not correctly mocked, behaviour of the
unit tests will depend on what happens to be running on localhost.

Jobs like gate-os-collect-config-python27-ubuntu-xenial are currently
broken because discover requests are being made to localhost and
something has change in the environment to cause test failure.

This change does the following:
- replace unit tests addresses from 127.0.0.1 to 192.0.2.1 (TEST-NET-1)
- fix mocking of tests which now stall because 192.0.2.1 isn't accessable
- change the mocking approach to keystoneclient.discover.Discover to
  a simple stub class

Change-Id: I6d9450d6854b9f50dac0f55f54f1ea1d579a27ac
2016-10-12 12:16:52 +13:00
John Trowbridge
2980adb21d Open os-collect-config for Ocata
To avoid pushing an artificial alpha tag, following PBR semver keyword
bumps major version. See http://docs.openstack.org/developer/pbr/#version

Change-Id: I1e60356351ccbad772a37ecb6547d21bfd8f5d29
Sem-Ver: api-break
2016-10-05 09:54:41 -04:00
James Slagle
9ee07c96c2 Revert "Treat ec2 collector data as immutable"
All CI jobs are timing out currently during NetworkDeployment
because 99-refresh-completed is not signaling to Heat due to
instance-id being detected as null by os-apply-config.

I suspect this patch is causing the issue.

This reverts commit 8717436db99a84588532e792f8c9ea909f3e3628.

Change-Id: Id0d122d7e855cd828a0742dabc5f7089fb6a913c
Closes-Bug: 1623754
5.0.0 5.0.0.0rc2
2016-09-14 16:02:22 +00:00
Steve Baker
8717436db9 Treat ec2 collector data as immutable
Currently the ec2 collector polls the nova metadata service every
$polling_period even though the data is not expected to change and no
known config actions have been written to respond to changes in these
values.

With larger overclouds, this metadata polling will cause noticeable
load on the undercloud, especially nova-api and neutron (for port
lookups).

This change allows collect calls to raise a SourceAlreadyCollected
exception if the data for this collector never changes. The ec2
collector raises this if its data has already been cached to
/var/lib/os-collect-config/ec2.json

Change-Id: Ib7df590601132857690c8ab64fe32098a81752d8
Closes-Bug: #1619072
2016-09-01 13:42:11 +12:00
Jenkins
20630c3360 Merge "Merge deployments data for zaqar collectors" 2016-08-30 21:19:47 +00:00
Dan Prince
90cb2721e9 Only re-exec on config change or HUP
There are 2 things at play here. First the logic added in
4cfeb28d12295154f1b57308022283cc2eaab079 is no longer needed
because our initial sleep time is very low and increases
gradually up to the max.

2nd, I'm proposing that we avoid reexecing unless the config
file actually changes. Not re-execing will give us the option
to optimize os-collect-config for some long running collectors
(like zaqar websockets).  Also, Os-collect-config updates would
already be handled by packaging restarts and or other deployment
system changes anyways.

Change-Id: I04b2752d007089f72af42c88c4249c3e11c8346f
2016-07-19 11:57:54 -04:00
Jenkins
c2bdb4f672 Merge "Set a timeout for all data fetches using requests" 5.0.0.0b3 2016-07-18 21:25:57 +00:00
Dan Prince
08ba371994 Merge deployments data for zaqar collectors
This is required so that a the TripleO undercloud can
switch to polling for metadata from a Zaqar rather than Heat or
Swift.

Change-Id: Ieb076185314a7aeadd8a708ab01b48dfbccef006
Closes-bug: 1604098
2016-07-18 14:13:22 -04:00