Commit Graph

281 Commits (9ee07c96c2b0d64485d111aed0af03fa394e9c4d)

Author SHA1 Message Date
James Slagle 9ee07c96c2 Revert "Treat ec2 collector data as immutable"
All CI jobs are timing out currently during NetworkDeployment
because 99-refresh-completed is not signaling to Heat due to
instance-id being detected as null by os-apply-config.

I suspect this patch is causing the issue.

This reverts commit 8717436db9.

Change-Id: Id0d122d7e855cd828a0742dabc5f7089fb6a913c
Closes-Bug: 1623754
2016-09-14 16:02:22 +00:00
Steve Baker 8717436db9 Treat ec2 collector data as immutable
Currently the ec2 collector polls the nova metadata service every
$polling_period even though the data is not expected to change and no
known config actions have been written to respond to changes in these

With larger overclouds, this metadata polling will cause noticeable
load on the undercloud, especially nova-api and neutron (for port

This change allows collect calls to raise a SourceAlreadyCollected
exception if the data for this collector never changes. The ec2
collector raises this if its data has already been cached to

Change-Id: Ib7df590601132857690c8ab64fe32098a81752d8
Closes-Bug: #1619072
2016-09-01 13:42:11 +12:00
Jenkins 20630c3360 Merge "Merge deployments data for zaqar collectors" 2016-08-30 21:19:47 +00:00
Jenkins c2bdb4f672 Merge "Set a timeout for all data fetches using requests" 2016-07-18 21:25:57 +00:00
Dan Prince 08ba371994 Merge deployments data for zaqar collectors
This is required so that a the TripleO undercloud can
switch to polling for metadata from a Zaqar rather than Heat or

Change-Id: Ieb076185314a7aeadd8a708ab01b48dfbccef006
Closes-bug: 1604098
2016-07-18 14:13:22 -04:00
Steve Baker e5518c11c7 Set a timeout for all data fetches using requests
A timeout config value is added for each collector which uses requests
to fetch data, and this value is used for any requests calls.

Without a timeout a request may stall indefinitely and
os-collect-config will stop polling.

A timeout default of 10 seconds is chosen as the default. This is used
for both the connection timeout and the read timeout.

Change-Id: I4ad0065b5a85393105c6385a15653d7204b4f880
Closes-Bug: #1600652
2016-07-11 11:14:57 +12:00
OpenStack Proposal Bot 7be429c33d Updated from global requirements
Change-Id: I6478d00910d360a763f24488b89e0b131b9c1f0f
2016-07-09 03:15:49 +00:00
OpenStack Proposal Bot a88e2733a7 Updated from global requirements
Change-Id: I1edecad4ec48c896c8beb03ccf0360512d1da049
2016-05-30 20:44:38 +00:00
OpenStack Proposal Bot 9a5c1688f2 Updated from global requirements
Change-Id: I3ca384d92e42c1ac6d1d9ff54b4995e873f8dca6
2016-05-30 00:37:57 +00:00
OpenStack Proposal Bot 9b78c81d20 Updated from global requirements
Change-Id: I1e91040e04c50889f595e5e8a47343af46f85ed6
2016-05-14 04:01:02 +00:00
Jenkins 1ed789dbe5 Merge "Switch to oslo.log again" 2016-05-13 22:03:37 +00:00
Ondřej Nový d45a8c5c7f Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:

Change-Id: I8ead8eefb940c3e2e6ec470f06e5288949a9a2bc
2016-04-23 15:19:25 +00:00
Ben Nemec f092ea133f Switch to oslo.log again
We're currently still using unmaintained oslo-incubator code for
our logging, which is bad.  This switches us to oslo.log as
described in [1].


Change-Id: Ibce86ab4ee24eeb55d0de1b0d5ff4ee4ea6ef66f
2016-03-22 15:38:05 +00:00
Ben Nemec ab5c66df45 Revert "Switch to oslo.log"
This isn't quite right and broke on stable/liberty.  Pushing
a revert in case I3c22d77dece399d21ab94783b74990789a1e1481
doesn't actually fix the problem.  We should probably merge
whichever passes first.

This reverts commit 69653318f4.

Change-Id: I9304429f25d28ca756e50b1788e149c5bb46b1d6
2016-03-21 20:50:25 +00:00
Ben Nemec 69653318f4 Switch to oslo.log
The old oslo-incubator log module isn't maintained (and doesn't even
exist anymore), so we don't really want to be using it.  It appears
this was the only incubator module we were actually using, so this
allows us to remove all of the unmaintained incubator code.

Change-Id: Ib4ad3b231360987a1ef4f95b5b5a8b656232efc4
2016-03-17 14:40:46 +00:00
Jenkins b04605e132 Merge "Replace assertEqual(None, *) with assertIsNone in tests" 2016-03-01 22:27:59 +00:00
Andreas Jaeger 54a60ddfdd Remove argparse from requirements
argparse was external in python 2.6 but not anymore, remove it from

This should help with pip 8.0 that gets confused in this situation.
Installation of the external argparse is not needed.

Change-Id: Ib7e74912b36c1b5ccb514e31fac35efeff57378d
2016-01-20 19:18:30 +01:00
Swapnil Kulkarni (coolsvap) 8b28ced627 Replace assertEqual(None, *) with assertIsNone in tests
Replace assertEqual(None, *) with assertIsNone in tests to have
more clear messages in case of failure.

Change-Id: I2cc0d913d83ea56c323475daddb66a0047564b42
2016-01-04 19:41:10 +05:30
OpenStack Proposal Bot ba1a17a1f2 Updated from global requirements
Change-Id: I98ef86f800c2c5872e08db6fa6030ecd7085e463
2015-12-15 18:16:00 +00:00
Dan Prince b63188eb25 shorter sleeps if metadata changes are detected
This patch updates os-collect-config so that the sleep interval
time is shortened if changes are detected. This should decrease
deployment time when using Heat templates which use depends_on
to step through a sequence of software deployment resources.

The new default sleep is set to 1 and increases
by sleep_time *= 2 until it reaches the default
sleep interval again.

Change-Id: I5cbd0956db2abebec876b15bee72b70ec64d5aef
2015-10-21 16:56:21 -04:00
Jenkins b10b844fec Merge "Change ignore-errors to ignore_errors" 2015-09-25 20:49:55 +00:00
Monty Taylor 0e0eca3687 Change ignore-errors to ignore_errors
Needed for coverage 4.0

Change-Id: Ic64cbb3d42ebc605248e97bd525b9ce90e500321
2015-09-21 14:41:46 +00:00
OpenStack Proposal Bot a86bafe438 Updated from global requirements
Change-Id: Ib69d0b27507247ebee3459084fa385ff7b2fe05c
2015-09-17 12:15:44 +00:00
Thomas Herve b13122918c Add support for zaqar transport
This adds a new zaqar collector able to read configuration from a
specific zaqar queue.

blueprint software-config-zaqar
Change-Id: Ie38af7b59e7a1aa370ac7760bb7819e37c2165c3
2015-08-26 11:33:10 +02:00
Jenkins ca089dfcc2 Merge "Merge deployments data for collectors heat, request" 2015-07-02 16:37:30 +00:00
Jenkins 7f2cced83f Merge "Refactor merging out of cfn collector" 2015-07-02 16:25:53 +00:00
Steve Baker acafb1b94a Merge deployments data for collectors heat, request
This is required so that a swift-enabled TripleO undercloud can switch
to polling for metadata from a TempURL rather than heat.

Change-Id: I73ac9e01f85e0c72ce7411e2c61c545322f3dccc
Closes-Bug: #1424913
2015-06-15 09:14:33 +12:00
Steve Baker 4b68dfd53d Refactor merging out of cfn collector
This is a pure refactoring change which moves the os-apply-config
deployment data merging into its own module. This will allow
other collectors (request, heat) to do the same merging.

Partial-Bug: #1424913
Change-Id: Ic78a60e3efebadbb06ebfd262ceb275ca519a3f2
2015-06-15 09:14:29 +12:00
OpenStack Proposal Bot 004869ab5e Updated from global requirements
Change-Id: I9d0e98344c52adcfc73ecbc5afc0d1a6d6ac7e79
2015-06-04 20:04:14 +00:00
Jenkins e80616d15a Merge "Log with info instead of warn when collector not configured" 2015-06-04 17:15:33 +00:00
Jenkins 163030c081 Merge "Drop use of 'oslo' namespace package" 2015-05-29 09:40:47 +00:00
Steve Baker d2c5bd1b0e Log with info instead of warn when collector not configured
Users who are debugging unrelated issues are frequently interpreting
logged warnings as an indication that os-collect-config is misconfigured
when the warning is actually harmless.

This change uses info instead of warn for this logging.

Change-Id: I28e0fb37b603364eea72cdc5299a29259489b820
Closes-Bug: #1437952
2015-05-15 11:54:23 +12:00
Doug Hellmann 4a41f0b3e5 Drop use of 'oslo' namespace package
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.

The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.

Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.

Blueprint: remove-namespace-packages

Change-Id: If51059c31c82d5235e2ae21143911b5561783ca6
2015-05-06 18:45:17 +00:00
Jenkins eefebd7682 Merge "Use non-versioned auth_url for keystone" 2015-05-06 14:27:25 +00:00
OpenStack Proposal Bot 11d69d9d3d Updated from global requirements
Change-Id: I780644b67e532cd82476a1c6ad072c99173eebfa
2015-04-24 13:03:37 +00:00
Yanyan Hu f6ca7618c4 Use non-versioned auth_url for keystone
keystoneclient discover can't get v3 auth_url using v2 auth_url. It
accepts an url without version number. This patch fixes this issue.

Change-Id: I45b4c016bd91afc39d39620920951043bb71df05
Closes-Bug: #1398254
2015-04-24 13:52:24 +02:00
OpenStack Proposal Bot 599fe89cc9 Updated from global requirements
Change-Id: I79d43903aff8d818ff1b583be2f5b46263d3b129
2015-03-21 00:16:39 +00:00
OpenStack Proposal Bot 450188d52e Updated from global requirements
Change-Id: I13c5693efba1114c8b8eaba31af10ab75985ed76
2015-03-05 04:57:02 +00:00
OpenStack Proposal Bot ed95ad3d04 Updated from global requirements
Change-Id: I5a542669fbe25cb83cfaba1f0403035dc051cde5
2015-01-08 18:50:19 +00:00
Endre Karlson c917665af9 Add ca_certificate option for SSL'd api
Change-Id: I12eb9dbbb8bee24e50ae342ffbc7356d4583a973
2014-12-05 13:59:55 +01:00
Jenkins de6d420f40 Merge "Add unit test for heat_local" 2014-11-25 17:53:11 +00:00
OpenStack Proposal Bot 8e603ed3c8 Updated from global requirements
Change-Id: I7503a6a56367f0d01a896441723df4fc3f3a7567
2014-11-20 14:11:12 +00:00
OpenStack Proposal Bot 61cb285d23 Updated from global requirements
Change-Id: I3d2eab22bfd5b35bb6d799b8b924dca147483ed1
2014-11-18 11:36:00 +00:00
Dougal Matthews 4bc37fd50e Bump hacking to 0.9.x series
Change-Id: I474266c647f31f0ad92ec9878fc03eb5a5c97f43
2014-11-13 13:11:00 +00:00
OpenStack Proposal Bot dc26874ab5 Updated from global requirements
Change-Id: I7ac9a3db15d6fff36e626d74fedfb35a5ecd0532
2014-11-10 23:28:59 +00:00
Gregory Haynes 46e950ddd1 Enable local collector by default
This is a useful collector and it seems odd to not have it on by

Change-Id: Ibc3be6ff27c6290ce32bc01d0e62acf3253545c3
2014-10-30 01:26:09 -07:00
Jenkins 72bd2854f6 Merge "Handle missing local collector directory" 2014-10-29 16:13:22 +00:00
Gregory Haynes d05f57b092 Handle missing local collector directory
We currently explode if local collector is enabled with the data
directory missing. Changing this to a warning allows us to have this
turned on by default without greatly inconveniencing users.

Change-Id: I36364ba1a0706a5a2c820eadd526f2ba424ac665
2014-10-28 13:34:48 -07:00
Steve Baker ad475ee927 Collector which does a simple HTTP GET
This change implements a collector which does an HTTP GET via
python requests to fetch the metadata.

It should work with any GET-able URL, however it is designed to
work with Swift TempURLs.

Swift objects are not consistent, so the Last-Modified header is
checked for each poll and metadata is not fetched if the last
modified is not newer than the previous successful poll.

This collector will be enabled for OS::Nova::Server
software_config_transport: POLL_TEMP_URL which is available
in the Juno release of Heat. Using POLL_TEMP_URL will result
in no metadata polling load on heat, which has historically been
an issue with tripleo scalability.

Change-Id: I22155c22bdcc3c81a5e945ca5436a8f29f196528
2014-10-17 09:40:23 +13:00
Jenkins fa3d4e34e8 Merge "Modify gitignore to ignore cover" 2014-10-16 12:48:51 +00:00