Commit Graph

20 Commits (master)

Author SHA1 Message Date
Lukas Bezdicka 8566ad549c Don't fail on non-valid json in cache files
Resolves: rhbz#1438096
Closes-Bug: #1678328

Change-Id: Iee1bb4a7769568cfd3bad8b23c9dff440679ac16
Co-Authored-By: Alex Schultz <aschultz@redhat.com>
6 years ago
Steve Baker 76975f04e5 ec2 collector attempt config-drive before metadata server
This change will first attempt to get ec2 metadata from a v2 config
drive before falling back to the nova metadata server if no config
drive is detected.

Config drive is enabled in overcloud nodes, so using this for
os-collect-config will allow the undercloud nova metadata server to be
disabled.

V2 of the config-drive was released in OpenStack Folsom, so a decision
was made to not support V1 discovery at all. This makes the block
device candidate selection much simpler than the cloud-init one[1].

[1] https://git.launchpad.net/cloud-init/tree/cloudinit/sources/DataSourceConfigDrive.py#n219

Change-Id: I2aec0544fe40c2e901aee38c79bb838a627d20da
Closes-Bug: #1619074
7 years ago
Steve Baker a950edb320 Treat ec2 collector data as immutable
Currently the ec2 collector polls the nova metadata service every
$polling_period even though the data is not expected to change and no
known config actions have been written to respond to changes in these
values.

With larger overclouds, this metadata polling will cause noticeable
load on the undercloud, especially nova-api and neutron (for port
lookups).

This change will check for data in /var/lib/os-collect-config/ec2.json
and simply return that if it exists.

Change-Id: I84156f8b005b319fccf8ac17af8943aefb0a8f6d
Closes-Bug: #1619072
7 years ago
James Slagle 9ee07c96c2 Revert "Treat ec2 collector data as immutable"
All CI jobs are timing out currently during NetworkDeployment
because 99-refresh-completed is not signaling to Heat due to
instance-id being detected as null by os-apply-config.

I suspect this patch is causing the issue.

This reverts commit 8717436db9.

Change-Id: Id0d122d7e855cd828a0742dabc5f7089fb6a913c
Closes-Bug: 1623754
7 years ago
Steve Baker 8717436db9 Treat ec2 collector data as immutable
Currently the ec2 collector polls the nova metadata service every
$polling_period even though the data is not expected to change and no
known config actions have been written to respond to changes in these
values.

With larger overclouds, this metadata polling will cause noticeable
load on the undercloud, especially nova-api and neutron (for port
lookups).

This change allows collect calls to raise a SourceAlreadyCollected
exception if the data for this collector never changes. The ec2
collector raises this if its data has already been cached to
/var/lib/os-collect-config/ec2.json

Change-Id: Ib7df590601132857690c8ab64fe32098a81752d8
Closes-Bug: #1619072
7 years ago
Steve Baker e5518c11c7 Set a timeout for all data fetches using requests
A timeout config value is added for each collector which uses requests
to fetch data, and this value is used for any requests calls.

Without a timeout a request may stall indefinitely and
os-collect-config will stop polling.

A timeout default of 10 seconds is chosen as the default. This is used
for both the connection timeout and the read timeout.

Change-Id: I4ad0065b5a85393105c6385a15653d7204b4f880
Closes-Bug: #1600652
7 years ago
Ben Nemec f092ea133f Switch to oslo.log again
We're currently still using unmaintained oslo-incubator code for
our logging, which is bad.  This switches us to oslo.log as
described in [1].

1: http://docs.openstack.org/developer/oslo.log/migration.html

Change-Id: Ibce86ab4ee24eeb55d0de1b0d5ff4ee4ea6ef66f
7 years ago
Ben Nemec ab5c66df45 Revert "Switch to oslo.log"
This isn't quite right and broke on stable/liberty.  Pushing
a revert in case I3c22d77dece399d21ab94783b74990789a1e1481
doesn't actually fix the problem.  We should probably merge
whichever passes first.

This reverts commit 69653318f4.

Change-Id: I9304429f25d28ca756e50b1788e149c5bb46b1d6
7 years ago
Ben Nemec 69653318f4 Switch to oslo.log
The old oslo-incubator log module isn't maintained (and doesn't even
exist anymore), so we don't really want to be using it.  It appears
this was the only incubator module we were actually using, so this
allows us to remove all of the unmaintained incubator code.

Change-Id: Ib4ad3b231360987a1ef4f95b5b5a8b656232efc4
7 years ago
Doug Hellmann 4a41f0b3e5 Drop use of 'oslo' namespace package
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.

The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.

Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.

Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages

Change-Id: If51059c31c82d5235e2ae21143911b5561783ca6
8 years ago
Steve Kowalik a3c1159ed4 Add support for Python 3
Make changes pretty much all over the code base with respect to
encoding strings and fixing imports to support Python 3.

Change-Id: Id1920129001b8e223474c1b2faf8bd9d527fe7e7
9 years ago
Clint Byrum 16158684a6 Change collector interface to return a list
With the new OS::Heat::StructuredDeployment resource, each Metadata
section may have multiple "deployments" in it. With this, we will return
a list with tuples of key and content to write to the cache.

Change-Id: I9f4272b0761e1dfd850bc5a5c6b27a78f126281f
Related-Bug: #1295787
9 years ago
Clint Byrum a13845a737 Run all collectors.
Also refactor internal API a bit to make code more generic.
10 years ago
Clint Byrum d8f60687dc Refactor to avoid monkeypatching requests 10 years ago
Clint Byrum ea19fd8416 Use a session so only one connection is needed. 10 years ago
Clint Byrum 5d961341c0 Migrate from httplib2 to requests. 10 years ago
Clint Byrum 7983481bc3 Implement --command as documented in README.md 10 years ago
Clint Byrum b34e62abda using global CONF object 10 years ago
Clint Byrum 21db93faba Add cachedir and command options 10 years ago
Clint Byrum ae75926f47 Move ec2 code into its own module. 10 years ago