13 Commits

Author SHA1 Message Date
HUGHES, ALEXANDER (ah8742)
8d0b847a03 Standardize Treasuremap code with YAPF
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase.  This is the effort
to do so for the Treasuremap project.

[0] https://review.opendev.org/#/c/671291/

Change-Id: Icd45f1b99a90e6c934a84fdd91f2f7f8af5a8ddb
2019-07-24 17:58:10 +00:00
Roman Gorshunov
f995e9e7c4 Remove nagios and prometheus exporter images from updater
Nagios and prometheus-openstack-exporter images and charts are sourced
from OpenStack-Helm project and do not need to be updated separately
here anymore.

Change-Id: I939329e1b602b465e4d43f19ec4b853dd07acee0
2019-07-12 16:32:08 +02:00
Roman Gorshunov
00cbd30abf Enhance skip list
This patch enhances skip list, so that images matching skip pattern are
skipped. Previous behavior was to skip only images ending with pattern.

Change-Id: Ic6548048412b6ab4e62398c66fedb888489bd605
2019-07-05 20:39:43 +00:00
Roman Gorshunov
5d02307117 Add cross-verification of Git commit ID'd and image tags
Versions updater utility will cross-verify latest Git commit ID's and
latest container image tags. It will print useful information on
possibly missing references in versions.yaml and Git <-> image tag
mismatches.

Slightly increase timeouts to reach quay.io.

Wrap lines and use double quotes everywhere.

Logger is declared out. Global variables are still global if declared
at the top level.

Change-Id: I267ca57182d54d26f4e75d143ad273cb6c30060e
2019-07-05 18:26:30 +00:00
Roman Gorshunov
07f287c77c Add tag filter parameter to the updater tool
New parameter allows to prefer image tags from quay.io matching filter,
e.g. "opensuse" or "ubuntu".

Change-Id: I75091645ea4492c0db0facfcb3e95f18a54a76c8
2019-05-23 14:22:43 +00:00
Evgeny L
40915a8cfc Fix chart repo urls from openstack to opendev
On April 19, OpenStack completed the migration from openstack.org to
opendev.org [0]. During this migration, Airship projects moved from the
"openstack" namespace to the "airship" namespace. This commit makes
preliminary updates to gate and developer scripts to account for the
migration and fix broken integration gates.

[0] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005011.html

Change-Id: Ie955a777016deceed0d3f7f1aa839fba3cfdcc3e
2019-04-23 15:30:20 +00:00
Roman Gorshunov
23b8e13c5b Add cache for results of requests to quay.io in Updater tool
All changes are in Updater tool.
1) Cache results of requests to quay.io
2) Fix formatting of strings and strings in logs
3) Improve logging for http requests to quay.io
4) Fix failure when --out-file is specified with path
5) Fix quay.io HTTP 301 permanent redirect from '.../tag' to '.../tag/'

Change-Id: I7d5cec65c713be261af0997774ee945545b5090a
2019-03-04 17:05:49 +01:00
Roman Gorshunov
78bb5d6801 New option --skip (charts, images)
Added new option --skip, which allows to skip updates to the
listed comma-delimited charts and images.

Change-Id: I08a458fc107019ee915e8e8086d23215642c9835
2018-10-24 00:39:24 +00:00
Felipe Monteiro
5a93c30c85 fix: Stop adding unwanted newlines in multiline strings
This patch set fixes an tools/updater.py YAML export issue in which pyyaml is
adding unwanted (multiple) newlines in multiline strings in the output. In
other words, currently, tools/updater.py keeps injecting additional newline
characters in multiline certificates, such that each newline is rendered, in
effect, as two newlines.

For example:

  -----BEGIN CERTIFICATE-----
  MIIDSDCCAjCgAwIBAgIUegkh/antB1XyDVHdP5dv+0MZyBcwDQYJKoZIhvcNAQEL
  BQAwKjETMBEGA1UEChMKS3ViZXJuZXRlczETMBEGA1UEAxMKa3ViZXJuZXRlczAe
  <snip>

Currently renders as:

  -----BEGIN CERTIFICATE-----

  MIIDSDCCAjCgAwIBAgIUegkh/antB1XyDVHdP5dv+0MZyBcwDQYJKoZIhvcNAQEL

  BQAwKjETMBEGA1UEChMKS3ViZXJuZXRlczETMBEGA1UEAxMKa3ViZXJuZXRlczAe
  <snip>

This patch set adds code to the updater.py tool to use the
solution prescribed in [0] to force pyyaml to ensure that the
rendered output for such multiline strings matches the original
input.

Change-Id: I8b5c5166a0a1b3d945281addd4f562368e1907c3
2018-10-17 18:03:00 +00:00
Roman Gorshunov
0cf96f8b55 New option --out-file, various fixes in logic
1) added --out-file option
2) fixed imports for time and json
3) fixed retries logic to exit loop once we get valid response
4) fixed reference to an undefined variable

Change-Id: Iebbb83079978eb22eb13e6c93642a35f5bd14a20
2018-09-24 16:49:19 +02:00
Kaspars Skels
f9afea172d Change image update logic to latest in quay.io
Currently images are not being published for each commit,
only for commits that have code change.

Also, gates at times sit in a queue and not available right away.

Set logic to check for latest tag that is not named latest or master.

Change-Id: I8b543e579d77ec1a1b9dfe3e9e70baa5cdb186d8
2018-09-19 14:55:28 +00:00
Roman Gorshunov
3efa98a31b versions.yaml updater tool: verify that images & tags do exist
This patch allows the tool to verify that container images with
specific tags equal to the HEAD git commit id's we reference, do
really exist on quay.io repository and are available for download.

Change-Id: I2205f4fa63a085d94ef3c0e9d2ff511665df43e6
2018-09-11 15:59:34 +02:00
Roman Gorshunov
b5ccec8d25 versions.yaml updater tool
Being run in directory with versions.yaml or provideded with path to
the versions.yaml, will create versions.new.yaml, with updated git
commit id's to the latest HEAD in:
  1) references of all charts
  2) tags of container images listed in dictionary image_repo_git_url
     dict

Change-Id: I163304da5bf5ec198deb7aaea0abac04c96c569e
2018-09-10 17:12:41 +02:00