1. Removed osh-infra-upgrade-host playbook from image building jobs,
as non-relevant
2. Removed osh-infra-deploy-docker playbook from image building jobs,
as docker-ce is deployed in run-image playbook
3. Removed osh-infra-build, osh-infra-deploy-k8s and security-bandit
playbooks - they are not referenced anywhere; bandit has already
been merged with pep8 job
Change-Id: I27f0fd8f346d0b52c55870b6005be525a1fec671
This PS enables shipyard to use PBR as rest
of the other OpenStack project.
Currently if shipyard is used as a requirement in another
project, the dependencies for shipyard doesn't get installed.
This PS fixes this bug.
Change-Id: I34d452649a886af9d865d4ad51eeab70399fe395
Arbitrary labels could be added as `make` parameter `LABEL=`, which is used in
att-comdev/cicd Jenkins pipelines.
Sample parameter:
'LABEL=org.label-schema.vcs-url=${GERRIT_CHANGE_URL} \
--label org.label-schema.base-image=${base_sha256}'
Sample usage:
See Jenkinsfile files under images/ directory in att-comdev/cicd repo.
In addition to that, if `COMMIT` variable is undefined when invoking `make`, we
use result of `git rev-parse HEAD` command, which should output latest git
commit ID.
[0] https://github.com/att-comdev/cicd
Change-Id: I7c6b63c865ac0ff63f4a85ec54863c20781fe98d
This patch set uses zuul templates for python35 and python36
because:
1) python35 template is maintained by project-config; don't need
to maintain separately in SY
2) python36 is now what SY will be building off in the Dockerfile
so it is better to also test this actively in CICD
Change-Id: I98230b51db69f1054d11428f48bd09dc2547f8b6
- Adds a query parameter 'cleartext-secrets' to get full raw documents.
- Adds CLI flag to get full raw documents.
Change-Id: If38974c8433c8360cc47ae1273720ad76e87a6fd
This change removes an extra space on line 375 of the Shipyard values
file that prevents Helm from rendering the Shipyard chart values. The
line is part of the Shipyard policy config.
Change-Id: I49f3e4c442a00be043218bac2fe6f2da3c723b0d
Enhances the workflow to include adding notes that contain the builddata
information associated with the Drydock steps. Part of adding this
support includes adding general notes support to all of the operators
that inherit from the UcpBaseOperator
Storyboard References:
Story: 2002797
Story: 2002796
Change-Id: I5e1a54d6373c4a523e2d4fe87796da4358f22055
While iterating on the next steps of using notes, it became clear that
several changes to the output and access methods for notes needed
enhancements. This change introduces a new way to access a note's URL
information via a new API/CLI, while removing the resolution of URLs
from the existing note output. This supports the concept of "builddata"
coming back with sizes of 800kb or more - which really can never work
out inline in other data, especially in cases where there is
multiplicity of the information across many items.
New API: GET /notedetails/{id}
CLI: shipyard get notedetails/{id} and/or shipyard get notedetails {id}
Returns the resolution of the URL for a note, outputting the raw info as
the response (not structured in a JSON response).
The CLI will attempt to minimally format the response if it has inline
\n characters by replacing them will real newlines in the output (if the
output-format is set to either cli or format. Raw format will be
returned as-is.
The existing notes responses are changed to not include the resolution
of the URL information inline, but rather provide the text:
Details at notedetails/{id}
The CLI will interpret this and present:
- Info available with 'describe notedetails/09876543210987654321098765'
This is an attempt to inform the user to access the note details that
way - luckily the API and CLI align on the term notedetails, as the word
details works well enough in the singular form presented by the CLI and
the plural form used by the API.
The ID returned is the unique id of the note (ULID format).
Notes that have no URL will return a 404 response from the API (and
an appropriately formatted value from the CLI).
This approach solves an issue beyond the large inline values from URLs;
providing a means to NOT resolve the URLs except in a one-at-a-time way.
Long lists of notes will no longer have the risk of long waits nor
needing of parallelization of retrieval of URLs for notes.
This change introduces an API-side sorting of notes by timestamp,
providing a chronological presentation of the information that may or
may not match the ULID or insertion ordering of the notes.
Additional feedback from peers about the output of noted indicated that
the CLI formatting of notes in general was in need of visual tuning. As
such, this change introduces changes to the formatting of the output
of notes from the CLI:
- Notes for describing an item will be presented with a more specific
header, e.g.: Action Notes: or Step Notes: instead of simply Notes.
- Tables with notes will change the header from "Notes" to "Footnotes"
give the user a better marker that the notes follow the current
table.
- Table footnotes will be presented in a table format similar to
the following, with headings matching the kind of note being
produced.
Step Footnotes Note
(1) > blah blah blah
> yakkity yakkity
(2) > stuff stuff stuff stuff stuff stuff stuff
stuff stuff stuff
- Info available with 'describe notedetails/...
> things things things
Change-Id: I1680505d5c555b2293419179ade995b0e8484e6d
Enhance the Shipyard API and CLI to retrieve notes that have been
specified against actions and steps. Includes a new reusable parameter
for verbosity.
Change-Id: I1c7f47c0346ce783dacd62b8bbc1fd35a0bf285b
Two testing only changes, one to mock the notes helper during the
action_api testing so it doesn't wait for network timeouts silently and
take a very long time to run, and secondly enhanced testing around the
production of node filters used with deployment groups to validate that
rack_names are being handled in the desired way when producing node
filters for Drydock.
Change-Id: I5439e82333f40e91c270fa52c466731b4bbf1f2f
Fixing duplicate items in "org.opencontainers.image.title" label of
the container images. Was: "airflow shipyard", now - separate and
correct for each if the images being built.
Change-Id: Iad180a100f8ce5ccc3ac718a447d10b0813aa665
Adds the common/shared code to support notes in Shipyard such that this
component can be reused between the Shipyard API and the workflows.
Includes a single example of creating and retreiving notes as part of
action creation, that will likely be changed in a future change.
Note that this change doesn't include changes to the CLI to represent
notes outwardly, but the first example can be seen using the
--output-format=format option (shipyard get actions)
Change-Id: I2f87713eb74dae312912ff4c36e6ae30a569ea38
The goal of this commit is to reduce the average time spent retrieving
validations from Deckhand. Currently, wait times when committing
configdocs can be significant due to unnecessary API calls. This change
reduces the number of API calls during this process by utilizing the
`/revisions/{{revision_id}}/validations/detail` endpoint exposed by
Deckhand. During testing, this introduced a 71% decrease in cumulative
time for committing configdocs. Note, this commit does not introduce
usage of the official Deckhand client, which will be addressed in a
future change.
Change-Id: I3c86fca6bae1a5a2f74963a87b2198c1705cf3a6
This commit adds the Werkzeug ProfilerMiddleware to Shipyard API
requests. This option can be enabled using the
`conf.shipyard.base.profiler` option and should not be used in
production.
Change-Id: I293840d78baf670478047faad87fdcfe2f8af70e
This patch modifies API behavior for
GET /v1.0/actions/{action_id}/steps/{step_id}/logs
such way:
- it returns the same status code as Airflow HTTP request returned
if Airflow responds with a status code of 400 or greater,
- it returns 500 error status code if an exception happens during
Airflow HTTP request (200 was before).
Warning: this change breaks API backward compatibility, now a client
could get 4xx or 5xx codes proxied from Airflow.
Change-Id: Ic5dceb3abc34415d21b4d8d4e71b4e5661a7363d
Tunes the interval between checks for tasks to run to every 5 seconds
instead of a continuous loop. In practice this looks like it should be
somewhere between 6 - 10 times less active against the database, and
likewise spawn the same factor less processes. Additionally tunes and
adds notes about some of the other tuning parameters.
Change-Id: I5e28b4c081c53a553072470ae53302915a90dd1a
This commit introduces an action, `test_site`, that invokes Helm
tests for all deployed releases using the
`ArmadaTestReleasesOperator` introduced in [1]. This action supports
the ability to invoke Helm tests for a specific release using the
`release` parameter and cleanup resources if the `cleanup` parameter
is set to `true`.
[1] https://review.openstack.org/#/c/603236/
Depends-On: https://review.openstack.org/#/c/603236/
Change-Id: Ib5f38fe4b8a6516ee2afae62774ec84f1d2eb1ad
This commit adds a new operator, `ArmadaTestReleases`, and DAG,
`test_site` that invokes Helm tests corresponding to each deployed
release across all namespaces. The `ArmadaTestReleases` operator also
supports the ability to invoke Helm tests for a specific release using
the `release` parameter and remove test pods if the `cleanup` parameter
is set to `true`.
Depends-On: https://review.openstack.org/#/c/603235/
Change-Id: Ie2e92c347b670258a7d6f49bff2e1e840b72067a
Currently, the `ArmadaGetReleasesOperator` does not save any releases
retrieved from the environment. In order to add future functionality
that invokes Helm tests for all releases, deployed releases must be
cataloged. This commit adds a method, `get_releases`, to the Armada
base operator and updates existing plugins to utilize it. Additionally,
this change moves the retrieval of tiller information before the Armada
client is fetched in order to limit subsequent calls to the
`get_tiller_info` method.
Change-Id: Ib4aaec762d509994ce90460d0854e526280c4592
The 'airflow-worker' and 'airflow-logrotate' containers do not need to
run as privileged containers to perform their jobs. Shipyard deploy_site
action was used to test the 'airflow-worker' as a deploy_site invokes
'airflow-worker'. When performing deploy_site action, all steps succeeded
and the 'airflow-worker' shows no errors when 'airflow-worker' is
non-privileged.
When 'airflow-logrotate' runs as non-privileged, the 'airflow-logrotate'
container still logs correctly and is able to delete/rotate logs without
problems.
Note: Making airflow-worker run with non-privileged containers means that
these containers will use the docker-default apparmor profile by default.
Change-Id: I26eda3eb8b7a36e67c2e7b593326f1d063600fc3
Now that Deckhand supports rolling back to revision 0 with
https://review.openstack.org/#/c/592725/ it is preferrable to
exercise this functionality instead of deleting all revisions
in Deckhand's database, so that previous revision history
isn't lost.
Deckhand is fast-forwarded to
0b5aa2e98a1ab5ab8a58c9dec3c1f88ef00d17a9 which contains
the commit required for SY to be able to make use of this
new change in Deckhand's client, as well as to align with
Promenade's Deckhand fast-forward of Deckhand in
https://review.openstack.org/#/c/604459/
Depends-On: https://review.openstack.org/#/c/592725/
Change-Id: Iebbfeb5e6c74b3e2f09b2137ec14482cc8fa4f44
Roman Gorshunov
Zuul
lvxianguo
Nishant Kumar
Felipe Monteiro
Aaron Sheffield
Drew Walters
j330k
Bryan Strassner
skovaleff
Andrey Volkov
Rick Bartra