Currently, any document name or schema referenced in the Shipyard
code base is a hard-coded string. Often times, these strings are
repeated throughout the code. This patch set adds a new configuration
section to shipyard.conf to define document names and schemas so they
can then be referenced in the Shipyard code via the oslo
configuration object. This functionality will be important for
upcoming Shipyard features which will call for more documents to be
validated as well as some new Shipyard-created docs.
Change-Id: I34ae8cd578bab730d004c3d176e3817b5a45c89e
By default, all sudo commands are logged to auth.log with their
full command line. Previously, Shipyard scripts called 'sudo docker'
with -e OS_PASSWORD=foo, resulting in the password value appearing
in auth.log in plaintext. This change adds -E to the sudo command
to pass the user's environment through, and removes the value from
-e OS_PASSWORD which tells Docker to use the environment value
directly. This prevents the password value from being logged.
Change-Id: Ifcf7f6525876144a609ff42be42da57a3f7f6f60
A recent Shipyard change [0] introduced user-context tracing support in
Shipyard, which requires the passing of an additional parameter to the
Deckhand and Drydock API clients. The Deckhand [1] and Drydock [2]
clients were updated to handle the additional parameter, but the
versions of those components used by Shipyard was not updated. This
change uplifts the versions of Deckhand and Drydock in requirements.txt
for the Airflow component of Shipyard to a version that supports the
aforementioned, additional, user-context parameter.
[0] https://review.openstack.org/633873
[1] https://review.openstack.org/634068
[2] https://review.openstack.org/638115
Change-Id: I096191f436c5123b6c263fe92c0d69b9e4085c66
Updates the definition for queue mirroring to use the same vhost as the
queue itself is defined to be.
Change-Id: Ibb631d7adb237fa17c5e853a9e9e35297a525782
The current script is fragile to the introduction of a DEBUG logging
level. This change addresses the specifics of logging records being
returned when invoking the airflow command enough to make the dag_status
result end up being the last result as the script desires, so the script
is returned to working order, even if the logging level is set to DEBUG.
This change goes a little further to the point of removing all records
that indicate a logging level of DEBUG|INFO|WARN|ERROR.
Change-Id: Ic72358097e3a476e20fa3713008d0d0bcd35463e
When Shipyard fails to execute some of the steps, the status
of the action is always "Complete", determine the success
of the execution based not only on action status, but
also on the status of every step within the action.
Change-Id: If7f71b55c2aed0322edef9c811ed1906400b0913
The airflow image has a requirements file that is separate from the one
located in src/bin/shipyard_airflow, which was missed in a previous
uplift patch [0]. This change updates all versions in the image
requirements file to match the source code requirements.
[0] https://review.openstack.org/652736
Change-Id: I238bba4038e5cd8364711c2c5d1f78f51fc76948
A recent Shipyard change [0] introduced user-context tracing support in
Shipyard, which requires the passing of an additional parameter to the
Armada API client. The Armada client was updated to handle the
additional parameter [1]; however, the Shipyard version Armada uses was
not updated. This change uplifts the version of Armada in
requirements.txt for the Airflow component of Shipyard to a version that
supports the aforementioned, additional, user-context parameter.
[0] https://review.openstack.org/633873
[1] https://review.openstack.org/638124
Change-Id: I7dfbda842eb136858fbb6d891b96d057a977c913
This PS adds entry in log for user id and passes on the context
maker to other Airship components from Shipyard during API call.
This will ensure easy tracing of user and context through log
tracing.
Change-Id: Ib9bfa8f20b641f8bb6c2dca967d9388e30d5735c
This commit introduces a non-voting job to lint Helm charts against the
latest version of Helm toolkit from OpenStack-Helm Infra. This job
should serve as an indicator of when it's safe to advance the version of
Helm toolkit used by Airship.
Additionally, this commit modifies all Helm chart lint jobs to run on
each commit, regardless of the files modified by a change. This should
not introduce a noticeable difference in CI runtime, as these jobs
execute quicker than the tox jobs.
Change-Id: Ife3b285a2f9a160df9724e9dcb230243d06be3ab
- deployment-shipyard
This updates the shipyard chart to include the pod
security context on the pod template.
This also adds the container security context to set
allowPrivilegeEscalation to false and readOnlyRootFilesystem to true
Change-Id: Idb1b848847eaec2b6e24389c063b7ece2973c4dc
Currently Shipyard is attempting to retrieve Tiller information to pass
to Armada. Once the dependent change is merged, Tiller will be moved
into the same pod as Armada, and this information will be retrieved
from the Armada configuration file. Therefore, Shipyard should no
longer be retrieving this information for Armada.
Co-Authored-By: Michael Beaver <michaelbeaver64@gmail.com>
Depends-On: https://review.openstack.org/#/c/632788/
Change-Id: I689e396ef9f184525f09f819f13c1490344481f2
Helm v2.13.1 has been released [0]. This change uplifts Helm to v2.13.1
to match other Airship projects and pins the Shipyard chart linting gate
to the latest version of Helm toolkit.
[0] https://github.com/helm/helm/releases/tag/v2.13.1
Depends-On: https://review.openstack.org/647812
Change-Id: Id9b15ad5f3253332ef0f8129b62a7b7510ab475c
This commit changes the default image tag for Shipyard and Airflow
images from `untagged` to `latest` to match other Airship projects.
Change-Id: Id1f9d56784e51f0631280188ba1b898fde8f56ea
- Zuul updated ansible to 2.7, no longer allows missing variables.
- Using default value when it isn't available.
Based on Aaron Sheffield's PS for Pegleg: https://review.openstack.org/#/c/645631/
Change-Id: I02495bc793021b429e0be62ecac0ed45e930484c
This enables passing any combination of actions and parameters to this
script. This also makes it easier to add scripts for additional actions
by not having to modify execute_shipyard_action.
Updated redeploy_server to pass in servers as a param.
Change-Id: I590183c28e8c66997ab85470902dbe9576c5c5f6
Before if condition was looking for non-existence of ${server}. `server`
is never defined, so this condition is always true.
Now if condition is looking for existence of ${servers}, which is
sometimes defined.
Change-Id: I31cd4a2765705176a9d5239a01b04ff341b36540
Updates the dependencies on Armada, Drydock, and Deckhand to current
values. The primary reason for this is to take advantage of a new set of
Armada requirements that allows for a reduced image size (no longer
includes grpc tools)
Change-Id: I8f208ab91664479a7af96c543a4e72d8f6d154ef
Adds the SLUGIFY_USES_TEXT_UNIDECODE=yes option to the appropriate tox
jobs to allow document builds/regeneration of directories occur in
alignment with license requirements
Change-Id: Ie0d25183e6759d1ae48aa45c504479e1d0592d52
Change to use ubuntu base image instead of python
Refactor Shipyard Dockerfile to reduce image size significantly
BREAKING CHANGE: The `make images` PYTHON_BASE_IMAGE arg is now renamed to BASE_IMAGE.
Change-Id: I3338dfbbb91b5514fa4fd205bdfc4136d0abc2e5
Moves the airflow web server container from its own pod into the
Shipyard pod. This removes exposed network surface area from the
Shipyard suite of software. Shipyard, after this change accesses the
Airflow API using localhost in the same k8s pod.
Change-Id: Ied4bd415a8d78c393b7256ead27a6a2176f4a2d6
Changes to make the docker image build to include the workflows from
Shipyard, rather than adding them to the container during Helm install
of Shipyard. This also removes the "prod" switch, as it is now always
built the same way, with the workflows in place.
Change-Id: I4acd6195cbec32193e15621e75ccaeb9879455f5
Moves the airflow sceduler to a container in the airflow-worker
statefulset so that its version lifecycle matches that of the worker.
Leaves the stand-alone scheduler in place to support upgradability from
prior installations that included a standalone scheduler. New
installations are advised to turn off the scheduler template from
rendering using the values.yaml flag.
This is an attempt to make disruptive upgrades to airflow less impactful
to a "update_site" action from Shipyard.
Additionally this removes the template for airflow-flower, which is not in use.
Change-Id: I0608793ee6aba1eb3ce0f5e9567655287014a0ca
This commit introduces a chart build gate that triggers when changes are
made to charts in the charts/ directory.
Change-Id: Iec1c6da785d26e2e7273b37fd1028f3bf47ec199
Currently, the airship-shipyard-ubuntu job deploys K8s using outdated
playbooks. The job fails due to a change [0] that introduced an AppArmor
requirement to Calico. This change removes the failing job, as its
success does not depend on changes in Shipyard.
In the future, jobs requiring K8s should be deployed using the new
Minikube gate scripts from the OpenStack-Helm infra repository. They
will be added in future changes once they are required. Additionally, a
future change will introduce a Helm linting gate for charts hosted in
the Shipyard repository.
[0] https://review.openstack.org/614805
[1] https://git.openstack.org/cgit/openstack/openstack-helm-infra/tree/tools/deployment/common/005-deploy-k8s.sh
Change-Id: I72aaff9050eab844ef0856794b059de14e2a4a28