This PS replaces old airskiff integration gate
with new kubeadm based airskiff integration gate.
The main goal of this gate is to test new deckhand
image and chart together with other Airship
components.
Change-Id: I4221b7be64e9a7e07964fa817d42c21a3f172db0
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1
Change-Id: If7191c8291e31488a8a5f26107981616e14ea12c
This PS rolls back python deps update done after adding xattr==0.10.1
https: //review.opendev.org/c/airship/deckhand/+/906424
Change-Id: I0d246d3a72555e1fb6c7542a20af207e7dd74ba8
Update helm toolkit ref so that ingress will be created with a pathType
of Prefix.
https: //review.opendev.org/c/openstack/openstack-helm-infra/+/905757
Change-Id: I261603e3c68337347955996f77b86bd60a2f2af5
Upgrading htk to version 0.2.55, which deprecates the ingress class
annotation (kubernetes.io/ingress.class) with .spec.ingressClassName
https://review.opendev.org/c/openstack/openstack-helm-infra/+/891720
Change-Id: I573a926ab6fb07f10c0c4d9020746ba80e6d8dbd
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
This PS updates python modules and code to match Airflow 2.6.2:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfrmed based on
airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9
- postgresql image updated to 14.8
Change-Id: I65a1b86473ee3e988aae353b59fb5473d75851f9
This PS makes the following changes:
- uses deploy-k8s.sh from treasuremap
- makes sure the airskiff-deploy playbook is using 80Gb partition if
available
- adds available security updates to docker images
Change-Id: I0f330cb15ec32b12703f0bc6620b3f3c797a25bb
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.
Also the latest osh-infra commit was used with kubernetes 1.27.1 for
integration tests.
Change-Id: Ia2c951f27b96774b553e7c0c7c6809172312f753
This PS delivers the following updates:
- fixed sample config and policy files generation in tox
- rolled back chart version incremention back to 0.2.0
Change-Id: I509030319a724b18bb21f45f7ede7c07ab18e894
- adjusted .gitignore to keep fresh egg-info and omit build artifacts
- fresh egg-info data is needed for promenade that depends on Deckhand
- restored deckhand-functional-uwsgi-py38 gate
- restored deckhand-integration-uwsgi-py38 gate
- made deckhand-airskiff-deployment gate voting ( treasuremap project
has been updated)
- removed bionic gates
- updated focal dockerfile
- added more binary deps into bindep.txt
- updated deckhand chart values to latest images - focal and wallaby
- fixed python code to compy with CVE's found by fresh version of bandit
- implemented pip freeze approach
- added tox -e freeze profile to manage it
- requirements-frozen.txt is now main file with requirements
- requirements-direct.txt is the file to control deps
- updated setup.cfg to adjust to newer version of setuptools
- fixed airskiff-deploy gate
- fixed docker-image-build playbook to restore Quay repo image publish
- updated other playbooks to include roles from zuul/base-jobs in order
to setup build hosts properly
- removed workaround with hardcoded dns resolver ip 10.96.0.10 as it
became obsolette due to recent fix in openstack-helm-infra
- adjusted tools/whitespace-linter.sh script
- tox.ini has been brought to compliance with tox4 requirements
- replaced str() calls with six.text_type() according to D325 Deckhand specific
commandment from Hacking.rst
- locked python-barbicanclient version with 5.2.0 because of breaking
changes in the upper versions
Change-Id: I1cd3c97e83569c4db7e958b3400bdd4b7ea5e668
update dockerfile for python deckhand install
add deckhand version to chart 1.0
add chart version 0.2.0
update all packages to latest in requirements.txt
update zuul jobs for focal and python 3.8
remove zuul job functional-uwsgi-py38 in favor of functional-docker-py38
update tox config
typecast to string in re.sub() function
add stestr to test-requirements.txt
add SQLAlchemy jsonpickle sphinx-rtd-theme stestr to requirements.txt
deprecated function: BarbicanException -> BarbicanClientException
fix mock import using unittest
fix import collections to collections.abc
fix for collections modules for older than python 3.10 versions.
deprecated function: json -> to_json
deprecated function: werkzeug.contrib.profiler ->
werkzeug.middleware.profiler
deprecated function: falcon.AIP -> falcon.App
deprecation warning: switch from resp.body to resp.text
rename fixtures to dh_fixtures because there is an imported module
fixtures
switch from stream.read to bounded_stream.read
deprecated function: falcon process_response needed additional parameter
deprecated function: falcon default_exception_handler changed parameter
order
move from MagicMock object to falcon test generated object to fix
incompatability with upgraded Falcon module.
Adjust gabbi tests to fix incompatability with upgraded DeepDiff module
update Makefile to execute ubuntu_focal
update HTK (helmtoolkit)
unpin barbican to pass integration tests
Use helm 3 in chart build.
`helm serve` is removed in helm 3 so this moves
to using local `file://` dependencies [0] instead.
Change-Id: I180416f480edea1b8968d80c993b3e1fcc95c08d
When performing substitutions, there are occasions when the source value
does not exactly match the format required by the destination document
(e.g. the values.yaml structure of an Armada chart).
This change provides the ability extract a substring of the source
value, and substitute that into the destination document.
Two optional fields are added to `src` under `metadata.substitutions`:
* `pattern`: a regular expression, with optional capture groups
* `match_group`: the number of the desired capture group
The canonical use case is a chart that requires an image with the repo
name and tag in separate fields, while the substitution source has the
full image path as a single value.
For example, assuming that the source document "software-versions" has:
data:
images:
hello: docker.io/library/hello-world:latest
Then the following set of substitutions would put the repo and tag in
the applicable values in the destination document:
metadata:
substitutions:
- src:
schema: pegleg/SoftwareVersions/v1
name: software-versions
path: .images.hello
pattern: '^(.*):(.*)'
match_group: 1
dest:
path: .values.images.hello.repo
- src:
schema: pegleg/SoftwareVersions/v1
name: software-versions
path: .images.hello
pattern: '^(.*):(.*)'
match_group: 2
dest:
path: .values.images.hello.tag
data:
values:
images:
hello:
repo: # docker.io/library/hello-world
tag: # latest
Change-Id: I2fcb0d2b8e2fe3d85479ac2bad0b7b90f434eb77
The following job is failing, seemingly due to changes in openstack-helm
and openstack-helm-infra:
deckhand-integration-docker-py36-ubuntu_bionic
For now, disable the job.
Change-Id: I55e441079a470d6f346d70857655f0a77ddc67c3
Update tox configuration and Zuul config to use Python 3.6, and make the
xenial and opensuse gates non-voting.
Change-Id: Iab98ad6cf269d57acf0598ae431b2f661673b234
Unpin python3-six for Opensuse image build.
Update helm-toolkit stable commit to merge of this change:
https://review.opendev.org/#/c/803654/
Update the helm installation script to download and install v2.17.0
Fix integration tests by pinning Barbican to stable commit.
Pinn jsonschema to 4.0.0a2 to fix tox tests
Change-Id: I2badd0e2f6c934098f0c9f5ef7e52354756c12e0
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].
These changes account for the fact that Shipyard imports Armada, Drydock,
Promenade, and Deckhand. Having said that, with pip 20.3, the pip
packages amongst those projects cannot conflict. A follow-up change may
be needed if more conflicts are found.
[0] https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020
Change-Id: Id75acea82ddf5d915a8b8805e076dac49cab800f
Patch PyYAML (via the pylibyaml library) to automatically enable the
LibYAML parser and emitter, which are faster than the Python versions.
https://pypi.org/project/pylibyaml/
Change-Id: Iebcc50b5db87518b3b7e0fac124c712afd06da2b
Updates Dockerfiles to build the LibYAML library, which can provide much
faster YAML parsing and emitting than the native Python library.
https://pyyaml.org/wiki/LibYAML
Change-Id: I1c6f41a72c7d32e810cf64f572dc2a1cc6a1e710
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0
Change-Id: I547a7f0e6106fee2f560b62671e1eceb312e5c4e
Updated obsolete uwsgi default configuration parameters for better
performance.
Increased number of worker threads to increase performance.
Uplifted uwsgi to the latest for bug fixes since 2018.
For more info please see:
https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html
Change-Id: Ifedb9c6279e64be86deb6ec375810c5ecf97958a
Adds configmap-hash annotations to the job-db-init and job-db-sync
for configmap-bin and configmap-etc.
These annotations ensure that if configmaps change, the pods
are redeployed according to their upgrade strategy.
Change-Id: I8ff282d8279c934590d5308e9c26efaf65685e2b
This updates the deckhand chart to include the pod
security context on the pod template.
This also adds the container security context to set
readOnlyRootFilesystem flag to true
Change-Id: I9bfd889b163e280cf17c4e7b49974a077e889f2f
Use pip3 in event system has both pip2 and pip3 installed.
Use apt to install setuptools for Ansible's consumption.
Change-Id: I6929ecb0cce2ec8ac70e9261acb9f87dc7031153
Co-authored-by: Alexander Hughes <Alexander.Hughes@pm.me>
Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.
This change removes all references to this copyright by the
non-existent group and any blank lines underneath.
Change-Id: Ib0b21b33d8bf91ea6da4c2421cc81355cf2b23b1
Policy validation in Deckhand was not implemented completely. Refer link
below:
https://airshipit.readthedocs.io/projects/deckhand/en/latest/users/validation.html#policy-validations
This PS removes some of the code related to the feature which was being
used in a code path when a set of documents are uploaded to Deckhand.
In standard Airship deployments the number of documents could be quite
high and this leads to significant delay (more than 300seconds in some
cases). As there are no plans to implement the policy validation feature,
it makes sense to remove it from code path which could cause delay and
sometimes timeouts while uploading documents.
This has been tested on a Baremetal lab: GF and BF.
Change-Id: I2ff3f40a7fe37bed5a589fab00d829db726604fe
