This PS updates python modules and code to match Airflow 2.6.2:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9
Change-Id: If6f57325339995216d2553c7a5ff56e7673b5acc
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integratinal
tests.
Also the latest osh-infra commit was used with kubernetes 1.27.1 or
integration tests.
Make images section of Makefile was enriched with run_armada step in
order to quicktest the freshly built images - this is vital for focal
and bionic images differences.
Change-Id: Ida43eebd4aa51fff6261ac9cf5f8259359bc99a9
- armada-airskiff-deploy is voting gate again
- fixed falcon.API deprecation - -> falcon.App
- fixed collections.abc.defaultdict not found error
- fixed tox4 requirements
- implemented requirements-frozen.txt approach to make allike as other
Airship projects
- uplifted docker version in the image building and publishing gate
Change-Id: I337ec07cd6d082acabd9ad65dd9eefb728a43b12
Update kubernetes client to v26.1.0
Updating armada to focal base image
Remove xenial and opensuse dockerfiles
Update tox python from py35 to py38
Add apparmor for docker build
Uplift HTK chart version 0.2.52
Bumping up some python dependencies to get in sync with shipyard
Added clear-firewall role for airskiff-deploy playbook
Change-Id: If06a3f60466702d05a21c24a7cb8041bed41507a
Bumping k8s client to v25.3.0
Cronjob batch v1beta1 no longer available in k8s 1.25
Update tox.ini file to be compatible with v4
Change-Id: Iac79c52c97c9ef1223ae8d502da1572ef8d068fa
For now we leave the tiller status enpdpoint, until
Shipyard has had a release to stop depending on it [0].
[0]: https://review.opendev.org/c/airship/shipyard/+/802718
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: If8a02d7118f6840fdbbe088b4086aee9a18ababb
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase. This is the effort
to do so for the Armada project.
[0] https://review.opendev.org/#/c/671291/
Change-Id: I4fe916d6e330618ea3a1fccfa4bdfdfabb9ffcb2
Bandit 1.6.0 changes the behavior of the '-x' option so that it now
supports glob patterns. Update our tox file to use to correctly exclude
test code from bandit scans. This requires bumping our minimum bandit version.
As an aside, changing the behavior of an option like this in a minor
version if bad form :(
Change-Id: I11bee26373ea9894b47d7f964eac88ef70859732
CI jobs are failing due to new YAPF style guidelines [0] that are not
PEP8 compliant. This commit pins YAPF to 0.26.0 to prevent non-compliant
regressions in the gate.
[0] https://github.com/google/yapf/blob/master/CHANGELOG
Change-Id: I030031f3748cddc5fce3020e7bcfe324b666cbd8
Flake8 3.6.0 now warns about both line break after and *before* binary
operator, you have to choose whether you use W503 or W504. Disable the
newer W504.
Fix "F841 local variable 'e' is assigned to but never used".
Handle warnings about invalid escape sequence in regex.
Handle invalid escape sequence in string.
Change-Id: I68efbde4e9dd2e6e9455d91313eb45c9c79d35ce
This patch set combines whitespace linter and pep8 together
(as both do code linting) into the same job by dropping the
.zuul.yaml whitespace-linter job and rolling it up into the
same pep8 tox.ini check, just like Deckhand [0].
[0] 0462b7b929/tox.ini (L106)
Change-Id: I31ed60adfd431157ffc96a0f4d5d77d54af37352
The output with yapf version 0.24.0 has some changes which are in
conflict with the flake8 E126 rule. Hence pinning to <=0.23.0 for now.
Change-Id: I80b4094673eeca6dddc4fde0c324e0f5d9ec5d99
This patch set updates the various tox targets to use python 3 first
per Stein goal outlined in [0]. Also, added minversion to tox.
[0] https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I60ad774e089df248f3b1993a815dc6ac7232c3fc
Signed-off-by: Tin Lam <tin@irrational.io>
- Adding yapf diff to pep8 target
- Adding yapf tox target to do actual format
** The rest of this PS contains formatted code only, no other changes
Change-Id: Idfef60f53565add2d0cf65bb8e5b91072cf0aded
This patch set adds command to clean releasenote prior to building
releasenotes. This also clean up unnecessary whitelist command.
Change-Id: Ib9439235a24132111a56a9e35f0c9640d2892f5e
Signed-off-by: Tin Lam <tin@irrational.io>
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I3f2720a6e8e6c685d2e5ca288008be20785a8309
The tox coverage refers to a non-existing rc file. This patch set
redresses this by putting in a proper .coveragerc file similar to
all the other projects.
This patchset also makes the tox coverage job compatible w/ Zuul.
Change-Id: I96a3d13482053b591f7d01e1c5c1e906d84f8fb7
Depends-On: If3cba68492ba9c7a0e78258c7b669d1b39fcdeba
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set puts the bandit check as part of the pep8 gating job
for airship-armada and remove the separate bandit gate.
Change-Id: I3871bb20f04aedb80a1ffca4038e94a156156a14
Signed-off-by: Tin Lam <tin@irrational.io>
This patchset renames the tox.ini coverage job to cover to
comply with OpenStack standards [0]. This is needed so that [1]
works seamlessly (it will enable py35 and cover jobs for Armada).
[0] e.g. 04469a5181/tox.ini (L119)
[1] I174d5df008f6e000da1a3878afe75919312ea7aa
Change-Id: I353a60313bef2f8aa0319a4e89e659cd23c8a4a9
This patch set removes E722 pep8 exclusion that allows for "bare"
except: statement.
Change-Id: Icdce885366541b88aabbef35166cf196a588676b
Signed-off-by: Tin Lam <tin@irrational.io>
This adds better exception handling ang logging to
_append_file_to_result helper in get_files. When reading
arbitrary file data and attempting to encode to utf-8
this can cause UnicodeDecodeError to be raised.
However, Armada will not skip over such files; it will
raise an exception with appropriate details instead.
Closes #195
Closes #196
Change-Id: Id7c32c17e351d1ffe042e3755c116c36b6380223
ARMADA-29: CI/CD Armada Testing Framework Entrypoints defined
It does not appear the the docker run command to call tox tests/unit
is necessary. The Makefile already includes several testing checks.
Added a few additional targets as specified in the upc-integration
code-convenstions to satisfy the clcp Makefile standards.
Change-Id: Ia920a856ed6d607fcf38d5f85dbb9a0c8aae4fa6
- adding .editorconfig file
- minor cleanup in various files related to .editorconfig
- typos, whitespace, etc.
- other general housekeeping items on the codebase
Change-Id: I104f8dcb06aafb180da12f7ee4c0ded41fc07b9d
This PS unskips the last remaining git test in test_source which
had an unconditional unittests.skip() decorator applied to it.
The source_cleanup test now issues a warning if the repo path
isn't found and the associated test (passing in a bad path)
simply asserts that the warning was called.
This also adds a conditional skip to the git clone unit tests
in the event that the user doesn't have network connectivity.
Change-Id: Ic300c48a4e0c361e37ec15cd3675415f8170a817
This PS simply allows tox -e py35 to take in regular expressions
for better testing experience. For example:
tox -e py35 # runs all unit tests
tox -e py35 -- test_armada_controller # only run unit tests that match
Change-Id: Iab5a70fa0d51e1572a31943d8f38bc9b0d6a7c18
This allows the user to apply and/or validate a manifest using
either a filepath (as before) or URL.
Addition by sh8121att:
- Create a general document resolver class to handle local paths
and URIs
- Allow multiple filenames and combine them into a single document
set
- Change API to allow for passing document reference URIs to be
resolved server-side rather
- Update validation API to conform to UCP specification
- Dockerfile updates to speed up build
- Fix unit tests
Closes #96
Change-Id: I5a57779f10d1b63ffc161a14afec851a34ae9efe
This PS adds whitelist externals to Armada's tox.ini to avoid
warnings that are issued while running tox -e py35 or tox -e pep8.
Example:
WARNING:test command found but not installed in testenv
cmd: /usr/bin/find
env: /opt/stack/armada/.tox/py35
This PS also resolves a critical error while running
tox -e coverage, because tox defaults to python2.7 for
the coverage job, and Armada's hapi module is not compatible
with python2.7. The fix involves setting the testenv's
basepython to python3.5.
Change-Id: Ice0074c3453c80290886f27a4518fbed01722572
This PS can help the developer run just one unittest.
like below.
tox -e py35 armada/tests/unit/api/test_api.py::TestAPI::test_armada_apply
Change-Id: I846c52bb4f3b133942f7c201f750dc15607f93ea
- generic exception in errors
The goal of this is to move all exceptions into one file and generic format.
This aligns with the UCP projects.
Change-Id: I1862f08409182c77b03eaeb9e6bd3209d72c5cee
Update handler for chart pre-upgrade Jobs deletion
to rely on Kubernetes propagationPolicy for deleting
child Pods so that more generic labels can exist in
an Armada manifest without impacting job-unrelated
pods
- Update K8s API integration to use propagationPolicy for job delete
- Make default propagationPolicy 'Foreground'
- Update documents to clarify structure of specifying pre-upgrade hooks
- Fix tox file to support running unit tests behind a HTTP proxy
Change-Id: I650543cfe05cc6a9661ab375e831bb425b7eeeab
This patch set makes Armada pep8 compliant. Note the hapi/** is
autogenerated and therefore should be excluded from linting.
Change-Id: I123eefb543f9bd9cf0bc6bd98ed95646d8d72cc3
- Register Oslo logging configurations in server.py, test_armada.py, test_chartbuilder.py, and test_tiller.py
- Add mock to test-requirements.txt
- Update .travis.yml
- Add mock case for tiller IP
