2262 Commits

Author SHA1 Message Date
Sreejith Punnapuzha
0620debbbe Add git commit hash and build date to version
This commit adds git commit hash and build date to airshipctl version
command.

Relates-To: #630

Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
Change-Id: Ic27db40cf80153cb5d3af3accacd33e85d36010d
2021-09-21 08:57:37 -05:00
Andrii Ostapenko
419300b277
Fix layout
Current inheritance model we use site -> type -> global is currently
violated inheriting in global phases from type/gating.

Also makes capd job voting.

Change-Id: I9dd6e8b9050fa37b7f38690aebf4fcae71771b80
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2021-09-20 15:54:06 -06:00
Zuul
6cdf28fb40 Merge "Clear bad defaults in AIAP/airshipctl replacements" 2021-09-20 16:44:45 +00:00
Zuul
dad459f5cb Merge "Nextgen secrets implementation with separation per cluster" 2021-09-18 06:24:57 +00:00
Zuul
00d74a4500 Merge "Allow to specify multiple clusters per get-kubeconfig request" 2021-09-18 06:24:55 +00:00
Ian Howell
b77f9e2910 AIAP: Cleanup completion statuses
This adds the `status-checker` container, which aggregates the statuses
of the task containers. This is useful for quickly checking whether AIAP
is in a failed or successful state, allowing for early exit during
gating or testing. This also prevents the containers from stopping for
any reason, allowing for easy debugging.

Change-Id: I1571d006fb3c856e4d2bedee0befdccae6082a66
2021-09-16 16:06:52 -05:00
Zuul
a99722694c Merge "Use quay.io/airshipit/golang images" 2021-09-16 20:16:24 +00:00
Zuul
ec48d6cba4 Merge "Implement gating type" 2021-09-16 20:16:22 +00:00
Zuul
d58ddbdb05 Merge "Fix CAPD deployment using phase plan" 2021-09-16 20:05:28 +00:00
Zuul
4f42338019 Merge "Removed the hardcoded wait logic" 2021-09-15 23:28:17 +00:00
Zuul
82cab8e4e7 Merge "Add possibility to specify poll interval" 2021-09-15 21:25:28 +00:00
Zuul
ecccdd7641 Merge "Cleanup logs" 2021-09-15 18:00:30 +00:00
Ian Howell
bedca67420 Clear bad defaults in AIAP/airshipctl replacements
Change-Id: I17dd940448c1263f00e865239b6e69201e61d55b
2021-09-15 10:20:05 -05:00
Siraj Yasin
c7be426de3 Removed the hardcoded wait logic
* Removed the hardcoded wait condition ( 30s delay )
  for docker and libvirtd service
* Added logic to wait until docker is available for artifact-setup.
* Added logic to wait until libvritd is available for infra-builder.
* Added "imagePullPolicy: IfNotPresent" to make use of local changes.

Closes: #629
Change-Id: I8956cf5927be6142fcc0a3b1a112ea82511fd9bb
2021-09-15 15:12:53 +00:00
Ruslan Aliev
8b405161fc Use quay.io/airshipit/golang images
Currently we use golang images from dockerhub, which sometimes
entail pull rate limit error.

Change-Id: Id8bca8c17c89556c8c424bbdaa4b11efe66d312c
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Closes: #635
2021-09-14 15:19:50 -05:00
Zuul
d5690e20fe Merge "AIAP: Update docs" 2021-09-14 15:26:19 +00:00
Ruslan Aliev
b67242f532 Add possibility to specify poll interval
Currently, there is no way to specify poll interval for
kubernetes-apply phases, however such possibility exists
in cli-utils API and could be helpful for tuning apply
process.

Change-Id: I605767d589920d0f1e10343afd9672c8d615a46a
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2021-09-13 22:09:16 -05:00
Ian Howell
fd04e9cd00 AIAP: Update docs
This change aims to clarify the differentiation between deploying AIAP
to Azure and deploying to a native Linux environment.

Change-Id: I3e014589dc02cce1bac5ff32a69c29ef6fc5b9f3
2021-09-13 09:17:30 -05:00
Matthew Fuller
0fd3e16f2c Implement gating type
Fully implements gating type to be used by all provider types.

Closes: #575

Change-Id: I30ebb76466b3c273519ca42a6f81d7df0ef52800
2021-09-10 21:46:53 +00:00
Ian Howell
fc20c79e47 AIAP: Keep containers alive in successful runs
Currently the task containers will reach a conclusive state (success or
failure), and will promptly stop running. This creates the following
problems:
  1. In the case of failure, a user doesn't have the ability to exec into
     the container for debugging purposes.
  2. In the case a success, the readiness probe for each task container
     will begin to fail. This happens because readiness is defined by
     the existence of a file; if the container stops running, the file
     no longer exists.

This change provides a solution to the 2nd bullet, keeping the container
alive in case of success.

Change-Id: Iae432caada3d2a338493b1534da72957808185d2
2021-09-10 11:21:42 -05:00
Zuul
885187f37c Merge "AIAP: Fix filepath name for manifests" 2021-09-09 15:30:30 +00:00
Ian Howell
e241430712 AIAP: Update documentation
This updates the documentation for working with AIAP. This includes
changes to the README as well as adding all configurable values to the
examples (having their default values).

Change-Id: I7c1b92d6a324b0fbd07f27ec0607fc62765c4fa2
2021-09-09 00:10:17 +00:00
Ian Howell
b921364ece AIAP: Fix filepath name for manifests
AIAP was unable to modify certain manifest files when the manifest url
ended in `.git`. This fixes that by stripping off the file extension
from the target path directory.

Change-Id: I3915deeb109fb1df726b27e1dd1deb1c987e39c5
2021-09-08 16:38:29 -05:00
Ian Howell
cde9648718 AIAP: Cause AIAP to fail faster
Currently, when a container in AIAP fails, the container continues to
run in the backgroud. This causes the container to be marked as failed
so that the deployment will fail faster.

Closes: #626

Change-Id: I34de366390b6ddc0ef434f7e642e73797a790edf
2021-09-07 17:02:52 -05:00
Ian Howell
b32e9b669b AIAP: Utilize document pull
Closes: #623

Change-Id: I559c160eb33710c3e3105e5454d035a833ce4abf
2021-09-07 16:54:14 -05:00
Zuul
3a537cc859 Merge "AIAP: Export KUBECONFIG var to environment" 2021-09-07 21:44:54 +00:00
Zuul
7dab6429d6 Merge "AIAP: Cleanup kubeconfig after running aks script" 2021-09-07 19:53:54 +00:00
Zuul
5ccc79ab18 Merge "Configure AIAP pod anti-affinity" 2021-09-07 19:33:26 +00:00
Zuul
ccc5bbb5e4 Merge "AIAP: Add environment variables for auth settings" 2021-09-07 16:41:21 +00:00
Zuul
e24d828f73 Merge "Update 22_test_configs.sh to use new document pull" 2021-09-07 16:41:20 +00:00
Zuul
be518102df Merge "Add auth to template for 22_test_configs.sh" 2021-09-07 16:00:34 +00:00
Ratnopam Chakrabarti
441ef0e3da Fix CAPD deployment using phase plan
This PS introduces phase plan for CAPD provider
 * Adds phase to merge kubeconfig
 * Patches type/gating plan to use capd phase plan
 * Patches cluster_map and executor to work with "default" namespace
 * Fixes CAPD zuul failures

Relates-To: #564
Relates-To: #580
Relates-To: #587
Change-Id: I5007970c907bc87dccf6dd9fcb052afc1b5c13f7
2021-09-07 15:00:25 +00:00
Ian Howell
f6f2372a8b AIAP: Cleanup kubeconfig after running aks script
The `az aks get-credentials` command adds a context, user, and cluster
to the kubeconfig. This cleans that up when CLEANUP_GROUP is true.

Change-Id: I6563d278e4700363d4d33c2ab746ac18dd0e51dc
2021-09-04 19:04:38 +00:00
Alexey Odinokov
e2c56108ee Nextgen secrets implementation with separation per cluster
1. Extending templater with kyaml functions and creating combined catalogue
to be able to request/update the existing resources.
This is based on 'everything is transformer' concept introduced in kustomize 4.x
That includes gathering all secrets into 1 variable catalogue and
special mechanism to regenerate/merge with manual secrets.

2. Implementing 'catalogue per cluster' approach for secrets.

3. Rearranging secrets so it's possible to use:
pgp (each person may have his own key), age, Hachicorp Vault and etc
and the list of people who can decrypt documents is set in a special file.
Since in some cases there should be a separate list of people who can decrypt
data - this list is set for each cluster (ephemeral and target) separatelly.

Closes: #586
Change-Id: I038f84dd138d5ad4a35f4862c61ff2124c2fd530
2021-09-03 20:46:15 +00:00
Ian Howell
0cb8e0f5f4 AIAP: Parameterize base image
This parameterizes the base image location used for the libvirt,
artifact-setup, infra-builder, and runner images.

Closes: #628

Change-Id: Idca33b5571017700fead0eb758af1900b4a4fcf4
2021-09-03 12:32:38 -05:00
Zuul
362cc4b880 Merge "Add a script to run AIAP in AKS" 2021-09-03 14:35:52 +00:00
Ian Howell
4dffb330ec AIAP: Add environment variables for auth settings
This adds the required auth variables to the runner environment so that
a user can configure passwords/keys needed to pull manifests from a
private repository.

Closes: #627

Change-Id: I185a86c0c82be38a0d41bb69f425a8fac8b5b913
2021-09-03 09:20:29 -05:00
Ian Howell
1d09fa090c Add auth to template for 22_test_configs.sh
This adds the following vairables to be used with the 22_test_configs.sh
script, so that it can generate configs which have access to private
repos.

* AIRSHIP_CONFIG_MANIFEST_REPO_AUTH_TYPE
* AIRSHIP_CONFIG_MANIFEST_REPO_AUTH_KEY_PASSWORD
* AIRSHIP_CONFIG_MANIFEST_REPO_AUTH_KEY_PATH
* AIRSHIP_CONFIG_MANIFEST_REPO_AUTH_HTTP_PASSWORD
* AIRSHIP_CONFIG_MANIFEST_REPO_AUTH_SSH_PASSWORD
* AIRSHIP_CONFIG_MANIFEST_REPO_AUTH_USERNAME

Relates-To: #627

Change-Id: I791e45eb6747e76f0e69a7cdcfde90db570fa1d1
2021-09-03 09:16:36 -05:00
Zuul
22de5f130f Merge "Temporarily disable AIAP script runner" 2021-09-03 06:39:21 +00:00
Zuul
9e0e8cd86a Merge "AIAP: Remove tty setting from libvirt container" 2021-09-03 05:58:18 +00:00
Matt McEuen
a67286151f Configure AIAP pod anti-affinity
This adds an anti-affinity rule to airship-in-a-pod, so that only
a single AIAP can run on a given node at a time.  This is currently
required because of the way AIAP configures host mounts; future work
will allow multiple AIAPs to co-exist on a node, at which point this
anti-affinity rule can be taken out.

Change-Id: Ib8f5062a3bd2b4ea6d0a95c42cce08ac9789c362
2021-09-02 19:59:16 -05:00
Matt McEuen
47de62a11c Add a script to run AIAP in AKS
This adds a script that a developer can use to run airship in a pod
in a dynamically created AKS cluster.  By default a resource group is
created and then cleaned up following the test.

Further work needs to be done to surface some indication of a failed
test out of the pod, and watch for that condition in this script.

Change-Id: I29f462b9becd6d1de0a9e0e1d7877c6019cbd6ee
2021-09-02 18:11:10 -05:00
James Gu
3298be2d67 Allow one single qcow2 image in qcow bundle
Allow one image for both controller and worker nodes without having to
override site manifests

Change-Id: Ia8000051026de1897ccf487877711459a88c8ce8
2021-09-02 21:00:42 +00:00
Ian Howell
e8d8b1e135 AIAP: Remove tty setting from libvirt container
The `tty` setting may be required in some instances dependent upon the
environment in which AIAP is deployed. However, these instances are much
less frequent than successful deployments when the setting is set to
false. Further investigation will be needed to determine when it is
appropriate to use this flag.

Change-Id: Ibbd44cbf0697a71f8081cd170dbdf5f7b4e4f106
2021-09-02 13:28:59 -05:00
Ian Howell
3d2372c0d7 Temporarily disable AIAP script runner
Due to time constraints for several highly valuable feature changes to
AIAP, this change temporarily disables the AIAP script runner, with
plans to re-enable it as soon as all relevant features have merged.

Change-Id: I2c085ed56b8a6bb03a63c06070112b6923e2cfbe
2021-09-02 12:20:29 -05:00
Ian Howell
95a43b48ae Update 22_test_configs.sh to use new document pull
This causes the 22_test_configs.sh to create an airship config file
which supports the new document pull feature introduced in
cb080a2066

Change-Id: I057bd2420f488a9f91ed69d09f73b1336827e7a2

Relates-To: #623
Change-Id: I7bbf052877e4918e168fc4005ea899aa700bae41
2021-09-02 09:09:12 -05:00
Zuul
d78f3e5c11 Merge "skip generating etherenet_mac_address when null" 2021-09-01 07:39:59 +00:00
Zuul
bd3b7f8436 Merge "Add proxy variable to AIAP" 2021-09-01 07:39:57 +00:00
Zuul
fe825bc4c4 Merge "AIAP: Fix SOPS environment variables" 2021-09-01 04:37:12 +00:00
Battina, Sai (sb464f)
409f9f8833 Add proxy variable to AIAP
This PS add's proxy env variables to artifact-setup
infra-builder, and runner containers in aiap pod to
allow users to pass proxy configuration id required.

Change-Id: I9747597c3534e70f4b69651c044a9cfe21ebc06d
2021-08-30 16:43:59 -05:00