86 Commits

Author SHA1 Message Date
Vladislav Kuzmin
3de8b5b2b2 Implement kustomize sink in generic container executor
Sink function allows to write configurations to an external system.

Change-Id: If9c6904239a542ea4c2bef2920965b6d87feb1e6
Relates-To: #202
Relates-To: #369
2021-01-13 17:56:04 +04:00
Ruslan Aliev
5a6f1096eb Groom phase/executors package
All the duplicated methods were removed. Unit tests reorganized.
Test coverage increased.

Change-Id: I0f7bd3eea63c101195ea50c0369e62945d73f297
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Closes: #432
2021-01-12 16:31:43 -06:00
Zuul
932d92664e Merge "Add PhaseEntryPointBasePath to phase helper" 2021-01-12 19:40:38 +00:00
Zuul
78e0e81bc0 Merge "Bootstrap Ephemeral - Executor methods" 2021-01-11 21:02:20 +00:00
Sidney Shiba
4b31d3b936 Bootstrap Ephemeral - Executor methods
This patchset includes the go code for the Executor for the Bootstrap
container/Ephemeral cluster and respective unit test code.

This executor allows to deploy a K8S cluster on a public/private cloud
provider through a dedicated Bootstrap container.

This patchset does not include the manifests for the executor to avoid
too big patchset. It will be included in a different patchset.

Change-Id: I3f9d2ba1bb65b344522e105fd26310af24e3a1b1
2021-01-08 10:34:01 -06:00
Vladislav Kuzmin
49c0ad4a23 Add PhaseEntryPointBasePath to phase helper
This parameter allows to get full path to current site.
Also change phaseRoot to phaseBundleRoot and
change the corresponding methods.

Change-Id: Ia4b1b55ceaa8c544ed429ec314bc8772a2fee6de
Relates-To: #379
2021-01-07 16:02:04 +00:00
bijayasharma
f9378e74b0 Add phase tree command in package module
* This commit is the smaller patchset from earlier work:
  https://review.opendev.org/#/c/750449/

* This is the first series of patchset

Change-Id: Ib1ff74eba65de7c7c59cf8f4fd26b15e388ba368
Signed-off-by: bijayasharma <vetbijaya@gmail.com>
Relates-To: #296
2021-01-06 19:38:54 +00:00
Vladislav Kuzmin
53ef504bd6 Refactor generic container executor
Refactor executor functions in go lang style

Change-Id: I81f5ff32e0ac354790c7afbc0bf4db91f58397b4
Relates-To: #202
Relates-To: #369
2021-01-05 19:56:25 +00:00
Zuul
b3cc2e4410 Merge "Improve document transformation in container executor" 2020-12-18 12:28:14 +00:00
Anderson, Craig (ca846m)
2daacf5f2a image-builder integration for ISO builds
This introduces airshipctl integration with image-builder [0], which
replaces the existing isogen tool for ephemeral ISO generation.

The airshipctl isogen executor has been updated for building ephemeral
ISOs using the image-builder container. The ability for user-declared
filenames for cloud-init user data and network data was removed, since
the user's only interest is in supplying the relevant overrides, not in
transparent naming coordination with the image-builder container. A new
object is added to the document package to identify the document kind,
label, and key to retrieve data from since this is pattern we will
reuse elsewhere.

Progress flag removed as requsted. Progress is reported directly by the
image-builder container.

Isogen debug flag removed in favor of using log.DebugEnabled()

[0] https://review.opendev.org/#/c/730777/

Depends-On: https://review.opendev.org/c/airship/images/+/730777/
Change-Id: I545004feaf2116f8ffb29faf6f7f7f5fcfe24fff
2020-12-17 14:47:53 -08:00
Vladislav Kuzmin
7f040fed97 Improve document transformation in container executor
Start using yaml utils to prepare documents for
function input.

Change-Id: I4adf4781d10ff8c23eea4bec4e85f8c5b4769a8b
Relates-To: #202
Relates-To: #369
2020-12-17 18:32:23 +04:00
Zuul
4e7053bfb5 Merge "Stop clusterctl execution if error is encountered" 2020-12-16 16:52:12 +00:00
Zuul
b6d52343dc Merge "Allow to perform phase render using document entrypoint" 2020-12-16 16:51:46 +00:00
Kostiantyn Kalynovskyi
0f7707937b Stop clusterctl execution if error is encountered
Change-Id: Ia5b2b2bf58dbb243bf96de6ccfd66f3c354c1515
2020-12-15 13:44:00 -06:00
Kostiantyn Kalynovskyi
a6987107bd Allow to perform phase render using document entrypoint
In current setup, we rely on executor to perform render operation,
however there are many usecases, when we want to have a simple
`kustomize build` performed against document entrypoint in phase.
Now this will be a default behaviour and if a user would like
executor to perform rendering to actually understand what it's
doing, he will be able to specify --executor boolean flag.

Relates-To: #403

Change-Id: I181830f9814ff48a19ba0a1284e89900187bc7d8
2020-12-15 11:28:16 -06:00
Ruslan Aliev
01266f036e Move container phase executor to a separate package
Having an executor within container package creates
potential import cycling. This patch moves it to a separate
package which can be used to conveniently store all the executors
at one place.

Change-Id: Ibd07c2ba46d1971604bdbd9e5e360759bb68a659
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #432
2020-12-14 13:16:35 -06:00
Ruslan Aliev
71dc1d4703 Move clusterctl phase executor to a separate package
Having an executor within clusterctl.client package creates
potential import cycling. This patch moves it to a separate
package which can be used to conveniently store all the executors
at one place.

Change-Id: Ib0a6072a393e68885d9ef911aa2894a0de055668
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #374
2020-12-11 14:09:41 -06:00
Ruslan Aliev
b85f8fa8f9 Move kubernetes-apply phase executor to a separate package
Having an executor within applier package creates
potential import cycling. This patch moves it to a separate
package which can be used to conveniently store all the executors
at one place.

Change-Id: I68d909489b691e4fb7129446ef9a3fb085f8683c
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #432
2020-12-11 14:09:41 -06:00
Ruslan Aliev
36303006af Move isogen phase executor to a separate package
Having an executor within isogen package creates
potential import cycling. This patch moves it to a separate
package which can be used to conveniently store all the executors
at one place.

Change-Id: I2154c0af6c28f47abb2448863332766260106759
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #432
2020-12-11 14:09:41 -06:00
Vladislav Kuzmin
258b81122e Add generic container executor
This executor based on kyaml.RunFns[1].
It allows to execute arbitrary container as RunFns function
and handle any input data as ExecutorBundle.
Be aware that runtimeutils.FunctionSpec used for version 0.4.1[2]
and it can be changed in the future. It means that yaml scheme
of executor can be changed during kyaml uplift.
This executor transforms executor yaml to RunFns function acceptable format.

[1] https://github.com/kubernetes-sigs/kustomize/blob/master/kyaml/runfn/runfn.go
[2] https://github.com/kubernetes-sigs/kustomize/blob/kyaml/v0.4.1/kyaml/fn/runtime/runtimeutil/functiontypes.go#L22

Change-Id: I9eb648783f1ed462d2b8359d9dc86320c4db2a83
Closes: #202
Closes: #369
2020-11-26 17:11:56 +04:00
Kostiantyn Kalynovskyi
c89cd5fba0 Small improvement to Events processor interface
In this commit, we make sure that there are no left over channels
after phase.Run is complete

Change-Id: Ia9b7a37f19aad892e77fdca5ad02efe8322e612b
2020-11-17 14:56:22 -06:00
Michael Beaver
193dc935f1 Refactor subPath out of config manifests
This change removes the SubPath entry portion of the config manifests, and
refactors several areas that used it. This also updates unit tests to
reflect the new changes.

Closes: #255

Change-Id: Ibbb519a3f1fb28a1594e971274c5de54b99c867e
2020-11-04 09:30:03 -06:00
Zuul
0f287089d9 Merge "Add check for executor ref in phase helper" 2020-10-27 22:32:32 +00:00
Vladimir Kozhukalov
a608d4c56d Add docEntryPointPrefix field to the metadata.yaml
* The phase.docEntryPointPrefix field in the metadata.yaml
   allows you to define a common part of the documentEntryPoint
   for all phases in the phase bundle.

   For example, let metadata.yaml be
   ---
   phase:
     path: manifests/phases
     docEntryPointPrefix: manifests/site/test-site

   and a phase be
   ---
   apiVersion: airshipit.org/v1alpha1
   kind: Phase
   metadata:
     name: initinfra-ephemeral
     clusterName: ephemeral-cluster
   config:
     executorRef:
       apiVersion: airshipit.org/v1alpha1
       kind: KubernetesApply
       name: kubernetes-apply
   documentEntryPoint: ephemeral/initinfra

   Then the documentEntryPoint for executor will be prepended with
   docEntryPointPrefix and the path to the executor bundle will be
   manifests/site/test-site/ephemeral/initinfra

Change-Id: I29ec14378790d95b66c3ff1fe6120bb200f91a50
Relates-To: #356
2020-10-22 13:29:05 +03:00
Zuul
23224e57c5 Merge "Phase docs are targetPath + phaseRepo + phase.DocumentEntrypoint" 2020-10-16 15:02:04 +00:00
Vladimir Kozhukalov
dc68640389 Phase docs are targetPath + phaseRepo + phase.DocumentEntrypoint
Change-Id: I1d5ea75d1c19eb4ebaf37b28918ccde771bcef85
Relates-To: #356
2020-10-15 13:55:59 +03:00
Ruslan Aliev
a6363f1c1e Run image build command like a phase
We perform almost the same activities prior calling createBootstrapIso
function like we do that in phase executor, also image build command
now is techically outdated since it doesn't work because it fails with
"phase document was not found" error. This patch removes unnecessary
functions like GenerateBootstrapIso and runs image build command in
a phase way. Test coverage increases slightly (by 0.2%) by removing
partially duplicating code which was not tested properly. Minor bug
fixes included.

Change-Id: I545004cea721164838b296ae647a7651cde248ff
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-10-13 17:48:55 -05:00
Zuul
f92e8bd042 Merge "Replace PrimaryRepositoryName with PhaseRepositoryName" 2020-10-13 20:50:53 +00:00
Kostiantyn Kalynovskyi
7095d09de2 Fix kubeconfig flag to be passed to executor
Change-Id: I6e9951785158cecfb952255426d8b5bb40be37ea
Closes: #366
Relates-To: #366
2020-10-13 01:07:49 +00:00
Kostiantyn Kalynovskyi
eddfc6338a Add check for executor ref in phase helper
Relates-To: #365
Closes: #365

Change-Id: I95251a0ef49436d86f6f882bf23a91ceb1b1a952
2020-10-12 13:41:48 -05:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Ruslan Aliev
fc33287204 Remove kubeconfig from config struct
Change-Id: I5ab8722dd151d9e652e0b20e1d82988cd98505af
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-10-07 14:35:07 -05:00
Zuul
29a6906efc Merge "Add test to phase bundleFactory" 2020-09-30 14:26:37 +00:00
Zuul
e4a2c68d3e Merge "Use BundleFactory instead of bundle in executors" 2020-09-30 14:24:52 +00:00
Zuul
d4ea864eeb Merge "Fix ListPhases to return not empty phase docs" 2020-09-29 17:47:31 +00:00
Kostiantyn Kalynovskyi
300486482c Add test to phase bundleFactory
The test makes sure that correct error is returned when bundle
factory fails, and increases test coverage of executor() function

Change-Id: Ibfc5d8ea1941a5d4d304a8afaedc5c89efe7c398
2020-09-29 00:40:15 +00:00
Kostiantyn Kalynovskyi
e6e56ffa2e Use BundleFactory instead of bundle in executors
This will allow to avoid using nil bundle and performing
additional checks inside the executors

Change-Id: Ia061cccf7e0a7948bbc5c8fce24e0e6f8b3256e1
2020-09-28 19:39:05 -05:00
Zuul
e70187641a Merge "Add timeout flag for phase run in cli" 2020-09-28 22:06:44 +00:00
Sirajudeen
3e4c228638 Add timeout flag for phase run in cli
Syntax:
	airshipctl phase run <PHASE_NAME> --timeout <time_unit>

Exmaple:
	airshipctl phase run initinfra-ephemeral --debug --wait-timeout 1000s

Change-Id: Ic8c699f5302eb482f32aee0e7d7c593df2ed7d3b
2020-09-28 16:58:18 +00:00
Kostiantyn Kalynovskyi
065888d6c7 Add functions for default api objects
This allows to avoid nil pointers when working with unmarshaled
objects, that may be missing some fields.

Next step would be to revisit why pointers are used in such obj
as Clusterctl and ImageConfiguration

Change-Id: I99a711529a53943ef1b0cb1ebb4d240f2482fac8
2020-09-23 18:37:46 -05:00
Ruslan Aliev
2f5402ae80 Remove phase apply command
Command phase run will be used instead, so phase apply became no
longer needed.

Change-Id: I8bdbd6bcad7efbb03632978d8952700fc39d55ba
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-23 01:42:16 -05:00
Ruslan Aliev
0c736af2e8 Implement render methods
Change-Id: I03b7aad33576eaa9ef7dfaabfb579ac90cc26a12
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-23 03:06:51 +00:00
Kostiantyn Kalynovskyi
9f01c87648 Fix ListPhases to return not empty phase docs
Change-Id: I44dd36c41279b34357d6d86a92fc49bfca7b871b
2020-09-18 15:47:28 -05:00
Vladislav Kuzmin
27e0ab455c Add isogen executor
Relates-To: #344
Change-Id: I1b9b04f1f723d8df6d6aad63b64a917566b22176
2020-09-18 13:49:01 +04:00
Ruslan Aliev
5d52339bd6 Remove clusters, users, reconcilation and config import
All of these entities are no longer required and should be deleted.

Change-Id: Id4776efe668265df6961a38ce984b8ac37b27097
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-16 18:35:12 -05:00
Zuul
3601c2b59c Merge "Cleanup phase package" 2020-09-15 20:23:36 +00:00
Kostiantyn Kalynovskyi
285130106a Use cluster name as context in executors
Change-Id: I4fcf524dd2ff466290b80df7d966f90aa0409471
Relates-To: #342
2020-09-15 12:29:16 -05:00
Kostiantyn Kalynovskyi
b63c9e3888 Cleanup phase package
Change-Id: Ib849da5faa2f69f058bd9df814b55e117ecefb8a
Relates-To: #342
2020-09-15 17:27:40 +00:00
Kostiantyn Kalynovskyi
c46cdebe48 Add phase command objects
Commit adds phase command object that allow easy calls from cmd
package.

Change-Id: I23a9ab3755dbe366ad9b7ce163f75f64f48148cd
2020-09-12 15:21:12 +00:00
Kostiantyn Kalynovskyi
d79c71e94d Implement phase and phase.Client interface
Relates-To: #342

Change-Id: I8882204c0e9eae05fca30c093fcd3bad58e308e1
2020-09-11 16:24:04 -05:00