155 Commits

Author SHA1 Message Date
uday.ruddarraju
9a608de653 Encrypt and decrypt using sops
Design document: https://docs.google.com/document/d/1EjiCuXoiy8DEEXe15KxVJ4iWrwogCyG113_0LdzcWzQ/edit?usp=drive_web&ouid=102644738301620637153

Demo readme: https://hackmd.io/@WE7PUWXBRVeQJzCZkXkOLw/ryoW-aOLv

This patchset comprises of:
- package library to interact with sops
- integrate airshipctl encrypt/decrypt with sops

Change-Id: I2ca3ff3c8661d146708084728cb3f87365a4f39e
2020-10-23 02:27:45 -07:00
Vladimir Kozhukalov
a608d4c56d Add docEntryPointPrefix field to the metadata.yaml
* The phase.docEntryPointPrefix field in the metadata.yaml
   allows you to define a common part of the documentEntryPoint
   for all phases in the phase bundle.

   For example, let metadata.yaml be
   ---
   phase:
     path: manifests/phases
     docEntryPointPrefix: manifests/site/test-site

   and a phase be
   ---
   apiVersion: airshipit.org/v1alpha1
   kind: Phase
   metadata:
     name: initinfra-ephemeral
     clusterName: ephemeral-cluster
   config:
     executorRef:
       apiVersion: airshipit.org/v1alpha1
       kind: KubernetesApply
       name: kubernetes-apply
   documentEntryPoint: ephemeral/initinfra

   Then the documentEntryPoint for executor will be prepended with
   docEntryPointPrefix and the path to the executor bundle will be
   manifests/site/test-site/ephemeral/initinfra

Change-Id: I29ec14378790d95b66c3ff1fe6120bb200f91a50
Relates-To: #356
2020-10-22 13:29:05 +03:00
Ruslan Aliev
0c870e5244 Remove clustertype-related functions
We don't use clustertype anymore, so all the related functions must
be deleted or adjusted as well as appropriate config fields.

Change-Id: I3931fdc71d4318e916f8bbc2d94e062c9df5f641
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #349
2020-10-16 11:31:58 -05:00
Vladimir Kozhukalov
dc68640389 Phase docs are targetPath + phaseRepo + phase.DocumentEntrypoint
Change-Id: I1d5ea75d1c19eb4ebaf37b28918ccde771bcef85
Relates-To: #356
2020-10-15 13:55:59 +03:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Ruslan Aliev
fc33287204 Remove kubeconfig from config struct
Change-Id: I5ab8722dd151d9e652e0b20e1d82988cd98505af
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-10-07 14:35:07 -05:00
Zuul
7502da5c34 Merge "Add document pull step to gate script runner" 2020-09-28 22:45:02 +00:00
Zuul
462b8d8442 Merge "Fix TestPersistConfig" 2020-09-28 22:13:37 +00:00
Ruslan Aliev
d8a6139e81 Add document pull step to gate script runner
Currently we don't test document pull step by rewriting
manifest_directory variable as current source path, which
makes a mess in target_path, manifest_path and related paths
(concatenation of target_path+manifest_path won't return
an existing and proper location). This patch organizes
mentioned variables and enables document pull step which
starts working properly. The latest repo state used as
git source for this command.

Change-Id: I5abce73877441c7529f2f77add79cf410e2226d8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-25 00:39:28 -05:00
Sirajudeen
2769c62c8a Update metadataPath using set-manifest in cli
* Added metatada-path as a flag to set-manifest

Change-Id: Id26f20c4a6350b4aae34da850bbfcab42cbe9f21
2020-09-24 19:33:15 +00:00
bijayasharma
e99adf50ad Removed K8s Authors
To make it consistent, this commit will remove K8s authors
from copyright section.

Change-Id: I4e670cc76ea1feb6456965707a8deff258e5f215
2020-09-23 11:36:43 -04:00
Ruslan Aliev
2e85043c20 Fix typos
Change-Id: I0ce87ec00d2b5974cf861d9c9685f23d7d0f5fc1
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-21 02:49:33 -05:00
Ruslan Aliev
5d52339bd6 Remove clusters, users, reconcilation and config import
All of these entities are no longer required and should be deleted.

Change-Id: Id4776efe668265df6961a38ce984b8ac37b27097
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-16 18:35:12 -05:00
uday.ruddarraju
f328c43295 Adding encryption config to airshipctl config
Design document: https://docs.google.com/document/d/1EjiCuXoiy8DEEXe15KxVJ4iWrwogCyG113_0LdzcWzQ/edit?usp=drive_web&ouid=102644738301620637153

This patchset comprises of:
- airship config now supports encryption configs to store encryption and decryption keys
from local file system or the kubernetes api server that will be used to encrypt
and decrypt secrets in a future patchset

This is the first of multiple patchsets to support encryption and decryption in airshipctl

Complete feature: https://review.opendev.org/#/c/742695/

Change-Id: I195e8e254b7cc6b3e04e45d67e0a0e3797183816
2020-09-15 14:11:04 -07:00
Vladimir Kozhukalov
b76ddc98ac Fix TestPersistConfig
testutil.InitConfig writes sample configs to files,
 and if we just check that these files exist after
 calling PersistConfig function, then don't test anything.

 We have to change config file paths before calling PersistConfig
 and then we can check if these files exist.

Change-Id: Ibaee17a57a86251f7016bb57b2d8bd9976cefac0
2020-09-15 14:13:47 +03:00
Kostiantyn Kalynovskyi
76b1ffd722 Implement helper interface
Change-Id: If65d7d8cd3fa082facc077753bb06275da2a6643
Relates-To: #342
2020-09-10 15:05:00 +00:00
Ruslan Aliev
fba618225a Remove AirshipCTLSettings from root level cmd
AirshipCTLSettings will no longer be used by any command, so this
patch removes it from root level.

Change-Id: I2aa1d80ab0785b498af1ac2c86334f9b0d9ff047
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-03 23:52:49 -05:00
Ruslan Aliev
2166062f2e Refactor config init command
Config init command will no longer use AirshipCTLSettings for
defining a config object which needs to be saved.

Change-Id: I303f05a8df1e7439d1b3dcfd41cf2679d729a9f8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-03 23:52:49 -05:00
Ruslan Aliev
91529ca046 Introduce new approach of config object loading
The new approach of config loading allows us to combine and move all
the config loading and verifications actions from cmd to pkg module;
it will be executed on demand only and save us from non expected
runtime errors. The examples of usage will be introduced in further
changes.

Change-Id: I1b4c29f9c5b1de3e46a1faf5c28fea5c7b3eb4e1
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-02 23:41:05 -05:00
Zuul
eaa66fded7 Merge "Fixed TODO in package config" 2020-09-01 22:17:51 +00:00
Vladimir Kozhukalov
e9c8425c30 Move bootstrapInfo config section to manifests
This is a part of two activities
  * Refactor config and store all parameters to document model
  * Implement everything as phases (which are supposed to be purely document driven)

 This patch removes bootstrapInfo section from the airshipctl config and
 makes these to commands
  * airshipctl baremetal isogen
  * airthipctl baremetal remotedirect
 to take necessary parameters from documents

 We introduce two airship API kinds ImageGenerator and RemoteDirect.
 Instead of storing config parameters in cm/secrets we use these two
 API objects.

Relates-To: #246
Closes: #254
Change-Id: I42903c45dce1c73da184c07277fec76fd88c700f
2020-08-31 16:54:46 +03:00
Stanislav Egorov
88da96cdf5 Fixed TODO in package config
Implemented strongly-typed errors for config

Change-Id: I6c91f5abbdba2fb89b831dd897808453d172a826
2020-08-28 22:56:41 -07:00
Zuul
f396702824 Merge "Exclude kubeconfig from use-context update" 2020-08-20 05:00:23 +00:00
Alexander Noskov
582894d389 Switch to Ubuntu focal (20.04) based image
Since we have switched from debian to ubuntu based image and changed the
name (in https://review.opendev.org/733078) we need to align it with
airshipctl repository.
Renaming the following names:
1) debian-custom.iso -> ubuntu-focal.iso
2) quay.io/airshipit/isogen:latest-debian_stable -> quay.io/airshipit/isogen:latest-ubuntu_focal

Updating the following packages and resources:
1) k8s 1.17.3 -> 1.18.6
2) docker 19.03.9 -> 19.03.12
3) Calico 3.9 -> 3.15

Change-Id: I7eaf382acb79016a511db6e0955fa932c02963c4
2020-08-13 13:11:25 -05:00
niharikab
315ed5994b Implements proper permissions on airship2 generated files
*This commit sets default permissions for .airship directory & config
  file.
*It allows operators to override default permissions in config file

Closes: #188

Change-Id: I5713662ecc71b71c0d0bd0f61691d326c7727b0f
2020-08-11 01:05:41 +05:30
Vladislav Kuzmin
43fad9ad85 Replace Non-ASCII symbols
Update golanci version, because we need to add asciicheck linter.
Because of update appears some gosec issues.

Closes: #312

Change-Id: I54759e7301f0fb9dcb60eb7d9a2d1e9cebfaf359
2020-08-05 13:22:59 +00:00
muly
9d4347b0c1 documentation changes
1) modifications to exported functions/variables documentation
2) moved some methods into a different folder

Change-Id: I212442516424bde3284f75b326d152c290d20c80
2020-07-29 09:17:00 -04:00
bijayasharma
241714d042 [#308] Removed code style issue on Error string
* This commit will remove code style issue on Error string

Change-Id: Ib2344de75768362a88c077354193878366ce4b0c
2020-07-24 14:19:57 -04:00
Vladislav Kuzmin
d12c42d043 Exclude kubeconfig from use-context update
Since kubeconfig was moved to manifest we don't
need to change it during use-context update.

Change-Id: Ib6a0ff19eee89bc86a343e374d50938482e33941
Closes: #294
2020-07-16 18:58:31 +04:00
Kostiantyn Kalynovskyi
66fc562667 Fix metadata loading by config module
In this fix, we make sure that targetPath is taken into the account
when forming path to metadata file

Change-Id: Ie94c3508161b8c3b9254db08f4bf809348ff54f0
Relates-To: #295
Closes: #295
2020-07-15 19:18:35 -05:00
Sirajudeen
35d119d3d1 Added unit test for dochelper, config
* added more negative test cases to dochelper, config to
  increase the test coverage

Change-Id: I3179823f9196157e72009ff998cfb8dd348fee1f
2020-07-14 16:48:03 -05:00
Ruslan Aliev
37c044a5b7 Fix config init command
airshipctl config init command currently fails with "no such file or directory"
error, this patch fixes it.

Change-Id: I78cff26ed71bdb400ce897630102350e72dc495d
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-07-14 15:50:29 +00:00
Zuul
9892e5ea81 Merge "Add repository metadata" 2020-07-10 21:57:50 +00:00
Zuul
bfa6344335 Merge "Remove AirshipError" 2020-07-08 17:43:06 +00:00
Ruslan Aliev
0314c5c848 Return a proper error if phase document was not found
Change-Id: I0b3b7a819b6c76fdc2e20ee2f6f8ece047fac858
Relates-To: #257
Closes: #257
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-06-29 19:38:15 +00:00
Kostiantyn Kalynovskyi
852b0ad29e Add repository metadata
This commit adds repository metadata object and method to load it
from manifests.

Metadata is to be changed and extended in the future, when phases
start to be implemented.

Change-Id: Idc9374220f62df8b1a80c276278361efa4503e12
Relates-To: #263
Relates-To: #255
Relates-To: #240
2020-06-29 13:33:04 -05:00
Sreejith Punnapuzha
a1226660d6 New CLI option to get/set manifest
This commit adds get-manifests and set-manifests option for airshipctl
config. get-manifests prints all the manifests in current config file.
set-manifests helps in setting/creating new manifests in current config
file.

Closes: #115
Change-Id: I7c677ed491a1d456744d27bf105f21e2b3c4e6aa
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-25 18:47:51 -05:00
Ruslan Aliev
ad7f80f2bf Return an error if there is a problem to access config file
Currently, if config file doesn't exist - airshipctl doesn't
throw an error and uses empty config object instead. It seems
this behavior was an attempt to hide an error if there was a call
of subcommand which is not require proper config file. Since we
moved config load calls to the specific subcommands from
root level - current behavior should be changed and we have to
throw an error instead if it's not possible to access config file.

Change-Id: I7e949fbc9d48d682e26500e0b6b6b9bd12665e24
Relates-To: #199
Closes: #199
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-06-12 11:22:47 -05:00
Zuul
ccb83371ec Merge "Add set-management-configuration command" 2020-06-10 16:23:06 +00:00
Vladislav Kuzmin
d5e0498546 Improve config package organization pt.3
This is a series of patches that refactor config.go into
several smaller modules, each relating specifically
to one component of the configuration structure.

This commit moves the types in the modules where they are used.

Change-Id: I5bbeb765d558f02e04ebf8e41987a436874b83d4
Closes: #35
2020-06-01 11:54:25 +04:00
Drew Walters
383cffed4d Add set-management-configuration command
This change adds a set-management-configuration command that allows the
user, typically CI, to change management configuration settings
imperatively.

Change-Id: I7a106a46f57fef0ea2a8c5dd8c8786d8a103305c
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-28 21:59:44 +00:00
Ian Howell
c030bcb034 Remove AirshipError
This removes the common, as it doesn't add any additional information
when an error occurs

Change-Id: Id67e38604445a2a044aae949d933f556bb29b05c
2020-05-28 14:22:44 -05:00
Drew Walters
58ba1d94e0 Add management config validation
The management configuration does not have central validation, leading
to late validation errors. Additionally, there is an effort to provide
better config validation on a command-by-command basis.

This change adds central validation to the management configuration that
will be used several ways:

  1. For quicker validation today
  2. For a new CLI command that will be introduced in a future change
     that enables imperative modification of the management config
  3. Possibly in future efforts to provide command-by-command config
     validation by calling the exported Validate() function

Change-Id: I19eafddc818e8d478b9afd053d4ab387c7ad38b3
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-21 14:21:59 +00:00
Zuul
99234bad5d Merge "Remove EnsureComplete config object validation from GetCurrentContext" 2020-05-20 04:01:46 +00:00
Drew Walters
fa2b0eecff Fix cluster reconciliation logic
When reconciling clusters, airshipctl points the bootstrapInfo and
managementConfiguration to an entry named default, and the user is
unaware of this change. The result is that user settings are ignored
unless the user defines settings under entries named "default". This
change updates the cluster reconciliation logic to do this only when a
cluster is added to the airshipconfig by airshipctl.

Fixes #244

Change-Id: I2cefbf83a74d9cd566015509a4f768698d274746
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-19 20:59:23 +00:00
Yasin, Siraj (SY495P)
8d16fffb7c Fix Lint warnings: Missing comments for functions
* Added comments wherever missing for exported functions/constants

Change-Id: I7d0b79b5f56fc6c3edb60a3b58c0962ebd76314c
Relates-To: #148
2020-05-19 16:54:09 +00:00
Ruslan Aliev
9616cd0756 Remove EnsureComplete config object validation from GetCurrentContext
It's redundant and contradictory to call EnsureComplete method from
GetCurrentContext function since this method is called twice
during config load and we perform this check inside of appropriate
action function (like phase apply or cluster init).

Change-Id: Idb9ab7e7665dcc6d199bc96c21ae1dc6d059ac2e
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-19 01:57:24 -05:00
Matt McEuen
4e64e74b6f Add a setting for kustomize plugins
This adds an env var-driven configuration for the filesystem path
in which kustomize should expect its plugins (including airshipctl)
to be.  The value defaults to a subfolder off the `.airshipit` folder.

The config is persisted as a singleton rather than a member of
AirshipCTLSettings (or the Config) because
1. the settings object would have had to have been passed around
   a couple dozen additional files/functions,
2. it's reasonable to expect the plugin location to be consistent
   across threads in a multi-threaded, airshipctl-as-library context.

Settings_test.go was moved in to an environment_test package to avoid
a circular import dependency.

Change-Id: Icdd21bd3687ef42492e388af982d7b490af3eff3
2020-05-15 15:06:32 -05:00
Alexander Noskov
aab8325af5 Fix default checkout options in Airshipctl config
`airshipctl config init` command, generates default Airshipctl config
with mutually exclusive checkout options.
This CR fixing config generation by setting the only branch.
The `commitHash` and `tag` remain empty as an example in config file..

Change-Id: I12ec9b713e8f6769ddbd5301e000ab0242ea8ff4
Closes: #233
2020-05-14 04:35:19 +00:00
Alexander Noskov
84f2bc60c9 Fix default Airshipctl config parameters
Problems description:
Once we execute airshipctl config init, the default config in
~/.airship/config doesn't contain default values for:
currentContext: ~
clusters:
  default:
    clusterType:
      target:
        bootstrapInfo: ~
        managementConfiguration: ~
---
This PS set default values for Airshipctl config to use it
out of the box.

Closes: #234

Change-Id: I57449e678f19ab1c0e2212c38b01ea195f62aa44
2020-05-12 16:02:05 -05:00