68 Commits

Author SHA1 Message Date
Kostiantyn Kalynovskyi
decc12b0a9 Move remotedirect method to remote interface
Please note that this is first commit in series and in next commits
remote/remote_direct* files will be remove completely, so if you
have any comments that address issues in these files, keep in mind
that they will be remove in next patchsets.

Change-Id: I52e9b8438a4f8e1a2395cbe90238b9cb654772d4
2021-01-21 23:09:54 +00:00
Ruslan Aliev
94a17694ea Move MockDocument implementation to testutils
Currently MockDocument implementation located under pkg/bootstrap/isogen
package, which is not relevant for it and makes it impossible to import
in other test packages. Also _test.go filename is not proper one for
this source file since it doesn't contain any unit tests inside.

Change-Id: I9b3f28df653637701f5d6fca4376da5a22f39658
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #432
2020-12-11 14:09:41 -06:00
Ruslan Aliev
d7ca864295 Move mockContainer implementation to testutil
Currently mockContainer implementation is located under
pkg/bootstrap/isogen and it's not possible to import it from
other packages. This patch moves it to appropriate new package
under testutil.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To : #432

Change-Id: I36510e8e065434de597e7316dfbe51da0825d343
2020-12-11 14:09:41 -06:00
Ruslan Aliev
05b5ca599b Extend document filesystem with chmod and dir methods
These methods will be used instead of calling their direct
analogs in the code in order to have more consistency.

Change-Id: Ie2eaf03b1fbd049ed5788f1a769a332ef863494b
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #415
2020-12-02 09:03:17 -06:00
Ruslan Aliev
ca71de3951 Move document filesystem to a separate package
In order to widely use filesystem interface in airshipctl
there is need to move it to separate package to avoid
importing unnecessary dependencies from document
package and, as a consequence, possible cyclic dependencies.
 * filesystem moved from document/fs to pkg/fs

Change-Id: I3b6298462f03db43594a9fa26bf23ab7687c5589
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #415
2020-11-30 19:51:30 -06:00
Zuul
0c2edba495 Merge "Revert "Encrypt and decrypt using sops"" 2020-11-05 22:32:28 +00:00
Sirisha Gopigiri
f705ff0000 Revert "Encrypt and decrypt using sops"
This reverts commit 9a608de653ffe1ce8f60b5f16249e2152b3522ad.

Requesting revert because of the sops+gpg testcase failure issues. The failed testcases are impacting airship-airshipctl-lint-unit zuul gate.

For more details please refer to https://github.com/airshipit/airshipctl/issues/378

Change-Id: I60f6b5daaeebc038b68556c60413b3d0c6db6d68
2020-11-05 13:53:15 +00:00
Michael Beaver
193dc935f1 Refactor subPath out of config manifests
This change removes the SubPath entry portion of the config manifests, and
refactors several areas that used it. This also updates unit tests to
reflect the new changes.

Closes: #255

Change-Id: Ibbb519a3f1fb28a1594e971274c5de54b99c867e
2020-11-04 09:30:03 -06:00
Zuul
f60458a6a8 Merge "Remove context from BMH object." 2020-10-28 20:41:00 +00:00
Kostiantyn Kalynovskyi
690d8d88c1 Remove context from BMH object.
This is first step in refactoring remote package to support
future inventory interface, also relates to remote-direct
executor, as it simplifies usage of Host objects

Relates-To: #362

Change-Id: Ief2fece134a465916ce461960cd7d16c9e237eb7
Relates-To: #362
2020-10-26 10:25:44 -05:00
uday.ruddarraju
9a608de653 Encrypt and decrypt using sops
Design document: https://docs.google.com/document/d/1EjiCuXoiy8DEEXe15KxVJ4iWrwogCyG113_0LdzcWzQ/edit?usp=drive_web&ouid=102644738301620637153

Demo readme: https://hackmd.io/@WE7PUWXBRVeQJzCZkXkOLw/ryoW-aOLv

This patchset comprises of:
- package library to interact with sops
- integrate airshipctl encrypt/decrypt with sops

Change-Id: I2ca3ff3c8661d146708084728cb3f87365a4f39e
2020-10-23 02:27:45 -07:00
Vladimir Kozhukalov
dc68640389 Phase docs are targetPath + phaseRepo + phase.DocumentEntrypoint
Change-Id: I1d5ea75d1c19eb4ebaf37b28918ccde771bcef85
Relates-To: #356
2020-10-15 13:55:59 +03:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Ruslan Aliev
fc33287204 Remove kubeconfig from config struct
Change-Id: I5ab8722dd151d9e652e0b20e1d82988cd98505af
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-10-07 14:35:07 -05:00
Ruslan Aliev
2e85043c20 Fix typos
Change-Id: I0ce87ec00d2b5974cf861d9c9685f23d7d0f5fc1
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-21 02:49:33 -05:00
Ruslan Aliev
5d52339bd6 Remove clusters, users, reconcilation and config import
All of these entities are no longer required and should be deleted.

Change-Id: Id4776efe668265df6961a38ce984b8ac37b27097
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-16 18:35:12 -05:00
uday.ruddarraju
f328c43295 Adding encryption config to airshipctl config
Design document: https://docs.google.com/document/d/1EjiCuXoiy8DEEXe15KxVJ4iWrwogCyG113_0LdzcWzQ/edit?usp=drive_web&ouid=102644738301620637153

This patchset comprises of:
- airship config now supports encryption configs to store encryption and decryption keys
from local file system or the kubernetes api server that will be used to encrypt
and decrypt secrets in a future patchset

This is the first of multiple patchsets to support encryption and decryption in airshipctl

Complete feature: https://review.opendev.org/#/c/742695/

Change-Id: I195e8e254b7cc6b3e04e45d67e0a0e3797183816
2020-09-15 14:11:04 -07:00
Vladimir Kozhukalov
e9c8425c30 Move bootstrapInfo config section to manifests
This is a part of two activities
  * Refactor config and store all parameters to document model
  * Implement everything as phases (which are supposed to be purely document driven)

 This patch removes bootstrapInfo section from the airshipctl config and
 makes these to commands
  * airshipctl baremetal isogen
  * airthipctl baremetal remotedirect
 to take necessary parameters from documents

 We introduce two airship API kinds ImageGenerator and RemoteDirect.
 Instead of storing config parameters in cm/secrets we use these two
 API objects.

Relates-To: #246
Closes: #254
Change-Id: I42903c45dce1c73da184c07277fec76fd88c700f
2020-08-31 16:54:46 +03:00
muly
b6e92132db [#310] Declaration redundancy: Unused exported function
* removed a unused exported function

Closes: #310

Change-Id: Ib174b31cd68bdc2cf9a63dd89efd9beb8ab7d78e
2020-08-24 14:35:57 +00:00
Kostiantyn Kalynovskyi
da12ae925d Add kubeconfig interface
This will allow to use various sources for kubeconfig and provide
utility to write, save, and clean kubeconfig files, as well as
use kubeconfig interface for phase executors.

Relates-To: #264

Change-Id: Ia5b5af00700a0adf55124e1dd335d570515ed804
2020-08-17 15:22:17 -05:00
niharikab
315ed5994b Implements proper permissions on airship2 generated files
*This commit sets default permissions for .airship directory & config
  file.
*It allows operators to override default permissions in config file

Closes: #188

Change-Id: I5713662ecc71b71c0d0bd0f61691d326c7727b0f
2020-08-11 01:05:41 +05:30
Vladislav Kuzmin
43fad9ad85 Replace Non-ASCII symbols
Update golanci version, because we need to add asciicheck linter.
Because of update appears some gosec issues.

Closes: #312

Change-Id: I54759e7301f0fb9dcb60eb7d9a2d1e9cebfaf359
2020-08-05 13:22:59 +00:00
Ruslan Aliev
37c044a5b7 Fix config init command
airshipctl config init command currently fails with "no such file or directory"
error, this patch fixes it.

Change-Id: I78cff26ed71bdb400ce897630102350e72dc495d
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-07-14 15:50:29 +00:00
Kostiantyn Kalynovskyi
6a2fba6f21 Add events package
The package will have Event object, that would represent any event
that airshipctl can have, also this package provides Processor interface
that is to be used to process, print, or track progress of the executor
runners

Change-Id: I184408f7b6130653cdbabbe5e5adaac2dd758663
2020-07-07 09:44:22 -05:00
Sreejith Punnapuzha
a1226660d6 New CLI option to get/set manifest
This commit adds get-manifests and set-manifests option for airshipctl
config. get-manifests prints all the manifests in current config file.
set-manifests helps in setting/creating new manifests in current config
file.

Closes: #115
Change-Id: I7c677ed491a1d456744d27bf105f21e2b3c4e6aa
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-25 18:47:51 -05:00
Kostiantyn Kalynovskyi
6a896704b5 Move mock document file system to testutils
This commit makes mock filesystem reusable by moving it to importable
package in testutils

Change-Id: I4885480a9ef3b8c81da0058db708c32187af8e12
Relates-To: #277
Relates-To: #238
Closes: #277
2020-06-17 22:31:12 +00:00
Kostiantyn Kalynovskyi
cd7e99ac15 Add more genernic Kubectl Factory interface
Adds FakeFactory function, that allows to inject custom handlers
for HTTP and Unstructured client, which in turn enables better
testing for kubernetes api related packages

Relates-To: #276
Relates-To: #238

Change-Id: Ic27352bdc64bfccb91cc6a49afa6164e4624b1e1
Closes: #276
2020-06-17 17:30:14 -05:00
Yasin, Siraj (SY495P)
641248f46b Fix Lint warnings: Missing comments for functions
* Added comments wherever missing for exported functions/constants

Change-Id: I0942e710077ba7c6ac35d661023c87070fe0a23b
Closes: #148
2020-05-19 16:54:13 +00:00
Yasin, Siraj (SY495P)
8d16fffb7c Fix Lint warnings: Missing comments for functions
* Added comments wherever missing for exported functions/constants

Change-Id: I7d0b79b5f56fc6c3edb60a3b58c0962ebd76314c
Relates-To: #148
2020-05-19 16:54:09 +00:00
Yasin, Siraj (SY495P)
318895d89d Encoding secret data in airshipctl config files
* encoding while saving credentials in authInfo
* decoding while fethcing credentials from authInfo
* Credentials are found only in ~/.airship/kubeconfig, so did not find
  anything in ~/.airship/config to be encoded.

Change-Id: I13f3d49b2ad7ccd1388cabd015fe5a93be2c7b96
Closes: #155
2020-05-08 15:09:17 +00:00
Ruslan Aliev
90f80d55b9 Ensure map is initialized before assignment
Attempts to write to a nil map will cause a runtime panic,
since we don't have explicit checks whether the map is nil
or not - they were added with proper initialization using make.
Appropriate test configuration was added.

Change-Id: Ib50686de44b782fdc881bd4608dd74588fce93bd
Relates-To: #190
Closes: #190
2020-05-07 16:22:52 -05:00
Drew Walters
69e8c886fe Add strong typing for baremetal power statuses
Baremetal host power statuses are handled as strings in airshipctl.
Invoking remotedirect on a host that is powered off will result in a
remotedirect failure; therefore, the power status of a host needs to be
verified before remotedirect can begin. This change adds strict typing
to baremetal power statuses so airshipctl can verify the status of a
remote host before performing remotedirect regardless of the client type
(e.g. redfish, other).

Change-Id: I22f1784006add018ee1d67c18f94499aa9544752
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-01 17:25:31 +00:00
Drew Walters
1ce5005cd6 Add baremetal ejectmedia command
Early airshipctl usage has identified the need to eject virtual media
on-demand using airshipctl in order to prevent booting from stuck media.
This change adds a command to eject all virtual media attached to a
baremetal node.

Change-Id: Id67fa00489093dcb84ce54216e0553fa6a737ea6
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-28 23:10:36 +00:00
Drew Walters
7ddb858135 Add baremetal host poweron command
This change introduces a command to power on baremetal hosts.

Closes: #5

Change-Id: Ie76cd7c044cc44629648ef0b02bb7e492ccccdcf
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-28 23:10:32 +00:00
Drew Walters
995538829e Add management support to non-ephemeral hosts
Management support is only available to the ephemeral host. This change
extends support of management features to hosts not labeled as the
ephemeral host.

Change-Id: I0ac917f2633f659e56e67cafc52d2db9d967754f
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-28 16:29:19 +00:00
Drew Walters
0986773bdd Add management configuration module
Some management options are not directly related to the remote direct
functionality, and the remote direct config should not be validated when
they are needed. This change moves those management options to a
dedicated configuration module.

Change-Id: If4388bccbc70b2c77d7804cb4c38644775404242
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-24 21:14:16 +00:00
Stanislav Egorov
cd9b4dddb4 Refactored airshipctl config
Removed Modules from config. There are no other types of modules
that need additional configuration. BootstrapInfo is defined
in the config directly.

Used camelCase for config parameters.

Change-Id: I3531a061026f3b2f1e72af6272ba4709f28d1b3d
2020-04-22 19:26:07 -07:00
Alan Meadows
e2076191b7 Add iDRAC ephemeral boot media support
The Dell Redfish implementation slightly deviates from the DMTF Redfish
specification. One variation is the Dell specification's classification
of virtual media, in which the Dell variation adds support for virtual
CDs and virtual floppy drives [0]. In order to perform some actions on Dell
hardware, airshipctl needs support for vendor specific clients.

This change introduces ephemeral boot media support for iDRAC systems
using a proprietary API in the Dell client.

During the creation of this change, it was also observed that Redfish
calls fail when media is already inserted. This change adds a check
to eject media if media is already inserted.

[0] https://www.dell.com/support/manuals/us/en/04/idrac9-lifecycle-controller-v3.2-series/idrac_3.21.21.21_redfishapiguide/virtualmedia?guid=guid-d9e76cf6-627d-4cb9-a3de-3f2b88b74cfb&lang=en-us

Closes: #139

Co-authored-by: Drew Walters <andrew.walters@att.com>
Change-Id: Ic9fd9e1493b1ff1bb20e956ae5f821d137c74760
2020-04-17 19:51:24 +00:00
Yasin, Siraj (SY495P)
bc5dbed20f Aligned the function definiton to its signature
* The function definition for WithToDiscoveryClientByError and
  WithDynamicClientByError seems to be swapped by mistake.
* So aligned the function definiton to its signature

Change-Id: I7e293b148765f460d007aa4a6dc931defeb879b5
2020-04-16 11:07:23 -05:00
Zuul
1ec7a24b54 Merge "Uplift go-git" 2020-04-15 13:08:33 +00:00
Alexander Hughes
c70f4a2742 Refactor of codebase addressing IDE errors
Goland is complaining of a number of bad practices, and unused code.
This patchset addresses some of those complaints, such as:
- struct initialization without field names
- error string should not be capitalized or end with punctuation
- name starts with package name
- duplicate yaml keys
- snake case
- redundant parentheses

For the http.Transport redundant parentheses, the lint target fails
without the use of //nolint:errcheck so it has been added as
(currently) there is no potential for there to be an error returned
see [0] for example.

[0] https://golang.org/src/net/http/transport.go#L42

Change-Id: Ib1b20639c9b6e9be097ef1f158ecd7472f578488
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
2020-04-14 15:55:04 -04:00
Ian Howell
7c6de22354 Uplift go-git
The go-git module (and helper modules)  recently moved from
`gopkg.in/src-d/go-git` to `github.com/go-git/go-git` [0]. This commit
switches to the new module, as well as uplifts from v4 to v5.

[0] https://github.com/src-d/go-git/issues/1295#issuecomment-592965250

Change-Id: I2e73ba7bb1220a3c75c0d81aa5ab0f1f13479161
2020-04-14 09:07:28 -05:00
Matt McEuen
e52ed80363 Update Kustomize integration to api/v0.3.1
This updates the Kustomize dependency for airshipctl to
api/v0.3.1, which is the oldest version which will support the
Replacement Transformer plugin.

Some changes were needed to accomodate the fact that various apis
that airshipctl was relying on were moved to kustomize-insternal
packages, namely:
- Integrated with the krusty.Kustomizer to drive kustomization
- Removed the custom plugin loader which leveraged the Unknown type
- Worked around NoFieldError becoming private, inc. removing a test

As a follow on we'll need to re-integrate plugin functionality somehow.

Also, in this release Kustomize has implemented support for the
"config.kubernetes.io/local-config" annotation, which we'd planned
to use to to avoid deploying some documents to the Kubernetes API.
It turns out the semantics are different than we anticipated;
Kustomize also fails to return these docs via document *selection*.
Therefore, this change reverts to an earlier approach which uses
a custom airshipit.org/deploy-k8s label.

Change-Id: I7022e12464ea7b6a3ca8609f99f3699bf8da0edd
2020-04-13 10:06:41 -05:00
Yasin, Siraj (SY495P)
c25d223c7b Add copyright for missing files
* added license templates for go, bash & yaml in tools dir
* added a script that will add license information for all
    missing files. Type:  go, yaml, yml, sh
* skip adding license for all files within testdata
* Syntax:
   > ./tools/add_license.sh

* Skip license for manifests folder
* Added one extra line after licene for yaml files
* Added License after Hashbang for bash.
* Add an extra line after hashbang and before license
* Updated the go template to use multiline comments

New Files:
  1. tools/add_license.sh
  2. tools/license_go.txt
  3. tools/license_yaml.txt
  4. tools/license_bash.txt

Change-Id: Ia4da5b261e7cd518d446896b72c810421877472a
Realtes-To:#147
2020-04-09 08:35:59 -05:00
Zuul
780c6b8459 Merge "Update Kustomize to v3.3.1" 2020-04-09 10:07:30 +00:00
Zuul
9ad4db66f4 Merge "Fix for Lint warnings" 2020-04-09 10:01:26 +00:00
Yasin, Siraj (SY495P)
f696ed5f11 Fix for Lint warnings
* This commit handles only warnings of mentioned type
  warning:
    comment on exported function <funcname> should be
      of the form "<funcname> ..."
* In this case, there were comments for the exported
  function/constant/var for which warnings raised,
  but not in expected format.
* So this fix is mostly formating the comment to avoid
  lint warnings.

  Observations:
    * comment did not had function name
    * function name in comment was immediately follwed by comma
        => function followed by space and any description is expected
    * function was not mentioned in the first line of the comment
        => when there is multiline comment, first line should start with
           function name

Change-Id: Ife97104ebc1054f4e34259dca51e8bdb42b33bbd
Relates-To:#148
2020-04-07 14:07:45 +00:00
Drew Walters
263f2ecd6f Add remote system shutdown command
This change introduces a system shutdown command that can be used to
power off remote hosts.

Relates-To: #5

Change-Id: I4fa52db055dcb17e344283fbdd772506e12f77fe
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-07 13:21:27 +00:00
Drew Walters
0c3eefe036 Add remote host power status command
This change introduces a command that allows the user to retrieve the
power status of a remote host.

Relates-To: #5

Change-Id: I4d3ded6667a5427ad6814c3d000da3becaec50a1
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 21:18:18 +00:00
Drew Walters
a15f978cad Use new Redfish client
The remote package in airshipctl is tightly coupled to redfish. In the
future, we may need to introduce IPMI or SMASH; however, adding those
clients now would be difficult because of our tight dependence on
redfish. This change implements usage of the new Redfish client, which
implements the generic remote client. It also separates remoteDirect
functionality from the redfish package in order to make it more loosely
coupled.

Relates #5
Closes #122

Change-Id: I45d4ea6e2a4146ea519e94ea701a3ad527e50ca0
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 18:10:53 +00:00