548 Commits

Author SHA1 Message Date
Alan Meadows
e2076191b7 Add iDRAC ephemeral boot media support
The Dell Redfish implementation slightly deviates from the DMTF Redfish
specification. One variation is the Dell specification's classification
of virtual media, in which the Dell variation adds support for virtual
CDs and virtual floppy drives [0]. In order to perform some actions on Dell
hardware, airshipctl needs support for vendor specific clients.

This change introduces ephemeral boot media support for iDRAC systems
using a proprietary API in the Dell client.

During the creation of this change, it was also observed that Redfish
calls fail when media is already inserted. This change adds a check
to eject media if media is already inserted.

[0] https://www.dell.com/support/manuals/us/en/04/idrac9-lifecycle-controller-v3.2-series/idrac_3.21.21.21_redfishapiguide/virtualmedia?guid=guid-d9e76cf6-627d-4cb9-a3de-3f2b88b74cfb&lang=en-us

Closes: #139

Co-authored-by: Drew Walters <andrew.walters@att.com>
Change-Id: Ic9fd9e1493b1ff1bb20e956ae5f821d137c74760
2020-04-17 19:51:24 +00:00
Drew Walters
cb59c859cb Add Dell Redfish client
The Dell Redfish implementation slightly deviates from the DMTF Redfish
specification. One variation is the Dell specification's classification
of virtual media, in which the Dell variation adds support for virtual
CDs and virtual floppy drives [0]. In order to perform some actions on Dell
hardware, airshipctl needs support for vendor specific clients.

This change introduces a vendor-specific Dell client to handle some
Redfish operations.

[0] https://www.dell.com/support/manuals/us/en/04/idrac9-lifecycle-controller-v3.2-series/idrac_3.21.21.21_redfishapiguide/virtualmedia?guid=guid-d9e76cf6-627d-4cb9-a3de-3f2b88b74cfb&lang=en-us

Relates-To: #139

Change-Id: Icc4500177e859d5a4607b98ec2bd2737521d00b1
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-17 19:51:24 +00:00
Dmitry Ukov
1100217edd Add CAPI bootstrap provider kubeadm
Forked kustomization from 0.3.3 release

Change-Id: Id507ca50518a103ec7df30b201928bba6f76840e
2020-04-17 18:20:27 +00:00
Zuul
e32b94e2aa Merge "Introduce initinfra composite" 2020-04-17 14:43:15 +00:00
Zuul
88ef95d678 Merge "Enhance the fake Client" 2020-04-17 13:49:38 +00:00
Zuul
8c4c7be2fb Merge "Add env variables for necessary roles." 2020-04-17 13:44:52 +00:00
Dmitry Ukov
225ae4a6d1 Introduce initinfra composite
Change-Id: I3f67f315f60ea81c24ae8c5c8612ecc8b086fd39
2020-04-17 13:39:54 +00:00
Zuul
32d0ded751 Merge "Change bootstrap bundle entrypoint to phase dir" 2020-04-17 12:57:47 +00:00
Kostiantyn Kalynovskyi
da2799bee3 Change bootstrap bundle entrypoint to phase dir
Change-Id: I85151f42e510eb113bcc3bb86d56043d51bf348a
Relates-To: #175
2020-04-16 20:02:45 -05:00
sai battina
505a8faccf Add env variables for necessary roles.
This PS also updates airship config site details to be passed as a variable

Change-Id: I299c2cd932a2d44486907a2435306348ea3e4d00
2020-04-16 20:42:31 +00:00
Yasin, Siraj (SY495P)
bc5dbed20f Aligned the function definiton to its signature
* The function definition for WithToDiscoveryClientByError and
  WithDynamicClientByError seems to be swapped by mistake.
* So aligned the function definiton to its signature

Change-Id: I7e293b148765f460d007aa4a6dc931defeb879b5
2020-04-16 11:07:23 -05:00
Ian Howell
e15f1218f6 Enhance the fake Client
This change accomplishes the following:
* Add a constructor for fake.Clients
* Add the ResourceAccumulator type and several instances of
  ResourceAccumulators, each of which is intended to supply a fake.Client
  with arbitrary kubernetes resources.
* Add the client.Factory type, which provides an easier method of
  providing a fake.Client in place of a real one.

Change-Id: I97f5a613df3ca14bc4fdcf726d3e20c5413cbb5b
2020-04-15 14:27:40 -05:00
Ian Howell
e9f8ac3ac3 Add APIExtensions ClientSet to airship Client
This change adds the APIExtension ClientSet to airship's client wrapper,
allowing interactions with CRDs.

Change-Id: Ic156551fde132754915cdb957dd3d82221744cc3
2020-04-15 12:22:03 -05:00
Zuul
1ec7a24b54 Merge "Uplift go-git" 2020-04-15 13:08:33 +00:00
Alexander Hughes
c70f4a2742 Refactor of codebase addressing IDE errors
Goland is complaining of a number of bad practices, and unused code.
This patchset addresses some of those complaints, such as:
- struct initialization without field names
- error string should not be capitalized or end with punctuation
- name starts with package name
- duplicate yaml keys
- snake case
- redundant parentheses

For the http.Transport redundant parentheses, the lint target fails
without the use of //nolint:errcheck so it has been added as
(currently) there is no potential for there to be an error returned
see [0] for example.

[0] https://golang.org/src/net/http/transport.go#L42

Change-Id: Ib1b20639c9b6e9be097ef1f158ecd7472f578488
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
2020-04-14 15:55:04 -04:00
Ian Howell
7c6de22354 Uplift go-git
The go-git module (and helper modules)  recently moved from
`gopkg.in/src-d/go-git` to `github.com/go-git/go-git` [0]. This commit
switches to the new module, as well as uplifts from v4 to v5.

[0] https://github.com/src-d/go-git/issues/1295#issuecomment-592965250

Change-Id: I2e73ba7bb1220a3c75c0d81aa5ab0f1f13479161
2020-04-14 09:07:28 -05:00
Zuul
8438935cfe Merge "Update Kustomize integration to api/v0.3.1" 2020-04-14 11:36:33 +00:00
Zuul
f520dc1190 Merge "Change Label and Annotate to accept map of string" 2020-04-13 19:07:47 +00:00
Zuul
9eb5449b5c Merge "Switching local gating scripts to Python3 to comply with Zuul" 2020-04-13 18:24:24 +00:00
Zuul
e7380b3ba6 Merge "Remove getters/setters of AirshipCTLSettings" 2020-04-13 16:15:02 +00:00
Matt McEuen
e52ed80363 Update Kustomize integration to api/v0.3.1
This updates the Kustomize dependency for airshipctl to
api/v0.3.1, which is the oldest version which will support the
Replacement Transformer plugin.

Some changes were needed to accomodate the fact that various apis
that airshipctl was relying on were moved to kustomize-insternal
packages, namely:
- Integrated with the krusty.Kustomizer to drive kustomization
- Removed the custom plugin loader which leveraged the Unknown type
- Worked around NoFieldError becoming private, inc. removing a test

As a follow on we'll need to re-integrate plugin functionality somehow.

Also, in this release Kustomize has implemented support for the
"config.kubernetes.io/local-config" annotation, which we'd planned
to use to to avoid deploying some documents to the Kubernetes API.
It turns out the semantics are different than we anticipated;
Kustomize also fails to return these docs via document *selection*.
Therefore, this change reverts to an earlier approach which uses
a custom airshipit.org/deploy-k8s label.

Change-Id: I7022e12464ea7b6a3ca8609f99f3699bf8da0edd
2020-04-13 10:06:41 -05:00
Zuul
121adcddc8 Merge "Fixed TODO about error definitions" 2020-04-13 13:34:51 +00:00
Alexey Odinokov
1505353333 Switching local gating scripts to Python3 to comply with Zuul
The initial version of gating playbook contained ability
to run them either on Python2 or Python3 ansible.
But it causes a lot of issues e.g. a need to install[1] both
versions of libs to support both possible ways to run,
discrepancy between local and Zuul run[2]
Zuul testing is done with ansible running on Python3 [3].
Meanwhile ppa repo for ansible contains ansible pachage
that depends only from python2. There is no package for
Python3 and we used only Python2-ansyble.
The fact that Zuul runs it with different configuration
means that we'll keep facing the issues with dependencies
in future, since Gating doesn't test it.
This patch changes makes local scripts run as Zuul.

PS
The script partially takes care of reverting intallation
from its previous runs. To cleanup completely it's possible
to run [4] in case these pacakges are not used by anyone else

[1]
https://opendev.org/airship/airshipctl/src/branch/master/roles/libvirt-install/tasks/main.yaml#L38
[2]
https://review.opendev.org/#/c/717841/
https://review.opendev.org/#/c/718590/
[3]
https://zuul.opendev.org/t/openstack/build/4096513604a84cd2b2f2578fc3f98660/log/zuul-info/host-info.primary.yaml#498
[4]
if dpkg -l | grep "ansible.*ppa~bionic"; then
      sudo apt -y remove ansible
      sudo apt -y autoremove
      sudo add-apt-repository -r -y ppa:ansible/ansible
      sudo apt-get -y update
fi
+ possibly
sudo apt remove virtinst python-libvirt python-docker python-requests python-lxml python-libvirt

Change-Id: Ia1ac3aa4128f6c4797dea1709ccb8516ae342a66
2020-04-11 06:53:40 +00:00
Zuul
788dc35b0a Merge "Add copyright for missing files" 2020-04-10 17:51:51 +00:00
Kostiantyn Kalynovskyi
b485f00c58 Change Label and Annotate to accept map of string
The methods are useful when incoming data are not key value pairs,
rather than iterating two times, one time when we call Annotate()
or Label() over the map, and then iterate again inside the function,
it would be better if we could skip iterating the first time, and
let everything to be handled by Annotate or Label functions

Relates-To: #162

Change-Id: Ia9d989a0534d15e4024317c4606a187ac021b751
2020-04-09 16:34:05 -05:00
Zuul
992efae971 Merge "Fixed local gate scripts" 2020-04-09 13:39:55 +00:00
Yasin, Siraj (SY495P)
c25d223c7b Add copyright for missing files
* added license templates for go, bash & yaml in tools dir
* added a script that will add license information for all
    missing files. Type:  go, yaml, yml, sh
* skip adding license for all files within testdata
* Syntax:
   > ./tools/add_license.sh

* Skip license for manifests folder
* Added one extra line after licene for yaml files
* Added License after Hashbang for bash.
* Add an extra line after hashbang and before license
* Updated the go template to use multiline comments

New Files:
  1. tools/add_license.sh
  2. tools/license_go.txt
  3. tools/license_yaml.txt
  4. tools/license_bash.txt

Change-Id: Ia4da5b261e7cd518d446896b72c810421877472a
Realtes-To:#147
2020-04-09 08:35:59 -05:00
Zuul
780c6b8459 Merge "Update Kustomize to v3.3.1" 2020-04-09 10:07:30 +00:00
Zuul
9ad4db66f4 Merge "Fix for Lint warnings" 2020-04-09 10:01:26 +00:00
Stanislav Egorov
9e0b8b2a96 Fixed local gate scripts
Change-Id: I3fc0c597f1b4f050b40905b5415069c5d0b8101f
2020-04-08 18:31:30 -07:00
Zuul
a306429749 Merge "Create documentation for new developers" 2020-04-08 15:58:57 +00:00
Noor Muhammad Malik
e650468379 Create documentation for new developers
This patch updates the develop documentation in docs/source/developers.md.
Adds details on environment setup and uses some of the scripts available
in tools/gate directory.

Change-Id: I76d426746f668f89ab4f9e43ac00f75f47735702
Relates-To: #72
Closes: #72
2020-04-08 19:22:27 +05:00
Ian Howell
5efc26d9f0 Remove getters/setters of AirshipCTLSettings
This change removes the getters and setters for the following attributes
of pkg/environment.AirshipCTLSettings:

* airshipConfigPath
* kubeConfigPath
* config

The above attributes have been exported to account for the removal of
these methods.

Change-Id: I4e7d38bf6771f7521179a2fe2990475b936b5d95
2020-04-08 09:00:32 -05:00
Yasin, Siraj (SY495P)
f696ed5f11 Fix for Lint warnings
* This commit handles only warnings of mentioned type
  warning:
    comment on exported function <funcname> should be
      of the form "<funcname> ..."
* In this case, there were comments for the exported
  function/constant/var for which warnings raised,
  but not in expected format.
* So this fix is mostly formating the comment to avoid
  lint warnings.

  Observations:
    * comment did not had function name
    * function name in comment was immediately follwed by comma
        => function followed by space and any description is expected
    * function was not mentioned in the first line of the comment
        => when there is multiline comment, first line should start with
           function name

Change-Id: Ife97104ebc1054f4e34259dca51e8bdb42b33bbd
Relates-To:#148
2020-04-07 14:07:45 +00:00
Drew Walters
263f2ecd6f Add remote system shutdown command
This change introduces a system shutdown command that can be used to
power off remote hosts.

Relates-To: #5

Change-Id: I4fa52db055dcb17e344283fbdd772506e12f77fe
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-07 13:21:27 +00:00
Stanislav Egorov
f7a2e33b8b Fixed TODO about error definitions
Use SelectOne to replace Select and additional checks.

Change-Id: I40b4780d62fa4b3a0e01cc54ef7dbe4b32a4763d
2020-04-06 11:23:42 -07:00
Drew Walters
a4001d255f Add remote host reboot command
This change introduces a reboot command that allows the user to reboot a
remote host. The reboot command supports any airshipctl remote client
and simply uses the one defined in the remote options.

Relates #5

Change-Id: I6e11430aa60a09172eb3ac0f44d1a6ff17ca91b0
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-06 09:26:57 -05:00
Drew Walters
0c3eefe036 Add remote host power status command
This change introduces a command that allows the user to retrieve the
power status of a remote host.

Relates-To: #5

Change-Id: I4d3ded6667a5427ad6814c3d000da3becaec50a1
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 21:18:18 +00:00
Drew Walters
d25c22a538 Add remote command
This change introduces the remote command. The remote command will be
used to control the power of remote nodes and eventually include the
"bootstrap remotedirect" functionality.

This change is merely an outline of the command. No functionality is
included in this change.

Closes #117

Change-Id: I3291405f36d6a7acc5c8c1f07bd90ad9c2986cd3
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 20:52:22 +00:00
Drew Walters
b4b792a04f Fix function name misspelling
Change-Id: I8ffb5fc459360fb9cf4cb84dc0d6ab0aa4f30d21
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 20:49:17 +00:00
Drew Walters
a15f978cad Use new Redfish client
The remote package in airshipctl is tightly coupled to redfish. In the
future, we may need to introduce IPMI or SMASH; however, adding those
clients now would be difficult because of our tight dependence on
redfish. This change implements usage of the new Redfish client, which
implements the generic remote client. It also separates remoteDirect
functionality from the redfish package in order to make it more loosely
coupled.

Relates #5
Closes #122

Change-Id: I45d4ea6e2a4146ea519e94ea701a3ad527e50ca0
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 18:10:53 +00:00
Drew Walters
bca0a96603 Add mock Redfish client
Remote direct tests mock the go-redfish client in order to avoid making
Redfish API calls. We should be able to mock airshipctl's internal
Redfish API client for unit tests with less side effects. This change
adds a mock Redfish client to reduce the side effects in our remote
direct unit tests.

Relates #5, #122

Change-Id: Ic968c5eefa90aa2065a3f15bb9880edbacf73ece
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-03 18:01:45 +00:00
Matt McEuen
e690a4c453 Update Kustomize to v3.3.1
This updates the Kustomize module dependency to v3.3.1,
which has replacement functionality required by the Replacement Transformer
plugin, among other enhancements.

Change-Id: I058114c398464394cfb6b2626cb475d763c1bacc
2020-04-03 12:11:33 -05:00
Zuul
ad8c87611b Merge "Use 16GB flavor for gate job and let it vote" 2020-04-03 17:09:27 +00:00
Zuul
ddfb6ebdbe Merge "Add a HTTPS-->HTTP reverse proxy in front of redfish-emulator" 2020-04-03 15:03:51 +00:00
Zuul
01ca21754d Merge "Fix ironic deployment template" 2020-04-03 14:59:02 +00:00
Zuul
effc58d511 Merge "Add Bare Metal Operator Ironic entrypoints" 2020-04-03 14:46:59 +00:00
Zuul
004e9c47ac Merge "Switch gate to official isogen image" 2020-04-03 08:07:42 +00:00
Zuul
a200af5810 Merge "Remove external role dependency from gate job" 2020-04-03 00:42:01 +00:00
Stanislav Egorov
3063e207b3 Use 16GB flavor for gate job and let it vote
Gate job can be run on a 16GB flavor as well as 32GB.
It's better to use less resources if this works for tests.

Depends-On: https://review.opendev.org/#/c/715055/

Change-Id: I198d640584e80dc6ca3626a6b23855c2aba6bf60
2020-04-02 21:28:34 +00:00