91 Commits

Author SHA1 Message Date
Sreejith Punnapuzha
c64ad2db4f Update kubectl version
This commit updates kubectl version from 1.16.2 to 1.17.4 as part of
gate setup

Change-Id: I88728be65474ee7ce57f243e1356b84c7609a45f
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 14:31:31 -05:00
Sreejith Punnapuzha
c3002cabda Add bash scripts to run airshipctl
* Add playbook to run the scripts
 * Add script to install kubectl
 * Add script to create airshipctl executable
 * Add script to generate test configuration
 * Add script to pull site documents
 * Add script to build ephemeral iso
 * Add script to deploy ephemeral node
 * Add script to deploy control plane

Closes: #223
Closes: #224
Closes: #225
Closes: #226
Closes: #227

Change-Id: Ied63e2a733c3abcc6b1a822624c5ffce92e05a44
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 15:44:06 +00:00
Sreejith Punnapuzha
1108d4d2bf Add python3-setuptools to needed packages
This commit adds python3-setuptools to needed packages so that
sushy-tool installation will not fail

Change-Id: I7b6de0ebf09b7c187061030463327831c7d856f3
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 08:25:13 -05:00
Zuul
953e55f04b Merge "Remove deprecated fields of remoteDirect" 2020-06-12 19:59:49 +00:00
Yasin, Siraj (SY495P)
54e13ceaad Added new command for Image Build.
* command: airshipctl image build
* For now it is just a wrapper for baremetal isogen.
* Removed isogen subcommand from baremetal

Change-Id: I5c1260c4b167e5768eba27a7cd83c9b60ab29184
Relates-To:#251
Closes:#251
2020-06-12 03:01:26 +00:00
Dmitry Ukov
df2fff0acf Add control plane deployment
K8s control plane is deployed by KubeadmControlPlane controller. This
controller creates CAPI machines and infrastructure objects
(Metal3Machine). Metal3Machine objects are created based on a template
which contins host selector label. Control plane label is assigned to a
particular BareMetalHots object defined inside of the shared
kustomization.

Relates-To: #149
Closes: #221
Change-Id: I3be1750aacf9736ece2944045c036f405e404561
2020-06-11 13:44:42 +00:00
Ian Howell
5a187e1810 Remove deprecated fields of remoteDirect
This change removes the deprecated "remoteType", "useProxy", and
"insecure" fields from the remoteDirect field of bootstrapInfo

Change-Id: I64d28b715b81464c871f99ad0348fadc2e9fc5f6
Relates-To: #242
2020-06-10 10:23:41 -05:00
Stanislav Egorov
37bba4e2fc Locked version of sushy-tools
0.9.1 is not working with current implementation of airshipctl
Error below:
  Setting boot device to 'CD'.
  BMC responded '500 INTERNAL SERVER ERROR'. Attempting to unmarshal the raw BMC error response.
  Unable to decode BMC response. "Unable to decode Redfish response.
  Key 'error.@Message.ExtendedInfo.Message' is missing or has unknown format."
  redfish client encountered an error: BMC responded '500 INTERNAL SERVER ERROR'.

0.9.0 works fine

Change-Id: I34907cc42bbe78624e2741a581bc95c2e2f68bea
2020-05-28 15:24:36 -07:00
Dmitry Ukov
a301de202b Add CAPI initialization steps to gating
This change enables execute 'airshipctl cluster init' command
as a part of zuul pipeline

Co-Authored-By: Kostyantyn Kalynovskyi <kkalynovskyi@mirantis.com>
Change-Id: I9d499ae66f8d847759c2e61e5d4415f2c9f3c860
2020-05-20 15:17:20 +04:00
Zuul
66980ed432 Merge "Replace become: true notation" 2020-05-20 04:27:57 +00:00
Ruslan Aliev
76d8716968 Replace become: true notation
Ansible documentation recommends to use yes/no as boolean value,
the last mention of possible using true/yes related to version 2.4 [1].
Starting from version 2.5 only become: yes specified in docs [2][3].
Since we have both variations, this patch aligns become statement to
use only yes value.

[1] https://docs.ansible.com/ansible/2.4/become.html
[2] https://docs.ansible.com/ansible/2.5/user_guide/become.html
[3] https://docs.ansible.com/ansible/latest/user_guide/become.html

Change-Id: I706e6e1b9d706d6bfcc38c1a3d0d7304575946ce
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:48:42 -05:00
Ruslan Aliev
917d6896b6 Remove "cluster initinfra" command
Command "cluster initinfra" should be removed since
"phase apply" was introduced instead.

Change-Id: I263f8988a4456fc03314093a55ee1c1da8048404
Relates-To: #162
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-05-16 01:45:34 -05:00
Alexander Noskov
84f2bc60c9 Fix default Airshipctl config parameters
Problems description:
Once we execute airshipctl config init, the default config in
~/.airship/config doesn't contain default values for:
currentContext: ~
clusters:
  default:
    clusterType:
      target:
        bootstrapInfo: ~
        managementConfiguration: ~
---
This PS set default values for Airshipctl config to use it
out of the box.

Closes: #234

Change-Id: I57449e678f19ab1c0e2212c38b01ea195f62aa44
2020-05-12 16:02:05 -05:00
Zuul
60ece21b96 Merge "Fix system power regressions" 2020-05-12 16:42:31 +00:00
Zuul
528407edc7 Merge "Add pod log collecting role" 2020-05-12 16:10:18 +00:00
Dmitry Ukov
ea79cd6a71 Add pod log collecting role
Closes: #220
Change-Id: I5a4b682f8f4b94fa6827e6d304d07d825811916b
2020-05-12 18:08:03 +04:00
Drew Walters
4017e9c221 Fix system power regressions
A change [0] recently merged that makes additional remote parameters
configurable; however, it changed the default wait value to two seconds
and made the wait value a duration in time between two nanoseconds.

This change restores the default wait value of thirty seconds and makes
the wait value representative of seconds again.

[0] https://review.opendev.org/717812

Change-Id: I9fad30d479419efe95fca7dc78154bfc3ca66697
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-11 20:33:50 +00:00
Zuul
936b5639f5 Merge "Fix image push zuul job" 2020-05-11 16:14:11 +00:00
Zuul
3fb7c774e7 Merge "Reusing apache-server instead of httpd" 2020-05-11 15:39:30 +00:00
Sreejith Punnapuzha
ad71f4704b Fix image push zuul job
This commit changes airshipctl-publish-images role to upload images
to quay.io

Closes: #217
Change-Id: I67b3e07ddc170da6df7c34dce929356449863c10
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-05-08 18:54:39 -05:00
Zuul
f334b45638 Merge "Make remote parameters configurable" 2020-05-08 22:01:18 +00:00
James Gu
301de51467 Fixed document pull issue in gate script 21
Changed systemwide-executable to run in local_src_dir because the script needs
to build airshipctl binary first, before doc pull command can be invoked and populate
the remote_work_dir. Changed the order of the roles to allow systemwide-executable
to run before test-config and pull-docs.

Change-Id: Ia47423d91cdfb2dbbf0259344daab1c7b8cd47f5
Signed-off-by: James Gu <james.gu@att.com>
2020-05-08 18:30:51 +00:00
Drew Walters
f729fdd1a1 Replace venv with tox for documentation builds
OpenStack infra has created Zuul project templates that will enable us
to push our documentation to Read the Docs and the Airship website.
While adding a tox file to this project is not ideal, it appears to be
the only way to leverage these jobs at the moment.

This change adds a tox.ini file to airshipctl to build our documentation
and removes the venv solution. The Airship Working Committee will work
with the OpenStack infra team, if possible, to make the publish jobs
more flexible in the future and remove the tox.ini file.

Change-Id: I274cf69a7c79e0aad1d4c9b1decfaf5630f0b18f
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-08 15:47:41 +00:00
Vladislav Kuzmin
9c1724f6d8 Make remote parameters configurable
Remove constants and define these parameteres
in config file for remote configuration.

Closes: #138

Change-Id: Ib9250b8d39c01ef43a1262b62e4d37215481bc43
2020-05-08 17:23:59 +04:00
Zuul
a9211bf787 Merge "[#112] Add doc pull in deployment scripts" 2020-05-05 14:58:01 +00:00
sai battina
f844596a05 Wait for control pods to be ready before moving to initinfra
Change-Id: I091fe3f712c4bc29d9e0c5f03e234c75523dfa90
2020-05-02 19:21:44 +00:00
Zuul
54d7b5f229 Merge "Add additional debug logging to Redfish calls" 2020-05-01 15:47:26 +00:00
Zuul
6f6b441ffb Merge "Fix libvirt-volume error conditional" 2020-05-01 14:19:52 +00:00
Drew Walters
9ff4c67e05 Add additional debug logging to Redfish calls
When tracking down issues on baremetal nodes using remotedirect, it has
been difficult to pinpoint which Redfish calls fail. This change adds
additional debug logging to make the troubleshooting process simpler
during pre-alpha development.

Change-Id: Ic32d34d010c44b2e7e2a4dfe2b623a451ad94019
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-01 14:14:31 +00:00
Alexey Odinokov
16d2c8d8c1 Reusing apache-server instead of httpd
Since we already introduced apache server for redfish,
it will be a good idea to use it for sharing iso also.
These changes add file-exchanger config that can be
used to share iso. In addition apache allows the
specified users (see rbac config) to upload files.
This may help in future to better emulate deployment
environment, where it will be necessary to do this
step after airshipctl generated iso and before
airshipctl invokes redfish to run that iso.

Change-Id: Ice47fe977fb71cf9df14d39c5ac26e365e2006f2
2020-05-01 03:17:02 +00:00
James Gu
d26df73b4b [#112] Add doc pull in deployment scripts
Added the optional use of doc pull in gate deployment scripts. When the
remote_work_dir is defined, the new pull doc role will invoke airshipctl
document pull command to download the manifests to the remote_work_dir.
This role is not included in zuul job so the zuul job will continue to use
the already checked out manifests in the zuul.project.src directory.

Fixes #112

Change-Id: I754c08661ad0b635819cb3730e05a6986d90f5ca
Signed-off-by: James Gu <james.gu@att.com>
2020-04-30 15:11:44 -07:00
Dmitry Ukov
25e7d92d48 Rename gating test site
* New name for gating site is 'test-site'
* Renamed 'test-bootstrap' type. New name is 'gating'.
* Moved BareMetalHost and its settings from ephemeral function to
  shared directory within the site

Change-Id: I510e74c1a553bb2b1f84e66f8e4a7796640b59cc
2020-04-29 17:32:40 +04:00
James Gu
afcfdb5c6e Proxy support enhancement
Added gate scripts and playbook enhancement to run airshipctl behind proxy.

Change-Id: Icc3cdcd6fd150420103f907cd1871feba1ccd223
Signed-off-by: James Gu <james.gu@att.com>
2020-04-28 18:04:55 +00:00
Drew Walters
e34edc0ce2 Move bootstrap cmd functionality to baremetal cmd
Early airshipctl usage has identified the need to control the power of
remote hosts directly from airshipctl. This functionality is typically
required during the bootstrapping phase of airshipctl; however, the
functionality could be used anytime. A logical home for this
functionality would be in a baremetal command, not a bootstrap command.
Since all functionality performed by the bootstrap command is performed on
baremetal hosts, a natural need has developed to group this
functionality together under one baremetal command.

This change moves all functionality from the bootstrap command to a new
baremetal command.

airshipctl baremetal isogen         generate ephemeral node ISO
airshipctl baremetal remotedirect   bootstrap ephemeral node

Change-Id: Ia615224686d97c95d78c3773c2b9f41df9d06ed6
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-28 16:29:19 +00:00
Drew Walters
995538829e Add management support to non-ephemeral hosts
Management support is only available to the ephemeral host. This change
extends support of management features to hosts not labeled as the
ephemeral host.

Change-Id: I0ac917f2633f659e56e67cafc52d2db9d967754f
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-28 16:29:19 +00:00
Drew Walters
e0d612f437 Fix libvirt-volume error conditional
The libvirt-volume error conditional does not register errors when it
fails to create a volume due to an incorrect stdout check. This change
fixes the conditional.

Change-Id: I436324a22accf75a353c65ef3852838e992414e8
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-04-27 14:23:48 +00:00
Zuul
585f885409 Merge "Add initinfra gating tests" 2020-04-23 18:28:27 +00:00
Zuul
102ec4ec72 Merge "Refactored airshipctl config" 2020-04-23 16:13:27 +00:00
Vamsi Savaram
e40b5ffec5 Add initinfra gating tests
Relates-To: #180

Change-Id: Ie7a8daa532596f2d1d0fb2059ef4630607511711
2020-04-23 14:41:45 +00:00
Stanislav Egorov
cd9b4dddb4 Refactored airshipctl config
Removed Modules from config. There are no other types of modules
that need additional configuration. BootstrapInfo is defined
in the config directly.

Used camelCase for config parameters.

Change-Id: I3531a061026f3b2f1e72af6272ba4709f28d1b3d
2020-04-22 19:26:07 -07:00
Vamsi Savaram
2d8bc4cf9c Use HTTPS to deploy ephemeral node
PS [1] delivered a change to add apache running
sushy-emulator as wsgi backend. This allows us
to have a deployment that looks like a real BMC.
This PS switches gate test to use basic auth and
https on port (8443).

[1]:https://review.opendev.org/#/c/719320/

Relates-To: #136

Change-Id: I7d12bea92cb2c10954cdf1018898d352618c6694
2020-04-22 16:00:09 +00:00
Zuul
a0cb765d73 Merge "Gather container logs in check pipeline" 2020-04-20 17:04:06 +00:00
Vamsi Savaram
46989a520d Gather container logs in check pipeline
This patch configures docker daemon to use
journald logging driver

Relates-To: #89

Change-Id: I2e3c4df06163caa032d2d9b7a2032123d311915a
2020-04-20 15:03:59 +00:00
Zuul
af2a5330e0 Merge "Substituting redfish-emulator and reverse-proxy with Apache" 2020-04-20 14:50:59 +00:00
Alexey Odinokov
bb7bd1c58e Substituting redfish-emulator and reverse-proxy with Apache
The current implementation of airship-libvirt-gate is using
sushy-emulator binary to emulate redfish. Sushy-emulator works
only for http and also can’t authenticate users out-of-box if
ran by itself. In order to check https and authentication the
reverse-proxy was introduced. This approach had several
drawbacks:
1) http still doesn’t check auth
2) to use apache for https only is too heavy solution for https
This change converts reverse proxy to apache running
sushy-emulator as wsgi backend, that gives an ability to check
authentication for both http and https.
We’re also getting rid of ad-hoc sushy-emulator service and
using out-of-box apache service implementation.
The code also introduces gathering of apache resulting configs
and logs for quicker debug if needed.
Right now authentication is disabled, since manifests are
written in a way so they don’t use them. If it’s necessary to
enable it, just set username here[1]

PS
There is ability to use apache for http-server [2], but it’s
better to do as a separate PR

[1]
roles/airship-libvirt-gate/defaults/main.yaml
[2]
roles/http-fileserver

Change-Id: I43b5bca41519c88b01535c156b2db0e9edaa81bb
2020-04-17 15:55:48 +00:00
sai battina
505a8faccf Add env variables for necessary roles.
This PS also updates airship config site details to be passed as a variable

Change-Id: I299c2cd932a2d44486907a2435306348ea3e4d00
2020-04-16 20:42:31 +00:00
Alexey Odinokov
1505353333 Switching local gating scripts to Python3 to comply with Zuul
The initial version of gating playbook contained ability
to run them either on Python2 or Python3 ansible.
But it causes a lot of issues e.g. a need to install[1] both
versions of libs to support both possible ways to run,
discrepancy between local and Zuul run[2]
Zuul testing is done with ansible running on Python3 [3].
Meanwhile ppa repo for ansible contains ansible pachage
that depends only from python2. There is no package for
Python3 and we used only Python2-ansyble.
The fact that Zuul runs it with different configuration
means that we'll keep facing the issues with dependencies
in future, since Gating doesn't test it.
This patch changes makes local scripts run as Zuul.

PS
The script partially takes care of reverting intallation
from its previous runs. To cleanup completely it's possible
to run [4] in case these pacakges are not used by anyone else

[1]
https://opendev.org/airship/airshipctl/src/branch/master/roles/libvirt-install/tasks/main.yaml#L38
[2]
https://review.opendev.org/#/c/717841/
https://review.opendev.org/#/c/718590/
[3]
https://zuul.opendev.org/t/openstack/build/4096513604a84cd2b2f2578fc3f98660/log/zuul-info/host-info.primary.yaml#498
[4]
if dpkg -l | grep "ansible.*ppa~bionic"; then
      sudo apt -y remove ansible
      sudo apt -y autoremove
      sudo add-apt-repository -r -y ppa:ansible/ansible
      sudo apt-get -y update
fi
+ possibly
sudo apt remove virtinst python-libvirt python-docker python-requests python-lxml python-libvirt

Change-Id: Ia1ac3aa4128f6c4797dea1709ccb8516ae342a66
2020-04-11 06:53:40 +00:00
Yasin, Siraj (SY495P)
c25d223c7b Add copyright for missing files
* added license templates for go, bash & yaml in tools dir
* added a script that will add license information for all
    missing files. Type:  go, yaml, yml, sh
* skip adding license for all files within testdata
* Syntax:
   > ./tools/add_license.sh

* Skip license for manifests folder
* Added one extra line after licene for yaml files
* Added License after Hashbang for bash.
* Add an extra line after hashbang and before license
* Updated the go template to use multiline comments

New Files:
  1. tools/add_license.sh
  2. tools/license_go.txt
  3. tools/license_yaml.txt
  4. tools/license_bash.txt

Change-Id: Ia4da5b261e7cd518d446896b72c810421877472a
Realtes-To:#147
2020-04-09 08:35:59 -05:00
Vamsi Savaram
5bf96b27d6 Add a HTTPS-->HTTP reverse proxy in front of redfish-emulator
This patch introduces a reverse proxy that terminates an HTTPS
connection and forwards it to the redfish emulator behind that
is running HTTP. Also the reverse proxy presents a self-signed
certificate.

Closes: #136
Relates-To: #136

Change-Id: If6ee705247ae8866d2674bff1ff034277f9c9177
2020-04-02 10:21:37 -05:00
Stanislav Egorov
4e50491e9c Fixed runtime log gathering
Change-Id: I1944157230ca1387d400c255e0b4020c5e532407
2020-03-26 15:13:28 -07:00