66 Commits

Author SHA1 Message Date
Ruslan Aliev
f7fca469de Add KRM function to validate site documents
Proposed KRM function can perform static validation of the site
documents based on Kubeval. It allows to specify additional CRD
locations to be able to validate custom resources. As input,
it uses document bundle created from desired phase's document
entry point or output from render command.

Change-Id: Ib76d88a6e6c7f3b29b29cab0abe038eef380686f
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #19
2021-04-22 11:13:40 -05:00
Roman Gorshunov
7998615a7b Add base images override support
Go builder and base images override support for downstream
deployment customizations for both airshipctl and plugins.
If required, adjust the following Zuul job variables:
 - docker.base_go_image
 - docker.base_release_image
 - docker.base_plugins_build_image
 - docker.base_plugins_release_image

Closes: #515
Change-Id: Iaf4b54a353207b06c9ed6bdcae876537e73f6e44
2021-04-12 19:47:31 +02:00
Vladislav Kuzmin
d5c0377207 Add toolbox krm function
Add krm function to execute bash scripts inside container.

Closes: #494
Change-Id: I3074a27a022f65e87f190ab5a39c252f225ca1fa
2021-03-31 13:28:16 +00:00
Ian Howell
edea4a2a36 Suppress the "go command not found" message
When a make target is run, make will try to find the GOBIN using a go
command. If go isn't installed, the user sees the message:

  /bin/bash: go: command not found

This can be confusing, especially when a docker-image target is being
run. This change suppresses that message.

Change-Id: Ie27a4a82a3692db60445601febf3bd51f956d575
2021-01-20 09:40:56 -06:00
Roman Gorshunov
de72491f2c Add commit and other labels to the airshipctl container image
Relates-To: #399

Change-Id: I0e0be8fe6dd37b346637ae9847523e4cd3734081
2021-01-11 14:52:59 +00:00
Anderson, Craig (ca846m)
2cf38f248d Add deepcopy generate gate enforcement
Reject commits to the airshipctl repo that contain a
manually edited zz_generated.deepcopy.go file (i.e. one that does not
match the auto-generated version).

This will also prevent code from merging which fails to generate a
deepcopy file.

Change-Id: I4f06cd852237f7b3d8643d859775c16c3cd47530
2020-11-30 06:39:17 -08:00
Dmitry Ukov
3acd9ab8f5 Add DeepCopy generation target to Makefile
Change-Id: I974f8472f2ad8231a6612849f1bc73895c96ef81
Closes: #384
2020-11-18 18:20:25 +04:00
Sreejith Punnapuzha
c2e5cb13f2 Change image tags
* Change templater image tag from dev to latest
  * Change replacement-transformer image tag from dev to latest
  * Change Makefile to build images with tag latest

Change-Id: Iff074d6fe771cce48dae7190fbc69cc437138817
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-11-10 12:23:25 -06:00
Sirajudeen
c0f116a68c Separate lint job from the test
* Split lint and test jobs
* Add lint job to be run against all files

Change-Id: I74da7999324283984d61f38ae9fb394ee5b37ce6
Closes: #386
2020-11-05 22:35:32 +00:00
Dmitry Ukov
d783d71f05 Introduce Replacement Transformer plugin container
Relates-To: #341
Change-Id: I65a5b8cf3bcf94321fc02f240b1d8075bca45c0c
2020-10-16 16:00:56 +04:00
Zuul
4566efc325 Merge "improve logging" 2020-09-01 12:48:29 +00:00
Schiefelbein, Andrew
20d2a99190 improve logging
This will help on debugging and / or error messages which
will show what package and what line a message came from

This is pared down from what is currently being proposed for the UI
https://review.opendev.org/#/c/747473/

Change-Id: I68eb8b0141a8d56d5ea14aa623a25f9287ff1e75
2020-08-27 16:02:38 -05:00
Sreejith Punnapuzha
06ae80d3b0 Update Kubectl Version
Kubernetes was uplifted to v1.18.6 uplifting kubectl to the same
version.

Change-Id: Id80a4ee169c06c1957c575ef98f87f6ad56c7947
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-08-24 09:27:43 -05:00
Sirajudeen
8091c98566 Ignore errors from test coverage
*) Most the errors.go has 0% coverage due to usage of
Fatal.
*) so ignoring errors from coverage till we cleanup the references to Fatal
   and use custom errors/warnings.

Change-Id: I56f838ec33336ccca3c14226e791842d159a95fb
2020-07-10 10:43:28 -05:00
Dmitry Ukov
cad8177fee Make API compatible with k8s apimachinary
Each object must implement runtime.Object interface. Therefore we
need DeepCopy methods implemented for each API struct.
Implementation is generated by controller-get utility.

Relates-To: #259
Change-Id: I535bf9ec3ff61597a778f73ca449313b7455b989
2020-06-23 13:01:16 +04:00
Sreejith Punnapuzha
c64ad2db4f Update kubectl version
This commit updates kubectl version from 1.16.2 to 1.17.4 as part of
gate setup

Change-Id: I88728be65474ee7ce57f243e1356b84c7609a45f
Signed-off-by: Sreejith Punnapuzha <sreejith.punnapuzha@outlook.com>
2020-06-19 14:31:31 -05:00
Matt McEuen
958d783919 Add site doc validation gate
This adds a gate which loops over all phases in all sites,
and performs an airshipctl apply --dry-run on them to ensure YAML
validity and schema adherence.  Aside from installation tasks,
the gate is run via a makefile entrypoint so that it can be
easily consumed by developers or by non-zuul CICD platforms.

Change-Id: Ie4ab246848a580ab20c3153af1e3749a27e3f770
2020-06-04 18:04:47 -05:00
Yasin, Siraj (SY495P)
90e27b542e Add check_copyright as part of gate script
* Added check_copyright to validate if all source files has
  expected copyright header

Change-Id: I8d9f3510db9cb5cbc510748a2f71ad95aa8d366b
2020-06-01 18:52:28 +00:00
Ian Howell
7243176e44 Add golint as a non-voting gate
Change-Id: I9850528a720ff517a3f400553fa8ab66bf4b1b3b
Closes: #253
2020-05-29 11:40:45 -05:00
Zuul
01deda7825 Merge "Fixed docker build space issue" 2020-05-19 20:00:56 +00:00
Drew Walters
0d7dc4de24 Add automated CLI documentation
This change creates a tool that generates CLI documentation each time
`make update-golden` is executed.

Change-Id: I4bcdf299cd424f4cff1ecf6503822d304e9a3947
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-13 19:43:10 +00:00
Stanislav Egorov
e873bbb281 Fixed docker build space issue
By default docker build doesn't remove intermediate containers
when build is failed. They are accumulated and we can see disk space issue.

Enabled force mode to remove intermediate containers even if build is failed.
It can be overriden for debugging purposes via env vars for make.

Example: DOCKER_FORCE_CLEAN=false make docker-image

Change-Id: Ia68916b78fc91704e296ebe1ae05d1168a17fdc2
2020-05-12 23:58:15 -07:00
Drew Walters
f729fdd1a1 Replace venv with tox for documentation builds
OpenStack infra has created Zuul project templates that will enable us
to push our documentation to Read the Docs and the Airship website.
While adding a tox file to this project is not ideal, it appears to be
the only way to leverage these jobs at the moment.

This change adds a tox.ini file to airshipctl to build our documentation
and removes the venv solution. The Airship Working Committee will work
with the OpenStack infra team, if possible, to make the publish jobs
more flexible in the future and remove the tox.ini file.

Change-Id: I274cf69a7c79e0aad1d4c9b1decfaf5630f0b18f
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-05-08 15:47:41 +00:00
Stanislav Egorov
5be10c5814 [#204] Refactoring for version cmd
Now the version can be changed from Makefile/env.
By default version is 'devel' if there is no ldflags in build args.
It can be overriden by providing ldflags in Makefile.

Change-Id: Ifef5932ec953ccfaeca8a2ef58eb951c900c7a59
2020-05-06 09:05:19 -07:00
Alexander Hughes
abb28716c8 [#81] makefile to check for unused test data
This change adds logic to the docker-image-unit-tests to ensure that
code is checked within the container for coverage, valid test data,
and reduces the number of containers created in the gate - reducing
the runtime of each gate.

Relates-To: #81

Change-Id: Iab3d9bd74266d32404e6eb06ecebc234df04fd5c
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
2020-03-08 00:05:48 +00:00
Zuul
15d45199a8 Merge "[#58] Add golint" 2020-03-06 20:25:55 +00:00
Ian Howell
e8892c4e99 [#58] Add golint
This adds the golint linter to the `make lint` target. As of this
commit, the results from golint are not counted as errors. Further
changes to the code will be required to bring it up to golint's
standards, at which point we will allow golint to cause failures for the
`lint` target.

Change-Id: I34134e3deb080e6da0bc0aac299caca6c3b5d0a9
2020-02-26 11:15:10 -06:00
Drew Walters
bcfcd6f9a0 [#33] Publish airshipctl image on Quay
This change introduces a job that publishes the latest airshipctl image
to quay.io/airshipit/airshipctl each time a new change is merged.

Closes #33

Change-Id: I8468b10a8064333d1749015960151e98adb2be97
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-02-26 10:49:22 -06:00
Drew Walters
c834b2d22f [#59] Publish documentation output to html dir
OpenDev publishes Ansible roles that enable documentation previews.
These roles expect an html or pdf directory in the Sphinx build dir.
This change updates the documentation output directory to the
doc/build/html so documentation previews are available on each PS.

Related #59

Change-Id: I0853f603b6bc147e7fc33834b5d195c16f71d5ab
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-02-20 11:21:04 -06:00
Drew Walters
3a5f25c22e [#59] Build docs with Python 3
Most systems have /usr/bin/python, the interpreter the airshipctl
Makefile uses to build documentation, linked to an end-of-life
interpreter. This change updates the Makefile to use the Python 3
interpreter.

Related #59

Change-Id: I19b4d623a5a76dc455184bac2f54d8bfe0947530
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-02-20 10:16:47 -06:00
Ian Howell
9d72a01fc3 [#41] Update make godoc target
Ideally, installing and using an external tool should not modify the
go.mod or go.sum files in anyway. Unfortunately, this may not be
completely possible until Go1.14
(https://github.com/golang/go/issues/30515). This change prevents the
godoc target from modifying the go.mod file, though there may still be
anomalies with modifying the go.sum file (e.g. this change).

Further, since the godoc tool defaults to -goroot=$(go env GOROOT), it
doesn't make sense to provide a default value for -goroot, as it may not
be what the user has set as $(go env GOROOT)

Change-Id: I92de6bf8af4c00464b857db363bd07fdb55ee143
2020-02-12 11:06:10 -06:00
Ian Howell
1b8a965105 Add a simple whitespace linter
The goimports tool does not check for trailing whitespaces in multiline
strings. This commit adds a simple linter to address this deficiency.

This change will also have the benefit of providing light linting of any
non-go files.

Change-Id: I253dd7c4895f476fa3a6f0c4fccf7c9cada16366
2020-01-15 10:57:44 -06:00
Ian H. Pittwood
b8a2bb3b27 Updates to Makefile
Implements `make docs` command to create rtd-themed docs using Sphinx
from the existing Markdown documents in the docs directory.

Changes the `get-modules` command to `depend` to more closely match
OpenStack guidelines in [0].

Adds `fmt` command as specified in [0], currently referencing the `lint`
command.

Adds `godoc` command that will host a local godoc server instance with a
link directly to airshipctl technical documentation.

Adds `releasenotes` commands specified in [0], currently
noted as "TODO".

Adds `images` command as found in other Airship projects, referencing
`docker-image`.

[0] https://governance.openstack.org/tc/reference/pti/golang.html

Change-Id: I134fc2c2714bce6d23dc9cf8b86c3016d68def7e
2020-01-10 09:20:10 -06:00
Ian H. Pittwood
5b7de06eec Add proxy options to Makefile docker build
Adds options relating to proxy settings in the Makefile to be used
during the execution of docker build. The settings are optional to be
used and are set up similarly to other Airship projects such as [0] and
[1].

To set proxy settings during docker build, execute the command as
follows:

```
make docker-image \
  -e PROXY=<The proxy URL> \
  -e NO_PROXY=<comma-separated list of URLs/IPs not using the proxy> \
  -e USE_PROXY=true
```

[0] https://opendev.org/airship/pegleg/src/branch/master/Makefile
[1] https://opendev.org/airship/promenade/src/branch/master/Makefile

Change-Id: I92258465d9638b40797d38ba5d8b835fc38df23f
2019-12-19 18:22:41 +00:00
Ian Howell
7ecae507e0 Add make tidy to check for up-to-date go.mod
This commit adds the `tidy` target. This target will fail if the go.mod
or go.sum files are not up to date with the code. It will also be a
prerequisite to the `lint` command, implying that it will also be caught
by the Zuul gates.

Change-Id: Ie33de2d793beca100435670346eb0e2ea7b3b0dd
2019-11-15 09:41:41 -06:00
Zuul
1a1a9391ed Merge "Fix the make cover target" 2019-11-05 15:52:14 +00:00
Ian Howell
6fb4af06a1 Fix the make cover target
This change has multiple benefits:

* Allows testing/coverage Makefile flags to be set from the command
  line. This allows a dev to easily test both correctness and coverage of
  a subset of packages by simply providing a value for the `PKG` make
  variable.

* Fixes an issue where an error message was being printed to stderr on
  each Zuul run, since the `COVER_PKG` variable required that Go is
  installed, and yet we don't install Go to the Zuul VM.

* Fixes an issue where Zuul VMs were crashing due to an Out Of Memory
  error. This was happening because of the way that the `coverpkg` flag
  works, which caused tests to occasionally require more than 8G of
  memory, exceeding the amount of memory on each Zuul VM.

* Cleans up the output when running `make cover`. Previously, the list
  of all packages was printed for coverage for each package, creating a
  lot of noise on the console.

However, it has the following negative:

* With the removal of the `coverpkg` flag, the `go test` command will no
  longer check coverage of packages which do not include any `_test.go`
  files. This puts the responsibility of assuring that coverage is
  correctly reflected onto the shoulders of developers and reviewers.

Change-Id: I7fef1dd26ef19b5f2ab4cf2d1be223bf82210492
2019-11-01 15:28:56 -05:00
Ian Howell
5b25f0bd00 This removes golangci-lint as a go module dependency.
golangci-lint is GPL-licensed, and its presence in go.mod might lead
people to believe that airshipctl links against a GPL-licensed
dependency. In reality airshipctl does not link against golangci-lint
(thanks to the // +build tools comment), but this change helps to avoid
that misconception from being made.

Change-Id: I95c223a7f6597e1aa07a025ed2cebbbd6bd7cbd8
2019-10-30 09:26:42 -05:00
James Gu
d294a6ba37 Move up to go 1.13.1
Updated go version to 1.13.1 and updated dependency versions in mod
file. Updated Dockerfile to use golang 1.13.1 image. Removed GO111MODULE
env in Makefile which is no longer necessary.

Change-Id: Ib120e8bdfbd3724b72169c4b5ce2cbc7fd9f3ce6
2019-10-04 15:31:45 +00:00
Ian Howell
dc9c78b210 Fix the coverage tests
This fixes a bug where `make cover` was missing packages. The target now
covers all code except for code placed in the top level package (such as
main.go) and anything placed in the testutils directory.

This also fixes minor issues with the Dockerfile and the coverage_check
script

Note that this commit also strives to increase code coverage beyond the
80% margin

Change-Id: I9e1cbcf841cc869345a00f05e39774cb3da10065
2019-10-02 15:18:56 -05:00
Ian Howell
1c999e2095 Add coverage checks
This commit adds a makefile target for generating a report for unit test
coverage. It also adds the coverage_check tool to assert that the actual
test coverage meets the specified requirement.

This also improves various aspects of the testing utilities:
* The "test" package has been renamed to "testutil"
* The "Objs" member has been removed from the CmdTest object
* The "Cmd" member has been added to the CmdTest object. This allows
  testing of multiple variants of a root airshipctl command

Finally, this commit includes additional tests for root.go. These are
required in order to meet the required coverage threshold.

Change-Id: Id48343166c0488c543a405ec3143e4a75355ba43
2019-08-15 11:11:51 -05:00
Ian Howell
c3f61a37c7 Fix various minor nits in the Makefile
* Remove the unused SCRIPTS_DIR variable
* Make bracket/parenthesis usage consistent
* Add the delete-golden target and add it as a pre-req for
  update-golden. This will help to keep the testdata directories from
  accruing bloat

Change-Id: Ie4b7377872d48371e5e134fe18ed9347e89b7c70
2019-07-17 08:17:54 -05:00
Ian Howell
129ccb3e3e Add a configuration file for golangci-lint
This adds the .golangci.example.yaml, allowing for more precise control
over how linting is done.

This configuration is highly opinionated and is subject to change.

Change-Id: Ida13bf0dca12e152b75427f5060ad4931e426c72
2019-07-15 12:11:08 -05:00
Pete Birley
5cf0b2327c CI: Add a functional test for interacting with a existing k8s cluster
This adds Ansible playbooks to install kubernetes using minikube and
then assert that airshipctl is able to communicate with the cluster.

Change-Id: Ice5cf3de8d9192d508c79c68caf8f5a61896da9c
Signed-off-by: Pete Birley <pete@port.direct>
2019-07-09 21:04:15 +00:00
Ian Howell
6e41a56a7d Rename module to reflect its new location
Change-Id: I5c8c524c4d3f569b21b5b829722b3c49952cb0ab
2019-07-01 12:15:29 -05:00
Ian Howell
d6b06a8787 Zuul: Switch from travis to zuul
This PS adds the skeleton for a set of zuul checks and gates for
airshipctl and removes the travis config.

This PS also removes some dead code from the util package. This change
is required to get unit tests passing.

Change-Id: Ifb1be49cb1bb82c62a0085b6da9b8ff1b261a95b
2019-07-01 11:00:26 -05:00
Ian Howell
50f29ca75d Remove coverage tests 2019-06-06 08:59:59 -05:00
Ian Howell
cb0d15bdb8 Fix coverage to handle all packages sans generated code 2019-05-30 15:34:10 -05:00
Ian Howell
e4b45002c9 Enable the linter, and fix uncovered linting issues
This uses the current master commit of golangci-lint. We will want to
update to use a tagged version as soon as the golangci-lint team pushes
a new tag
2019-05-30 10:02:54 -05:00
Ian Howell
d68cfdfe6b Force coverage to check all packages 2019-05-28 15:38:24 -05:00