130 Commits

Author SHA1 Message Date
Ruslan Aliev
0e44b580f0 Fix pulling docs from remote branch
Currently airshipctl is not able to checkout remote branch
due to using simple branch reference name instead of remote one.
This patch fixes appropriate problem and allows user to specify
what kind of branch to use - local or remote one (since reference
name will be different in these scenarios).

Change-Id: I1fea29f84097b9e7160597003d0e8961f4d3aca6
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2021-01-14 12:35:56 -06:00
bijayasharma
f9378e74b0 Add phase tree command in package module
* This commit is the smaller patchset from earlier work:
  https://review.opendev.org/#/c/750449/

* This is the first series of patchset

Change-Id: Ib1ff74eba65de7c7c59cf8f4fd26b15e388ba368
Signed-off-by: bijayasharma <vetbijaya@gmail.com>
Relates-To: #296
2021-01-06 19:38:54 +00:00
Sirisha Gopigiri
154bcec95c Adding Filesystem check function to templater
The following code uses the documentFs interface and adds
simple fileExists function to templater plugin.

Change-Id: Ia53c573e54188960eaf99f7c48469359d4e3688b
2021-01-05 05:51:01 +00:00
Sirisha Gopigiri
91b2164a4f Adding Regex support for Templater
Adding go code to templater to support generation of strings
based on the regular expression passed.

Closes: #438
Change-Id: Ife0c4f8d5dd8032408875c73ba098919470be9e9
2021-01-04 18:18:27 +05:30
Sirisha Gopigiri
247047f2a3 Update Sprig library and add Uint32 support
Below code updates sprig library from v2.22 to
v3.2. Also added support for uint32 coversion.

Change-Id: I263cf6f1915d2f892f6f77efdcdd477ad902f527
2020-12-21 17:40:11 +05:30
Anderson, Craig (ca846m)
2daacf5f2a image-builder integration for ISO builds
This introduces airshipctl integration with image-builder [0], which
replaces the existing isogen tool for ephemeral ISO generation.

The airshipctl isogen executor has been updated for building ephemeral
ISOs using the image-builder container. The ability for user-declared
filenames for cloud-init user data and network data was removed, since
the user's only interest is in supplying the relevant overrides, not in
transparent naming coordination with the image-builder container. A new
object is added to the document package to identify the document kind,
label, and key to retrieve data from since this is pattern we will
reuse elsewhere.

Progress flag removed as requsted. Progress is reported directly by the
image-builder container.

Isogen debug flag removed in favor of using log.DebugEnabled()

[0] https://review.opendev.org/#/c/730777/

Depends-On: https://review.opendev.org/c/airship/images/+/730777/
Change-Id: I545004feaf2116f8ffb29faf6f7f7f5fcfe24fff
2020-12-17 14:47:53 -08:00
Dmitry Ukov
aa46185975 Uplift kustomize api and kyaml versions
Change-Id: Id8334a74efb7f1004a36f2bfc0c6eca81380ef4d
Relates-to: #369
2020-12-04 13:43:26 +04:00
Ruslan Aliev
ca71de3951 Move document filesystem to a separate package
In order to widely use filesystem interface in airshipctl
there is need to move it to separate package to avoid
importing unnecessary dependencies from document
package and, as a consequence, possible cyclic dependencies.
 * filesystem moved from document/fs to pkg/fs

Change-Id: I3b6298462f03db43594a9fa26bf23ab7687c5589
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #415
2020-11-30 19:51:30 -06:00
Dmitry Ukov
9c69138990 Remove document plugin subcommand
This change finalises shift to new version of kustomize plugins

Change-Id: Ie47f42a9e2c4dc03fc4dca437020ad71a564d894
2020-11-11 14:59:14 +04:00
Sirisha Gopigiri
d82e04ea5e Adding decoding and encoding feature to Replacement Transformer
This PS adds decoding features to ReplacementTransformer to decode
the source objectRef when source is of `kind: Secret` and has `data`
field. It also encodes the value in the target fieldRefs before
replacement if the target is `kind: Secret` and has `data` fields.
Throws an error if the target fieldRefs have both `data` and `stringData`
for replacement.

Change-Id: I1d918058409b3511faa9a99512d25574027bda86
2020-10-27 17:45:29 +05:30
Dmitry Ukov
96cbb045a4 Remove legacy code from ReplacementTransformer
Change-Id: Ia247ee15efd5f66b5e36cf49339474548d94765d
2020-10-23 14:27:55 +04:00
Stanislav Egorov
71f04ed976 Proxy config for control plane
Along with the change to support proxy configuration for control plane
there is a change for replacement transformer to support arrays of strings.

The current implementation of cloud-init configuration for the control plane
based on KubeadmControlPlane object and uses an array of strings for commands.

Change-Id: Id0a49cc1a0fdbc392c7c7a56859b21737065b0d6
2020-10-20 21:38:18 -07:00
Zuul
d812b6c165 Merge "Introduce Replacement Transformer plugin container" 2020-10-19 18:51:49 +00:00
Ruslan Aliev
0c870e5244 Remove clustertype-related functions
We don't use clustertype anymore, so all the related functions must
be deleted or adjusted as well as appropriate config fields.

Change-Id: I3931fdc71d4318e916f8bbc2d94e062c9df5f641
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #349
2020-10-16 11:31:58 -05:00
Dmitry Ukov
d783d71f05 Introduce Replacement Transformer plugin container
Relates-To: #341
Change-Id: I65a5b8cf3bcf94321fc02f240b1d8075bca45c0c
2020-10-16 16:00:56 +04:00
Zuul
f92e8bd042 Merge "Replace PrimaryRepositoryName with PhaseRepositoryName" 2020-10-13 20:50:53 +00:00
Dmitry Ukov
ca1a3a2d0b Migrate Replacement Transformer plugin
Plugin extended to support new kustomize plugin framework which
consider each plugin as a container

Change-Id: If55b7093f711401165b7d4fd3f5b1059fde464ff
Relates-To: #340
2020-10-13 20:43:55 +04:00
Vladimir Kozhukalov
0dc4ab7491 Replace PrimaryRepositoryName with PhaseRepositoryName
Change-Id: I2276a3b3ea50b8e81dd240815cd00a0a2a2af96e
Relates-To: #356
2020-10-12 15:28:27 +03:00
Dmitry Ukov
025c2172d6 Implement JSON path filter for kyaml.RNode
Change introduces kyaml.RNode Pipe Filter which uses k8s go-client JSON
path parser. This enables to use JSON queries defined by
https://goessner.net/articles/JsonPath/

Change-Id: I6c2276f27652190ed9d183cea0e45eb118346c6b
Relates-To: #340
2020-10-07 15:27:19 +04:00
Dmitry Ukov
0d475ec6c5 Extend plugin interface with execute method
Change introduces evolution of airshipctl document plugins which
are compatible to the new approach for kustomize plugins.

Change-Id: I422110ba523b97c8a80a0b82cb43b70a4ee8e558
Relates-To: #322
2020-10-05 13:24:03 +04:00
Zuul
e4a2c68d3e Merge "Use BundleFactory instead of bundle in executors" 2020-09-30 14:24:52 +00:00
Kostiantyn Kalynovskyi
e6e56ffa2e Use BundleFactory instead of bundle in executors
This will allow to avoid using nil bundle and performing
additional checks inside the executors

Change-Id: Ia061cccf7e0a7948bbc5c8fce24e0e6f8b3256e1
2020-09-28 19:39:05 -05:00
Ruslan Aliev
d8a6139e81 Add document pull step to gate script runner
Currently we don't test document pull step by rewriting
manifest_directory variable as current source path, which
makes a mess in target_path, manifest_path and related paths
(concatenation of target_path+manifest_path won't return
an existing and proper location). This patch organizes
mentioned variables and enables document pull step which
starts working properly. The latest repo state used as
git source for this command.

Change-Id: I5abce73877441c7529f2f77add79cf410e2226d8
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-25 00:39:28 -05:00
Zuul
233b87a103 Merge "Move plugins configurations to API module" 2020-09-24 15:34:08 +00:00
Ruslan Aliev
2e85043c20 Fix typos
Change-Id: I0ce87ec00d2b5974cf861d9c9685f23d7d0f5fc1
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2020-09-21 02:49:33 -05:00
Dmitry Ukov
034efc3682 Move plugins configurations to API module
Data structure representing plugin configurations should be a part of
airshipctl API module. Plugin implementations will reside in document
package.

Change-Id: Id2e359b747a16a5573052cfb05c1148d346db508
Relates-To: #322
2020-09-17 19:22:07 +04:00
Matt McEuen
9997e7427e Allow substring replacement of numerics
This change adds the ability to use substring substitution to inject
numeric source values into string targets.  This is required for our
networking catalogue, since a numeric k8s API port needs to be inserted into
numeric and (sub)string targets in different phases.

Co-Authored-By: Ian Howell <ian.howell0@gmail.com>
Change-Id: I24beb46a2bda4e118406129a0a922b0c56142c76
2020-09-14 16:03:51 -05:00
Ruslan Aliev
285aeeba8c Refactor document pull command
This command was refactored for usage with new config factory.
Config object now is being initialized in pkg module on demand,
which provides more flexibility and cleaner code.

Change-Id: Iff75eb66db2fee922e6db1bf2930892c02c3b601
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-02 23:41:05 -05:00
Zuul
ad21e64a98 Merge "Substring replacement fix in ReplacementTranformer" 2020-09-01 16:56:23 +00:00
Ruslan Aliev
78364d7d44 Refactor document plugin command
AirshipCTLSettings are not used here, so it was removed.

Change-Id: If4bf64b9991b4ac05898632c79bacbacd7872635
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #327
2020-09-01 10:15:58 -05:00
Sirajudeen
7e391bfc1b Substring replacement fix in ReplacementTranformer
* Substring replacement failing in ReplacementTranformer when
target substring contains periods (.)

* This is temporary fix till the kpt-function way of replacement
  transformer is implemented

* In this fix it is assumed that the substring condition can only occur at
  the end of the path.

  example:
   expected: replace the nginx version from 1.7.9 to 1.17.0 for container named
             `nginx-tagged`

   target condition:
      spec.template.spec.containers[name=nginx-tagged].image%1.7.9%

Change-Id: I76fb65f69a6eedf3cbdd692d3bc1835a214dc8fa
Relates-To: #336
Closes: #336
2020-08-31 17:43:21 +00:00
Stanislav Egorov
61ed8d30b7 Fixed TODO in document/repo package
Implemented strongly-typed errors for document/repo

Change-Id: I572489b4c48ff4cd75a4e2f9a6b1d27ad1f840dd
2020-08-28 12:24:54 -07:00
Zuul
1c6427248b Merge "Add unit tests for missing Bundle functions" 2020-08-10 16:23:22 +00:00
Ian Howell
3410000626 Add unit tests for missing Bundle functions
This increases the unit test coverage of the pkg/document package

Change-Id: I037a66879fc4ce1f31e394dc2bda5ca61391c3e2
2020-08-05 09:31:09 -05:00
bijayasharma
be03e9067c [#309] Removed unused constant
* This commit will remove unused constant.
* According to issue, we have one unused constant:
  pkg/document/constants.go

Change-Id: Ib93fa01dbd60bb48cf6b8b77298bd0ab4d5b82a4
2020-08-05 12:48:11 +00:00
Vladislav Kuzmin
d7101bcb2c Refactor environment module
Move code from environment module to util and document
modules to prevent circular dependencies.

Relates-To: #264

Change-Id: Ifec9ab3f1ada01cc662e655ff4a6f2cfebe4150a
2020-07-31 14:03:39 +04:00
Zuul
645aeeb293 Merge "Introduce KubeConfig API object" 2020-07-30 20:22:19 +00:00
Zuul
94af5394b1 Merge "documentation changes" 2020-07-30 19:51:49 +00:00
Dmitry Ukov
5f1b93604e Introduce KubeConfig API object
Change-Id: I5894000377b1d22aac02c339ab1938ba35bcf651
2020-07-30 16:20:37 +00:00
muly
9d4347b0c1 documentation changes
1) modifications to exported functions/variables documentation
2) moved some methods into a different folder

Change-Id: I212442516424bde3284f75b326d152c290d20c80
2020-07-29 09:17:00 -04:00
Ruslan Aliev
71f7a8bc85 Improve document pull command
This patch refactors some code and adds useful debug/info log output.

Change-Id: I590f6e5c300e5c91443af0a8fdb1073229602a67
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Relates-To: #278
2020-07-28 22:01:20 +00:00
Matt McEuen
0324993d60 Add an m3 host generator function
This PS has a function which constructs a collection of Metal3 BareMetalHost
resources, along with associated configuration Secrets.
It solves for a couple of things:

1. pulling the nitty gritty details for generating BMH into one reusable place,
2. allowing the site-specific details to be filled in via catalogues of values

This function leverages a couple of different plugins in sequence:
The airshipctl Replacement plugin, which pulls the site-specific data from
the catalogue documents into a Templater plugin configuration; and then
the airshipctl Templater plugin, which generates a variable number of
BMHs in a data-driven fashion.

More details can be found in the README.md in this patchset.

Closes: #245

Change-Id: I3ddbd36dc53ea6afbd633098c985f4b28bcbb793
2020-07-17 15:30:26 -05:00
Sirajudeen
1d34132b87 Added unit test for missing document functions
* Added more unit tests to increase test coverage

Change-Id: I292ddcd1c98ae49b37032a7d6111e063be397de3
2020-07-16 15:36:04 -05:00
Sirajudeen
35d119d3d1 Added unit test for dochelper, config
* added more negative test cases to dochelper, config to
  increase the test coverage

Change-Id: I3179823f9196157e72009ff998cfb8dd348fee1f
2020-07-14 16:48:03 -05:00
Kostiantyn Kalynovskyi
edf09b0fd5 Add Append method to bundle interface
This method would allow us to append bundle with new documents

Change-Id: I201a2a0f2e7f707556a53495538c0f075ce6fe17
Relates-To: #238
2020-07-07 14:45:26 +00:00
Dmitry Ukov
c05286d3f6 Implement document filtering using api objects
Change-Id: I9ac70ed85c949224d091e9e3afead70084b38170
Co-Authored-By: Vladimir Kozhukalov <kozhukalov@gmail.com>
2020-06-23 13:12:33 +04:00
Kostiantyn Kalynovskyi
36facd30eb Add new constructor to create documents
The change would allow to build a document object from bytes

Change-Id: I15c13a960b4e989cffa4525c85b8f32f2a51a0e1
Relates-To: #238
Relates-To: #279
Closes: #279
2020-06-17 22:31:24 +00:00
Dmitry Ukov
efec7ec5e1 Move clusterctl api structs to gloabal api module
Change-Id: I4cd707d286ddac7cdc4aadc66b8257e413e231aa
Closes: #269
2020-06-08 21:08:08 +04:00
Dmitry Ukov
155db7dca5 Extend document interface with utility methods
* GetGroup, GetVersion and GetKind now can be used to construct GVK
  structs
* ToObject deserialize document to a particular object. This is
  needed to precess custom resources which were defined for airship

Change-Id: I527e2c73288e865455853c5553bffd508a2d2eb6
2020-06-02 15:20:01 +04:00
Dmitry Ukov
6d05ae69ee Implement document rendering sub-command
Kustomize filtering and rendering mechanisms are used.

Related: #14
Related: #15
Related: #16
Relates-To: #247
Closes: #247
Co-authored-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ied06b95577772cc51f51f3ebb3acd441388d5cda
2020-05-22 12:05:44 +04:00