airship/airshipctl
Code Issues Proposed changes

Merge "Reverting secret-generation template and adding examples"

Browse Source
This commit is contained in:
Zuul 2021-02-02 05:40:18 +00:00 committed by Gerrit Code Review
commit aca738afa1
15 changed files with 195 additions and 260 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

118
manifests/function/generate-secrets-example/README.md Normal file
View File

@ -0,0 +1,118 @@
Function: generate-secrets-example
=================================
This function provide an example on how to generate secrets using templator
and variable catalogue. The generated secrets are usually of
`kind: VariableCatalogue`. These generated secrets then be used in
conjuction with `kind: ReplacementTransformer` to subsitute accordingly
in the site manifests. If the generated secrets needs to be deployed
on the cluster then define the secret as `kind: Secret` and appropriately
mark it with `deploy-k8s: true` annotation.
## Generating & Encrypting Secrets
Make a copy of this folder to the appropraite site for which secrets has to
be generated and then edit the [secret-generation.yaml](secret-generation.yaml)
with the required secret generation details.
For example refer to [generator](../../site/test-site/target/generator/) folder.
Once the secret definitions are in place in the site manifests, we can
add a new phase to generate secrets pointing to the folder in site manifests.
Below is an example of how to add phase to the [phases.yaml](../../phases/phases.yaml).
```
apiVersion: airshipit.org/v1alpha1
kind: Phase
metadata:
name: secret-generate
config:
executorRef:
apiVersion: airshipit.org/v1alpha1
kind: GenericContainer
name: encrypter
documentEntryPoint: target/generator
```
The executorRef is of `kind: GenericContainer` and should also have the
following definition in [executor.yaml](../../phases/executor.yaml)
```
---
apiVersion: airshipit.org/v1alpha1
kind: GenericContainer
metadata:
name: encrypter
labels:
airshipit.org/deploy-k8s: "false"
kustomizeSinkOutputDir: "target/generator/results/generated"
spec:
container:
image: quay.io/aodinokov/sops:v0.0.3
envs:
- SOPS_IMPORT_PGP
- SOPS_PGP_FP
config: |
apiVersion: v1
kind: ConfigMap
data:
cmd: encrypt
unencrypted-regex: '^(kind|apiVersion|group|metadata)$'
```
The container spec in the `kind: GenericContainer` is specified with
sops spec so that the generated secrets would be encrypted and
then stored in the `kustomizeSinkOutputDir` directory. Sops uses pgp keys
and sops fingerprint key environment variable from the terminal to
perform encryption on the generated secrets.
## Steps to execute using airshipctl command
1. Sops environment variable has to be exported which will be
used for encryption. Download the sops key file. If you want to use
custom sops key copy it to the current location with filename as `key.asc`.
`curl -fsSL -o key.asc https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc`
2. Export key file and set corresponding fingerprint which will be
used for encryption.
`export SOPS_IMPORT_PGP="$(cat key.asc)" && export SOPS_PGP_FP="FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4"`
3. Then run the airshipctl command
`airshipctl phase run <secret-generate>`
Once the command executes successfully, we can see the generated and
encrypted secrets will be placed in `kustomizeSinkOutputDir`.
## Generate Secrets without encryption(Not recommended)
In case if no encryption is required for the secrets then use the below
`kind: GenericContainer` definition in the [executor.yaml](../../phases/executor.yaml)
```
---
apiVersion: airshipit.org/v1alpha1
kind: GenericContainer
metadata:
name: encrypter
labels:
airshipit.org/deploy-k8s: "false"
kustomizeSinkOutputDir: "target/generator/results/generated"
spec:
container:
image: quay.io/airshipit/templater:latest
config: |
foo: bar
```
## Decrypt to read the secrets
To decrypt the secrets for readability purposes run the kustomize build
command on the generated secrets folder with the [kustomization.yaml](../../site/test-site/target/generator/results/kustomization.yaml) and [decrypt-secrets.yaml](../../site/test-site/target/generator/results/decrypt-secrets.yaml)
files in place in the same folder.
Kustomize command to decrypt:
`KUSTOMIZE_PLUGIN_HOME=$(pwd)/manifests SOPS_IMPORT_PGP=$(cat key.asc) kustomize build \ --enable_alpha_plugins \
manifests/site/test-site/target/generator/results`

36
manifests/function/generate-secrets-example/SecretGenerateExamples.md Normal file
View File

@ -0,0 +1,36 @@
## Secret Generation Examples
Below examples show different function calls in templater that can be used for creation of secrets.
1. derivePassword function to generate password
```
password: {{ derivePassword 1 "long" (randAscii 10) "user" "example.com" }}
```
2. To generate randonm Ascii or AlphaNum or Numeric or Alpha keys
```
test1: {{ randAlphaNum <len> }}
test2: {{ randAlpha 5 }}
test3: {{ randNumeric 12 }}
test4: {{ randAscii 10 }}
```
3. Generate keys matching regex
```
regexkey: {{ regexGen "abc[x-z]+" 6 }}
```
4. To generate certificate authorities
```
{{- $ca := genCA <commonName> <validity> }}
{{- $ca := genCA "foo-ca" 365 }}
{{- $ca := genCAWithKey "foo-ca" 365 (genPrivateKey "rsa") }}
```
5. Certificate generation(selfsigned and signed)
```
{{- $cert := genSelfSignedCert <cn> <ip_list> <dns_list> <validity> }}
{{- $cert := genSelfSignedCertWithKey "foo.com" (list "10.0.0.1" "10.0.0.2") (list "bar.com" "bat.com") 365 (genPrivateKey "ecdsa") }}
{{- $cert := genSignedCert "foo.com" (list "10.0.0.1" "10.0.0.2") (list "bar.com" "bat.com") 365 $ca }}
{{- $cert := genSignedCert "foo.com" (list "10.0.0.1" "10.0.0.2") (list "bar.com" "bat.com") 365 $ca (genPrivateKey "ed25519")}}
```

2
manifests/function/generate-secrets-example/kustomization.yaml Normal file
View File

@ -0,0 +1,2 @@
generators:
- secret-generation.yaml

39
manifests/function/generate-secrets-example/secret-generation.yaml Normal file
View File

@ -0,0 +1,39 @@
apiVersion: airshipit.org/v1alpha1
kind: Templater
metadata:
name: secret-template
annotations:
config.kubernetes.io/function: |
container:
image: quay.io/airshipit/templater:latest
values:
clusterCa:
cn: "Kubernetes API"
validity: 3650
password:
len: 1
cap: "long"
masterKey: "test"
siteKey: "user"
sitePassword: "example.org"
template: |
apiVersion: airshipit.org/v1alpha1
kind: VariableCatalogue
metadata:
labels:
airshipit.org/deploy-k8s: "false"
name: password-secret
annotations:
config.kubernetes.io/path: secrets.yaml
passkey: {{ derivePassword (.password.len|toUint32) .password.cap .password.masterKey .password.siteKey .password.sitePassword }}
rand1: {{ randAlphaNum 4 }}
rand2: {{ randAlpha 5 }}
rand3: {{ randNumeric 12 }}
rand4: {{ randAscii 10 }}
regexkey: {{ regexGen "ed[0-9]*[xY]{3,7}" 7 }}
{{- $ca := genCA .clusterCa.cn .clusterCa.validity }}
{{- $cert := genSignedCert "foo.com" (list "10.0.0.1" "10.0.0.2") (list "bar.com" "bat.com") 365 $ca }}
tls.key: {{ $ca.Key|b64enc|quote }}
tls.crt: {{ $ca.Cert|b64enc|quote }}
tlsCert.key: {{ $cert.Cert|b64enc|quote }}
tlsCert.crt: {{ $cert.Key|b64enc|quote }}

41
manifests/function/generate-secrets/generate-certificates-template.yaml
View File

@ -1,41 +0,0 @@
apiVersion: airshipit.org/v1alpha1
kind: Templater
metadata:
name: generate-certificates-template
annotations:
config.kubernetes.io/function: |-
container:
image: quay.io/airshipit/templater:latest
values:
certificates:
template: |
{{- range $key, $val := .certificates }}
{{- $secretName := $key }}
{{- $secret := $val }}
{{- $ca := "" }}
{{- if not .validity }}
{{- $_ := set . "validity" 365 }}
{{- end }}
{{- if not .cn }}
{{- $_ := set . "cn" "kubernetes" }}
{{- end }}
{{- if .keyEncoding }}
{{- $ca = genCAWithKey .cn .validity (genPrivateKey .keyEncoding)}}
{{- else}}
{{- $ca = genCA .cn .validity }}
{{end -}}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ $secretName }}
{{- if $secret.deployk8s }}
namespace: {{ $secret.namespace | default "default" }}
{{- end }}
labels:
airshipit.org/deploy-k8s: {{ $secret.deployk8s | default "false" }}
data:
tls.crt: {{ $ca.Cert|b64enc|quote }}
tls.key: {{ $ca.Key|b64enc|quote }}
type: kubernetes.io/tls
{{ end -}}

53
manifests/function/generate-secrets/generate-passphrases-template.yaml
View File

@ -1,53 +0,0 @@
apiVersion: airshipit.org/v1alpha1
kind: Templater
metadata:
name: generate-passphrases-template
annotations:
config.kubernetes.io/function: |-
container:
image: quay.io/airshipit/templater:latest
values:
passphrases:
template: |
{{- range $key, $val := .passphrases }}
{{- $secretName := $key }}
{{- $secret := $val }}
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: {{ $secretName }}
{{- if $secret.deployk8s }}
namespace: {{ $secret.namespace | default "default" }}
{{- end }}
labels:
airshipit.org/deploy-k8s: {{ $secret.deployk8s | default "false" }}
data:
{{range $secret.values -}}
{{- if not .keyName }}
{{- $_ := set . "keyName" "password" }}
{{- end }}
{{ if not .generationType -}}
{{- fail "no valid generationType specified!" }}
{{ end -}}
{{if eq .generationType "static" -}}
{{ .keyName }}: {{ .value | b64enc }}
{{else if eq .generationType "randAscii" -}}
{{ .keyName }}: {{ randAscii .length | b64enc }}
{{else if eq .generationType "randAlpha" -}}
{{ .keyName }}: {{ randAlpha .length | b64enc }}
{{else if eq .generationType "randAlphaNum" -}}
{{ .keyName }}: {{ randAlphaNum .length | b64enc }}
{{else if eq .generationType "randNumeric" -}}
{{ .keyName }}: {{ randNumeric .length | b64enc }}
{{else if eq .generationType "regexGen" -}}
{{ .keyName }}: {{ regexGen .regex (.limit | int) | b64enc }}
{{else if eq .generationType "derivePassword" -}}
{{ .keyName }}: {{ derivePassword (.length | toUint32) .passwordType .masterPassword .user .site | b64enc }}
{{else -}}
{{ $error := printf "%s is not a valid generationType!" .generationType }}
{{- fail $error }}
{{end}}
{{end -}}
{{end -}}

5
manifests/function/generate-secrets/kustomization.yaml
View File

@ -1,5 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- generate-passphrases-template.yaml
- generate-certificates-template.yaml

4
manifests/function/generate-secrets/replacements/kustomization.yaml
View File

@ -1,4 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- secrets.yaml

27
manifests/function/generate-secrets/replacements/secrets.yaml
View File

@ -1,27 +0,0 @@
apiVersion: airshipit.org/v1alpha1
kind: ReplacementTransformer
metadata:
name: generate-secret-replacements
annotations:
config.kubernetes.io/function: |-
container:
image: quay.io/airshipit/replacement-transformer:latest
replacements:
- source:
objref:
name: generate-secret-catalogue
fieldref: "{.generate.passphrases}"
target:
objref:
kind: Templater
name: generate-passphrases-template
fieldrefs: ["{.values.passphrases}"]
- source:
objref:
name: generate-secret-catalogue
fieldref: "{.generate.certificates}"
target:
objref:
kind: Templater
name: generate-certificates-template
fieldrefs: ["{.values.certificates}"]

28
manifests/function/generatesecrets-example/README.md
View File

@ -1,28 +0,0 @@
Function: generatesecrets-example
=================================
This function defines a secrets variable catalogue profile that
can be consumed by the generate-secrets function to generate secrets.
Using this example we can build other catalogues to generate passphrases
and certificates.
In the `example` defined passphrases and certificates fields are defined.
Sprig library templater functions and other custom defined functions
will be called to generate the respective passphrases and certificates.
In passphrases catalogue the `generationType` field has to be specified, so that the
passphrase generation happens based on the function. Here is the list of valid
`generationType` functions supported as of now: `randAscii`, `randAlpha`,
`randAlphaNum`, `randNumeric`, `derivePassword`, `regexGen`. Along with the
`generationType` the corresponding fields for that function has to be specified.
Refer to the `example` for the required fields for specific `generationType`.
If no `generationType` or inavlid type is specified an appropriate
error will be thrown and execution fails.
For certificate generation, commonName(`cn`), `validity`, `keyEncoding` are
the valid fields that are to be specified. If `cn` and `validity` are not
specified they take "kubernetes" and "365" days as default values.
The `/replacements` kustomization contains a substitution rule that injects
the variables specified into the generate-secrets function template, which will be
used to generate the respective passphrases and certificates based on the variables.

20
manifests/function/generatesecrets-example/certificates.yaml
View File

@ -1,20 +0,0 @@
apiVersion: airshipit.org/v1alpha1
kind: VariableCatalogue
metadata:
# NOTE: change this when copying this example
name: certificates-example
labels:
airshipit.org/deploy-k8s: "false"
generate:
certificates:
ca-cert:
namespace: dummy
cn: kubernetes
validity: 20
ca-cert-key:
deployk8s: true
keyEncoding: "rsa"
namespace: test
cn: k8
validity: 365

5
manifests/function/generatesecrets-example/kustomization.yaml
View File

@ -1,5 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- passphrases.yaml
- certificates.yaml

40
manifests/function/generatesecrets-example/passphrases.yaml
View File

@ -1,40 +0,0 @@
apiVersion: airshipit.org/v1alpha1
kind: VariableCatalogue
metadata:
# NOTE: change this when copying this example
name: passphrases-example
labels:
airshipit.org/deploy-k8s: "false"
generate:
passphrases:
secret1:
namespace: ns1
deployk8s: true
values:
- keyName: key1
generationType: derivePassword
passwordType: long
user: test
site: example.com
masterPassword: master
length: 2
- generationType: randAlpha
length: 3
- keyName: key3
generationType: static
value: mypass
- keyName: key4
generationType: randAlphaNum
length: 4
- keyName: key5
generationType: randNumeric
length: 5
secret2:
namespace: test
values:
- generationType: randAscii
length: 3
- keyName: key2
generationType: regexGen
regex: "[efghul][a-z]{2,3}[0-9]{2,8}"
limit: 10

4
manifests/function/generatesecrets-example/replacements/kustomization.yaml
View File

@ -1,4 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- secrets-replace.yaml

33
manifests/function/generatesecrets-example/replacements/secrets-replace.yaml
View File

@ -1,33 +0,0 @@
# These rules inject passphrases and certificate variable values
# from the `generate-secret-catalogue` into the `generate-passphrases-template`
# and `generate-certificates-template` function's Template plugin configs respectively.
apiVersion: airshipit.org/v1alpha1
kind: ReplacementTransformer
metadata:
# NOTE: change this when copying this example
name: generatesecrets-example-replacements
annotations:
config.kubernetes.io/function: |-
container:
image: quay.io/airshipit/replacement-transformer:latest
replacements:
- source:
objref:
# NOTE: change this to match your passphrases's metadata.name
name: passphrases-example
fieldref: "{.generate.passphrases}"
target:
objref:
kind: Templater
name: generate-passphrases-template
fieldrefs: ["{.values.passphrases}"]
- source:
objref:
# NOTE: change this to match your certificates's metadata.name
name: certificates-example
fieldref: "{.generate.certificates}"
target:
objref:
kind: Templater
name: generate-certificates-template
fieldrefs: ["{.values.certificates}"]