airship
/
treasuremap
Code Issues Proposed changes

Use kpt for rook 

This PS is to enable kpt for rook-ceph.

The situation is odd in that we need to use

https://github.com/rook/rook/tree/master/cluster/examples/kubernetes/ceph

as an upstream for each base directory but only need a single file in
each case.

The solution here is to clone the entire upstream directory into each
base directory. Gitignore files are used to prevent having to commit
hundreds of files to the repo.

This is obviously not very efficient but it is the best I could come
up with.

Hopefully kpt will be able to sync single files on the future.

Change-Id: I2a5fa63b56749653880be72333a7fce1620bdb6c
This commit is contained in:
Frank Ritchie 2021-06-11 21:38:07 -04:00
parent b20777dd7d
commit 2b08977431
42 changed files with 224 additions and 105 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
manifests/composite/storage-cluster/Kptfile Normal file
View File

@ -0,0 +1,9 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
dependencies:
- name: upstream
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

22
manifests/composite/storage-cluster/README.md
View File

@ -1,19 +1,11 @@
In-place edits to the local copies of upstream Rook Custom Resource
examples are not recommended. The upstream examples can be considered
an immutable starting point.
# Rook Ceph CephCluster
Changes to the upstream examples should be made via Kustomize.
Kubernetes manifests for deploying a Rook CephCluster Custom Resource.
Rook Custom Resource Examples:
## Update Manifests
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/cluster.yaml
Local: cluster.yaml
Tag: v1.6.3
To update the upstream manifests in this function:
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/toolbox.yaml
Local: toolbox.yaml
Tag: v1.6.3
Kustomize Doc:
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization
1. Update the git references in `Kptfile`.
2. Run `kpt pkg sync .` from this directory.
3. Update any `Rook` container image references defined in version catalogs.

8
manifests/composite/storage-cluster/kustomization.yaml
View File

@ -3,8 +3,8 @@ kind: Kustomization
namespace: rook-ceph
resources:
- ceph-conf.yaml
- cluster.yaml
- toolbox.yaml
- upstream/cluster.yaml
- upstream/toolbox.yaml
# Below is the functions section. You are free to comment out or in oppsite to remove comments
# enabling or disabling any function in the list.
# All functions are independent, and can be deployed simultaneously at any moment.
@ -26,7 +26,7 @@ patchesJSON6902:
- op: replace
path: /spec/dashboard/ssl
value: true
# If you enabling ssl for the dashboard, do not forget
# If you enable ssl for the dashboard, don't forget
# to enable the patch below
- target:
kind: Service
@ -39,7 +39,7 @@ patchesJSON6902:
port: 8443
protocol: TCP
targetPort: 8443
patchesStrategicMerge:
patches:
- |-
apiVersion: ceph.rook.io/v1
kind: CephCluster

4
manifests/composite/storage-cluster/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,4 @@
*
!Kptfile
!cluster.yaml
!toolbox.yaml

11
manifests/composite/storage-cluster/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

0
manifests/composite/storage-cluster/cluster.yaml → manifests/composite/storage-cluster/upstream/cluster.yaml
View File

0
manifests/composite/storage-cluster/toolbox.yaml → manifests/composite/storage-cluster/upstream/toolbox.yaml
View File

28
manifests/function/rook-cluster/Kptfile Normal file
View File

@ -0,0 +1,28 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
dependencies:
- name: cephfs/base/upstream
git:
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3
- name: dashboard/base/upstream
git:
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3
- name: pools/base/upstream
git:
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3
- name: storageclasses/block/upstream
git:
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph/csi/rbd
ref: v1.6.3
- name: storageclasses/file/upstream
git:
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph/csi/cephfs
ref: v1.6.3

33
manifests/function/rook-cluster/README.md
View File

@ -1,31 +1,10 @@
In-place edits to the local copies of upstream Rook Custom Resource
examples are not recommended. The upstream examples can be considered
an immutable starting point.
# Rook Ceph Custom Resources
Changes to the upstream examples should be made via Kustomize.
Kubernetes manifests for deploying select Rook Ceph Custom Resources.
Rook Custom Resource Examples:
## Update Manifests
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/filesystem.yaml
Local: cephfs/base/filesystem.yaml
Tag: v1.6.3
To update the upstream manifests in this function:
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/dashboard-external-http.yaml
Local: dashboard/base/dashboard-external-http.yaml
Tag: v1.6.3
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/pool.yaml
Local: pools/base/pool.yaml
Tag: v1.6.3
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/csi/rbd/storageclass.yaml
Local: storageclasses/block/storageclass.yaml
Tag: v1.6.3
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/csi/cephfs/storageclass.yaml
Local: storageclasses/file/storageclass.yaml
Tag: v1.6.3
Kustomize Doc:
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization
1. Update the git references in `Kptfile`.
2. Run `kpt pkg sync .` from this directory.

2
manifests/function/rook-cluster/cephfs/base/kustomization.yaml
View File

@ -1,5 +1,5 @@
resources:
- filesystem.yaml
- upstream/filesystem.yaml
patchesJSON6902:
- target:
kind: CephFilesystem

3
manifests/function/rook-cluster/cephfs/base/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
*
!Kptfile
!filesystem.yaml

11
manifests/function/rook-cluster/cephfs/base/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

0
manifests/function/rook-cluster/cephfs/base/filesystem.yaml → manifests/function/rook-cluster/cephfs/base/upstream/filesystem.yaml
View File

4
manifests/function/rook-cluster/cephfs/kustomization.yaml
View File

@ -1,5 +1,5 @@
resources:
- ./base
patchesStrategicMerge:
- base
patches:
- cephfs-mds.yaml
- cephfs-pool.yaml

2
manifests/function/rook-cluster/dashboard/base/kustomization.yaml
View File

@ -1,5 +1,5 @@
resources:
- dashboard-external-http.yaml
- upstream/dashboard-external-http.yaml
patchesJSON6902:
- target:
kind: Service

3
manifests/function/rook-cluster/dashboard/base/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
*
!Kptfile
!dashboard-external-http.yaml

11
manifests/function/rook-cluster/dashboard/base/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

0
manifests/function/rook-cluster/dashboard/base/dashboard-external-http.yaml → manifests/function/rook-cluster/dashboard/base/upstream/dashboard-external-http.yaml
View File

6
manifests/function/rook-cluster/pools/base/kustomization.yaml
View File

@ -1,6 +1,6 @@
resources:
- pool.yaml
patchesJSON6902:
- upstream/pool.yaml
patches:
- target:
kind: CephBlockPool
name: replicapool
@ -8,5 +8,3 @@ patchesJSON6902:
- op: replace
path: /metadata/name
value: pool
- op: remove
path: /spec/annotations

3
manifests/function/rook-cluster/pools/base/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
*
!Kptfile
!pool.yaml

11
manifests/function/rook-cluster/pools/base/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

0
manifests/function/rook-cluster/pools/base/pool.yaml → manifests/function/rook-cluster/pools/base/upstream/pool.yaml
View File

2
manifests/function/rook-cluster/pools/kustomization.yaml
View File

@ -1,2 +1,2 @@
resources:
- ./rbd
- rbd

12
manifests/function/rook-cluster/pools/rbd/rbd-pool.yaml
View File

@ -4,9 +4,9 @@ metadata:
name: pool
namespace: rook-ceph # namespace:cluster
spec:
failureDomain: host
replicated:
size: 3
quotas:
maxSize: "0" # e.g. "10Gi" - valid suffixes include k, M, G, T, P, E, Ki, Mi, Gi, Ti, Pi, Ei
maxObjects: 0 # 1000000000 = billion objects, 0 means no quotas
failureDomain: host
replicated:
size: 3
quotas:
maxSize: "0" # e.g. "10Gi" - valid suffixes include k, M, G, T, P, E, Ki, Mi, Gi, Ti, Pi, Ei
maxObjects: 0 # 1000000000 = billion objects, 0 means no quotas

4
manifests/function/rook-cluster/storageclasses/block/kustomization.yaml
View File

@ -1,5 +1,5 @@
resources:
- storageclass.yaml
- upstream/storageclass.yaml
patchesJSON6902:
- target:
kind: StorageClass
@ -8,6 +8,6 @@ patchesJSON6902:
- op: replace
path: /metadata/name
value: block-storage-sc
patchesStrategicMerge:
patches:
- patches/delete-pool.yaml
- patches/rook-ceph-block.yaml

3
manifests/function/rook-cluster/storageclasses/block/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
*
!Kptfile
!storageclass.yaml

11
manifests/function/rook-cluster/storageclasses/block/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph/csi/rbd
ref: v1.6.3

0
manifests/function/rook-cluster/storageclasses/block/storageclass.yaml → manifests/function/rook-cluster/storageclasses/block/upstream/storageclass.yaml
View File

4
manifests/function/rook-cluster/storageclasses/file/kustomization.yaml
View File

@ -1,5 +1,5 @@
resources:
- storageclass.yaml
- upstream/storageclass.yaml
patchesJSON6902:
- target:
kind: StorageClass
@ -14,5 +14,5 @@ patchesJSON6902:
- op: replace
path: /parameters/pool
value: cephfs-data0
patchesStrategicMerge:
patches:
- patches/rook-ceph-cephfs.yaml

3
manifests/function/rook-cluster/storageclasses/file/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
*
!Kptfile
!storageclass.yaml

11
manifests/function/rook-cluster/storageclasses/file/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph/csi/cephfs
ref: v1.6.3

0
manifests/function/rook-cluster/storageclasses/file/storageclass.yaml → manifests/function/rook-cluster/storageclasses/file/upstream/storageclass.yaml
View File

9
manifests/function/rook-operator/Kptfile Normal file
View File

@ -0,0 +1,9 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
dependencies:
- name: upstream
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

11
manifests/function/rook-operator/README.md Normal file
View File

@ -0,0 +1,11 @@
# Rook Ceph Operator
Kubernetes manifests for deploying the Rook Ceph Operator.
## Update Manifests
To update the upstream manifests in this function:
1. Update the git references in `Kptfile`.
2. Run `kpt pkg sync .` from this directory.
3. Update any `Rook` container image references defined in version catalogs.

20
manifests/function/rook-operator/kustomization.yaml
View File

@ -1,4 +1,20 @@
resources:
- upstream
patchesStrategicMerge:
- upstream/common.yaml
- upstream/crds.yaml
- upstream/operator.yaml
patches:
- rook-operator-patch-tolerations.yaml
- |-
kind: ConfigMap
apiVersion: v1
metadata:
name: rook-ceph-operator-config
namespace: rook-ceph
data:
CSI_ENABLE_HOST_NETWORK: "true"
ROOK_CSI_CEPH_IMAGE: "quay.io/cephcsi/cephcsi:v3.3.1"
ROOK_CSI_REGISTRAR_IMAGE: "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.0.1"
ROOK_CSI_RESIZER_IMAGE: "k8s.gcr.io/sig-storage/csi-resizer:v1.0.1"
ROOK_CSI_PROVISIONER_IMAGE: "k8s.gcr.io/sig-storage/csi-provisioner:v2.0.4"
ROOK_CSI_SNAPSHOTTER_IMAGE: "k8s.gcr.io/sig-storage/csi-snapshotter:v4.0.0"
ROOK_CSI_ATTACHER_IMAGE: "k8s.gcr.io/sig-storage/csi-attacher:v3.0.2"

5
manifests/function/rook-operator/upstream/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
*
!Kptfile
!common.yaml
!crds.yaml
!operator.yaml

11
manifests/function/rook-operator/upstream/Kptfile Normal file
View File

@ -0,0 +1,11 @@
apiVersion: kpt.dev/v1alpha1
kind: Kptfile
metadata:
name: upstream
upstream:
type: git
git:
commit: 69591248f69e23964734f0192944ef2442bc7885
repo: https://github.com/rook/rook
directory: cluster/examples/kubernetes/ceph
ref: v1.6.3

11
manifests/function/rook-operator/upstream/README.md
View File

@ -1,11 +0,0 @@
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/common.yaml
Local: common.yaml
Tag: v1.6.3
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/crds.yaml
Local: crds.yaml
Tag: v1.6.3
Upstream: https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/operator.yaml
Local: operator.yaml
Tag: v1.6.3

8
manifests/function/rook-operator/upstream/common.yaml
View File

@ -9,6 +9,14 @@
#
# Most of the sections are prefixed with a 'OLM' keyword which is used to build our CSV for an OLM (Operator Life Cycle manager)
###################################################################################################################
# Namespace where the operator and other rook resources are created
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph # namespace:cluster
# OLM: BEGIN OBJECTBUCKET ROLEBINDING
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:

14
manifests/function/rook-operator/upstream/kustomization.yaml
View File

@ -1,14 +0,0 @@
resources:
- namespace.yaml
- crds.yaml
- common.yaml
- operator.yaml
patchesStrategicMerge:
- |-
kind: ConfigMap
apiVersion: v1
metadata:
name: rook-ceph-operator-config
namespace: rook-ceph
data:
CSI_ENABLE_HOST_NETWORK: "true"

7
manifests/function/rook-operator/upstream/namespace.yaml
View File

@ -1,7 +0,0 @@
# Namespace where the operator and other rook resources are created
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph # namespace:cluster
# OLM: BEGIN OBJECTBUCKET ROLEBINDING
---

12
manifests/function/rook-operator/upstream/operator.yaml
View File

@ -69,12 +69,12 @@ data:
# The default version of CSI supported by Rook will be started. To change the version
# of the CSI driver to something other than what is officially supported, change
# these images to the desired release of the CSI driver.
ROOK_CSI_CEPH_IMAGE: "quay.io/cephcsi/cephcsi:v3.3.1"
ROOK_CSI_REGISTRAR_IMAGE: "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.0.1"
ROOK_CSI_RESIZER_IMAGE: "k8s.gcr.io/sig-storage/csi-resizer:v1.0.1"
ROOK_CSI_PROVISIONER_IMAGE: "k8s.gcr.io/sig-storage/csi-provisioner:v2.0.4"
ROOK_CSI_SNAPSHOTTER_IMAGE: "k8s.gcr.io/sig-storage/csi-snapshotter:v4.0.0"
ROOK_CSI_ATTACHER_IMAGE: "k8s.gcr.io/sig-storage/csi-attacher:v3.0.2"
# ROOK_CSI_CEPH_IMAGE: "quay.io/cephcsi/cephcsi:v3.3.1"
# ROOK_CSI_REGISTRAR_IMAGE: "k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.0.1"
# ROOK_CSI_RESIZER_IMAGE: "k8s.gcr.io/sig-storage/csi-resizer:v1.0.1"
# ROOK_CSI_PROVISIONER_IMAGE: "k8s.gcr.io/sig-storage/csi-provisioner:v2.0.4"
# ROOK_CSI_SNAPSHOTTER_IMAGE: "k8s.gcr.io/sig-storage/csi-snapshotter:v4.0.0"
# ROOK_CSI_ATTACHER_IMAGE: "k8s.gcr.io/sig-storage/csi-attacher:v3.0.2"
# (Optional) set user created priorityclassName for csi plugin pods.
# CSI_PLUGIN_PRIORITY_CLASSNAME: "system-node-critical"