diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst index 64506c63d..813d05182 100644 --- a/doc/source/configuration/index.rst +++ b/doc/source/configuration/index.rst @@ -24,8 +24,10 @@ Kubernetes Configuration .. toctree:: :maxdepth: 1 - k8s_persistent_vol k8s_auth_winactivedir + k8s_persistent_vol + k8s_pod_sec_policies + k8s_res_policies k8s_upgrade ----------------------- diff --git a/doc/source/configuration/k8s_pod_sec_policies.rst b/doc/source/configuration/k8s_pod_sec_policies.rst new file mode 100644 index 000000000..516e15c32 --- /dev/null +++ b/doc/source/configuration/k8s_pod_sec_policies.rst @@ -0,0 +1,127 @@ +===================== +Pod Security Policies +===================== + +This guide describes how to enable Kubernetes pod security policies. + +.. contents:: + :local: + :depth: 1 + +-------- +Overview +-------- + +Pod Security Policies (PSPs) enable fine-grained authorization of pod creation +and updates. :abbr:`PSPs (Pod Security Policies)` control access to security +sensitive aspects of pod specifications such as running of privileged +containers, use of host file system, running as root, etc. PSPs define a set of +conditions that a pod must run with in order to be accepted into the system, as +well as defaults for the related fields. PSPs are assigned to users using +Kubernetes RBAC RoleBindings. See +https://kubernetes.io/docs/concepts/policy/pod-security-policy/ for details. + +When enabled, pod security policy checking authorizes all Kubernetes API +commands against the PSPs which the issuer of the command has access to. If +there are no PSPs defined in the system or the issuer does not have access to +any PSPs, PSP checking will fail to authorize the command. + +StarlingX provides a system service parameter to enable pod security policy +checking. Setting this service parameter also creates two PSPs (privileged and +restricted). Users with the cluster-admin role can access all resources and +therefore have PSPs to authorize against. The parameter also creates two +corresponding roles for specifying access to these PSPs (``privileged-psp-user`` +and ``restricted-psp-user``) for binding to other non-admin type subjects. + +------------------- +Enable PSP checking +------------------- + +Perform the following steps. + +#. Set the Kubernetes ``kube_apiserver admission_plugins`` system parameter to + include PodSecurityPolicy. + + :: + + system service-parameter-add kubernetes kube_apiserver admission_plugins=PodSecurityPolicy + +#. Apply the Kubernetes system parameters. + + :: + + system service-parameter-apply kubernetes + +Use the following commands to view the automatically added PSPs, as well as +privileged and restricted PSPs. + +:: + + kubectl get psp + kubectl describe psp privileged + kubectl describe psp restricted + +------------------------------- +Update role for non-admin users +------------------------------- + +After enabling Pod security policy checking in StarlingX, all users +with the cluster-admin role are unaffected because they have access to the +privileged PSP. However, other users require a new ``RoleBinding`` to either +the privileged-psp-user role or the restricted-psp-user role. + +For example, the following ``RoleBinding`` assigns the restricted PSP to +``basic-user`` in the ``billing-dept-ns`` namespace: + +:: + + apiVersion: rbac.authorization.k8s.io/v1 + kind: RoleBinding + metadata: + name: basic-restricted-psp-users + namespace: billing-dept-ns + subjects: + - kind: ServiceAccount + name: basic-user + namespace: kube-system + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: restricted-psp-user + +This enables ``basic-user`` to create pods in the ``billing-dept-ns`` namespace +subject to the restricted PSP policy. + +--------------------------------- +Bind to the PSP for the namespace +--------------------------------- + +An unexpected behavior when PSP checking is enabled is that the above +``basic-user`` is able to create pods in ``billing-dept-ns`` (subject to the +restricted PSP), however they are **not** able to create deployments. This is +because the pods of the deployment are created using the replicaSet +controller’s serviceAccount and RBAC bindings, not the ``basic-user`` +serviceAccount and RBAC bindings. + +The typical approach for addressing this is to bind all the serviceAccounts in +kube-system (which includes replicaSet controller serviceAccounts) to the +appropriate PSP for the specific namespace. + +For example, the following RoleBinding assigns the restricted PSP to all +kube-system serviceAccounts operating in the ``billing-dept-ns`` namespace. + +:: + + apiVersion: rbac.authorization.k8s.io/v1 + kind: RoleBinding + metadata: + name: kube-system-restricted-psp-users + namespace: billing-dept-ns + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: restricted-psp-user + subjects: + - kind: Group + name: system:serviceaccounts:kube-system + apiGroup: rbac.authorization.k8s.io diff --git a/doc/source/configuration/k8s_res_policies.rst b/doc/source/configuration/k8s_res_policies.rst new file mode 100644 index 000000000..ead399bb5 --- /dev/null +++ b/doc/source/configuration/k8s_res_policies.rst @@ -0,0 +1,123 @@ +================= +Resource Policies +================= + +This guide describes two Kubernetes resource policies, LimitRange and +ResourceQuota, which are enabled by default on StarlingX. + +.. contents:: + :local: + :depth: 1 + +---------- +LimitRange +---------- + +By default, containers run with unbounded resources on a Kubernetes cluster. +This is not ideal in production environments, as a single pod could monopolize +all available resources on a worker node. LimitRange is a policy to constrain +resource allocations (for pods or containers) in a particular namespace. + +Specifically a LimitRange policy provides constraints that can: + +* Enforce minimum and maximum compute resources usage per pod or container in + a namespace. +* Enforce minimum and maximum storage request per PersistentVolumeClaim in a + namespace. +* Enforce a ratio between request and limit for a resource in a namespace. +* Set the default request/limit for compute resources in a namespace and + automatically inject them to containers at runtime. + +See https://kubernetes.io/docs/concepts/policy/limit-range/ for more details. + +An example of LimitRange policies for the ``billing-dept-ns`` namespace in the +:doc:`k8s_pod_sec_policies` example is shown below: + +:: + + apiVersion: v1 + kind: LimitRange + metadata: + name: mem-cpu-per-container-limit + namespace: billing-dept-ns + spec: + limits: + - max: + cpu: "800m" + memory: "1Gi" + min: + cpu: "100m" + memory: "99Mi" + default: + cpu: "700m" + memory: "700Mi" + defaultRequest: + cpu: "110m" + memory: "111Mi" + type: Container + --- + apiVersion: v1 + kind: LimitRange + metadata: + name: mem-cpu-per-pod-limit + namespace: billing-dept-ns + spec: + limits: + - max: + cpu: "2" + memory: "2Gi" + type: Pod + --- + apiVersion: v1 + kind: LimitRange + metadata: + name: pvc-limit + namespace: billing-dept-ns + spec: + limits: + - type: PersistentVolumeClaim + max: + storage: 3Gi + min: + storage: 1Gi + --- + apiVersion: v1 + kind: LimitRange + metadata: + name: memory-ratio-pod-limit + namespace: billing-dept-ns + spec: + limits: + - maxLimitRequestRatio: + memory: 10 + type: Pod + +------------- +ResourceQuota +------------- + +A ResourceQuota policy object provides constraints that limit aggregate resource +consumption per namespace. It can limit the quantity of objects that can be +created in a namespace by type, as well as the total amount of compute resources +that may be consumed by resources in that project. ResourceQuota limits can be +created for CPU, memory, storage, and resource counts for all standard +namespaced resource types such as secrets, configmaps, and others. + +See https://kubernetes.io/docs/concepts/policy/resource-quotas/ for more details. + +An example of ResourceQuota policies for the ``billing-dept-ns`` namespace of +the :doc:`k8s_pod_sec_policies` example is shown below: + +:: + + apiVersion: v1 + kind: ResourceQuota + metadata: + name: resource-quotas + namespace: billing-dept-ns + spec: + hard: + persistentvolumeclaims: "1" + services.loadbalancers: "2" + services.nodeports: "0" +