Spec: Support OCI image registry with authentication turned on
This specification proposes support for deploying openstack services using OSH with OCI image registry which has authentication turned on. Change-Id: I26e34a5a39c06e9d481af58c15fb930d3fe9b1ef Implements: blueprint support-oci-image-registry-with-authentication-turned-on Signed-off-by: Angie Wang <angie.wang@windriver.com>
This commit is contained in:
Angie Wang
Pete Birley
parent
f191c62d13
commit
75f3083d24
@ -0,0 +1,204 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
..
|
||||
|
||||
========================================================
|
||||
Support OCI image registry with authentication turned on
|
||||
========================================================
|
||||
|
||||
Blueprint:
|
||||
support-oci-image-registry-with-authentication-turned-on_
|
||||
|
||||
.. _support-oci-image-registry-with-authentication-turned-on: https://blueprints.launchpad.net/openstack-helm/+spec/support-oci-image-registry-with-authentication-turned-on
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
In the current openstack-helm, all charts provide an ``images:`` section in
|
||||
their ``values.yaml`` that have the container images references. By default,
|
||||
the container images are all downloaded from a registry hosted by Docker or Quay.
|
||||
However, the image references can be overridden by operators to download images
|
||||
from any OCI image registry. In the case that the OCI image registry has
|
||||
authentication turned on, kubelet would fail to download the images because the
|
||||
current Openstack-Helm does not provide a way to pass the OCI image registry
|
||||
credentials to kubernetes when pulling images.
|
||||
|
||||
|
||||
Use case
|
||||
========
|
||||
Operators should be able to use Openstack-Helm to deploy containerized openstack
|
||||
services with a docker registry has authentication turned on.
|
||||
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
To be able to pull images from an OCI image registry which has the authentication
|
||||
turned on, kubernetes needs credentials. For each chart, a new ``endpoints:``
|
||||
entry could be added in ``values.yaml`` to provide image credentials, a secret
|
||||
needs to be generated to hold the credentials and the ``imagePullSecrets:`` field
|
||||
should be added in each service account to specify which secret should be used
|
||||
to get the credentials from when pulling images by kubelet.
|
||||
|
||||
The detailed proposes change are described as following:
|
||||
|
||||
1. For each chart, add a new entry ``oci_image_registry:`` under ``endpoints:`` in
|
||||
``values.yaml``. The entry ``oci_image_registry:`` has the ``auth:`` section which
|
||||
provides the credentials for accessing registry images and an option ``enabled:``
|
||||
to determine whether images authentication is required or not. The registry basic
|
||||
information would also be included for generating the registry URL by the endpoint
|
||||
lookup functions. Also add a new entry ``oci_image_registry:`` under ``secrets:``
|
||||
to indicate the secret name. In order to create the secret that holds the provided
|
||||
credentials, add a new component ``secret_registry`` in ``manifests:`` section.
|
||||
For example:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
secrets:
|
||||
oci_image_registry:
|
||||
nova: nova-oci-image-registry-key
|
||||
|
||||
endpoints:
|
||||
...
|
||||
oci_image_registry:
|
||||
name: oci-image-registry
|
||||
namespace: oci-image-registry
|
||||
auth:
|
||||
enabled: false
|
||||
nova:
|
||||
username: nova
|
||||
password: password
|
||||
hosts:
|
||||
default: localhost
|
||||
host_fqdn_override:
|
||||
default: null
|
||||
port:
|
||||
registry:
|
||||
default: 5000
|
||||
|
||||
manifests:
|
||||
secret_registry: true
|
||||
|
||||
The option ``enabled:`` under ``auth:`` and the manifest ``secret_registry:``
|
||||
provide the ability for operator to determine whether they would like to have
|
||||
secrets generated and passed to kubernetes for pulling images.
|
||||
|
||||
The secret would not be created with the default option ``enabled: false`` and
|
||||
``secret_registry: true``. To enable secret creation, operator should override
|
||||
``enabled:`` to true. The above example shows the default credentials, operator
|
||||
should override the ``username:`` and ``password:`` under ``auth:`` section to
|
||||
provide their own credentials.
|
||||
|
||||
Then, add manifest ``secret-registry.yaml`` in ``templates/`` to leverage
|
||||
the function that will be added in helm-toolkit to create the secret. For example:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
{{- if and .Values.manifests.secret_registry .Values.endpoints.oci_image_registry.auth.enabled }}
|
||||
{{ include "helm-toolkit.manifests.secret_registry" ( dict "envAll" . "registryUser" .Chart.Name ) }}
|
||||
{{- end }}
|
||||
|
||||
2. Add a helm-toolkit function ``helm-toolkit.manifests.secret_registry`` to create a
|
||||
manifest for secret generation. For example:
|
||||
|
||||
.. code-block:: rst
|
||||
|
||||
{{- define "helm-toolkit.manifests.secret_registry" -}}
|
||||
{{- $envAll := index . "envAll" }}
|
||||
{{- $registryUser := index . "registryUser" }}
|
||||
{{- $secretName := index $envAll.Values.secrets.oci_image_registry $registryUser }}
|
||||
{{- $registryHost := tuple "oci_image_registry" "internal" $envAll | include "helm-toolkit.endpoints.endpoint_host_lookup" }}
|
||||
{{- $registryPort := tuple "oci_image_registry" "internal" "registry" $envAll | include "helm-toolkit.endpoints.endpoint_port_lookup" }}
|
||||
{{- $imageCredentials := index $envAll.Values.endpoints.oci_image_registry.auth $registryUser }}
|
||||
{{- $dockerAuthToken := printf "%s:%s" $imageCredentials.username $imageCredentials.password | b64enc }}
|
||||
{{- $dockerAuth := printf "{\"auths\": {\"%s:%s\": {\"auth\": \"%s\"}}}" $registryHost $registryPort $dockerAuthToken | b64enc }}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: {{ $secretName }}
|
||||
type: kubernetes.io/dockerconfigjson
|
||||
data:
|
||||
.dockerconfigjson: {{ $dockerAuth }}
|
||||
{{- end }}
|
||||
|
||||
3. Reference the created secret by adding the ``imagePullSecrets:`` field to ServiceAccount
|
||||
resource template [2]_ in ``helm-toolkit/snippets/_kubernetes_pod_rbac_serviceaccount.tpl``.
|
||||
To handle it as optional, the field is wrapped in a conditional. For example,
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
...
|
||||
{{- if $envAll.Values.endpoints.oci_image_registry.auth.enabled }}
|
||||
imagePullSecrets:
|
||||
- name: {{ index $envAll.Values.secrets.oci_image_registry $envAll.Chart.Name }}
|
||||
{{- end }}
|
||||
|
||||
If .Values.endpoints.oci_image_registry.auth.enabled will be set to true, then any
|
||||
containers created with the current service account will have the ``imagePullSecrets``
|
||||
automatically added to their spec and the secret will be passed to kubelet to be
|
||||
used for pulling images.
|
||||
|
||||
|
||||
Security Impact
|
||||
---------------
|
||||
The credentials for the registry could be exposed by running the kubectl command:
|
||||
kubectl get secret <secret-name> --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode
|
||||
|
||||
Authentication should be enabled for normal users to access Kube API server via
|
||||
either kubectl command or kube REST API call.
|
||||
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
No performance impact
|
||||
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
Before using Openstack-Helm to deploy openstack services,
|
||||
|
||||
1. Put .docker/config.json in docker/kubelet root directory on all nodes
|
||||
2. Pre-pulling images on all nodes
|
||||
|
||||
But above alternatives have limitations and security impact. i.e...require root access
|
||||
to configure on all nodes, all pods can read any configured private registries, all pods
|
||||
can use any images cached on a node [1]_
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignees:
|
||||
|
||||
* Angie Wang (angiewang)
|
||||
|
||||
|
||||
Work Items
|
||||
----------
|
||||
#. Provide the credentials and add the manifest across all charts in OSH and OSH-infra
|
||||
#. Update helm-toolkit to provide manifest to create secret for registry authentication
|
||||
#. Update helm-toolkit serviceaccount template to pass the secret in a conditional
|
||||
|
||||
|
||||
Testing
|
||||
=======
|
||||
None
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
Documentation of how to enable the registry secret generation
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
.. [1] https://kubernetes.io/docs/concepts/containers/images
|
||||
.. [2] https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account
|
||||
Loading…
Reference in New Issue
Block a user