Spec: Support OCI image registry with authentication turned on
This specification proposes support for deploying openstack services using OSH with OCI image registry which has authentication turned on. Change-Id: I26e34a5a39c06e9d481af58c15fb930d3fe9b1ef Implements: blueprint support-oci-image-registry-with-authentication-turned-on Signed-off-by: Angie Wang <angie.wang@windriver.com>
This commit is contained in:
Angie Wang
Pete Birley
parent
f191c62d13
commit
75f3083d24
@ -0,0 +1,204 @@
|
|||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
..
|
||||||
|
|
||||||
|
========================================================
|
||||||
|
Support OCI image registry with authentication turned on
|
||||||
|
========================================================
|
||||||
|
|
||||||
|
Blueprint:
|
||||||
|
support-oci-image-registry-with-authentication-turned-on_
|
||||||
|
|
||||||
|
.. _support-oci-image-registry-with-authentication-turned-on: https://blueprints.launchpad.net/openstack-helm/+spec/support-oci-image-registry-with-authentication-turned-on
|
||||||
|
|
||||||
|
Problem Description
|
||||||
|
===================
|
||||||
|
In the current openstack-helm, all charts provide an ``images:`` section in
|
||||||
|
their ``values.yaml`` that have the container images references. By default,
|
||||||
|
the container images are all downloaded from a registry hosted by Docker or Quay.
|
||||||
|
However, the image references can be overridden by operators to download images
|
||||||
|
from any OCI image registry. In the case that the OCI image registry has
|
||||||
|
authentication turned on, kubelet would fail to download the images because the
|
||||||
|
current Openstack-Helm does not provide a way to pass the OCI image registry
|
||||||
|
credentials to kubernetes when pulling images.
|
||||||
|
|
||||||
|
|
||||||
|
Use case
|
||||||
|
========
|
||||||
|
Operators should be able to use Openstack-Helm to deploy containerized openstack
|
||||||
|
services with a docker registry has authentication turned on.
|
||||||
|
|
||||||
|
|
||||||
|
Proposed Change
|
||||||
|
===============
|
||||||
|
To be able to pull images from an OCI image registry which has the authentication
|
||||||
|
turned on, kubernetes needs credentials. For each chart, a new ``endpoints:``
|
||||||
|
entry could be added in ``values.yaml`` to provide image credentials, a secret
|
||||||
|
needs to be generated to hold the credentials and the ``imagePullSecrets:`` field
|
||||||
|
should be added in each service account to specify which secret should be used
|
||||||
|
to get the credentials from when pulling images by kubelet.
|
||||||
|
|
||||||
|
The detailed proposes change are described as following:
|
||||||
|
|
||||||
|
1. For each chart, add a new entry ``oci_image_registry:`` under ``endpoints:`` in
|
||||||
|
``values.yaml``. The entry ``oci_image_registry:`` has the ``auth:`` section which
|
||||||
|
provides the credentials for accessing registry images and an option ``enabled:``
|
||||||
|
to determine whether images authentication is required or not. The registry basic
|
||||||
|
information would also be included for generating the registry URL by the endpoint
|
||||||
|
lookup functions. Also add a new entry ``oci_image_registry:`` under ``secrets:``
|
||||||
|
to indicate the secret name. In order to create the secret that holds the provided
|
||||||
|
credentials, add a new component ``secret_registry`` in ``manifests:`` section.
|
||||||
|
For example:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
oci_image_registry:
|
||||||
|
nova: nova-oci-image-registry-key
|
||||||
|
|
||||||
|
endpoints:
|
||||||
|
...
|
||||||
|
oci_image_registry:
|
||||||
|
name: oci-image-registry
|
||||||
|
namespace: oci-image-registry
|
||||||
|
auth:
|
||||||
|
enabled: false
|
||||||
|
nova:
|
||||||
|
username: nova
|
||||||
|
password: password
|
||||||
|
hosts:
|
||||||
|
default: localhost
|
||||||
|
host_fqdn_override:
|
||||||
|
default: null
|
||||||
|
port:
|
||||||
|
registry:
|
||||||
|
default: 5000
|
||||||
|
|
||||||
|
manifests:
|
||||||
|
secret_registry: true
|
||||||
|
|
||||||
|
The option ``enabled:`` under ``auth:`` and the manifest ``secret_registry:``
|
||||||
|
provide the ability for operator to determine whether they would like to have
|
||||||
|
secrets generated and passed to kubernetes for pulling images.
|
||||||
|
|
||||||
|
The secret would not be created with the default option ``enabled: false`` and
|
||||||
|
``secret_registry: true``. To enable secret creation, operator should override
|
||||||
|
``enabled:`` to true. The above example shows the default credentials, operator
|
||||||
|
should override the ``username:`` and ``password:`` under ``auth:`` section to
|
||||||
|
provide their own credentials.
|
||||||
|
|
||||||
|
Then, add manifest ``secret-registry.yaml`` in ``templates/`` to leverage
|
||||||
|
the function that will be added in helm-toolkit to create the secret. For example:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
{{- if and .Values.manifests.secret_registry .Values.endpoints.oci_image_registry.auth.enabled }}
|
||||||
|
{{ include "helm-toolkit.manifests.secret_registry" ( dict "envAll" . "registryUser" .Chart.Name ) }}
|
||||||
|
{{- end }}
|
||||||
|
|
||||||
|
2. Add a helm-toolkit function ``helm-toolkit.manifests.secret_registry`` to create a
|
||||||
|
manifest for secret generation. For example:
|
||||||
|
|
||||||
|
.. code-block:: rst
|
||||||
|
|
||||||
|
{{- define "helm-toolkit.manifests.secret_registry" -}}
|
||||||
|
{{- $envAll := index . "envAll" }}
|
||||||
|
{{- $registryUser := index . "registryUser" }}
|
||||||
|
{{- $secretName := index $envAll.Values.secrets.oci_image_registry $registryUser }}
|
||||||
|
{{- $registryHost := tuple "oci_image_registry" "internal" $envAll | include "helm-toolkit.endpoints.endpoint_host_lookup" }}
|
||||||
|
{{- $registryPort := tuple "oci_image_registry" "internal" "registry" $envAll | include "helm-toolkit.endpoints.endpoint_port_lookup" }}
|
||||||
|
{{- $imageCredentials := index $envAll.Values.endpoints.oci_image_registry.auth $registryUser }}
|
||||||
|
{{- $dockerAuthToken := printf "%s:%s" $imageCredentials.username $imageCredentials.password | b64enc }}
|
||||||
|
{{- $dockerAuth := printf "{\"auths\": {\"%s:%s\": {\"auth\": \"%s\"}}}" $registryHost $registryPort $dockerAuthToken | b64enc }}
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: {{ $secretName }}
|
||||||
|
type: kubernetes.io/dockerconfigjson
|
||||||
|
data:
|
||||||
|
.dockerconfigjson: {{ $dockerAuth }}
|
||||||
|
{{- end }}
|
||||||
|
|
||||||
|
3. Reference the created secret by adding the ``imagePullSecrets:`` field to ServiceAccount
|
||||||
|
resource template [2]_ in ``helm-toolkit/snippets/_kubernetes_pod_rbac_serviceaccount.tpl``.
|
||||||
|
To handle it as optional, the field is wrapped in a conditional. For example,
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: ServiceAccount
|
||||||
|
...
|
||||||
|
{{- if $envAll.Values.endpoints.oci_image_registry.auth.enabled }}
|
||||||
|
imagePullSecrets:
|
||||||
|
- name: {{ index $envAll.Values.secrets.oci_image_registry $envAll.Chart.Name }}
|
||||||
|
{{- end }}
|
||||||
|
|
||||||
|
If .Values.endpoints.oci_image_registry.auth.enabled will be set to true, then any
|
||||||
|
containers created with the current service account will have the ``imagePullSecrets``
|
||||||
|
automatically added to their spec and the secret will be passed to kubelet to be
|
||||||
|
used for pulling images.
|
||||||
|
|
||||||
|
|
||||||
|
Security Impact
|
||||||
|
---------------
|
||||||
|
The credentials for the registry could be exposed by running the kubectl command:
|
||||||
|
kubectl get secret <secret-name> --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode
|
||||||
|
|
||||||
|
Authentication should be enabled for normal users to access Kube API server via
|
||||||
|
either kubectl command or kube REST API call.
|
||||||
|
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
No performance impact
|
||||||
|
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
Before using Openstack-Helm to deploy openstack services,
|
||||||
|
|
||||||
|
1. Put .docker/config.json in docker/kubelet root directory on all nodes
|
||||||
|
2. Pre-pulling images on all nodes
|
||||||
|
|
||||||
|
But above alternatives have limitations and security impact. i.e...require root access
|
||||||
|
to configure on all nodes, all pods can read any configured private registries, all pods
|
||||||
|
can use any images cached on a node [1]_
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignees:
|
||||||
|
|
||||||
|
* Angie Wang (angiewang)
|
||||||
|
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
#. Provide the credentials and add the manifest across all charts in OSH and OSH-infra
|
||||||
|
#. Update helm-toolkit to provide manifest to create secret for registry authentication
|
||||||
|
#. Update helm-toolkit serviceaccount template to pass the secret in a conditional
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
None
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
Documentation of how to enable the registry secret generation
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
.. [1] https://kubernetes.io/docs/concepts/containers/images
|
||||||
|
.. [2] https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account
|
||||||
Loading…
Reference in New Issue
Block a user