Merge "Create Docker Registry with Pull Through Cache"

doc/source/developer_resources/local_registry.rst

@@ -13,8 +13,23 @@ This will also speed up the bootstrap process as images will be downloaded only
 once. This guide assumes that you are installing this local external docker
 registry on a Linux system with Docker installed and configured.
 
+Optionally, you can also create multiple registries with pull-through cache,
+skipping the need to pre-populate the registry and keeping the images and tags
+up-to-date.
+
 .. rubric:: |proc|
 
+.. tabs::
+  .. group-tab:: Private Registry
+
+    .. note::
+      In this section, you will pre-populate a local registry with multiple
+      images. Currently, the generated images list will not contain all
+      images, making it necessary to manually pull some images used by the
+      kubeadm tool. Alternatively, you can follow the "pull-through cache"
+      section, allowing docker to get the image from remote, if it doesn't
+      exists locally.
+
     #.  Create folders to store your local registry images (``storage``) and to
         place setup files that will be used later on (``images``):
 
@@ -64,16 +79,16 @@ registry on a Linux system with Docker installed and configured.
                 registry:2
 
         .. note::
-        *Optional*: the ``-p`` parameter configures a mapping between the host
+           *Optional*: the ``-p`` parameter configures a mapping between the
-        port and the container port. If you want to listen on another port on
+           host port and the container port. If you want to listen on another
-        your host, say 9000, change from ``-p 5000:5000 \`` to
+           port on your host, say 9000, change from ``-p 5000:5000 \`` to
            ``-p 9000:5000 \``.
 
     #.  Create the list of images that will populate the registry.
 
-    Obtain the Kubernetes version your |prod| uses. This can be found in the
+        Obtain the Kubernetes version your |prod| uses. This can be found in
-    ``fresh_install_k8s_version`` value of the `Kubernetes versions`_ file.
+        the ``fresh_install_k8s_version`` value of the `Kubernetes versions`_
-    Use the branches and tags to find the value for your version.
+        file. Use the branches and tags to find the value for your version.
 
         With the Kubernetes version, you can find the corresponding folder in
         `system images`_ and locate the ``system-images.yml`` file. This file
@@ -92,9 +107,26 @@ registry on a Linux system with Docker installed and configured.
 
             curl -s ${IMAGES_YAML_RAW_FILE} | grep -v '\-\-\-' | grep -v '^#' | cut -d ':' -f2,3 | tr -d ' ' > $HOME/docker-registry/images/list.lst
 
-    .. note:: *Optional*: If you have a running |prod| setup, you can run
+        The expected image list will be presented in this format:
-        the following to create an Ansible Playbook to get the exact images
+
-        you will need instead:
+        .. code-block:: text
+
+            <REGISTRY_URL/IMAGE_NAME>:<IMAGE_TAG>
+            <REGISTRY_URL/IMAGE_NAME>:<IMAGE_TAG>
+            <REGISTRY_URL/IMAGE_NAME>:<IMAGE_TAG>...
+
+        .. important::
+           Due to ``kubeadm`` dynamically pulling the necessary images for
+           creating the k8s cluster, based on the ``Kubernetes version`` used,
+           this list doesn't contain all necessary images.
+           You'll need to manually add the extra images to the ``list.lst``
+           file, using `kubeadm`_. You can check the official k8s docs on how
+           to get them.
+
+        .. note::
+           *Optional*: If you have a running |prod| setup, you can run the
+           following to create an Ansible Playbook to get the exact images you
+           will need instead:
 
             .. code-block:: shell
 
@@ -122,8 +154,8 @@ registry on a Linux system with Docker installed and configured.
            You will find the Kubernetes version to assign to the `K8S_VERSION`
            variable on the aforementioned `Kubernetes versions`_ file.
 
-#.  Create and run a script that will populate the registry based on the list
+    #.  Create and run a script that will populate the registry based on the
-    of images:
+        list of images:
 
         .. code-block:: shell
 
@@ -169,9 +201,117 @@ registry on a Linux system with Docker installed and configured.
             $REG_SCRIPT $HOME/docker-registry/images/list.lst
 
         .. note::
-        The ``populate_registry.sh`` script checks if each image in the list is
+            The ``populate_registry.sh`` script checks if each image in the
-        already present, which means you can update the list and re-run the script
+            list is already present, which means you can update the list and
-        to get new images whenever necessary.
+            re-run the script to get new images whenever necessary.
+
+  .. group-tab:: With Pull-through cache
+
+      #.  Create folders to store your local registry images (``storage``) and to place setup files that will be used later on (``images``):
+
+          .. code-block:: shell
+
+              mkdir -p $HOME/docker-registry/storage
+              mkdir -p $HOME/docker-registry/images
+              mkdir -p $HOME/docker-registry/config
+
+      #.  Create a configuration file that will be used by Docker's official
+          registry image later on. The script will replace the PORT and
+          REGISTRY values:
+
+          .. code-block:: shell
+
+              cat > $HOME/docker-registry/config.yml << EOF
+              version: 0.1
+              log:
+                fields:
+                  service: registry
+              storage:
+                cache:
+                  blobdescriptor: inmemory
+                filesystem:
+                  rootdirectory: /var/lib/registry
+              http:
+                addr: <PORT>
+                headers:
+                  X-Content-Type-Options: [nosniff]
+              health:
+                storagedriver:
+                  enabled: true
+                  interval: 10s
+                  threshold: 3
+              proxy:
+                remoteurl: https://<REGISTRY>
+              EOF
+
+      #.  Create and run a script that will create several private docker
+          registries:
+
+          .. code-block:: shell
+
+              export REG_CACHE_SCRIPT=$HOME/docker-registry/images/create_registries.sh
+              cat > $REG_CACHE_SCRIPT <<'EOF'
+              #!/bin/bash
+
+              if [ -z "$1" ]; then
+                  echo "No argument supplied"
+                  exit 1
+              fi
+
+              export LOCAL_REG=$HOME/docker-registry
+              INSECURE_REGISTRY_LIST=()
+              LOCALHOST_IP=$1
+              [ -z "$2" ] && BASE_PORT=5000 || BASE_PORT=$2
+              REGISTRY_LIST=("quay.io" "gcr.io" "k8s.gcr.io" "registry-1.docker.io"
+                  "docker.elastic.co" "ghcr.io" "registry.k8s.io" "icr.io")
+              CONFIG_BASE=$(cat "$LOCAL_REG"/config.yml)
+
+              for cur_registry in "${REGISTRY_LIST[@]}"; do
+                  CONTAINER_NAME="registry_$cur_registry"
+                  NEW_CONFIG_FILE_NAME="config_$cur_registry.yml"
+                  NEW_CONFIG_FILE="$LOCAL_REG/configs/$NEW_CONFIG_FILE_NAME"
+                  NEW_CONFIG=$(echo "$CONFIG_BASE" | sed "s/<PORT>/$BASE_PORT/" | sed "s/<REGISTRY>/$cur_registry/")
+                  echo "$NEW_CONFIG" >"$NEW_CONFIG_FILE"
+
+                  docker run -d \
+                      --restart=always \
+                      --name "$CONTAINER_NAME" \
+                      -v "$LOCAL_REG"/storage:/var/lib/registry \
+                      -v "$NEW_CONFIG_FILE":/etc/docker/registry/config.yml \
+                      -e REGISTRY_HTTP_ADDR=0.0.0.0:"$BASE_PORT" \
+                      -p "$BASE_PORT":"$BASE_PORT" \
+                      registry:2.6.2
+
+                  INSECURE_REGISTRY_LIST+=("$cur_registry - $LOCALHOST_IP:$BASE_PORT")
+                  ((BASE_PORT++))
+
+              done
+
+              for reg in "${INSECURE_REGISTRY_LIST[@]}"; do
+                  echo "$reg"
+              done
+              EOF
+
+              chmod +x $REG_CACHE_SCRIPT
+              $REG_CACHE_SCRIPT <your ip address> <starting port (default 5000)>
+
+          .. important::
+              The ``create_registries.sh`` script create 8 registry
+              containers. You'll need 8 free ports, in sequence, starting from
+              the specified starting port.
+              Ex.: Passing port ``6000`` as the second argument, will create containers on ``6000-6007`` port range.
+
+          .. note::
+              On it's current version, Docker CLI throws a warning when
+              pulling images that're still using the v1 manifest format.
+              The official Docker image registry, on it's latest version, does
+              not allow images that still uses them.
+              A few images used by the StarlingX OS are outdated, so, as of
+              now, it'll be necessary to use a older version of the registry
+              image (v2.6.2).
+              Check the official `registry as a pull through cache`_ docker
+              guide for more info.
+
 
 .. note::
     The Docker CLI exclusively permits insecure (HTTP) registries when on
@@ -188,6 +328,9 @@ Your registry is ready! On your next |prod| installation, update your
 ``/home/sysadmin/localhost.yml`` bootstrap overrides file with the
 following lines to use it:
 
+.. tabs::
+  .. group-tab:: Private Registry
+
     .. code-block:: yaml
 
         docker_registries:
@@ -211,6 +354,31 @@ following lines to use it:
             type: docker
             secure: false
 
+  .. group-tab:: With Pull-through cache
+
+    .. code-block:: yaml
+
+        docker_registries:
+          quay.io:
+            url: <your IP address>:<port_1>
+          gcr.io:
+            url: <your IP address>:<port_2>
+          k8s.gcr.io:
+            url: <your IP address>:<port_3>
+          docker.io:
+            url: <your IP address>:<port_4>
+          docker.elastic.co:
+            url: <your IP address>:<port_5>
+          ghcr.io:
+            url: <your IP address>:<port_6>
+          registry.k8s.io:
+            url: <your IP address>:<port_7>
+          icr.io:
+            url: <your IP address>:<port_8>
+          defaults:
+            type: docker
+            secure: false
+
 .. note::
     This procedure configured |prod| to use an insecure registry via the
     ``docker_registries.defaults.secure`` parameter set to ``false`` in the
@@ -219,3 +387,5 @@ following lines to use it:
 
 .. _Kubernetes versions: https://opendev.org/starlingx/ansible-playbooks/src/branch/master/playbookconfig/src/playbooks/roles/bootstrap/validate-config/vars/main.yml
 .. _system images: https://opendev.org/starlingx/ansible-playbooks/src/branch/master/playbookconfig/src/playbooks/roles/common/load-images-information/vars
+.. _registry as a pull through cache: https://docs.docker.com/docker-hub/mirror/
+.. _kubeadm: https://kubernetes.io/pt-br/docs/reference/setup-tools/kubeadm/kubeadm-config/#cmd-config-images-list