Add a local registry guide for developers

The guide is aimed at developers working in virtual environments that
need to build or run several controller-0 setups (specifically, to
developers in need of running the Ansible bootstrap playbook multiple
times).

By configuring a local registry and downloading the needed images
beforehand, all subsequent installations can point to this local
registry to download images from it to the controller-0. Not going to
the internet everytime, devs are not subject to public registries' rate
limits.

Story: 2005051
Task: 47895
Partial-Bug: 1834504
Partial-Bug: 1888610

Change-Id: Ie3e3cbc09030a6491f7b573a4ea5fd57e3158d54
Signed-off-by: Bruno Muniz <bruno.muniz@encora.com>

doc/source/developer_resources/index.rst

@@ -27,5 +27,5 @@ Developer Resources
    starlingx_patching
    stx_ipv6_deployment
    stx_opcua
-
+   local_registry
 

doc/source/developer_resources/local_registry.rst

@@ -0,0 +1,221 @@
+
+.. _virtual-create-and-bootstrap-from-a-private-docker-registry:
+
+==================================================================
+Create and Bootstrap from a Local External Private Docker Registry
+==================================================================
+
+You can bootstrap controller-0 from a private local external Docker registry.
+This is useful in case you plan to perform multiple installations on your own
+machine and do not want to be rate-limited by public registries.
+
+This will also speed up the bootstrap process as images will be downloaded only
+once. This guide assumes that you are installing this local external docker
+registry on a Linux system with Docker installed and configured.
+
+.. rubric:: |proc|
+
+#.  Create folders to store your local registry images (``storage``) and to
+    place setup files that will be used later on (``images``):
+
+    .. code-block:: shell
+
+        mkdir -p $HOME/docker-registry/storage
+        mkdir -p $HOME/docker-registry/images
+
+#.  Create a configuration file that will be used by Docker's official
+    Registry image later on:
+
+    .. code-block:: shell
+
+        cat > $HOME/docker-registry/config.yml << EOF
+        version: 0.1
+        log:
+          fields:
+            service: registry
+        storage:
+          cache:
+            blobdescriptor: inmemory
+          filesystem:
+            rootdirectory: /var/lib/registry
+        http:
+          addr: :5000
+          headers:
+            X-Content-Type-Options: [nosniff]
+        health:
+          storagedriver:
+            enabled: true
+            interval: 10s
+            threshold: 3
+        EOF
+
+#.  Run the docker container registry:
+
+    .. code-block:: shell
+
+        export LOCAL_REG=$HOME/docker-registry
+        docker run -d \
+            --restart=always \
+            --name registry \
+            -v "$LOCAL_REG"/storage:/var/lib/registry \
+            -v "$LOCAL_REG"/config.yml:/etc/docker/registry/config.yml \
+            -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
+            -p 5000:5000 \
+            registry:2
+
+    .. note::
+        *Optional*: the ``-p`` parameter configures a mapping between the host
+        port and the container port. If you want to listen on another port on
+        your host, say 9000, change from ``-p 5000:5000 \`` to
+        ``-p 9000:5000 \``.
+
+#.  Create the list of images that will populate the registry.
+
+    Obtain the Kubernetes version your |prod| uses. This can be found in the
+    ``fresh_install_k8s_version`` value of the `Kubernetes versions`_ file.
+    Use the branches and tags to find the value for your version.
+
+    With the Kubernetes version, you can find the corresponding folder in
+    `system images`_ and locate the ``system-images.yml`` file. This file
+    contains the list of images to be loaded into your registry.
+
+    To make the list of images for |prod| 8.0, take the raw address of the
+    corresponding ``system-images.yml`` file and set a variable with it:
+
+    .. code-block:: shell
+
+        export IMAGES_YAML_RAW_FILE="https://opendev.org/starlingx/ansible-playbooks/raw/branch/master/playbookconfig/src/playbooks/roles/common/load-images-information/vars/k8s-v1.24.4/system-images.yml"
+
+    Use the command to generate a ``list.lst`` file:
+
+    .. code-block:: shell
+
+        curl -s ${IMAGES_YAML_RAW_FILE} | grep -v '\-\-\-' | grep -v '^#' | cut -d ':' -f2 | tr -d ' ' > $HOME/docker-registry/images/list.lst
+
+    .. note:: *Optional*: If you have a running |prod| setup, you can run
+        the following to create an Ansible Playbook to get the exact images
+        you will need instead:
+
+        .. code-block:: shell
+
+            cat > list-images.yml << EOF
+            ---
+            - hosts: localhost
+              gather_facts: true
+              tasks:
+                - name: Load image info
+                  include_role:
+                    name: /usr/share/ansible/stx-ansible/playbooks/roles/common/load-images-information
+
+                - name: Print image list
+                  debug:
+                    msg: "{{ (kubernetes_images + networking_images + static_images + storage_images + security_images) }}"
+            EOF
+
+        Then, run the following to execute the Ansible Playbook:
+
+        .. code-block:: shell
+
+            K8S_VERSION=<version>
+            ansible-playbook list-images.yml -e "kubernetes_version=${K8S_VERSION}"
+
+        You will find the Kubernetes version to assign to the `K8S_VERSION`
+        variable on the aforementioned `Kubernetes versions`_ file.
+
+#.  Create and run a script that will populate the registry based on the list
+    of images:
+
+    .. code-block:: shell
+
+        export REG_SCRIPT=$HOME/docker-registry/images/populate_registry.sh
+        cat > $REG_SCRIPT <<'EOF'
+        #!/bin/bash
+
+        if [[ -z $1 ]]; then
+                echo "Please provide a file with a list of Docker images."
+            exit 1
+        fi
+
+        TAGS_FILE=$1
+        LOCAL_REGISTRY=localhost:5000
+
+        while read DOCKER_IMAGE;
+        do
+            echo ""
+            echo -n "--- ${DOCKER_IMAGE}: ";
+
+            IMAGE_ARRAY=($(echo $DOCKER_IMAGE | tr ":" " "))
+            REPO=${IMAGE_ARRAY[0]}
+            TAG=${IMAGE_ARRAY[1]}
+            REPO_TAGS_URL="http://${LOCAL_REGISTRY}/v2/${REPO}/tags/list"
+            if curl -s -X GET --insecure ${REPO_TAGS_URL} | jq | grep ${TAG} &>/dev/null; then
+                echo -n "Skipping..."
+                continue
+            fi
+
+            echo "Pulling..."
+
+            set -x
+            docker pull ${DOCKER_IMAGE};
+            REGISTRY_IMAGE=${LOCAL_REGISTRY}/${DOCKER_IMAGE}
+            docker tag ${DOCKER_IMAGE} ${REGISTRY_IMAGE};
+            docker push ${REGISTRY_IMAGE};
+            docker rmi ${DOCKER_IMAGE} ${REGISTRY_IMAGE};
+            set +x
+
+        done < $TAGS_FILE
+        EOF
+        chmod +x $REG_SCRIPT
+        $REG_SCRIPT $HOME/docker-registry/images/list.lst
+
+    .. note::
+        The ``populate_registry.sh`` script checks if each image in the list is
+        already present, which means you can update the list and re-run the script
+        to get new images whenever necessary.
+
+    .. note::
+        The Docker CLI exclusively permits insecure (HTTP) registries when on
+        the local host. When executing the provided script remotely, in
+        addition to modifying the ``LOCAL_REGISTRY`` variable to match the IP
+        address of the registry's location, it is necessary to insert an entry
+        in the ``insecure-registries:`` section within the
+        ``etc/docker/daemon.json`` file. Following this adjustment, you must
+        restart the Docker service.
+
+.. rubric:: |result|
+
+Your registry is ready! On your next |prod| installation, update your
+``/home/sysadmin/localhost.yml`` bootstrap overrides file with the
+following lines to use it:
+
+.. code-block:: yaml
+
+    docker_registries:
+      quay.io:
+        url: <your IP address>:5000/quay.io
+      gcr.io:
+        url: <your IP address>:5000/gcr.io
+      k8s.gcr.io:
+        url: <your IP address>:5000/k8s.gcr.io
+      docker.io:
+        url: <your IP address>:5000/docker.io
+      docker.elastic.co:
+        url: <your IP address>:5000/docker.elastic.co
+      ghcr.io:
+        url: <your IP address>:5000/ghcr.io
+      registry.k8s.io:
+        url: <your IP address>:5000/registry.k8s.io
+      icr.io:
+        url: <your IP address>:5000/icr.io
+      defaults:
+        type: docker
+        secure: false
+
+.. note::
+    This procedure configured |prod| to use an insecure registry via the
+    ``docker_registries.defaults.secure`` parameter set to ``false`` in the
+    excerpt above. Make sure you only use this on your own development
+    environment.
+
+.. _Kubernetes versions: https://opendev.org/starlingx/ansible-playbooks/src/branch/master/playbookconfig/src/playbooks/roles/bootstrap/validate-config/vars/main.yml
+.. _system images: https://opendev.org/starlingx/ansible-playbooks/src/branch/master/playbookconfig/src/playbooks/roles/common/load-images-information/vars