Add support to mixed upgrade for overcloud-prep-container role.

During mixed version upgrade, we want to deploy the overcloud with the
N-1 release docker images and then switch to the N release images
file.

That review[1] added the N-1 necessary tags.  That review[2] was an
attempt to use the proper parameters in that role for deployment in the
context of mixed version.  This fails.

The failure comes from the fact that we load the release file using
"-e ". All those variables then have the highest precedence[3]
possible and cannot be overridden.

Furthermore we use `use_overcloud_mixed_upgrade` flag to prevent the
yum upgrade to run inside container during initial overcloud
deployment.  The problem here is that the repo configuration is taken
from the undercloud which is N.  Thus the N-1 images are upgraded to
N.  That means that we can't gate "from" patches.  In a p->q setup, a
change in pike cannot be tested if it relates to the docker image.

Eventually we need a way to run the container role even when the
containerized_overcloud is false.  For Fast Forward Upgrade, the
starting point is Newton which doesn't use container.  So we introduce
`prep_container_upgrade_run` flag that can be set from the featureset
to ensure that the container get deployed during upgrade.

[1] https://review.openstack.org/#/c/537833/
[2] https://review.openstack.org/#/c/539920/1/toci-quickstart/playbooks/multinode-overcloud-prep.yml
[3] https://docs.ansible.com/ansible/latest/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable

Related-Bug: #1745348

This Tech-debt is introduced until the refactor of the release loading
is done.
Related-Bug: #1749978

Change-Id: I4df06c8a0d5d8133ba64c38b90cef34793776422

playbooks/multinode-overcloud-prep.yml

@@ -33,6 +33,7 @@
     - overcloud-prep-containers
   roles:
     - role: overcloud-prep-containers
+      use_overcloud_mixed_upgrade: "{{mixed_upgrade|default(false)}}"
       when: containerized_overcloud|bool
 
 - name: Run tripleo-validations pre-deployment tests

playbooks/multinode-overcloud-upgrade-new.yml

@@ -23,6 +23,12 @@
     - role: overcloud-prep-containers
       # This need the new repo in place.
       update_containers: true
+      # This enable us to run it even on non-containerized overcloud deployment
+      # ie, ffu upgrade where deployment is on newton.
+      containerized_overcloud: "{{prep_container_upgrade_run|default(false)}}"
+      # Do not overwrite the deployment log if any.
+      overcloud_prep_containers_log: "upgrade_overcloud_prep_containers.log"
+      overcloud_prep_containers_script: "upgrade_overcloud_prep_containers.sh"
       when: containerized_overcloud_upgrade|bool
 
 #FIXME(chem): tech-debt LP#1749740

roles/overcloud-prep-containers/README.md

@@ -33,7 +33,10 @@ overcloud-prep-containers variables
 * delete_docker_cache: <false> -- whether to stop Docker, wipe all the Docker data, and restart
 * update_containers: <false> -- whether to run container-check to update containers
 * container_process_count: <8> -- number of concurrent processes to run when updating containers
-
+* use_overcloud_mixed_upgrade: <false> - when true, we use overcloud release tag and build_id for
+                                         container images in mixed upgrade context.
+* prep_container_upgrade_run: <false> - in mixed version upgrade offer a flag to have container
+                                        deployed during upgrade.
 
 overcloud-prep-config variables
 -------------------------------

roles/overcloud-prep-containers/defaults/main.yml

@@ -7,3 +7,5 @@ overcloud_prep_containers_log: "overcloud_prep_containers.log"
 prepare_service_env_args: -e {{ overcloud_templates_path }}/environments/docker.yaml
 update_containers: false
 container_process_count: 8
+use_overcloud_mixed_upgrade: false
+prep_container_upgrade_run: false

roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2

@@ -13,6 +13,7 @@ set -eux
 ## ------------------------
 
 ## * Add an additional insecure registry if needed
+## ::
 {% if additional_insecure_registry|bool %}
 if egrep -q "^INSECURE_REGISTRY=.*{{docker_registry_host}}.*" /etc/sysconfig/docker; then
     echo "/etc/sysconfig/docker contains the correct settings"
@@ -25,9 +26,21 @@ fi
 
 ## * get build id
 ## ::
+{% set docker_registry_namespace_used = docker_registry_namespace %}
 {% if get_build_command is defined %}
 BUILD_ID={{ get_build_command }}
+
+##   a) Checking if we are deploying a N-1 (relative to release) docker images
+##      So that we use the the N-1 BUILD_ID and the N-1 docker_registry_namespace
+##      for deployment.  This is used in mixed upgrade deployment.
+## ::
+{% elif use_overcloud_mixed_upgrade|default(false)|bool %}
+BUILD_ID={{ overcloud_docker_image_tag }}
+{% set docker_registry_namespace_used = overcloud_docker_registry_namespace %}
 {% else %}
+
+##   b) We are using the release version of the docker images.
+## ::
 BUILD_ID={{ docker_image_tag }}
 {% endif %}
 
@@ -39,7 +52,7 @@ PREPARE_ARGS=${PREPARE_ARGS:-"{{ prepare_service_env_args }}"}
 openstack overcloud container image prepare \
     --images-file {{ working_dir }}/overcloud_containers.yaml \
     ${PREPARE_ARGS} \
-    --namespace {{ docker_registry_host }}/{{ docker_registry_namespace }} \
+    --namespace {{ docker_registry_host }}/{{ docker_registry_namespace_used }} \
     --tag $BUILD_ID \
 {% if docker_prep_prefix is defined %}
     --prefix {{ docker_prep_prefix}} \
@@ -67,7 +80,7 @@ openstack overcloud container image prepare \
     --images-file {{ working_dir }}/overcloud_containers.yaml \
     --env-file {{ working_dir }}/containers-default-parameters.yaml \
     ${PREPARE_ARGS} \
-    --namespace {{ local_docker_registry_host }}:8787/{{ docker_registry_namespace }} \
+    --namespace {{ local_docker_registry_host }}:8787/{{ docker_registry_namespace_used }} \
     --set ceph_namespace={{ local_docker_registry_host }}:8787/ceph \
 {% if docker_prep_prefix is defined %}
     --prefix={{ docker_prep_prefix}} \
@@ -83,7 +96,7 @@ openstack overcloud container image prepare \
 {% endif %}
     --tag $BUILD_ID
 
-{% if update_containers|bool %}
+{% if update_containers|bool and not use_overcloud_mixed_upgrade|default(false)|bool %}
 # See https://github.com/imain/container-check for script and documentation
 
 ## * rename the base-os yum repos to disable them. This will speed up the update