Ensure FluxCD image upgrade for multi-node

It is observed on multi-node systems that during updade-activate
step images may not be present in local registry. Although the
root issue was [1], it was discovered that
upgrade-static-images.yml is called each hour.

I'm thinking of this scenario: upgrade-static-images.yml was called
earlier than workaround [2] for [1]. Upgrade activate is called
earlier than next upgrade-static-images.yml. FluxCD pods are not
present now. This results in FluxCD upgrade script failing, because
of timeout for deployment (pods are not up, but in
ImagePullBackOff state).

Instead of relying on timing, add determinism here and make sure Flux
images are present before deploying pods.

Tests on AIO-SX:
Although real tests would be done on multi-node system, emulated
upgrade instead. Playbook finish without errors for each of the tests.

DESC: Emulate bad multi-node scenario.
      Remove fluxcd pods from local registry and CRI.
      Call upgrade-fluxcd-controllers.yml.
PASS: Observe flux images populated

DESC: Emulate AIO-SX scenario.
      Already know flux images are present before unlock.
      Call upgrade-fluxcd-controllers.yml with image present.
PASS: Observe flux images populated again though we didn't need them

[1]: https://bugs.launchpad.net/starlingx/+bug/1999182
[2]: https://review.opendev.org/c/starlingx/config/+/867053
Partial-Bug: 1999032
Signed-off-by: Dan Voiculeasa <dan.voiculeasa@windriver.com>
Change-Id: Id33870f28ec027d99c0763b5d89626e2546fecf9

playbookconfig/src/playbooks/roles/common/push-docker-images/tasks/main.yml

@@ -62,6 +62,14 @@
     download_images_list: "{{ download_images_list|default([]) + static_images + security_images }}"
   when: mode == 'upgrade_static_images'
 
+# We only want fluxcd images for fluxcd upgrades, but can live with a broader
+# target for the sake of keeping it variable definitions contained to
+# a variable: static_images
+- name: Set download images list to static images for FluxCD image upgrade
+  set_fact:
+    download_images_list: "{{ download_images_list|default([]) + static_images }}"
+  when: mode == 'upgrade_fluxcd_images'
+
 # Only download k8s networking images if k8s networking upgrade
 - name: Set download images list to k8s network images if upgrading k8s networking
   set_fact:

playbookconfig/src/playbooks/upgrade-fluxcd-controllers.yml

@@ -9,7 +9,10 @@
 
   vars:
     playbooks_root: "/usr/share/ansible/stx-ansible/playbooks"
+    mode: "upgrade_fluxcd_images"
 
   # Main play
   roles:
+    - { role: common/get-kube-version, become: yes }
+    - { role: common/push-docker-images, become: yes }
     - { role: common/fluxcd-controllers, become: yes }