Updates for containerized undercloud

This patch contains changes to deploy a containerized undercloud by
using: "openstack undercloud install --use-heat" command.

Details of the changes:

- The experimental containerized undercloud deployments driven via
  `undercloud deploy` CLI is not supported anymore for Pike.
- overcloud-prep-containers: fix an jinja2 syntax issue with "is defined"
- overcloud-prep-images: "--http-boot=/var/lib/ironic/httpboot" is
  added to the "openstack overcloud image upload" command.
- undercloud_rpm_dependencies parameter has been created, default to
  undercloud_tripleo_pkg value.
- Remove "undercloud_tripleo_pkg" so we only use
  "undercloud_rpm_dependencies". Note that "undercloud_tripleo_pkg"
  was only used in this repository.
- Remove "undercloud_update_packages" option which was technical debt.
- Add "undercloud_install_cli_options" option which is used to add extra
  options to "openstack undercloud install" command.
- Add "undercloud_container_images_file" which is the environment file
  with default parameters for containers to use with undercloud install CLI.
- Add "undercloud_custom_env_files" which provides custom t-h-t env files
  for the "undercloud install" `undercloud.conf` interface.
- Generate the undercloud prep containers script. This script will
  reproduce what we expect from our users to run with
  "openstack overcloud container image prepare" command, for the
  undercloud.
- Update undercloud-deploy.sh.j2, remove tech debt and update for new
  bits.
- Install heat libraries and other dependencies for containerized UC
  from python-tripleoclient-heat-installer. This should go away
  once python-tripleoclient installs python-tripleoclient-heat-installer
  automatically.
- Update undercloud-install.sh to use undercloud_install_cli_options
  which is empty by default but will be used in the container-undercloud
  featureset to override with the right parameters.

Partially-Implements: bp containerized-undercloud

Co-Authored-By: Bogdan Dobrelya <bdobreli@redhat.com>
Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Ian Main <imain@redhat.com>
Co-Authored-By: Martin André <m.andre@redhat.com>
Co-Authored-By: Wes Hayutin <weshayutin@gmail.com>

Change-Id: I7d056b208d5ac0639e3277f3f4ae453a9ee639e8

releasenotes/notes/undercloud-deploy-pike-f14645c4a5442ef8.yaml

@@ -0,0 +1,5 @@
+---
+deprecations:
+  - |
+    The experimental containerized undercloud deployments driven via
+    `undercloud deploy` CLI is not supported anymore for Pike.

roles/baremetal-undercloud/packages/defaults/main.yml

@@ -1,6 +1,3 @@
-# Base tripleo package
-undercloud_tripleo_pkg: python-tripleoclient
-
 # Additional packages
 undercloud_additional_pkgs:
   - vim

roles/baremetal-undercloud/packages/tasks/main.yml

@@ -5,7 +5,7 @@
 - name: Install TripleO package
   become: true
   package:
-    name: "{{ undercloud_tripleo_pkg }}"
+    name: "{{ undercloud_rpm_dependencies }}"
     state: present
   tags:
     - undercloud-tripleo-pkg

roles/extras-common/README.md

@@ -35,3 +35,4 @@ Role Variables
   TripleO deployment
 - `composable_scenario`: <''> -- path to the composable scenarios used at deployment time
 - `upgrade_composable_scenario`: <''> -- path to the composable scenarios used at upgrade time
+- `undercloud_rpm_dependencies`: <'python-tripleoclient'> -- Dependency packages for undercloud deployments.

roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2

@@ -25,7 +25,7 @@ fi
 
 ## * get build id
 ## ::
-{% if get_build_command is defined: %}
+{% if get_build_command is defined %}
 BUILD_ID={{ get_build_command }}
 {% else %}
 BUILD_ID={{ docker_image_tag }}

roles/overcloud-prep-images/templates/overcloud-prep-images.sh.j2

@@ -32,8 +32,7 @@ tar xvfp {{ overcloud_full_tar_name }}
 ## * Upload images to glance.
 ## ::
 
-openstack overcloud image upload {% if bash_deploy_ramdisk|bool %}--old-deploy-image{% endif %} {% if whole_disk_images|bool %}--whole-disk{% endif %}
+openstack overcloud image upload --http-boot=/var/lib/ironic/httpboot {% if bash_deploy_ramdisk|bool %}--old-deploy-image{% endif %} {% if whole_disk_images|bool %}--whole-disk{% endif %}
-
 {% endif %}
 
 {% if step_glance_upload|bool %}

roles/undercloud-deploy/README.md

@@ -33,8 +33,7 @@ user sessions to provide greater flexixiblity to our users. ** additional docume
 http://docs.openstack.org/developer/tripleo-quickstart/accessing-libvirt.html
 - `undercloud_conf_extra`: <''> -- extra options to be added to ~/undercloud.conf
 - `undercloud_extra_args`: <''> -- extra options for undercloud deploy command.
-- `undercloud_update_packages`: <'null'> -- a string with a list of packages to update as dependencies for
+- `undercloud_install_cli_options`: <''> -- extra options for undercloud install command.
-your hacking setup. By defaults it updates nothing, which is backwards compatible.
 - `undercloud_enable_mistral`: <'true'> -- sets up the 'enable_mistral' option
   in undercloud.conf.
 - `undercloud_enable_monitoring`: <'false'> -- sets up the 'enable_monitoring'
@@ -83,9 +82,17 @@ your hacking setup. By defaults it updates nothing, which is backwards compatibl
 - `undercloud_docker_image_tag`: <`docker_image_tag`> -- tag of docker images used for
   undercloud deployment. Defaults to the value provided for overcloud. Defaults to the
   value provided for overcloud.
+- `undercloud_container_images_file`: <"{{ working_dir }}/undercloud-containers-default-parameters.yaml"> --
+  the environment file with default parameters for containers to use with undercloud install CLI.
 - `undercloud_extra_services_args`: -- additional t-h-t (TripleO heat templates)
   environment files for extra services deployed on containerized underclouds. By default,
-  includes environments for docker, ironic, mistral and zaqar containerized services.
+  includes environments for docker. Those are only applied for the `undercloud deploy` CLI and not
+  for the `undercloud install`'s `undercloud.conf`. Use `undercloud_custom_env_files` for the latter.
+- `undercloud_custom_env_files`: <"{{ working_dir }}/undercloud-parameter-defaults.yaml"> --
+  custom t-h-t env files for the `undercloud install`'s `undercloud.conf` interface. By default,
+  it provides the default parameters generated from `undercloud_resource_registry_args` and
+  `undercloud_network_environment_args`. For the `undercloud deploy`, use
+  `undercloud_extra_services_args` instead.
 
 Role Network Variables
 ----------------------
@@ -201,21 +208,6 @@ Also note that you should normaly rely on the openstack-tripleo-heat-templates
 package. The custom t-h-t repo vars may break that package and should be used with
 caution.
 
-You can as well specify a list of the packages to be updated to the most
-recent versions. For example:
-
-```yaml
-undercloud_update_packages: >-
-  openstack-tripleo-common
-  openstack-tripleo-heat-templates
-  puppet-tripleo
-  python-tripleoclient
-  openstack-heat-agents
-```
-
-Or use the ``undercloud_update_packages: "'*'"`` to update all packages
-(may take a long).
-
 Note, checkout/install steps for the remaining yet unpackaged custom changes
 like dev branches for puppet modules, tripleo client, heat agent hooks, need
 to be covered in the custom ``undercloud-install.sh`` script body (rendered
@@ -228,18 +220,19 @@ example you may want to deploy:
  * containerized Keystone,
  * optional containerized Etcd,
 
-with disabled telemetry, docker images from `docker.io/tripleomaster`,
+with disabled telemetry, default docker images from `docker.io` namespace,
 debug logs for services and puppet, given a decent timeout, and keeping
 the ephemeral undercloud heat agent running for debug purposes:
 
 ```yaml
-undercloud_extra_args: >-
+undercloud_extra_services_args: >-
   -e {{overcloud_templates_path}}/environments/disable-telemetry.yaml
   -e {{overcloud_templates_path}}/environments/docker-minimal.yaml
   -e {{overcloud_templates_path}}/environments/services-docker/etcd.yaml
   -e {{overcloud_templates_path}}/environments/services/octavia.yaml
   -e {{overcloud_templates_path}}/environments/debug.yaml
   -e {{overcloud_templates_path}}/environments/config-debug.yaml
+undercloud_extra_args: >-
   --timeout 60
   --keep-running
 ```
@@ -256,7 +249,6 @@ resource_registry:
 
 parameter_defaults:
   DockerNamespace: tripleomaster
-  DockerNamespaceIsRegistry: false
   ComputeServices: {}
   SwiftCeilometerPipelineEnabled: false
 ```
@@ -303,19 +295,14 @@ And an example playbook to call the role is:
     undercloud_install_script: undercloud-deploy.sh.j2
     overcloud_templates_repo: https://github.com/johndoe/tripleo-heat-templates
     overcloud_templates_branch: dev
-    undercloud_update_packages: >-
+    undercloud_extra_services_args: >-
-      openstack-tripleo-common
-      openstack-tripleo-heat-templates
-      puppet-tripleo
-      python-tripleoclient
-      openstack-heat-agents
-    undercloud_extra_args: >-
       -e {{overcloud_templates_path}}/environments/disable-telemetry.yaml
       -e {{overcloud_templates_path}}/environments/docker-minimal.yaml
       -e {{overcloud_templates_path}}/environments/services-docker/etcd.yaml
       -e {{overcloud_templates_path}}/environments/services/octavia.yaml
       -e {{overcloud_templates_path}}/environments/debug.yaml
       -e {{overcloud_templates_path}}/environments/config-debug.yaml
+    undercloud_extra_args: >-
       --timeout 60
       --keep-running
   roles:

roles/undercloud-deploy/defaults/main.yml

@@ -1,8 +1,11 @@
 ---
 
 undercloud_config_file: undercloud.conf.j2
+undercloud_install_cli_options: ""
 undercloud_post_install_script: undercloud-install-post.sh.j2
 undercloud_install_log: "{{ working_dir }}/undercloud_install.log"
+undercloud_container_images_file: "{{ working_dir }}/undercloud-containers-default-parameters.yaml"
+undercloud_custom_env_files: "{{ working_dir }}/undercloud-parameter-defaults.yaml"
 
 # default undercloud install method
 # containerized_undercloud is an EXPERIMENTAL FEATURE
@@ -20,10 +23,6 @@ undercloud_roles_data: "{{overcloud_templates_path}}/roles_data_undercloud.yaml"
 # defines extra args and service templates for the undercloud deploy command
 undercloud_extra_args: ""
 undercloud_extra_services_args: >-
-  -e {{overcloud_templates_path}}/environments/services-docker/ironic.yaml
-  -e {{overcloud_templates_path}}/environments/services-docker/ironic-inspector.yaml
-  -e {{overcloud_templates_path}}/environments/services-docker/mistral.yaml
-  -e {{overcloud_templates_path}}/environments/services-docker/zaqar.yaml
   -e {{overcloud_templates_path}}/environments/docker.yaml
 
 # The undercloud can also be deployed with heat
@@ -69,12 +68,15 @@ libvirt_default_network_address: 192.168.122.1
 nameserver_from_virthost: false
 virthost_nameservers: []
 
+undercloud_enable_ironic: true
+undercloud_enable_ironic_inspector: true
 undercloud_enable_mistral: true
 undercloud_enable_monitoring: false
-undercloud_enable_telemetry: true
+undercloud_enable_telemetry: false
 undercloud_enable_tempest: true
 undercloud_enable_ui: true
 undercloud_enable_validations: true
+undercloud_enable_zaqar: true
 undercloud_ipxe_deploy: true
 
 undercloud_undercloud_debug: true

roles/undercloud-deploy/tasks/create-scripts.yml

@@ -73,3 +73,9 @@
     src: undercloud-parameter-defaults.yaml.j2
     dest: "{{ working_dir }}/undercloud-parameter-defaults.yaml"
     mode: 0600
+
+- name: Create the undercloud prep containers script
+  template:
+    src: undercloud-prep-containers.sh.j2
+    dest: "{{ working_dir }}/undercloud-prep-containers.sh"
+    mode: 0755

roles/undercloud-deploy/tasks/docker_prep.yml

@@ -1,13 +0,0 @@
----
-- name: Adding docker group
-  group:
-    name: docker
-    state: present
-  become: true
-
-- name: Adding "undercloud_user" to docker group
-  user:
-    name: "{{undercloud_user}}"
-    groups: docker
-    append: true
-  become: true

roles/undercloud-deploy/tasks/main.yml

@@ -3,15 +3,25 @@
   tags:
     - undercloud-install
 
-- include: docker_prep.yml
+# FIXME(bogdando) remove this when those become installed automagically
-  when: containerized_overcloud|bool
+# by python-tripleoclient (in Rocky)
+- name: Install dependencies for a containerized undercloud
+  become: true
+  package:
+    name: python-tripleoclient-heat-installer
   tags:
-    - containers
+    - undercloud-install
+    - undercloud-tripleo-pkg
+  when: containerized_undercloud|bool
 
 - include: create-scripts.yml
   tags:
     - undercloud-scripts
 
+- include: undercloud-prep-containers.yml
+  tags:
+    - undercloud-scripts
+
 - include: install-undercloud.yml
   tags:
     - undercloud-install

roles/undercloud-deploy/tasks/undercloud-prep-containers.yml

@@ -0,0 +1,6 @@
+---
+- name: Prepare containers for the undercloud containerized deployment
+  shell: >
+    set -o pipefail &&
+    {{ working_dir }}/undercloud-prep-containers.sh 2>&1 {{ timestamper_cmd }} >
+    {{ working_dir }}/undercloud_prep_containers.log

roles/undercloud-deploy/templates/undercloud-deploy.sh.j2

@@ -5,6 +5,8 @@ set -eux
 ### --start_docs
 ## Deploying the undercloud
 ## ========================
+
+## .. note:: In progress documentation is available at https://github.com/dprince/undercloud_containers/blob/master/doit.sh
 ## ::
 
 ## * Export the required environment variables.
@@ -14,71 +16,14 @@ set -eux
 export {{ var }}
 {% endfor %}
 
-## * Update requested packages.
-## ::
-
-{% if undercloud_update_packages | default(false) %}
-sudo yum update -y {{ undercloud_update_packages }}
-{% endif %}
-
-# TODO: remove this once we land patch to install them via RPM
-sudo yum install -y \
-    openstack-heat-api \
-    openstack-heat-engine \
-    openstack-heat-monolith \
-    openstack-heat-agents \
-    python-ipaddr \
-    python-tripleoclient \
-    openvswitch \
-    puppet-tripleo
-
-## * Start the required services.
-## ::
-
-sudo systemctl start openvswitch
-
-## * Update the puppet modules or install custom modules.
-##   NOTE: breaks installed packages, is a pure hack in for dev envs only!
-## ::
-
-# Amend "foo/bar" and uncomment as needed:
-#puppet module install foo/bar || puppet module upgrade foo/bar
-
-#MODULE_PATH="{{ working_dir }}/.puppet/code/modules"
-#rm -rf "${MODULE_PATH}/bar"
-#mkdir -p "${MODULE_PATH}"
-#cd "${MODULE_PATH}"
-#git clone -b dev https://github.com/foo/puppet-bar bar
-
 ## * get build id
 ## ::
-{% if get_build_command is defined: %}
+{% if get_build_command is defined %}
 BUILD_ID={{ get_build_command }}
 {% else %}
 BUILD_ID={{ undercloud_docker_image_tag }}
 {% endif %}
 
-# Copy installed/cloned modules and make a symlink
-#sudo rsync -avxH "${MODULE_PATH}" /usr/share/openstack-puppet/
-sudo mkdir -p /etc/puppet/modules/
-sudo ln -f -s /usr/share/openstack-puppet/modules/* /etc/puppet/modules/
-
-## * Configure the undercloud containers default parameters
-## ::
-{% if containerized_undercloud|bool %}
-openstack overcloud container image prepare \
-    --env-file {{ working_dir }}/undercloud-containers-default-parameters.yaml \
-    {{ undercloud_extra_services_args }} \
-    -r {{undercloud_roles_data}} \
-    --namespace {{ undercloud_docker_registry_host }}{% if undercloud_docker_registry_port is defined %}:{{ undercloud_docker_registry_port }}{% endif%}/{{ undercloud_docker_registry_namespace }} \
-    --tag $BUILD_ID
-{% endif %}
-
-echo "============================="
-echo "Undercloud containers default parameters:"
-cat {{ working_dir }}/undercloud-containers-default-parameters.yaml
-echo "============================="
-
 ## * Set the local ip used for the undercloud install
 ## ::
 
@@ -99,17 +44,32 @@ sudo openstack undercloud deploy \
 {% endif %}
     -e {{ working_dir }}/undercloud-parameter-defaults.yaml \
 {% if containerized_undercloud|bool %}
-    {{ undercloud_extra_services_args }} \
+{% if undercloud_enable_mistral|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/mistral.yaml \
+{% endif %}
+{% if undercloud_enable_ironic|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/ironic.yaml \
+{% endif %}
+{% if undercloud_enable_ironic_inspector|bool %}
+    -e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/ironic-inspector.yaml \
+{% endif %}
+{% if undercloud_enable_zaqar|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/zaqar.yaml \
+{% endif %}
+{% if undercloud_enable_telemetry|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-gnocchi.yaml \
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-aodh.yaml \
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-panko.yaml \
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-ceilometer.yaml \
+{% endif %}
     -e {{ working_dir }}/undercloud-containers-default-parameters.yaml \
 {% endif %}
+{% if undercloud_extra_services_args %}
+    {{ undercloud_extra_services_args }} \
+{% endif %}
+{% if undercloud_extra_args %}
     {{ undercloud_extra_args }} \
+{% endif %}
     -e {{ working_dir }}/hieradata-overrides-t-h-t-undercloud.yaml
 
-## * Copy the Openstack credentials file to the non root user.
-## ::
-
-sudo cp /root/stackrc {{ working_dir }}/stackrc
-sudo chown {{ undercloud_user }}:{{ undercloud_user }} {{ working_dir }}/stackrc
-
 ### --stop_docs
-

roles/undercloud-deploy/templates/undercloud-install.sh.j2

@@ -11,6 +11,6 @@ set -eux
 export {{ var }}
 {% endfor %}
 
-openstack undercloud install
+openstack undercloud install {{ undercloud_install_cli_options }}
 
 ### --stop_docs

roles/undercloud-deploy/templates/undercloud-prep-containers.sh.j2

@@ -0,0 +1,57 @@
+#!/bin/bash
+
+set -eux
+
+### --start_docs
+## Prepare the undercloud for deploying the containerized compute node
+## ===================================================================
+
+## .. note:: In progress documentation is available at https://github.com/dprince/undercloud_containers/blob/master/doit.sh
+## ::
+
+## * get build id
+## ::
+{% if get_build_command is defined %}
+BUILD_ID={{ get_build_command }}
+{% else %}
+BUILD_ID={{ undercloud_docker_image_tag }}
+{% endif %}
+
+## * Generate {{ working_dir }}/undercloud-containers-default-parameters.yaml
+## ::
+
+openstack overcloud container image prepare \
+    --output-env-file {{ working_dir }}/undercloud-containers-default-parameters.yaml \
+    --template-file /usr/share/openstack-tripleo-common/container-images/overcloud_containers.yaml.j2 \
+{% if undercloud_enable_mistral|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/mistral.yaml \
+{% endif %}
+{% if undercloud_enable_ironic|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/ironic.yaml \
+{% endif %}
+{% if undercloud_enable_ironic_inspector|bool %}
+    -e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/ironic-inspector.yaml \
+{% endif %}
+{% if undercloud_enable_zaqar|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/zaqar.yaml \
+{% endif %}
+{% if undercloud_enable_telemetry|bool %}
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-gnocchi.yaml \
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-aodh.yaml \
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-panko.yaml \
+    -e {{overcloud_templates_path}}/environments/services-docker/undercloud-ceilometer.yaml \
+{% endif %}
+{% if undercloud_extra_services_args %}
+    {{ undercloud_extra_services_args }} \
+{% endif %}
+    -r {{undercloud_roles_data}} \
+    --namespace {{ undercloud_docker_registry_host }}{% if undercloud_docker_registry_port is defined %}:{{ undercloud_docker_registry_port }}{% endif%}/{{ undercloud_docker_registry_namespace }} \
+    --tag $BUILD_ID
+
+echo "============================="
+echo "Containers default parameters:"
+cat {{ working_dir }}/undercloud-containers-default-parameters.yaml
+echo "============================="
+
+
+### --stop_docs

roles/undercloud-deploy/templates/undercloud.conf.j2

@@ -203,6 +203,18 @@ enable_ui = {{undercloud_enable_ui}}
 # Whether to install Mistral in the Undercloud. (boolean value)
 enable_mistral = {{undercloud_enable_mistral}}
 
+# Whether to install Ironic in the Undercloud. (boolean value)
+enable_ironic = {{undercloud_enable_ironic}}
+
+# Whether to install Ironic-inspector in the Undercloud. (boolean value)
+enable_ironic_inspector = {{undercloud_enable_ironic_inspector}}
+
+# Whether to install Zaqar in the Undercloud. (boolean value)
+enable_zaqar = {{undercloud_enable_zaqar}}
+
+# FIXME(mandre) turn this into ansible variable
+heat_native=true
+
 # Whether to use iPXE for deploy by default. (boolean value)
 ipxe_deploy = {{undercloud_ipxe_deploy}}
 
@@ -231,6 +243,25 @@ docker_registry_mirror = {{undercloud_docker_registry_mirror}}
 # docker_registry_mirror =
 {% endif %}
 
+# additional env files for undercloud, used with containers
+
+{% if undercloud_container_images_file is defined %}
+container_images_file = {{ undercloud_container_images_file }}
+{% else %}
+#container_images_file = <None>
+{% endif %}
+
+{% if undercloud_custom_env_files is defined %}
+custom_env_files = {{ undercloud_custom_env_files }}
+{% else %}
+#custom_env_files = <None>
+{% endif %}
+
+# Extra config elements.
+{% if undercloud_conf_extra != "" %}
+{{ undercloud_conf_extra }}
+{% endif %}
+
 [auth]
 
 #
@@ -412,7 +443,3 @@ undercloud_swift_hash_suffix = {{undercloud_undercloud_swift_hash_suffix}}
 {% else %}
 #undercloud_swift_hash_suffix = <None>
 {% endif %}
-
-{% if undercloud_conf_extra != "" %}
-{{ undercloud_conf_extra }}
-{% endif %}

roles/undercloud-setup/defaults/main.yml

@@ -24,6 +24,3 @@ external_interface: eth2
 external_interface_ip: 10.0.0.1
 external_interface_netmask: 255.255.255.0
 custom_nameserver: 8.8.8.8
-
-undercloud_rpm_dependencies: >-
-  python-tripleoclient

roles/undercloud-setup/templates/install_packages.sh.j2

@@ -8,7 +8,6 @@
 ## * Install python-tripleoclient
 ## ::
 
-#sudo yum install -y python-tripleoclient ceph-ansible
 sudo yum install -y {{ undercloud_rpm_dependencies }}
 
 ### --stop_docs