Fixes to allow install Ironic on remote node

This change introduces fixes, which allows install Ironic on remote
server. This may not complete list of changes required for all
operations.
Also added group 'target' to inventory file, which is same as localhost
by default, and host in this group can be replaced by remote host.

Change-Id: I59c942d2556c5e1b7eee661fb8cf13c4ae02b2d1

README.rst

@@ -108,7 +108,7 @@ Bifrost source code should be pulled directly from git first::
   cd bifrost
 
 Edit ``./playbooks/inventory/group_vars/*`` to match your environment. The
-localhost file is intended for steps executed upon the localhost, such as
+target file is intended for steps executed upon the target server, such as
 installation, or image generation.  The baremetal file is geared for steps
 performed on baremetal nodes, such as enrollment, deployment, or any other
 custom playbooks that a user may bolt on to this toolkit.
@@ -180,9 +180,9 @@ restarted.
 Run::
 
   If you have passwordless sudo enabled, run:
-     ansible-playbook -vvvv -i inventory/localhost install.yaml
+     ansible-playbook -vvvv -i inventory/target install.yaml
   Otherwise, add -K option to let Ansible prompting for the sudo  password:
-     ansible-playbook -K -vvvv -i inventory/localhost install.yaml
+     ansible-playbook -K -vvvv -i inventory/target install.yaml
 
 With regard to testing, ironic's node cleaning capability is disabled by
 default as it can be an unexpected surprise for a new user that their test
@@ -191,7 +191,7 @@ node is unusable for however long it takes for the disks to be wiped.
 If you wish to enable cleaning, you can achieve this by passing the option
 ``-e cleaning=true`` to the command line or executing the command below::
 
-  ansible-playbook -K -vvvv -i inventory/localhost install.yaml -e cleaning=true
+  ansible-playbook -K -vvvv -i inventory/target install.yaml -e cleaning=true
 
 After you have performed an installation, you can edit /etc/ironic/ironic.conf
 to enable or disable cleaning as desired, however it is highly encouraged to
@@ -202,7 +202,7 @@ These drivers and information about them can be found `here <http://git.openstac
 If you would like to install the ironic staging drivers, simply pass
 ``-e staging_drivers_include=true`` when executing the install playbook::
 
-  ansible-playbook -K -vvvv -i inventory/localhost install.yaml -e staging_drivers_include=true
+  ansible-playbook -K -vvvv -i inventory/target install.yaml -e staging_drivers_include=true
 
 Manual CLI use
 --------------
@@ -441,11 +441,11 @@ ironic conductor will use this key to connect to the host machine and
 run virsh commands.
 
 #. Set ``testing`` to *true* in the
-   ``playbooks/inventory/group_vars/localhost`` file.
+   ``playbooks/inventory/group_vars/target`` file.
 #. You may need to adjust the value for ``ssh_public_key_path``.
 #. Run the install step, as documented above, however adding ``-e
    testing=true`` to the Ansible command line.
-#. Execute the ``ansible-playbook -vvvv -i inventory/localhost
+#. Execute the ``ansible-playbook -vvvv -i inventory/target
    test-bifrost-create-vm.yaml`` command to create a test virtual
    machine.
 #. Set the environment variable of ``BIFROST_INVENTORY_SOURCE`` to the
@@ -545,6 +545,6 @@ ansible into it, run ``env-setup.sh`` as follows::
 
 Then run the install playbook with the following arguments::
 
-  ansible-playbook -vvvv -i inventory/localhost install.yaml
+  ansible-playbook -vvvv -i inventory/target install.yaml
 
 This will install ironic and its dependencies into the virtual environment.

doc/source/deploy/keystone.rst

@@ -18,10 +18,10 @@ Installation with Keystone
 Bifrost can now install and make use of keystone. In order to enable
 this as part of the installation, the ``enable_keystone`` variable
 must be set to ``true``.
-Either in ``playbooks/inventory/group_vars/localhost`` or on the
+Either in ``playbooks/inventory/group_vars/target`` or on the
 command line during installation. Example::
 
-    ansible-playbook -vvvv -i inventory/localhost install.yaml -e enable_keystone=true
+    ansible-playbook -vvvv -i inventory/target install.yaml -e enable_keystone=true
 
 However, prior to installation, overriding credentials should be set
 in order to customize the deployment to meet your needs. See::
@@ -38,7 +38,7 @@ will need to set the appropriate defaults, via
 ``playbooks/roles/bifrost-ironic-install/defaults/main.yml``
 which would be a good source for the role level defaults.
 Ideally, when setting new defaults, they should be set in the
-``playbooks/inventory/group_vars/localhost`` file.
+``playbooks/inventory/group_vars/target`` file.
 
 Creation of clouds.yaml
 -----------------------
@@ -66,8 +66,8 @@ following steps need to take place.
 
        -e noauth_mode=false -e cloud_name=bifrost
 
-#. Set the global defaults for localhost
-   (``master/playbooks/inventory/group_vars/localhost``).
+#. Set the global defaults for tagret
+   (``master/playbooks/inventory/group_vars/target``).
 
 
 OpenStack Client

playbooks/install.yaml

@@ -1,7 +1,6 @@
 ---
-- hosts: localhost
-  name: "Install Ironic on the local host."
-  connection: local
+- hosts: target
+  name: "Install Ironic on the target host."
   become: yes
   gather_facts: yes
   roles:

playbooks/inventory/group_vars/target

@@ -0,0 +1,72 @@
+---
+# The ironic API URL for bifrost operations.  Defaults to localhost.
+# ironic_url: "http://localhost:6385/"
+
+# The network interface that bifrost will be operating on.  Defaults
+# to virbr0 in roles, can be overridden here.
+# network_interface: "virbr0"
+
+
+# ironic_db_password ironic user password for rabbit
+ironic_db_password: aSecretPassword473z
+# mysql_username: Default mysql admin username
+mysql_username: root
+# mysql_password: Default mysql admin user password
+mysql_password:
+
+# The path to the SSH key to be utilized for testing and burn-in
+# to configuration drives. When set, it should be set in both baremetal
+# and localhost groups, however this is only an override to the default.
+# ssh_public_key_path: "{{ ansible_env.HOME }}/.ssh/id_rsa.pub"
+
+# If testing is true, then the environment is setup for using libvirt
+# virtual machines for the hardware instead of real hardware.
+# testing: true
+
+# The default port to download files via.  Required for IPA URL generation.
+# Presently the defaults are located in the roles, however if changed both
+# the localhost and baremetal group files must be updated.
+# file_url_port: 8080
+
+# IPA Image parameters.  If these are changed, they must be changed in
+# Both localhost and baremetal groups.  Presently the defaults
+# in each role should be sufficent for proper operation.
+# ipa_kernel: "{{http_boot_folder}}/coreos_production_pxe.vmlinuz"
+# ipa_ramdisk: "{{http_boot_folder}}/coreos_production_pxe_image-oem.cpio.gz"
+# ipa_kernel_url: "http://{{ hostvars[inventory_hostname]['ansible_' + network_interface]['ipv4']['address'] }}:{{file_url_port}}/coreos_production_pxe.vmlinuz"
+# ipa_ramdisk_url: "http://{{ hostvars[inventory_hostname]['ansible_' + network_interface]['ipv4']['address'] }}:{{file_url_port}}/coreos_production_pxe_image-oem.cpio.gz"
+
+# The http_boot_folder defines the root folder for the webserver.
+# If this setting is changed, it must be applied to both the baremetal
+# and localhost groups. Presently the role defaults are set to the value
+# below.
+# http_boot_folder: /httpboot
+
+# The settings for the name of the image to be deployed along with the
+# on disk location are below.  If changed, these settings must be applied
+# to both the baremetal and localhost groups.  If the file is already on
+# disk, then the image generation will not take place, otherwise an image
+# will be generated using diskimage-builder.
+# deploy_image_filename: "deployment_image.qcow2"
+# deploy_image: "{{http_boot_folder}}/{{deploy_image_filename}}"
+
+# Setting to utilize diskimage-builder to create a bootable image.
+create_image_via_dib: true
+
+# Transform boot image is intended for use with the Ubuntu trusty image.
+# It makes the image bootable by installing Grub.
+# Setting to prepend a partition image with a boot sector and partition table.
+transform_boot_image: false
+
+# Create IPA image instead of downloading an pre-made CoreOS IPA image.
+create_ipa_image: false
+
+# Dnsmasq default route for clients. If not defined, dnsmasq will push to
+# clients as default route the same IP of the dnsmasq server.
+# Default: undefined
+# dnsmasq_router: {{ ipv4_gateway }}
+
+# Dnsmasq default nameserver for clients. If not defined, this setting
+# will be ignored.
+# Default: undefined
+# dnsmasq_dns_servers: 8.8.8.8,8.8.4.4

playbooks/inventory/target

@@ -0,0 +1,2 @@
+[target]
+127.0.0.1 ansible_connection=local

playbooks/roles/bifrost-ironic-install/defaults/required_defaults_RedHat_family.yml

@@ -42,4 +42,6 @@ required_packages:
 # NOTE(cinerama): amtterm is not currently packaged for RHEL/CentOS 7.
   - libvirt-daemon
   - debootstrap
+  - gcc
+  - python2-pip
 iscsi_client_package: "iscsi-initiator-utils"

playbooks/roles/bifrost-ironic-install/defaults/required_defaults_Ubuntu_16.04.yml

@@ -2,3 +2,37 @@
 init_template: systemd_template.j2
 init_dest_dir: /lib/systemd/system/
 init_ext: .service
+required_packages:
+  - mysql-server
+  - rabbitmq-server
+  - python-dev
+  - python-mysqldb
+  - python-configparser
+  - libffi-dev
+  - libxslt1-dev
+  - libssl-dev
+  - libxml2-dev
+  - ipxe
+  - tftpd-hpa
+  - tftp-hpa
+  - syslinux-common
+  - syslinux
+  - xinetd
+  - parted
+  - ipmitool
+  - psmisc
+  - nginx
+  - wget
+  - genisoimage
+  - kpartx
+  - qemu-utils
+  - python-openwsman
+  - amtterm
+  - sgabios
+  - libvirt-bin
+  - debootstrap
+  - uuid-runtime
+  - curl
+  - dnsmasq
+  - python-pip
+  - python-pymysql

playbooks/roles/bifrost-ironic-install/tasks/bootstrap.yml

@@ -123,17 +123,15 @@
   copy:
     src: "{{ ironic_git_folder }}/etc/ironic/rootwrap.conf"
     dest: "/etc/ironic/rootwrap.conf"
+    remote_src: yes
     mode: 0644
     owner: root
     group: root
   when: skip_install is not defined and enable_pxe_drivers | bool == true
+# Note(ashestakov): "copy" module in ansible doesn't support recursive
+# copying on remote host. "cp" command used instead.
 - name: "Copy rootwrap.d contents from ironic source folder"
-  copy:
-    src: "{{ ironic_git_folder }}/etc/ironic/rootwrap.d/"
-    dest: "/etc/ironic/rootwrap.d"
-    mode: 0644
-    owner: root
-    group: root
+  command: cp -r "{{ ironic_git_folder }}/etc/ironic/rootwrap.d/" "/etc/ironic/rootwrap.d"
   when: skip_install is not defined and enable_pxe_drivers | bool == true
 
 - name: "Populate keystone for Bifrost"
@@ -146,6 +144,7 @@
   copy:
     src: "{{ ironic_git_folder }}/etc/ironic/policy.json"
     dest: "/etc/ironic/"
+    remote_src: yes
     owner: "ironic"
     group: "ironic"
     mode: 0644

playbooks/roles/bifrost-ironic-install/tasks/create_tftpboot.yml

@@ -37,7 +37,7 @@
   register: test_pxelinux
 
 - name: "Place pxelinux.0"
-  copy: src={{ syslinux_tftp_dir }}/pxelinux.0 dest=/tftpboot
+  copy: src={{ syslinux_tftp_dir }}/pxelinux.0 dest=/tftpboot remote_src=true
   when: test_pxelinux.stat.exists == false
 
 - name: "Place boot.ipxe helper script /etc/ironic"
@@ -54,14 +54,14 @@
   when: download_ipxe | bool == true
 
 - name: "Copy iPXE image into place"
-  copy: src={{ ipxe_dir }}/undionly.kpxe dest=/tftpboot/
+  copy: src={{ ipxe_dir }}/undionly.kpxe dest=/tftpboot/ remote_src=true
 
 # NOTE(TheJulia): Copy full iPXE chain loader images in case they are required.
 - name: "Copy full iPXE image into /httpboot"
-  copy: src={{ ipxe_dir }}/{{ ipxe_full_binary }} dest=/httpboot/
+  copy: src={{ ipxe_dir }}/{{ ipxe_full_binary }} dest=/httpboot/ remote_src=true
 
 - name: "Copy full iPXE image into /tftpboot"
-  copy: src={{ ipxe_dir }}/{{ ipxe_full_binary }} dest=/tftpboot/
+  copy: src={{ ipxe_dir }}/{{ ipxe_full_binary }} dest=/tftpboot/ remote_src=true
 
 # Similar logic to below can be utilized to retrieve files
 - name: "Determine if folder exists, else create and populate folder."

playbooks/roles/bifrost-keystone-install/defaults/required_defaults_Ubuntu_16.04.yml

@@ -2,3 +2,20 @@
 init_template: systemd_template.j2
 init_dest_dir: /lib/systemd/system/
 init_ext: .service
+required_packages:
+  - mysql-server
+  - rabbitmq-server
+  - python-dev
+  - python-mysqldb
+  - python-configparser
+  - libffi-dev
+  - libxslt1-dev
+  - libssl-dev
+  - libxml2-dev
+  - nginx
+  - uwsgi
+  - uwsgi-core
+  # - python-django-uwsgi - This may not be needed
+  - uwsgi-plugin-python
+  - python-pip
+  - python-pymysql

playbooks/roles/bifrost-keystone-install/tasks/bootstrap.yml

@@ -117,6 +117,7 @@
   copy:
     src: "{{ keystone_git_folder }}/etc/policy.json"
     dest: "/etc/keystone/"
+    remote_src: true
     owner: "keystone"
     group: "keystone"
     mode: 0644
@@ -125,6 +126,7 @@
   copy:
     src: "{{ keystone_git_folder }}/etc/keystone-paste.ini"
     dest: "/etc/keystone/"
+    remote_src: true
     owner: "keystone"
     group: "keystone"
     mode: 0644
@@ -184,18 +186,26 @@
     group: "{{ nginx_user }}" # TODO(TheJulia): Split webserver user/group.
     mode: 0755
 
+# Note(ashestakov): "copy" module in ansible doesn't support recursive
+# copying on remote host. "cp" command used instead.
 - name: "Copy keystone-wsgi-public  to /var/www/keystone/public"
-  copy:
-    src: "{{ keystone_install_prefix.stdout }}/keystone-wsgi-public"
-    dest: /var/www/keystone/public
+  command: cp -r "{{ keystone_install_prefix.stdout }}/keystone-wsgi-public" /var/www/keystone/public
+
+- name: "Ensure owner and mode of keystone-wsgi-public"
+  file:
+    path: /var/www/keystone/public
     owner: "keystone"
     group: "{{nginx_user}}"
     mode: 0754
 
+# Note(ashestakov): "copy" module in ansible doesn't support recursive
+# copying on remote host. "cp" command used instead.
 - name: "Copy keystone-wsgi-admin to /var/www/keystone/admin"
-  copy:
-    src: "{{ keystone_install_prefix.stdout }}/keystone-wsgi-admin"
-    dest: /var/www/keystone/admin
+  command: cp -r "{{ keystone_install_prefix.stdout }}/keystone-wsgi-admin" /var/www/keystone/admin
+
+- name: "Ensure owner and mode of keystone-wsgi-admin"
+  file:
+    path: /var/www/keystone/admin
     owner: "keystone"
     group: "{{nginx_user}}"
     mode: 0754
@@ -267,7 +277,7 @@
         service_name: 'uwsgi',
         username: "{{ nginx_user }}",
         exec_start_pre: "/usr/bin/install -m 755 -o {{ nginx_user }} -g {{ nginx_user }} -d /run/uwsgi",
-        args: '--master --emperor /etc/uwsgi/apps-enabled'}
+        args: '--master --emperor /etc/uwsgi/apps-enabled'} # TODO(TheJulia): Split webserver user/group.
 
 # NOTE(ashestakov) https://github.com/ansible/ansible-modules-core/issues/3764
 - name: "Remove uwsgi sysvinit init script"

releasenotes/notes/allow-install-remote-101eea1ea4009fe1.yaml

@@ -0,0 +1,6 @@
+---
+features:
+  - Allow install Ironic on remote server.
+    Added group 'target' to inventory, which is same as localhost by default.
+    For install ironic remotely, address and ssh credentials should be
+    configured in playbooks/inventory/target file.