Initial skeleton for devstack-tempest base job

Setup the initial folder and play to run tempest.
This simply runs tempest full for now, with not support for config
options.

Change-Id: I5a76dd23900a1b5fb1764fafd837d69baf9ed8b1
Depends-on: Iffe54fbccbccd68db08f79a1b51dd7f76dbff408

.zuul.yaml

@@ -0,0 +1,23 @@
+- job:
+    name: devstack-tempest
+    parent: devstack
+    description: Base Tempest job.
+    required-projects:
+      - openstack/tempest
+    timeout: 7200
+    roles:
+      - zuul: openstack-dev/devstack
+    vars:
+      devstack_services:
+        tempest: True
+    run: playbooks/devstack-tempest.yaml
+
+- project:
+    name: openstack/tempest
+    check:
+      jobs:
+        - devstack-tempest:
+            files:
+              - ^playbooks/
+              - ^roles/
+              - ^.zuul.yaml$

playbooks/devstack-tempest.yaml

@@ -0,0 +1,14 @@
+# Changes that run through devstack-tempest are likely to have an impact on
+# the devstack part of the job, so we keep devstack in the main play to
+# avoid zuul retrying on legitimate failures.
+- hosts: all
+  roles:
+    - run-devstack
+
+# We run tests only on one node, regardless how many nodes are in the system
+- hosts: tempest
+  roles:
+    - setup-tempest-run-dir
+    - setup-tempest-data-dir
+    - acl-devstack-files
+    - run-tempest

roles/acl-devstack-files/README.rst

@@ -0,0 +1,10 @@
+Grant global read access to devstack `files` folder.
+
+This is handy to grant the `tempest` user access to VM images for testing.
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_data_dir
+   :default: /opt/stack/data
+
+   The devstack data directory.

roles/acl-devstack-files/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_data_dir: /opt/stack/data

roles/acl-devstack-files/tasks/main.yaml

@@ -0,0 +1,6 @@
+- name: Grant global read access to devstack files
+  file:
+    path: "{{devstack_data_dir}}/files"
+    mode: "o+rx"
+    recurse: yes
+  become: yes

roles/run-tempest/README.rst

@@ -0,0 +1,18 @@
+Run Tempest
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_base_dir
+   :default: /opt/stack
+
+   The devstack base directory.
+
+.. zuul:rolevar:: tempest_concurrency
+   :default: 0
+
+   The number of parallel test processes.
+
+.. zuul:rolevar:: tox_venvlist
+   :default: smoke
+
+   The Tempest tox environment to run.

roles/run-tempest/defaults/main.yaml

@@ -0,0 +1,2 @@
+devstack_base_dir: /opt/stack
+tox_venvlist: smoke

roles/run-tempest/tasks/main.yaml

@@ -0,0 +1,28 @@
+# NOTE(andreaf) The number of vcpus is not available on all systems.
+# See https://github.com/ansible/ansible/issues/30688
+# When not available, we fall back to ansible_processor_cores
+- name: Get hw.logicalcpu from sysctl
+  shell: sysctl hw.logicalcpu | cut -d' ' -f2
+  register: sysctl_hw_logicalcpu
+  when: ansible_processor_vcpus is not defined
+
+- name: Number of cores
+  set_fact:
+    num_cores: "{{ansible_processor_vcpus|default(sysctl_hw_logicalcpu.stdout)}}"
+
+- name: Set concurrency for cores == 3 or less
+  set_fact:
+    default_concurrency: "{{ num_cores }}"
+  when: num_cores|int <= 3
+
+- name: Limit max concurrency when more than 3 vcpus are available
+  set_fact:
+    default_concurrency: "{{ num_cores|int // 2 }}"
+  when: num_cores|int > 3
+
+- name: Run Tempest
+  command: tox -e {{tox_venvlist}} -- --concurrency={{tempest_concurrency|default(default_concurrency)}}
+  args:
+    chdir: "{{devstack_base_dir}}/tempest"
+  become: true
+  become_user: tempest

roles/setup-tempest-data-dir/README.rst

@@ -0,0 +1,12 @@
+Setup the `tempest` user as owner of Tempest's data folder.
+
+Tempest's devstack plugin creates the data folder, but it has no knowledge
+of the `tempest` user, so we need a role to fix ownership on the data folder.
+
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_data_dir
+   :default: /opt/stack/data
+
+   The devstack data directory.

roles/setup-tempest-data-dir/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_data_dir: /opt/stack/data

roles/setup-tempest-data-dir/tasks/main.yaml

@@ -0,0 +1,7 @@
+- name: Set tempest as owner of Tempest data folder
+  file:
+    path: "{{devstack_data_dir}}/tempest"
+    owner: tempest
+    group: stack
+    recurse: yes
+  become: yes

roles/setup-tempest-run-dir/README.rst

@@ -0,0 +1,14 @@
+Setup Tempest run folder.
+
+To support isolation between multiple runs, separate run folders are required.
+Set `tempest` as owner of Tempest's current run folder.
+There is an implicit assumption here of a one to one relationship between
+devstack versions and Tempest runs.
+
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_base_dir
+   :default: /opt/stack
+
+   The devstack base directory.

roles/setup-tempest-run-dir/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_base_dir: /opt/stack

roles/setup-tempest-run-dir/tasks/main.yaml

@@ -0,0 +1,7 @@
+- name: Set tempest as owner of Tempest run folder
+  file:
+    path: "{{devstack_base_dir}}/tempest"
+    owner: tempest
+    group: stack
+    recurse: yes
+  become: yes