Initial skeleton for devstack-tempest base job

Setup the initial folder and play to run tempest.
This simply runs tempest full for now, with not support for config
options.

Change-Id: Ifcb4f8f36e5cc0cd0f25b13220146c4d97f7947a

.zuul.yaml

@@ -51,13 +51,25 @@
     post-run:
       playbooks/post
 
+- job:
+    name: devstack-tempest
+    parent: devstack
+    description: Base Tempest job.
+    required-projects:
+      - openstack/tempest
+    timeout: 7200
+    vars:
+      devstack_services:
+        tempest: True
+
 - project:
     name: openstack-infra/devstack-gate
     check:
       jobs:
-        - devstack:
+        - devstack-tempest:
             files:
               - ^playbooks/pre
               - ^playbooks/post
               - ^playbooks/devstack
+              - ^playbooks/devstack-tempest
               - ^roles/

playbooks/devstack-tempest.yaml

@@ -0,0 +1,15 @@
+# Changes that run through devstack-tempest are likely to have an impact on
+# the devstack part of the job, so we keep devstack in the main play to
+# avoid zuul retrying on legitimate failures.
+- hosts: all
+  roles:
+    - run-devstack
+
+# NOTE(andreaf) For now run on all, since we don't have anything in the
+# inventory to select a test runner.
+- hosts: all
+  roles:
+    - setup-tempest-run-dir
+    - setup-tempest-data-dir
+    - acl-devstack-files
+    - run-tempest

roles/acl-devstack-files/README.rst

@@ -0,0 +1,10 @@
+Grant global read access to devstack `files` folder.
+
+This is handy to grant the `tempest` user access to VM images for testing.
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_data_dir
+   :default: /opt/stack/data
+
+   The devstack data directory.

roles/acl-devstack-files/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_data_dir: /opt/stack/data

roles/acl-devstack-files/tasks/main.yaml

@@ -0,0 +1,6 @@
+- name: Grant global read access to devstack files
+  file:
+    path: "{{devstack_data_dir}}/files"
+    mode: "o+rx"
+    recurse: yes
+  become: yes

roles/run-tempest/README.rst

@@ -0,0 +1,13 @@
+Run Tempest
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_base_dir
+   :default: /opt/stack
+
+   The devstack base directory.
+
+.. zuul:rolevar:: tempest_concurrency
+   :default: 0
+
+   The number of parallel test processes.

roles/run-tempest/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_base_dir: /opt/stack

roles/run-tempest/tasks/main.yaml

@@ -0,0 +1,24 @@
+# NOTE(andreaf) The number of vcpus is not available on all systems.
+# See https://github.com/ansible/ansible/issues/30688
+# When not available, we fall back to ansible_processor_cores
+- name: Get hw.logicalcpu from sysctl
+  shell: sysctl hw.logicalcpu | cut -d' ' -f2
+  register: sysctl_hw_logicalcpu
+  when: ansible_processor_vcpus is not defined
+
+- name: Get default concurrency
+  set_fact:
+    default_concurrency: "{{ansible_processor_vcpus|default(sysctl_hw_logicalcpu.stdout)|int}}"
+
+- name: Limit max concurrency when more than 3 vcpus are available
+  set_fact:
+    default_concurrency: "{{default_concurrency|int // 2}}"
+  when: default_concurrency|int > 3
+
+- name: Run Tempest
+  command: tox -e full -- --concurrency={{tempest_concurrency|default(default_concurrency)}}
+  args:
+    chdir: "{{devstack_base_dir}}/tempest"
+  become: true
+  become_user: tempest
+

roles/setup-tempest-data-dir/README.rst

@@ -0,0 +1,12 @@
+Setup the `tempest` user as owner of Tempest's data folder.
+
+Tempest's devstack plugin creates the data folder, but it has no knowledge
+of the `tempest` user, so we need a role to fix ownership on the data folder.
+
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_data_dir
+   :default: /opt/stack/data
+
+   The devstack data directory.

roles/setup-tempest-data-dir/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_data_dir: /opt/stack/data

roles/setup-tempest-data-dir/tasks/main.yaml

@@ -0,0 +1,7 @@
+- name: Set tempest as owner of Tempest data folder
+  file:
+    path: "{{devstack_data_dir}}/tempest"
+    owner: tempest
+    group: stack
+    recurse: yes
+  become: yes

roles/setup-tempest-run-dir/README.rst

@@ -0,0 +1,14 @@
+Setup Tempest run folder.
+
+To support isolation between multiple runs, separate run folders are required.
+Set `tempest` as owner of Tempest's current run folder.
+There is an implicit assumption here of a one to one relationship between
+devstack versions and Tempest runs.
+
+
+**Role Variables**
+
+.. zuul:rolevar:: devstack_base_dir
+   :default: /opt/stack
+
+   The devstack base directory.

roles/setup-tempest-run-dir/defaults/main.yaml

@@ -0,0 +1 @@
+devstack_base_dir: /opt/stack

roles/setup-tempest-run-dir/tasks/main.yaml

@@ -0,0 +1,7 @@
+- name: Set tempest as owner of Tempest run folder
+  file:
+    path: "{{devstack_base_dir}}/tempest"
+    owner: tempest
+    group: stack
+    recurse: yes
+  become: yes