Implement tripleo-systemd-wrapper role

This patch adds a new role that will be used to manage side containers
with systemd instead of docker.socket or nsenter.  The main use case here
is Neutron, although this role is designed to work with any service.

This role will create a series of systemd files to monitor a file which
gets mounted into a container. Additionally a wrapper script is
generated which is mounted in the container that will provide the
arguments that should be used to launch new containers.

Blueprint: safe-side-containers
Change-Id: I4821b7ca0260e4dfd1717ba976cef700d160f84f
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Alex Schultz <aschultz@redhat.com>

doc/source/roles/role-tripleo-systemd-wrapper.rst

@@ -0,0 +1,90 @@
+==============================
+Role - tripleo-systemd-wrapper
+==============================
+
+An Ansible role to manage systemd wrappers in TripleO.
+
+
+What it does
+------------
+
+In a nutshell, this role helps to configure systemd so it manages side car
+containers (e.g. dnsmasq, HAproxy, keepalived, etc, requested by Neutron
+agents).
+
+Underneath, this role creates four files:
+
+- /etc/systemd/system/<service>.path
+
+This file will allow the host to monitor changes to
+/var/lib/<service>/<service>-processes-timestamp which keeps track of the
+service processes in a text file.
+<service>-processes-timestamp file is managed by the <service>-wrapper script
+with a flock to avoid race conditions.
+
+- /etc/systemd/system/<service>.service
+
+This file is the SystemD service that will run the synchronization of
+processes. It is run as "Type=oneshot" because we just want the unit to execute
+the <service>-process-sync script without keeping active processes.
+In this Ansible role, we automatically enable and start this service.
+
+- /var/lib/<service>/<service>/wrapper
+
+Script that wrap the service lifecycle management. It takes care of starting
+the side containers everytime the service is called.
+Because it's a wrapper, the script has to be bind mounted from the host into
+the container.
+
+e.g.: /var/lib/neutron/neutron-dnsmasq/wrapper:/usr/local/bin/dnsmasq:ro
+
+So in the case of Neutron DHCP agent, when an operator will create a network,
+Neutron will call dnsmasq which will actually call our side container wrapper.
+
+- /var/lib/neutron/<service>/process-sync
+
+This script helps to keep the list of processes (side containers) up to date,
+so we don't create more than one container per namespace. We use flock to avoid
+a race condition if at the same time the wrapper is called. The flock protects
+the list of processes and also the timestamps.
+
+
+Requirements
+------------
+
+It requires systemd on the host. This role isn't designed nor tested to run
+within a container.
+
+Role variables
+--------------
+
+- tripleo_systemd_wrapper_cmd: -- Command to run in the container.
+- tripleo_systemd_wrapper_config_bind_mount: -- Bind-mount used for container config.
+- tripleo_systemd_wrapper_container_cli: -- Name of the container cli command to use (podman | docker).
+- tripleo_systemd_wrapper_docker_additional_sockets: -- Additional docker sockets to use when interacting with docker
+- tripleo_systemd_wrapper_image_name: -- Container image name.
+- tripleo_systemd_wrapper_service_dir: -- Directory where state files will be created.
+- tripleo_systemd_wrapper_service_kill_script: -- Name of the script to create for the kill action
+- tripleo_systemd_wrapper_service_name: -- Name of the service to wrap in Systemd.
+
+Example Playbook
+----------------
+
+Sample playbook to call the role::
+
+  - name: Create Neutron dnsmasq systemd wrapper
+    hosts: all
+    roles:
+      - tripleo-systemd-wrapper
+    vars:
+      tripleo_systemd_wrapper_cmd: "/usr/sbin/dnsmasq -k"
+      tripleo_systemd_wrapper_config_bind_mount: "/var/lib/config-data/puppet-generated/neutron/etc/neutron:/etc/neutron:ro"
+      tripleo_systemd_wrapper_container_cli: podman
+      tripleo_systemd_wrapper_image_name: "docker.io/tripleomaster/centos-binary-neutron-dhcp-agent:current-tripleo"
+      tripleo_systemd_wrapper_service_dir: /var/lib/neutron
+      tripleo_systemd_wrapper_service_kill_script: dnsmasq-kill
+      tripleo_systemd_wrapper_service_name: neutron-dnsmasq
+
+
+.. ansibleautoplugin::
+  :role: tripleo_ansible/roles/tripleo-systemd-wrapper

tripleo_ansible/roles/tripleo-systemd-wrapper/defaults/main.yml

@@ -0,0 +1,22 @@
+---
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+# All variables intended for modification should be placed in this file.
+
+# All variables within this role should have a prefix of "tripleo_systemd_wrapper"
+tripleo_systemd_wrapper_debug: false
+tripleo_systemd_wrapper_container_cli: podman

tripleo_ansible/roles/tripleo-systemd-wrapper/meta/main.yml

@@ -0,0 +1,44 @@
+---
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+galaxy_info:
+  author: OpenStack
+  description: TripleO OpenStack Role -- tripleo-systemd-wrapper
+  company: Red Hat
+  license: Apache-2.0
+  min_ansible_version: 2.7
+  #
+  # Provide a list of supported platforms, and for each platform a list of versions.
+  # If you don't wish to enumerate all versions for a particular platform, use 'all'.
+  # To view available platforms and versions (or releases), visit:
+  # https://galaxy.ansible.com/api/v1/platforms/
+  #
+  platforms:
+    - name: Fedora
+      versions:
+        - 28
+    - name: CentOS
+      versions:
+        - 7
+
+  galaxy_tags:
+    - tripleo
+
+
+# List your role dependencies here, one per line. Be sure to remove the '[]' above,
+# if you add dependencies to this list.
+dependencies: []

tripleo_ansible/roles/tripleo-systemd-wrapper/molecule/default/Dockerfile

@@ -0,0 +1,37 @@
+# Molecule managed
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+{% if item.registry is defined %}
+FROM {{ item.registry.url }}/{{ item.image }}
+{% else %}
+FROM {{ item.image }}
+{% endif %}
+
+RUN if [ $(command -v apt-get) ]; then apt-get update && apt-get install -y python sudo bash ca-certificates && apt-get clean; \
+    elif [ $(command -v dnf) ]; then dnf makecache && dnf --assumeyes install sudo python*-devel python*-dnf bash {{ item.pkg_extras | default('') }} && dnf clean all; \
+    elif [ $(command -v yum) ]; then yum makecache fast && yum install -y python sudo yum-plugin-ovl python-setuptools bash {{ item.pkg_extras | default('') }} && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
+    elif [ $(command -v zypper) ]; then zypper refresh && zypper install -y python sudo bash python-xml {{ item.pkg_extras | default('') }} && zypper clean -a; \
+    elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates {{ item.pkg_extras | default('') }}; \
+    elif [ $(command -v xbps-install) ]; then xbps-install -Syu && xbps-install -y python sudo bash ca-certificates {{ item.pkg_extras | default('') }} && xbps-remove -O; fi
+
+{% for pkg in item.easy_install | default([]) %}
+# install pip for centos where there is no python-pip rpm in default repos
+RUN easy_install {{ pkg }}
+{% endfor %}
+
+
+CMD ["sh", "-c", "while true; do sleep 10000; done"]

tripleo_ansible/roles/tripleo-systemd-wrapper/molecule/default/molecule.yml

@@ -0,0 +1,60 @@
+---
+driver:
+  name: docker
+
+log: true
+
+platforms:
+  - name: centos7
+    hostname: centos7
+    image: centos:7
+    dockerfile: Dockerfile
+    pkg_extras: python-setuptools
+    easy_install:
+      - pip
+    environment: &env
+      http_proxy: "{{ lookup('env', 'http_proxy') }}"
+      https_proxy: "{{ lookup('env', 'https_proxy') }}"
+    # needed for systemd stuff
+    command: /sbin/init
+    capabilities:
+      - SYS_ADMIN
+    volumes:
+      - /sys/fs/cgroup:/sys/fs/cgroup:ro
+
+  - name: centos8
+    hostname: centos8
+    image: centos:8
+    dockerfile: Dockerfile
+    pkg_extras: python*-setuptools
+    environment:
+      <<: *env
+    # needed for systemd stuff
+    command: /sbin/init
+    capabilities:
+      - SYS_ADMIN
+    volumes:
+      - /sys/fs/cgroup:/sys/fs/cgroup:ro
+
+provisioner:
+  name: ansible
+  log: true
+  env:
+    ANSIBLE_STDOUT_CALLBACK: yaml
+
+scenario:
+  test_sequence:
+    - destroy
+    - create
+    - prepare
+    - converge
+    - verify
+    - destroy
+
+lint:
+  enabled: false
+
+verifier:
+  name: testinfra
+  lint:
+    name: flake8

tripleo_ansible/roles/tripleo-systemd-wrapper/molecule/default/playbook.yml

@@ -0,0 +1,29 @@
+---
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+- name: Converge
+  hosts: all
+  roles:
+    - "tripleo-systemd-wrapper"
+  vars:
+    tripleo_systemd_wrapper_cmd: "/usr/sbin/dnsmasq -k"
+    tripleo_systemd_wrapper_config_bind_mount: "/var/lib/config-data/puppet-generated/neutron/etc/neutron:/etc/neutron:ro"
+    tripleo_systemd_wrapper_container_cli: podman
+    tripleo_systemd_wrapper_image_name: "docker.io/tripleomaster/centos-binary-neutron-dhcp-agent:current-tripleo"
+    tripleo_systemd_wrapper_service_dir: /var/lib/neutron
+    tripleo_systemd_wrapper_service_kill_script: dnsmasq-kill
+    tripleo_systemd_wrapper_service_name: neutron-dnsmasq

tripleo_ansible/roles/tripleo-systemd-wrapper/molecule/default/prepare.yml

@@ -0,0 +1,21 @@
+---
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+- name: Prepare
+  hosts: all
+  roles:
+    - role: test_deps

tripleo_ansible/roles/tripleo-systemd-wrapper/molecule/default/verify.yml

@@ -0,0 +1,15 @@
+---
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.

tripleo_ansible/roles/tripleo-systemd-wrapper/tasks/main.yml

@@ -0,0 +1,112 @@
+---
+# Copyright 2019 Red Hat, Inc.
+# All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+
+# "tripleo-systemd-wrapper" will search for and load any operating system variable file
+
+# found within the "vars/" path. If no OS files are found the task will skip.
+- name: Gather variables for each operating system
+  include_vars: "{{ item }}"
+  with_first_found:
+    - skip: true
+      files:
+        - "{{ ansible_distribution | lower }}-{{ ansible_distribution_version | lower }}.yml"
+        - "{{ ansible_distribution | lower }}-{{ ansible_distribution_major_version | lower }}.yml"
+        - "{{ ansible_os_family | lower }}-{{ ansible_distribution_major_version | lower }}.yml"
+        - "{{ ansible_distribution | lower }}.yml"
+        - "{{ ansible_os_family | lower }}-{{ ansible_distribution_version.split('.')[0] }}.yml"
+        - "{{ ansible_os_family | lower }}.yml"
+  tags:
+    - always
+
+- name: "Ensure {{ tripleo_systemd_wrapper_service_dir }} exists"
+  become: true
+  file:
+    path: "{{ tripleo_systemd_wrapper_service_dir }}"
+    state: directory
+    setype: svirt_sandbox_file_t
+    selevel: s0
+
+- name: "Ensure {{ tripleo_systemd_wrapper_service_dir }}/kill_scripts exists"
+  become: true
+  file:
+    path: "{{ tripleo_systemd_wrapper_service_dir }}/kill_scripts"
+    state: directory
+    setype: svirt_sandbox_file_t
+    selevel: s0
+  when: tripleo_systemd_wrapper_service_kill_script is defined
+
+- name: "Ensure {{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }} exists"
+  become: true
+  file:
+    path: "{{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}"
+    state: directory
+    setype: svirt_sandbox_file_t
+    selevel: s0
+    mode: '4750'
+
+- name: "Create {{ tripleo_systemd_wrapper_service_name }} process command script"
+  become: true
+  template:
+    dest: "{{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/command"
+    src: service_command.j2
+    mode: '0750'
+
+# TODO(emilien) figure out secure permissions & ownership & labeling
+- name: "Create {{ tripleo_systemd_wrapper_service_name }} wrapper script"
+  become: true
+  template:
+    dest: "{{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/wrapper"
+    src: service_wrapper.j2
+    mode: '0750'
+
+# TODO(emilien) figure out secure permissions & ownership & labeling
+- name: "Create {{ tripleo_systemd_wrapper_service_name }} process sync script"
+  become: true
+  template:
+    dest: "{{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/sync"
+    src: service_sync.j2
+    mode: '0750'
+
+- name: "Create {{ tripleo_systemd_wrapper_service_name }} service kill script"
+  become: true
+  template:
+    dest: "{{ tripleo_systemd_wrapper_service_dir }}/kill_scripts/{{ tripleo_systemd_wrapper_service_kill_script }}"
+    src: service_kill.j2
+    mode: '0755'
+  when: tripleo_systemd_wrapper_service_kill_script is defined
+
+- name: "Create {{ tripleo_systemd_wrapper_service_name }} systemd path file"
+  become: true
+  template:
+    dest: "/etc/systemd/system/{{ tripleo_systemd_wrapper_service_name }}.path"
+    src: service.path.j2
+    mode: '0644'
+
+- name: "Create {{ tripleo_systemd_wrapper_service_name }} systemd service file"
+  become: true
+  template:
+    dest: "/etc/systemd/system/{{ tripleo_systemd_wrapper_service_name }}.service"
+    src: service.service.j2
+    mode: '0644'
+
+- name: "Start {{ tripleo_systemd_wrapper_service_name }} path"
+  become: true
+  systemd:
+    name: "{{ tripleo_systemd_wrapper_service_name }}.path"
+    enabled: true
+    state: started
+    daemon_reload: true

tripleo_ansible/roles/tripleo-systemd-wrapper/templates/service.path.j2

@@ -0,0 +1,5 @@
+[Path]
+PathModified={{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/processes-timestamp
+
+[Install]
+WantedBy=multi-user.target

tripleo_ansible/roles/tripleo-systemd-wrapper/templates/service.service.j2

@@ -0,0 +1,10 @@
+[Unit]
+Description=Tripleo {{ tripleo_systemd_wrapper_service_name }} sync service
+
+[Service]
+{% if tripleo_systemd_wrapper_debug %}
+Environment=SYSTEMD_LOG_LEVEL=debug
+{% endif %}
+Type=oneshot
+ExecStart={{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/sync
+User=root

tripleo_ansible/roles/tripleo-systemd-wrapper/templates/service_command.j2

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+#
+# We wrap the command in a bash script so we can do complex logic
+# to make the command dynamic based on the internals of the container.
+# This is necessary for backwards compatibily when commands change their
+# args based on version (I'm looking at you haproxy).
+#
+{{ tripleo_systemd_wrapper_cmd }} $@

tripleo_ansible/roles/tripleo-systemd-wrapper/templates/service_kill.j2

@@ -0,0 +1,71 @@
+#!/bin/bash
+{% if tripleo_systemd_wrapper_debug|bool -%}
+set -x
+{% endif -%}
+add_date() {
+  echo "$(date) $@"
+}
+
+# Set up script logging for debugging purpose.
+# It will be taken care of by logrotate since there is the .log
+# suffix.
+exec 3>&1 4>&2
+trap 'exec 2>&4 1>&3' 0 1 2 3
+exec 1>>/var/log/neutron/kill-script.log 2>&1
+
+SIG=$1
+PID=$2
+NETNS=$(ip netns identify ${PID})
+
+if [ "x${NETNS}" == "x" ]; then
+  add_date "No network namespace detected, exiting"
+  exit 1
+fi
+
+{% if tripleo_systemd_wrapper_container_cli == 'podman' %}
+CLI="nsenter --net=/run/netns/${NETNS} --preserve-credentials -m -t 1 podman"
+{% elif tripleo_systemd_wrapper_container_cli == 'docker' %}
+{% if tripleo_systemd_wrapper_docker_additional_sockets and tripleo_systemd_wrapper_docker_additional_sockets|length > 0-%}
+export DOCKER_HOST=unix://{{ tripleo_systemd_wrapper_docker_additional_sockets[0] }}
+{% endif -%}
+CLI='docker'
+{% else %}
+CLI='echo noop'
+{% endif %}
+
+kill_container() {
+  add_date "Stopping container $1 ($2)"
+  $CLI stop $2
+  add_date "Deleting container $1 ($2)"
+  $CLI rm $2
+}
+
+hup_container() {
+  add_date "Sending HUP signal to $1 ($2)"
+  $CLI kill --signal HUP $2
+}
+
+{% raw -%}
+if [ -f /proc/$PID/cgroup ]; then
+  # Get container ID based on process cgroups
+  CT_ID=$(awk 'BEGIN {FS="[-.]"} /name=/{print $3}' /proc/$PID/cgroup)
+  CT_NAME=$($CLI inspect -f '{{.Name}}' $CT_ID)
+
+  case $SIG in
+    HUP)
+      hup_container $CT_NAME $CT_ID
+      ;;
+    9)
+      kill_container $CT_NAME $CT_ID
+      ;;
+    *)
+      add_date "Unknown action ${SIG} for ${$CT_NAME} ${CT_ID}"
+      exit 1
+      ;;
+  esac
+
+else
+  add_date "No such PID: ${PID}"
+  exit 1
+fi
+{% endraw %}

tripleo_ansible/roles/tripleo-systemd-wrapper/templates/service_sync.j2

@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+{% if tripleo_systemd_wrapper_debug %}
+set -x
+{% endif %}
+
+function start_service {
+    local NETNS=$1
+    shift
+    local NAME="{{ tripleo_systemd_wrapper_service_name }}-${NETNS}"
+    local CLI='{{ tripleo_systemd_wrapper_container_cli }}'
+    local CMD="{{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/command"
+    local CONTAINER_CMD="ip netns exec ${NETNS} ${CMD}"
+    {% if tripleo_systemd_wrapper_container_cli == 'podman' %}
+    local LOGGING="--log-driver k8s-file --log-opt path=/var/log/containers/stdouts/${NAME}.log"
+    {% else %}
+    local LOGGING=''
+    {% endif %}
+
+    $CLI stop $NAME &> /dev/null || true
+    $CLI rm -f $NAME &> /dev/null || true
+    $CLI run --detach \
+        -v "{{ tripleo_systemd_wrapper_config_bind_mount }}" \
+        -v "/run/netns:/run/netns:shared" \
+        -v "{{ tripleo_systemd_wrapper_service_dir }}:{{ tripleo_systemd_wrapper_service_dir }}:z,shared" \
+        -v "/dev/log:/dev/log" $LOGGING \
+        --net host \
+        --pid host \
+        --privileged \
+        --rm=true \
+        -u root \
+        --name $NAME \
+        {{ tripleo_systemd_wrapper_image_name }} \
+        $CONTAINER_CMD $@
+}
+
+exec {lock_fd}>/var/lock/{{ tripleo_systemd_wrapper_service_name }}-processes.lock || exit 1
+# In case service_wrapper script already locked the commands, we just wait.
+flock "$lock_fd"
+
+IFS=$'\n'
+for LINE in $(cat {{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/processes); do
+    NETNS=$(echo $LINE | awk '{ print $1 }')
+    IFS=$' ' ARGS=$(echo $LINE | sed -e "s|$NETNS ||" | xargs)
+    # TODO(emilien) investigate if we should rather run docker/podman ps instead of ps on the host
+    if ! ps -e -o pid,command | grep "$(echo $NETNS | sed 's|^[^-]*\-||')" | grep -v grep &> /dev/null; then
+        start_service $NETNS $ARGS
+    fi
+done
+# truncate the file so we don't start them again
+:> {{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/processes
+
+flock -u "$lock_fd"
+

tripleo_ansible/roles/tripleo-systemd-wrapper/templates/service_wrapper.j2

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+{% if tripleo_systemd_wrapper_debug %}
+set -x
+{% endif %}
+
+ARGS="$@"
+NETNS=$(ip netns identify)
+
+exec {lock_fd}>/var/lock/{{ tripleo_systemd_wrapper_service_name }}-processes.lock || exit 1
+# In case service_sync script already locked the commands, we just wait.
+flock "$lock_fd"
+
+echo "$NETNS $ARGS" >> {{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/processes
+# only update the timestamp which fires systemd if there was an update
+date > {{ tripleo_systemd_wrapper_service_dir }}/{{ tripleo_systemd_wrapper_service_name }}/processes-timestamp
+
+flock -u "$lock_fd"

zuul.d/molecule.yaml

@@ -41,6 +41,7 @@
       - tripleo-ansible-centos-7-molecule-tripleo-container-manage
       - tripleo-ansible-centos-7-molecule-tripleo-modules
       - tripleo-ansible-centos-7-molecule-tripleo-puppet-cache
+      - tripleo-ansible-centos-7-molecule-tripleo-systemd-wrapper
     gate:
       jobs:
       - tripleo-ansible-centos-7-molecule-aide
@@ -82,6 +83,7 @@
       - tripleo-ansible-centos-7-molecule-tripleo-container-manage
       - tripleo-ansible-centos-7-molecule-tripleo-modules
       - tripleo-ansible-centos-7-molecule-tripleo-puppet-cache
+      - tripleo-ansible-centos-7-molecule-tripleo-systemd-wrapper
     name: tripleo-ansible-molecule-jobs
 - job:
     files:
@@ -371,3 +373,10 @@
     parent: tripleo-ansible-centos-7-base
     vars:
       tripleo_role_name: tripleo-puppet-cache
+- job:
+    files:
+    - ^tripleo_ansible/roles/tripleo-systemd-wrapper/.*
+    name: tripleo-ansible-centos-7-molecule-tripleo-systemd-wrapper
+    parent: tripleo-ansible-centos-7-base
+    vars:
+      tripleo_role_name: tripleo-systemd-wrapper