Prepare for dynamic generation of grenade plugin registry

This copies the devstack plugin detection code, to be executed
from a nearly identical periodic proposal job, generating a list
of plugins in the same format.

Change-Id: Ifac7833b03409a4c476b3c9f0f9900987b5ad177

data/grenade-plugins-registry.header

@@ -0,0 +1,24 @@
+..
+  Note to patch submitters: this file is covered by a periodic proposal
+  job.  You should edit the files data/grenade-plugins-registry.footer
+  data/grenade-plugins-registry.header instead of this one.
+
+=========================
+ Grenade Plugin Registry
+=========================
+
+Since we've created the external plugin mechanism, it's gotten used by
+a lot of projects. The following is a list of plugins that currently
+exist. Any project that wishes to list their plugin here is welcomed
+to.
+
+Detected Plugins
+================
+
+The following are plugins that a script has found in the openstack/
+namespace, which includes but is not limited to official OpenStack
+projects.
+
++----------------------------+-------------------------------------------------------------------------+
+|Plugin Name                 |URL                                                                      |
++----------------------------+-------------------------------------------------------------------------+

doc/source/index.rst

@@ -7,6 +7,7 @@ Grenade - an OpenStack Community Upgrade Production
 
    readme
    plugins
+   plugin-registry
    hacking
 
 Code

doc/source/plugin-registry.rst

@@ -0,0 +1,24 @@
+..
+  Note to patch submitters: this file is about to be covered by a
+  periodic proposal job.  You should edit the files
+  data/grenade-plugins-registry.footer and
+  data/grenade-plugins-registry.header instead of this one.
+
+=========================
+ Grenade Plugin Registry
+=========================
+
+Since we've created the external plugin mechanism, it's gotten used by
+a lot of projects. The following will be a list of plugins that currently
+exist.
+
+Detected Plugins
+================
+
+The following are plugins that a script has found in the openstack/
+namespace, which includes but is not limited to official OpenStack
+projects.
+
++----------------------------+-------------------------------------------------------------------------+
+|Plugin Name                 |URL                                                                      |
++----------------------------+-------------------------------------------------------------------------+

tools/generate-grenade-plugins-list.py

@@ -0,0 +1,59 @@
+#! /usr/bin/env python
+
+# Copyright 2016 Hewlett Packard Enterprise Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+# This script is intended to be run as part of a periodic proposal bot
+# job in OpenStack infrastructure.
+#
+# In order to function correctly, the environment in which the
+# script runs must have
+#   * network access to the review.openstack.org Gerrit API
+#     working directory
+#   * network access to https://git.openstack.org/cgit
+
+import json
+import requests
+
+url = 'https://review.openstack.org/projects/'
+
+# This is what a project looks like
+'''
+  "openstack-attic/akanda": {
+    "id": "openstack-attic%2Fakanda",
+    "state": "READ_ONLY"
+  },
+'''
+
+def is_in_openstack_namespace(proj):
+    return proj.startswith('openstack/')
+
+# Rather than returning a 404 for a nonexistent file, cgit delivers a
+# 0-byte response to a GET request.  It also does not provide a
+# Content-Length in a HEAD response, so the way we tell if a file exists
+# is to check the length of the entire GET response body.
+def has_grenade_plugin(proj):
+    r = requests.get("https://git.openstack.org/cgit/%s/plain/devstack/upgrade/upgrade.sh" % proj)
+    if len(r.text) > 0:
+        return True
+    else:
+        False
+
+r = requests.get(url)
+projects = sorted(filter(is_in_openstack_namespace, json.loads(r.text[4:])))
+
+found_plugins = filter(has_grenade_plugin, projects)
+
+for project in found_plugins:
+    print project[10:]

tools/generate-grenade-plugins-list.sh

@@ -0,0 +1,64 @@
+#!/bin/bash -ex
+
+# Copyright 2016 Hewlett Packard Enterprise Development Company, L.P.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+# This script is intended to be run as a periodic proposal bot job
+# in OpenStack infrastructure, though you can run it as a one-off.
+#
+# In order to function correctly, the environment in which the
+# script runs must have
+#   * a writable doc/source directory relative to the current
+#     working directory
+#   AND ( (
+#   * git
+#   * all git repos meant to be searched for plugins cloned and
+#     at the desired level of up-to-datedness
+#   * the environment variable git_dir pointing to the location
+#   * of said git repositories
+#   ) OR (
+#   * network access to the review.openstack.org Gerrit API
+#     working directory
+#   * network access to https://git.openstack.org/cgit
+#   ))
+#
+# If a file named data/grenade-plugins-registry.header or
+# data/grenade-plugins-registry.footer is found relative to the
+# current working directory, it will be prepended or appended to
+# the generated reStructuredText plugins table respectively.
+
+(
+declare -A plugins
+
+if [[ -r data/grenade-plugins-registry.header ]]; then
+    cat data/grenade-plugins-registry.header
+fi
+
+sorted_plugins=$(python tools/generate-grenade-plugins-list.py)
+
+for k in ${sorted_plugins}; do
+    project=${k:0:28}
+    giturl="git://git.openstack.org/openstack/${k:0:26}"
+    printf "|%-28s|%-73s|\n" "${project}" "${giturl}"
+    printf "+----------------------------+-------------------------------------------------------------------------+\n"
+done
+
+if [[ -r data/grenade-plugins-registry.footer ]]; then
+    cat data/grenade-plugins-registry.footer
+fi
+) > doc/source/plugin-registry.rst
+
+if [[ -n ${1} ]]; then
+    cp doc/source/plugin-registry.rst ${1}/doc/source/plugin-registry.rst
+fi