Generate plugin registry based on releases repo

The plugin registry is maintained manually, so it tends to be out-of-date.
The deliverble files in openstack/releases repository provide information
on repositories and bug trackers. It sounds reasonable to me to generate
the plugin registry based on infomration in the openstack/releases repo.

The script added picks up deliverables marked as horizon-plugin.
mistral-dashboard and networking-bgpvpn dashboard are released in
different ways, so they are handled specially in the script.

How to run:
  ./tools/list-horizon-plugins.py \
      --csv-file plugin-registry.csv \
      ../releases ussuri

Note that this commit does not autogenerate the plugin registry.
The script still needs to be run manually.
Further automation would be the next step.

Note that bug trackers of senlin-dashboard and trove-dashboard in
the releases repo looks out-of-date. I will submit a fix for them soon.

The downside of the automation is that we cannot list non-official
horizon plugins but I believe it is a good compromise.
The only non-offical plugin listed now is the group-based-policy UI,
but it is questionable whether it is still maintained well.

Change-Id: Id56be13fb93c6dc123e7cfb1c906357cdf59732b

doc/source/install/plugin-registry.rst

@@ -4,98 +4,18 @@
 Plugin Registry
 ===============
 
+This is a list of horizon plugins which are part of the official
+OpenStack releases.
+
 .. note::
 
    Currently, Horizon plugins are responsible for their own compatibility.
    Check the individual repos for information on support.
 
-.. list-table::
-   :header-rows: 1
+.. The CSV file is generated based the openstack/releases deliverables
+   by tools/list-horizon-plugins.py. If you would like to know the detail
+   on how they are processed, see the script above.
 
-   * - Plugin
-     - Repository
-     - Bug Tracker
-   * - BGPVPN Dashboard
-     - :opendev-repo:`openstack/networking-bgpvpn`
-     - :launchpad:`bgpvpn`
-   * - Blazar Dashboard
-     - :opendev-repo:`openstack/blazar-dashboard`
-     - :launchpad:`blazar`
-   * - Cloudkitty Dashboard
-     - :opendev-repo:`openstack/cloudkitty-dashboard`
-     - :launchpad:`cloudkitty`
-   * - Congress Dashboard
-     - :opendev-repo:`openstack/congress-dashboard`
-     - :launchpad:`congress`
-   * - Designate Dashboard
-     - :opendev-repo:`openstack/designate-dashboard`
-     - :launchpad:`designate-dashboard`
-   * - Group Based Policy UI
-     - :opendev-repo:`x/group-based-policy-ui`
-     - :launchpad:`group-based-policy-ui`
-   * - Freezer Web UI
-     - :opendev-repo:`openstack/freezer-web-ui`
-     - :launchpad:`freezer`
-   * - Heat Dashboard
-     - :opendev-repo:`openstack/heat-dashboard`
-     - :storyboard:`openstack/heat-dashboard`
-   * - Ironic UI
-     - :opendev-repo:`openstack/ironic-ui`
-     - :launchpad:`ironic-ui`
-   * - Karbor Dashboard
-     - :opendev-repo:`openstack/karbor-dashboard`
-     - :launchpad:`karbor-dashboard`
-   * - Magnum UI
-     - :opendev-repo:`openstack/magnum-ui`
-     - :launchpad:`magnum-ui`
-   * - Manila UI
-     - :opendev-repo:`openstack/manila-ui`
-     - :launchpad:`manila-ui`
-   * - Mistral Dashboard
-     - :opendev-repo:`openstack/mistral-dashboard`
-     - :launchpad:`mistral`
-   * - Monasca UI
-     - :opendev-repo:`openstack/monasca-ui`
-     - :launchpad:`monasca`
-   * - Murano Dashboard
-     - :opendev-repo:`openstack/murano-dashboard`
-     - :launchpad:`murano`
-   * - Neutron FWaaS Dashboard
-     - :opendev-repo:`openstack/neutron-fwaas-dashboard`
-     - :launchpad:`neutron-fwaas-dashboard`
-   * - Neutron VPNaaS Dashboard
-     - :opendev-repo:`openstack/neutron-vpnaas-dashboard`
-     - :launchpad:`neutron-vpnaas-dashboard`
-   * - Octavia Dashboard
-     - :opendev-repo:`openstack/octavia-dashboard`
-     - :storyboard:`openstack/octavia-dashboard`
-   * - Sahara Dashboard
-     - :opendev-repo:`openstack/sahara-dashboard`
-     - :storyboard:`openstack/sahara-dashboard`
-   * - Searchlight UI
-     - :opendev-repo:`openstack/searchlight-ui`
-     - :storyboard:`openstack/searchlight-ui`
-   * - Senlin Dashboard
-     - :opendev-repo:`openstack/senlin-dashboard`
-     - :launchpad:`senlin-dashboard`
-   * - Solum Dashboard
-     - :opendev-repo:`openstack/solum-dashboard`
-     - :launchpad:`solum`
-   * - Tacker UI
-     - :opendev-repo:`openstack/tacker-horizon`
-     - :launchpad:`tacker`
-   * - Trove Dashboard
-     - :opendev-repo:`openstack/trove-dashboard`
-     - :launchpad:`trove-dashboard`
-   * - Vitrage Dashboard
-     - :opendev-repo:`openstack/vitrage-dashboard`
-     - :storyboard:`openstack/vitrage-dashboard`
-   * - Watcher Dashboard
-     - :opendev-repo:`openstack/watcher-dashboard`
-     - :launchpad:`watcher-dashboard`
-   * - Zaqar UI
-     - :opendev-repo:`openstack/zaqar-ui`
-     - :launchpad:`zaqar-ui`
-   * - Zun UI
-     - :opendev-repo:`openstack/zun-ui`
-     - :launchpad:`zun-ui`
+.. csv-table::
+   :header-rows: 1
+   :file: ../../../plugin-registry.csv

plugin-registry.csv

@@ -0,0 +1,30 @@
+Plugin,Repository,Bug Tracker
+blazar-dashboard,:opendev-repo:`openstack/blazar-dashboard`,:launchpad:`blazar`
+cloudkitty-dashboard,:opendev-repo:`openstack/cloudkitty-dashboard`,:storyboard:`openstack/cloudkitty-dashboard`
+congress-dashboard,:opendev-repo:`openstack/congress-dashboard`,:launchpad:`congress`
+designate-dashboard,:opendev-repo:`openstack/designate-dashboard`,:launchpad:`designate-dashboard`
+freezer-web-ui,:opendev-repo:`openstack/freezer-web-ui`,:storyboard:`openstack/freezer-web-ui`
+heat-dashboard,:opendev-repo:`openstack/heat-dashboard`,:storyboard:`openstack/heat-dashboard`
+ironic-ui,:opendev-repo:`openstack/ironic-ui`,:storyboard:`openstack/ironic-ui`
+karbor-dashboard,:opendev-repo:`openstack/karbor-dashboard`,:launchpad:`karbor-dashboard`
+magnum-ui,:opendev-repo:`openstack/magnum-ui`,:launchpad:`magnum-ui`
+manila-ui,:opendev-repo:`openstack/manila-ui`,:launchpad:`manila-ui`
+masakari-dashboard,:opendev-repo:`openstack/masakari-dashboard`,:launchpad:`masakari`
+mistral-dashboard,:opendev-repo:`openstack/mistral-dashboard`,:launchpad:`mistral`
+monasca-ui,:opendev-repo:`openstack/monasca-ui`,:launchpad:`monasca`
+murano-dashboard,:opendev-repo:`openstack/murano-dashboard`,:launchpad:`murano`
+networking-bgpvpn,:opendev-repo:`openstack/networking-bgpvpn`,:launchpad:`bgpvpn`
+neutron-fwaas-dashboard,:opendev-repo:`openstack/neutron-fwaas-dashboard`,:launchpad:`neutron-fwaas-dashboard`
+neutron-vpnaas-dashboard,:opendev-repo:`openstack/neutron-vpnaas-dashboard`,:launchpad:`neutron-vpnaas-dashboard`
+octavia-dashboard,:opendev-repo:`openstack/octavia-dashboard`,:storyboard:`openstack/octavia-dashboard`
+qinling-dashboard,:opendev-repo:`openstack/qinling-dashboard`,:storyboard:`openstack/qinling-dashboard`
+sahara-dashboard,:opendev-repo:`openstack/sahara-dashboard`,:storyboard:`openstack/sahara-dashboard`
+searchlight-ui,:opendev-repo:`openstack/searchlight-ui`,:storyboard:`openstack/searchlight-ui`
+senlin-dashboard,:opendev-repo:`openstack/senlin-dashboard`,:launchpad:`senlin-dashboard`
+solum-dashboard,:opendev-repo:`openstack/solum-dashboard`,:launchpad:`solum`
+tacker-horizon,:opendev-repo:`openstack/tacker-horizon`,:launchpad:`tacker`
+trove-dashboard,:opendev-repo:`openstack/trove-dashboard`,:launchpad:`trove-dashboard`
+vitrage-dashboard,:opendev-repo:`openstack/vitrage-dashboard`,:storyboard:`openstack/vitrage-dashboard`
+watcher-dashboard,:opendev-repo:`openstack/watcher-dashboard`,:launchpad:`watcher-dashboard`
+zaqar-ui,:opendev-repo:`openstack/zaqar-ui`,:launchpad:`zaqar-ui`
+zun-ui,:opendev-repo:`openstack/zun-ui`,:launchpad:`zun-ui`

tools/executable_files.txt

@@ -1,7 +1,8 @@
 ./manage.py
 ./node_modules
 ./tools/abandon_old_reviews.sh
+./tools/find_settings.py
 ./tools/gate/integration/post_test_hook.sh
 ./tools/gate/integration/pre_test_hook.sh
+./tools/list-horizon-plugins.py
 ./tools/unit_tests.sh
-./tools/find_settings.py

tools/list-horizon-plugins.py

@@ -0,0 +1,101 @@
+#!/usr/bin/env python3
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import argparse
+import csv
+import glob
+import os
+
+import yaml
+
+
+# The script picks up deliverables whose type is 'horizon-plugin' in the
+# openstack/releases repository. Some horizon plugins are released together
+# with other repositories and we cannot pick them up for such case.
+# EXTRA_PLUGINS is used to declare such horizon plugins.
+# Each entry is a tuple of
+# - the deliverable name in the releases repo, and
+# - the repository name of the dashboard.
+EXTRA_PLUGINS = {
+    'networking-bgpvpn': 'networking-bgpvpn',
+    'mistral': 'mistral-dashboard',
+}
+
+
+def read_plugin_data_from_releases(releases_repo, release):
+    plugins = {}
+    deliverables_glob = '%s/deliverables/%s/*.yaml' % (releases_repo, release)
+    for deliverable in glob.glob(deliverables_glob):
+        with open(deliverable) as f:
+            data = yaml.safe_load(f)
+
+        name = os.path.splitext(os.path.basename(deliverable))[0]
+        if data['type'] == 'horizon-plugin':
+            pass
+        elif name in EXTRA_PLUGINS:
+            name = EXTRA_PLUGINS[name]
+        else:
+            continue
+
+        repos = [repo for repo in data['repository-settings']
+                 if os.path.basename(repo) == name]
+        if not repos:
+            repos = list(data['repository-settings'].keys())[0]
+        data['repository'] = repos[0]
+        plugins[name] = data
+    return plugins
+
+
+def get_plugin_info(name, config):
+    repo = ':opendev-repo:`%s`' % config['repository']
+    if 'storyboard' in config:
+        bug_tracker = ':storyboard:`%s`' % config['storyboard']
+    elif 'launchpad' in config:
+        bug_tracker = ':launchpad:`%s`' % config['launchpad']
+    else:
+        bug_tracker = None
+    return [name, repo, bug_tracker]
+
+
+def write_csv(plugins, csv_file):
+    with open(csv_file, 'w', newline='') as csvfile:
+        csvwriter = csv.writer(csvfile, lineterminator='\n',
+                               quoting=csv.QUOTE_MINIMAL)
+        csvwriter.writerow(['Plugin', 'Repository', 'Bug Tracker'])
+        for name in sorted(plugins):
+            csvwriter.writerow(get_plugin_info(name, plugins[name]))
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        '--csv-file',
+        default='plugin-registry.csv',
+        help='Path to a CSV file which contains the plugin list.'
+    )
+    parser.add_argument(
+        'repo',
+        help='Path to openstack/releases repository cloned to local.'
+    )
+    parser.add_argument(
+        'release',
+        help='Release name like "ussuri'
+    )
+    args = parser.parse_args()
+    plugins = read_plugin_data_from_releases(args.repo, args.release)
+
+    write_csv(plugins, args.csv_file)
+
+
+if __name__ == '__main__':
+    main()