implement external plugin mechanism for grenade

Implement an external plugin mechanism for grenade in the spirit of
devstack plugins. This assumes that external plugins will live in the
devstack/upgrade tree in their respective repository.

External plugins are enabled by adding

enable_grenade_plugin <name> <giturl> [branch]

to ``pluginrc`` at the top of the grenade directory. (``localrc``
could not be used due to timing issues on parsing)

a new ``devstack_localrc <base|target> ...`` is also added to grenade
that allows settings files to add content to base and target devstack
localrcs. This is quite critical for out of tree support, because for
upgrade testing there will often be a need to add services to the
localrc.

Partially tested on the existing Heat plugin implemented in
I0847004a8ac023a113cabaee46896abc823a01da, though that doesn't yet
fully pass due to conflicts in it's upgrade process.

Include documentation for out of tree best practices.

Change-Id: I02a777077d40408766204b05ed284fe98efbce8e

PLUGINS.rst

@@ -174,3 +174,50 @@ covered other places. The value in the resource scripts is these side
 effects. Actual VMs running, actual iscsi targets running, etc. And
 ensuring these things are not disrupted when the control plane is
 shifted out from under them.
+
+Out of Tree Plugins
+===================
+
+A grenade plugin can be hosted out of tree in a project tree, similar
+to external devstack plugins. There are a few subtle differences when
+this happens.
+
+The plugin structure will live under ``$project/devstack/upgrade/``
+directory.
+
+The plugin is enabled by adding::
+
+  enable_grenade_plugin <$project> <giturl> [branch]
+
+To ``pluginrc`` in the ``GRENADE_DIR``. An additional rc file was
+required due to sequencing of when plugin functions become available.
+
+Changing Devstack Localrc
+-------------------------
+
+There is also a mechanism that allows a ``settings`` file change the
+devstack localrc files with the ``devstack_localrc`` function.
+
+::
+   devstack_localrc <base|target> arbitrary stuff to add
+
+Which will take all the rest of the stuff on that line and add it to
+the localrc for either the base or target devstack.
+
+Example settings
+----------------
+
+The following is a reasonable example ``settings`` for out of tree
+plugin::
+
+  register_project_for_upgrade heat
+  register_db_to_save heat
+  devstack_localrc base enable_service h-api h-api-cfn h-api-cw h-eng heat
+  devstack_localrc target enable_service h-api h-api-cfn h-api-cw h-eng heat
+
+This registers the project for upgrade, symbolicly enables the heat
+database for dump during upgrade, and adds the heat services into the
+service list for base and target.
+
+It's expected that most ``settings`` files for out of tree plugins
+will need equivalent lines.

grenade.sh

@@ -194,15 +194,17 @@ init_grenade_db
 # we are starting with.
 ENABLED_SERVICES=$(source $BASE_DEVSTACK_DIR/stackrc; echo $ENABLED_SERVICES)
 
+# Fetch all the grenade plugins which were registered in ``pluginrc``
+# via the ``enable_grenade_plugin`` stanza. This must be done before
+# settings are loaded, but has to be this late to give access to all
+# the devstack functions.
+fetch_grenade_plugins
+
 # Load the ``settings`` files for all the in tree ``projects/``. This
 # registers all the projects in order that we're going to be upgrading
 # when the time is right.
-#
-# TODO(sdague): this will be where we ``enable_plugins`` when we start
-# supporting out of tree content for grenade.
 load_settings
 
-
 # Run the base install of the environment
 if [[ "$RUN_BASE" == "True" ]]; then
 

inc/plugin

@@ -27,6 +27,7 @@
 GRENADE_DB=$SAVE_DIR/grenade_db.ini
 
 function load_settings {
+    # In tree plugins
     local in_tree_plugins=$RC_DIR/projects
     for dir in $in_tree_plugins/*; do
         local settings=$dir/settings
@@ -34,6 +35,26 @@ function load_settings {
             source $settings
         fi
     done
+
+    # External plugins
+    local plugins="${GRENADE_PLUGINS}"
+    local plugin
+
+    # short circuit if nothing to do
+    if [[ -z $plugins ]]; then
+        return
+    fi
+
+    echo "Loading plugin settings"
+    for plugin in ${plugins//,/ }; do
+        local dir=${GITDIR[$plugin]}
+        # source any known settings
+        if [[ -f $dir/devstack/upgrade/settings ]]; then
+            echo "Loading settings for $plugin from $dir/devstack/upgrade/settings"
+            source $dir/devstack/upgrade/settings
+        fi
+    done
+
     export UPGRADE_PROJECTS
 }
 
@@ -103,3 +124,58 @@ function resource_get {
     local value=$(iniget $GRENADE_DB $project $key)
     echo $value
 }
+
+# External plugin interface for grenade
+
+function enable_grenade_plugin {
+    local name=$1
+    local url=$2
+    local branch=${3:-$TARGET_DEVSTACK_BRANCH}
+    # the following allows us to set the PLUGIN_DIR for the gate to
+    # zuul checked out locations.
+    local plugin_dir=${PLUGIN_DIR:-$STACK_ROOT/plugins}
+    GRENADE_PLUGINS+=",$name"
+    # NOTE(sdague): we're intentional namespace colliding with
+    # devstack to reuse devstack architecture. I don't think this is
+    # going to get us in trouble, but it might. So here be dragons, or
+    # at least small fierce lizards of unknown provenance.
+    GITREPO[$name]=$url
+    GITDIR[$name]=$plugin_dir/$name
+    GITBRANCH[$name]=$branch
+}
+
+function devstack_localrc {
+    local settings_file=$(caller | awk '{print $2}')
+    local where=$1
+    shift
+    case $where in
+        base)
+            echo "# added by $settings_file" >> $BASE_DEVSTACK_DIR/localrc
+            echo "$@" >> $BASE_DEVSTACK_DIR/localrc
+            ;;
+        target)
+            echo "# added by $settings_file" >> $TARGET_DEVSTACK_DIR/localrc
+            echo "$@" >> $TARGET_DEVSTACK_DIR/localrc
+            ;;
+    esac
+}
+
+function fetch_grenade_plugins {
+    local plugins="${GRENADE_PLUGINS}"
+    local plugin
+
+    # short circuit if nothing to do
+    if [[ -z $plugins ]]; then
+        return
+    fi
+
+    echo "Fetching Grenade plugins"
+    for plugin in ${plugins//,/ }; do
+        git_clone_by_name $plugin
+    done
+}
+
+# this allows us to expose this late
+if [[ -f ${GRENADE_DIR}/pluginrc ]]; then
+    source ${GRENADE_DIR}/pluginrc
+fi