Support reloading of gerrit.config

This change introduces a mechanism to allow reloading of the
@GerritServerConfig. To execute a reload of gerrit.config, the SSH
command "reload-config" is provided.

Challenges and implementation:
------------------------------
Most callers that inject @GerritServerConfig are @Singletons. These
@Singletons will only parse the GerritServerConfig once and then
build and store more suitable datastructures to work from.

To allow these modules to react on config changes, a caller can now
implement GerritConfigListener, which will enable the caller to receive
notifications of future config updates.

The GerritConfigListener provides:

    void configUpdated(ConfigUpdatedEvent event);

The ConfigUpdatedEvent provides the old config and the new one.
Implementing classes of the listener are expected to do three things:

1. Investigate if any of the updated config values are of interest.
The ConfigUpdatedEvent provides helper methods for this purpose:

    public boolean isSectionUpdated(String section)
    public boolean isValueUpdated(String section, String subsection, String name)
    (+ various overloaded versions of these)

2. React to the configuration changes (if any) and apply them

3. Accept or reject the entries of interest:

    public void accept(Set<ConfigKey> entries)
    public void accept(String section)
    public void reject(Set<ConfigKey> entries)
    (+ various overloaded versions of these)

When a section (or specific config keys) are accepted or rejected, the
ConfigUpdatedEvent will build a diff of what changes were accepted and
present it to the administrator that issued a configuration reload.

For instance, in this case, 4 config rows where added(+), 2 removed(-1)
and 1 modified. All but one change were accepted, hence the user would get
this output:

$ ssh $HOST -p 29418 gerrit reload-config

Accepted configuration changes:
- suggest.maxSuggestedReviewers = 10
- suggest.accounts = 50
* addreviewer.maxAllowed = [100 => 1000]
+ sshd.requestLog = true
+ commentlink.changeid.link = #/q/$1
+ commentlink.changeid.match = (I[0-9a-f]{8,40})

Rejected configuration changes:
+ gc.startTime = Fri 10:30

Documentation:
--------------
We start by documenting the config entries that support config reloads
(which number will be increasing over time).

When we support reloading of the majority of config settings, the
documentation can switch to documenting the config entries that does
_not_ support config reloading instead.

Roadmap:
--------
This change will be followed by a couple of changes that introduces
reloadable config support for: suggest-reviewers, sshd-log and
commentlinks.

Other stuff TODO includes:
* REST end point
* Rejection messages (to allow for an explanation on why the value was
not accepted)
* Test cases
* An option to allow displaying the current effective configuration
* A lot of implementations of the GerritConfigListener.

Change-Id: I4bd6f389731af303ef9ba5d1d73f173d869c62e4

Documentation/cmd-index.txt

@@ -169,6 +169,9 @@ link:cmd-plugin-remove.html[gerrit plugin remove]::
 link:cmd-plugin-remove.html[gerrit plugin rm]::
 	Alias for 'gerrit plugin remove'.
 
+link:cmd-reload-config.html[gerrit reload-config]::
+	Apply an updated gerrit.config.
+
 link:cmd-set-account.html[gerrit set-account]::
 	Change an account's settings.
 

Documentation/cmd-reload-config.txt

@@ -0,0 +1,44 @@
+= plugin reload
+
+== NAME
+reload-config - Reloads the gerrit.config.
+
+== SYNOPSIS
+[verse]
+--
+_ssh_ -p <port> <host> _gerrit reload-config_
+  <NAME> ...
+--
+
+== DESCRIPTION
+Reloads the gerrit.config configuration.
+
+Not all configuration value can be picked up by this command. Which config
+sections and values that are supported is documented here:
+link:config-gerrit.html[Configuration]
+
+_The output shows only modified config values that are picked up by Gerrit
+and applied._
+
+If a config entry is added or removed from gerrit.config, but still brings
+no effect due to a matching default value, no output for this entry is shown.
+
+== ACCESS
+* Caller must be a member of the privileged 'Administrators' group.
+
+== SCRIPTING
+This command is intended to be used in scripts.
+
+== EXAMPLES
+Reload the gerrit configuration:
+
+----
+	ssh -p 29418 localhost gerrit reload-config
+----
+
+GERRIT
+------
+Part of link:index.html[Gerrit Code Review]
+
+SEARCHBOX
+---------

java/com/google/gerrit/server/config/ConfigKey.java

@@ -0,0 +1,46 @@
+// Copyright (C) 2018 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+package com.google.gerrit.server.config;
+
+import com.google.auto.value.AutoValue;
+import com.google.gerrit.common.Nullable;
+
+@AutoValue
+public abstract class ConfigKey {
+  public abstract String section();
+
+  @Nullable
+  public abstract String subsection();
+
+  public abstract String name();
+
+  public static ConfigKey create(String section, String subsection, String name) {
+    return new AutoValue_ConfigKey(section, subsection, name);
+  }
+
+  public static ConfigKey create(String section, String name) {
+    return new AutoValue_ConfigKey(section, null, name);
+  }
+
+  @Override
+  public String toString() {
+    StringBuilder sb = new StringBuilder();
+    sb.append(section()).append(".");
+    if (subsection() != null) {
+      sb.append(subsection()).append(".");
+    }
+    sb.append(name());
+    return sb.toString();
+  }
+}

java/com/google/gerrit/server/config/ConfigUpdatedEvent.java

@@ -0,0 +1,214 @@
+// Copyright (C) 2018 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+package com.google.gerrit.server.config;
+
+import com.google.common.collect.ImmutableList;
+import java.util.Collections;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Objects;
+import java.util.Set;
+import org.apache.commons.lang.StringUtils;
+import org.eclipse.jgit.lib.Config;
+
+/**
+ * This event is produced by {@link GerritServerConfigReloader} and forwarded to callers
+ * implementing {@link GerritConfigListener}.
+ *
+ * <p>The event intends to:
+ *
+ * <p>1. Help the callers figure out if any action should be taken, depending on which entries are
+ * updated in gerrit.config.
+ *
+ * <p>2. Provide the callers with a mechanism to accept/reject the entries of interest: @see
+ * accept(Set<ConfigKey> entries), @see accept(String section), @see reject(Set<ConfigKey> entries)
+ * (+ various overloaded versions of these)
+ */
+public class ConfigUpdatedEvent {
+  private final Config oldConfig;
+  private final Config newConfig;
+
+  public ConfigUpdatedEvent(Config oldConfig, Config newConfig) {
+    this.oldConfig = oldConfig;
+    this.newConfig = newConfig;
+  }
+
+  public Config getOldConfig() {
+    return this.oldConfig;
+  }
+
+  public Config getNewConfig() {
+    return this.newConfig;
+  }
+
+  public Update accept(ConfigKey entry) {
+    return accept(Collections.singleton(entry));
+  }
+
+  public Update accept(Set<ConfigKey> entries) {
+    return createUpdate(entries, UpdateResult.APPLIED);
+  }
+
+  public Update accept(String section) {
+    Set<ConfigKey> entries = getEntriesFromSection(oldConfig, section);
+    entries.addAll(getEntriesFromSection(newConfig, section));
+    return createUpdate(entries, UpdateResult.APPLIED);
+  }
+
+  public Update reject(Set<ConfigKey> entries) {
+    return createUpdate(entries, UpdateResult.REJECTED);
+  }
+
+  private static Set<ConfigKey> getEntriesFromSection(Config config, String section) {
+    Set<ConfigKey> res = new LinkedHashSet<>();
+    for (String name : config.getNames(section, true)) {
+      res.add(ConfigKey.create(section, name));
+    }
+    for (String sub : config.getSubsections(section)) {
+      for (String name : config.getNames(section, sub, true)) {
+        res.add(ConfigKey.create(section, sub, name));
+      }
+    }
+    return res;
+  }
+
+  private Update createUpdate(Set<ConfigKey> entries, UpdateResult updateResult) {
+    Update update = new Update(updateResult);
+    entries
+        .stream()
+        .filter(this::isValueUpdated)
+        .forEach(
+            key -> {
+              update.addConfigUpdate(
+                  new ConfigUpdateEntry(
+                      key,
+                      oldConfig.getString(key.section(), key.subsection(), key.name()),
+                      newConfig.getString(key.section(), key.subsection(), key.name())));
+            });
+    return update;
+  }
+
+  public boolean isSectionUpdated(String section) {
+    Set<ConfigKey> entries = getEntriesFromSection(oldConfig, section);
+    entries.addAll(getEntriesFromSection(newConfig, section));
+    return isEntriesUpdated(entries);
+  }
+
+  public boolean isValueUpdated(String section, String subsection, String name) {
+    return !Objects.equals(
+        oldConfig.getString(section, subsection, name),
+        newConfig.getString(section, subsection, name));
+  }
+
+  public boolean isValueUpdated(ConfigKey key) {
+    return isValueUpdated(key.section(), key.subsection(), key.name());
+  }
+
+  public boolean isValueUpdated(String section, String name) {
+    return isValueUpdated(section, null, name);
+  }
+
+  public boolean isEntriesUpdated(Set<ConfigKey> entries) {
+    for (ConfigKey entry : entries) {
+      if (isValueUpdated(entry.section(), entry.subsection(), entry.name())) {
+        return true;
+      }
+    }
+    return false;
+  }
+
+  public enum UpdateResult {
+    APPLIED,
+    REJECTED;
+
+    @Override
+    public String toString() {
+      return StringUtils.capitalize(name().toLowerCase());
+    }
+  }
+
+  /**
+   * One Accepted/Rejected Update have one or more config updates (ConfigUpdateEntry) tied to it.
+   */
+  public static class Update {
+    private UpdateResult result;
+    private final Set<ConfigUpdateEntry> configUpdates;
+
+    public Update(UpdateResult result) {
+      this.configUpdates = new LinkedHashSet<>();
+      this.result = result;
+    }
+
+    public UpdateResult getResult() {
+      return result;
+    }
+
+    public List<ConfigUpdateEntry> getConfigUpdates() {
+      return ImmutableList.copyOf(configUpdates);
+    }
+
+    public void addConfigUpdate(ConfigUpdateEntry entry) {
+      this.configUpdates.add(entry);
+    }
+  }
+
+  public enum ConfigEntryType {
+    ADDED,
+    REMOVED,
+    MODIFIED,
+    UNMODIFIED
+  }
+
+  public static class ConfigUpdateEntry {
+    public final ConfigKey key;
+    public final String oldVal;
+    public final String newVal;
+
+    public ConfigUpdateEntry(ConfigKey key, String oldVal, String newVal) {
+      this.key = key;
+      this.oldVal = oldVal;
+      this.newVal = newVal;
+    }
+
+    /** Note: The toString() is used to format the output from @see ReloadConfig. */
+    @Override
+    public String toString() {
+      switch (getUpdateType()) {
+        case ADDED:
+          return String.format("+ %s = %s", key, newVal);
+        case MODIFIED:
+          return String.format("* %s = [%s => %s]", key, oldVal, newVal);
+        case REMOVED:
+          return String.format("- %s = %s", key, oldVal);
+        case UNMODIFIED:
+          return String.format("  %s = %s", key, newVal);
+        default:
+          throw new IllegalStateException("Unexpected UpdateType: " + getUpdateType().name());
+      }
+    }
+
+    public ConfigEntryType getUpdateType() {
+      if (oldVal == null && newVal != null) {
+        return ConfigEntryType.ADDED;
+      }
+      if (oldVal != null && newVal == null) {
+        return ConfigEntryType.REMOVED;
+      }
+      if (Objects.equals(oldVal, newVal)) {
+        return ConfigEntryType.UNMODIFIED;
+      }
+      return ConfigEntryType.MODIFIED;
+    }
+  }
+}

java/com/google/gerrit/server/config/GerritConfigListener.java

@@ -0,0 +1,28 @@
+// Copyright (C) 2018 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.config;
+
+import com.google.gerrit.extensions.annotations.ExtensionPoint;
+import java.util.EventListener;
+import java.util.List;
+
+/**
+ * Implementations of the GerritConfigListener interface expects to react GerritServerConfig
+ * updates. @see ConfigUpdatedEvent.
+ */
+@ExtensionPoint
+public interface GerritConfigListener extends EventListener {
+  List<ConfigUpdatedEvent.Update> configUpdated(ConfigUpdatedEvent event);
+}

java/com/google/gerrit/server/config/GerritGlobalModule.java

@@ -284,6 +284,8 @@ public class GerritGlobalModule extends FactoryModule {
     bind(TransferConfig.class);
 
     bind(GcConfig.class);
+    DynamicSet.setOf(binder(), GerritConfigListener.class);
+
     bind(ChangeCleanupConfig.class);
     bind(AccountDeactivator.class);
 

java/com/google/gerrit/server/config/GerritServerConfigModule.java

@@ -76,8 +76,7 @@ public class GerritServerConfigModule extends AbstractModule {
     bind(TrackingFooters.class).toProvider(TrackingFootersProvider.class).in(SINGLETON);
     bind(Config.class)
         .annotatedWith(GerritServerConfig.class)
-        .toProvider(GerritServerConfigProvider.class)
-        .in(SINGLETON);
+        .toProvider(GerritServerConfigProvider.class);
     bind(SecureStore.class).toProvider(SecureStoreProvider.class).in(SINGLETON);
   }
 }

java/com/google/gerrit/server/config/GerritServerConfigProvider.java

@@ -22,6 +22,7 @@ import com.google.gerrit.server.securestore.SecureStore;
 import com.google.inject.Inject;
 import com.google.inject.Provider;
 import com.google.inject.ProvisionException;
+import com.google.inject.Singleton;
 import java.io.IOException;
 import java.nio.file.Path;
 import java.util.ArrayList;
@@ -36,25 +37,45 @@ import org.slf4j.LoggerFactory;
 /**
  * Provides {@link Config} annotated with {@link GerritServerConfig}.
  *
- * <p>Note that this class is not a singleton, so the few callers that need a reloaded-on-demand
- * config can inject a {@code GerritServerConfigProvider}. However, most callers won't need this,
- * and will just inject {@code @GerritServerConfig Config} directly, which is bound as a singleton
- * in {@link GerritServerConfigModule}.
+ * <p>To react on config updates, the caller should implement @see GerritConfigListener.
+ *
+ * <p>The few callers that need a reloaded-on-demand config can inject a {@code
+ * GerritServerConfigProvider} and request the lastest config with fetchLatestConfig().
  */
+@Singleton
 public class GerritServerConfigProvider implements Provider<Config> {
   private static final Logger log = LoggerFactory.getLogger(GerritServerConfigProvider.class);
 
   private final SitePaths site;
   private final SecureStore secureStore;
 
+  private final Object lock = new Object();
+
+  private GerritConfig gerritConfig;
+
   @Inject
   GerritServerConfigProvider(SitePaths site, SecureStore secureStore) {
     this.site = site;
     this.secureStore = secureStore;
+    this.gerritConfig = loadConfig();
   }
 
   @Override
   public Config get() {
+    synchronized (lock) {
+      return gerritConfig;
+    }
+  }
+
+  protected ConfigUpdatedEvent updateConfig() {
+    synchronized (lock) {
+      Config oldConfig = gerritConfig;
+      gerritConfig = loadConfig();
+      return new ConfigUpdatedEvent(oldConfig, gerritConfig);
+    }
+  }
+
+  public GerritConfig loadConfig() {
     FileBasedConfig baseConfig = loadConfig(null, site.gerrit_config);
     if (!baseConfig.getFile().exists()) {
       log.info("No " + site.gerrit_config.toAbsolutePath() + "; assuming defaults");

java/com/google/gerrit/server/config/GerritServerConfigReloader.java

@@ -0,0 +1,58 @@
+// Copyright (C) 2018 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.server.config;
+
+import com.google.gerrit.extensions.registration.DynamicSet;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
+import java.util.ArrayList;
+import java.util.List;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Issues a configuration reload from the GerritServerConfigProvider and notify all listeners. */
+@Singleton
+public class GerritServerConfigReloader {
+  private static final Logger log = LoggerFactory.getLogger(GerritServerConfigReloader.class);
+
+  private final GerritServerConfigProvider configProvider;
+  private final DynamicSet<GerritConfigListener> configListeners;
+
+  @Inject
+  GerritServerConfigReloader(
+      GerritServerConfigProvider configProvider, DynamicSet<GerritConfigListener> configListeners) {
+    this.configProvider = configProvider;
+    this.configListeners = configListeners;
+  }
+
+  /**
+   * Reloads the Gerrit Server Configuration from disk. Synchronized to ensure that one issued
+   * reload is fully completed before a new one starts.
+   */
+  public List<ConfigUpdatedEvent.Update> reloadConfig() {
+    log.info("Starting server configuration reload");
+    List<ConfigUpdatedEvent.Update> updates = fireUpdatedConfigEvent(configProvider.updateConfig());
+    log.info("Server configuration reload completed succesfully");
+    return updates;
+  }
+
+  public List<ConfigUpdatedEvent.Update> fireUpdatedConfigEvent(ConfigUpdatedEvent event) {
+    ArrayList<ConfigUpdatedEvent.Update> result = new ArrayList<>();
+    for (GerritConfigListener configListener : configListeners) {
+      result.addAll(configListener.configUpdated(event));
+    }
+    return result;
+  }
+}

java/com/google/gerrit/sshd/commands/DefaultCommandModule.java

@@ -53,6 +53,7 @@ public class DefaultCommandModule extends CommandModule {
     command(gerrit, ListGroupsCommand.class);
     command(gerrit, LsUserRefs.class);
     command(gerrit, Query.class);
+    command(gerrit, ReloadConfig.class);
     command(gerrit, ShowCaches.class);
     command(gerrit, ShowConnections.class);
     command(gerrit, ShowQueue.class);

java/com/google/gerrit/sshd/commands/ReloadConfig.java

@@ -0,0 +1,70 @@
+// Copyright (C) 2018 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.sshd.commands;
+
+import static com.google.gerrit.sshd.CommandMetaData.Mode.MASTER_OR_SLAVE;
+
+import com.google.gerrit.common.data.GlobalCapability;
+import com.google.gerrit.extensions.annotations.RequiresCapability;
+import com.google.gerrit.server.config.ConfigUpdatedEvent;
+import com.google.gerrit.server.config.ConfigUpdatedEvent.UpdateResult;
+import com.google.gerrit.server.config.GerritServerConfigReloader;
+import com.google.gerrit.sshd.CommandMetaData;
+import com.google.gerrit.sshd.SshCommand;
+import com.google.inject.Inject;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/** Issues a reload of gerrit.config. */
+@RequiresCapability(GlobalCapability.ADMINISTRATE_SERVER)
+@CommandMetaData(
+  name = "reload-config",
+  description = "Reloads the Gerrit configuration",
+  runsAt = MASTER_OR_SLAVE
+)
+public class ReloadConfig extends SshCommand {
+
+  @Inject private GerritServerConfigReloader gerritServerConfigReloader;
+
+  @Override
+  protected void run() throws Failure {
+    List<ConfigUpdatedEvent.Update> updates = gerritServerConfigReloader.reloadConfig();
+    if (updates.isEmpty()) {
+      stdout.println("No config entries updated!");
+      return;
+    }
+
+    // Print out UpdateResult.{ACCEPTED|REJECTED} entries grouped by their type
+    for (UpdateResult updateResult : UpdateResult.values()) {
+      List<ConfigUpdatedEvent.Update> filteredUpdates = filterUpdates(updates, updateResult);
+      if (filteredUpdates.isEmpty()) {
+        continue;
+      }
+      stdout.println(updateResult.toString() + " configuration changes:");
+      filteredUpdates
+          .stream()
+          .flatMap(update -> update.getConfigUpdates().stream())
+          .forEach(cfgEntry -> stdout.println(cfgEntry.toString()));
+    }
+  }
+
+  public static List<ConfigUpdatedEvent.Update> filterUpdates(
+      List<ConfigUpdatedEvent.Update> updates, UpdateResult result) {
+    return updates
+        .stream()
+        .filter(update -> update.getResult() == result)
+        .collect(Collectors.toList());
+  }
+}

javatests/com/google/gerrit/acceptance/ssh/SshCommandsIT.java

@@ -54,6 +54,7 @@ public class SshCommandsIT extends AbstractDaemonTest {
           "ls-projects",
           "ls-user-refs",
           "plugin",
+          "reload-config",
           "show-caches",
           "show-connections",
           "show-queue",