Merge branch 'stable-2.16' into stable-3.0

* stable-2.16:
  Error Prone: Enable and fix OperatorPrecedence
  GerritBaseTests: Activate test logging
  Revert "Enable optional DEBUG level logs for query tests"
  Allow to control the Gerrit log level for running tests from system var
  Revert "Acceptance: set log threshold level for tests"
  Reduce log level for tests to INFO
  Bazel: Update time attribute of file entries in plugin artifact
  Add Jetty connection metrics
  Add additional JGit WindowCache metrics
  Update JGit to 5.1.13.202002110435-r
  BucketedCallback: fix prune() to remove unset sub-metrics from registry
  Modify draft ref updates commits to point to an empty parent

CommentsIT: Adapt to the 3.0 version of Gerrit.

Change-Id: I532d092329bdd94a85cccdf2b34dd150944a6699

Documentation/config-gerrit.txt

@@ -1608,6 +1608,21 @@ Default on JGit is 10 MiB on all platforms.
 +
 Common unit suffixes of 'k', 'm', or 'g' are supported.
 
+[[core.packedGitUseStrongRefs]]core.packedGitUseStrongRefs::
++
+Set to `true` in order to use strong references to reference packfile
+pages cached in the WindowCache. Otherwise SoftReferences are used.
+If this option is set to `false`, the Java garbage collector will
+flush the WindowCache to free memory if the used heap comes close to
+the maximum heap size. This has the advantage that it can quickly
+reclaim memory which was used by the WindowCache but comes at the
+price that the previously cached pack file content needs to be again
+copied from the file system cache to the Gerrit process.
+Setting this option to `true` prevents flushing the WindowCache
+which provides more predictable performance.
++
+Default is `false`.
+
 [[core.deltaBaseCaseLimit]]core.deltaBaseCacheLimit::
 +
 Maximum number of bytes to reserve for caching base objects

Documentation/dev-bazel.txt

@@ -289,7 +289,7 @@ The WAR file will be placed in:
 Debugging tests:
 
 ----
-  bazel test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod  testTarget
+  bazel test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget
 ----
 
 Debug test example:
@@ -365,21 +365,25 @@ If Docker is not available, the Elasticsearch tests will be skipped.
 Note that Bazel currently does not show
 link:https://github.com/bazelbuild/bazel/issues/3476[the skipped tests].
 
-[[debug]]
+[[logging]]
-=== Index Query Tests
+=== Controlling logging level
 
-The `DEBUG` log level can optionally be enabled for the index query tests. That log level applies to
+Per default, logging level is set to `INFO` level for all tests. The `DEBUG`
-both Elasticsearch and Lucene tests.
+log level can be enabled for the tests.
 
-In Eclipse, set `-Ddebug=true` as a VM argument under the Run Configuration's `Arguments` tab.
+In IDE, set `-Dgerrit.logLevel=debug` as a VM argument. With `bazel`, pass
-
+`GERRIT_LOG_LEVEL=debug` environment variable:
-With `bazel`, here is an example for the Lucene `account` test:
 
 ----
-bazel test --jvmopt='-Ddebug=true' \
+  bazel test --test_filter=com.gerrit.server.notedb.ChangeNotesTest \
-javatests/com/google/gerrit/server/query/account:lucene_query_test
+  --test_env=GERRIT_LOG_LEVEL=debug \
+  javatests/com/google/gerrit/server:server_tests
 ----
 
+The log results can be found in:
+`bazel-testlogs/javatests/com/google/gerrit/server/server_tests/test.log`.
+
+
 == Dependencies
 
 Dependency JARs are normally downloaded as needed, but you can
@@ -574,7 +578,6 @@ To use the binary from the Bazel build, you need to use the `run_npm_binary.py`
 wrapper script. For an example, see the use of `crisper` in `tools/bzl/js.bzl`.
 
 
-
 [[RBE]]
 == Google Remote Build Support
 
@@ -609,8 +612,6 @@ bazel test --config=remote \
 ```
 
 
-
-
 GERRIT
 ------
 Part of link:index.html[Gerrit Code Review]

Documentation/metrics.txt

@@ -60,6 +60,15 @@ objects needing finalization.
 
 ==== Jetty
 
+* `http/server/jetty/connections/connections`: The current number of open connections
+* `http/server/jetty/connections/connections_total`: The total number of connections opened
+* `http/server/jetty/connections/connections_duration_max`: The max duration of a connection in ms
+* `http/server/jetty/connections/connections_duration_mean`: The mean duration of a connection in ms
+* `http/server/jetty/connections/connections_duration_stdev`: The standard deviation of the duration of a connection in ms
+* `http/server/jetty/connections/received_messages`: The total number of messages received
+* `http/server/jetty/connections/sent_messages`: The total number of messages sent
+* `http/server/jetty/connections/received_bytes`: Total number of bytes received by tracked connections
+* `http/server/jetty/connections/sent_bytes`: Total number of bytes sent by tracked connections"
 * `http/server/jetty/threadpool/active_threads`: Active threads
 * `http/server/jetty/threadpool/idle_threads`: Idle threads
 * `http/server/jetty/threadpool/reserved_threads`: Reserved threads
@@ -130,6 +139,18 @@ topic submissions that concluded successfully.
 
 * `jgit/block_cache/cache_used`: Bytes of memory retained in JGit block cache.
 * `jgit/block_cache/open_files`: File handles held open by JGit block cache.
+* `avg_load_time` Average time to load a cache entry for JGit block cache.
+* `eviction_count` : Cache evictions for JGit block cache.
+* `eviction_ratio` : Cache eviction ratio for JGit block cache.
+* `hit_count` : Cache hits for JGit block cache.
+* `hit_ratio` : Cache hit ratio for JGit block cache.
+* `load_failure_count` : Failed cache loads for JGit block cache.
+* `load_failure_ratio` : Failed cache load ratio for JGit block cache.
+* `load_success_count` : Successful cache loads for JGit block cache.
+* `miss_count` : Cache misses for JGit block cache.
+* `miss_ratio` : Cache miss ratio for JGit block cache.
+* `cache_used_per_repository` : Bytes of memory retained per repository for the top N repositories
+having most data in the cache. The number N of reported repositories is limited to 1000.
 
 === Git
 

java/com/google/gerrit/acceptance/BUILD

@@ -89,8 +89,6 @@ java_library2(
         "//lib/httpcomponents:httpcore",
         "//lib/jetty:servlet",
         "//lib/jgit/org.eclipse.jgit.junit:junit",
-        "//lib/log:impl-log4j",
-        "//lib/log:log4j",
         "//lib/mockito",
         "//lib/truth",
         "//lib/truth:truth-java8-extension",

java/com/google/gerrit/acceptance/GerritServer.java

@@ -18,13 +18,11 @@ import static com.google.common.base.Preconditions.checkArgument;
 import static com.google.common.base.Preconditions.checkState;
 import static java.lang.annotation.RetentionPolicy.RUNTIME;
 import static java.util.Objects.requireNonNull;
-import static org.apache.log4j.Logger.getLogger;
 
 import com.google.auto.value.AutoValue;
 import com.google.common.base.MoreObjects;
 import com.google.common.base.Strings;
 import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableMap;
 import com.google.gerrit.acceptance.testsuite.account.AccountOperations;
 import com.google.gerrit.acceptance.testsuite.account.AccountOperationsImpl;
 import com.google.gerrit.acceptance.testsuite.group.GroupOperations;
@@ -49,6 +47,7 @@ import com.google.gerrit.server.util.SystemLog;
 import com.google.gerrit.testing.FakeEmailSender;
 import com.google.gerrit.testing.InMemoryRepositoryManager;
 import com.google.gerrit.testing.SshMode;
+import com.google.gerrit.testing.TestLoggingActivator;
 import com.google.inject.AbstractModule;
 import com.google.inject.BindingAnnotation;
 import com.google.inject.Injector;
@@ -74,11 +73,6 @@ import java.util.concurrent.Executors;
 import java.util.concurrent.Future;
 import java.util.concurrent.TimeUnit;
 import java.util.stream.Stream;
-import org.apache.log4j.ConsoleAppender;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
-import org.apache.log4j.Logger;
-import org.apache.log4j.PatternLayout;
 import org.eclipse.jgit.lib.Config;
 import org.eclipse.jgit.lib.RepositoryCache;
 import org.eclipse.jgit.util.FS;
@@ -113,8 +107,7 @@ public class GerritServer implements AutoCloseable {
           null, // @GerritConfig is only valid on methods.
           null, // @GerritConfigs is only valid on methods.
           null, // @GlobalPluginConfig is only valid on methods.
-          null, // @GlobalPluginConfigs is only valid on methods.
+          null); // @GlobalPluginConfigs is only valid on methods.
-          getLogLevelThresholdAnnotation(testDesc));
     }
 
     public static Description forTestMethod(
@@ -136,8 +129,7 @@ public class GerritServer implements AutoCloseable {
           testDesc.getAnnotation(GerritConfig.class),
           testDesc.getAnnotation(GerritConfigs.class),
           testDesc.getAnnotation(GlobalPluginConfig.class),
-          testDesc.getAnnotation(GlobalPluginConfigs.class),
+          testDesc.getAnnotation(GlobalPluginConfigs.class));
-          getLogLevelThresholdAnnotation(testDesc));
     }
 
     private static boolean has(Class<? extends Annotation> annotation, Class<?> clazz) {
@@ -149,14 +141,6 @@ public class GerritServer implements AutoCloseable {
       return false;
     }
 
-    private static Level getLogLevelThresholdAnnotation(org.junit.runner.Description testDesc) {
-      LogThreshold logLevelThreshold = testDesc.getTestClass().getAnnotation(LogThreshold.class);
-      if (logLevelThreshold == null) {
-        return Level.DEBUG;
-      }
-      return Level.toLevel(logLevelThreshold.level());
-    }
-
     abstract org.junit.runner.Description testDescription();
 
     @Nullable
@@ -188,8 +172,6 @@ public class GerritServer implements AutoCloseable {
     @Nullable
     abstract GlobalPluginConfigs pluginConfigs();
 
-    abstract Level logLevelThreshold();
-
     private void checkValidAnnotations() {
       if (configs() != null && config() != null) {
         throw new IllegalStateException("Use either @GerritConfigs or @GerritConfig not both");
@@ -223,46 +205,6 @@ public class GerritServer implements AutoCloseable {
     }
   }
 
-  private static final ImmutableMap<String, Level> LOG_LEVELS =
-      ImmutableMap.<String, Level>builder()
-          .put("com.google.gerrit", Level.DEBUG)
-
-          // Silence non-critical messages from MINA SSHD.
-          .put("org.apache.mina", Level.WARN)
-          .put("org.apache.sshd.common", Level.WARN)
-          .put("org.apache.sshd.server", Level.WARN)
-          .put("org.apache.sshd.common.keyprovider.FileKeyPairProvider", Level.INFO)
-          .put("com.google.gerrit.sshd.GerritServerSession", Level.WARN)
-
-          // Silence non-critical messages from mime-util.
-          .put("eu.medsea.mimeutil", Level.WARN)
-
-          // Silence non-critical messages from openid4java.
-          .put("org.apache.xml", Level.WARN)
-          .put("org.openid4java", Level.WARN)
-          .put("org.openid4java.consumer.ConsumerManager", Level.FATAL)
-          .put("org.openid4java.discovery.Discovery", Level.ERROR)
-          .put("org.openid4java.server.RealmVerifier", Level.ERROR)
-          .put("org.openid4java.message.AuthSuccess", Level.ERROR)
-
-          // Silence non-critical messages from c3p0 (if used).
-          .put("com.mchange.v2.c3p0", Level.WARN)
-          .put("com.mchange.v2.resourcepool", Level.WARN)
-          .put("com.mchange.v2.sql", Level.WARN)
-
-          // Silence non-critical messages from apache.http.
-          .put("org.apache.http", Level.WARN)
-
-          // Silence non-critical messages from Jetty.
-          .put("org.eclipse.jetty", Level.WARN)
-
-          // Silence non-critical messages from JGit.
-          .put("org.eclipse.jgit.transport.PacketLineIn", Level.WARN)
-          .put("org.eclipse.jgit.transport.PacketLineOut", Level.WARN)
-          .put("org.eclipse.jgit.internal.storage.file.FileSnapshot", Level.WARN)
-          .put("org.eclipse.jgit.util.FS", Level.WARN)
-          .build();
-
   private static boolean forceLocalDisk() {
     String value = Strings.nullToEmpty(System.getenv("GERRIT_FORCE_LOCAL_DISK"));
     if (value.isEmpty()) {
@@ -378,7 +320,7 @@ public class GerritServer implements AutoCloseable {
       throws Exception {
     checkArgument(site != null, "site is required (even for in-memory server");
     desc.checkValidAnnotations();
-    configureLogging(desc.logLevelThreshold());
+    TestLoggingActivator.configureLogging();
     CyclicBarrier serverStarted = new CyclicBarrier(2);
     Daemon daemon =
         new Daemon(
@@ -482,25 +424,6 @@ public class GerritServer implements AutoCloseable {
     return new GerritServer(desc, site, createTestInjector(daemon), daemon, daemonService);
   }
 
-  private static void configureLogging(Level threshold) {
-    LogManager.resetConfiguration();
-
-    PatternLayout layout = new PatternLayout();
-    layout.setConversionPattern("%-5p %c %x: %m%n");
-
-    ConsoleAppender dst = new ConsoleAppender();
-    dst.setLayout(layout);
-    dst.setTarget("System.err");
-    dst.setThreshold(threshold);
-    dst.activateOptions();
-
-    Logger root = LogManager.getRootLogger();
-    root.removeAllAppenders();
-    root.addAppender(dst);
-
-    LOG_LEVELS.entrySet().stream().forEach(e -> getLogger(e.getKey()).setLevel(e.getValue()));
-  }
-
   private static void mergeTestConfig(Config cfg) {
     String forceEphemeralPort = String.format("%s:0", getLocalHost().getHostName());
     String url = "http://" + forceEphemeralPort + "/";

java/com/google/gerrit/acceptance/LogThreshold.java

@@ -1,29 +0,0 @@
-// Copyright (C) 2019 The Android Open Source Project
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-package com.google.gerrit.acceptance;
-
-import static java.lang.annotation.ElementType.METHOD;
-import static java.lang.annotation.ElementType.TYPE;
-import static java.lang.annotation.RetentionPolicy.RUNTIME;
-
-import java.lang.annotation.Inherited;
-import java.lang.annotation.Retention;
-import java.lang.annotation.Target;
-
-@Target({TYPE, METHOD})
-@Retention(RUNTIME)
-@Inherited
-public @interface LogThreshold {
-  String level() default "DEBUG";
-}

java/com/google/gerrit/metrics/dropwizard/BucketedCallback.java

@@ -22,6 +22,7 @@ import com.google.common.collect.Maps;
 import com.google.gerrit.metrics.Description;
 import com.google.gerrit.metrics.Field;
 import java.util.Map;
+import java.util.Set;
 import java.util.concurrent.ConcurrentHashMap;
 
 /** Abstract callback metric broken down into buckets. */
@@ -66,7 +67,13 @@ abstract class BucketedCallback<V> implements BucketedMetric {
   }
 
   void doPrune() {
-    cells.entrySet().removeIf(objectValueGaugeEntry -> !objectValueGaugeEntry.getValue().set);
+    Set<Map.Entry<Object, BucketedCallback<V>.ValueGauge>> entries = cells.entrySet();
+    for (Map.Entry<Object, ValueGauge> e : entries) {
+      if (!e.getValue().set) {
+        entries.remove(e);
+        registry.remove(submetric(e.getKey()));
+      }
+    }
   }
 
   void doEndSet() {

java/com/google/gerrit/metrics/proc/JGitMetricModule.java

@@ -14,12 +14,18 @@
 
 package com.google.gerrit.metrics.proc;
 
+import com.google.common.base.Supplier;
+import com.google.gerrit.metrics.CallbackMetric1;
 import com.google.gerrit.metrics.Description;
 import com.google.gerrit.metrics.Description.Units;
+import com.google.gerrit.metrics.Field;
 import com.google.gerrit.metrics.MetricMaker;
+import java.util.Map;
 import org.eclipse.jgit.storage.file.WindowCacheStats;
 
 public class JGitMetricModule extends MetricModule {
+  private static final long MAX_REPO_COUNT = 1000;
+
   @Override
   protected void configure(MetricMaker metrics) {
     metrics.newCallbackMetric(
@@ -28,12 +34,153 @@ public class JGitMetricModule extends MetricModule {
         new Description("Bytes of memory retained in JGit block cache.")
             .setGauge()
             .setUnit(Units.BYTES),
-        WindowCacheStats::getOpenBytes);
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getOpenByteCount();
+          }
+        });
 
     metrics.newCallbackMetric(
         "jgit/block_cache/open_files",
-        Integer.class,
+        Long.class,
         new Description("File handles held open by JGit block cache.").setGauge().setUnit("fds"),
-        WindowCacheStats::getOpenFiles);
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getOpenFileCount();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/avg_load_time",
+        Double.class,
+        new Description("Average time to load a cache entry for JGit block cache.")
+            .setGauge()
+            .setUnit(Units.NANOSECONDS),
+        new Supplier<Double>() {
+          @Override
+          public Double get() {
+            return WindowCacheStats.getStats().getAverageLoadTime();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/eviction_count",
+        Long.class,
+        new Description("Cache evictions for JGit block cache.").setGauge(),
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getEvictionCount();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/eviction_ratio",
+        Double.class,
+        new Description("Cache eviction ratio for JGit block cache.").setGauge(),
+        new Supplier<Double>() {
+          @Override
+          public Double get() {
+            return WindowCacheStats.getStats().getEvictionRatio();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/hit_count",
+        Long.class,
+        new Description("Cache hits for JGit block cache.").setGauge(),
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getHitCount();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/hit_ratio",
+        Double.class,
+        new Description("Cache hit ratio for JGit block cache.").setGauge(),
+        new Supplier<Double>() {
+          @Override
+          public Double get() {
+            return WindowCacheStats.getStats().getHitRatio();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/load_failure_count",
+        Long.class,
+        new Description("Failed cache loads for JGit block cache.").setGauge(),
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getLoadFailureCount();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/load_failure_ratio",
+        Double.class,
+        new Description("Failed cache load ratio for JGit block cache.").setGauge(),
+        new Supplier<Double>() {
+          @Override
+          public Double get() {
+            return WindowCacheStats.getStats().getLoadFailureRatio();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/load_success_count",
+        Long.class,
+        new Description("Successfull cache loads for JGit block cache.").setGauge(),
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getLoadSuccessCount();
+          }
+        });
+    metrics.newCallbackMetric(
+        "jgit/block_cache/miss_count",
+        Long.class,
+        new Description("Cache misses for JGit block cache.").setGauge(),
+        new Supplier<Long>() {
+          @Override
+          public Long get() {
+            return WindowCacheStats.getStats().getMissCount();
+          }
+        });
+
+    metrics.newCallbackMetric(
+        "jgit/block_cache/miss_ratio",
+        Double.class,
+        new Description("Cache miss ratio for JGit block cache.").setGauge(),
+        WindowCacheStats.getStats()::getMissRatio);
+
+    CallbackMetric1<String, Long> repoEnt =
+        metrics.newCallbackMetric(
+            "jgit/block_cache/cache_used_per_repository",
+            Long.class,
+            new Description(
+                    "Bytes of memory retained per repository for the top repositories "
+                        + "having most data in the cache.")
+                .setGauge()
+                .setUnit("byte"),
+            Field.ofString("repository_name"));
+    metrics.newTrigger(
+        repoEnt,
+        () -> {
+          Map<String, Long> cacheMap = WindowCacheStats.getStats().getOpenByteCountPerRepository();
+          if (cacheMap.isEmpty()) {
+            repoEnt.forceCreate("");
+          } else {
+            cacheMap.entrySet().stream()
+                .sorted(Map.Entry.<String, Long>comparingByValue().reversed())
+                .limit(MAX_REPO_COUNT)
+                .forEach(e -> repoEnt.set(e.getKey(), e.getValue()));
+            repoEnt.prune();
+          }
+        });
   }
 }

java/com/google/gerrit/pgm/http/jetty/JettyMetrics.java

@@ -17,6 +17,7 @@ import com.google.common.collect.ImmutableSet;
 import com.google.gerrit.metrics.CallbackMetric;
 import com.google.gerrit.metrics.CallbackMetric0;
 import com.google.gerrit.metrics.Description;
+import com.google.gerrit.metrics.Description.Units;
 import com.google.gerrit.metrics.MetricMaker;
 import com.google.inject.Inject;
 import com.google.inject.Singleton;
@@ -45,27 +46,83 @@ public class JettyMetrics {
         metrics.newCallbackMetric(
             "http/server/jetty/threadpool/idle_threads",
             Integer.class,
-            new Description("Idle threads").setGauge().setUnit("threads"));
+            new Description("Idle threads").setGauge());
     CallbackMetric0<Integer> busyThreads =
         metrics.newCallbackMetric(
             "http/server/jetty/threadpool/active_threads",
             Integer.class,
-            new Description("Active threads").setGauge().setUnit("threads"));
+            new Description("Active threads").setGauge());
     CallbackMetric0<Integer> reservedThreads =
         metrics.newCallbackMetric(
             "http/server/jetty/threadpool/reserved_threads",
             Integer.class,
-            new Description("Reserved threads").setGauge().setUnit("threads"));
+            new Description("Reserved threads").setGauge());
     CallbackMetric0<Integer> queueSize =
         metrics.newCallbackMetric(
             "http/server/jetty/threadpool/queue_size",
             Integer.class,
-            new Description("Queued requests waiting for a thread").setGauge().setUnit("requests"));
+            new Description("Queued requests waiting for a thread").setGauge());
     CallbackMetric0<Boolean> lowOnThreads =
         metrics.newCallbackMetric(
             "http/server/jetty/threadpool/is_low_on_threads",
             Boolean.class,
             new Description("Whether thread pool is low on threads").setGauge());
+    CallbackMetric0<Long> connections =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/connections",
+            Long.class,
+            new Description("The current number of open connections").setGauge());
+    CallbackMetric0<Long> connectionsTotal =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/connections_total",
+            Long.class,
+            new Description("The total number of connections opened").setGauge());
+    CallbackMetric0<Long> connectionDurationMax =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/connections_duration_max",
+            Long.class,
+            new Description("The max duration of a connection")
+                .setGauge()
+                .setUnit(Units.MILLISECONDS));
+    CallbackMetric0<Double> connectionDurationMean =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/connections_duration_mean",
+            Double.class,
+            new Description("The mean duration of a connection")
+                .setGauge()
+                .setUnit(Units.MILLISECONDS));
+    CallbackMetric0<Double> connectionDurationStDev =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/connections_duration_stdev",
+            Double.class,
+            new Description("The standard deviation of the duration of a connection")
+                .setGauge()
+                .setUnit(Units.MILLISECONDS));
+    CallbackMetric0<Long> receivedMessages =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/received_messages",
+            Long.class,
+            new Description("The total number of messages received").setGauge());
+    CallbackMetric0<Long> sentMessages =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/sent_messages",
+            Long.class,
+            new Description("The total number of messages sent").setGauge());
+    CallbackMetric0<Long> receivedBytes =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/received_bytes",
+            Long.class,
+            new Description("Total number of bytes received by tracked connections")
+                .setGauge()
+                .setUnit(Units.BYTES));
+    CallbackMetric0<Long> sentBytes =
+        metrics.newCallbackMetric(
+            "http/server/jetty/connections/sent_bytes",
+            Long.class,
+            new Description("Total number of bytes sent by tracked connections")
+                .setGauge()
+                .setUnit(Units.BYTES));
+
     JettyServer.Metrics jettyMetrics = jetty.getMetrics();
     metrics.newTrigger(
         ImmutableSet.<CallbackMetric<?>>of(
@@ -76,7 +133,16 @@ public class JettyMetrics {
             maxPoolSize,
             poolSize,
             queueSize,
-            lowOnThreads),
+            lowOnThreads,
+            connections,
+            connectionsTotal,
+            connectionDurationMax,
+            connectionDurationMean,
+            connectionDurationStDev,
+            receivedMessages,
+            sentMessages,
+            receivedBytes,
+            sentBytes),
         () -> {
           minPoolSize.set(jettyMetrics.getMinThreads());
           maxPoolSize.set(jettyMetrics.getMaxThreads());
@@ -86,6 +152,15 @@ public class JettyMetrics {
           reservedThreads.set(jettyMetrics.getReservedThreads());
           queueSize.set(jettyMetrics.getQueueSize());
           lowOnThreads.set(jettyMetrics.isLowOnThreads());
+          connections.set(jettyMetrics.getConnections());
+          connectionsTotal.set(jettyMetrics.getConnectionsTotal());
+          connectionDurationMax.set(jettyMetrics.getConnectionDurationMax());
+          connectionDurationMean.set(jettyMetrics.getConnectionDurationMean());
+          connectionDurationStDev.set(jettyMetrics.getConnectionDurationStdDev());
+          receivedMessages.set(jettyMetrics.getReceivedMessages());
+          sentMessages.set(jettyMetrics.getSentMessages());
+          receivedBytes.set(jettyMetrics.getReceivedBytes());
+          sentBytes.set(jettyMetrics.getSentBytes());
         });
   }
 }

java/com/google/gerrit/pgm/http/jetty/JettyServer.java

@@ -43,6 +43,7 @@ import java.util.concurrent.TimeUnit;
 import javax.servlet.DispatcherType;
 import javax.servlet.Filter;
 import org.eclipse.jetty.http.HttpScheme;
+import org.eclipse.jetty.io.ConnectionStatistics;
 import org.eclipse.jetty.jmx.MBeanContainer;
 import org.eclipse.jetty.server.Connector;
 import org.eclipse.jetty.server.ForwardedRequestCustomizer;
@@ -116,9 +117,11 @@ public class JettyServer {
 
   static class Metrics {
     private final QueuedThreadPool threadPool;
+    private ConnectionStatistics connStats;
 
-    Metrics(QueuedThreadPool threadPool) {
+    Metrics(QueuedThreadPool threadPool, ConnectionStatistics connStats) {
       this.threadPool = threadPool;
+      this.connStats = connStats;
     }
 
     public int getIdleThreads() {
@@ -152,12 +155,49 @@ public class JettyServer {
     public boolean isLowOnThreads() {
       return threadPool.isLowOnThreads();
     }
+
+    public long getConnections() {
+      return connStats.getConnections();
+    }
+
+    public long getConnectionsTotal() {
+      return connStats.getConnectionsTotal();
+    }
+
+    public long getConnectionDurationMax() {
+      return connStats.getConnectionDurationMax();
+    }
+
+    public double getConnectionDurationMean() {
+      return connStats.getConnectionDurationMean();
+    }
+
+    public double getConnectionDurationStdDev() {
+      return connStats.getConnectionDurationStdDev();
+    }
+
+    public long getReceivedMessages() {
+      return connStats.getReceivedMessages();
+    }
+
+    public long getSentMessages() {
+      return connStats.getSentMessages();
+    }
+
+    public long getReceivedBytes() {
+      return connStats.getReceivedBytes();
+    }
+
+    public long getSentBytes() {
+      return connStats.getSentBytes();
+    }
   }
 
   private final SitePaths site;
   private final Server httpd;
   private final Metrics metrics;
   private boolean reverseProxy;
+  private ConnectionStatistics connStats;
 
   @Inject
   JettyServer(
@@ -171,7 +211,11 @@ public class JettyServer {
     QueuedThreadPool pool = threadPool(cfg, threadSettingsConfig);
     httpd = new Server(pool);
     httpd.setConnectors(listen(httpd, cfg));
-    metrics = new Metrics(pool);
+    connStats = new ConnectionStatistics();
+    for (Connector connector : httpd.getConnectors()) {
+      connector.addBean(connStats);
+    }
+    metrics = new Metrics(pool, connStats);
 
     Handler app = makeContext(env, cfg);
     if (cfg.getBoolean("httpd", "requestLog", !reverseProxy)) {

java/com/google/gerrit/server/notedb/AbstractChangeUpdate.java

@@ -184,6 +184,14 @@ public abstract class AbstractChangeUpdate {
 
   protected abstract String getRefName();
 
+  protected void setParentCommit(CommitBuilder cb, ObjectId parentCommitId) {
+    if (!parentCommitId.equals(ObjectId.zeroId())) {
+      cb.setParentId(parentCommitId);
+    } else {
+      cb.setParentIds(); // Ref is currently nonexistent, commit has no parents.
+    }
+  }
+
   /**
    * Apply this update to the given inserter.
    *
@@ -216,11 +224,7 @@ public abstract class AbstractChangeUpdate {
     }
     cb.setAuthor(authorIdent);
     cb.setCommitter(new PersonIdent(serverIdent, when));
-    if (!curr.equals(z)) {
+    setParentCommit(cb, curr);
-      cb.setParentId(curr);
-    } else {
-      cb.setParentIds(); // Ref is currently nonexistent, commit has no parents.
-    }
     if (cb.getTreeId() == null) {
       if (curr.equals(z)) {
         cb.setTreeId(emptyTree(ins)); // No parent, assume empty tree.

java/com/google/gerrit/server/notedb/ChangeDraftUpdate.java

@@ -234,6 +234,11 @@ public class ChangeDraftUpdate extends AbstractChangeUpdate {
     return RefNames.refsDraftComments(getId(), accountId);
   }
 
+  @Override
+  protected void setParentCommit(CommitBuilder cb, ObjectId parentCommitId) {
+    cb.setParentIds(); // Draft updates should not keep history of parent commits
+  }
+
   @Override
   public boolean isEmpty() {
     return delete.isEmpty() && put.isEmpty();

java/com/google/gerrit/server/submit/SubmoduleOp.java

@@ -644,7 +644,7 @@ public class SubmoduleOp {
         int newSize = msgbuf.length() + bullet.length() + message.length();
         if (++numMessages > maxCommitMessages
             || newSize > maxCombinedCommitMessageSize
-            || iter.hasNext() && (newSize + ellipsis.length()) > maxCombinedCommitMessageSize) {
+            || (iter.hasNext() && (newSize + ellipsis.length()) > maxCombinedCommitMessageSize)) {
           msgbuf.append(ellipsis);
           break;
         }

java/com/google/gerrit/testing/BUILD

@@ -46,6 +46,8 @@ java_library(
         "//lib/guice:guice-servlet",
         "//lib/jgit/org.eclipse.jgit:jgit",
         "//lib/jgit/org.eclipse.jgit.junit:junit",
+        "//lib/log:impl-log4j",
+        "//lib/log:log4j",
         "//lib/truth",
     ],
 )

java/com/google/gerrit/testing/GerritBaseTests.java

@@ -15,6 +15,7 @@
 package com.google.gerrit.testing;
 
 import com.google.common.base.CharMatcher;
+import org.junit.BeforeClass;
 import org.junit.Ignore;
 import org.junit.Rule;
 import org.junit.rules.ExpectedException;
@@ -25,6 +26,11 @@ public abstract class GerritBaseTests {
   @Rule public ExpectedException exception = ExpectedException.none();
   @Rule public final TestName testName = new TestName();
 
+  @BeforeClass
+  public static void beforeClassTest() {
+    TestLoggingActivator.configureLogging();
+  }
+
   protected String getSanitizedMethodName() {
     String name = testName.getMethodName().toLowerCase();
     name =

java/com/google/gerrit/testing/TestLoggingActivator.java

@@ -0,0 +1,94 @@
+// Copyright (C) 2020 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package com.google.gerrit.testing;
+
+import static org.apache.log4j.Logger.getLogger;
+
+import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableMap;
+import org.apache.log4j.ConsoleAppender;
+import org.apache.log4j.Level;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+import org.apache.log4j.PatternLayout;
+
+public class TestLoggingActivator {
+  private static final ImmutableMap<String, Level> LOG_LEVELS =
+      ImmutableMap.<String, Level>builder()
+          .put("com.google.gerrit", getGerritLogLevel())
+
+          // Silence non-critical messages from MINA SSHD.
+          .put("org.apache.mina", Level.WARN)
+          .put("org.apache.sshd.common", Level.WARN)
+          .put("org.apache.sshd.server", Level.WARN)
+          .put("org.apache.sshd.common.keyprovider.FileKeyPairProvider", Level.INFO)
+          .put("com.google.gerrit.sshd.GerritServerSession", Level.WARN)
+
+          // Silence non-critical messages from mime-util.
+          .put("eu.medsea.mimeutil", Level.WARN)
+
+          // Silence non-critical messages from openid4java.
+          .put("org.apache.xml", Level.WARN)
+          .put("org.openid4java", Level.WARN)
+          .put("org.openid4java.consumer.ConsumerManager", Level.FATAL)
+          .put("org.openid4java.discovery.Discovery", Level.ERROR)
+          .put("org.openid4java.server.RealmVerifier", Level.ERROR)
+          .put("org.openid4java.message.AuthSuccess", Level.ERROR)
+
+          // Silence non-critical messages from c3p0 (if used).
+          .put("com.mchange.v2.c3p0", Level.WARN)
+          .put("com.mchange.v2.resourcepool", Level.WARN)
+          .put("com.mchange.v2.sql", Level.WARN)
+
+          // Silence non-critical messages from apache.http.
+          .put("org.apache.http", Level.WARN)
+
+          // Silence non-critical messages from Jetty.
+          .put("org.eclipse.jetty", Level.WARN)
+
+          // Silence non-critical messages from JGit.
+          .put("org.eclipse.jgit.transport.PacketLineIn", Level.WARN)
+          .put("org.eclipse.jgit.transport.PacketLineOut", Level.WARN)
+          .put("org.eclipse.jgit.internal.storage.file.FileSnapshot", Level.WARN)
+          .put("org.eclipse.jgit.util.FS", Level.WARN)
+          .build();
+
+  private static Level getGerritLogLevel() {
+    String value = Strings.nullToEmpty(System.getenv("GERRIT_LOG_LEVEL"));
+    if (value.isEmpty()) {
+      value = Strings.nullToEmpty(System.getProperty("gerrit.logLevel"));
+    }
+    return Level.toLevel(value, Level.INFO);
+  }
+
+  public static void configureLogging() {
+    LogManager.resetConfiguration();
+
+    PatternLayout layout = new PatternLayout();
+    layout.setConversionPattern("%-5p %c %x: %m%n");
+
+    ConsoleAppender dst = new ConsoleAppender();
+    dst.setLayout(layout);
+    dst.setTarget("System.err");
+    dst.setThreshold(Level.DEBUG);
+    dst.activateOptions();
+
+    Logger root = LogManager.getRootLogger();
+    root.removeAllAppenders();
+    root.addAppender(dst);
+
+    LOG_LEVELS.entrySet().stream().forEach(e -> getLogger(e.getKey()).setLevel(e.getValue()));
+  }
+}

javatests/com/google/gerrit/acceptance/server/change/CommentsIT.java

@@ -29,6 +29,7 @@ import com.google.common.collect.Lists;
 import com.google.gerrit.acceptance.AbstractDaemonTest;
 import com.google.gerrit.acceptance.NoHttpd;
 import com.google.gerrit.acceptance.PushOneCommit;
+import com.google.gerrit.acceptance.testsuite.project.ProjectOperations;
 import com.google.gerrit.acceptance.testsuite.request.RequestScopeOperations;
 import com.google.gerrit.extensions.api.changes.DeleteCommentInput;
 import com.google.gerrit.extensions.api.changes.DraftInput;
@@ -80,6 +81,7 @@ import org.junit.Test;
 public class CommentsIT extends AbstractDaemonTest {
   @Inject private ChangeNoteUtil noteUtil;
   @Inject private FakeEmailSender email;
+  @Inject private ProjectOperations projectOperations;
   @Inject private Provider<ChangesCollection> changes;
   @Inject private Provider<PostReview> postReview;
   @Inject private RequestScopeOperations requestScopeOperations;
@@ -356,6 +358,39 @@ public class CommentsIT extends AbstractDaemonTest {
         .containsExactlyElementsIn(expectedComments);
   }
 
+  /**
+   * This test makes sure that the commits in the refs/draft-comments ref in NoteDb have no parent
+   * commits. This is important so that each new draft update (add, modify, delete) does not keep
+   * track of previous history.
+   */
+  @Test
+  public void commitsInDraftCommentsRefHaveNoParent() throws Exception {
+    PushOneCommit.Result r = createChange();
+    String changeId = r.getChangeId();
+    String revId = r.getCommit().getName();
+    String draftRefName = RefNames.refsDraftComments(r.getChange().getId(), user.id());
+
+    DraftInput comment1 = newDraft("file_1", Side.REVISION, 1, "comment 1");
+    CommentInfo commentInfo1 = addDraft(changeId, revId, comment1);
+    assertThat(getHeadOfDraftCommentsRef(draftRefName).getParentCount()).isEqualTo(0);
+
+    DraftInput comment2 = newDraft("file_2", Side.REVISION, 2, "comment 2");
+    CommentInfo commentInfo2 = addDraft(changeId, revId, comment2);
+    assertThat(getHeadOfDraftCommentsRef(draftRefName).getParentCount()).isEqualTo(0);
+
+    deleteDraft(changeId, revId, commentInfo1.id);
+    assertThat(getHeadOfDraftCommentsRef(draftRefName).getParentCount()).isEqualTo(0);
+    assertThat(
+            getDraftComments(changeId, revId).values().stream()
+                .flatMap(List::stream)
+                .map(commentInfo -> commentInfo.message))
+        .containsExactly("comment 2");
+
+    deleteDraft(changeId, revId, commentInfo2.id);
+    assertThat(projectOperations.project(allUsers).hasHead(draftRefName)).isFalse();
+    assertThat(getDraftComments(changeId, revId).values().stream().flatMap(List::stream)).isEmpty();
+  }
+
   @Test
   public void putDraft() throws Exception {
     for (Integer line : lines) {
@@ -1095,6 +1130,12 @@ public class CommentsIT extends AbstractDaemonTest {
     }
   }
 
+  private RevCommit getHeadOfDraftCommentsRef(String refName) throws Exception {
+    try (Repository repo = repoManager.openRepository(allUsers)) {
+      return getHead(repo, refName);
+    }
+  }
+
   private static String extractComments(String msg) {
     // Extract lines between start "....." and end "-- ".
     Pattern p = Pattern.compile(".*[.]{5}\n+(.*)\\n+-- \n.*", Pattern.DOTALL);

javatests/com/google/gerrit/server/query/account/AbstractQueryAccountsTest.java

@@ -88,14 +88,10 @@ import java.util.HashMap;
 import java.util.Iterator;
 import java.util.List;
 import java.util.Optional;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
 import org.eclipse.jgit.lib.PersonIdent;
 import org.eclipse.jgit.lib.Repository;
 import org.junit.After;
-import org.junit.AfterClass;
 import org.junit.Before;
-import org.junit.BeforeClass;
 import org.junit.Ignore;
 import org.junit.Test;
 
@@ -144,20 +140,6 @@ public abstract class AbstractQueryAccountsTest extends GerritServerTests {
 
   protected abstract Injector createInjector();
 
-  @BeforeClass
-  public static void setLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.DEBUG);
-    }
-  }
-
-  @AfterClass
-  public static void resetLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.INFO);
-    }
-  }
-
   @Before
   public void setUpInjector() throws Exception {
     lifecycle = new LifecycleManager();

javatests/com/google/gerrit/server/query/account/BUILD

@@ -19,11 +19,9 @@ java_library(
         "//lib:guava",
         "//lib/guice",
         "//lib/jgit/org.eclipse.jgit:jgit",
-        "//lib/log:log4j",
         "//lib/truth",
         "//lib/truth:truth-java8-extension",
         "//prolog:gerrit-prolog-common",
-        "//resources:log4j-config",
     ],
 )
 

javatests/com/google/gerrit/server/query/change/AbstractQueryChangesTest.java

@@ -128,8 +128,6 @@ import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.concurrent.TimeUnit;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
 import org.eclipse.jgit.junit.TestRepository;
 import org.eclipse.jgit.lib.ObjectId;
 import org.eclipse.jgit.lib.ObjectInserter;
@@ -141,9 +139,7 @@ import org.eclipse.jgit.revwalk.RevCommit;
 import org.eclipse.jgit.revwalk.RevWalk;
 import org.eclipse.jgit.util.SystemReader;
 import org.junit.After;
-import org.junit.AfterClass;
 import org.junit.Before;
-import org.junit.BeforeClass;
 import org.junit.Ignore;
 import org.junit.Test;
 
@@ -203,20 +199,6 @@ public abstract class AbstractQueryChangesTest extends GerritServerTests {
 
   protected abstract Injector createInjector();
 
-  @BeforeClass
-  public static void setLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.DEBUG);
-    }
-  }
-
-  @AfterClass
-  public static void resetLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.INFO);
-    }
-  }
-
   @Before
   public void setUpInjector() throws Exception {
     lifecycle = new LifecycleManager();

javatests/com/google/gerrit/server/query/change/BUILD

@@ -25,9 +25,7 @@ java_library(
         "//lib/guice",
         "//lib/jgit/org.eclipse.jgit:jgit",
         "//lib/jgit/org.eclipse.jgit.junit:junit",
-        "//lib/log:log4j",
         "//lib/truth",
-        "//resources:log4j-config",
     ],
 )
 

javatests/com/google/gerrit/server/query/group/AbstractQueryGroupsTest.java

@@ -65,12 +65,8 @@ import java.util.Iterator;
 import java.util.List;
 import java.util.Locale;
 import java.util.Optional;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
 import org.junit.After;
-import org.junit.AfterClass;
 import org.junit.Before;
-import org.junit.BeforeClass;
 import org.junit.Ignore;
 import org.junit.Test;
 
@@ -113,20 +109,6 @@ public abstract class AbstractQueryGroupsTest extends GerritServerTests {
 
   protected abstract Injector createInjector();
 
-  @BeforeClass
-  public static void setLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.DEBUG);
-    }
-  }
-
-  @AfterClass
-  public static void resetLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.INFO);
-    }
-  }
-
   @Before
   public void setUpInjector() throws Exception {
     lifecycle = new LifecycleManager();

javatests/com/google/gerrit/server/query/group/BUILD

@@ -19,10 +19,8 @@ java_library(
         "//lib:guava",
         "//lib/guice",
         "//lib/jgit/org.eclipse.jgit:jgit",
-        "//lib/log:log4j",
         "//lib/truth",
         "//lib/truth:truth-java8-extension",
-        "//resources:log4j-config",
     ],
 )
 

javatests/com/google/gerrit/server/query/project/AbstractQueryProjectsTest.java

@@ -65,12 +65,8 @@ import java.util.Iterator;
 import java.util.List;
 import java.util.Locale;
 import java.util.Set;
-import org.apache.log4j.Level;
-import org.apache.log4j.LogManager;
 import org.junit.After;
-import org.junit.AfterClass;
 import org.junit.Before;
-import org.junit.BeforeClass;
 import org.junit.Ignore;
 import org.junit.Test;
 
@@ -109,20 +105,6 @@ public abstract class AbstractQueryProjectsTest extends GerritServerTests {
 
   protected abstract Injector createInjector();
 
-  @BeforeClass
-  public static void setLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.DEBUG);
-    }
-  }
-
-  @AfterClass
-  public static void resetLogLevel() {
-    if (Boolean.getBoolean("debug")) {
-      LogManager.getRootLogger().setLevel(Level.INFO);
-    }
-  }
-
   @Before
   public void setUpInjector() throws Exception {
     lifecycle = new LifecycleManager();

javatests/com/google/gerrit/server/query/project/BUILD

@@ -20,9 +20,7 @@ java_library(
         "//lib:guava",
         "//lib/guice",
         "//lib/jgit/org.eclipse.jgit:jgit",
-        "//lib/log:log4j",
         "//lib/truth",
-        "//resources:log4j-config",
     ],
 )
 

tools/BUILD

@@ -65,7 +65,7 @@ java_package_configuration(
         "-Xep:NullableConstructor:WARN",
         "-Xep:NullablePrimitive:WARN",
         "-Xep:NullableVoid:WARN",
-        "-Xep:OperatorPrecedence:WARN",
+        "-Xep:OperatorPrecedence:ERROR",
         "-Xep:OverridesGuiceInjectableMethod:WARN",
         "-Xep:PreconditionsInvalidPlaceholder:WARN",
         "-Xep:ProtoFieldPreconditionsCheckNotNull:WARN",

tools/bzl/plugin.bzl

@@ -49,16 +49,21 @@ def gerrit_plugin(
 
     # TODO(davido): Remove manual merge of manifest file when this feature
     # request is implemented: https://github.com/bazelbuild/bazel/issues/2009
+    # TODO(davido): Remove manual touch command when this issue is resolved:
+    # https://github.com/bazelbuild/bazel/issues/10789
     genrule2(
         name = name + target_suffix,
         stamp = 1,
         srcs = ["%s__non_stamped_deploy.jar" % name],
         cmd = " && ".join([
+            "TZ=UTC",
+            "export TZ",
             "GEN_VERSION=$$(cat bazel-out/stable-status.txt | grep -w STABLE_BUILD_%s_LABEL | cut -d ' ' -f 2)" % dir_name.upper(),
             "cd $$TMP",
             "unzip -q $$ROOT/$<",
             "echo \"Implementation-Version: $$GEN_VERSION\n$$(cat META-INF/MANIFEST.MF)\" > META-INF/MANIFEST.MF",
-            "zip -qr $$ROOT/$@ .",
+            "find . -exec touch '{}' ';'",
+            "zip -Xqr $$ROOT/$@ .",
         ]),
         outs = ["%s%s.jar" % (name, target_suffix)],
         visibility = ["//visibility:public"],