Merge branch 'stable-2.16' into stable-3.0

* stable-2.16:
  Error Prone: Enable and fix OperatorPrecedence
  GerritBaseTests: Activate test logging
  Revert "Enable optional DEBUG level logs for query tests"
  Allow to control the Gerrit log level for running tests from system var
  Revert "Acceptance: set log threshold level for tests"
  Reduce log level for tests to INFO
  Bazel: Update time attribute of file entries in plugin artifact
  Add Jetty connection metrics
  Add additional JGit WindowCache metrics
  Update JGit to 5.1.13.202002110435-r
  BucketedCallback: fix prune() to remove unset sub-metrics from registry
  Modify draft ref updates commits to point to an empty parent

CommentsIT: Adapt to the 3.0 version of Gerrit.

Change-Id: I532d092329bdd94a85cccdf2b34dd150944a6699
This commit is contained in:
Marco Miller 2020-02-17 11:04:07 -05:00
commit 6f57e5b45d
27 changed files with 503 additions and 224 deletions

View File

@ -1608,6 +1608,21 @@ Default on JGit is 10 MiB on all platforms.
+
Common unit suffixes of 'k', 'm', or 'g' are supported.
[[core.packedGitUseStrongRefs]]core.packedGitUseStrongRefs::
+
Set to `true` in order to use strong references to reference packfile
pages cached in the WindowCache. Otherwise SoftReferences are used.
If this option is set to `false`, the Java garbage collector will
flush the WindowCache to free memory if the used heap comes close to
the maximum heap size. This has the advantage that it can quickly
reclaim memory which was used by the WindowCache but comes at the
price that the previously cached pack file content needs to be again
copied from the file system cache to the Gerrit process.
Setting this option to `true` prevents flushing the WindowCache
which provides more predictable performance.
+
Default is `false`.
[[core.deltaBaseCaseLimit]]core.deltaBaseCacheLimit::
+
Maximum number of bytes to reserve for caching base objects

View File

@ -289,7 +289,7 @@ The WAR file will be placed in:
Debugging tests:
----
bazel test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget
bazel test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget
----
Debug test example:
@ -365,21 +365,25 @@ If Docker is not available, the Elasticsearch tests will be skipped.
Note that Bazel currently does not show
link:https://github.com/bazelbuild/bazel/issues/3476[the skipped tests].
[[debug]]
=== Index Query Tests
[[logging]]
=== Controlling logging level
The `DEBUG` log level can optionally be enabled for the index query tests. That log level applies to
both Elasticsearch and Lucene tests.
Per default, logging level is set to `INFO` level for all tests. The `DEBUG`
log level can be enabled for the tests.
In Eclipse, set `-Ddebug=true` as a VM argument under the Run Configuration's `Arguments` tab.
With `bazel`, here is an example for the Lucene `account` test:
In IDE, set `-Dgerrit.logLevel=debug` as a VM argument. With `bazel`, pass
`GERRIT_LOG_LEVEL=debug` environment variable:
----
bazel test --jvmopt='-Ddebug=true' \
javatests/com/google/gerrit/server/query/account:lucene_query_test
bazel test --test_filter=com.gerrit.server.notedb.ChangeNotesTest \
--test_env=GERRIT_LOG_LEVEL=debug \
javatests/com/google/gerrit/server:server_tests
----
The log results can be found in:
`bazel-testlogs/javatests/com/google/gerrit/server/server_tests/test.log`.
== Dependencies
Dependency JARs are normally downloaded as needed, but you can
@ -574,7 +578,6 @@ To use the binary from the Bazel build, you need to use the `run_npm_binary.py`
wrapper script. For an example, see the use of `crisper` in `tools/bzl/js.bzl`.
[[RBE]]
== Google Remote Build Support
@ -609,8 +612,6 @@ bazel test --config=remote \
```
GERRIT
------
Part of link:index.html[Gerrit Code Review]

View File

@ -60,6 +60,15 @@ objects needing finalization.
==== Jetty
* `http/server/jetty/connections/connections`: The current number of open connections
* `http/server/jetty/connections/connections_total`: The total number of connections opened
* `http/server/jetty/connections/connections_duration_max`: The max duration of a connection in ms
* `http/server/jetty/connections/connections_duration_mean`: The mean duration of a connection in ms
* `http/server/jetty/connections/connections_duration_stdev`: The standard deviation of the duration of a connection in ms
* `http/server/jetty/connections/received_messages`: The total number of messages received
* `http/server/jetty/connections/sent_messages`: The total number of messages sent
* `http/server/jetty/connections/received_bytes`: Total number of bytes received by tracked connections
* `http/server/jetty/connections/sent_bytes`: Total number of bytes sent by tracked connections"
* `http/server/jetty/threadpool/active_threads`: Active threads
* `http/server/jetty/threadpool/idle_threads`: Idle threads
* `http/server/jetty/threadpool/reserved_threads`: Reserved threads
@ -130,6 +139,18 @@ topic submissions that concluded successfully.
* `jgit/block_cache/cache_used`: Bytes of memory retained in JGit block cache.
* `jgit/block_cache/open_files`: File handles held open by JGit block cache.
* `avg_load_time` Average time to load a cache entry for JGit block cache.
* `eviction_count` : Cache evictions for JGit block cache.
* `eviction_ratio` : Cache eviction ratio for JGit block cache.
* `hit_count` : Cache hits for JGit block cache.
* `hit_ratio` : Cache hit ratio for JGit block cache.
* `load_failure_count` : Failed cache loads for JGit block cache.
* `load_failure_ratio` : Failed cache load ratio for JGit block cache.
* `load_success_count` : Successful cache loads for JGit block cache.
* `miss_count` : Cache misses for JGit block cache.
* `miss_ratio` : Cache miss ratio for JGit block cache.
* `cache_used_per_repository` : Bytes of memory retained per repository for the top N repositories
having most data in the cache. The number N of reported repositories is limited to 1000.
=== Git

View File

@ -89,8 +89,6 @@ java_library2(
"//lib/httpcomponents:httpcore",
"//lib/jetty:servlet",
"//lib/jgit/org.eclipse.jgit.junit:junit",
"//lib/log:impl-log4j",
"//lib/log:log4j",
"//lib/mockito",
"//lib/truth",
"//lib/truth:truth-java8-extension",

View File

@ -18,13 +18,11 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
import static java.util.Objects.requireNonNull;
import static org.apache.log4j.Logger.getLogger;
import com.google.auto.value.AutoValue;
import com.google.common.base.MoreObjects;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.gerrit.acceptance.testsuite.account.AccountOperations;
import com.google.gerrit.acceptance.testsuite.account.AccountOperationsImpl;
import com.google.gerrit.acceptance.testsuite.group.GroupOperations;
@ -49,6 +47,7 @@ import com.google.gerrit.server.util.SystemLog;
import com.google.gerrit.testing.FakeEmailSender;
import com.google.gerrit.testing.InMemoryRepositoryManager;
import com.google.gerrit.testing.SshMode;
import com.google.gerrit.testing.TestLoggingActivator;
import com.google.inject.AbstractModule;
import com.google.inject.BindingAnnotation;
import com.google.inject.Injector;
@ -74,11 +73,6 @@ import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.stream.Stream;
import org.apache.log4j.ConsoleAppender;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.RepositoryCache;
import org.eclipse.jgit.util.FS;
@ -113,8 +107,7 @@ public class GerritServer implements AutoCloseable {
null, // @GerritConfig is only valid on methods.
null, // @GerritConfigs is only valid on methods.
null, // @GlobalPluginConfig is only valid on methods.
null, // @GlobalPluginConfigs is only valid on methods.
getLogLevelThresholdAnnotation(testDesc));
null); // @GlobalPluginConfigs is only valid on methods.
}
public static Description forTestMethod(
@ -136,8 +129,7 @@ public class GerritServer implements AutoCloseable {
testDesc.getAnnotation(GerritConfig.class),
testDesc.getAnnotation(GerritConfigs.class),
testDesc.getAnnotation(GlobalPluginConfig.class),
testDesc.getAnnotation(GlobalPluginConfigs.class),
getLogLevelThresholdAnnotation(testDesc));
testDesc.getAnnotation(GlobalPluginConfigs.class));
}
private static boolean has(Class<? extends Annotation> annotation, Class<?> clazz) {
@ -149,14 +141,6 @@ public class GerritServer implements AutoCloseable {
return false;
}
private static Level getLogLevelThresholdAnnotation(org.junit.runner.Description testDesc) {
LogThreshold logLevelThreshold = testDesc.getTestClass().getAnnotation(LogThreshold.class);
if (logLevelThreshold == null) {
return Level.DEBUG;
}
return Level.toLevel(logLevelThreshold.level());
}
abstract org.junit.runner.Description testDescription();
@Nullable
@ -188,8 +172,6 @@ public class GerritServer implements AutoCloseable {
@Nullable
abstract GlobalPluginConfigs pluginConfigs();
abstract Level logLevelThreshold();
private void checkValidAnnotations() {
if (configs() != null && config() != null) {
throw new IllegalStateException("Use either @GerritConfigs or @GerritConfig not both");
@ -223,46 +205,6 @@ public class GerritServer implements AutoCloseable {
}
}
private static final ImmutableMap<String, Level> LOG_LEVELS =
ImmutableMap.<String, Level>builder()
.put("com.google.gerrit", Level.DEBUG)
// Silence non-critical messages from MINA SSHD.
.put("org.apache.mina", Level.WARN)
.put("org.apache.sshd.common", Level.WARN)
.put("org.apache.sshd.server", Level.WARN)
.put("org.apache.sshd.common.keyprovider.FileKeyPairProvider", Level.INFO)
.put("com.google.gerrit.sshd.GerritServerSession", Level.WARN)
// Silence non-critical messages from mime-util.
.put("eu.medsea.mimeutil", Level.WARN)
// Silence non-critical messages from openid4java.
.put("org.apache.xml", Level.WARN)
.put("org.openid4java", Level.WARN)
.put("org.openid4java.consumer.ConsumerManager", Level.FATAL)
.put("org.openid4java.discovery.Discovery", Level.ERROR)
.put("org.openid4java.server.RealmVerifier", Level.ERROR)
.put("org.openid4java.message.AuthSuccess", Level.ERROR)
// Silence non-critical messages from c3p0 (if used).
.put("com.mchange.v2.c3p0", Level.WARN)
.put("com.mchange.v2.resourcepool", Level.WARN)
.put("com.mchange.v2.sql", Level.WARN)
// Silence non-critical messages from apache.http.
.put("org.apache.http", Level.WARN)
// Silence non-critical messages from Jetty.
.put("org.eclipse.jetty", Level.WARN)
// Silence non-critical messages from JGit.
.put("org.eclipse.jgit.transport.PacketLineIn", Level.WARN)
.put("org.eclipse.jgit.transport.PacketLineOut", Level.WARN)
.put("org.eclipse.jgit.internal.storage.file.FileSnapshot", Level.WARN)
.put("org.eclipse.jgit.util.FS", Level.WARN)
.build();
private static boolean forceLocalDisk() {
String value = Strings.nullToEmpty(System.getenv("GERRIT_FORCE_LOCAL_DISK"));
if (value.isEmpty()) {
@ -378,7 +320,7 @@ public class GerritServer implements AutoCloseable {
throws Exception {
checkArgument(site != null, "site is required (even for in-memory server");
desc.checkValidAnnotations();
configureLogging(desc.logLevelThreshold());
TestLoggingActivator.configureLogging();
CyclicBarrier serverStarted = new CyclicBarrier(2);
Daemon daemon =
new Daemon(
@ -482,25 +424,6 @@ public class GerritServer implements AutoCloseable {
return new GerritServer(desc, site, createTestInjector(daemon), daemon, daemonService);
}
private static void configureLogging(Level threshold) {
LogManager.resetConfiguration();
PatternLayout layout = new PatternLayout();
layout.setConversionPattern("%-5p %c %x: %m%n");
ConsoleAppender dst = new ConsoleAppender();
dst.setLayout(layout);
dst.setTarget("System.err");
dst.setThreshold(threshold);
dst.activateOptions();
Logger root = LogManager.getRootLogger();
root.removeAllAppenders();
root.addAppender(dst);
LOG_LEVELS.entrySet().stream().forEach(e -> getLogger(e.getKey()).setLevel(e.getValue()));
}
private static void mergeTestConfig(Config cfg) {
String forceEphemeralPort = String.format("%s:0", getLocalHost().getHostName());
String url = "http://" + forceEphemeralPort + "/";

View File

@ -1,29 +0,0 @@
// Copyright (C) 2019 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.acceptance;
import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.TYPE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
import java.lang.annotation.Inherited;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;
@Target({TYPE, METHOD})
@Retention(RUNTIME)
@Inherited
public @interface LogThreshold {
String level() default "DEBUG";
}

View File

@ -22,6 +22,7 @@ import com.google.common.collect.Maps;
import com.google.gerrit.metrics.Description;
import com.google.gerrit.metrics.Field;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
/** Abstract callback metric broken down into buckets. */
@ -66,7 +67,13 @@ abstract class BucketedCallback<V> implements BucketedMetric {
}
void doPrune() {
cells.entrySet().removeIf(objectValueGaugeEntry -> !objectValueGaugeEntry.getValue().set);
Set<Map.Entry<Object, BucketedCallback<V>.ValueGauge>> entries = cells.entrySet();
for (Map.Entry<Object, ValueGauge> e : entries) {
if (!e.getValue().set) {
entries.remove(e);
registry.remove(submetric(e.getKey()));
}
}
}
void doEndSet() {

View File

@ -14,12 +14,18 @@
package com.google.gerrit.metrics.proc;
import com.google.common.base.Supplier;
import com.google.gerrit.metrics.CallbackMetric1;
import com.google.gerrit.metrics.Description;
import com.google.gerrit.metrics.Description.Units;
import com.google.gerrit.metrics.Field;
import com.google.gerrit.metrics.MetricMaker;
import java.util.Map;
import org.eclipse.jgit.storage.file.WindowCacheStats;
public class JGitMetricModule extends MetricModule {
private static final long MAX_REPO_COUNT = 1000;
@Override
protected void configure(MetricMaker metrics) {
metrics.newCallbackMetric(
@ -28,12 +34,153 @@ public class JGitMetricModule extends MetricModule {
new Description("Bytes of memory retained in JGit block cache.")
.setGauge()
.setUnit(Units.BYTES),
WindowCacheStats::getOpenBytes);
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getOpenByteCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/open_files",
Integer.class,
Long.class,
new Description("File handles held open by JGit block cache.").setGauge().setUnit("fds"),
WindowCacheStats::getOpenFiles);
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getOpenFileCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/avg_load_time",
Double.class,
new Description("Average time to load a cache entry for JGit block cache.")
.setGauge()
.setUnit(Units.NANOSECONDS),
new Supplier<Double>() {
@Override
public Double get() {
return WindowCacheStats.getStats().getAverageLoadTime();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/eviction_count",
Long.class,
new Description("Cache evictions for JGit block cache.").setGauge(),
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getEvictionCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/eviction_ratio",
Double.class,
new Description("Cache eviction ratio for JGit block cache.").setGauge(),
new Supplier<Double>() {
@Override
public Double get() {
return WindowCacheStats.getStats().getEvictionRatio();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/hit_count",
Long.class,
new Description("Cache hits for JGit block cache.").setGauge(),
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getHitCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/hit_ratio",
Double.class,
new Description("Cache hit ratio for JGit block cache.").setGauge(),
new Supplier<Double>() {
@Override
public Double get() {
return WindowCacheStats.getStats().getHitRatio();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/load_failure_count",
Long.class,
new Description("Failed cache loads for JGit block cache.").setGauge(),
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getLoadFailureCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/load_failure_ratio",
Double.class,
new Description("Failed cache load ratio for JGit block cache.").setGauge(),
new Supplier<Double>() {
@Override
public Double get() {
return WindowCacheStats.getStats().getLoadFailureRatio();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/load_success_count",
Long.class,
new Description("Successfull cache loads for JGit block cache.").setGauge(),
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getLoadSuccessCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/miss_count",
Long.class,
new Description("Cache misses for JGit block cache.").setGauge(),
new Supplier<Long>() {
@Override
public Long get() {
return WindowCacheStats.getStats().getMissCount();
}
});
metrics.newCallbackMetric(
"jgit/block_cache/miss_ratio",
Double.class,
new Description("Cache miss ratio for JGit block cache.").setGauge(),
WindowCacheStats.getStats()::getMissRatio);
CallbackMetric1<String, Long> repoEnt =
metrics.newCallbackMetric(
"jgit/block_cache/cache_used_per_repository",
Long.class,
new Description(
"Bytes of memory retained per repository for the top repositories "
+ "having most data in the cache.")
.setGauge()
.setUnit("byte"),
Field.ofString("repository_name"));
metrics.newTrigger(
repoEnt,
() -> {
Map<String, Long> cacheMap = WindowCacheStats.getStats().getOpenByteCountPerRepository();
if (cacheMap.isEmpty()) {
repoEnt.forceCreate("");
} else {
cacheMap.entrySet().stream()
.sorted(Map.Entry.<String, Long>comparingByValue().reversed())
.limit(MAX_REPO_COUNT)
.forEach(e -> repoEnt.set(e.getKey(), e.getValue()));
repoEnt.prune();
}
});
}
}

View File

@ -17,6 +17,7 @@ import com.google.common.collect.ImmutableSet;
import com.google.gerrit.metrics.CallbackMetric;
import com.google.gerrit.metrics.CallbackMetric0;
import com.google.gerrit.metrics.Description;
import com.google.gerrit.metrics.Description.Units;
import com.google.gerrit.metrics.MetricMaker;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@ -45,27 +46,83 @@ public class JettyMetrics {
metrics.newCallbackMetric(
"http/server/jetty/threadpool/idle_threads",
Integer.class,
new Description("Idle threads").setGauge().setUnit("threads"));
new Description("Idle threads").setGauge());
CallbackMetric0<Integer> busyThreads =
metrics.newCallbackMetric(
"http/server/jetty/threadpool/active_threads",
Integer.class,
new Description("Active threads").setGauge().setUnit("threads"));
new Description("Active threads").setGauge());
CallbackMetric0<Integer> reservedThreads =
metrics.newCallbackMetric(
"http/server/jetty/threadpool/reserved_threads",
Integer.class,
new Description("Reserved threads").setGauge().setUnit("threads"));
new Description("Reserved threads").setGauge());
CallbackMetric0<Integer> queueSize =
metrics.newCallbackMetric(
"http/server/jetty/threadpool/queue_size",
Integer.class,
new Description("Queued requests waiting for a thread").setGauge().setUnit("requests"));
new Description("Queued requests waiting for a thread").setGauge());
CallbackMetric0<Boolean> lowOnThreads =
metrics.newCallbackMetric(
"http/server/jetty/threadpool/is_low_on_threads",
Boolean.class,
new Description("Whether thread pool is low on threads").setGauge());
CallbackMetric0<Long> connections =
metrics.newCallbackMetric(
"http/server/jetty/connections/connections",
Long.class,
new Description("The current number of open connections").setGauge());
CallbackMetric0<Long> connectionsTotal =
metrics.newCallbackMetric(
"http/server/jetty/connections/connections_total",
Long.class,
new Description("The total number of connections opened").setGauge());
CallbackMetric0<Long> connectionDurationMax =
metrics.newCallbackMetric(
"http/server/jetty/connections/connections_duration_max",
Long.class,
new Description("The max duration of a connection")
.setGauge()
.setUnit(Units.MILLISECONDS));
CallbackMetric0<Double> connectionDurationMean =
metrics.newCallbackMetric(
"http/server/jetty/connections/connections_duration_mean",
Double.class,
new Description("The mean duration of a connection")
.setGauge()
.setUnit(Units.MILLISECONDS));
CallbackMetric0<Double> connectionDurationStDev =
metrics.newCallbackMetric(
"http/server/jetty/connections/connections_duration_stdev",
Double.class,
new Description("The standard deviation of the duration of a connection")
.setGauge()
.setUnit(Units.MILLISECONDS));
CallbackMetric0<Long> receivedMessages =
metrics.newCallbackMetric(
"http/server/jetty/connections/received_messages",
Long.class,
new Description("The total number of messages received").setGauge());
CallbackMetric0<Long> sentMessages =
metrics.newCallbackMetric(
"http/server/jetty/connections/sent_messages",
Long.class,
new Description("The total number of messages sent").setGauge());
CallbackMetric0<Long> receivedBytes =
metrics.newCallbackMetric(
"http/server/jetty/connections/received_bytes",
Long.class,
new Description("Total number of bytes received by tracked connections")
.setGauge()
.setUnit(Units.BYTES));
CallbackMetric0<Long> sentBytes =
metrics.newCallbackMetric(
"http/server/jetty/connections/sent_bytes",
Long.class,
new Description("Total number of bytes sent by tracked connections")
.setGauge()
.setUnit(Units.BYTES));
JettyServer.Metrics jettyMetrics = jetty.getMetrics();
metrics.newTrigger(
ImmutableSet.<CallbackMetric<?>>of(
@ -76,7 +133,16 @@ public class JettyMetrics {
maxPoolSize,
poolSize,
queueSize,
lowOnThreads),
lowOnThreads,
connections,
connectionsTotal,
connectionDurationMax,
connectionDurationMean,
connectionDurationStDev,
receivedMessages,
sentMessages,
receivedBytes,
sentBytes),
() -> {
minPoolSize.set(jettyMetrics.getMinThreads());
maxPoolSize.set(jettyMetrics.getMaxThreads());
@ -86,6 +152,15 @@ public class JettyMetrics {
reservedThreads.set(jettyMetrics.getReservedThreads());
queueSize.set(jettyMetrics.getQueueSize());
lowOnThreads.set(jettyMetrics.isLowOnThreads());
connections.set(jettyMetrics.getConnections());
connectionsTotal.set(jettyMetrics.getConnectionsTotal());
connectionDurationMax.set(jettyMetrics.getConnectionDurationMax());
connectionDurationMean.set(jettyMetrics.getConnectionDurationMean());
connectionDurationStDev.set(jettyMetrics.getConnectionDurationStdDev());
receivedMessages.set(jettyMetrics.getReceivedMessages());
sentMessages.set(jettyMetrics.getSentMessages());
receivedBytes.set(jettyMetrics.getReceivedBytes());
sentBytes.set(jettyMetrics.getSentBytes());
});
}
}

View File

@ -43,6 +43,7 @@ import java.util.concurrent.TimeUnit;
import javax.servlet.DispatcherType;
import javax.servlet.Filter;
import org.eclipse.jetty.http.HttpScheme;
import org.eclipse.jetty.io.ConnectionStatistics;
import org.eclipse.jetty.jmx.MBeanContainer;
import org.eclipse.jetty.server.Connector;
import org.eclipse.jetty.server.ForwardedRequestCustomizer;
@ -116,9 +117,11 @@ public class JettyServer {
static class Metrics {
private final QueuedThreadPool threadPool;
private ConnectionStatistics connStats;
Metrics(QueuedThreadPool threadPool) {
Metrics(QueuedThreadPool threadPool, ConnectionStatistics connStats) {
this.threadPool = threadPool;
this.connStats = connStats;
}
public int getIdleThreads() {
@ -152,12 +155,49 @@ public class JettyServer {
public boolean isLowOnThreads() {
return threadPool.isLowOnThreads();
}
public long getConnections() {
return connStats.getConnections();
}
public long getConnectionsTotal() {
return connStats.getConnectionsTotal();
}
public long getConnectionDurationMax() {
return connStats.getConnectionDurationMax();
}
public double getConnectionDurationMean() {
return connStats.getConnectionDurationMean();
}
public double getConnectionDurationStdDev() {
return connStats.getConnectionDurationStdDev();
}
public long getReceivedMessages() {
return connStats.getReceivedMessages();
}
public long getSentMessages() {
return connStats.getSentMessages();
}
public long getReceivedBytes() {
return connStats.getReceivedBytes();
}
public long getSentBytes() {
return connStats.getSentBytes();
}
}
private final SitePaths site;
private final Server httpd;
private final Metrics metrics;
private boolean reverseProxy;
private ConnectionStatistics connStats;
@Inject
JettyServer(
@ -171,7 +211,11 @@ public class JettyServer {
QueuedThreadPool pool = threadPool(cfg, threadSettingsConfig);
httpd = new Server(pool);
httpd.setConnectors(listen(httpd, cfg));
metrics = new Metrics(pool);
connStats = new ConnectionStatistics();
for (Connector connector : httpd.getConnectors()) {
connector.addBean(connStats);
}
metrics = new Metrics(pool, connStats);
Handler app = makeContext(env, cfg);
if (cfg.getBoolean("httpd", "requestLog", !reverseProxy)) {

View File

@ -184,6 +184,14 @@ public abstract class AbstractChangeUpdate {
protected abstract String getRefName();
protected void setParentCommit(CommitBuilder cb, ObjectId parentCommitId) {
if (!parentCommitId.equals(ObjectId.zeroId())) {
cb.setParentId(parentCommitId);
} else {
cb.setParentIds(); // Ref is currently nonexistent, commit has no parents.
}
}
/**
* Apply this update to the given inserter.
*
@ -216,11 +224,7 @@ public abstract class AbstractChangeUpdate {
}
cb.setAuthor(authorIdent);
cb.setCommitter(new PersonIdent(serverIdent, when));
if (!curr.equals(z)) {
cb.setParentId(curr);
} else {
cb.setParentIds(); // Ref is currently nonexistent, commit has no parents.
}
setParentCommit(cb, curr);
if (cb.getTreeId() == null) {
if (curr.equals(z)) {
cb.setTreeId(emptyTree(ins)); // No parent, assume empty tree.

View File

@ -234,6 +234,11 @@ public class ChangeDraftUpdate extends AbstractChangeUpdate {
return RefNames.refsDraftComments(getId(), accountId);
}
@Override
protected void setParentCommit(CommitBuilder cb, ObjectId parentCommitId) {
cb.setParentIds(); // Draft updates should not keep history of parent commits
}
@Override
public boolean isEmpty() {
return delete.isEmpty() && put.isEmpty();

View File

@ -644,7 +644,7 @@ public class SubmoduleOp {
int newSize = msgbuf.length() + bullet.length() + message.length();
if (++numMessages > maxCommitMessages
|| newSize > maxCombinedCommitMessageSize
|| iter.hasNext() && (newSize + ellipsis.length()) > maxCombinedCommitMessageSize) {
|| (iter.hasNext() && (newSize + ellipsis.length()) > maxCombinedCommitMessageSize)) {
msgbuf.append(ellipsis);
break;
}

View File

@ -46,6 +46,8 @@ java_library(
"//lib/guice:guice-servlet",
"//lib/jgit/org.eclipse.jgit:jgit",
"//lib/jgit/org.eclipse.jgit.junit:junit",
"//lib/log:impl-log4j",
"//lib/log:log4j",
"//lib/truth",
],
)

View File

@ -15,6 +15,7 @@
package com.google.gerrit.testing;
import com.google.common.base.CharMatcher;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.rules.ExpectedException;
@ -25,6 +26,11 @@ public abstract class GerritBaseTests {
@Rule public ExpectedException exception = ExpectedException.none();
@Rule public final TestName testName = new TestName();
@BeforeClass
public static void beforeClassTest() {
TestLoggingActivator.configureLogging();
}
protected String getSanitizedMethodName() {
String name = testName.getMethodName().toLowerCase();
name =

View File

@ -0,0 +1,94 @@
// Copyright (C) 2020 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.testing;
import static org.apache.log4j.Logger.getLogger;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableMap;
import org.apache.log4j.ConsoleAppender;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;
public class TestLoggingActivator {
private static final ImmutableMap<String, Level> LOG_LEVELS =
ImmutableMap.<String, Level>builder()
.put("com.google.gerrit", getGerritLogLevel())
// Silence non-critical messages from MINA SSHD.
.put("org.apache.mina", Level.WARN)
.put("org.apache.sshd.common", Level.WARN)
.put("org.apache.sshd.server", Level.WARN)
.put("org.apache.sshd.common.keyprovider.FileKeyPairProvider", Level.INFO)
.put("com.google.gerrit.sshd.GerritServerSession", Level.WARN)
// Silence non-critical messages from mime-util.
.put("eu.medsea.mimeutil", Level.WARN)
// Silence non-critical messages from openid4java.
.put("org.apache.xml", Level.WARN)
.put("org.openid4java", Level.WARN)
.put("org.openid4java.consumer.ConsumerManager", Level.FATAL)
.put("org.openid4java.discovery.Discovery", Level.ERROR)
.put("org.openid4java.server.RealmVerifier", Level.ERROR)
.put("org.openid4java.message.AuthSuccess", Level.ERROR)
// Silence non-critical messages from c3p0 (if used).
.put("com.mchange.v2.c3p0", Level.WARN)
.put("com.mchange.v2.resourcepool", Level.WARN)
.put("com.mchange.v2.sql", Level.WARN)
// Silence non-critical messages from apache.http.
.put("org.apache.http", Level.WARN)
// Silence non-critical messages from Jetty.
.put("org.eclipse.jetty", Level.WARN)
// Silence non-critical messages from JGit.
.put("org.eclipse.jgit.transport.PacketLineIn", Level.WARN)
.put("org.eclipse.jgit.transport.PacketLineOut", Level.WARN)
.put("org.eclipse.jgit.internal.storage.file.FileSnapshot", Level.WARN)
.put("org.eclipse.jgit.util.FS", Level.WARN)
.build();
private static Level getGerritLogLevel() {
String value = Strings.nullToEmpty(System.getenv("GERRIT_LOG_LEVEL"));
if (value.isEmpty()) {
value = Strings.nullToEmpty(System.getProperty("gerrit.logLevel"));
}
return Level.toLevel(value, Level.INFO);
}
public static void configureLogging() {
LogManager.resetConfiguration();
PatternLayout layout = new PatternLayout();
layout.setConversionPattern("%-5p %c %x: %m%n");
ConsoleAppender dst = new ConsoleAppender();
dst.setLayout(layout);
dst.setTarget("System.err");
dst.setThreshold(Level.DEBUG);
dst.activateOptions();
Logger root = LogManager.getRootLogger();
root.removeAllAppenders();
root.addAppender(dst);
LOG_LEVELS.entrySet().stream().forEach(e -> getLogger(e.getKey()).setLevel(e.getValue()));
}
}

View File

@ -29,6 +29,7 @@ import com.google.common.collect.Lists;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.NoHttpd;
import com.google.gerrit.acceptance.PushOneCommit;
import com.google.gerrit.acceptance.testsuite.project.ProjectOperations;
import com.google.gerrit.acceptance.testsuite.request.RequestScopeOperations;
import com.google.gerrit.extensions.api.changes.DeleteCommentInput;
import com.google.gerrit.extensions.api.changes.DraftInput;
@ -80,6 +81,7 @@ import org.junit.Test;
public class CommentsIT extends AbstractDaemonTest {
@Inject private ChangeNoteUtil noteUtil;
@Inject private FakeEmailSender email;
@Inject private ProjectOperations projectOperations;
@Inject private Provider<ChangesCollection> changes;
@Inject private Provider<PostReview> postReview;
@Inject private RequestScopeOperations requestScopeOperations;
@ -356,6 +358,39 @@ public class CommentsIT extends AbstractDaemonTest {
.containsExactlyElementsIn(expectedComments);
}
/**
* This test makes sure that the commits in the refs/draft-comments ref in NoteDb have no parent
* commits. This is important so that each new draft update (add, modify, delete) does not keep
* track of previous history.
*/
@Test
public void commitsInDraftCommentsRefHaveNoParent() throws Exception {
PushOneCommit.Result r = createChange();
String changeId = r.getChangeId();
String revId = r.getCommit().getName();
String draftRefName = RefNames.refsDraftComments(r.getChange().getId(), user.id());
DraftInput comment1 = newDraft("file_1", Side.REVISION, 1, "comment 1");
CommentInfo commentInfo1 = addDraft(changeId, revId, comment1);
assertThat(getHeadOfDraftCommentsRef(draftRefName).getParentCount()).isEqualTo(0);
DraftInput comment2 = newDraft("file_2", Side.REVISION, 2, "comment 2");
CommentInfo commentInfo2 = addDraft(changeId, revId, comment2);
assertThat(getHeadOfDraftCommentsRef(draftRefName).getParentCount()).isEqualTo(0);
deleteDraft(changeId, revId, commentInfo1.id);
assertThat(getHeadOfDraftCommentsRef(draftRefName).getParentCount()).isEqualTo(0);
assertThat(
getDraftComments(changeId, revId).values().stream()
.flatMap(List::stream)
.map(commentInfo -> commentInfo.message))
.containsExactly("comment 2");
deleteDraft(changeId, revId, commentInfo2.id);
assertThat(projectOperations.project(allUsers).hasHead(draftRefName)).isFalse();
assertThat(getDraftComments(changeId, revId).values().stream().flatMap(List::stream)).isEmpty();
}
@Test
public void putDraft() throws Exception {
for (Integer line : lines) {
@ -1095,6 +1130,12 @@ public class CommentsIT extends AbstractDaemonTest {
}
}
private RevCommit getHeadOfDraftCommentsRef(String refName) throws Exception {
try (Repository repo = repoManager.openRepository(allUsers)) {
return getHead(repo, refName);
}
}
private static String extractComments(String msg) {
// Extract lines between start "....." and end "-- ".
Pattern p = Pattern.compile(".*[.]{5}\n+(.*)\\n+-- \n.*", Pattern.DOTALL);

View File

@ -88,14 +88,10 @@ import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Optional;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.eclipse.jgit.lib.PersonIdent;
import org.eclipse.jgit.lib.Repository;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Test;
@ -144,20 +140,6 @@ public abstract class AbstractQueryAccountsTest extends GerritServerTests {
protected abstract Injector createInjector();
@BeforeClass
public static void setLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.DEBUG);
}
}
@AfterClass
public static void resetLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.INFO);
}
}
@Before
public void setUpInjector() throws Exception {
lifecycle = new LifecycleManager();

View File

@ -19,11 +19,9 @@ java_library(
"//lib:guava",
"//lib/guice",
"//lib/jgit/org.eclipse.jgit:jgit",
"//lib/log:log4j",
"//lib/truth",
"//lib/truth:truth-java8-extension",
"//prolog:gerrit-prolog-common",
"//resources:log4j-config",
],
)

View File

@ -128,8 +128,6 @@ import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.eclipse.jgit.junit.TestRepository;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.ObjectInserter;
@ -141,9 +139,7 @@ import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;
import org.eclipse.jgit.util.SystemReader;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Test;
@ -203,20 +199,6 @@ public abstract class AbstractQueryChangesTest extends GerritServerTests {
protected abstract Injector createInjector();
@BeforeClass
public static void setLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.DEBUG);
}
}
@AfterClass
public static void resetLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.INFO);
}
}
@Before
public void setUpInjector() throws Exception {
lifecycle = new LifecycleManager();

View File

@ -25,9 +25,7 @@ java_library(
"//lib/guice",
"//lib/jgit/org.eclipse.jgit:jgit",
"//lib/jgit/org.eclipse.jgit.junit:junit",
"//lib/log:log4j",
"//lib/truth",
"//resources:log4j-config",
],
)

View File

@ -65,12 +65,8 @@ import java.util.Iterator;
import java.util.List;
import java.util.Locale;
import java.util.Optional;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Test;
@ -113,20 +109,6 @@ public abstract class AbstractQueryGroupsTest extends GerritServerTests {
protected abstract Injector createInjector();
@BeforeClass
public static void setLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.DEBUG);
}
}
@AfterClass
public static void resetLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.INFO);
}
}
@Before
public void setUpInjector() throws Exception {
lifecycle = new LifecycleManager();

View File

@ -19,10 +19,8 @@ java_library(
"//lib:guava",
"//lib/guice",
"//lib/jgit/org.eclipse.jgit:jgit",
"//lib/log:log4j",
"//lib/truth",
"//lib/truth:truth-java8-extension",
"//resources:log4j-config",
],
)

View File

@ -65,12 +65,8 @@ import java.util.Iterator;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Test;
@ -109,20 +105,6 @@ public abstract class AbstractQueryProjectsTest extends GerritServerTests {
protected abstract Injector createInjector();
@BeforeClass
public static void setLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.DEBUG);
}
}
@AfterClass
public static void resetLogLevel() {
if (Boolean.getBoolean("debug")) {
LogManager.getRootLogger().setLevel(Level.INFO);
}
}
@Before
public void setUpInjector() throws Exception {
lifecycle = new LifecycleManager();

View File

@ -20,9 +20,7 @@ java_library(
"//lib:guava",
"//lib/guice",
"//lib/jgit/org.eclipse.jgit:jgit",
"//lib/log:log4j",
"//lib/truth",
"//resources:log4j-config",
],
)

View File

@ -65,7 +65,7 @@ java_package_configuration(
"-Xep:NullableConstructor:WARN",
"-Xep:NullablePrimitive:WARN",
"-Xep:NullableVoid:WARN",
"-Xep:OperatorPrecedence:WARN",
"-Xep:OperatorPrecedence:ERROR",
"-Xep:OverridesGuiceInjectableMethod:WARN",
"-Xep:PreconditionsInvalidPlaceholder:WARN",
"-Xep:ProtoFieldPreconditionsCheckNotNull:WARN",

View File

@ -49,16 +49,21 @@ def gerrit_plugin(
# TODO(davido): Remove manual merge of manifest file when this feature
# request is implemented: https://github.com/bazelbuild/bazel/issues/2009
# TODO(davido): Remove manual touch command when this issue is resolved:
# https://github.com/bazelbuild/bazel/issues/10789
genrule2(
name = name + target_suffix,
stamp = 1,
srcs = ["%s__non_stamped_deploy.jar" % name],
cmd = " && ".join([
"TZ=UTC",
"export TZ",
"GEN_VERSION=$$(cat bazel-out/stable-status.txt | grep -w STABLE_BUILD_%s_LABEL | cut -d ' ' -f 2)" % dir_name.upper(),
"cd $$TMP",
"unzip -q $$ROOT/$<",
"echo \"Implementation-Version: $$GEN_VERSION\n$$(cat META-INF/MANIFEST.MF)\" > META-INF/MANIFEST.MF",
"zip -qr $$ROOT/$@ .",
"find . -exec touch '{}' ';'",
"zip -Xqr $$ROOT/$@ .",
]),
outs = ["%s%s.jar" % (name, target_suffix)],
visibility = ["//visibility:public"],