opendev/gerrit
Code Issues Proposed changes

Replace reindexAfterRefUpdate setting with indexMergeable

Browse Source 
Gerrit can index a boolean field called 'mergeable' that determines
if a change can be merged into the target ref. This bit can change
whenever the target ref advances. Gerrit therefore has logic to
reindex all open changes when the target ref advances.

Depending on the number of open changes, the frequency of updates of
the target ref and the size of the repo, this can be a very expensive
operation. For large installations, 'reindexAfterRefUpdate' was added
as a setting back in 2015 (I88ae7f4ad) to turn off automatic
reindexing.

This setting however, leads to inconsistent behavior: Gerrit stops
updating documents when the target ref advances, so the 'mergeable'
bit in the indexed document can be stale. For large repos, it is
most likely stale. Users can still query for 'is:mergeable' though and
Gerrit happily serves that stale bit in any query response.

It is worth noting, that all of this does not affect the UI as that
sends a separate, asynchronous request to compute mergeablitly when
needed and does not rely on the index.

This commit cleans this behavior up by replacing reindexAfterRefUpdate
with indexMergeable. After this commit, there are two modes of operation:
1) Gerrit indexes 'mergable' and keeps it up to date when the target ref
   advances. Gerrit allows queries for 'is:mergeable'.
2) Gerrit does not index 'mergeable' at all. Gerrit does not allow
   queries for 'is:mergeable'.

This way, users always get a consistent and correct result.

Change-Id: I053af1b99616920db7f0dda8f8ec770e8683df5c
This commit is contained in:
Patrick Hiesel
2019-11-13 08:08:19 -08:00
parent 01f2af22af
commit d03143571d
25 changed files with 194 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
java/com/google/gerrit/index/Schema.java
View File

@@ -15,11 +15,12 @@
package com.google.gerrit.index;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.toImmutableList;
import com.google.common.base.MoreObjects;
import com.google.common.collect.FluentIterable;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import java.util.ArrayList;
import java.util.Arrays;
@@ -181,12 +182,17 @@ public class Schema<T> {
* <p>Null values are omitted, as are fields which cause errors, which are logged.
*
* @param obj input object.
* @param skipFields set of field names to skip when indexing the document
* @return all non-null field values from the object.
*/
public final Iterable<Values<T>> buildFields(T obj) {
return FluentIterable.from(fields.values())
.transform(
public final Iterable<Values<T>> buildFields(T obj, ImmutableSet<String> skipFields) {
return fields.values().stream()
.map(
f -> {
if (skipFields.contains(f.getName())) {
return null;
}
Object v;
try {
v = f.get(obj);
@@ -203,7 +209,8 @@ public class Schema<T> {
return new Values<>(f, Collections.singleton(v));
}
})
.filter(Objects::nonNull);
.filter(Objects::nonNull)
.collect(toImmutableList());
}
@Override