Add REST endpoint to check for auto-closeable changes in a project

The consistency check searches for open changes that can be auto-closed
because a commit of the change is already contained in the destination
branch or because the destination branch contains a commit with the same
Change-Id. Normally Gerrit auto-closes such changes when the
corresponding commits are pushed directly to the repository. However if
auto-closing on direct push fails and the push is still successful
change states get inconsistent (changes that are already part of the
destination branch are still open). This consistency check is intended
to detect and repair this situation.

Change-Id: I79e46f70d59258ef972ba5d72e40d952fccc0c0f
Signed-off-by: Edwin Kempin <ekempin@google.com>
This commit is contained in:
Edwin Kempin 2018-01-12 15:20:48 +01:00
parent b9287e0a0f
commit bf9df393af
11 changed files with 999 additions and 11 deletions

View File

@ -1397,6 +1397,95 @@ returns immediately.
Content-Disposition: attachment
----
[[check]]
=== Check project consistency
Performs consistency checks on the project.
Which consistency checks should be performed is controlled by the
link:#check-project-input[CheckProjectInput] entity in the request
body.
The following consistency checks are supported:
[[auto-closeable-changes-check]]
--
* AutoCloseableChangesCheck: Searches for open changes that can be
auto-closed because a patch set of the change is already contained in
the destination branch or because the destination branch contains a
commit with the same Change-Id. Normally Gerrit auto-closes such
changes when the corresponding commits are pushed directly to the
repository. However if a branch is updated behind Gerrit's back or if
auto-closing changes fails (and the push is still successful) change
states can get inconsistent (changes that are already part of the
destination branch are still open). This consistency check is
intended to detect and repair this situation.
--
To fix any problems that can be fixed automatically set the `fix` field
in the inputs for the consistency checks to `true`.
This REST endpoint requires the
link:access-control.html#capability_administrateServer[Administrate Server]
global capability.
.Request
----
POST /projects/MyProject/check HTTP/1.0
Content-Type: application/json; charset=UTF-8
{
"auto_closeable_changes_check": {
"fix": true,
"branch": "refs/heads/master",
"max_commits": 100
}
}
----
As response a link:#check-project-result-info[CheckProjectResultInfo]
entity is returned that results for the consistency checks.
.Response
----
HTTP/1.1 200 OK
Content-Disposition: attachment
Content-Type: application/json; charset=UTF-8
)]}'
{
"auto_closeable_changes_check_result": {
"auto_closeable_changes": {
"refs/heads/master": [
{
"id": "myProject~master~I8473b95934b5732ac55d26311a706c9c2bde9940",
"project": "myProject",
"branch": "master",
"change_id": "I8473b95934b5732ac55d26311a706c9c2bde9940",
"subject": "Implementing Feature X",
"status": "NEW",
"created": "2013-02-01 09:59:32.126000000",
"updated": "2013-02-21 11:16:36.775000000",
"insertions": 34,
"deletions": 101,
"_number": 3965,
"owner": {
"name": "John Doe"
},
"problems": [
{
"message": "Patch set 1 (2f15e416237ed9b561199f24184f5f5d2708c584) is merged into destination ref refs/heads/master (2f15e416237ed9b561199f24184f5f5d2708c584), but change status is NEW",
"status": "FIXED",
"outcome": "Marked change as merged"
}
]
}
]
}
}
}
----
[[branch-endpoints]]
== Branch Endpoints
@ -2792,6 +2881,52 @@ access
check. This defaults to `read`. If given, it `ref` must be given too.
|=========================================
[[auto_closeable_changes_check_input]]
=== AutoCloseableChangesCheckInput
The `AutoCloseableChangesCheckInput` entity contains options for running
the link:#auto-closeable-changes-check[AutoCloseableChangesCheck].
[options="header",cols="1,^2,4"]
|=============================
|Field Name ||Description
|`fix` |optional|
Whether auto-closeable changes should be closed automatically.
|`branch` ||
The branch for which the link:#auto-closeable-changes-check[
AutoCloseableChangesCheck] should be performed. The 'refs/heads/'
prefix for the branch name can be omitted.
|`skip_commits` |optional|
Number of commits that should be skipped when walking the commits of
the branch.
|`max_commits` |optional|
Maximum number of commits to walk. If not specified this defaults to
10,000 commits. 10,000 is also the maximum that can be set.
Auto-closing changes is an expensive operation and the more commits
are walked the slower it gets. This is why you should avoid walking too
many commits.
|=============================
[[auto_closeable_changes_check_result]]
=== AutoCloseableChangesCheckResult
The `AutoCloseableChangesCheckResult` entity contains the results of
running the link:#auto-closeable-changes-check[AutoCloseableChangesCheck]
on a project.
[options="header",cols="1,6"]
|====================================
|Field Name |Description
|`auto_closeable_changes`|
Changes that can be auto-closed as list of
link:rest-api-changes.html#change-info[ChangeInfo] entities. For each
returned link:rest-api-changes.html#change-info[ChangeInfo] entity the
`problems` field is populated that includes details about the detected
issues. If `fix` in the link:#auto_closeable_changes_check_input[
AutoCloseableChangesCheckInput] was set to `true`, `status` and
`outcome` in link:rest-api-changes.html#problem-info[ProblemInfo] are
populated. If the status says `FIXED` Gerrit was able to auto-close the
change now.
|====================================
[[ban-input]]
=== BanInput
The `BanInput` entity contains information for banning commits in a
@ -2849,6 +2984,36 @@ The base revision of the new branch. +
If not set, `HEAD` will be used as base revision.
|=======================
[[check-project-input]]
=== CheckProjectInput
The `CheckProjectInput` entity contains information about which
consistency checks should be run on a project.
[options="header",cols="1,^2,4"]
|===========================================
|Field Name ||Description
|`auto_closeable_changes_check`|optional|
Parameters for the link:#auto-closeable-changes-check[
AutoCloseableChangesCheck] as
link:rest-api-changes.html#auto_closeable_changes_check_input[
AutoCloseableChangesCheckInput] entity.
|===========================================
[[check-project-result-info]]
=== CheckProjectResultInfo
The `CheckProjectResultInfo` entity contains results for consistency
checks that have been run on a project.
[options="header",cols="1,^2,4"]
|==================================================
|Field Name ||Description
|`auto_closeable_changes_check_result`|optional|
Results for the link:#auto-closeable-changes-check[
AutoCloseableChangesCheck] as
link:rest-api-changes.html#auto_closeable_changes_check_result[
AutoCloseableChangesCheckResult] entity.
|==================================================
[[config-info]]
=== ConfigInfo
The `ConfigInfo` entity contains information about the effective project

View File

@ -0,0 +1,33 @@
// Copyright (C) 2018 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.extensions.api.projects;
public class CheckProjectInput {
public AutoCloseableChangesCheckInput autoCloseableChangesCheck;
public static class AutoCloseableChangesCheckInput {
/** Whether auto-closeable changes should be fixed by setting their status to MERGED. */
public Boolean fix;
/** Branch that should be checked for auto-closeable changes. */
public String branch;
/** Number of commits to skip. */
public Integer skipCommits;
/** Maximum number of commits to walk. */
public Integer maxCommits;
}
}

View File

@ -0,0 +1,26 @@
// Copyright (C) 2018 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.extensions.api.projects;
import com.google.gerrit.extensions.common.ChangeInfo;
import java.util.List;
public class CheckProjectResultInfo {
public AutoCloseableChangesCheckResult autoCloseableChangesCheckResult;
public static class AutoCloseableChangesCheckResult {
public List<ChangeInfo> autoCloseableChanges;
}
}

View File

@ -43,6 +43,8 @@ public interface ProjectApi {
AccessCheckInfo checkAccess(AccessCheckInput in) throws RestApiException;
CheckProjectResultInfo check(CheckProjectInput in) throws RestApiException;
ConfigInfo config() throws RestApiException;
ConfigInfo config(ConfigInput in) throws RestApiException;
@ -242,6 +244,11 @@ public interface ProjectApi {
throw new NotImplementedException();
}
@Override
public CheckProjectResultInfo check(CheckProjectInput in) throws RestApiException {
throw new NotImplementedException();
}
@Override
public ConfigInfo config() throws RestApiException {
throw new NotImplementedException();

View File

@ -105,6 +105,18 @@ public abstract class Predicate<T> {
return getChildren().get(i);
}
/** Get the number of leaf terms in this predicate. */
public int getLeafCount() {
int leafCount = 0;
for (Predicate<?> childPredicate : getChildren()) {
if (childPredicate instanceof IndexPredicate) {
leafCount++;
}
leafCount += childPredicate.getLeafCount();
}
return leafCount;
}
/** Create a copy of this predicate, with new children. */
public abstract Predicate<T> copy(Collection<? extends Predicate<T>> children);

View File

@ -24,6 +24,8 @@ import com.google.gerrit.extensions.api.config.AccessCheckInfo;
import com.google.gerrit.extensions.api.config.AccessCheckInput;
import com.google.gerrit.extensions.api.projects.BranchApi;
import com.google.gerrit.extensions.api.projects.BranchInfo;
import com.google.gerrit.extensions.api.projects.CheckProjectInput;
import com.google.gerrit.extensions.api.projects.CheckProjectResultInfo;
import com.google.gerrit.extensions.api.projects.ChildProjectApi;
import com.google.gerrit.extensions.api.projects.CommitApi;
import com.google.gerrit.extensions.api.projects.ConfigInfo;
@ -53,6 +55,7 @@ import com.google.gerrit.server.permissions.GlobalPermission;
import com.google.gerrit.server.permissions.PermissionBackend;
import com.google.gerrit.server.project.ProjectJson;
import com.google.gerrit.server.project.ProjectResource;
import com.google.gerrit.server.restapi.project.Check;
import com.google.gerrit.server.restapi.project.CheckAccess;
import com.google.gerrit.server.restapi.project.ChildProjectsCollection;
import com.google.gerrit.server.restapi.project.CommitsCollection;
@ -115,6 +118,7 @@ public class ProjectApiImpl implements ProjectApi {
private final CommitApiImpl.Factory commitApi;
private final DashboardApiImpl.Factory dashboardApi;
private final CheckAccess checkAccess;
private final Check check;
private final Provider<ListDashboards> listDashboards;
private final GetHead getHead;
private final SetHead setHead;
@ -148,6 +152,7 @@ public class ProjectApiImpl implements ProjectApi {
CommitApiImpl.Factory commitApi,
DashboardApiImpl.Factory dashboardApi,
CheckAccess checkAccess,
Check check,
Provider<ListDashboards> listDashboards,
GetHead getHead,
SetHead setHead,
@ -181,6 +186,7 @@ public class ProjectApiImpl implements ProjectApi {
commitApi,
dashboardApi,
checkAccess,
check,
listDashboards,
getHead,
setHead,
@ -216,6 +222,7 @@ public class ProjectApiImpl implements ProjectApi {
CommitApiImpl.Factory commitApi,
DashboardApiImpl.Factory dashboardApi,
CheckAccess checkAccess,
Check check,
Provider<ListDashboards> listDashboards,
GetHead getHead,
SetHead setHead,
@ -249,6 +256,7 @@ public class ProjectApiImpl implements ProjectApi {
commitApi,
dashboardApi,
checkAccess,
check,
listDashboards,
getHead,
setHead,
@ -284,6 +292,7 @@ public class ProjectApiImpl implements ProjectApi {
CommitApiImpl.Factory commitApi,
DashboardApiImpl.Factory dashboardApi,
CheckAccess checkAccess,
Check check,
Provider<ListDashboards> listDashboards,
GetHead getHead,
SetHead setHead,
@ -316,6 +325,7 @@ public class ProjectApiImpl implements ProjectApi {
this.createAccessChange = createAccessChange;
this.dashboardApi = dashboardApi;
this.checkAccess = checkAccess;
this.check = check;
this.listDashboards = listDashboards;
this.getHead = getHead;
this.setHead = setHead;
@ -371,15 +381,6 @@ public class ProjectApiImpl implements ProjectApi {
}
}
@Override
public AccessCheckInfo checkAccess(AccessCheckInput in) throws RestApiException {
try {
return checkAccess.apply(checkExists(), in);
} catch (Exception e) {
throw asRestApiException("Cannot check access rights", e);
}
}
@Override
public ProjectAccessInfo access(ProjectAccessInput p) throws RestApiException {
try {
@ -398,6 +399,24 @@ public class ProjectApiImpl implements ProjectApi {
}
}
@Override
public AccessCheckInfo checkAccess(AccessCheckInput in) throws RestApiException {
try {
return checkAccess.apply(checkExists(), in);
} catch (Exception e) {
throw asRestApiException("Cannot check access rights", e);
}
}
@Override
public CheckProjectResultInfo check(CheckProjectInput in) throws RestApiException {
try {
return check.apply(checkExists(), in);
} catch (Exception e) {
throw asRestApiException("Cannot check project", e);
}
}
@Override
public void description(DescriptionInput in) throws RestApiException {
try {

View File

@ -0,0 +1,333 @@
// Copyright (C) 2018 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.project;
import static com.google.gerrit.common.FooterConstants.CHANGE_ID;
import static com.google.gerrit.index.query.Predicate.and;
import static com.google.gerrit.index.query.Predicate.or;
import static com.google.gerrit.server.query.change.ChangeStatusPredicate.open;
import static java.util.stream.Collectors.toSet;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Strings;
import com.google.common.base.Throwables;
import com.google.common.collect.ImmutableList;
import com.google.gerrit.extensions.api.changes.FixInput;
import com.google.gerrit.extensions.api.projects.CheckProjectInput;
import com.google.gerrit.extensions.api.projects.CheckProjectInput.AutoCloseableChangesCheckInput;
import com.google.gerrit.extensions.api.projects.CheckProjectResultInfo;
import com.google.gerrit.extensions.api.projects.CheckProjectResultInfo.AutoCloseableChangesCheckResult;
import com.google.gerrit.extensions.client.ListChangesOption;
import com.google.gerrit.extensions.common.ChangeInfo;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.ResourceConflictException;
import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.extensions.restapi.UnprocessableEntityException;
import com.google.gerrit.index.IndexConfig;
import com.google.gerrit.index.query.Predicate;
import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.client.RefNames;
import com.google.gerrit.server.change.ChangeJson;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.index.change.ChangeField;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.gerrit.server.query.change.ChangeIdPredicate;
import com.google.gerrit.server.query.change.CommitPredicate;
import com.google.gerrit.server.query.change.InternalChangeQuery;
import com.google.gerrit.server.query.change.ProjectPredicate;
import com.google.gerrit.server.query.change.RefPredicate;
import com.google.gerrit.server.update.RetryHelper;
import com.google.gerrit.server.update.RetryHelper.ActionType;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevSort;
import org.eclipse.jgit.revwalk.RevWalk;
@Singleton
public class ProjectsConsistencyChecker {
@VisibleForTesting public static final int AUTO_CLOSE_MAX_COMMITS_LIMIT = 10000;
private final GitRepositoryManager repoManager;
private final RetryHelper retryHelper;
private final Provider<InternalChangeQuery> changeQueryProvider;
private final ChangeJson.Factory changeJsonFactory;
private final IndexConfig indexConfig;
@Inject
ProjectsConsistencyChecker(
GitRepositoryManager repoManager,
RetryHelper retryHelper,
Provider<InternalChangeQuery> changeQueryProvider,
ChangeJson.Factory changeJsonFactory,
IndexConfig indexConfig) {
this.repoManager = repoManager;
this.retryHelper = retryHelper;
this.changeQueryProvider = changeQueryProvider;
this.changeJsonFactory = changeJsonFactory;
this.indexConfig = indexConfig;
}
public CheckProjectResultInfo check(Project.NameKey projectName, CheckProjectInput input)
throws IOException, OrmException, RestApiException {
CheckProjectResultInfo r = new CheckProjectResultInfo();
if (input.autoCloseableChangesCheck != null) {
r.autoCloseableChangesCheckResult =
checkForAutoCloseableChanges(projectName, input.autoCloseableChangesCheck);
}
return r;
}
private AutoCloseableChangesCheckResult checkForAutoCloseableChanges(
Project.NameKey projectName, AutoCloseableChangesCheckInput input)
throws IOException, OrmException, RestApiException {
AutoCloseableChangesCheckResult r = new AutoCloseableChangesCheckResult();
if (Strings.isNullOrEmpty(input.branch)) {
throw new BadRequestException("branch is required");
}
boolean fix = input.fix != null ? input.fix : false;
if (input.maxCommits != null && input.maxCommits > AUTO_CLOSE_MAX_COMMITS_LIMIT) {
throw new BadRequestException(
"max commits can at most be set to " + AUTO_CLOSE_MAX_COMMITS_LIMIT);
}
int maxCommits = input.maxCommits != null ? input.maxCommits : AUTO_CLOSE_MAX_COMMITS_LIMIT;
// Result that we want to return to the client.
List<ChangeInfo> autoCloseableChanges = new ArrayList<>();
// Remember the change IDs of all changes that we already included into the result, so that we
// can avoid including the same change twice.
Set<Change.Id> seenChanges = new HashSet<>();
try (Repository repo = repoManager.openRepository(projectName);
RevWalk rw = new RevWalk(repo)) {
String branch = RefNames.fullName(input.branch);
Ref ref = repo.exactRef(branch);
if (ref == null) {
throw new UnprocessableEntityException(
String.format("branch '%s' not found", input.branch));
}
rw.reset();
rw.markStart(rw.parseCommit(ref.getObjectId()));
rw.sort(RevSort.TOPO);
rw.sort(RevSort.REVERSE);
// Cache the SHA1's of all merged commits. We need this for knowing which commit merged the
// change when auto-closing changes by commit.
List<ObjectId> mergedSha1s = new ArrayList<>();
// Cache the Change-Id to commit SHA1 mapping for all Change-Id's that we find in merged
// commits. We need this for knowing which commit merged the change when auto-closing
// changes by Change-Id.
Map<Change.Key, ObjectId> changeIdToMergedSha1 = new HashMap<>();
// Base predicate which is fixed for every change query.
Predicate<ChangeData> basePredicate =
and(new ProjectPredicate(projectName.get()), new RefPredicate(branch), open());
int maxLeafPredicates = indexConfig.maxTerms() - basePredicate.getLeafCount();
// List of predicates by which we want to find open changes for the branch. These predicates
// will be combined with the 'or' operator.
List<Predicate<ChangeData>> predicates = new ArrayList<>(maxLeafPredicates);
RevCommit commit;
int skippedCommits = 0;
int walkedCommits = 0;
while ((commit = rw.next()) != null) {
if (input.skipCommits != null && skippedCommits < input.skipCommits) {
skippedCommits++;
continue;
}
if (walkedCommits >= maxCommits) {
break;
}
walkedCommits++;
ObjectId commitId = commit.copy();
mergedSha1s.add(commitId);
// Consider all Change-Id lines since this is what ReceiveCommits#autoCloseChanges does.
List<String> changeIds = commit.getFooterLines(CHANGE_ID);
// Number of predicates that we need to add for this commit, 1 per Change-Id plus one for
// the commit.
int newPredicatesCount = changeIds.size() + 1;
// We accumulated the max number of query terms that can be used in one query, execute
// the query and start a new one.
if (predicates.size() + newPredicatesCount > maxLeafPredicates) {
autoCloseableChanges.addAll(
executeQueryAndAutoCloseChanges(
basePredicate, seenChanges, predicates, fix, changeIdToMergedSha1, mergedSha1s));
mergedSha1s.clear();
changeIdToMergedSha1.clear();
predicates.clear();
if (newPredicatesCount > maxLeafPredicates) {
// Whee, a single commit generates more than maxLeafPredicates predicates. Give up.
throw new ResourceConflictException(
String.format(
"commit %s contains more Change-Ids than we can handle", commit.name()));
}
}
changeIds.forEach(
changeId -> {
// It can happen that there are multiple merged commits with the same Change-Id
// footer (e.g. if a change was cherry-picked to a stable branch stable branch which
// then got merged back into master, or just by directly pushing several commits
// with the same Change-Id). In this case it is hard to say which of the commits
// should be used to auto-close an open change with the same Change-Id (and branch).
// Possible approaches are:
// 1. use the oldest commit with that Change-Id to auto-close the change
// 2. use the newest commit with that Change-Id to auto-close the change
// Possibility 1. has the disadvantage that the commit may have been merged before
// the change was created in which case it is strange how it could auto-close the
// change. Also this strategy would require to walk all commits since otherwise we
// cannot be sure that we have seen the oldest commit with that Change-Id.
// Possibility 2 has the disadvantage that it doesn't produce the same result as if
// auto-closing on push would have worked, since on direct push the first commit with
// a Change-Id of an open change would have closed that change. Also for this we
// would need to consider all commits that are skipped.
// Since both possibilities are not perfect and require extra effort we choose the
// easiest approach, which is use the newest commit with that Change-Id that we have
// seen (this means we ignore skipped commits). This should be okay since the
// important thing for callers is that auto-closable changes are closed. Which of the
// commits is used to auto-close a change if there are several candidates is of minor
// importance and hence can be non-deterministic.
Change.Key changeKey = new Change.Key(changeId);
if (!changeIdToMergedSha1.containsKey(changeKey)) {
changeIdToMergedSha1.put(changeKey, commitId);
}
// Find changes that have a matching Change-Id.
predicates.add(new ChangeIdPredicate(changeId));
});
// Find changes that have a matching commit.
predicates.add(new CommitPredicate(commit.name()));
}
if (predicates.size() > 0) {
// Execute the query with the remaining predicates that were collected.
autoCloseableChanges.addAll(
executeQueryAndAutoCloseChanges(
basePredicate, seenChanges, predicates, fix, changeIdToMergedSha1, mergedSha1s));
}
}
r.autoCloseableChanges = autoCloseableChanges;
return r;
}
private List<ChangeInfo> executeQueryAndAutoCloseChanges(
Predicate<ChangeData> basePredicate,
Set<Change.Id> seenChanges,
List<Predicate<ChangeData>> predicates,
boolean fix,
Map<Change.Key, ObjectId> changeIdToMergedSha1,
List<ObjectId> mergedSha1s)
throws OrmException {
if (predicates.isEmpty()) {
return ImmutableList.of();
}
try {
List<ChangeData> queryResult =
retryHelper.execute(
ActionType.INDEX_QUERY,
() -> {
// Execute the query.
return changeQueryProvider
.get()
.setRequestedFields(ChangeField.CHANGE, ChangeField.PATCH_SET)
.query(and(basePredicate, or(predicates)));
},
OrmException.class::isInstance);
// Result for this query that we want to return to the client.
List<ChangeInfo> autoCloseableChangesByBranch = new ArrayList<>();
for (ChangeData autoCloseableChange : queryResult) {
// Skip changes that we have already processed, either by this query or by
// earlier queries.
if (seenChanges.add(autoCloseableChange.getId())) {
retryHelper.execute(
ActionType.CHANGE_UPDATE,
() -> {
// Auto-close by change
if (changeIdToMergedSha1.containsKey(autoCloseableChange.change().getKey())) {
autoCloseableChangesByBranch.add(
changeJson(
fix, changeIdToMergedSha1.get(autoCloseableChange.change().getKey()))
.format(autoCloseableChange));
return null;
}
// Auto-close by commit
for (ObjectId patchSetSha1 :
autoCloseableChange
.patchSets()
.stream()
.map(ps -> ObjectId.fromString(ps.getRevision().get()))
.collect(toSet())) {
if (mergedSha1s.contains(patchSetSha1)) {
autoCloseableChangesByBranch.add(
changeJson(fix, patchSetSha1).format(autoCloseableChange));
break;
}
}
return null;
},
OrmException.class::isInstance);
}
}
return autoCloseableChangesByBranch;
} catch (Exception e) {
Throwables.throwIfUnchecked(e);
Throwables.throwIfInstanceOf(e, OrmException.class);
throw new OrmException(e);
}
}
private ChangeJson changeJson(Boolean fix, ObjectId mergedAs) {
ChangeJson changeJson = changeJsonFactory.create(ListChangesOption.CHECK);
if (fix != null && fix.booleanValue()) {
FixInput fixInput = new FixInput();
fixInput.expectMergedAs = mergedAs.name();
changeJson.fix(fixInput);
}
return changeJson;
}
}

View File

@ -138,7 +138,21 @@ public class InternalChangeQuery extends InternalQuery<ChangeData> {
}
public List<ChangeData> byBranchKey(Branch.NameKey branch, Change.Key key) throws OrmException {
return query(and(ref(branch), project(branch.getParentKey()), change(key)));
return query(byBranchKeyPred(branch, key));
}
public List<ChangeData> byBranchKeyOpen(Project.NameKey project, String branch, Change.Key key)
throws OrmException {
return query(and(byBranchKeyPred(new Branch.NameKey(project, branch), key), open()));
}
public static Predicate<ChangeData> byBranchKeyOpenPred(
Project.NameKey project, String branch, Change.Key key) {
return and(byBranchKeyPred(new Branch.NameKey(project, branch), key), open());
}
private static Predicate<ChangeData> byBranchKeyPred(Branch.NameKey branch, Change.Key key) {
return and(ref(branch), project(branch.getParentKey()), change(key));
}
public List<ChangeData> byProject(Project.NameKey project) throws OrmException {
@ -264,13 +278,28 @@ public class InternalChangeQuery extends InternalQuery<ChangeData> {
public List<ChangeData> byBranchCommit(String project, String branch, String hash)
throws OrmException {
return query(and(new ProjectPredicate(project), new RefPredicate(branch), commit(hash)));
return query(byBranchCommitPred(project, branch, hash));
}
public List<ChangeData> byBranchCommit(Branch.NameKey branch, String hash) throws OrmException {
return byBranchCommit(branch.getParentKey().get(), branch.get(), hash);
}
public List<ChangeData> byBranchCommitOpen(String project, String branch, String hash)
throws OrmException {
return query(and(byBranchCommitPred(project, branch, hash), open()));
}
public static Predicate<ChangeData> byBranchCommitOpenPred(
Project.NameKey project, String branch, String hash) {
return and(byBranchCommitPred(project.get(), branch, hash), open());
}
private static Predicate<ChangeData> byBranchCommitPred(
String project, String branch, String hash) {
return and(new ProjectPredicate(project), new RefPredicate(branch), commit(hash));
}
public List<ChangeData> bySubmissionId(String cs) throws OrmException {
if (Strings.isNullOrEmpty(cs)) {
return Collections.emptyList();

View File

@ -0,0 +1,48 @@
// Copyright (C) 2018 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.restapi.project;
import com.google.gerrit.extensions.api.projects.CheckProjectInput;
import com.google.gerrit.extensions.api.projects.CheckProjectResultInfo;
import com.google.gerrit.extensions.restapi.AuthException;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.ResourceConflictException;
import com.google.gerrit.extensions.restapi.RestModifyView;
import com.google.gerrit.server.permissions.GlobalPermission;
import com.google.gerrit.server.permissions.PermissionBackend;
import com.google.gerrit.server.project.ProjectResource;
import com.google.gerrit.server.project.ProjectsConsistencyChecker;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@Singleton
public class Check implements RestModifyView<ProjectResource, CheckProjectInput> {
private final PermissionBackend permissionBackend;
private final ProjectsConsistencyChecker projectsConsistencyChecker;
@Inject
Check(
PermissionBackend permissionBackend, ProjectsConsistencyChecker projectsConsistencyChecker) {
this.permissionBackend = permissionBackend;
this.projectsConsistencyChecker = projectsConsistencyChecker;
}
@Override
public CheckProjectResultInfo apply(ProjectResource rsrc, CheckProjectInput input)
throws AuthException, BadRequestException, ResourceConflictException, Exception {
permissionBackend.user(rsrc.getUser()).check(GlobalPermission.ADMINISTRATE_SERVER);
return projectsConsistencyChecker.check(rsrc.getNameKey(), input);
}
}

View File

@ -54,6 +54,8 @@ public class Module extends RestApiModule {
post(PROJECT_KIND, "check.access").to(CheckAccess.class);
get(PROJECT_KIND, "check.access").to(CheckAccessReadView.class);
post(PROJECT_KIND, "check").to(Check.class);
get(PROJECT_KIND, "parent").to(GetParent.class);
put(PROJECT_KIND, "parent").to(SetParent.class);

View File

@ -0,0 +1,314 @@
// Copyright (C) 2018 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.acceptance.api.project;
import static com.google.common.truth.Truth.assertThat;
import static com.google.gerrit.acceptance.GitUtil.pushHead;
import static java.util.stream.Collectors.toList;
import static java.util.stream.Collectors.toSet;
import com.google.common.collect.Iterables;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.PushOneCommit;
import com.google.gerrit.extensions.api.projects.CheckProjectInput;
import com.google.gerrit.extensions.api.projects.CheckProjectInput.AutoCloseableChangesCheckInput;
import com.google.gerrit.extensions.api.projects.CheckProjectResultInfo;
import com.google.gerrit.extensions.client.ChangeStatus;
import com.google.gerrit.extensions.client.InheritableBoolean;
import com.google.gerrit.extensions.common.ChangeInfo;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.UnprocessableEntityException;
import com.google.gerrit.server.project.ProjectsConsistencyChecker;
import org.eclipse.jgit.internal.storage.dfs.InMemoryRepository;
import org.eclipse.jgit.junit.TestRepository;
import org.eclipse.jgit.lib.PersonIdent;
import org.eclipse.jgit.revwalk.RevCommit;
import org.junit.Before;
import org.junit.Test;
public class CheckProjectIT extends AbstractDaemonTest {
private TestRepository<InMemoryRepository> serverSideTestRepo;
@Before
public void setUp() throws Exception {
serverSideTestRepo =
new TestRepository<>((InMemoryRepository) repoManager.openRepository(project));
}
@Test
public void noProblem() throws Exception {
PushOneCommit.Result r = createChange("refs/for/master");
String branch = r.getChange().change().getDest().get();
ChangeInfo info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectResultInfo checkResult =
gApi.projects().name(project.get()).check(checkProjectInputForAutoCloseableCheck(branch));
assertThat(checkResult.autoCloseableChangesCheckResult.autoCloseableChanges).isEmpty();
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
}
@Test
public void detectAutoCloseableChangeByCommit() throws Exception {
RevCommit commit = pushCommitWithoutChangeIdForReview();
ChangeInfo change =
Iterables.getOnlyElement(gApi.changes().query("commit:" + commit.name()).get());
String branch = "refs/heads/master";
serverSideTestRepo.branch(branch).update(testRepo.getRevWalk().parseCommit(commit));
ChangeInfo info = gApi.changes().id(change._number).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectResultInfo checkResult =
gApi.projects().name(project.get()).check(checkProjectInputForAutoCloseableCheck(branch));
assertThat(
checkResult
.autoCloseableChangesCheckResult
.autoCloseableChanges
.stream()
.map(i -> i._number)
.collect(toList()))
.containsExactly(change._number);
info = gApi.changes().id(change._number).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
}
@Test
public void fixAutoCloseableChangeByCommit() throws Exception {
RevCommit commit = pushCommitWithoutChangeIdForReview();
ChangeInfo change =
Iterables.getOnlyElement(gApi.changes().query("commit:" + commit.name()).get());
String branch = "refs/heads/master";
serverSideTestRepo.branch(branch).update(commit);
ChangeInfo info = gApi.changes().id(change._number).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectInput input = checkProjectInputForAutoCloseableCheck(branch);
input.autoCloseableChangesCheck.fix = true;
CheckProjectResultInfo checkResult = gApi.projects().name(project.get()).check(input);
assertThat(
checkResult
.autoCloseableChangesCheckResult
.autoCloseableChanges
.stream()
.map(i -> i._number)
.collect(toSet()))
.containsExactly(change._number);
info = gApi.changes().id(change._number).info();
assertThat(info.status).isEqualTo(ChangeStatus.MERGED);
}
@Test
public void detectAutoCloseableChangeByChangeId() throws Exception {
PushOneCommit.Result r = createChange("refs/for/master");
String branch = r.getChange().change().getDest().get();
RevCommit amendedCommit = serverSideTestRepo.amend(r.getCommit()).create();
serverSideTestRepo.branch(branch).update(amendedCommit);
ChangeInfo info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectResultInfo checkResult =
gApi.projects().name(project.get()).check(checkProjectInputForAutoCloseableCheck(branch));
assertThat(
checkResult
.autoCloseableChangesCheckResult
.autoCloseableChanges
.stream()
.map(i -> i._number)
.collect(toSet()))
.containsExactly(r.getChange().getId().get());
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
}
@Test
public void fixAutoCloseableChangeByChangeId() throws Exception {
PushOneCommit.Result r = createChange("refs/for/master");
String branch = r.getChange().change().getDest().get();
RevCommit amendedCommit = serverSideTestRepo.amend(r.getCommit()).create();
serverSideTestRepo.branch(branch).update(amendedCommit);
ChangeInfo info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectInput input = checkProjectInputForAutoCloseableCheck(branch);
input.autoCloseableChangesCheck.fix = true;
CheckProjectResultInfo checkResult = gApi.projects().name(project.get()).check(input);
assertThat(
checkResult
.autoCloseableChangesCheckResult
.autoCloseableChanges
.stream()
.map(i -> i._number)
.collect(toSet()))
.containsExactly(r.getChange().getId().get());
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.MERGED);
}
@Test
public void maxCommits() throws Exception {
PushOneCommit.Result r = createChange("refs/for/master");
String branch = r.getChange().change().getDest().get();
RevCommit amendedCommit = serverSideTestRepo.amend(r.getCommit()).create();
serverSideTestRepo.branch(branch).update(amendedCommit);
serverSideTestRepo.commit(amendedCommit);
ChangeInfo info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectInput input = checkProjectInputForAutoCloseableCheck(branch);
input.autoCloseableChangesCheck.fix = true;
input.autoCloseableChangesCheck.maxCommits = 1;
CheckProjectResultInfo checkResult = gApi.projects().name(project.get()).check(input);
assertThat(checkResult.autoCloseableChangesCheckResult.autoCloseableChanges).isEmpty();
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
input.autoCloseableChangesCheck.maxCommits = 2;
checkResult = gApi.projects().name(project.get()).check(input);
assertThat(
checkResult
.autoCloseableChangesCheckResult
.autoCloseableChanges
.stream()
.map(i -> i._number)
.collect(toSet()))
.containsExactly(r.getChange().getId().get());
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.MERGED);
}
@Test
public void skipCommits() throws Exception {
PushOneCommit.Result r = createChange("refs/for/master");
String branch = r.getChange().change().getDest().get();
RevCommit amendedCommit = serverSideTestRepo.amend(r.getCommit()).create();
serverSideTestRepo.branch(branch).update(amendedCommit);
serverSideTestRepo.commit(amendedCommit);
ChangeInfo info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
CheckProjectInput input = checkProjectInputForAutoCloseableCheck(branch);
input.autoCloseableChangesCheck.fix = true;
input.autoCloseableChangesCheck.maxCommits = 1;
CheckProjectResultInfo checkResult = gApi.projects().name(project.get()).check(input);
assertThat(checkResult.autoCloseableChangesCheckResult.autoCloseableChanges).isEmpty();
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.NEW);
input.autoCloseableChangesCheck.skipCommits = 1;
checkResult = gApi.projects().name(project.get()).check(input);
assertThat(
checkResult
.autoCloseableChangesCheckResult
.autoCloseableChanges
.stream()
.map(i -> i._number)
.collect(toSet()))
.containsExactly(r.getChange().getId().get());
info = gApi.changes().id(r.getChange().getId().get()).info();
assertThat(info.status).isEqualTo(ChangeStatus.MERGED);
}
@Test
public void noBranch() throws Exception {
CheckProjectInput input = new CheckProjectInput();
input.autoCloseableChangesCheck = new AutoCloseableChangesCheckInput();
exception.expect(BadRequestException.class);
exception.expectMessage("branch is required");
gApi.projects().name(project.get()).check(input);
}
@Test
public void nonExistingBranch() throws Exception {
CheckProjectInput input = checkProjectInputForAutoCloseableCheck("non-existing");
exception.expect(UnprocessableEntityException.class);
exception.expectMessage("branch 'non-existing' not found");
gApi.projects().name(project.get()).check(input);
}
@Test
public void branchPrefixCanBeOmitted() throws Exception {
CheckProjectInput input = checkProjectInputForAutoCloseableCheck("master");
gApi.projects().name(project.get()).check(input);
}
@Test
public void setLimitForMaxCommits() throws Exception {
CheckProjectInput input = checkProjectInputForAutoCloseableCheck("refs/heads/master");
input.autoCloseableChangesCheck.maxCommits =
ProjectsConsistencyChecker.AUTO_CLOSE_MAX_COMMITS_LIMIT;
gApi.projects().name(project.get()).check(input);
}
@Test
public void tooLargeMaxCommits() throws Exception {
CheckProjectInput input = checkProjectInputForAutoCloseableCheck("refs/heads/master");
input.autoCloseableChangesCheck.maxCommits =
ProjectsConsistencyChecker.AUTO_CLOSE_MAX_COMMITS_LIMIT + 1;
exception.expect(BadRequestException.class);
exception.expectMessage(
"max commits can at most be set to "
+ ProjectsConsistencyChecker.AUTO_CLOSE_MAX_COMMITS_LIMIT);
gApi.projects().name(project.get()).check(input);
}
private RevCommit pushCommitWithoutChangeIdForReview() throws Exception {
setRequireChangeId(InheritableBoolean.FALSE);
RevCommit commit =
testRepo
.branch("HEAD")
.commit()
.message("A change")
.author(admin.getIdent())
.committer(new PersonIdent(admin.getIdent(), testRepo.getDate()))
.create();
pushHead(testRepo, "refs/for/master");
return commit;
}
private static CheckProjectInput checkProjectInputForAutoCloseableCheck(String branch) {
CheckProjectInput input = new CheckProjectInput();
input.autoCloseableChangesCheck = new AutoCloseableChangesCheckInput();
input.autoCloseableChangesCheck.branch = branch;
return input;
}
}