Introduce SubmitPreview REST API call

Before submitting a whole topic or submission including parents
existed and the usage of Superproject subscriptions was low, it
was easy to predict, what would happen if a change was submitted
as it was just one change that was submitted. This is changing as
the submission process gets bigger unknowns by having coupling of
changes via their ancestors, topic submissions and superproject
subscriptions.

We would like to provide aid answering "What would happen if
I submit this change?" even more than just the submitted together
tab, as that would not show e.g. superproject subscriptions or the
underlying Git DAG. In case of merging, this also doesn't show
the exact tree afterwards.

This introduces a REST API call that will show exactly what will
happen by providing the exact Git DAG that would be produced if
a change was submitted now. Requirements for such a call:
* It has to be capable of dealing with multiple projects.
  (superproject subscriptions, submitting topics across projects)
* easy to deal with on the client side, while transporting the
  whole Git DAG.

This call returns a zip file that contains thin bundles which
can be pulled from to see the exact state after a hypothetical
submission.

As projects can be nested (e.g. project and project/foo), we cannot
just name the bundles as the projects as that may produce a
directory/file conflict inside the zip file. Projects have
limitations on how they can be named, which is enforced upon
project creation in the LocalDiskRepository class. One of the
rules is that no project name contains ".git/", which makes ".git"
a suffix that will guarantee no directory/file conflicts, which
is why the names are chosen to be "${projectname}.git"

In case of an error, no zip file is returned, but just a plain
message string.

We considered having a "dry run" submission process, that allows
submission of a change with a prefix to the target branch(es). As
an example:
    You would call submit_with_prefix(change-id=234,
    prefix=refs/testing/) which would create a branch
    refs/testing/<change target branch> which is updated
    to what that branch would look like. But also a
    superproject would get a refs/testing/<superproject branch>
    which could be inspected to what actually happened.

That has some disadvantages:
* Ref updates are expensive in Gerrit on Googles infrastructure as
  ref updates are replicated across the globe. For such testing refs
  we do not need a strong replication as we can reproduce them any
  time.
* The ACLs are unclear for these testing branches (infer them from
  the actual branches?)
* We’d need to cleanup these testing branches regularly (time to live
  of e.g. 1 week?)
* Making sure the prefix is unique for all projects that are
  involved before test submission

Change-Id: I3c976257c2b20de32373cbade9b99811586e926c
Signed-off-by: Stefan Beller <sbeller@google.com>
This commit is contained in:
Stefan Beller 2016-09-16 12:20:02 -07:00
parent 6ac85596f5
commit dfa1ef377d
13 changed files with 708 additions and 29 deletions

View File

@ -3257,6 +3257,59 @@ Query parameter `download` (e.g. `/changes/.../patch?download`)
will suggest the browser save the patch as `commitsha1.diff.base64`,
for later processing by command line tools.
[[submit-preview]]
===Submit Preview
--
'GET /changes/link:#change-id[\{change-id\}]/revisions/link:#revision-id[\{revision-id\}]/preview_submit'
--
Gets a file containing thin bundles of all modified projects if this
change was submitted. The bundles are named `${ProjectName}.git`.
Each thin bundle contains enough to construct the state in which a project would
be in if this change were submitted. The base of the thin bundles are the
current target branches, so to make use of this call in a non-racy way, first
get the bundles and then fetch all projects contained in the bundle.
(This assumes no non-fastforward pushes).
You need to give a parameter '?format=zip' or '?format=tar' to specify the
format for the outer container.
To make good use of this call, you would roughly need code as found at:
----
$ curl -Lo preview_submit_test.sh http://review.example.com:8080/tools/scripts/preview_submit_test.sh
----
.Request
----
GET /changes/myProject~master~I8473b95934b5732ac55d26311a706c9c2bde9940/revisions/current/preview_submit?zip HTTP/1.0
----
.Response
----
HTTP/1.1 200 OK
Date: Tue, 13 Sep 2016 19:13:46 GMT
Content-Disposition: attachment; filename="submit-preview-147.zip"
X-Content-Type-Options: nosniff
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Content-Type: application/x-zip
Transfer-Encoding: chunked
[binary stuff]
----
In case of an error, the response is not a zip file but a regular json response,
containing only the error message:
.Response
----
HTTP/1.1 200 OK
Content-Disposition: attachment
Content-Type: application/json; charset=UTF-8
)]}'
"Anonymous users cannot submit"
----
[[get-mergeable]]
=== Get Mergeable
--

View File

@ -26,6 +26,7 @@ import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Iterables;
import com.google.common.collect.Iterators;
import com.google.common.collect.Sets;
import com.google.common.primitives.Chars;
import com.google.gerrit.acceptance.AcceptanceTestRequestScope.Context;
@ -49,6 +50,7 @@ import com.google.gerrit.extensions.client.SubmitType;
import com.google.gerrit.extensions.common.ActionInfo;
import com.google.gerrit.extensions.common.ChangeInfo;
import com.google.gerrit.extensions.common.EditInfo;
import com.google.gerrit.extensions.restapi.BinaryResult;
import com.google.gerrit.extensions.restapi.IdString;
import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.reviewdb.client.AccountGroup;
@ -103,12 +105,19 @@ import org.eclipse.jgit.internal.storage.dfs.InMemoryRepository;
import org.eclipse.jgit.junit.TestRepository;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.NullProgressMonitor;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.lib.PersonIdent;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTree;
import org.eclipse.jgit.revwalk.RevWalk;
import org.eclipse.jgit.transport.FetchResult;
import org.eclipse.jgit.transport.RefSpec;
import org.eclipse.jgit.transport.Transport;
import org.eclipse.jgit.transport.TransportBundleStream;
import org.eclipse.jgit.transport.URIish;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
@ -120,14 +129,20 @@ import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runners.model.Statement;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
@RunWith(ConfigSuite.class)
public abstract class AbstractDaemonTest {
@ -949,7 +964,8 @@ public abstract class AbstractDaemonTest {
protected RevCommit getHead(Repository repo, String name) throws Exception {
try (RevWalk rw = new RevWalk(repo)) {
return rw.parseCommit(repo.exactRef(name).getObjectId());
Ref r = repo.exactRef(name);
return r != null ? rw.parseCommit(r.getObjectId()) : null;
}
}
@ -1008,4 +1024,73 @@ public abstract class AbstractDaemonTest {
saveProjectConfig(allProjects, cfg);
return ca;
}
/**
* Fetches each bundle into a newly cloned repository, then it applies
* the bundle, and returns the resulting tree id.
*/
protected Map<Branch.NameKey, RevTree>
fetchFromBundles(BinaryResult bundles) throws Exception {
assertThat(bundles.getContentType()).isEqualTo("application/x-zip");
File tempfile = File.createTempFile("test", null);
bundles.writeTo(new FileOutputStream(tempfile));
Map<Branch.NameKey, RevTree> ret = new HashMap<>();
try (ZipFile readback = new ZipFile(tempfile);) {
for (ZipEntry entry : ImmutableList.copyOf(
Iterators.forEnumeration(readback.entries()))) {
String bundleName = entry.getName();
InputStream bundleStream = readback.getInputStream(entry);
int len = bundleName.length();
assertThat(bundleName).endsWith(".git");
String repoName = bundleName.substring(0, len - 4);
Project.NameKey proj = new Project.NameKey(repoName);
TestRepository<?> localRepo = cloneProject(proj);
try (TransportBundleStream tbs = new TransportBundleStream(
localRepo.getRepository(), new URIish(bundleName), bundleStream);) {
FetchResult fr = tbs.fetch(NullProgressMonitor.INSTANCE,
Arrays.asList(new RefSpec("refs/*:refs/preview/*")));
for (Ref r : fr.getAdvertisedRefs()) {
String branchName = r.getName();
Branch.NameKey n = new Branch.NameKey(proj, branchName);
RevCommit c = localRepo.getRevWalk().parseCommit(r.getObjectId());
ret.put(n, c.getTree());
}
}
}
}
return ret;
}
/**
* Assert that the given branches have the given tree ids.
*/
protected void assertRevTrees(Project.NameKey proj,
Map<Branch.NameKey, RevTree> trees) throws Exception {
TestRepository<?> localRepo = cloneProject(proj);
GitUtil.fetch(localRepo, "refs/*:refs/*");
Map<String, Ref> refs = localRepo.getRepository().getAllRefs();
Map<Branch.NameKey, RevTree> refValues = new HashMap<>();
for (Branch.NameKey b : trees.keySet()) {
if (!b.getParentKey().equals(proj)) {
continue;
}
Ref r = refs.get(b.get());
assertThat(r).isNotNull();
RevWalk rw = localRepo.getRevWalk();
RevCommit c = rw.parseCommit(r.getObjectId());
refValues.put(b, c.getTree());
assertThat(trees.get(b)).isEqualTo(refValues.get(b));
}
assertThat(refValues.keySet()).containsAnyIn(trees.keySet());
}
}

View File

@ -6,6 +6,7 @@ acceptance_tests(
deps = [
':submodule_util',
':push_for_review',
'//gerrit-extension-api:api',
],
labels = ['git'],
)

View File

@ -21,15 +21,22 @@ import static com.google.gerrit.acceptance.GitUtil.getChangeId;
import com.google.gerrit.acceptance.NoHttpd;
import com.google.gerrit.extensions.api.changes.ReviewInput;
import com.google.gerrit.extensions.client.ChangeStatus;
import com.google.gerrit.extensions.client.SubmitType;
import com.google.gerrit.extensions.restapi.BinaryResult;
import com.google.gerrit.reviewdb.client.Branch;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.testutil.ConfigSuite;
import org.eclipse.jgit.junit.TestRepository;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTree;
import org.eclipse.jgit.transport.RefSpec;
import org.junit.Test;
import java.util.Map;
@NoHttpd
public class SubmoduleSubscriptionsWholeTopicMergeIT
extends AbstractSubmoduleSubscription {
@ -101,22 +108,50 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
gApi.changes().id(id2).current().review(ReviewInput.approve());
gApi.changes().id(id3).current().review(ReviewInput.approve());
BinaryResult request = gApi.changes().id(id1).current().submitPreview();
Map<Branch.NameKey, RevTree> preview =
fetchFromBundles(request);
gApi.changes().id(id1).current().submit();
ObjectId subRepoId = subRepo.git().fetch().setRemote("origin").call()
.getAdvertisedRef("refs/heads/master").getObjectId();
expectToHaveSubmoduleState(superRepo, "master",
"subscribed-to-project", subRepoId);
// As the submodules have changed commits, the superproject tree will be
// different, so we cannot directly compare the trees here, so make
// assumptions only about the changed branches:
Project.NameKey p1 = new Project.NameKey(name("super-project"));
Project.NameKey p2 = new Project.NameKey(name("subscribed-to-project"));
assertThat(preview).containsKey(
new Branch.NameKey(p1, "refs/heads/master"));
assertThat(preview).containsKey(
new Branch.NameKey(p2, "refs/heads/master"));
if (getSubmitType() == SubmitType.CHERRY_PICK) {
// each change is updated and the respective target branch is updated:
assertThat(preview).hasSize(5);
} else if (getSubmitType() == SubmitType.REBASE_IF_NECESSARY) {
// Either the first is used first as is, then the second and third need
// rebasing, or those two stay as is and the first is rebased.
// add in 2 master branches, expect 3 or 4:
assertThat(preview.size()).isAnyOf(3, 4);
} else {
assertThat(preview).hasSize(2);
}
}
@Test
public void testSubscriptionUpdateIncludingChangeInSuperproject() throws Exception {
public void testSubscriptionUpdateIncludingChangeInSuperproject()
throws Exception {
TestRepository<?> superRepo = createProjectWithPush("super-project");
TestRepository<?> subRepo = createProjectWithPush("subscribed-to-project");
allowMatchingSubmoduleSubscription("subscribed-to-project", "refs/heads/master",
"super-project", "refs/heads/master");
allowMatchingSubmoduleSubscription("subscribed-to-project",
"refs/heads/master", "super-project", "refs/heads/master");
createSubmoduleSubscription(superRepo, "master", "subscribed-to-project", "master");
createSubmoduleSubscription(superRepo, "master",
"subscribed-to-project", "master");
ObjectId subHEAD = subRepo.branch("HEAD").commit().insertChangeId()
.message("some change")
@ -310,7 +345,8 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
createSubmoduleSubscription(midRepo, "master", "bottom-project", "master");
ObjectId bottomHead =
pushChangeTo(bottomRepo, "refs/for/master", "some message", "same-topic");
pushChangeTo(bottomRepo, "refs/for/master",
"some message", "same-topic");
ObjectId topHead =
pushChangeTo(topRepo, "refs/for/master", "some message", "same-topic");
@ -322,8 +358,10 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
gApi.changes().id(id1).current().submit();
expectToHaveSubmoduleState(midRepo, "master", "bottom-project", bottomRepo, "master");
expectToHaveSubmoduleState(topRepo, "master", "mid-project", midRepo, "master");
expectToHaveSubmoduleState(midRepo, "master", "bottom-project",
bottomRepo, "master");
expectToHaveSubmoduleState(topRepo, "master", "mid-project",
midRepo, "master");
}
@Test
@ -346,7 +384,8 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
pushSubmoduleConfig(topRepo, "master", config);
ObjectId bottomHead =
pushChangeTo(bottomRepo, "refs/for/master", "some message", "same-topic");
pushChangeTo(bottomRepo, "refs/for/master",
"some message", "same-topic");
ObjectId topHead =
pushChangeTo(topRepo, "refs/for/master", "some message", "same-topic");
@ -358,13 +397,16 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
gApi.changes().id(id1).current().submit();
expectToHaveSubmoduleState(midRepo, "master", "bottom-project", bottomRepo, "master");
expectToHaveSubmoduleState(topRepo, "master", "mid-project", midRepo, "master");
expectToHaveSubmoduleState(topRepo, "master", "bottom-project", bottomRepo, "master");
expectToHaveSubmoduleState(midRepo, "master",
"bottom-project", bottomRepo, "master");
expectToHaveSubmoduleState(topRepo, "master",
"mid-project", midRepo, "master");
expectToHaveSubmoduleState(topRepo, "master",
"bottom-project", bottomRepo, "master");
}
@Test
public void testBranchCircularSubscription() throws Exception {
private String prepareBranchCircularSubscription() throws Exception {
TestRepository<?> topRepo = createProjectWithPush("top-project");
TestRepository<?> midRepo = createProjectWithPush("mid-project");
TestRepository<?> bottomRepo = createProjectWithPush("bottom-project");
@ -385,15 +427,23 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
String changeId = getChangeId(bottomRepo, bottomMasterHead).get();
approve(changeId);
exception.expectMessage("Branch level circular subscriptions detected");
exception.expectMessage("top-project,refs/heads/master");
exception.expectMessage("mid-project,refs/heads/master");
exception.expectMessage("bottom-project,refs/heads/master");
gApi.changes().id(changeId).current().submit();
return changeId;
}
assertThat(hasSubmodule(midRepo, "master", "bottom-project")).isFalse();
assertThat(hasSubmodule(topRepo, "master", "mid-project")).isFalse();
@Test
public void testBranchCircularSubscription() throws Exception {
String changeId = prepareBranchCircularSubscription();
gApi.changes().id(changeId).current().submit();
}
@Test
public void testBranchCircularSubscriptionPreview() throws Exception {
String changeId = prepareBranchCircularSubscription();
gApi.changes().id(changeId).current().submitPreview();
}
@Test
@ -401,8 +451,8 @@ public class SubmoduleSubscriptionsWholeTopicMergeIT
TestRepository<?> superRepo = createProjectWithPush("super-project");
TestRepository<?> subRepo = createProjectWithPush("subscribed-to-project");
allowMatchingSubmoduleSubscription("subscribed-to-project", "refs/heads/master",
"super-project", "refs/heads/master");
allowMatchingSubmoduleSubscription("subscribed-to-project",
"refs/heads/master", "super-project", "refs/heads/master");
allowMatchingSubmoduleSubscription("super-project", "refs/heads/dev",
"subscribed-to-project", "refs/heads/dev");

View File

@ -43,6 +43,7 @@ import com.google.gerrit.extensions.common.ChangeInfo;
import com.google.gerrit.extensions.common.ChangeMessageInfo;
import com.google.gerrit.extensions.common.LabelInfo;
import com.google.gerrit.extensions.restapi.AuthException;
import com.google.gerrit.extensions.restapi.BinaryResult;
import com.google.gerrit.extensions.restapi.ResourceConflictException;
import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.extensions.webui.UiAction;
@ -71,13 +72,16 @@ import org.eclipse.jgit.lib.PersonIdent;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTree;
import org.eclipse.jgit.revwalk.RevWalk;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.io.ByteArrayOutputStream;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@NoHttpd
public abstract class AbstractSubmit extends AbstractDaemonTest {
@ -116,9 +120,155 @@ public abstract class AbstractSubmit extends AbstractDaemonTest {
@Test
@TestProjectInput(createEmptyCommit = false)
public void submitToEmptyRepo() throws Exception {
RevCommit initialHead = getRemoteHead();
PushOneCommit.Result change = createChange();
BinaryResult request = submitPreview(change.getChangeId());
RevCommit headAfterSubmitPreview = getRemoteHead();
assertThat(headAfterSubmitPreview).isEqualTo(initialHead);
Map<Branch.NameKey, RevTree> actual =
fetchFromBundles(request);
assertThat(actual).hasSize(1);
submit(change.getChangeId());
assertThat(getRemoteHead().getId()).isEqualTo(change.getCommit());
assertRevTrees(project, actual);
}
@Test
public void submitSingleChange() throws Exception {
RevCommit initialHead = getRemoteHead();
PushOneCommit.Result change = createChange();
BinaryResult request = submitPreview(change.getChangeId());
RevCommit headAfterSubmit = getRemoteHead();
assertThat(headAfterSubmit).isEqualTo(initialHead);
assertRefUpdatedEvents();
assertChangeMergedEvents();
Map<Branch.NameKey, RevTree> actual =
fetchFromBundles(request);
if (getSubmitType() == SubmitType.CHERRY_PICK) {
// The change is updated as well:
assertThat(actual).hasSize(2);
} else {
assertThat(actual).hasSize(1);
}
submit(change.getChangeId());
assertRevTrees(project, actual);
}
@Test
public void submitMultipleChangesOtherMergeConflictPreview()
throws Exception {
RevCommit initialHead = getRemoteHead();
PushOneCommit.Result change =
createChange("Change 1", "a.txt", "content");
submit(change.getChangeId());
RevCommit headAfterFirstSubmit = getRemoteHead();
testRepo.reset(initialHead);
PushOneCommit.Result change2 = createChange("Change 2",
"a.txt", "other content");
PushOneCommit.Result change3 = createChange("Change 3", "d", "d");
PushOneCommit.Result change4 = createChange("Change 4", "e", "e");
// change 2 is not approved, but we ignore labels
approve(change3.getChangeId());
BinaryResult request = null;
String msg = null;
try {
request = submitPreview(change4.getChangeId());
} catch (Exception e) {
msg = e.getMessage();
}
if (getSubmitType() == SubmitType.CHERRY_PICK) {
Map<Branch.NameKey, RevTree> s =
fetchFromBundles(request);
submit(change4.getChangeId());
assertRevTrees(project, s);
} else if (getSubmitType() == SubmitType.FAST_FORWARD_ONLY) {
assertThat(msg).isEqualTo(
"Failed to submit 3 changes due to the following problems:\n" +
"Change " + change2.getChange().getId() + ": internal error: " +
"change not processed by merge strategy\n" +
"Change " + change3.getChange().getId() + ": internal error: " +
"change not processed by merge strategy\n" +
"Change " + change4.getChange().getId() + ": Project policy " +
"requires all submissions to be a fast-forward. Please " +
"rebase the change locally and upload again for review.");
RevCommit headAfterSubmit = getRemoteHead();
assertThat(headAfterSubmit).isEqualTo(headAfterFirstSubmit);
assertRefUpdatedEvents(initialHead, headAfterFirstSubmit);
assertChangeMergedEvents(change.getChangeId(),
headAfterFirstSubmit.name());
} else if(getSubmitType() == SubmitType.REBASE_IF_NECESSARY) {
String change2hash = change2.getChange().currentPatchSet()
.getRevision().get();
assertThat(msg).isEqualTo(
"Cannot rebase " + change2hash + ": The change could " +
"not be rebased due to a conflict during merge.");
RevCommit headAfterSubmit = getRemoteHead();
assertThat(headAfterSubmit).isEqualTo(headAfterFirstSubmit);
assertRefUpdatedEvents(initialHead, headAfterFirstSubmit);
assertChangeMergedEvents(change.getChangeId(),
headAfterFirstSubmit.name());
} else {
assertThat(msg).isEqualTo(
"Failed to submit 3 changes due to the following problems:\n" +
"Change " + change2.getChange().getId() + ": Change could not be " +
"merged due to a path conflict. Please rebase the change " +
"locally and upload the rebased commit for review.\n" +
"Change " + change3.getChange().getId() + ": Change could not be " +
"merged due to a path conflict. Please rebase the change " +
"locally and upload the rebased commit for review.\n" +
"Change " + change4.getChange().getId() + ": Change could not be " +
"merged due to a path conflict. Please rebase the change " +
"locally and upload the rebased commit for review.");
RevCommit headAfterSubmit = getRemoteHead();
assertThat(headAfterSubmit).isEqualTo(headAfterFirstSubmit);
assertRefUpdatedEvents(initialHead, headAfterFirstSubmit);
assertChangeMergedEvents(change.getChangeId(),
headAfterFirstSubmit.name());
}
}
@Test
public void submitMultipleChangesPreview() throws Exception {
RevCommit initialHead = getRemoteHead();
PushOneCommit.Result change2 = createChange("Change 2",
"a.txt", "other content");
PushOneCommit.Result change3 = createChange("Change 3", "d", "d");
PushOneCommit.Result change4 = createChange("Change 4", "e", "e");
// change 2 is not approved, but we ignore labels
approve(change3.getChangeId());
BinaryResult request = submitPreview(change4.getChangeId());
Map<String, Map<String, Integer>> expected = new HashMap<>();
expected.put(project.get(), new HashMap<String, Integer>());
expected.get(project.get()).put("refs/heads/master", 3);
Map<Branch.NameKey, RevTree> actual =
fetchFromBundles(request);
assertThat(actual).containsKey(
new Branch.NameKey(project, "refs/heads/master"));
if (getSubmitType() == SubmitType.CHERRY_PICK) {
assertThat(actual).hasSize(2);
} else {
assertThat(actual).hasSize(1);
}
// check that the submit preview did not actually submit
RevCommit headAfterSubmit = getRemoteHead();
assertThat(headAfterSubmit).isEqualTo(initialHead);
assertRefUpdatedEvents();
assertChangeMergedEvents();
// now check we actually have the same content:
approve(change2.getChangeId());
submit(change4.getChangeId());
assertRevTrees(project, actual);
}
@Test
@ -326,6 +476,10 @@ public abstract class AbstractSubmit extends AbstractDaemonTest {
assertMerged(change.changeId);
}
protected BinaryResult submitPreview(String changeId) throws Exception {
return gApi.changes().id(changeId).current().submitPreview();
}
protected void assertSubmittable(String changeId) throws Exception {
assertThat(gApi.changes().id(changeId).info().submittable)
.named("submit bit on ChangeInfo")

View File

@ -15,6 +15,7 @@
package com.google.gerrit.acceptance.rest.change;
import static com.google.common.truth.Truth.assertThat;
import static org.junit.Assert.fail;
import com.google.gerrit.acceptance.GitUtil;
import com.google.gerrit.acceptance.PushOneCommit;
@ -24,14 +25,19 @@ import com.google.gerrit.extensions.api.changes.CherryPickInput;
import com.google.gerrit.extensions.api.changes.ReviewInput;
import com.google.gerrit.extensions.api.projects.BranchInput;
import com.google.gerrit.extensions.client.SubmitType;
import com.google.gerrit.extensions.restapi.BinaryResult;
import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.reviewdb.client.Branch;
import com.google.gerrit.reviewdb.client.Project;
import org.eclipse.jgit.junit.TestRepository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevTree;
import org.eclipse.jgit.transport.RefSpec;
import org.junit.Test;
import java.util.List;
import java.util.Map;
public class SubmitByMergeIfNecessaryIT extends AbstractSubmitByMerge {
@ -144,6 +150,12 @@ public class SubmitByMergeIfNecessaryIT extends AbstractSubmitByMerge {
approve(change2a.getChangeId());
approve(change2b.getChangeId());
approve(change3.getChangeId());
// get a preview before submitting:
BinaryResult request = submitPreview(change1b.getChangeId());
Map<Branch.NameKey, RevTree> preview =
fetchFromBundles(request);
submit(change1b.getChangeId());
RevCommit tip1 = getRemoteLog(p1, "master").get(0);
@ -158,11 +170,28 @@ public class SubmitByMergeIfNecessaryIT extends AbstractSubmitByMerge {
change2b.getCommit().getShortMessage());
assertThat(tip3.getShortMessage()).isEqualTo(
change3.getCommit().getShortMessage());
// check that the preview matched what happened:
assertThat(preview).hasSize(3);
assertThat(preview).containsKey(
new Branch.NameKey(p1, "refs/heads/master"));
assertRevTrees(p1, preview);
assertThat(preview).containsKey(
new Branch.NameKey(p2, "refs/heads/master"));
assertRevTrees(p2, preview);
assertThat(preview).containsKey(
new Branch.NameKey(p3, "refs/heads/master"));
assertRevTrees(p3, preview);
} else {
assertThat(tip2.getShortMessage()).isEqualTo(
initialHead2.getShortMessage());
assertThat(tip3.getShortMessage()).isEqualTo(
initialHead3.getShortMessage());
assertThat(preview).hasSize(1);
assertThat(preview.get(new Branch.NameKey(p1, "refs/heads/master"))).isNotNull();
}
}
@ -215,11 +244,23 @@ public class SubmitByMergeIfNecessaryIT extends AbstractSubmitByMerge {
approve(change3.getChangeId());
if (isSubmitWholeTopicEnabled()) {
submitWithConflict(change1b.getChangeId(),
String msg =
"Failed to submit 5 changes due to the following problems:\n" +
"Change " + change3.getChange().getId() + ": Change could not be " +
"merged due to a path conflict. Please rebase the change locally " +
"and upload the rebased commit for review.");
"and upload the rebased commit for review.";
// Get a preview before submitting:
try {
// We cannot just use the ExpectedException infrastructure as provided
// by AbstractDaemonTest, as then we'd stop early and not test the
// actual submit.
submitPreview(change1b.getChangeId());
fail("expected failure");
} catch (RestApiException e) {
assertThat(e.getMessage()).isEqualTo(msg);
}
submitWithConflict(change1b.getChangeId(), msg);
} else {
submit(change1b.getChangeId());
}

View File

@ -36,6 +36,7 @@ public interface RevisionApi {
void submit() throws RestApiException;
void submit(SubmitInput in) throws RestApiException;
BinaryResult submitPreview() throws RestApiException;
void publish() throws RestApiException;
ChangeApi cherryPick(CherryPickInput in) throws RestApiException;
ChangeApi rebase() throws RestApiException;
@ -240,6 +241,11 @@ public interface RevisionApi {
throw new NotImplementedException();
}
@Override
public BinaryResult submitPreview() throws RestApiException {
throw new NotImplementedException();
}
@Override
public SubmitType testSubmitType(TestSubmitRuleInput in)
throws RestApiException {

View File

@ -58,6 +58,7 @@ import com.google.gerrit.server.change.RebaseUtil;
import com.google.gerrit.server.change.Reviewed;
import com.google.gerrit.server.change.RevisionResource;
import com.google.gerrit.server.change.Submit;
import com.google.gerrit.server.change.PreviewSubmit;
import com.google.gerrit.server.change.TestSubmitType;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.UpdateException;
@ -88,6 +89,7 @@ class RevisionApiImpl implements RevisionApi {
private final Rebase rebase;
private final RebaseUtil rebaseUtil;
private final Submit submit;
private final PreviewSubmit submitPreview;
private final PublishDraftPatchSet publish;
private final Reviewed.PutReviewed putReviewed;
private final Reviewed.DeleteReviewed deleteReviewed;
@ -118,6 +120,7 @@ class RevisionApiImpl implements RevisionApi {
Rebase rebase,
RebaseUtil rebaseUtil,
Submit submit,
PreviewSubmit submitPreview,
PublishDraftPatchSet publish,
Reviewed.PutReviewed putReviewed,
Reviewed.DeleteReviewed deleteReviewed,
@ -147,6 +150,7 @@ class RevisionApiImpl implements RevisionApi {
this.rebaseUtil = rebaseUtil;
this.review = review;
this.submit = submit;
this.submitPreview = submitPreview;
this.publish = publish;
this.files = files;
this.putReviewed = putReviewed;
@ -193,6 +197,12 @@ class RevisionApiImpl implements RevisionApi {
}
}
@Override
public BinaryResult submitPreview() throws RestApiException {
submitPreview.setFormat("zip");
return submitPreview.apply(revision);
}
@Override
public void publish() throws RestApiException {
try {

View File

@ -14,6 +14,10 @@
package com.google.gerrit.server.change;
import org.apache.commons.compress.archivers.ArchiveEntry;
import org.apache.commons.compress.archivers.ArchiveOutputStream;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.zip.ZipArchiveEntry;
import org.eclipse.jgit.api.ArchiveCommand;
import org.eclipse.jgit.archive.TarFormat;
import org.eclipse.jgit.archive.Tbz2Format;
@ -21,12 +25,40 @@ import org.eclipse.jgit.archive.TgzFormat;
import org.eclipse.jgit.archive.TxzFormat;
import org.eclipse.jgit.archive.ZipFormat;
import java.io.IOException;
import java.io.OutputStream;
public enum ArchiveFormat {
TGZ("application/x-gzip", new TgzFormat()),
TAR("application/x-tar", new TarFormat()),
TBZ2("application/x-bzip2", new Tbz2Format()),
TXZ("application/x-xz", new TxzFormat()),
ZIP("application/x-zip", new ZipFormat());
TGZ("application/x-gzip", new TgzFormat()) {
@Override
public ArchiveEntry prepareArchiveEntry(String fileName) {
return new TarArchiveEntry(fileName);
}
},
TAR("application/x-tar", new TarFormat()) {
@Override
public ArchiveEntry prepareArchiveEntry(String fileName) {
return new TarArchiveEntry(fileName);
}
},
TBZ2("application/x-bzip2", new Tbz2Format()) {
@Override
public ArchiveEntry prepareArchiveEntry(String fileName) {
return new TarArchiveEntry(fileName);
}
},
TXZ("application/x-xz", new TxzFormat()) {
@Override
public ArchiveEntry prepareArchiveEntry(String fileName) {
return new TarArchiveEntry(fileName);
}
},
ZIP("application/x-zip", new ZipFormat()) {
@Override
public ArchiveEntry prepareArchiveEntry(String fileName) {
return new ZipArchiveEntry(fileName);
}
};
private final ArchiveCommand.Format<?> format;
private final String mimeType;
@ -52,4 +84,11 @@ public enum ArchiveFormat {
Iterable<String> getSuffixes() {
return format.suffixes();
}
}
public ArchiveOutputStream createArchiveOutputStream(OutputStream o)
throws IOException {
return (ArchiveOutputStream)this.format.createArchiveOutputStream(o);
}
public abstract ArchiveEntry prepareArchiveEntry(final String fileName);
}

View File

@ -93,6 +93,7 @@ public class Module extends RestApiModule {
get(REVISION_KIND, "related").to(GetRelated.class);
get(REVISION_KIND, "review").to(GetReview.class);
post(REVISION_KIND, "review").to(PostReview.class);
get(REVISION_KIND, "preview_submit").to(PreviewSubmit.class);
post(REVISION_KIND, "submit").to(Submit.class);
post(REVISION_KIND, "rebase").to(Rebase.class);
get(REVISION_KIND, "patch").to(GetPatch.class);

View File

@ -0,0 +1,143 @@
// Copyright (C) 2016 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.change;
import org.eclipse.jgit.lib.NullProgressMonitor;
import org.eclipse.jgit.lib.ObjectId;
import com.google.common.base.Strings;
import com.google.gerrit.extensions.api.changes.SubmitInput;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.BinaryResult;
import com.google.gerrit.extensions.restapi.MethodNotAllowedException;
import com.google.gerrit.extensions.restapi.PreconditionFailedException;
import com.google.gerrit.extensions.restapi.RestApiException;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.git.MergeOp;
import com.google.gerrit.server.git.MergeOpRepoManager;
import com.google.gerrit.server.git.MergeOpRepoManager.OpenRepo;
import com.google.gerrit.server.project.ChangeControl;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import org.eclipse.jgit.transport.BundleWriter;
import org.eclipse.jgit.transport.ReceiveCommand;
import org.kohsuke.args4j.Option;
import org.apache.commons.compress.archivers.ArchiveOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.Collection;
import java.util.Set;
@Singleton
public class PreviewSubmit implements RestReadView<RevisionResource> {
private final Provider<ReviewDb> dbProvider;
private final Provider<MergeOp> mergeOpProvider;
private final AllowedFormats allowedFormats;
private String format;
@Option(name = "--format")
public void setFormat(String f) {
this.format = f;
}
@Inject
PreviewSubmit(Provider<ReviewDb> dbProvider,
Provider<MergeOp> mergeOpProvider,
AllowedFormats allowedFormats) {
this.dbProvider = dbProvider;
this.mergeOpProvider = mergeOpProvider;
this.allowedFormats = allowedFormats;
}
@Override
public BinaryResult apply(RevisionResource rsrc) throws RestApiException {
if (Strings.isNullOrEmpty(format)) {
throw new BadRequestException("format is not specified");
}
ArchiveFormat f = allowedFormats.extensions.get("." + format);
if (f == null) {
throw new BadRequestException("unknown archive format");
}
Change change = rsrc.getChange();
if (!change.getStatus().isOpen()) {
throw new PreconditionFailedException("change is " + Submit.status(change));
}
ChangeControl control = rsrc.getControl();
if (!control.getUser().isIdentifiedUser()) {
throw new MethodNotAllowedException("Anonymous users cannot submit");
}
try (BinaryResult b = getBundles(rsrc, f)) {
b.disableGzip()
.setContentType(f.getMimeType())
.setAttachmentName("submit-preview-"
+ change.getChangeId() + "." + format);
return b;
} catch (OrmException | IOException e) {
throw new RestApiException("Error generating submit preview");
}
}
private BinaryResult getBundles(RevisionResource rsrc, final ArchiveFormat f)
throws OrmException, RestApiException {
ReviewDb db = dbProvider.get();
ChangeControl control = rsrc.getControl();
IdentifiedUser caller = control.getUser().asIdentifiedUser();
Change change = rsrc.getChange();
BinaryResult bin;
try (MergeOp op = mergeOpProvider.get()) {
op.merge(db, change, caller, false, new SubmitInput(), true);
final MergeOpRepoManager orm = op.getMergeOpRepoManager();
final Set<Project.NameKey> projects = op.getAllProjects();
bin = new BinaryResult() {
@Override
public void writeTo(OutputStream out) throws IOException {
ArchiveOutputStream aos = f.createArchiveOutputStream(out);
for (Project.NameKey p : projects) {
OpenRepo or = orm.getRepo(p);
BundleWriter bw = new BundleWriter(or.getRepo());
bw.setObjectCountCallback(null);
bw.setPackConfig(null);
Collection<ReceiveCommand> refs = or.getUpdate().getRefUpdates();
for (ReceiveCommand r : refs) {
bw.include(r.getRefName(), r.getNewId());
if (!r.getOldId().equals(ObjectId.zeroId())) {
bw.assume(or.getCodeReviewRevWalk().parseCommit(r.getOldId()));
}
}
// This naming scheme cannot produce directory/file conflicts
// as no projects contains ".git/":
aos.putArchiveEntry(f.prepareArchiveEntry(p.get() + ".git"));
bw.writeBundle(NullProgressMonitor.INSTANCE, aos);
aos.closeArchiveEntry();
}
aos.finish();
}
};
}
return bin;
}
}

View File

@ -90,11 +90,19 @@ public class MergeOpRepoManager implements AutoCloseable {
return ob;
}
public Repository getRepo() {
return repo;
}
Project.NameKey getProjectName() {
return project.getProject().getNameKey();
}
BatchUpdate getUpdate() {
public CodeReviewRevWalk getCodeReviewRevWalk() {
return rw;
}
public BatchUpdate getUpdate() {
checkState(db != null, "call setContext before getUpdate");
if (update == null) {
update = batchUpdateFactory.create(db, getProjectName(), caller, ts)

View File

@ -0,0 +1,88 @@
#!/usr/bin/env bash
#
# Copyright (C) 2016 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# A sceleton script to demonstrate how to use the preview_submit REST API call.
#
#
if test -z $server
then
echo "The variable 'server' needs to point to your Gerrit instance"
exit 1
fi
if test -z $changeId
then
echo "The variable 'changeId' must contain a valid change Id"
exit 1
fi
if test -z $gerrituser
then
echo "The variable 'gerrituser' must contain a user/password"
exit 1
fi
curl --digest -u $gerrituser -w '%{http_code}' -o preview \
$server/a/changes/$changeId/revisions/current/preview_submit?format=zip >http_code
if ! grep 200 http_code >/dev/null
then
# error out:
echo "Error previewing submit $changeId due to:"
cat preview
echo
else
# valid zip file, extract and obtain a bundle for each project
mkdir tmp-bundles
unzip preview -d tmp-bundles
for project in $(cd tmp-bundles && find -type f)
do
# Projects may contain slashes, so create the required
# directory structure
mkdir -p $(dirname $project)
# $project is in the format of "./path/name/project.git"
# remove the leading ./
proj=${project:-./}
git clone $server/$proj $proj
# First some nice output:
echo "Verify that the bundle is good:"
GIT_WORK_TREE=$proj GIT_DIR=$proj/.git \
git bundle verify tmp-bundles/$proj
echo "Checking that the bundle only contains one branch..."
if test \
"$(GIT_WORK_TREE=$proj GIT_DIR=$proj/.git \
git bundle list-heads tmp-bundles/$proj |wc -l)" != 1
then
echo "Submitting $changeId would affect the project"
echo "$proj"
echo "on multiple branches:"
git bundle list-heads
echo "This script does not demonstrate this use case."
exit 1
fi
# find the target branch:
branch=$(GIT_WORK_TREE=$proj GIT_DIR=$proj/.git \
git bundle list-heads tmp-bundles/$proj | awk '{print $2}')
echo "found branch $branch"
echo "fetch the bundle into the repository"
GIT_WORK_TREE=$proj GIT_DIR=$proj/.git \
git fetch tmp-bundles/$proj $branch
echo "and checkout the state"
git -C $proj checkout FETCH_HEAD
done
echo "Now run a test for all of: $(cd tmp-bundles && find -type f)"
fi