Merge branch 'stable-2.6'

* stable-2.6:
  Support triggering GC for one project via REST
  Add support to retrieve repository statistics for a project via REST
  Document the response codes that are returned by the REST endpoints
This commit is contained in:
Shawn Pearce 2013-03-26 15:11:44 -04:00
commit a2a4fa4c23
13 changed files with 566 additions and 44 deletions

View File

@ -1009,6 +1009,7 @@ on `refs/for/refs/heads/<branch>` rather than permissions to upload changes
on `refs/heads/<branch>`.
[[system_capabilities]]
System capabilities
-------------------

View File

@ -374,6 +374,79 @@ As response the new ref to which `HEAD` points is returned.
"refs/heads/stable"
----
[[get-repository-statistics]]
Get Repository Statistics
~~~~~~~~~~~~~~~~~~~~~~~~~
[verse]
'GET /projects/link:#project-name[\{project-name\}]/statistics.git'
Return statistics for the repository of a project.
.Request
----
GET /projects/plugins%2Freplication/statistics.git HTTP/1.0
----
The repository statistics are returned as a
link:#repository-statistics-info[RepositoryStatisticsInfo] entity.
.Response
----
HTTP/1.1 200 OK
Content-Disposition: attachment
Content-Type: application/json;charset=UTF-8
)]}'
{
"number_of_loose_objects": 127,
"number_of_loose_refs": 15,
"number_of_pack_files": 15,
"number_of_packed_objects": 67,
"number_of_packed_refs": 0,
"size_of_loose_objects": 29466,
"size_of_packed_objects": 9646
}
----
[[run-gc]]
Run GC
~~~~~~
[verse]
'POST /projects/link:#project-name[\{project-name\}]/gc'
Run the Git garbage collection for the repository of a project.
.Request
----
POST /projects/plugins%2Freplication/gc HTTP/1.0
----
The response is the streamed output of the garbage collection.
.Response
----
HTTP/1.1 200 OK
Content-Disposition: attachment
Content-Type: text/plain;charset=UTF-8
collecting garbage for "plugins/replication":
Pack refs: 100% (21/21)
Counting objects: 20
Finding sources: 100% (20/20)
Getting sizes: 100% (13/13)
Compressing objects: 83% (5/6)
Writing objects: 100% (20/20)
Selecting commits: 100% (7/7)
Building bitmaps: 100% (7/7)
Finding sources: 100% (41/41)
Getting sizes: 100% (25/25)
Compressing objects: 52% (12/23)
Writing objects: 100% (41/41)
Prune loose objects also found in pack files: 100% (36/36)
Prune loose, unreferenced objects: 100% (36/36)
done.
----
[[dashboard-endpoints]]
Dashboard Endpoints
-------------------
@ -805,6 +878,24 @@ Message that should be used to commit the change of the project parent
in the `project.config` file to the `refs/meta/config` branch.
|=============================
[[repository-statistics-info]]
RepositoryStatisticsInfo
~~~~~~~~~~~~~~~~~~~~~~~~
The `RepositoryStatisticsInfo` entity contains information about
statistics of a Git repository.
[options="header",width="50%",cols="1,6"]
|======================================
|Field Name |Description
|`number_of_loose_objects` |Number of loose objects.
|`number_of_loose_refs` |Number of loose refs.
|`number_of_pack_files` |Number of pack files.
|`number_of_packed_objects`|Number of packed objects.
|`number_of_packed_refs` |Number of packed refs.
|`size_of_loose_objects` |Size of loose objects in bytes.
|`size_of_packed_objects` |Size of packed objects in bytes.
|======================================
GERRIT
------

View File

@ -74,6 +74,77 @@ Encoding
All IDs that appear in the URL of a REST call (e.g. project name, group name)
must be URL encoded.
[[response-codes]]
Response Codes
~~~~~~~~~~~~~~
HTTP status codes are well defined and the Gerrit REST endpoints use
them as described in the HTTP spec.
Here are examples for some HTTP status codes that show how they are
used in the context of the Gerrit REST API.
400 Bad Request
^^^^^^^^^^^^^^^
`400 Bad Request` is used if the request is not understood by the
server due to malformed syntax.
E.g. `400 Bad Request` is returned if JSON input is expected but the
'Content-Type' of the request is not 'application/json' or the request
body doesn't contain valid JSON.
`400 Bad Request` is also used if required input fields are not set or
if options are set which cannot be used together.
403 Forbidden
^^^^^^^^^^^^^
`403 Forbidden` is used if the operation is not allowed because the
calling user has no sufficient permissions.
E.g. some REST endpoints require that the calling user has certain
link:access-control.html#system_capabilities[global capabilities]
assigned.
`403 Forbidden` is also used if `self` is used as account ID and the
REST call was done without authentication.
404 Not Found
^^^^^^^^^^^^^
`404 Not Found` is returned if the resource that is specified by the
URL is not found or is not visible to the calling user. A resource
cannot be found if the URL contains a non-existing ID or view.
405 Method Not Allowed
^^^^^^^^^^^^^^^^^^^^^^
`405 Method Not Allowed` is used if the resource exists but doesn't
support the operation.
E.g. some of the `/groups/` endpoints are only supported for Gerrit
internal groups, if they are invoked for an external group the response
is `405 Method Not Allowed`.
409 Conflict
^^^^^^^^^^^^
`409 Conflict` is used if the request cannot be completed because the
current state of the resource doesn't allow the operation.
E.g. if you try to submit a change that is abandoned, this fails with
`409 Conflict` because the state of the change doesn't allow the submit
operation.
`409 Conflict` is also used if you try to create a resource but the
name is already occupied by an existing resource.
412 Precondition Failed
^^^^^^^^^^^^^^^^^^^^^^^
`412 Precondition Failed` is used if a precondition from the request
header fields is not fulfilled as described in the link:#preconditions[
Preconditions] section.
422 Unprocessable Entity
^^^^^^^^^^^^^^^^^^^^^^^^
`422 Unprocessable Entity` is returned if the ID of a resource that is
specified in the request body cannot be resolved.
Endpoints
---------
link:rest-api-accounts.html[/accounts/]::

View File

@ -0,0 +1,70 @@
// Copyright (C) 2013 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.acceptance;
import static org.junit.Assert.assertTrue;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.inject.Inject;
import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.lib.Repository;
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
public class GcAssert {
private final GitRepositoryManager repoManager;
@Inject
public GcAssert(GitRepositoryManager repoManager) {
this.repoManager = repoManager;
}
public void assertHasPackFile(Project.NameKey... projects)
throws RepositoryNotFoundException, IOException {
for (Project.NameKey p : projects) {
assertTrue("Project " + p.get() + " has no pack files.",
getPackFiles(p).length > 0);
}
}
public void assertHasNoPackFile(Project.NameKey... projects)
throws RepositoryNotFoundException, IOException {
for (Project.NameKey p : projects) {
assertTrue("Project " + p.get() + " has pack files.",
getPackFiles(p).length == 0);
}
}
private String[] getPackFiles(Project.NameKey p)
throws RepositoryNotFoundException, IOException {
Repository repo = repoManager.openRepository(p);
try {
File packDir = new File(repo.getDirectory(), "objects/pack");
return packDir.list(new FilenameFilter() {
@Override
public boolean accept(File dir, String name) {
return name.endsWith(".pack");
}
});
} finally {
repo.close();
}
}
}

View File

@ -56,6 +56,10 @@ public class RestSession {
return new RestResponse(getClient().execute(put));
}
public RestResponse post(String endPoint) throws IOException {
return post(endPoint, null);
}
public RestResponse post(String endPoint, Object content) throws IOException {
HttpPost post = new HttpPost("http://localhost:8080/a" + endPoint);
if (content != null) {

View File

@ -0,0 +1,102 @@
// Copyright (C) 2013 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.acceptance.rest.project;
import static com.google.gerrit.acceptance.git.GitUtil.createProject;
import static org.junit.Assert.assertEquals;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.AccountCreator;
import com.google.gerrit.acceptance.GcAssert;
import com.google.gerrit.acceptance.RestResponse;
import com.google.gerrit.acceptance.RestSession;
import com.google.gerrit.acceptance.SshSession;
import com.google.gerrit.acceptance.TestAccount;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.config.AllProjectsName;
import com.google.gerrit.server.git.GarbageCollectionQueue;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
import com.jcraft.jsch.JSchException;
import org.apache.http.HttpStatus;
import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
public class GarbageCollectionIT extends AbstractDaemonTest {
@Inject
private AccountCreator accounts;
@Inject
private AllProjectsName allProjects;
@Inject
private GarbageCollectionQueue gcQueue;
@Inject
private GcAssert gcAssert;
private TestAccount admin;
private RestSession session;
private Project.NameKey project1;
private Project.NameKey project2;
@Before
public void setUp() throws Exception {
admin =
accounts.create("admin", "admin@example.com", "Administrator",
"Administrators");
SshSession sshSession = new SshSession(admin);
project1 = new Project.NameKey("p1");
createProject(sshSession, project1.get());
project2 = new Project.NameKey("p2");
createProject(sshSession, project2.get());
session = new RestSession(admin);
}
@Test
public void testGcNonExistingProject_NotFound() throws IOException {
assertEquals(HttpStatus.SC_NOT_FOUND, POST("/projects/non-existing/gc"));
}
@Test
public void testGcNotAllowed_Forbidden() throws IOException, OrmException, JSchException {
assertEquals(HttpStatus.SC_FORBIDDEN,
new RestSession(accounts.create("user", "user@example.com", "User"))
.post("/projects/" + allProjects.get() + "/gc").getStatusCode());
}
@Test
public void testGcOneProject() throws JSchException, IOException {
assertEquals(HttpStatus.SC_OK, POST("/projects/" + allProjects.get() + "/gc"));
gcAssert.assertHasPackFile(allProjects);
gcAssert.assertHasNoPackFile(project1, project2);
}
private int POST(String endPoint) throws IOException {
RestResponse r = session.post(endPoint);
r.consume();
return r.getStatusCode();
}
}

View File

@ -21,6 +21,7 @@ import static org.junit.Assert.assertTrue;
import com.google.gerrit.acceptance.AbstractDaemonTest;
import com.google.gerrit.acceptance.AccountCreator;
import com.google.gerrit.acceptance.GcAssert;
import com.google.gerrit.acceptance.SshSession;
import com.google.gerrit.acceptance.TestAccount;
import com.google.gerrit.common.data.GarbageCollectionResult;
@ -28,19 +29,14 @@ import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.config.AllProjectsName;
import com.google.gerrit.server.git.GarbageCollection;
import com.google.gerrit.server.git.GarbageCollectionQueue;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
import com.jcraft.jsch.JSchException;
import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.lib.Repository;
import org.junit.Before;
import org.junit.Test;
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.util.Arrays;
import java.util.Locale;
@ -50,9 +46,6 @@ public class GarbageCollectionIT extends AbstractDaemonTest {
@Inject
private AccountCreator accounts;
@Inject
private GitRepositoryManager repoManager;
@Inject
private AllProjectsName allProjects;
@ -62,6 +55,9 @@ public class GarbageCollectionIT extends AbstractDaemonTest {
@Inject
private GarbageCollectionQueue gcQueue;
@Inject
private GcAssert gcAssert;
private TestAccount admin;
private SshSession sshSession;
private Project.NameKey project1;
@ -93,8 +89,8 @@ public class GarbageCollectionIT extends AbstractDaemonTest {
+ project2.get() + "\"");
assertFalse(sshSession.hasError());
assertNoError(response);
assertHasPackFile(project1, project2);
assertHasNoPackFile(allProjects, project3);
gcAssert.assertHasPackFile(project1, project2);
gcAssert.assertHasNoPackFile(allProjects, project3);
}
@Test
@ -102,7 +98,7 @@ public class GarbageCollectionIT extends AbstractDaemonTest {
String response = sshSession.exec("gerrit gc --all");
assertFalse(sshSession.hasError());
assertNoError(response);
assertHasPackFile(allProjects, project1, project2, project3);
gcAssert.assertHasPackFile(allProjects, project1, project2, project3);
}
@Test
@ -132,36 +128,4 @@ public class GarbageCollectionIT extends AbstractDaemonTest {
private void assertNoError(String response) {
assertFalse(response, response.toLowerCase(Locale.US).contains("error"));
}
private void assertHasPackFile(Project.NameKey... projects)
throws RepositoryNotFoundException, IOException {
for (Project.NameKey p : projects) {
assertTrue("Project " + p.get() + "has no pack files.",
getPackFiles(p).length > 0);
}
}
private void assertHasNoPackFile(Project.NameKey... projects)
throws RepositoryNotFoundException, IOException {
for (Project.NameKey p : projects) {
assertTrue("Project " + p.get() + "has pack files.",
getPackFiles(p).length == 0);
}
}
private String[] getPackFiles(Project.NameKey p)
throws RepositoryNotFoundException, IOException {
Repository repo = repoManager.openRepository(p);
try {
File packDir = new File(repo.getDirectory(), "objects/pack");
return packDir.list(new FilenameFilter() {
@Override
public boolean accept(File dir, String name) {
return name.endsWith(".pack");
}
});
} finally {
repo.close();
}
}
}

View File

@ -0,0 +1,23 @@
// Copyright (C) 2013 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.extensions.restapi;
import java.io.IOException;
import java.io.OutputStream;
public interface StreamingResponse {
public String getContentType();
public void stream(OutputStream out) throws IOException;
}

View File

@ -61,6 +61,7 @@ import com.google.gerrit.extensions.restapi.RestModifyView;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.extensions.restapi.RestResource;
import com.google.gerrit.extensions.restapi.RestView;
import com.google.gerrit.extensions.restapi.StreamingResponse;
import com.google.gerrit.extensions.restapi.TopLevelResource;
import com.google.gerrit.extensions.restapi.UnprocessableEntityException;
import com.google.gerrit.httpd.WebSession;
@ -301,7 +302,11 @@ public class RestApiServlet extends HttpServlet {
}
res.setStatus(status);
if (result != Response.none()) {
if (result instanceof StreamingResponse) {
StreamingResponse r = (StreamingResponse) result;
res.setContentType(r.getContentType());
r.stream(res.getOutputStream());
} else if (result != Response.none()) {
result = Response.unwrap(result);
if (result instanceof BinaryResult) {
replyBinaryResult(req, res, (BinaryResult) result);

View File

@ -0,0 +1,93 @@
// Copyright (C) 2013 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.project;
import com.google.common.base.Charsets;
import com.google.gerrit.common.data.GarbageCollectionResult;
import com.google.gerrit.common.data.GlobalCapability;
import com.google.gerrit.extensions.annotations.RequiresCapability;
import com.google.gerrit.extensions.restapi.RestModifyView;
import com.google.gerrit.extensions.restapi.StreamingResponse;
import com.google.gerrit.server.git.GarbageCollection;
import com.google.gerrit.server.project.GarbageCollect.Input;
import com.google.inject.Inject;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.util.Collections;
@RequiresCapability(GlobalCapability.RUN_GC)
public class GarbageCollect implements RestModifyView<ProjectResource, Input> {
public static class Input {
}
private GarbageCollection.Factory garbageCollectionFactory;
@Inject
GarbageCollect(GarbageCollection.Factory garbageCollectionFactory) {
this.garbageCollectionFactory = garbageCollectionFactory;
}
@Override
public StreamingResponse apply(final ProjectResource rsrc, Input input) {
return new StreamingResponse() {
@Override
public String getContentType() {
return "text/plain;charset=UTF-8";
}
@Override
public void stream(OutputStream out) throws IOException {
PrintWriter writer = new PrintWriter(
new OutputStreamWriter(out, Charsets.UTF_8)) {
@Override
public void println() {
write('\n');
}
};
try {
GarbageCollectionResult result = garbageCollectionFactory.create().run(
Collections.singletonList(rsrc.getNameKey()), writer);
if (result.hasErrors()) {
for (GarbageCollectionResult.Error e : result.getErrors()) {
String msg;
switch (e.getType()) {
case REPOSITORY_NOT_FOUND:
msg = "error: project \"" + e.getProjectName() + "\" not found";
break;
case GC_ALREADY_SCHEDULED:
msg = "error: garbage collection for project \""
+ e.getProjectName() + "\" was already scheduled";
break;
case GC_FAILED:
msg = "error: garbage collection for project \"" + e.getProjectName()
+ "\" failed";
break;
default:
msg = "error: garbage collection for project \"" + e.getProjectName()
+ "\" failed: " + e.getType();
}
writer.println(msg);
}
}
} finally {
writer.flush();
}
}
};
}
}

View File

@ -0,0 +1,63 @@
// Copyright (C) 2013 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.project;
import com.google.gerrit.common.data.GlobalCapability;
import com.google.gerrit.extensions.annotations.RequiresCapability;
import com.google.gerrit.extensions.restapi.ResourceConflictException;
import com.google.gerrit.extensions.restapi.ResourceNotFoundException;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.inject.Inject;
import org.eclipse.jgit.api.GarbageCollectCommand;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.JGitInternalException;
import org.eclipse.jgit.lib.Repository;
import java.io.IOException;
import java.util.Properties;
@RequiresCapability(GlobalCapability.RUN_GC)
public class GetStatistics implements RestReadView<ProjectResource> {
private final GitRepositoryManager repoManager;
@Inject
GetStatistics(GitRepositoryManager repoManager) {
this.repoManager = repoManager;
}
@Override
public RepositoryStatistics apply(ProjectResource rsrc)
throws ResourceNotFoundException, ResourceConflictException {
try {
Repository repo = repoManager.openRepository(rsrc.getNameKey());
try {
GarbageCollectCommand gc = Git.wrap(repo).gc();
return new RepositoryStatistics(gc.getStatistics());
} catch (GitAPIException e) {
throw new ResourceConflictException(e.getMessage());
} catch (JGitInternalException e) {
throw new ResourceConflictException(e.getMessage());
} finally {
repo.close();
}
} catch (IOException e) {
throw new ResourceNotFoundException(rsrc.getName());
}
}
}

View File

@ -43,6 +43,9 @@ public class Module extends RestApiModule {
get(PROJECT_KIND, "HEAD").to(GetHead.class);
put(PROJECT_KIND, "HEAD").to(SetHead.class);
get(PROJECT_KIND, "statistics.git").to(GetStatistics.class);
post(PROJECT_KIND, "gc").to(GarbageCollect.class);
child(PROJECT_KIND, "dashboards").to(DashboardsCollection.class);
get(DASHBOARD_KIND).to(GetDashboard.class);
put(DASHBOARD_KIND).to(SetDashboard.class);

View File

@ -0,0 +1,32 @@
// Copyright (C) 2013 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.project;
import com.google.common.base.CaseFormat;
import java.util.Map.Entry;
import java.util.Properties;
import java.util.TreeMap;
class RepositoryStatistics extends TreeMap<String, Object> {
private static final long serialVersionUID = 1L;
public RepositoryStatistics(Properties p) {
for (Entry<Object, Object> e : p.entrySet()) {
put(CaseFormat.LOWER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE,
e.getKey().toString()), e.getValue());
}
}
}