opendev/gerrit
Code Issues Proposed changes

Merge changes from topic 'group-index'

Browse Source 
* changes:
  Index groups that are created during init
  Implement group index for ElasticSearch
  Add REST endpoint to reindex a single group
  Support default query for groups
  Add query operator that matches groups that are visible to all users
  Add group query operator to match groups by owner
  Add group query operator to match groups by description
  Support querying groups by name and name part
  Set '_more_groups' on last group of query result
  Add tests for group index
  Groups API: Add query methods
  Support arbitrary group queries via REST
  Add new optional interface for REST collections to get request params
  Add GroupQueryProcessor
  IsVisibleToPredicate: Move describe method into base class
  Index group on creation and whenever a group is evicted from cache
  Initial implementation of group index in Lucene
  Add initial GroupQueryBuilder and define key predicate for group index
  Add schema and index definitions for groups index
This commit is contained in:
David Pursehouse 2017-01-12 03:56:48 +00:00 committed by Gerrit Code Review
commit e93b40b3e1
66 changed files with 2930 additions and 120 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

99
Documentation/rest-api-groups.txt
View File

@ -211,6 +211,83 @@ error out.)
}
----
[[query-groups]]
=== Query Groups
--
'GET /groups/?query2=<query>'
--
Queries internal groups visible to the caller. The
link:user-search-groups.html#_search_operators[query string] must be
provided by the `query2` parameter. The `start` and `limit` parameters
can be used to skip/limit results.
As result a list of link:#group-info[GroupInfo] entities is returned.
.Request
----
GET /groups/?query2=inname:test HTTP/1.0
----
.Response
----
HTTP/1.1 200 OK
Content-Disposition: attachment
Content-Type: application/json; charset=UTF-8
)]}'
[
{
"url": "#/admin/groups/uuid-68236a40ca78de8be630312d8ba50250bc5638ae",
"options": {},
"description": "Group for running tests on MyProject",
"group_id": 20,
"owner": "MyProject-Test-Group",
"owner_id": "59b92f35489e62c80d1ab1bf0c2d17843038df8b",
"id": "68236a40ca78de8be630312d8ba50250bc5638ae"
},
{
"url": "#/admin/groups/uuid-99a534526313324a2667025c3f4e089199b736aa",
"options": {},
"description": "Testers for ProjectX",
"group_id": 17,
"owner": "ProjectX-Testers",
"owner_id": "59b92f35489e62c80d1ab1bf0c2d17843038df8b",
"id": "99a534526313324a2667025c3f4e089199b736aa"
}
]
----
If the number of groups matching the query exceeds either the internal
limit or a supplied `limit` query parameter, the last group object has
a `_more_groups: true` JSON field set.
[[group-query-limit]]
==== Group Limit
The `/groups/?query2=<query>` URL also accepts a limit integer in the
`limit` parameter. This limits the results to `limit` groups.
Query the first 25 groups in group list.
----
GET /groups/?query2=<query>&limit=25 HTTP/1.0
----
The `/groups/` URL also accepts a start integer in the `start`
parameter. The results will skip `start` groups from group list.
Query 25 groups starting from index 50.
----
GET /groups/?query2=<query>&limit=25&start=50 HTTP/1.0
----
[[group-query-options]]
==== Group Options
Additional fields can be obtained by adding `o` parameters. Each option
requires more lookups and slows down the query response time to the
client so they are generally disabled by default. The supported fields
are described in the context of the link:#group-options[List Groups]
REST endpoint.
[[get-group]]
=== Get Group
--
@ -714,6 +791,24 @@ newest audit event comes first.
]
----
[[index-group]]
=== Index Group
--
'POST /groups/link:#group-id[\{group-id\}]/index'
--
Adds or updates the internal group in the secondary index.
.Request
----
POST /groups/fdda826a0815859ab48d22a05a43472f0f55f89a/index HTTP/1.0
----
.Response
----
HTTP/1.1 204 No Content
----
[[group-member-endpoints]]
== Group Member Endpoints
@ -1277,6 +1372,10 @@ permits users to apply to join the group, or manage their membership.
|`group_id` |only for internal groups|The numeric ID of the group.
|`owner` |only for internal groups|The name of the owner group.
|`owner_id` |only for internal groups|The URL encoded UUID of the owner group.
|`_more_groups`|optional, only for internal groups, not set if `false`|
Whether the query would deliver more results if not limited. +
Only set on the last group that is returned by a
link:#query-groups[group query].
|`members` |optional, only for internal groups|
A list of link:rest-api-accounts.html#account-info[AccountInfo]
entities describing the direct members. +

82
Documentation/user-search-groups.txt Normal file
View File

@ -0,0 +1,82 @@
= Gerrit Code Review - Searching Groups
Group queries only match internal groups. External groups and system
groups are not included in the query result.
== Basic Group Search
Similar to many popular search engines on the web, just enter some
text and let Gerrit figure out the meaning:
[options="header"]
|======================================================
|Description | Examples
|Name | Foo-Verifiers
|UUID | 6a1e70e1a88782771a91808c8af9bbb7a9871389
|Description | deprecated
|======================================================
[[search-operators]]
== Search Operators
Operators act as restrictions on the search. As more operators
are added to the same query string, they further restrict the
returned results. Search can also be performed by typing only a text
with no operator, which will match against a variety of fields.
[[description]]
description:'DESCRIPTION'::
+
Matches groups that have a description that contains 'DESCRIPTION'
(case-insensitive).
[[inname]]
inname:'NAMEPART'::
+
Matches groups that have a name part that starts with 'NAMEPART'
(case-insensitive).
[[is]]
[[is-visibleToAll]]
is:visibleToAll::
+
Matches groups that are in the groups options marked as visible to all
registered users.
[[name]]
name:'NAME'::
+
Matches groups that have the name 'NAME' (case-insensitive).
[[owner]]
owner:'UUID'::
+
Matches groups that are owned by a group that has the UUID 'UUID'.
[[uuid]]
uuid:'UUID'::
+
Matches groups that have the UUID 'UUID'.
== Magical Operators
[[is-visible]]
is:visible::
+
Magical internal flag to prove the current user has access to read
the group. This flag is always added to any query.
[[limit]]
limit:'CNT'::
+
Limit the returned results to no more than 'CNT' records. This is
automatically set to the page size configured in the current user's
preferences. Including it in a web query may lead to unpredictable
results with regards to pagination.
GERRIT
------
Part of link:index.html[Gerrit Code Review]
SEARCHBOX
---------

27
gerrit-acceptance-tests/src/test/java/com/google/gerrit/acceptance/api/group/GroupsIT.java
View File

@ -45,6 +45,7 @@ import org.junit.Test;
import java.sql.Timestamp;
import java.util.Arrays;
import java.util.Collections;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
@ -481,6 +482,32 @@ public class GroupsIT extends AbstractDaemonTest {
}
}
// reindex is tested by {@link AbstractQueryGroupsTest#reindex}
@Test
public void reindexPermissions() throws Exception {
TestAccount groupOwner = accounts.user2();
GroupInput in = new GroupInput();
in.name = name("group");
in.members = Collections.singleton(groupOwner).stream()
.map(u -> u.id.toString()).collect(toList());
in.visibleToAll = true;
GroupInfo group = gApi.groups().create(in).get();
// admin can reindex any group
setApiUser(admin);
gApi.groups().id(group.id).index();
// group owner can reindex own group (group is owned by itself)
setApiUser(groupOwner);
gApi.groups().id(group.id).index();
// user cannot reindex any group
setApiUser(user);
exception.expect(AuthException.class);
exception.expectMessage("not allowed to index group");
gApi.groups().id(group.id).index();
}
private void assertAuditEvent(GroupAuditEventInfo info, Type expectedType,
Account.Id expectedUser, Account.Id expectedMember) {
assertThat(info.user._accountId).isEqualTo(expectedUser.get());

228
gerrit-elasticsearch/src/main/java/com/google/gerrit/elasticsearch/ElasticGroupIndex.java Normal file
View File

@ -0,0 +1,228 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.elasticsearch;
import static com.google.gson.FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Lists;
import com.google.gerrit.elasticsearch.ElasticMapping.MappingProperties;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.FieldDef.FillArgs;
import com.google.gerrit.server.index.IndexUtils;
import com.google.gerrit.server.index.QueryOptions;
import com.google.gerrit.server.index.Schema;
import com.google.gerrit.server.index.group.GroupField;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.gerrit.server.query.DataSource;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.QueryParseException;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.ResultSet;
import com.google.inject.Provider;
import com.google.inject.assistedinject.Assisted;
import com.google.inject.assistedinject.AssistedInject;
import org.eclipse.jgit.lib.Config;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
import io.searchbox.client.JestResult;
import io.searchbox.core.Bulk;
import io.searchbox.core.Bulk.Builder;
import io.searchbox.core.Search;
import io.searchbox.core.search.sort.Sort;
import io.searchbox.core.search.sort.Sort.Sorting;
public class ElasticGroupIndex
extends AbstractElasticIndex<AccountGroup.UUID, AccountGroup>
implements GroupIndex {
static class GroupMapping {
MappingProperties groups;
GroupMapping(Schema<AccountGroup> schema) {
this.groups = ElasticMapping.createMapping(schema);
}
}
static final String GROUPS = "groups";
static final String GROUPS_PREFIX = GROUPS + "_";
private static final Logger log =
LoggerFactory.getLogger(ElasticGroupIndex.class);
private final Gson gson;
private final GroupMapping mapping;
private final Provider<GroupCache> groupCache;
private final ElasticQueryBuilder queryBuilder;
@AssistedInject
ElasticGroupIndex(
@GerritServerConfig Config cfg,
FillArgs fillArgs,
SitePaths sitePaths,
Provider<GroupCache> groupCache,
@Assisted Schema<AccountGroup> schema) {
super(cfg, fillArgs, sitePaths, schema, GROUPS_PREFIX);
this.groupCache = groupCache;
this.mapping = new GroupMapping(schema);
this.queryBuilder = new ElasticQueryBuilder();
this.gson = new GsonBuilder()
.setFieldNamingPolicy(LOWER_CASE_WITH_UNDERSCORES).create();
}
@Override
public void replace(AccountGroup group) throws IOException {
Bulk bulk = new Bulk.Builder()
.defaultIndex(indexName)
.defaultType(GROUPS)
.addAction(insert(GROUPS, group))
.refresh(refresh)
.build();
JestResult result = client.execute(bulk);
if (!result.isSucceeded()) {
throw new IOException(
String.format("Failed to replace group %s in index %s: %s",
group.getGroupUUID().get(), indexName, result.getErrorMessage()));
}
}
@Override
public DataSource<AccountGroup> getSource(Predicate<AccountGroup> p,
QueryOptions opts) throws QueryParseException {
return new QuerySource(p, opts);
}
@Override
protected Builder addActions(Builder builder, AccountGroup.UUID c) {
return builder.addAction(delete(GROUPS, c));
}
@Override
protected String getMappings() {
ImmutableMap<String, GroupMapping> mappings =
ImmutableMap.of("mappings", mapping);
return gson.toJson(mappings);
}
@Override
protected String getId(AccountGroup group) {
return group.getGroupUUID().get();
}
private class QuerySource implements DataSource<AccountGroup> {
private final Search search;
private final Set<String> fields;
QuerySource(Predicate<AccountGroup> p, QueryOptions opts)
throws QueryParseException {
QueryBuilder qb = queryBuilder.toQueryBuilder(p);
fields = IndexUtils.groupFields(opts);
SearchSourceBuilder searchSource = new SearchSourceBuilder()
.query(qb)
.from(opts.start())
.size(opts.limit())
.fields(Lists.newArrayList(fields));
Sort sort = new Sort(GroupField.UUID.getName(), Sorting.ASC);
sort.setIgnoreUnmapped();
search = new Search.Builder(searchSource.toString())
.addType(GROUPS)
.addIndex(indexName)
.addSort(ImmutableList.of(sort))
.build();
}
@Override
public int getCardinality() {
return 10;
}
@Override
public ResultSet<AccountGroup> read() throws OrmException {
try {
List<AccountGroup> results = Collections.emptyList();
JestResult result = client.execute(search);
if (result.isSucceeded()) {
JsonObject obj = result.getJsonObject().getAsJsonObject("hits");
if (obj.get("hits") != null) {
JsonArray json = obj.getAsJsonArray("hits");
results = Lists.newArrayListWithCapacity(json.size());
for (int i = 0; i < json.size(); i++) {
results.add(toAccountGroup(json.get(i)));
}
}
} else {
log.error(result.getErrorMessage());
}
final List<AccountGroup> r = Collections.unmodifiableList(results);
return new ResultSet<AccountGroup>() {
@Override
public Iterator<AccountGroup> iterator() {
return r.iterator();
}
@Override
public List<AccountGroup> toList() {
return r;
}
@Override
public void close() {
// Do nothing.
}
};
} catch (IOException e) {
throw new OrmException(e);
}
}
@Override
public String toString() {
return search.toString();
}
private AccountGroup toAccountGroup(JsonElement json) {
JsonElement source = json.getAsJsonObject().get("_source");
if (source == null) {
source = json.getAsJsonObject().get("fields");
}
AccountGroup.UUID uuid = new AccountGroup.UUID(
source.getAsJsonObject().get(GroupField.UUID.getName()).getAsString());
// Use the GroupCache rather than depending on any stored fields in the
// document (of which there shouldn't be any).
return groupCache.get().get(uuid);
}
}
}

9
gerrit-elasticsearch/src/main/java/com/google/gerrit/elasticsearch/ElasticIndexModule.java
View File

@ -21,6 +21,7 @@ import com.google.gerrit.server.index.IndexModule;
import com.google.gerrit.server.index.SingleVersionModule;
import com.google.gerrit.server.index.account.AccountIndex;
import com.google.gerrit.server.index.change.ChangeIndex;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.inject.Provides;
import com.google.inject.Singleton;
import com.google.inject.assistedinject.FactoryModuleBuilder;
@ -49,14 +50,18 @@ public class ElasticIndexModule extends LifecycleModule {
@Override
protected void configure() {
install(
new FactoryModuleBuilder()
.implement(AccountIndex.class, ElasticAccountIndex.class)
.build(AccountIndex.Factory.class));
install(
new FactoryModuleBuilder()
.implement(ChangeIndex.class, ElasticChangeIndex.class)
.build(ChangeIndex.Factory.class));
install(
new FactoryModuleBuilder()
.implement(AccountIndex.class, ElasticAccountIndex.class)
.build(AccountIndex.Factory.class));
.implement(GroupIndex.class, ElasticGroupIndex.class)
.build(GroupIndex.Factory.class));
install(new IndexModule(threads));
install(new SingleVersionModule(singleVersions));

69
gerrit-elasticsearch/src/test/java/com/google/gerrit/elasticsearch/ElasticQueryGroupsTest.java Normal file
View File

@ -0,0 +1,69 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.elasticsearch;
import com.google.gerrit.elasticsearch.ElasticTestUtils.ElasticNodeInfo;
import com.google.gerrit.server.query.group.AbstractQueryGroupsTest;
import com.google.gerrit.testutil.InMemoryModule;
import com.google.inject.Guice;
import com.google.inject.Injector;
import org.eclipse.jgit.lib.Config;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import java.util.concurrent.ExecutionException;
public class ElasticQueryGroupsTest extends AbstractQueryGroupsTest {
private static ElasticNodeInfo nodeInfo;
@BeforeClass
public static void startIndexService()
throws InterruptedException, ExecutionException {
if (nodeInfo != null) {
// do not start Elasticsearch twice
return;
}
nodeInfo = ElasticTestUtils.startElasticsearchNode();
ElasticTestUtils.createAllIndexes(nodeInfo);
}
@AfterClass
public static void stopElasticsearchServer() {
if (nodeInfo != null) {
nodeInfo.node.close();
nodeInfo.elasticDir.delete();
nodeInfo = null;
}
}
@After
public void cleanupIndex() {
if (nodeInfo != null) {
ElasticTestUtils.deleteAllIndexes(nodeInfo);
ElasticTestUtils.createAllIndexes(nodeInfo);
}
}
@Override
protected Injector createInjector() {
Config elasticsearchConfig = new Config(config);
InMemoryModule.setDefaults(elasticsearchConfig);
ElasticTestUtils.configure(elasticsearchConfig, nodeInfo.port);
return Guice.createInjector(
new InMemoryModule(elasticsearchConfig, notesMigration));
}
}

18
gerrit-elasticsearch/src/test/java/com/google/gerrit/elasticsearch/ElasticTestUtils.java
View File

@ -19,16 +19,20 @@ import static com.google.gerrit.elasticsearch.ElasticAccountIndex.ACCOUNTS_PREFI
import static com.google.gerrit.elasticsearch.ElasticChangeIndex.CHANGES_PREFIX;
import static com.google.gerrit.elasticsearch.ElasticChangeIndex.CLOSED_CHANGES;
import static com.google.gerrit.elasticsearch.ElasticChangeIndex.OPEN_CHANGES;
import static com.google.gerrit.elasticsearch.ElasticGroupIndex.GROUPS_PREFIX;
import com.google.common.base.Strings;
import com.google.common.io.Files;
import com.google.gerrit.elasticsearch.ElasticAccountIndex.AccountMapping;
import com.google.gerrit.elasticsearch.ElasticChangeIndex.ChangeMapping;
import com.google.gerrit.elasticsearch.ElasticGroupIndex.GroupMapping;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.account.AccountState;
import com.google.gerrit.server.index.IndexModule.IndexType;
import com.google.gerrit.server.index.Schema;
import com.google.gerrit.server.index.account.AccountSchemaDefinitions;
import com.google.gerrit.server.index.change.ChangeSchemaDefinitions;
import com.google.gerrit.server.index.group.GroupSchemaDefinitions;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.gson.FieldNamingPolicy;
import com.google.gson.Gson;
@ -154,6 +158,20 @@ final class ElasticTestUtils {
.addMapping(ElasticAccountIndex.ACCOUNTS, gson.toJson(accountMapping))
.execute()
.actionGet();
Schema<AccountGroup> groupSchema =
GroupSchemaDefinitions.INSTANCE.getLatest();
GroupMapping groupMapping = new GroupMapping(groupSchema);
nodeInfo.node
.client()
.admin()
.indices()
.prepareCreate(
String.format(
"%s%04d", GROUPS_PREFIX, groupSchema.getVersion()))
.addMapping(ElasticGroupIndex.GROUPS, gson.toJson(groupMapping))
.execute()
.actionGet();
}
private static String getHttpPort(Node node)

14
gerrit-extension-api/src/main/java/com/google/gerrit/extensions/api/groups/GroupApi.java
View File

@ -143,6 +143,15 @@ public interface GroupApi {
*/
List<? extends GroupAuditEventInfo> auditLog() throws RestApiException;
/**
* Reindexes the group.
*
* Only supported for internal groups.
*
* @throws RestApiException
*/
void index() throws RestApiException;
/**
* A default implementation which allows source compatibility
* when adding new methods to the interface.
@ -239,5 +248,10 @@ public interface GroupApi {
throws RestApiException {
throw new NotImplementedException();
}
@Override
public void index() {
throw new NotImplementedException();
}
}
}

107
gerrit-extension-api/src/main/java/com/google/gerrit/extensions/api/groups/Groups.java
View File

@ -52,6 +52,25 @@ public interface Groups {
/** @return new request for listing groups. */
ListRequest list();
/**
* Query groups.
* <p>
* Example code:
* {@code query().withQuery("inname:test").withLimit(10).get()}
*
* @return API for setting parameters and getting result.
*/
QueryRequest query();
/**
* Query groups.
* <p>
* Shortcut API for {@code query().withQuery(String)}.
*
* @see #query()
*/
QueryRequest query(String query);
abstract class ListRequest {
private final EnumSet<ListGroupsOption> options =
EnumSet.noneOf(ListGroupsOption.class);
@ -181,6 +200,84 @@ public interface Groups {
}
}
/**
* API for setting parameters and getting result.
* Used for {@code query()}.
*
* @see #query()
*/
abstract class QueryRequest {
private String query;
private int limit;
private int start;
private EnumSet<ListGroupsOption> options =
EnumSet.noneOf(ListGroupsOption.class);
/**
* Execute query and returns the matched groups as list.
*/
public abstract List<GroupInfo> get() throws RestApiException;
/**
* Set query.
*
* @param query needs to be in human-readable form.
*/
public QueryRequest withQuery(String query) {
this.query = query;
return this;
}
/**
* Set limit for returned list of groups.
* Optional; server-default is used when not provided.
*/
public QueryRequest withLimit(int limit) {
this.limit = limit;
return this;
}
/**
* Set number of groups to skip.
* Optional; no groups are skipped when not provided.
*/
public QueryRequest withStart(int start) {
this.start = start;
return this;
}
public QueryRequest withOption(ListGroupsOption options) {
this.options.add(options);
return this;
}
public QueryRequest withOptions(ListGroupsOption... options) {
this.options.addAll(Arrays.asList(options));
return this;
}
public QueryRequest withOptions(EnumSet<ListGroupsOption> options) {
this.options = options;
return this;
}
public String getQuery() {
return query;
}
public int getLimit() {
return limit;
}
public int getStart() {
return start;
}
public EnumSet<ListGroupsOption> getOptions() {
return options;
}
}
/**
* A default implementation which allows source compatibility
* when adding new methods to the interface.
@ -205,5 +302,15 @@ public interface Groups {
public ListRequest list() {
throw new NotImplementedException();
}
@Override
public QueryRequest query() {
throw new NotImplementedException();
}
@Override
public QueryRequest query(String query) {
throw new NotImplementedException();
}
}
}

1
gerrit-extension-api/src/main/java/com/google/gerrit/extensions/common/GroupInfo.java
View File

@ -25,6 +25,7 @@ public class GroupInfo extends GroupBaseInfo {
public Integer groupId;
public String owner;
public String ownerId;
public Boolean _moreGroups;
// These fields are only supplied for internal groups, and only if requested.
public List<AccountInfo> members;

32
gerrit-extension-api/src/main/java/com/google/gerrit/extensions/restapi/NeedsParams.java Normal file
View File

@ -0,0 +1,32 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.extensions.restapi;
import com.google.common.collect.Multimap;
/**
* Optional interface for {@link RestCollection}.
* <p>
* Collections that implement this interface can get to know about the request
* parameters.
*/
public interface NeedsParams {
/**
* Sets the request parameter.
*
* @param params the request parameter
*/
void setParams(Multimap<String, String> params);
}

10
gerrit-httpd/src/main/java/com/google/gerrit/httpd/restapi/RestApiServlet.java
View File

@ -74,6 +74,7 @@ import com.google.gerrit.extensions.restapi.DefaultInput;
import com.google.gerrit.extensions.restapi.ETagView;
import com.google.gerrit.extensions.restapi.IdString;
import com.google.gerrit.extensions.restapi.MethodNotAllowedException;
import com.google.gerrit.extensions.restapi.NeedsParams;
import com.google.gerrit.extensions.restapi.NotImplementedException;
import com.google.gerrit.extensions.restapi.PreconditionFailedException;
import com.google.gerrit.extensions.restapi.RawInput;
@ -245,6 +246,7 @@ public class RestApiServlet extends HttpServlet {
long responseBytes = -1;
Object result = null;
Multimap<String, String> params = LinkedHashMultimap.create();
Multimap<String, String> config = LinkedHashMultimap.create();
Object inputRequestBody = null;
RestResource rsrc = TopLevelResource.INSTANCE;
ViewData viewData = null;
@ -257,6 +259,8 @@ public class RestApiServlet extends HttpServlet {
checkCors(req, res);
checkUserSession(req);
ParameterParser.splitQueryString(req.getQueryString(), config, params);
List<IdString> path = splitPath(req);
RestCollection<RestResource, RestResource> rc = members.get();
CapabilityUtils.checkRequiresCapability(globals.currentUser,
@ -265,6 +269,10 @@ public class RestApiServlet extends HttpServlet {
viewData = new ViewData(null, null);
if (path.isEmpty()) {
if (rc instanceof NeedsParams) {
((NeedsParams)rc).setParams(params);
}
if (isRead(req)) {
viewData = new ViewData(null, rc.list());
} else if (rc instanceof AcceptsPost && "POST".equals(req.getMethod())) {
@ -357,8 +365,6 @@ public class RestApiServlet extends HttpServlet {
return;
}
Multimap<String, String> config = LinkedHashMultimap.create();
ParameterParser.splitQueryString(req.getQueryString(), config, params);
if (!globals.paramParser.get().parse(viewData.view, params, req, res)) {
return;
}

199
gerrit-lucene/src/main/java/com/google/gerrit/lucene/LuceneGroupIndex.java Normal file
View File

@ -0,0 +1,199 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.lucene;
import static com.google.gerrit.server.index.group.GroupField.UUID;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.IndexUtils;
import com.google.gerrit.server.index.QueryOptions;
import com.google.gerrit.server.index.Schema;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.gerrit.server.query.DataSource;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.QueryParseException;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.ResultSet;
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.assistedinject.Assisted;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.SearcherFactory;
import org.apache.lucene.search.Sort;
import org.apache.lucene.search.SortField;
import org.apache.lucene.search.TopFieldDocs;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.store.RAMDirectory;
import org.eclipse.jgit.lib.Config;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.ExecutionException;
public class LuceneGroupIndex extends
AbstractLuceneIndex<AccountGroup.UUID, AccountGroup> implements GroupIndex {
private static final Logger log =
LoggerFactory.getLogger(LuceneGroupIndex.class);
private static final String GROUPS = "groups";
private static final String UUID_SORT_FIELD = sortFieldName(UUID);
private static Term idTerm(AccountGroup group) {
return idTerm(group.getGroupUUID());
}
private static Term idTerm(AccountGroup.UUID uuid) {
return QueryBuilder.stringTerm(UUID.getName(), uuid.get());
}
private final GerritIndexWriterConfig indexWriterConfig;
private final QueryBuilder<AccountGroup> queryBuilder;
private final Provider<GroupCache> groupCache;
private static Directory dir(Schema<AccountGroup> schema, Config cfg,
SitePaths sitePaths) throws IOException {
if (LuceneIndexModule.isInMemoryTest(cfg)) {
return new RAMDirectory();
}
Path indexDir =
LuceneVersionManager.getDir(sitePaths, GROUPS + "_", schema);
return FSDirectory.open(indexDir);
}
@Inject
LuceneGroupIndex(
@GerritServerConfig Config cfg,
SitePaths sitePaths,
Provider<GroupCache> groupCache,
@Assisted Schema<AccountGroup> schema) throws IOException {
super(schema, sitePaths, dir(schema, cfg, sitePaths), GROUPS, null,
new GerritIndexWriterConfig(cfg, GROUPS), new SearcherFactory());
this.groupCache = groupCache;
indexWriterConfig =
new GerritIndexWriterConfig(cfg, GROUPS);
queryBuilder = new QueryBuilder<>(schema, indexWriterConfig.getAnalyzer());
}
@Override
public void replace(AccountGroup group) throws IOException {
try {
// No parts of FillArgs are currently required, just use null.
replace(idTerm(group), toDocument(group, null)).get();
} catch (ExecutionException | InterruptedException e) {
throw new IOException(e);
}
}
@Override
public void delete(AccountGroup.UUID key) throws IOException {
try {
delete(idTerm(key)).get();
} catch (ExecutionException | InterruptedException e) {
throw new IOException(e);
}
}
@Override
public DataSource<AccountGroup> getSource(Predicate<AccountGroup> p,
QueryOptions opts) throws QueryParseException {
return new QuerySource(opts, queryBuilder.toQuery(p),
new Sort(new SortField(UUID_SORT_FIELD, SortField.Type.STRING, false)));
}
private class QuerySource implements DataSource<AccountGroup> {
private final QueryOptions opts;
private final Query query;
private final Sort sort;
private QuerySource(QueryOptions opts, Query query, Sort sort) {
this.opts = opts;
this.query = query;
this.sort = sort;
}
@Override
public int getCardinality() {
return 10;
}
@Override
public ResultSet<AccountGroup> read() throws OrmException {
IndexSearcher searcher = null;
try {
searcher = acquire();
int realLimit = opts.start() + opts.limit();
TopFieldDocs docs = searcher.search(query, realLimit, sort);
List<AccountGroup> result = new ArrayList<>(docs.scoreDocs.length);
for (int i = opts.start(); i < docs.scoreDocs.length; i++) {
ScoreDoc sd = docs.scoreDocs[i];
Document doc = searcher.doc(sd.doc, IndexUtils.groupFields(opts));
result.add(toAccountGroup(doc));
}
final List<AccountGroup> r = Collections.unmodifiableList(result);
return new ResultSet<AccountGroup>() {
@Override
public Iterator<AccountGroup> iterator() {
return r.iterator();
}
@Override
public List<AccountGroup> toList() {
return r;
}
@Override
public void close() {
// Do nothing.
}
};
} catch (IOException e) {
throw new OrmException(e);
} finally {
if (searcher != null) {
try {
release(searcher);
} catch (IOException e) {
log.warn("cannot release Lucene searcher", e);
}
}
}
}
}
private AccountGroup toAccountGroup(Document doc) {
AccountGroup.UUID uuid =
new AccountGroup.UUID(doc.getField(UUID.getName()).stringValue());
// Use the GroupCache rather than depending on any stored fields in the
// document (of which there shouldn't be any).
return groupCache.get().get(uuid);
}
}

9
gerrit-lucene/src/main/java/com/google/gerrit/lucene/LuceneIndexModule.java
View File

@ -22,6 +22,7 @@ import com.google.gerrit.server.index.IndexModule;
import com.google.gerrit.server.index.SingleVersionModule;
import com.google.gerrit.server.index.account.AccountIndex;
import com.google.gerrit.server.index.change.ChangeIndex;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.inject.Provides;
import com.google.inject.Singleton;
import com.google.inject.assistedinject.FactoryModuleBuilder;
@ -59,14 +60,18 @@ public class LuceneIndexModule extends LifecycleModule {
@Override
protected void configure() {
install(
new FactoryModuleBuilder()
.implement(AccountIndex.class, LuceneAccountIndex.class)
.build(AccountIndex.Factory.class));
install(
new FactoryModuleBuilder()
.implement(ChangeIndex.class, LuceneChangeIndex.class)
.build(ChangeIndex.Factory.class));
install(
new FactoryModuleBuilder()
.implement(AccountIndex.class, LuceneAccountIndex.class)
.build(AccountIndex.Factory.class));
.implement(GroupIndex.class, LuceneGroupIndex.class)
.build(GroupIndex.Factory.class));
install(new IndexModule(threads));
if (singleVersions == null) {

7
gerrit-lucene/src/main/java/com/google/gerrit/lucene/QueryBuilder.java
View File

@ -15,6 +15,7 @@
package com.google.gerrit.lucene;
import static com.google.common.base.Preconditions.checkArgument;
import static java.nio.charset.StandardCharsets.UTF_8;
import static org.apache.lucene.search.BooleanClause.Occur.MUST;
import static org.apache.lucene.search.BooleanClause.Occur.MUST_NOT;
import static org.apache.lucene.search.BooleanClause.Occur.SHOULD;
@ -54,6 +55,12 @@ public class QueryBuilder<V> {
return new Term(name, builder.get());
}
static Term stringTerm(String name, String value) {
BytesRefBuilder builder = new BytesRefBuilder();
builder.append(value.getBytes(UTF_8), 0, value.length());
return new Term(name, builder.get());
}
private final Schema<V> schema;
private final org.apache.lucene.util.QueryBuilder queryBuilder;

4
gerrit-pgm/BUCK
View File

@ -41,12 +41,14 @@ java_sources(
java_library(
name = 'init',
srcs = glob([SRCS + 'init/*.java']),
srcs = glob([SRCS + 'init/**/*.java']),
resources = glob([RSRCS + 'init/*']),
deps = DEPS + [
':init-api',
':util',
'//gerrit-common:annotations',
'//gerrit-elasticsearch:elasticsearch',
'//gerrit-lucene:lucene',
'//lib:args4j',
'//lib:derby',
'//lib:gwtjsonrpc',

3
gerrit-pgm/BUILD
View File

@ -40,13 +40,14 @@ java_library(
java_library(
name = "init",
srcs = glob([SRCS + "init/*.java"]),
srcs = glob([SRCS + "init/**/*.java"]),
resources = glob([RSRCS + "init/*"]),
visibility = ["//visibility:public"],
deps = DEPS + [
":init-api",
":util",
"//gerrit-common:annotations",
'//gerrit-elasticsearch:elasticsearch',
"//gerrit-launcher:launcher", # We want this dep to be provided_deps
"//gerrit-lucene:lucene",
"//lib:args4j",

77
gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/BaseInit.java
View File

@ -28,12 +28,16 @@ import com.google.gerrit.pgm.init.api.InitFlags;
import com.google.gerrit.pgm.init.api.InstallAllPlugins;
import com.google.gerrit.pgm.init.api.InstallPlugins;
import com.google.gerrit.pgm.init.api.LibraryDownload;
import com.google.gerrit.pgm.init.index.IndexManagerOnInit;
import com.google.gerrit.pgm.init.index.elasticsearch.ElasticIndexModuleOnInit;
import com.google.gerrit.pgm.init.index.lucene.LuceneIndexModuleOnInit;
import com.google.gerrit.pgm.util.SiteProgram;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.config.GerritServerConfigModule;
import com.google.gerrit.server.config.SitePath;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.index.IndexModule;
import com.google.gerrit.server.plugins.JarScanner;
import com.google.gerrit.server.schema.SchemaUpdater;
import com.google.gerrit.server.schema.UpdateUI;
@ -343,48 +347,58 @@ public class BaseInit extends SiteProgram {
final SchemaUpdater schemaUpdater;
final SchemaFactory<ReviewDb> schema;
final GitRepositoryManager repositoryManager;
final IndexManagerOnInit indexManager;
@Inject
SiteRun(final ConsoleUI ui, final SitePaths site, final InitFlags flags,
final SchemaUpdater schemaUpdater,
final SchemaFactory<ReviewDb> schema,
final GitRepositoryManager repositoryManager) {
SiteRun(ConsoleUI ui,
SitePaths site,
InitFlags flags,
SchemaUpdater schemaUpdater,
SchemaFactory<ReviewDb> schema,
GitRepositoryManager repositoryManager,
IndexManagerOnInit indexManager) {
this.ui = ui;
this.site = site;
this.flags = flags;
this.schemaUpdater = schemaUpdater;
this.schema = schema;
this.repositoryManager = repositoryManager;
this.indexManager = indexManager;
}
void upgradeSchema() throws OrmException {
final List<String> pruneList = new ArrayList<>();
schemaUpdater.update(new UpdateUI() {
@Override
public void message(String msg) {
System.err.println(msg);
System.err.flush();
}
try {
indexManager.start();
schemaUpdater.update(new UpdateUI() {
@Override
public void message(String msg) {
System.err.println(msg);
System.err.flush();
}
@Override
public boolean yesno(boolean def, String msg) {
return ui.yesno(def, msg);
}
@Override
public boolean yesno(boolean def, String msg) {
return ui.yesno(def, msg);
}
@Override
public boolean isBatch() {
return ui.isBatch();
}
@Override
public boolean isBatch() {
return ui.isBatch();
}
@Override
public void pruneSchema(StatementExecutor e, List<String> prune) {
for (String p : prune) {
if (!pruneList.contains(p)) {
pruneList.add(p);
@Override
public void pruneSchema(StatementExecutor e, List<String> prune) {
for (String p : prune) {
if (!pruneList.contains(p)) {
pruneList.add(p);
}
}
}
}
});
});
} finally {
indexManager.stop();
}
if (!pruneList.isEmpty()) {
StringBuilder msg = new StringBuilder();
@ -426,7 +440,18 @@ public class BaseInit extends SiteProgram {
bind(InitFlags.class).toInstance(init.flags);
}
});
sysInjector = createDbInjector(SINGLE_USER).createChildInjector(modules);
Injector dbInjector = createDbInjector(SINGLE_USER);
switch (IndexModule.getIndexType(dbInjector)) {
case LUCENE:
modules.add(new LuceneIndexModuleOnInit());
break;
case ELASTICSEARCH:
modules.add(new ElasticIndexModuleOnInit());
break;
default:
throw new IllegalStateException("unsupported index.type");
}
sysInjector = dbInjector.createChildInjector(modules);
}
return sysInjector;
}

55
gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/index/IndexManagerOnInit.java Normal file
View File

@ -0,0 +1,55 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.pgm.init.index;
import com.google.gerrit.extensions.events.LifecycleListener;
import com.google.gerrit.server.index.IndexDefinition;
import com.google.inject.Inject;
import com.google.inject.name.Named;
import java.util.Collection;
/**
* This class starts/stops the indexes from the init program so that init can
* write updates the indexes.
*/
public class IndexManagerOnInit {
private final LifecycleListener indexManager;
private final Collection<IndexDefinition<?, ?, ?>> defs;
@Inject
IndexManagerOnInit(
@Named(IndexModuleOnInit.INDEX_MANAGER) LifecycleListener indexManager,
Collection<IndexDefinition<?, ?, ?>> defs) {
this.indexManager = indexManager;
this.defs = defs;
}
public void start() {
indexManager.start();
for (IndexDefinition<?, ?, ?> def : defs) {
def.getIndexCollection().start();
}
}
public void stop() {
indexManager.stop();
for (IndexDefinition<?, ?, ?> def : defs) {
def.getIndexCollection().stop();
}
}
}

85
gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/index/IndexModuleOnInit.java Normal file
View File

@ -0,0 +1,85 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.pgm.init.index;
import com.google.common.collect.FluentIterable;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.gerrit.extensions.events.LifecycleListener;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.index.IndexDefinition;
import com.google.gerrit.server.index.SchemaDefinitions;
import com.google.gerrit.server.index.SingleVersionModule;
import com.google.gerrit.server.index.SingleVersionModule.SingleVersionListener;
import com.google.gerrit.server.index.group.AllGroupsIndexer;
import com.google.gerrit.server.index.group.GroupIndexCollection;
import com.google.gerrit.server.index.group.GroupIndexDefinition;
import com.google.gerrit.server.index.group.GroupSchemaDefinitions;
import com.google.inject.AbstractModule;
import com.google.inject.Provides;
import com.google.inject.ProvisionException;
import com.google.inject.TypeLiteral;
import com.google.inject.name.Names;
import com.google.inject.util.Providers;
import java.util.Collection;
import java.util.Map;
import java.util.Set;
public class IndexModuleOnInit extends AbstractModule {
static final String INDEX_MANAGER = "IndexModuleOnInit/IndexManager";
@Override
protected void configure() {
// The GroupIndex implementations (LuceneGroupIndex and ElasticGroupIndex)
// need GroupCache only for reading from the index. On init we only want to
// write to the index, hence we don't need the group cache.
bind(GroupCache.class).toProvider(Providers.of(null));
// GroupIndexDefinition wants to have AllGroupsIndexer but it is only used
// by the Reindex program and the OnlineReindexer which are both not used
// during init, hence we don't need AllGroupsIndexer.
bind(AllGroupsIndexer.class).toProvider(Providers.of(null));
bind(GroupIndexCollection.class);
bind(new TypeLiteral<Map<String, Integer>>() {})
.annotatedWith(Names.named(SingleVersionModule.SINGLE_VERSIONS))
.toInstance(ImmutableMap.<String, Integer> of());
bind(LifecycleListener.class).annotatedWith(Names.named(INDEX_MANAGER))
.to(SingleVersionListener.class);
}
@Provides
Collection<IndexDefinition<?, ?, ?>> getIndexDefinitions(
GroupIndexDefinition groups) {
Collection<IndexDefinition<?, ?, ?>> result =
ImmutableList.<IndexDefinition<?, ?, ?>> of(
groups);
Set<String> expected =
FluentIterable.of(GroupSchemaDefinitions.INSTANCE)
.transform(SchemaDefinitions::getName)
.toSet();
Set<String> actual = FluentIterable.from(result)
.transform(IndexDefinition::getName)
.toSet();
if (!expected.equals(actual)) {
throw new ProvisionException(
"need index definitions for all schemas: "
+ expected + " != " + actual);
}
return result;
}
}

34
gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/index/elasticsearch/ElasticIndexModuleOnInit.java Normal file
View File

@ -0,0 +1,34 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.pgm.init.index.elasticsearch;
import com.google.gerrit.elasticsearch.ElasticGroupIndex;
import com.google.gerrit.pgm.init.index.IndexModuleOnInit;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.inject.AbstractModule;
import com.google.inject.assistedinject.FactoryModuleBuilder;
public class ElasticIndexModuleOnInit extends AbstractModule {
@Override
protected void configure() {
install(
new FactoryModuleBuilder()
.implement(GroupIndex.class, ElasticGroupIndex.class)
.build(GroupIndex.Factory.class));
install(new IndexModuleOnInit());
}
}

33
gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/index/lucene/LuceneIndexModuleOnInit.java Normal file
View File

@ -0,0 +1,33 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.pgm.init.index.lucene;
import com.google.gerrit.lucene.LuceneGroupIndex;
import com.google.gerrit.pgm.init.index.IndexModuleOnInit;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.inject.AbstractModule;
import com.google.inject.assistedinject.FactoryModuleBuilder;
public class LuceneIndexModuleOnInit extends AbstractModule {
@Override
protected void configure() {
install(
new FactoryModuleBuilder()
.implement(GroupIndex.class, LuceneGroupIndex.class)
.build(GroupIndex.Factory.class));
install(new IndexModuleOnInit());
}
}

10
gerrit-server/src/main/java/com/google/gerrit/server/account/GroupCache.java
View File

@ -18,6 +18,8 @@ import com.google.common.collect.ImmutableList;
import com.google.gerrit.common.Nullable;
import com.google.gerrit.reviewdb.client.AccountGroup;
import java.io.IOException;
/** Tracks group objects in memory for efficient access. */
public interface GroupCache {
AccountGroup get(AccountGroup.Id groupId);
@ -36,10 +38,10 @@ public interface GroupCache {
ImmutableList<AccountGroup> all();
/** Notify the cache that a new group was constructed. */
void onCreateGroup(AccountGroup.NameKey newGroupName);
void onCreateGroup(AccountGroup.NameKey newGroupName) throws IOException;
void evict(AccountGroup group);
void evict(AccountGroup group) throws IOException;
void evictAfterRename(final AccountGroup.NameKey oldName,
final AccountGroup.NameKey newName);
void evictAfterRename(AccountGroup.NameKey oldName,
AccountGroup.NameKey newName) throws IOException;
}

18
gerrit-server/src/main/java/com/google/gerrit/server/account/GroupCacheImpl.java
View File

@ -21,11 +21,13 @@ import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.client.AccountGroupName;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.cache.CacheModule;
import com.google.gerrit.server.index.group.GroupIndexer;
import com.google.gwtorm.server.OrmDuplicateKeyException;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.SchemaFactory;
import com.google.inject.Inject;
import com.google.inject.Module;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import com.google.inject.TypeLiteral;
import com.google.inject.name.Named;
@ -33,6 +35,7 @@ import com.google.inject.name.Named;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.ExecutionException;
@ -76,17 +79,20 @@ public class GroupCacheImpl implements GroupCache {
private final LoadingCache<String, Optional<AccountGroup>> byName;
private final LoadingCache<String, Optional<AccountGroup>> byUUID;
private final SchemaFactory<ReviewDb> schema;
private final Provider<GroupIndexer> indexer;
@Inject
GroupCacheImpl(
@Named(BYID_NAME) LoadingCache<AccountGroup.Id, Optional<AccountGroup>> byId,
@Named(BYNAME_NAME) LoadingCache<String, Optional<AccountGroup>> byName,
@Named(BYUUID_NAME) LoadingCache<String, Optional<AccountGroup>> byUUID,
SchemaFactory<ReviewDb> schema) {
SchemaFactory<ReviewDb> schema,
Provider<GroupIndexer> indexer) {
this.byId = byId;
this.byName = byName;
this.byUUID = byUUID;
this.schema = schema;
this.indexer = indexer;
}
@Override
@ -101,7 +107,7 @@ public class GroupCacheImpl implements GroupCache {
}
@Override
public void evict(final AccountGroup group) {
public void evict(final AccountGroup group) throws IOException {
if (group.getId() != null) {
byId.invalidate(group.getId());
}
@ -111,17 +117,19 @@ public class GroupCacheImpl implements GroupCache {
if (group.getGroupUUID() != null) {
byUUID.invalidate(group.getGroupUUID().get());
}
indexer.get().index(group.getGroupUUID());
}
@Override
public void evictAfterRename(final AccountGroup.NameKey oldName,
final AccountGroup.NameKey newName) {
final AccountGroup.NameKey newName) throws IOException {
if (oldName != null) {
byName.invalidate(oldName.get());
}
if (newName != null) {
byName.invalidate(newName.get());
}
indexer.get().index(get(newName).getGroupUUID());
}
@Override
@ -161,8 +169,10 @@ public class GroupCacheImpl implements GroupCache {
}
@Override
public void onCreateGroup(AccountGroup.NameKey newGroupName) {
public void onCreateGroup(AccountGroup.NameKey newGroupName)
throws IOException {
byName.invalidate(newGroupName.get());
indexer.get().index(get(newGroupName).getGroupUUID());
}
private static AccountGroup missing(AccountGroup.Id key) {

21
gerrit-server/src/main/java/com/google/gerrit/server/api/groups/GroupApiImpl.java
View File

@ -34,6 +34,7 @@ import com.google.gerrit.server.group.GetName;
import com.google.gerrit.server.group.GetOptions;
import com.google.gerrit.server.group.GetOwner;
import com.google.gerrit.server.group.GroupResource;
import com.google.gerrit.server.group.Index;
import com.google.gerrit.server.group.ListIncludedGroups;
import com.google.gerrit.server.group.ListMembers;
import com.google.gerrit.server.group.PutDescription;
@ -71,6 +72,7 @@ class GroupApiImpl implements GroupApi {
private final DeleteIncludedGroups deleteGroups;
private final GetAuditLog getAuditLog;
private final GroupResource rsrc;
private final Index index;
@AssistedInject
GroupApiImpl(
@ -91,6 +93,7 @@ class GroupApiImpl implements GroupApi {
AddIncludedGroups addGroups,
DeleteIncludedGroups deleteGroups,
GetAuditLog getAuditLog,
Index index,
@Assisted GroupResource rsrc) {
this.getGroup = getGroup;
this.getDetail = getDetail;
@ -109,6 +112,7 @@ class GroupApiImpl implements GroupApi {
this.addGroups = addGroups;
this.deleteGroups = deleteGroups;
this.getAuditLog = getAuditLog;
this.index = index;
this.rsrc = rsrc;
}
@ -143,7 +147,7 @@ class GroupApiImpl implements GroupApi {
putName.apply(rsrc, in);
} catch (NoSuchGroupException e) {
throw new ResourceNotFoundException(name, e);
} catch (OrmException e) {
} catch (OrmException | IOException e) {
throw new RestApiException("Cannot put group name", e);
}
}
@ -163,7 +167,7 @@ class GroupApiImpl implements GroupApi {
in.owner = owner;
try {
putOwner.apply(rsrc, in);
} catch (OrmException e) {
} catch (OrmException | IOException e) {
throw new RestApiException("Cannot put group owner", e);
}
}
@ -179,7 +183,7 @@ class GroupApiImpl implements GroupApi {
in.description = description;
try {
putDescription.apply(rsrc, in);
} catch (OrmException e) {
} catch (OrmException | IOException e) {
throw new RestApiException("Cannot put group description", e);
}
}
@ -193,7 +197,7 @@ class GroupApiImpl implements GroupApi {
public void options(GroupOptionsInfo options) throws RestApiException {
try {
putOptions.apply(rsrc, options);
} catch (OrmException e) {
} catch (OrmException | IOException e) {
throw new RestApiException("Cannot put group options", e);
}
}
@ -270,4 +274,13 @@ class GroupApiImpl implements GroupApi {
throw new RestApiException("Cannot get audit log", e);
}
}
@Override
public void index() throws RestApiException {
try {
index.apply(rsrc, new Index.Input());
} catch (IOException e) {
throw new RestApiException("Cannot index group", e);
}
}
}

37
gerrit-server/src/main/java/com/google/gerrit/server/api/groups/GroupsImpl.java
View File

@ -20,6 +20,7 @@ import static com.google.gerrit.server.account.CapabilityUtils.checkRequiresCapa
import com.google.gerrit.extensions.api.groups.GroupApi;
import com.google.gerrit.extensions.api.groups.GroupInput;
import com.google.gerrit.extensions.api.groups.Groups;
import com.google.gerrit.extensions.client.ListGroupsOption;
import com.google.gerrit.extensions.common.GroupInfo;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.IdString;
@ -30,6 +31,7 @@ import com.google.gerrit.server.account.AccountsCollection;
import com.google.gerrit.server.group.CreateGroup;
import com.google.gerrit.server.group.GroupsCollection;
import com.google.gerrit.server.group.ListGroups;
import com.google.gerrit.server.group.QueryGroups;
import com.google.gerrit.server.project.ProjectsCollection;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
@ -37,6 +39,7 @@ import com.google.inject.Provider;
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.List;
import java.util.SortedMap;
@Singleton
@ -45,6 +48,7 @@ class GroupsImpl implements Groups {
private final GroupsCollection groups;
private final ProjectsCollection projects;
private final Provider<ListGroups> listGroups;
private final Provider<QueryGroups> queryGroups;
private final Provider<CurrentUser> user;
private final CreateGroup.Factory createGroup;
private final GroupApiImpl.Factory api;
@ -55,6 +59,7 @@ class GroupsImpl implements Groups {
GroupsCollection groups,
ProjectsCollection projects,
Provider<ListGroups> listGroups,
Provider<QueryGroups> queryGroups,
Provider<CurrentUser> user,
CreateGroup.Factory createGroup,
GroupApiImpl.Factory api) {
@ -62,6 +67,7 @@ class GroupsImpl implements Groups {
this.groups = groups;
this.projects = projects;
this.listGroups = listGroups;
this.queryGroups = queryGroups;
this.user = user;
this.createGroup = createGroup;
this.api = api;
@ -145,4 +151,35 @@ class GroupsImpl implements Groups {
throw new RestApiException("Cannot list groups", e);
}
}
@Override
public QueryRequest query() {
return new QueryRequest() {
@Override
public List<GroupInfo> get() throws RestApiException {
return GroupsImpl.this.query(this);
}
};
}
@Override
public QueryRequest query(String query) {
return query().withQuery(query);
}
private List<GroupInfo> query(QueryRequest r)
throws RestApiException {
try {
QueryGroups myQueryGroups = queryGroups.get();
myQueryGroups.setQuery(r.getQuery());
myQueryGroups.setLimit(r.getLimit());
myQueryGroups.setStart(r.getStart());
for (ListGroupsOption option : r.getOptions()) {
myQueryGroups.addOption(option);
}
return myQueryGroups.apply(TopLevelResource.INSTANCE);
} catch (OrmException e) {
throw new RestApiException("Cannot query groups", e);
}
}
}

33
gerrit-server/src/main/java/com/google/gerrit/server/group/GroupsCollection.java
View File

@ -14,6 +14,7 @@
package com.google.gerrit.server.group;
import com.google.common.collect.Multimap;
import com.google.gerrit.common.data.GroupDescription;
import com.google.gerrit.common.data.GroupDescriptions;
import com.google.gerrit.common.data.GroupReference;
@ -22,6 +23,7 @@ import com.google.gerrit.extensions.registration.DynamicMap;
import com.google.gerrit.extensions.restapi.AcceptsCreate;
import com.google.gerrit.extensions.restapi.AuthException;
import com.google.gerrit.extensions.restapi.IdString;
import com.google.gerrit.extensions.restapi.NeedsParams;
import com.google.gerrit.extensions.restapi.ResourceNotFoundException;
import com.google.gerrit.extensions.restapi.RestCollection;
import com.google.gerrit.extensions.restapi.RestView;
@ -35,34 +37,43 @@ import com.google.gerrit.server.account.GroupBackends;
import com.google.gerrit.server.account.GroupControl;
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
@Singleton
public class GroupsCollection implements
RestCollection<TopLevelResource, GroupResource>,
AcceptsCreate<TopLevelResource> {
AcceptsCreate<TopLevelResource>, NeedsParams {
private final DynamicMap<RestView<GroupResource>> views;
private final Provider<ListGroups> list;
private final Provider<QueryGroups> queryGroups;
private final CreateGroup.Factory createGroup;
private final GroupControl.Factory groupControlFactory;
private final GroupBackend groupBackend;
private final Provider<CurrentUser> self;
private boolean hasQuery2;
@Inject
GroupsCollection(final DynamicMap<RestView<GroupResource>> views,
final Provider<ListGroups> list,
final CreateGroup.Factory createGroup,
final GroupControl.Factory groupControlFactory,
final GroupBackend groupBackend,
final Provider<CurrentUser> self) {
GroupsCollection(DynamicMap<RestView<GroupResource>> views,
Provider<ListGroups> list,
Provider<QueryGroups> queryGroups,
CreateGroup.Factory createGroup,
GroupControl.Factory groupControlFactory,
GroupBackend groupBackend,
Provider<CurrentUser> self) {
this.views = views;
this.list = list;
this.queryGroups = queryGroups;
this.createGroup = createGroup;
this.groupControlFactory = groupControlFactory;
this.groupBackend = groupBackend;
this.self = self;
}
@Override
public void setParams(Multimap<String, String> params) {
// The --query2 option is defined in QueryGroups
this.hasQuery2 = params.containsKey("query2");
}
@Override
public RestView<TopLevelResource> list() throws ResourceNotFoundException,
AuthException {
@ -73,6 +84,10 @@ public class GroupsCollection implements
throw new ResourceNotFoundException();
}
if (hasQuery2) {
return queryGroups.get();
}
return list.get();
}

59
gerrit-server/src/main/java/com/google/gerrit/server/group/Index.java Normal file
View File

@ -0,0 +1,59 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.group;
import com.google.gerrit.common.data.GroupDescriptions;
import com.google.gerrit.extensions.restapi.AuthException;
import com.google.gerrit.extensions.restapi.Response;
import com.google.gerrit.extensions.restapi.RestModifyView;
import com.google.gerrit.extensions.restapi.UnprocessableEntityException;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.group.Index.Input;
import com.google.inject.Inject;
import com.google.inject.Singleton;
import java.io.IOException;
@Singleton
public class Index implements RestModifyView<GroupResource, Input> {
public static class Input {
}
private final GroupCache groupCache;
@Inject
Index(GroupCache groupCache) {
this.groupCache = groupCache;
}
@Override
public Response<?> apply(GroupResource rsrc, Input input)
throws IOException, AuthException, UnprocessableEntityException {
if (!rsrc.getControl().isOwner()) {
throw new AuthException("not allowed to index group");
}
AccountGroup group = GroupDescriptions.toAccountGroup(rsrc.getGroup());
if (group == null) {
throw new UnprocessableEntityException(String
.format("External Group Not Allowed: %s", rsrc.getGroupUUID().get()));
}
// evicting the group from the cache, reindexes the account
groupCache.evict(group);
return Response.none();
}
}

2
gerrit-server/src/main/java/com/google/gerrit/server/group/ListGroups.java
View File

@ -106,7 +106,7 @@ public class ListGroups implements RestReadView<TopLevelResource> {
this.owned = owned;
}
@Option(name = "-q", usage = "group to inspect")
@Option(name = "--query", aliases = {"-q"}, usage = "group to inspect")
public void addGroup(AccountGroup.UUID id) {
groupsToInspect.add(id);
}

1
gerrit-server/src/main/java/com/google/gerrit/server/group/Module.java
View File

@ -39,6 +39,7 @@ public class Module extends RestApiModule {
get(GROUP_KIND).to(GetGroup.class);
put(GROUP_KIND).to(PutGroup.class);
get(GROUP_KIND, "detail").to(GetDetail.class);
post(GROUP_KIND, "index").to(Index.class);
post(GROUP_KIND, "members").to(AddMembers.class);
post(GROUP_KIND, "members.add").to(AddMembers.class);
post(GROUP_KIND, "members.delete").to(DeleteMembers.class);

3
gerrit-server/src/main/java/com/google/gerrit/server/group/PutDescription.java
View File

@ -30,6 +30,7 @@ import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.Collections;
@Singleton
@ -51,7 +52,7 @@ public class PutDescription implements RestModifyView<GroupResource, Input> {
@Override
public Response<String> apply(GroupResource resource, Input input)
throws AuthException, MethodNotAllowedException,
ResourceNotFoundException, OrmException {
ResourceNotFoundException, OrmException, IOException {
if (input == null) {
input = new Input(); // Delete would set description to null.
}

6
gerrit-server/src/main/java/com/google/gerrit/server/group/PutName.java
View File

@ -36,6 +36,7 @@ import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.Collections;
import java.util.Date;
import java.util.TimeZone;
@ -70,7 +71,8 @@ public class PutName implements RestModifyView<GroupResource, Input> {
@Override
public String apply(GroupResource rsrc, Input input)
throws MethodNotAllowedException, AuthException, BadRequestException,
ResourceConflictException, OrmException, NoSuchGroupException {
ResourceConflictException, OrmException, NoSuchGroupException,
IOException {
if (rsrc.toAccountGroup() == null) {
throw new MethodNotAllowedException();
} else if (!rsrc.getControl().isOwner()) {
@ -92,7 +94,7 @@ public class PutName implements RestModifyView<GroupResource, Input> {
private GroupDetail renameGroup(AccountGroup group, String newName)
throws ResourceConflictException, OrmException,
NoSuchGroupException {
NoSuchGroupException, IOException {
AccountGroup.Id groupId = group.getId();
AccountGroup.NameKey old = group.getNameKey();
AccountGroup.NameKey key = new AccountGroup.NameKey(newName);

3
gerrit-server/src/main/java/com/google/gerrit/server/group/PutOptions.java
View File

@ -28,6 +28,7 @@ import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.Collections;
@Singleton
@ -45,7 +46,7 @@ public class PutOptions
@Override
public GroupOptionsInfo apply(GroupResource resource, GroupOptionsInfo input)
throws MethodNotAllowedException, AuthException, BadRequestException,
ResourceNotFoundException, OrmException {
ResourceNotFoundException, OrmException, IOException {
if (resource.toAccountGroup() == null) {
throw new MethodNotAllowedException();
} else if (!resource.getControl().isOwner()) {

3
gerrit-server/src/main/java/com/google/gerrit/server/group/PutOwner.java
View File

@ -33,6 +33,7 @@ import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import java.io.IOException;
import java.util.Collections;
@Singleton
@ -60,7 +61,7 @@ public class PutOwner implements RestModifyView<GroupResource, Input> {
public GroupInfo apply(GroupResource resource, Input input)
throws ResourceNotFoundException, MethodNotAllowedException,
AuthException, BadRequestException, UnprocessableEntityException,
OrmException {
OrmException, IOException {
AccountGroup group = resource.toAccountGroup();
if (group == null) {
throw new MethodNotAllowedException();

132
gerrit-server/src/main/java/com/google/gerrit/server/group/QueryGroups.java Normal file
View File

@ -0,0 +1,132 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.group;
import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import com.google.gerrit.common.data.GroupDescriptions;
import com.google.gerrit.extensions.client.ListGroupsOption;
import com.google.gerrit.extensions.common.GroupInfo;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.extensions.restapi.MethodNotAllowedException;
import com.google.gerrit.extensions.restapi.RestReadView;
import com.google.gerrit.extensions.restapi.TopLevelResource;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.gerrit.server.index.group.GroupIndexCollection;
import com.google.gerrit.server.query.QueryParseException;
import com.google.gerrit.server.query.QueryResult;
import com.google.gerrit.server.query.group.GroupQueryBuilder;
import com.google.gerrit.server.query.group.GroupQueryProcessor;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
import org.kohsuke.args4j.Option;
import java.util.ArrayList;
import java.util.EnumSet;
import java.util.List;
public class QueryGroups implements RestReadView<TopLevelResource> {
private final GroupIndexCollection indexes;
private final GroupQueryBuilder queryBuilder;
private final GroupQueryProcessor queryProcessor;
private final GroupJson json;
private String query;
private int limit;
private int start;
private EnumSet<ListGroupsOption> options =
EnumSet.noneOf(ListGroupsOption.class);
/** --query (-q) is already used by {@link ListGroups} */
@Option(name = "--query2", aliases = {"-q2"}, usage = "group query")
public void setQuery(String query) {
this.query = query;
}
@Option(name = "--limit", aliases = {"-n"}, metaVar = "CNT",
usage = "maximum number of groups to list")
public void setLimit(int limit) {
this.limit = limit;
}
@Option(name = "--start", aliases = {"-S"}, metaVar = "CNT",
usage = "number of groups to skip")
public void setStart(int start) {
this.start = start;
}
@Option(name = "-o", usage = "Output options per group")
public void addOption(ListGroupsOption o) {
options.add(o);
}
@Option(name = "-O", usage = "Output option flags, in hex")
public void setOptionFlagsHex(String hex) {
options.addAll(ListGroupsOption.fromBits(Integer.parseInt(hex, 16)));
}
@Inject
protected QueryGroups(GroupIndexCollection indexes,
GroupQueryBuilder queryBuilder,
GroupQueryProcessor queryProcessor,
GroupJson json) {
this.indexes = indexes;
this.queryBuilder = queryBuilder;
this.queryProcessor = queryProcessor;
this.json = json;
}
@Override
public List<GroupInfo> apply(TopLevelResource resource)
throws BadRequestException, MethodNotAllowedException, OrmException {
if (Strings.isNullOrEmpty(query)) {
throw new BadRequestException("missing query field");
}
GroupIndex searchIndex = indexes.getSearchIndex();
if (searchIndex == null) {
throw new MethodNotAllowedException("no group index");
}
if (start != 0) {
queryProcessor.setStart(start);
}
if (limit != 0) {
queryProcessor.setLimit(limit);
}
try {
QueryResult<AccountGroup> result =
queryProcessor.query(queryBuilder.parse(query));
List<AccountGroup> groups = result.entities();
ArrayList<GroupInfo> groupInfos =
Lists.newArrayListWithCapacity(groups.size());
json.addOptions(options);
for (AccountGroup group : groups) {
groupInfos.add(json.format(GroupDescriptions.forAccountGroup(group)));
}
if (!groupInfos.isEmpty() && result.more()) {
groupInfos.get(groupInfos.size() - 1)._moreGroups = true;
}
return groupInfos;
} catch (QueryParseException e) {
throw new BadRequestException(e.getMessage());
}
}
}

10
gerrit-server/src/main/java/com/google/gerrit/server/index/DummyIndexModule.java
View File

@ -14,10 +14,12 @@
package com.google.gerrit.server.index;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.account.AccountState;
import com.google.gerrit.server.index.account.AccountIndex;
import com.google.gerrit.server.index.change.ChangeIndex;
import com.google.gerrit.server.index.change.DummyChangeIndex;
import com.google.gerrit.server.index.group.GroupIndex;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.inject.AbstractModule;
@ -36,6 +38,13 @@ public class DummyIndexModule extends AbstractModule {
}
}
private static class DummyGroupIndexFactory implements GroupIndex.Factory {
@Override
public GroupIndex create(Schema<AccountGroup> schema) {
throw new UnsupportedOperationException();
}
}
@Override
protected void configure() {
install(new IndexModule(1));
@ -43,5 +52,6 @@ public class DummyIndexModule extends AbstractModule {
bind(Index.class).toInstance(new DummyChangeIndex());
bind(AccountIndex.Factory.class).toInstance(new DummyAccountIndexFactory());
bind(ChangeIndex.Factory.class).toInstance(new DummyChangeIndexFactory());
bind(GroupIndex.Factory.class).toInstance(new DummyGroupIndexFactory());
}
}

7
gerrit-server/src/main/java/com/google/gerrit/server/index/IndexDefinition.java
View File

@ -15,6 +15,7 @@
package com.google.gerrit.server.index;
import com.google.common.collect.ImmutableSortedMap;
import com.google.inject.Provider;
/**
* Definition of an index over a Gerrit data type.
@ -32,13 +33,13 @@ public abstract class IndexDefinition<K, V, I extends Index<K, V>> {
private final SchemaDefinitions<V> schemaDefs;
private final IndexCollection<K, V, I> indexCollection;
private final IndexFactory<K, V, I> indexFactory;
private final SiteIndexer<K, V, I> siteIndexer;
private final Provider<SiteIndexer<K, V, I>> siteIndexer;
protected IndexDefinition(
SchemaDefinitions<V> schemaDefs,
IndexCollection<K, V, I> indexCollection,
IndexFactory<K, V, I> indexFactory,
SiteIndexer<K, V, I> siteIndexer) {
Provider<SiteIndexer<K, V, I>> siteIndexer) {
this.schemaDefs = schemaDefs;
this.indexCollection = indexCollection;
this.indexFactory = indexFactory;
@ -66,6 +67,6 @@ public abstract class IndexDefinition<K, V, I extends Index<K, V>> {
}
public final SiteIndexer<K, V, I> getSiteIndexer() {
return siteIndexer;
return siteIndexer.get();
}
}

25
gerrit-server/src/main/java/com/google/gerrit/server/index/IndexModule.java
View File

@ -36,6 +36,12 @@ import com.google.gerrit.server.index.change.ChangeIndexDefinition;
import com.google.gerrit.server.index.change.ChangeIndexRewriter;
import com.google.gerrit.server.index.change.ChangeIndexer;
import com.google.gerrit.server.index.change.ChangeSchemaDefinitions;
import com.google.gerrit.server.index.group.GroupIndexCollection;
import com.google.gerrit.server.index.group.GroupIndexDefinition;
import com.google.gerrit.server.index.group.GroupIndexRewriter;
import com.google.gerrit.server.index.group.GroupIndexer;
import com.google.gerrit.server.index.group.GroupIndexerImpl;
import com.google.gerrit.server.index.group.GroupSchemaDefinitions;
import com.google.inject.Injector;
import com.google.inject.Key;
import com.google.inject.Provides;
@ -61,7 +67,8 @@ public class IndexModule extends LifecycleModule {
public static final ImmutableCollection<SchemaDefinitions<?>> ALL_SCHEMA_DEFS =
ImmutableList.<SchemaDefinitions<?>> of(
AccountSchemaDefinitions.INSTANCE,
ChangeSchemaDefinitions.INSTANCE);
ChangeSchemaDefinitions.INSTANCE,
GroupSchemaDefinitions.INSTANCE);
/** Type of secondary index. */
public static IndexType getIndexType(Injector injector) {
@ -98,15 +105,22 @@ public class IndexModule extends LifecycleModule {
bind(ChangeIndexCollection.class);
listener().to(ChangeIndexCollection.class);
factory(ChangeIndexer.Factory.class);
bind(GroupIndexRewriter.class);
bind(GroupIndexCollection.class);
listener().to(GroupIndexCollection.class);
factory(GroupIndexerImpl.Factory.class);
}
@Provides
Collection<IndexDefinition<?, ?, ?>> getIndexDefinitions(
AccountIndexDefinition accounts,
ChangeIndexDefinition changes) {
ChangeIndexDefinition changes,
GroupIndexDefinition groups) {
Collection<IndexDefinition<?, ?, ?>> result =
ImmutableList.<IndexDefinition<?, ?, ?>> of(
accounts,
groups,
changes);
Set<String> expected = FluentIterable.from(ALL_SCHEMA_DEFS)
.transform(SchemaDefinitions::getName)
@ -140,6 +154,13 @@ public class IndexModule extends LifecycleModule {
return factory.create(executor, indexes);
}
@Provides
@Singleton
GroupIndexer getGroupIndexer(GroupIndexerImpl.Factory factory,
GroupIndexCollection indexes) {
return factory.create(indexes);
}
@Provides
@Singleton
@IndexExecutor(INTERACTIVE)

14
gerrit-server/src/main/java/com/google/gerrit/server/index/IndexUtils.java
View File

@ -14,7 +14,6 @@
package com.google.gerrit.server.index;
import static com.google.gerrit.server.index.account.AccountField.ID;
import static com.google.gerrit.server.index.change.ChangeField.CHANGE;
import static com.google.gerrit.server.index.change.ChangeField.LEGACY_ID;
import static com.google.gerrit.server.index.change.ChangeField.PROJECT;
@ -23,6 +22,8 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.account.AccountField;
import com.google.gerrit.server.index.group.GroupField;
import org.eclipse.jgit.errors.ConfigInvalidException;
@ -47,9 +48,9 @@ public final class IndexUtils {
public static Set<String> accountFields(QueryOptions opts) {
Set<String> fs = opts.fields();
return fs.contains(ID.getName())
return fs.contains(AccountField.ID.getName())
? fs
: Sets.union(fs, ImmutableSet.of(ID.getName()));
: Sets.union(fs, ImmutableSet.of(AccountField.ID.getName()));
}
public static Set<String> changeFields(QueryOptions opts) {
@ -68,6 +69,13 @@ public final class IndexUtils {
ImmutableSet.of(LEGACY_ID.getName(), PROJECT.getName()));
}
public static Set<String> groupFields(QueryOptions opts) {
Set<String> fs = opts.fields();
return fs.contains(GroupField.UUID.getName())
? fs
: Sets.union(fs, ImmutableSet.of(GroupField.UUID.getName()));
}
private IndexUtils() {
// hide default constructor
}

8
gerrit-server/src/main/java/com/google/gerrit/server/index/SchemaUtil.java
View File

@ -90,12 +90,16 @@ public class SchemaUtil {
if (person == null) {
return ImmutableSet.of();
}
return getPersonParts(
return getNameParts(
person.getName(),
Collections.singleton(person.getEmailAddress()));
}
public static Set<String> getPersonParts(String name,
public static Set<String> getNameParts(String name) {
return getNameParts(name, Collections.emptySet());
}
public static Set<String> getNameParts(String name,
Iterable<String> emails) {
Splitter at = Splitter.on('@');
Splitter s = Splitter.on(CharMatcher.anyOf("@.- ")).omitEmptyStrings();

4
gerrit-server/src/main/java/com/google/gerrit/server/index/SingleVersionModule.java
View File

@ -33,7 +33,7 @@ import java.util.Set;
@Singleton
public class SingleVersionModule extends LifecycleModule {
static final String SINGLE_VERSIONS = "IndexModule/SingleVersions";
public static final String SINGLE_VERSIONS = "IndexModule/SingleVersions";
private final Map<String, Integer> singleVersions;
@ -50,7 +50,7 @@ public class SingleVersionModule extends LifecycleModule {
}
@Singleton
static class SingleVersionListener implements LifecycleListener {
public static class SingleVersionListener implements LifecycleListener {
private final Set<String> disabled;
private final Collection<IndexDefinition<?, ?, ?>> defs;
private final Map<String, Integer> singleVersions;

2
gerrit-server/src/main/java/com/google/gerrit/server/index/account/AccountField.java
View File

@ -56,7 +56,7 @@ public class AccountField {
@Override
public Iterable<String> get(AccountState input, FillArgs args) {
String fullName = input.getAccount().getFullName();
Set<String> parts = SchemaUtil.getPersonParts(
Set<String> parts = SchemaUtil.getNameParts(
fullName,
Iterables.transform(
input.getExternalIds(),

3
gerrit-server/src/main/java/com/google/gerrit/server/index/account/AccountIndexDefinition.java
View File

@ -18,6 +18,7 @@ import com.google.gerrit.reviewdb.client.Account;
import com.google.gerrit.server.account.AccountState;
import com.google.gerrit.server.index.IndexDefinition;
import com.google.inject.Inject;
import com.google.inject.util.Providers;
public class AccountIndexDefinition
extends IndexDefinition<Account.Id, AccountState, AccountIndex> {
@ -28,6 +29,6 @@ public class AccountIndexDefinition
AccountIndex.Factory indexFactory,
AllAccountsIndexer allAccountsIndexer) {
super(AccountSchemaDefinitions.INSTANCE, indexCollection, indexFactory,
allAccountsIndexer);
Providers.of(allAccountsIndexer));
}
}

3
gerrit-server/src/main/java/com/google/gerrit/server/index/change/ChangeIndexDefinition.java
View File

@ -18,6 +18,7 @@ import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.server.index.IndexDefinition;
import com.google.gerrit.server.query.change.ChangeData;
import com.google.inject.Inject;
import com.google.inject.util.Providers;
public class ChangeIndexDefinition
extends IndexDefinition<Change.Id, ChangeData, ChangeIndex> {
@ -28,6 +29,6 @@ public class ChangeIndexDefinition
ChangeIndex.Factory indexFactory,
AllChangesIndexer allChangesIndexer) {
super(ChangeSchemaDefinitions.INSTANCE, indexCollection, indexFactory,
allChangesIndexer);
Providers.of(allChangesIndexer));
}
}

138
gerrit-server/src/main/java/com/google/gerrit/server/index/group/AllGroupsIndexer.java Normal file
View File

@ -0,0 +1,138 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import static com.google.gerrit.server.git.QueueProvider.QueueType.BATCH;
import com.google.common.base.Stopwatch;
import com.google.common.util.concurrent.Futures;
import com.google.common.util.concurrent.ListenableFuture;
import com.google.common.util.concurrent.ListeningExecutorService;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.index.IndexExecutor;
import com.google.gerrit.server.index.SiteIndexer;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.SchemaFactory;
import com.google.inject.Inject;
import com.google.inject.Singleton;
import org.eclipse.jgit.lib.ProgressMonitor;
import org.eclipse.jgit.lib.TextProgressMonitor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
@Singleton
public class AllGroupsIndexer
extends SiteIndexer<AccountGroup.UUID, AccountGroup, GroupIndex> {
private static final Logger log =
LoggerFactory.getLogger(AllGroupsIndexer.class);
private final SchemaFactory<ReviewDb> schemaFactory;
private final ListeningExecutorService executor;
private final GroupCache groupCache;
@Inject
AllGroupsIndexer(
SchemaFactory<ReviewDb> schemaFactory,
@IndexExecutor(BATCH) ListeningExecutorService executor,
GroupCache groupCache) {
this.schemaFactory = schemaFactory;
this.executor = executor;
this.groupCache = groupCache;
}
@Override
public SiteIndexer.Result indexAll(GroupIndex index) {
ProgressMonitor progress =
new TextProgressMonitor(new PrintWriter(progressOut));
progress.start(2);
Stopwatch sw = Stopwatch.createStarted();
List<AccountGroup.UUID> uuids;
try {
uuids = collectGroups(progress);
} catch (OrmException e) {
log.error("Error collecting groups", e);
return new SiteIndexer.Result(sw, false, 0, 0);
}
return reindexGroups(index, uuids, progress);
}
private SiteIndexer.Result reindexGroups(GroupIndex index,
List<AccountGroup.UUID> uuids, ProgressMonitor progress) {
progress.beginTask("Reindexing groups", uuids.size());
List<ListenableFuture<?>> futures = new ArrayList<>(uuids.size());
AtomicBoolean ok = new AtomicBoolean(true);
final AtomicInteger done = new AtomicInteger();
final AtomicInteger failed = new AtomicInteger();
Stopwatch sw = Stopwatch.createStarted();
for (final AccountGroup.UUID uuid : uuids) {
final String desc = "group " + uuid;
ListenableFuture<?> future = executor.submit(
new Callable<Void>() {
@Override
public Void call() throws Exception {
try {
AccountGroup oldGroup = groupCache.get(uuid);
if (oldGroup != null) {
groupCache.evict(oldGroup);
}
index.replace(groupCache.get(uuid));
verboseWriter.println("Reindexed " + desc);
done.incrementAndGet();
} catch (Exception e) {
failed.incrementAndGet();
throw e;
}
return null;
}
});
addErrorListener(future, desc, progress, ok);
futures.add(future);
}
try {
Futures.successfulAsList(futures).get();
} catch (ExecutionException | InterruptedException e) {
log.error("Error waiting on group futures", e);
return new SiteIndexer.Result(sw, false, 0, 0);
}
progress.endTask();
return new SiteIndexer.Result(sw, ok.get(), done.get(), failed.get());
}
private List<AccountGroup.UUID> collectGroups(ProgressMonitor progress)
throws OrmException {
progress.beginTask("Collecting groups", ProgressMonitor.UNKNOWN);
List<AccountGroup.UUID> uuids = new ArrayList<>();
try (ReviewDb db = schemaFactory.open()) {
for (AccountGroup group : db.accountGroups().all()) {
uuids.add(group.getGroupUUID());
}
}
progress.endTask();
return uuids;
}
}

95
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupField.java Normal file
View File

@ -0,0 +1,95 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.FieldDef;
import com.google.gerrit.server.index.FieldType;
import com.google.gerrit.server.index.SchemaUtil;
import com.google.gwtorm.server.OrmException;
/** Secondary index schemas for groups. */
public class GroupField {
/** Legacy group ID. */
public static final FieldDef<AccountGroup, Integer> ID =
new FieldDef.Single<AccountGroup, Integer>(
"id", FieldType.INTEGER, false) {
@Override
public Integer get(AccountGroup input, FillArgs args) {
return input.getId().get();
}
};
/** Group UUID. */
public static final FieldDef<AccountGroup, String> UUID =
new FieldDef.Single<AccountGroup, String>(
"uuid", FieldType.EXACT, true) {
@Override
public String get(AccountGroup input, FillArgs args) {
return input.getGroupUUID().get();
}
};
/** Group owner UUID. */
public static final FieldDef<AccountGroup, String> OWNER_UUID =
new FieldDef.Single<AccountGroup, String>(
"owner_uuid", FieldType.EXACT, false) {
@Override
public String get(AccountGroup input, FillArgs args) {
return input.getOwnerGroupUUID().get();
}
};
/** Group name. */
public static final FieldDef<AccountGroup, String> NAME =
new FieldDef.Single<AccountGroup, String>(
"name", FieldType.EXACT, false) {
@Override
public String get(AccountGroup input, FillArgs args) {
return input.getName();
}
};
/** Fuzzy prefix match on group name parts. */
public static final FieldDef<AccountGroup, Iterable<String>> NAME_PART =
new FieldDef.Repeatable<AccountGroup, String>(
"name_part", FieldType.PREFIX, false) {
@Override
public Iterable<String> get(AccountGroup input, FillArgs args) {
return SchemaUtil.getNameParts(input.getName());
}
};
/** Group description. */
public static final FieldDef<AccountGroup, String> DESCRIPTION =
new FieldDef.Single<AccountGroup, String>(
"description", FieldType.FULL_TEXT, false) {
@Override
public String get(AccountGroup input, FillArgs args) {
return input.getDescription();
}
};
/** Whether the group is visible to all users. */
public static final FieldDef<AccountGroup, String> IS_VISIBLE_TO_ALL =
new FieldDef.Single<AccountGroup, String>(
"is_visible_to_all", FieldType.EXACT, false) {
@Override
public String get(AccountGroup input, FillArgs args)
throws OrmException {
return input.isVisibleToAll() ? "1" : "0";
}
};
}

32
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupIndex.java Normal file
View File

@ -0,0 +1,32 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.Index;
import com.google.gerrit.server.index.IndexDefinition;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.group.GroupPredicates;
public interface GroupIndex extends Index<AccountGroup.UUID, AccountGroup> {
public interface Factory extends
IndexDefinition.IndexFactory<AccountGroup.UUID, AccountGroup, GroupIndex> {
}
@Override
default Predicate<AccountGroup> keyPredicate(AccountGroup.UUID uuid) {
return GroupPredicates.uuid(uuid);
}
}

30
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupIndexCollection.java Normal file
View File

@ -0,0 +1,30 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import com.google.common.annotations.VisibleForTesting;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.IndexCollection;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@Singleton
public class GroupIndexCollection
extends IndexCollection<AccountGroup.UUID, AccountGroup, GroupIndex> {
@Inject
@VisibleForTesting
public GroupIndexCollection() {
}
}

33
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupIndexDefinition.java Normal file
View File

@ -0,0 +1,33 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License
package com.google.gerrit.server.index.group;
import com.google.gerrit.common.Nullable;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.IndexDefinition;
import com.google.inject.Inject;
import com.google.inject.util.Providers;
public class GroupIndexDefinition
extends IndexDefinition<AccountGroup.UUID, AccountGroup, GroupIndex> {
@Inject
GroupIndexDefinition(GroupIndexCollection indexCollection,
GroupIndex.Factory indexFactory,
@Nullable AllGroupsIndexer allGroupsIndexer) {
super(GroupSchemaDefinitions.INSTANCE, indexCollection, indexFactory,
Providers.of(allGroupsIndexer));
}
}

43
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupIndexRewriter.java Normal file
View File

@ -0,0 +1,43 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import static com.google.common.base.Preconditions.checkNotNull;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.IndexRewriter;
import com.google.gerrit.server.index.QueryOptions;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.QueryParseException;
import com.google.inject.Inject;
import com.google.inject.Singleton;
@Singleton
public class GroupIndexRewriter implements IndexRewriter<AccountGroup> {
private final GroupIndexCollection indexes;
@Inject
GroupIndexRewriter(GroupIndexCollection indexes) {
this.indexes = indexes;
}
@Override
public Predicate<AccountGroup> rewrite(Predicate<AccountGroup> in,
QueryOptions opts) throws QueryParseException {
GroupIndex index = indexes.getSearchIndex();
checkNotNull(index, "no active search index configured for groups");
return new IndexedGroupQuery(index, in, opts);
}
}

29
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupIndexer.java Normal file
View File

@ -0,0 +1,29 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import com.google.gerrit.reviewdb.client.AccountGroup;
import java.io.IOException;
public interface GroupIndexer {
/**
* Synchronously index a group.
*
* @param uuid group UUID to index.
*/
void index(AccountGroup.UUID uuid) throws IOException;
}

71
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupIndexerImpl.java Normal file
View File

@ -0,0 +1,71 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import com.google.common.collect.ImmutableSet;
import com.google.gerrit.common.Nullable;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.index.Index;
import com.google.inject.assistedinject.Assisted;
import com.google.inject.assistedinject.AssistedInject;
import java.io.IOException;
import java.util.Collection;
import java.util.Collections;
public class GroupIndexerImpl implements GroupIndexer {
public interface Factory {
GroupIndexerImpl create(GroupIndexCollection indexes);
GroupIndexerImpl create(@Nullable GroupIndex index);
}
private final GroupCache groupCache;
private final GroupIndexCollection indexes;
private final GroupIndex index;
@AssistedInject
GroupIndexerImpl(GroupCache groupCache,
@Assisted GroupIndexCollection indexes) {
this.groupCache = groupCache;
this.indexes = indexes;
this.index = null;
}
@AssistedInject
GroupIndexerImpl(GroupCache groupCache,
@Assisted GroupIndex index) {
this.groupCache = groupCache;
this.indexes = null;
this.index = index;
}
@Override
public void index(AccountGroup.UUID uuid) throws IOException {
for (Index<?, AccountGroup> i : getWriteIndexes()) {
i.replace(groupCache.get(uuid));
}
}
private Collection<GroupIndex> getWriteIndexes() {
if (indexes != null) {
return indexes.getWriteIndexes();
}
return index != null
? Collections.singleton(index)
: ImmutableSet.of();
}
}

39
gerrit-server/src/main/java/com/google/gerrit/server/index/group/GroupSchemaDefinitions.java Normal file
View File

@ -0,0 +1,39 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import static com.google.gerrit.server.index.SchemaUtil.schema;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.Schema;
import com.google.gerrit.server.index.SchemaDefinitions;
public class GroupSchemaDefinitions extends SchemaDefinitions<AccountGroup> {
static final Schema<AccountGroup> V1 = schema(
GroupField.ID,
GroupField.UUID,
GroupField.OWNER_UUID,
GroupField.NAME,
GroupField.NAME_PART,
GroupField.DESCRIPTION,
GroupField.IS_VISIBLE_TO_ALL);
public static final GroupSchemaDefinitions INSTANCE =
new GroupSchemaDefinitions();
private GroupSchemaDefinitions() {
super("groups", AccountGroup.class);
}
}

34
gerrit-server/src/main/java/com/google/gerrit/server/index/group/IndexedGroupQuery.java Normal file
View File

@ -0,0 +1,34 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.index.group;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.Index;
import com.google.gerrit.server.index.IndexedQuery;
import com.google.gerrit.server.index.QueryOptions;
import com.google.gerrit.server.query.DataSource;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.QueryParseException;
public class IndexedGroupQuery
extends IndexedQuery<AccountGroup.UUID, AccountGroup>
implements DataSource<AccountGroup> {
public IndexedGroupQuery(Index<AccountGroup.UUID, AccountGroup> index,
Predicate<AccountGroup> pred, QueryOptions opts)
throws QueryParseException {
super(index, pred, opts.convertForBackend());
}
}

14
gerrit-server/src/main/java/com/google/gerrit/server/query/IsVisibleToPredicate.java
View File

@ -14,9 +14,23 @@
package com.google.gerrit.server.query;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.query.change.SingleGroupUser;
public abstract class IsVisibleToPredicate<T> extends OperatorPredicate<T>
implements Matchable<T> {
public IsVisibleToPredicate(String name, String value) {
super(name, value);
}
protected static String describe(CurrentUser user) {
if (user.isIdentifiedUser()) {
return user.getAccountId().toString();
}
if (user instanceof SingleGroupUser) {
return "group:" + user.getEffectiveGroups()
.getKnownGroups().iterator().next().toString();
}
return user.toString();
}
}

13
gerrit-server/src/main/java/com/google/gerrit/server/query/account/AccountIsVisibleToPredicate.java
View File

@ -14,26 +14,13 @@
package com.google.gerrit.server.query.account;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.account.AccountControl;
import com.google.gerrit.server.account.AccountState;
import com.google.gerrit.server.query.IsVisibleToPredicate;
import com.google.gerrit.server.query.change.SingleGroupUser;
import com.google.gwtorm.server.OrmException;
public class AccountIsVisibleToPredicate
extends IsVisibleToPredicate<AccountState> {
private static String describe(CurrentUser user) {
if (user.isIdentifiedUser()) {
return user.getAccountId().toString();
}
if (user instanceof SingleGroupUser) {
return "group:" + user.getEffectiveGroups().getKnownGroups() //
.iterator().next().toString();
}
return user.toString();
}
private final AccountControl accountControl;
AccountIsVisibleToPredicate(AccountControl accountControl) {

11
gerrit-server/src/main/java/com/google/gerrit/server/query/change/ChangeIsVisibleToPredicate.java
View File

@ -25,17 +25,6 @@ import com.google.gwtorm.server.OrmException;
import com.google.inject.Provider;
class ChangeIsVisibleToPredicate extends IsVisibleToPredicate<ChangeData> {
private static String describe(CurrentUser user) {
if (user.isIdentifiedUser()) {
return user.getAccountId().toString();
}
if (user instanceof SingleGroupUser) {
return "group:" + user.getEffectiveGroups().getKnownGroups() //
.iterator().next().toString();
}
return user.toString();
}
private final Provider<ReviewDb> db;
private final ChangeNotes.Factory notesFactory;
private final ChangeControl.GenericFactory changeControl;

52
gerrit-server/src/main/java/com/google/gerrit/server/query/group/GroupIsVisibleToPredicate.java Normal file
View File

@ -0,0 +1,52 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.query.group;
import com.google.gerrit.common.errors.NoSuchGroupException;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.account.GroupControl;
import com.google.gerrit.server.query.IsVisibleToPredicate;
import com.google.gerrit.server.query.account.AccountQueryBuilder;
import com.google.gwtorm.server.OrmException;
public class GroupIsVisibleToPredicate
extends IsVisibleToPredicate<AccountGroup> {
private final GroupControl.GenericFactory groupControlFactory;
private final CurrentUser user;
GroupIsVisibleToPredicate(GroupControl.GenericFactory groupControlFactory,
CurrentUser user) {
super(AccountQueryBuilder.FIELD_VISIBLETO, describe(user));
this.groupControlFactory = groupControlFactory;
this.user = user;
}
@Override
public boolean match(AccountGroup group) throws OrmException {
try {
return groupControlFactory.controlFor(user, group.getGroupUUID())
.isVisible();
} catch (NoSuchGroupException e) {
// Ignored
return false;
}
}
@Override
public int getCost() {
return 1;
}
}

83
gerrit-server/src/main/java/com/google/gerrit/server/query/group/GroupPredicates.java Normal file
View File

@ -0,0 +1,83 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.query.group;
import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.index.FieldDef;
import com.google.gerrit.server.index.IndexPredicate;
import com.google.gerrit.server.index.group.GroupField;
import com.google.gerrit.server.query.Predicate;
import java.util.List;
import java.util.Locale;
public class GroupPredicates {
public static Predicate<AccountGroup> defaultPredicate(String query) {
// Adapt the capacity of this list when adding more default predicates.
List<Predicate<AccountGroup>> preds = Lists.newArrayListWithCapacity(5);
preds.add(uuid(new AccountGroup.UUID(query)));
preds.add(name(query));
preds.add(inname(query));
if (!Strings.isNullOrEmpty(query)) {
preds.add(description(query));
}
preds.add(owner(query));
return Predicate.or(preds);
}
public static Predicate<AccountGroup> uuid(AccountGroup.UUID uuid) {
return new GroupPredicate(GroupField.UUID,
GroupQueryBuilder.FIELD_UUID, uuid.get());
}
public static Predicate<AccountGroup> description(String description) {
return new GroupPredicate(GroupField.DESCRIPTION,
GroupQueryBuilder.FIELD_DESCRIPTION, description);
}
public static Predicate<AccountGroup> inname(String name) {
return new GroupPredicate(GroupField.NAME_PART,
GroupQueryBuilder.FIELD_INNAME, name.toLowerCase(Locale.US));
}
public static Predicate<AccountGroup> name(String name) {
return new GroupPredicate(GroupField.NAME,
GroupQueryBuilder.FIELD_NAME, name.toLowerCase(Locale.US));
}
public static Predicate<AccountGroup> owner(String owner) {
return new GroupPredicate(GroupField.OWNER_UUID,
GroupQueryBuilder.FIELD_OWNER, owner);
}
public static Predicate<AccountGroup> isVisibleToAll() {
return new GroupPredicate(GroupField.IS_VISIBLE_TO_ALL, "1");
}
static class GroupPredicate extends IndexPredicate<AccountGroup> {
GroupPredicate(FieldDef<AccountGroup, ?> def, String value) {
super(def, value);
}
GroupPredicate(FieldDef<AccountGroup, ?> def, String name, String value) {
super(def, name, value);
}
}
private GroupPredicates() {
}
}

101
gerrit-server/src/main/java/com/google/gerrit/server/query/group/GroupQueryBuilder.java Normal file
View File

@ -0,0 +1,101 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.query.group;
import com.google.common.base.Strings;
import com.google.common.primitives.Ints;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.query.LimitPredicate;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.QueryBuilder;
import com.google.gerrit.server.query.QueryParseException;
import com.google.inject.Inject;
/**
* Parses a query string meant to be applied to group objects.
*/
public class GroupQueryBuilder extends QueryBuilder<AccountGroup> {
public static final String FIELD_UUID = "uuid";
public static final String FIELD_DESCRIPTION = "description";
public static final String FIELD_INNAME = "inname";
public static final String FIELD_NAME = "name";
public static final String FIELD_OWNER = "owner";
public static final String FIELD_LIMIT = "limit";
private static final QueryBuilder.Definition<AccountGroup, GroupQueryBuilder> mydef =
new QueryBuilder.Definition<>(GroupQueryBuilder.class);
@Inject
GroupQueryBuilder() {
super(mydef);
}
@Operator
public Predicate<AccountGroup> uuid(String uuid) {
return GroupPredicates.uuid(new AccountGroup.UUID(uuid));
}
@Operator
public Predicate<AccountGroup> description(String description)
throws QueryParseException {
if (Strings.isNullOrEmpty(description)) {
throw error("description operator requires a value");
}
return GroupPredicates.description(description);
}
@Operator
public Predicate<AccountGroup> inname(String namePart) {
if (namePart.isEmpty()) {
return name(namePart);
}
return GroupPredicates.inname(namePart);
}
@Operator
public Predicate<AccountGroup> name(String name) {
return GroupPredicates.name(name);
}
@Operator
public Predicate<AccountGroup> owner(String owner) {
return GroupPredicates.owner(owner);
}
@Operator
public Predicate<AccountGroup> is(String value) throws QueryParseException {
if ("visibleToAll".equalsIgnoreCase(value)) {
return GroupPredicates.isVisibleToAll();
}
throw error("Invalid query");
}
@Override
protected Predicate<AccountGroup> defaultField(String query) {
return GroupPredicates.defaultPredicate(query);
}
@Operator
public Predicate<AccountGroup> limit(String query)
throws QueryParseException {
Integer limit = Ints.tryParse(query);
if (limit == null) {
throw error("Invalid limit: " + query);
}
return new LimitPredicate<>(FIELD_LIMIT, limit);
}
}

63
gerrit-server/src/main/java/com/google/gerrit/server/query/group/GroupQueryProcessor.java Normal file
View File

@ -0,0 +1,63 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.query.group;
import static com.google.common.base.Preconditions.checkState;
import static com.google.gerrit.server.query.group.GroupQueryBuilder.FIELD_LIMIT;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.account.GroupControl;
import com.google.gerrit.server.index.IndexConfig;
import com.google.gerrit.server.index.IndexPredicate;
import com.google.gerrit.server.index.group.GroupIndexCollection;
import com.google.gerrit.server.index.group.GroupIndexRewriter;
import com.google.gerrit.server.index.group.GroupSchemaDefinitions;
import com.google.gerrit.server.query.AndSource;
import com.google.gerrit.server.query.Predicate;
import com.google.gerrit.server.query.QueryProcessor;
import com.google.inject.Inject;
import com.google.inject.Provider;
public class GroupQueryProcessor extends QueryProcessor<AccountGroup> {
private final GroupControl.GenericFactory groupControlFactory;
static {
// It is assumed that basic rewrites do not touch visibleto predicates.
checkState(
!GroupIsVisibleToPredicate.class.isAssignableFrom(IndexPredicate.class),
"GroupQueryProcessor assumes visibleto is not used by the index rewriter.");
}
@Inject
protected GroupQueryProcessor(Provider<CurrentUser> userProvider,
Metrics metrics,
IndexConfig indexConfig,
GroupIndexCollection indexes,
GroupIndexRewriter rewriter,
GroupControl.GenericFactory groupControlFactory) {
super(userProvider, metrics, GroupSchemaDefinitions.INSTANCE, indexConfig,
indexes, rewriter, FIELD_LIMIT);
this.groupControlFactory = groupControlFactory;
}
@Override
protected Predicate<AccountGroup> enforceVisibility(
Predicate<AccountGroup> pred) {
return new AndSource<>(pred,
new GroupIsVisibleToPredicate(groupControlFactory, userProvider.get()),
start);
}
}

55
gerrit-server/src/main/java/com/google/gerrit/server/schema/SchemaCreator.java
View File

@ -24,6 +24,7 @@ import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.account.GroupUUID;
import com.google.gerrit.server.config.SitePath;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.index.group.GroupIndexCollection;
import com.google.gwtorm.jdbc.JdbcExecutor;
import com.google.gwtorm.jdbc.JdbcSchema;
import com.google.gwtorm.server.OrmException;
@ -46,6 +47,7 @@ public class SchemaCreator {
private final AllUsersCreator allUsersCreator;
private final PersonIdent serverUser;
private final DataSourceType dataSourceType;
private final GroupIndexCollection indexCollection;
private AccountGroup admin;
private AccountGroup batch;
@ -55,20 +57,23 @@ public class SchemaCreator {
AllProjectsCreator ap,
AllUsersCreator auc,
@GerritPersonIdent PersonIdent au,
DataSourceType dst) {
this(site.site_path, ap, auc, au, dst);
DataSourceType dst,
GroupIndexCollection ic) {
this(site.site_path, ap, auc, au, dst, ic);
}
public SchemaCreator(@SitePath Path site,
AllProjectsCreator ap,
AllUsersCreator auc,
@GerritPersonIdent PersonIdent au,
DataSourceType dst) {
DataSourceType dst,
GroupIndexCollection ic) {
site_path = site;
allProjectsCreator = ap;
allUsersCreator = auc;
serverUser = au;
dataSourceType = dst;
indexCollection = ic;
}
public void create(final ReviewDb db) throws OrmException, IOException,
@ -82,6 +87,7 @@ public class SchemaCreator {
sVer.versionNbr = SchemaVersion.getBinaryVersion();
db.schemaVersion().insert(Collections.singleton(sVer));
createDefaultGroups(db);
initSystemConfig(db);
allProjectsCreator
.setAdministrators(GroupReference.forGroup(admin))
@ -93,6 +99,30 @@ public class SchemaCreator {
dataSourceType.getIndexScript().run(db);
}
private void createDefaultGroups(ReviewDb db)
throws OrmException, IOException {
admin = newGroup(db, "Administrators", null);
admin.setDescription("Gerrit Site Administrators");
db.accountGroups().insert(Collections.singleton(admin));
db.accountGroupNames()
.insert(Collections.singleton(new AccountGroupName(admin)));
index(admin);
batch = newGroup(db, "Non-Interactive Users", null);
batch.setDescription("Users who perform batch actions on Gerrit");
batch.setOwnerGroupUUID(admin.getGroupUUID());
db.accountGroups().insert(Collections.singleton(batch));
db.accountGroupNames()
.insert(Collections.singleton(new AccountGroupName(batch)));
index(batch);
}
private void index(AccountGroup group) throws IOException {
if (indexCollection.getSearchIndex() != null) {
indexCollection.getSearchIndex().replace(group);
}
}
private AccountGroup newGroup(ReviewDb c, String name, AccountGroup.UUID uuid)
throws OrmException {
if (uuid == null) {
@ -104,27 +134,14 @@ public class SchemaCreator {
uuid);
}
private SystemConfig initSystemConfig(final ReviewDb c) throws OrmException {
admin = newGroup(c, "Administrators", null);
admin.setDescription("Gerrit Site Administrators");
c.accountGroups().insert(Collections.singleton(admin));
c.accountGroupNames().insert(
Collections.singleton(new AccountGroupName(admin)));
batch = newGroup(c, "Non-Interactive Users", null);
batch.setDescription("Users who perform batch actions on Gerrit");
batch.setOwnerGroupUUID(admin.getGroupUUID());
c.accountGroups().insert(Collections.singleton(batch));
c.accountGroupNames().insert(
Collections.singleton(new AccountGroupName(batch)));
final SystemConfig s = SystemConfig.create();
private SystemConfig initSystemConfig(ReviewDb db) throws OrmException {
SystemConfig s = SystemConfig.create();
try {
s.sitePath = site_path.toRealPath().normalize().toString();
} catch (IOException e) {
s.sitePath = site_path.toAbsolutePath().normalize().toString();
}
c.systemConfig().insert(Collections.singleton(s));
db.systemConfig().insert(Collections.singleton(s));
return s;
}
}

453
gerrit-server/src/test/java/com/google/gerrit/server/query/group/AbstractQueryGroupsTest.java Normal file
View File

@ -0,0 +1,453 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.query.group;
import static com.google.common.truth.Truth.assertThat;
import static java.util.stream.Collectors.toList;
import com.google.common.collect.ImmutableList;
import com.google.gerrit.extensions.api.GerritApi;
import com.google.gerrit.extensions.api.groups.GroupInput;
import com.google.gerrit.extensions.api.groups.Groups.QueryRequest;
import com.google.gerrit.extensions.common.AccountInfo;
import com.google.gerrit.extensions.common.GroupInfo;
import com.google.gerrit.extensions.restapi.BadRequestException;
import com.google.gerrit.lifecycle.LifecycleManager;
import com.google.gerrit.reviewdb.client.Account;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.AnonymousUser;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.account.AccountCache;
import com.google.gerrit.server.account.AccountManager;
import com.google.gerrit.server.account.AuthRequest;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.config.AllProjectsName;
import com.google.gerrit.server.query.account.InternalAccountQuery;
import com.google.gerrit.server.schema.SchemaCreator;
import com.google.gerrit.server.util.ManualRequestContext;
import com.google.gerrit.server.util.OneOffRequestContext;
import com.google.gerrit.server.util.RequestContext;
import com.google.gerrit.server.util.ThreadLocalRequestContext;
import com.google.gerrit.testutil.ConfigSuite;
import com.google.gerrit.testutil.GerritServerTests;
import com.google.gerrit.testutil.InMemoryDatabase;
import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Provider;
import com.google.inject.util.Providers;
import org.eclipse.jgit.lib.Config;
import org.junit.After;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
import java.util.Arrays;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Locale;
@Ignore
public abstract class AbstractQueryGroupsTest extends GerritServerTests {
@ConfigSuite.Default
public static Config defaultConfig() {
Config cfg = new Config();
cfg.setInt("index", null, "maxPages", 10);
return cfg;
}
@Rule
public final TestName testName = new TestName();
@Inject
protected AccountCache accountCache;
@Inject
protected AccountManager accountManager;
@Inject
protected GerritApi gApi;
@Inject
protected IdentifiedUser.GenericFactory userFactory;
@Inject
private Provider<AnonymousUser> anonymousUser;
@Inject
protected InMemoryDatabase schemaFactory;
@Inject
protected SchemaCreator schemaCreator;
@Inject
protected ThreadLocalRequestContext requestContext;
@Inject
protected OneOffRequestContext oneOffRequestContext;
@Inject
protected InternalAccountQuery internalAccountQuery;
@Inject
protected AllProjectsName allProjects;
@Inject
protected GroupCache groupCache;
protected LifecycleManager lifecycle;
protected ReviewDb db;
protected AccountInfo currentUserInfo;
protected CurrentUser user;
protected abstract Injector createInjector();
@Before
public void setUpInjector() throws Exception {
lifecycle = new LifecycleManager();
Injector injector = createInjector();
lifecycle.add(injector);
injector.injectMembers(this);
lifecycle.start();
db = schemaFactory.open();
schemaCreator.create(db);
Account.Id userId = createAccount("user", "User", "user@example.com", true);
user = userFactory.create(userId);
requestContext.setContext(newRequestContext(userId));
currentUserInfo = gApi.accounts().id(userId.get()).get();
}
protected RequestContext newRequestContext(Account.Id requestUserId) {
final CurrentUser requestUser =
userFactory.create(requestUserId);
return new RequestContext() {
@Override
public CurrentUser getUser() {
return requestUser;
}
@Override
public Provider<ReviewDb> getReviewDbProvider() {
return Providers.of(db);
}
};
}
protected void setAnonymous() {
requestContext.setContext(new RequestContext() {
@Override
public CurrentUser getUser() {
return anonymousUser.get();
}
@Override
public Provider<ReviewDb> getReviewDbProvider() {
return Providers.of(db);
}
});
}
@After
public void tearDownInjector() {
if (lifecycle != null) {
lifecycle.stop();
}
requestContext.setContext(null);
if (db != null) {
db.close();
}
InMemoryDatabase.drop(schemaFactory);
}
@Test
public void byUuid() throws Exception {
assertQuery("uuid:6d70856bc40ded50f2585c4c0f7e179f3544a272");
assertQuery("uuid:non-existing");
GroupInfo group = createGroup(name("group"));
assertQuery("uuid:" + group.id, group);
GroupInfo admins = gApi.groups().id("Administrators").get();
assertQuery("uuid:" + admins.id, admins);
}
@Test
public void byName() throws Exception {
assertQuery("name:non-existing");
GroupInfo group = createGroup(name("group"));
assertQuery("name:" + group.name, group);
assertQuery("name:" + group.name.toUpperCase(Locale.US), group);
// only exact match
GroupInfo groupWithHyphen = createGroup(name("group-with-hyphen"));
createGroup(name("group-no-match-with-hyphen"));
assertQuery("name:" + groupWithHyphen.name, groupWithHyphen);
}
@Test
public void byInname() throws Exception {
String namePart = testName.getMethodName();
GroupInfo group1 = createGroup("group-" + namePart);
GroupInfo group2 = createGroup("group-" + namePart + "-2");
GroupInfo group3 = createGroup("group-" + namePart + "3");
assertQuery("inname:" + namePart, group1, group2, group3);
assertQuery("inname:" + namePart.toUpperCase(Locale.US), group1, group2,
group3);
assertQuery("inname:" + namePart.toLowerCase(Locale.US), group1, group2,
group3);
}
@Test
public void byDescription() throws Exception {
GroupInfo group1 =
createGroupWithDescription(name("group1"), "This is a test group.");
GroupInfo group2 =
createGroupWithDescription(name("group2"), "ANOTHER TEST GROUP.");
createGroupWithDescription(name("group3"), "Maintainers of project foo.");
assertQuery("description:test", group1, group2);
assertQuery("description:non-existing");
exception.expect(BadRequestException.class);
exception.expectMessage("description operator requires a value");
assertQuery("description:\"\"");
}
@Test
public void byOwner() throws Exception {
assertQuery("owner:non-existing");
GroupInfo ownerGroup = createGroup(name("owner-group"));
GroupInfo group = createGroupWithOwner(name("group"), ownerGroup);
createGroup(name("group2"));
// ownerGroup owns itself
assertQuery("owner:" + ownerGroup.id, group, ownerGroup);
}
@Test
public void byIsVisibleToAll() throws Exception {
assertQuery("is:visibleToAll");
GroupInfo groupThatIsVisibleToAll =
createGroupThatIsVisibleToAll(name("group-that-is-visible-to-all"));
createGroup(name("group"));
assertQuery("is:visibleToAll", groupThatIsVisibleToAll);
}
@Test
public void byDefaultField() throws Exception {
GroupInfo group1 = createGroup(name("foo-group"));
GroupInfo group2 = createGroup(name("group2"));
GroupInfo group3 = createGroupWithDescription(name("group3"),
"decription that contains foo and the UUID of group2: " + group2.id);
assertQuery("non-existing");
assertQuery("foo", group1, group3);
assertQuery(group2.id, group2, group3);
}
@Test
public void withLimit() throws Exception {
GroupInfo group1 = createGroup(name("group1"));
GroupInfo group2 = createGroup(name("group2"));
GroupInfo group3 = createGroup(name("group3"));
String query =
"uuid:" + group1.id + " OR uuid:" + group2.id + " OR uuid:" + group3.id;
List<GroupInfo> result = assertQuery(query, group1, group2, group3);
assertThat(result.get(result.size() - 1)._moreGroups).isNull();
result = assertQuery(newQuery(query).withLimit(2), result.subList(0, 2));
assertThat(result.get(result.size() - 1)._moreGroups).isTrue();
}
@Test
public void withStart() throws Exception {
GroupInfo group1 = createGroup(name("group1"));
GroupInfo group2 = createGroup(name("group2"));
GroupInfo group3 = createGroup(name("group3"));
String query =
"uuid:" + group1.id + " OR uuid:" + group2.id + " OR uuid:" + group3.id;
List<GroupInfo> result = assertQuery(query, group1, group2, group3);
assertQuery(newQuery(query).withStart(1), result.subList(1, 3));
}
@Test
public void asAnonymous() throws Exception {
GroupInfo group = createGroup(name("group"));
setAnonymous();
assertQuery("uuid:" + group.id);
}
// reindex permissions are tested by {@link GroupsIT#reindexPermissions}
@Test
public void reindex() throws Exception {
GroupInfo group1 = createGroupWithDescription(name("group"), "barX");
// update group in the database so that group index is stale
String newDescription = "barY";
AccountGroup group =
db.accountGroups().get(new AccountGroup.Id(group1.groupId));
group.setDescription(newDescription);
db.accountGroups().update(Collections.singleton(group));
assertQuery("description:" + group1.description, group1);
assertQuery("description:" + newDescription);
gApi.groups().id(group1.id).index();
assertQuery("description:" + group1.description);
assertQuery("description:" + newDescription, group1);
}
private Account.Id createAccount(String username, String fullName,
String email, boolean active) throws Exception {
try (ManualRequestContext ctx = oneOffRequestContext.open()) {
Account.Id id =
accountManager.authenticate(AuthRequest.forUser(username)).getAccountId();
if (email != null) {
accountManager.link(id, AuthRequest.forEmail(email));
}
Account a = db.accounts().get(id);
a.setFullName(fullName);
a.setPreferredEmail(email);
a.setActive(active);
db.accounts().update(ImmutableList.of(a));
accountCache.evict(id);
return id;
}
}
protected GroupInfo createGroup(String name, AccountInfo... members)
throws Exception {
return createGroupWithDescription(name, null, members);
}
protected GroupInfo createGroupWithDescription(String name,
String description, AccountInfo... members) throws Exception {
GroupInput in = new GroupInput();
in.name = name;
in.description = description;
in.members = Arrays.asList(members).stream()
.map(a -> String.valueOf(a._accountId)).collect(toList());
return gApi.groups().create(in).get();
}
protected GroupInfo createGroupWithOwner(String name, GroupInfo ownerGroup)
throws Exception {
GroupInput in = new GroupInput();
in.name = name;
in.ownerId = ownerGroup.id;
return gApi.groups().create(in).get();
}
protected GroupInfo createGroupThatIsVisibleToAll(String name)
throws Exception {
GroupInput in = new GroupInput();
in.name = name;
in.visibleToAll = true;
return gApi.groups().create(in).get();
}
protected GroupInfo getGroup(AccountGroup.UUID uuid) throws Exception {
return gApi.groups().id(uuid.get()).get();
}
protected List<GroupInfo> assertQuery(Object query, GroupInfo... groups)
throws Exception {
return assertQuery(newQuery(query), groups);
}
protected List<GroupInfo> assertQuery(QueryRequest query,
GroupInfo... groups) throws Exception {
return assertQuery(query, Arrays.asList(groups));
}
protected List<GroupInfo> assertQuery(QueryRequest query,
List<GroupInfo> groups) throws Exception {
List<GroupInfo> result = query.get();
Iterable<String> uuids = uuids(result);
assertThat(uuids).named(format(query, result, groups))
.containsExactlyElementsIn(uuids(groups));
return result;
}
protected QueryRequest newQuery(Object query) {
return gApi.groups().query(query.toString());
}
protected String format(QueryRequest query, List<GroupInfo> actualGroups,
List<GroupInfo> expectedGroups) {
StringBuilder b = new StringBuilder();
b.append("query '").append(query.getQuery())
.append("' with expected groups ");
b.append(format(expectedGroups));
b.append(" and result ");
b.append(format(actualGroups));
return b.toString();
}
protected String format(Iterable<GroupInfo> groups) {
StringBuilder b = new StringBuilder();
b.append("[");
Iterator<GroupInfo> it = groups.iterator();
while (it.hasNext()) {
GroupInfo g = it.next();
b.append("{").append(g.id).append(", ").append("name=").append(g.name)
.append(", ").append("groupId=").append(g.groupId).append(", ")
.append("url=").append(g.url).append(", ").append("ownerId=")
.append(g.ownerId).append(", ").append("owner=").append(g.owner)
.append(", ").append("description=").append(g.description)
.append(", ").append("visibleToAll=")
.append(toBoolean(g.options.visibleToAll)).append("}");
if (it.hasNext()) {
b.append(", ");
}
}
b.append("]");
return b.toString();
}
protected static boolean toBoolean(Boolean b) {
return b == null ? false : b;
}
protected static Iterable<String> ids(GroupInfo... groups) {
return uuids(Arrays.asList(groups));
}
protected static Iterable<String> uuids(List<GroupInfo> groups) {
return groups.stream().map(g -> g.id).collect(toList());
}
protected String name(String name) {
if (name == null) {
return null;
}
return name + "_" + testName.getMethodName().toLowerCase();
}
}

31
gerrit-server/src/test/java/com/google/gerrit/server/query/group/LuceneQueryGroupsTest.java Normal file
View File

@ -0,0 +1,31 @@
// Copyright (C) 2017 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.query.group;
import com.google.gerrit.testutil.InMemoryModule;
import com.google.inject.Guice;
import com.google.inject.Injector;
import org.eclipse.jgit.lib.Config;
public class LuceneQueryGroupsTest extends AbstractQueryGroupsTest {
@Override
protected Injector createInjector() {
Config luceneConfig = new Config(config);
InMemoryModule.setDefaults(luceneConfig);
return Guice.createInjector(
new InMemoryModule(luceneConfig, notesMigration));
}
}

19
gerrit-server/src/test/java/com/google/gerrit/testutil/InMemoryDatabase.java
View File

@ -20,12 +20,14 @@ import com.google.gerrit.lifecycle.LifecycleManager;
import com.google.gerrit.reviewdb.client.CurrentSchemaVersion;
import com.google.gerrit.reviewdb.client.SystemConfig;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.index.group.GroupIndexCollection;
import com.google.gerrit.server.schema.SchemaCreator;
import com.google.gerrit.server.schema.SchemaVersion;
import com.google.gwtorm.jdbc.Database;
import com.google.gwtorm.jdbc.SimpleDataSource;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.SchemaFactory;
import com.google.inject.AbstractModule;
import com.google.inject.Guice;
import com.google.inject.Inject;
import com.google.inject.Injector;
@ -77,6 +79,23 @@ public class InMemoryDatabase implements SchemaFactory<ReviewDb> {
private boolean created;
@Inject
InMemoryDatabase(Injector injector) throws OrmException {
// Don't inject SchemaCreator directly.
// SchemaCreator needs to get GroupIndexCollection injected, but
// GroupIndexCollection was not bound yet. Creating a child injector with a
// binding for GroupIndexCollection to create an instance of SchemaCreator
// prevents that Guice creates a just-in-time binding for
// GroupIndexCollection in the root injector. If a binding for
// GroupIndexCollection is created in the root injector then IndexModule
// fails to create this binding later, because it already exists.
this(injector.createChildInjector(new AbstractModule() {
@Override
protected void configure() {
bind(GroupIndexCollection.class);
}
}).getInstance(SchemaCreator.class));
}
InMemoryDatabase(SchemaCreator schemaCreator) throws OrmException {
this.schemaCreator = schemaCreator;

5
gerrit-sshd/src/main/java/com/google/gerrit/sshd/commands/RenameGroupCommand.java
View File

@ -28,6 +28,8 @@ import com.google.inject.Inject;
import org.kohsuke.args4j.Argument;
import java.io.IOException;
@CommandMetaData(name = "rename-group", description = "Rename an account group")
public class RenameGroupCommand extends SshCommand {
@Argument(index = 0, required = true, metaVar = "GROUP", usage = "name of the group to be renamed")
@ -50,7 +52,8 @@ public class RenameGroupCommand extends SshCommand {
PutName.Input input = new PutName.Input();
input.name = newGroupName;
putName.apply(rsrc, input);
} catch (RestApiException | OrmException | NoSuchGroupException e) {
} catch (RestApiException | OrmException | IOException
| NoSuchGroupException e) {
throw die(e);
}
}