1439 Commits

Author SHA1 Message Date
David Pursehouse
8a695c953e Merge "Move the 'isPatchSetLocked' check out from the ChangeControl" 2018-05-02 08:15:06 +00:00
Changcheng Xiao
30325c2b7a Move the 'isPatchSetLocked' check out from the ChangeControl
PermissionBackend is supposed to only perform permission
related checks. The 'isPatchSetLocked' check is a patch
set status check to decide whether a change is able to
be rebased, abandoned, etc. It's not permission related
and thus should be moved out from the ChangeControl.

This commit moves the 'isPatchSetLocked' method to the
PatchSetUtil class which looks like the best destination
of this method.

After this move, each time the caller tries to check
the REBASE/ABANDON/RESTORE/ADD_PATCH_SET permission, it
must verify the patch set of the change is not locked. If
it's locked the request should be rejected directly without
checking any further permissions. This commit makes sure
every existing usage has done this prerequest check in
advance.

Change-Id: Ic5cef11228f152cb3f51eea0a7aee100332f263a
2018-05-02 09:46:25 +02:00
Patrick Hiesel
7f6eb9ab58 Add a config option to gate parallelism in ChangeJson
ChangeJson will need some more work until we can enable parallel
formatting. Until then, a new boolean config option can gate the
experimental feature.

Change-Id: I5cdd05a409194c1fcf98d06de0c98c1be88939ff
2018-05-02 09:23:25 +02:00
Patrick Hiesel
d4d83c50f1 Merge "Remove index field existence check for ChangeField.ASSIGNEE" 2018-05-02 06:34:05 +00:00
David Pursehouse
18821f60af Merge branch 'stable-2.15'
* stable-2.15:
  Fix the online index for changes when upgrading ES gerrit site
  Update git submodules
  Update git submodules
  config-mail: Remove redundant colons from section titles

Change-Id: Ia424576f92edb038d4581d484d90a6378432ea9d
2018-05-02 11:21:18 +09:00
Dave Borowitz
907813a404 SubmoduleOp: Don't cache server identity
Doing so results in reusing the same timestamp for all automatic
submodule updates.

Change-Id: Ieab268c55bbaccb68265e880c6ce050a3aed801f
2018-05-01 07:35:53 -04:00
Dave Borowitz
1294d22343 Merge "Revert "ExternalIdNotes: Prevent new duplicate emails"" 2018-05-01 11:20:39 +00:00
Maxime Guerreiro
34de18130e Improve String operations
- Don't call #toString on strings.
- Don't use + in StringBuilder#append-s methods.

Change-Id: I4e0f99ea417aabb72ecaaab170c66eb434d3ad37
2018-05-01 12:16:59 +09:00
Maxime Guerreiro
505fa92b0c Replace String#concat with +
Change-Id: I563b9c290d39a447ad63abd6aff34a430a222f8e
2018-05-01 03:12:46 +00:00
David Pursehouse
b785df6878 Merge "Collapse catch blocks when possible" 2018-05-01 03:12:23 +00:00
David Pursehouse
eedc16f57e Merge "Replace indexOf calls with String#contains" 2018-05-01 03:12:08 +00:00
David Pursehouse
3adab799c8 Merge branch 'stable-2.15'
* stable-2.15:
  CommandExecutorQueueProvider: Fix singleton binding
  ReviewerinPredicate: Only return users in state REVIEWER
  Remove documentation about cleaning the build cache
  AbstractQueryChangesTest#reviewerin: Expand to cover CC

Change-Id: I8bc29e2635480ab975bc1a474b333f048bce4eae
2018-05-01 10:49:23 +09:00
Dave Borowitz
7ce1988491 TagCache: Make EntryVal class public
Change-Id: Ie6d0df1ce60d55f8904df63562e800d9738e3ae1
2018-04-30 14:22:52 -04:00
Dave Borowitz
a71ae95d35 CacheModule: Expose bindings for (Persistent)CacheDef
These will be useful for some large tests for the googlesource.com
implementation, which wants to manually call the keySerializer/
valueSerializer.

Change-Id: I5037e3edd5dcead49eeb53dd8ae3b6f5a139d7e5
2018-04-30 11:30:04 -04:00
xchangcheng
9813123504 Merge "Revert "Expose instanceName in the subject template"" 2018-04-30 15:07:07 +00:00
xchangcheng
bfd75384c2 Revert "Expose instanceName in the subject template"
This reverts commit eb4d2d8da7bfc29511154fffe19f89f769a6a03c.

Reason for revert:

we observe

com.google.template.soy.tofu.SoyTofuException: In 'print' tag, expression "$instanceAndProjectName" evaluates to undefined.
        at com.google.gerrit.server.mail.template.ChangeSubject(ChangeSubject.soy:9)

in production

Change-Id: I1701b5bcb4720086783536c5c987866bf73e54f4
2018-04-30 14:06:49 +00:00
Edwin Kempin
cf824194bf Revert "ExternalIdNotes: Prevent new duplicate emails"
The check for duplicate emails can be too slow if there are many
external IDs. E.g. this can make the login of users very slow.

This reverts commit ce80b6ca7752420a0f7a0ef8e05ef8d5a5c9add0.

Change-Id: I80c5f60203f97230e4d94c9c89d88323db403604
Signed-off-by: Edwin Kempin <ekempin@google.com>
2018-04-30 08:48:45 +02:00
Dave Borowitz
11f0c64df9 Custom serializers for ChangeKindCache, including protobuf
Change-Id: I7f2e56f78fd346170922c366c57465d6dfb136ef
2018-04-27 12:25:45 -04:00
Dave Borowitz
49c1c16097 Manage persistent cache versioning outside of Java serialization
The flipside of switching to a new serialization format that, unlike
Java serialization, doesn't store the version in the serialized
representation, is that we need to manage versioning on our own.

Define a version field in PersistentCacheDef, and store this in a new
column in the H2 database. This puts slightly more burden on the cache
implementation to manually manage versions, but makes the process of
updating caches safer; the latter happens much more often than the
former, so the extra up-front effort is worth it.

For existing caches using Java serialization, continue using the default
version of 0. This keeps the same invalidation semantics, and we can
bump the version to 1 when we start implementing new serialization
strategies.

Change-Id: I0d9cccea713fe6dcf368d8b95be59465c336bbb3
2018-04-27 12:25:45 -04:00
Dave Borowitz
7ae7fb1cc0 Allow alternative serialization strategies for persistent caches
We want to have finer control over when persistent cache data is
invalidated due to the stored format changing. Depending on the Java
serialVersionUID, as we have done in the past, has two significant
downsides:
 * Adding new fields which can have a reasonable default, or removing
   old obsolete fields, always requires bumping the serialVersionUID and
   thus flushing the cache. With an alternative serialization strategy
   like protobuf, we can continue reading old entries in some cases.
 * Trivial refactorings which do not change the data semantics, such as
   renaming classes or fields, also flush the cache.

Both of these types of cache flushes have happened to PatchListCache in
the past year. Other serialization strategies such as protocol buffers
allows us to avoid unnecessary cache invalidations.

This change introduces a new interface CacheSerializer, which is used to
convert arbitrary objects to and from a simple byte array. This is
backwards compatible with the existing OTHER column type in H2, which is
implemented as a byte array, and accepts arbitrary data written with
PreparedStatement#setBytes and similar, not just #setObject.

Each SqlStore instance has a pair of CacheSerializers for the key and
value types. However, note that when the key type is String, the
existing StringKeyTypeImpl does *not* use a CacheSerializer for
serialization, since unlike the OTHER column type, the VARCHAR type
cannot just take a simple byte array. In the long term, it may make
sense to drop the KeyType class entirely, and instead use the
CacheSerializer to convert to bytes even for strings.

Change-Id: I150c1c8b8cd35b98c1a0792572d2e1ded4172774
2018-04-27 12:25:45 -04:00
Dave Borowitz
9bb008bbeb Factor KeyType singleton implementations into their own classes
Change-Id: Ia27c1304e5c8bc19a5313508730aaf1a6a53f7f4
2018-04-27 12:25:45 -04:00
Dave Borowitz
e719105e9f Factor out KeyType interface from H2CacheImpl
* Move type to new file.
* Extract an interface and an implementation KeyTypeImpl.

Change-Id: I52c7994509bba79ec61d4bce9ecfceef81439bb3
2018-04-27 12:25:45 -04:00
Dave Borowitz
f951073af0 Add Persistent subclasses of Cache{Binding,Def,Provider}
The persistent cache definition contains all the same configuration
options as the non-persistent definition, plus more that apply to only
persistent caches. Currently, there is just one option, diskLimit, but
in the future there will be more.

Change-Id: I2079e485fe4d61a470b6dd32429acccdf8591b36
2018-04-27 12:25:45 -04:00
Dave Borowitz
12b62b0538 Split setter/getter interfaces out of CacheBinding
The CacheBinding interface now contains only setters. With "binding" in
the name, it indicates that this is the thing used from within
CacheModule to configure the interface.

Move all the getter methods into a new interface, CacheDef. This is the
type that is passed into {Memory,Persistent}CacheFactory, where the
cache setup internals need to query the configured values, but calling
setters at this point would be programmer error.

This has the side advantage of eliminating the need for throwing
RuntimeExceptions from H2CacheBindingProxy, and it is now clearer that
this is simply a delegate for the getter half of the interface.

Both of these interfaces are still implemented by the same CacheProvider
class, and it still retains the freeze-during-get behavior. It may be
possible to split it up, for example converting it to a normal builder
which produces a read-only CacheDef from its build method, but I wasn't
able to get that working with the asEagerSingleton binding magic used by
the current implementation.

Change-Id: I5c90f6eb17d257a127b2c52d70558c2f80655ec2
2018-04-27 12:25:45 -04:00
Maxime Guerreiro
87de0c0622 Use Collections.addAll to copy elements from an array to collections
Change-Id: I048d005a0f75b0803be14f2128af682f0f834f8d
2018-04-27 13:36:46 +00:00
Maxime Guerreiro
d6f9e9c498 Collapse catch blocks when possible
Change-Id: I139ec81fc96b47de467ba0bcbaae6f6a0cce04b1
2018-04-27 13:36:37 +00:00
Maxime Guerreiro
249d47bf6b Replace indexOf calls with String#contains
Change-Id: Ib5f46fb846097d032490a85095669cb37b42c709
2018-04-27 13:36:30 +00:00
Maxime Guerreiro
3f7d29dde7 Allow submission with a FORCED status
Change-Id: I021661b59e36d1b76b0cf258e535dc5ef6aba564
2018-04-27 12:54:36 +00:00
Maxime Guerreiro
54f8947d77 Merge "Use method reference when possible" 2018-04-27 11:28:06 +00:00
Maxime Guerreiro
f2c79185f5 Use method reference when possible
Change-Id: I9bf98b91fbd09c68588b23f116f694f58de0dedc
2018-04-27 08:16:33 +00:00
Patrick Hiesel
ea2d0a5b5a Return a WorkQueue executor with two threads for FanOut in tests
We usually use a direct executor in tests as it is hard to catch
multi-thread bugs in tests since they usually just result in a flake.

However, the improvements in ChangeJson and AccountCache are new and
at least for now it'd be nice to get *some* test coverage on
parallelization.

Change-Id: I5697f6b4fdf92dabbbe924f234a0d4c1e4e3a48c
2018-04-27 08:46:10 +02:00
Patrick Hiesel
a8bae8376d Synchronize AccountLoader#get and #put
These methods are called from ChangeJson which runs formatting in
parallel. Since these methods modify local variables that are not
thread-safe, they need to be synchronized.

Both methods modify distinct variables so that an overall
synchronization is not required.

Change-Id: I328dc1265c3329f2c7ad66fb5946f30ca4e5f3b8
2018-04-27 08:34:38 +02:00
Aaron Gable
e8e7328f38 Add ability to specify base commit in Create Change API
The Create Change API currently supports the ability to specify
a `base_change`, and the change which you create will use that
existing change as its parent. However, it has no support for
specifying a commit sha1 which already exists on the target
branch.

This change introduces a new `base_commit` field, which takes
a 40 character sha1 as its value. It also introduces processing
to validate that the given sha1 is valid and exists on the
target branch. In also includes documentation and tests for the
above behavior.

Bug: Issue 8857
Change-Id: I7f1daa008b9a8d144c745327495dac65f581a5bf
2018-04-26 11:09:30 -07:00
Patrick Hiesel
2e4ac69fb5 Remove index field existence check for ChangeField.ASSIGNEE
Gerrit 2.15 released a new version of the change index that contains the
ASSIGNEE field. We can remove this check now since when people upgrade
to Gerrit 2.16 (which will include this commit) they will have the
ASSIGNEE field available in their index.

Change-Id: I56fb285ca31219d024babc043d64ef050e701c53
2018-04-26 09:30:04 +02:00
Dave Borowitz
346a204e14 Move PrologEnvironment creation out of ProjectState
The PrologMachineCopy field in ProjectState was an obtuse way of
upgrading the weak reference stored in the RulesCache to a strong
reference. Now that RulesCache stores strong references on its own, we
can just call into RulesCache at exactly the time it's needed, namely in
PrologRuleEvaluator.

Moving this out of ProjectState helps with untangling the cacheable and
non-cacheable pieces of this class; for context, see:
https://gerrit-review.googlesource.com/c/gerrit/+/167053

Change-Id: I4beda2c9ddcf0014c606c0bee01ec49dde419361
2018-04-25 13:07:06 -04:00
Dave Borowitz
08aa8bb89d RulesCache: Replace manual weak cache with regular Guava cache
Previously, PrologMachineCopy instances were stored with weak references
in RulesCache's internal cache, which had no other size constraints.
Instances were copied as hard references to ProjectState on demand,
meaning we would only spend memory on PrologMachineCopy instances that
were active in a project we care about in the cache. This was an awkward
and non-obvious two-layer cache.

We can achieve very similar memory performance if we simply convert
RulesCache to a standard Guava cache with hard references, sized to the
number of projects in the cache. Typically, we will store up to one
PrologMachineCopy per project in the cache, and the project cache is
generally sized to the total number of projects in the site. We can let
them expire out of the RulesCache in normal LRU fashion.

Under this implementation, it's possible we will hold on to stale rules
longer than necessary, particularly if rules.pl changes frequently and
the project cache is sized much larger than the total number of
projects. This could be avoided if we convert RulesCache to a
Cache<Project.NameKey, Pair<ObjectId, PrologMachineCopy>> and revalidate
the ObjectId on load; but at this point, that's a premature
optimization.

Tie the configuration of the RulesCache to the ProjectCache, so that it
is sized appropriately. This means that it can't be configured
independently, but we still need to at least mention it in the
documentation, since it will show up in the list of caches exposed to
administrators at runtime.

Change-Id: I1d3e3c013c30dd6d8a2a5d12f45a8e276d6051cd
2018-04-25 13:07:06 -04:00
Dave Borowitz
19c3360e14 Allow overriding the config key used to configure a cache
This will allow us to tie the size of one cache to match the size of
another cache.

Change-Id: I6e2fa78db8aec209fdc71ace74c28778c8192675
2018-04-25 13:06:40 -04:00
Hugo Arès
64e1054309 Merge "Add validation listener for account activation" 2018-04-25 13:27:15 +00:00
Hugo Arès
d66a5d2da7 Merge "Expose instanceName in the subject template" 2018-04-25 13:23:18 +00:00
Edwin Kempin
dc49e46dd0 Merge "Prevent duplicate permissions in AccessSection" 2018-04-25 13:17:44 +00:00
Maxime Guerreiro
eb4d2d8da7 Expose instanceName in the subject template
Add a variable `instanceName` to the variables offered in the subject
template.

Remove the instanceAndProjectName as it is easy to reconstruct it using
the projectName and instanceName variables. It was also incorrect, as
it relied on the short name and not on the project name.

It was introduced recently, so _I_ don't think any gerrit instance will
break.

Change-Id: Iaf46fcbc1df6db637cb95711bac84a4ead5fc036
2018-04-25 12:59:13 +00:00
Maxime Guerreiro
1b70a3d7d3 Merge "Rename setUp/tearDown methods in abstract tests" 2018-04-25 12:56:28 +00:00
Edwin Kempin
8044bb1b87 Prevent duplicate permissions in AccessSection
Duplicate permissions in an access section are not allowed. This is why
AccessSection#setPermissions(List<Permission>) fails with an
IllegalArgumentException if the provided list contains duplicate
permissions. However by calling AccessSection#addPermission(Permission)
it was possible to add duplicate permissions.

Duplicate permissions on an access section of the All-Projects project
make the permissions of the All-Project project unreadable and Gerrit
effectively stops working. This is because
ProjectState#getLocalAccessSections() filters out some permissions on
the All-Projects project by getting the list of permissions from
AccessSection and setting the filtered list back on AccessSection. If
the list that was retrieved from AccessSection contained duplicate
permissions, setting back the list failed with IllegalArgumentException
since AccessSection#setPermissions(List<Permission>) doesn't allow
duplicate permissions.

Also see change I29298232a3 which demonstrates this bug.

This change makes the behavior of the methods in AccessSection
consistent, AccessSection#setPermissions(List<Permission>) and
AccessSection#addPermission(Permission) now both reject duplicate
permissions.

The caller of AccessSection#addPermission(Permission) that tried to add
duplicate permissions was removed by change I139ba7aa97 which reverted
the in-place migration of refs/for/ permissions. When this code is added
back the new implementation must care about merging duplicate
permissions, otherwise the migration code will now fail.

In general it's better to fail at this earlier stage and prevent the
creation of a AccessSection with duplicate permissions as this prevents
that such a bad AccessSection is ever stored.

Change-Id: I5222cd9174f0b17baae2e66702c4aec6133a14ba
Signed-off-by: Edwin Kempin <ekempin@google.com>
2018-04-25 14:51:51 +02:00
Edwin Kempin
2890801f7e Merge "Extend project resetter to groups" 2018-04-25 12:12:40 +00:00
Han-Wen Nienhuys
e989032906 Extend project resetter to groups
Change-Id: I26681ede7468924143ed38ea6d758eebe1a8d3d8
2018-04-25 13:49:54 +02:00
Maxime Guerreiro
c229c4e710 Merge "Expose the label copy scores config keys" 2018-04-25 11:41:42 +00:00
Maxime Guerreiro
e55f9bff06 Expose the label copy scores config keys
Change-Id: I3ec093e2a9e75fc36951b57f5852b38afd5d8954
2018-04-25 10:49:28 +00:00
Maxime Guerreiro
55065875f9 Rename setUp/tearDown methods in abstract tests
By replacing the name from "setUp" to more explicit names, like
"setUpTestPlugin", we make sure nobody uses the name "setUp" by mistake,
thus overriding and cancelling the positive effects provided by the
base test class.

Change-Id: I0f74b437589e3b837343e4648d3679b34875109c
2018-04-25 09:15:57 +00:00
Edwin Kempin
6b5c28b02c Merge "Push down more AbstractDaemonTest methods" 2018-04-25 08:49:59 +00:00
Edwin Kempin
ef4166ff42 Merge "Allow to set active flag from new test API for accounts" 2018-04-25 08:46:03 +00:00