1439 Commits

Author SHA1 Message Date
Gustaf Lundh
e0f52ff170 Support reloading of gerrit.config
This change introduces a mechanism to allow reloading of the
@GerritServerConfig. To execute a reload of gerrit.config, the SSH
command "reload-config" is provided.

Challenges and implementation:
------------------------------
Most callers that inject @GerritServerConfig are @Singletons. These
@Singletons will only parse the GerritServerConfig once and then
build and store more suitable datastructures to work from.

To allow these modules to react on config changes, a caller can now
implement GerritConfigListener, which will enable the caller to receive
notifications of future config updates.

The GerritConfigListener provides:

    void configUpdated(ConfigUpdatedEvent event);

The ConfigUpdatedEvent provides the old config and the new one.
Implementing classes of the listener are expected to do three things:

1. Investigate if any of the updated config values are of interest.
The ConfigUpdatedEvent provides helper methods for this purpose:

    public boolean isSectionUpdated(String section)
    public boolean isValueUpdated(String section, String subsection, String name)
    (+ various overloaded versions of these)

2. React to the configuration changes (if any) and apply them

3. Accept or reject the entries of interest:

    public void accept(Set<ConfigKey> entries)
    public void accept(String section)
    public void reject(Set<ConfigKey> entries)
    (+ various overloaded versions of these)

When a section (or specific config keys) are accepted or rejected, the
ConfigUpdatedEvent will build a diff of what changes were accepted and
present it to the administrator that issued a configuration reload.

For instance, in this case, 4 config rows where added(+), 2 removed(-1)
and 1 modified. All but one change were accepted, hence the user would get
this output:

$ ssh $HOST -p 29418 gerrit reload-config

Accepted configuration changes:
- suggest.maxSuggestedReviewers = 10
- suggest.accounts = 50
* addreviewer.maxAllowed = [100 => 1000]
+ sshd.requestLog = true
+ commentlink.changeid.link = #/q/$1
+ commentlink.changeid.match = (I[0-9a-f]{8,40})

Rejected configuration changes:
+ gc.startTime = Fri 10:30

Documentation:
--------------
We start by documenting the config entries that support config reloads
(which number will be increasing over time).

When we support reloading of the majority of config settings, the
documentation can switch to documenting the config entries that does
_not_ support config reloading instead.

Roadmap:
--------
This change will be followed by a couple of changes that introduces
reloadable config support for: suggest-reviewers, sshd-log and
commentlinks.

Other stuff TODO includes:
* REST end point
* Rejection messages (to allow for an explanation on why the value was
not accepted)
* Test cases
* An option to allow displaying the current effective configuration
* A lot of implementations of the GerritConfigListener.

Change-Id: I4bd6f389731af303ef9ba5d1d73f173d869c62e4
2018-04-19 13:21:24 +02:00
Edwin Kempin
d391c0a9ff Merge changes I80e06b27,I01cbebd7,Iccce5845
* changes:
  Fix the MERGE_CONFLICT name in ChangeStates
  Replace "the the" occurences with "the" (wrong repetition)
  Merge branch 'stable-2.15'
2018-04-19 08:41:47 +00:00
Dave Borowitz
c040bd2eaa Merge changes I4bfe80d4,Ifb317d8a
* changes:
  Add a proof-of-concept test for error messages in ReceiveCommits
  Add a Truth Subject for asserting about JGit PushResults
2018-04-19 08:30:31 +00:00
Maxime Guerreiro
9febabb7b0 Replace "the the" occurences with "the" (wrong repetition)
Change-Id: I01cbebd79e1d3470a4f00e55a228d57ac156b676
2018-04-19 09:38:45 +02:00
David Pursehouse
5a1e6cafa7 Merge branch 'stable-2.15'
* stable-2.15:
  Fix typo in cmd-stream-events.txt
  Replace "the the" occurences with "the" (wrong repetition)

Change-Id: Iccce5845c697be54e0cd2257ab39f90a02bfe27e
2018-04-19 09:38:17 +02:00
Maxime Guerreiro
f13ee1e78a Try to stop plugin if enabling it failed
Without this, a plugin failing to load might break other unrelated
plugins. This occurs because a Guice environment is created and a
serverManager is created but never started.

This should only be triggered when a bogus plugin is installed, which
is more likely to occur during plugin development.

Also initialize the fields outside of the constructor.

Change-Id: I0bcbe331697e320430711a776357800f1f69c8a3
2018-04-19 07:36:07 +00:00
Patrick Hiesel
cd0b77954f Merge "Fix nits in PerThreadCache" 2018-04-19 07:32:24 +00:00
Patrick Hiesel
4e73cdb26f Fix nits in PerThreadCache
Change-Id: Ibb806919b07b0df76e25efe222ed0e8fae5a1041
2018-04-19 07:08:30 +00:00
Hugo Arès
d00c30dfa1 Merge "Remove explicit binding to DefaultCacheFactory" 2018-04-19 14:37:55 +00:00
Jacek Centkowski
fcfb958fa4 Remove explicit binding to DefaultCacheFactory
Rely on implicit binding instead.

Change-Id: I13057cb192cec65ca6b673676c309d02d1927661
Signed-off-by: Jacek Centkowski <jcentkowski@collab.net>
2018-04-19 16:18:34 +02:00
Gustaf Lundh
3a26660215 Merge "Create Review Permission" 2018-04-19 14:11:08 +00:00
Gustaf Lundh
97afc05387 Create Review Permission
This is the first change in a series that intends to getting rid of
refs/for/ from ACLs.

The migration from "push refs/for/*" to "create-review refs/heads/*"
is handled on-the-fly when the configuration is loaded.

Any "push refs/for/*" permission will be  automatially migrated to the
new "Create Review" permission and the old permission will be removed
if possible (e.g. push refs/* will create a new Create Review permission,
but keep the old permission since it will affect refs/{heads,meta}/*).

When the rest of the refs/for/ dependent permissions are migrated, an
offline schema migration should be available to help correct all configs,
though the online auto migration should be available for a few major
versions to help fix ACL-mistakes by users.

Change-Id: Ia112adeb7593a7b4a7a8b5d8b96f72f73c9b70b8
2018-04-19 13:13:54 +02:00
Edwin Kempin
28c50a4ee5 Don’t limit query results to open changes when no index predicate is used
If a change query doesn’t contain any index predicate, but only post
filter predicates the query result was limited to open changes only.
E.g. “reviewerin:Users” was finding only open changes while
“reviewerin:Users message:foo” considered open and closed changes. This
behaviour was confusing users.

Bug: Issue 8597
Change-Id: I88d93d60a5572345a4c4e8a5d58628b4c1a4851d
Signed-off-by: Edwin Kempin <ekempin@google.com>
2018-04-19 08:43:04 +02:00
Edwin Kempin
393e008070 Delete ChangeOperatorPredicate and use PostFilterPredicate instead
Using PostFilterPredicate better describes what these predicates do,
they post filter the result of change queries. With this change the
behaviour doesn’t change, especially for queries with a post filter
predicate but no index predicate we still fall back to query only open
changes. This should be addressed in a follow-up change.

Change-Id: I6527e15777c174e7616a56a82fdd1c69ad98f9c5
Signed-off-by: Edwin Kempin <ekempin@google.com>
2018-04-18 22:47:21 +02:00
David Pursehouse
79ed9a7504 Merge "Document AbstractDaemonTest#projectInput" 2018-04-18 15:18:58 +00:00
David Pursehouse
150bf7e3de Merge "Trim AbstractDaemonTest#createProject methods" 2018-04-18 15:15:44 +00:00
Dave Borowitz
fb315314ba Merge "AbstractDaemonTest: Make updating project config cleaner & safer" 2018-04-18 15:08:00 +00:00
Dave Borowitz
f7073d855a Merge "Drop trivial method AbstractDaemonTest#fail" 2018-04-18 15:01:40 +00:00
Patrick Hiesel
d34ab6c744 Merge "PerThreadCache: Add static convenience method" 2018-04-18 15:01:27 +00:00
Dave Borowitz
430ea67939 Merge "Push down more AbstractDaemonTest methods" 2018-04-18 15:01:19 +00:00
Patrick Hiesel
6709133248 Merge "PerThreadCache: Fix or suppress type warnings" 2018-04-18 15:01:12 +00:00
Dave Borowitz
dac9360fca Merge "H2CacheFactory: Respect diskLimit setting in CacheBinding" 2018-04-18 14:29:05 +00:00
Han-Wen Nienhuys
c0d03824a6 Document AbstractDaemonTest#projectInput
Change-Id: I81f63008b34f8292f322a43b0df9bd4c543f48cd
2018-04-18 16:22:41 +02:00
Dave Borowitz
f474dbcc78 PerThreadCache: Add static convenience method
Change-Id: Id599161b12c7b0e2c39ed28f01ff769d9f71b91e
2018-04-18 16:18:24 +02:00
Dave Borowitz
81f373de9e PerThreadCache: Fix or suppress type warnings
Change-Id: Iaf5ad96cfad344b502f050113d735d4d8bb76497
2018-04-18 16:18:24 +02:00
Han-Wen Nienhuys
445378d574 Trim AbstractDaemonTest#createProject methods
Change-Id: I4d23742454db49a9e23d4fd7757d1d8eb717a381
2018-04-18 16:17:18 +02:00
Dave Borowitz
710bce9bb7 AbstractDaemonTest: Make updating project config cleaner & safer
The old set of saveProjectConfig methods is rather unsafe:
 * It was possible to pass in a ProjectConfig corresponding to a
   different project than the Project.NameKey passed to the same method.
   I know I have written tests that failed because I mixed up the config
   and name with allProjects vs. project.
 * Convenience methods like allow would read the ProjectState from the
   ProjectCache and mutate its ProjectConfig in-place. This would be
   highly unsafe to do in a real server where the cached ProjectState is
   accessed from multiple threads; we're just lucky that this only
   happened in a single-threaded test context, and the project cache was
   always subsequently flushed.
 * Many callers elsewhere in various tests emulated this bad behavior of
   reading from the cache.

This change introduces a new test utility ProjectConfigUpdate which
handles opening the MetaDataUpdate, saving and evicting, and closing
when finished. This may not be the ideal interface we ultimately end up
with for updating project configs, but it's definitely safer and cleaner
than the status quo, and migrating callers was simple and mechanical.

Change-Id: Id4ad7bcdafa2b61a23bc09f10a703bbf358820d4
2018-04-18 16:03:58 +02:00
Dave Borowitz
248e9d3b49 ChangeJson: Fix javadoc link to ChangeData
The lazyLoad field has no javadoc on it, so a link to it is not useful.
(Eclipse also complains that the field itself is private).

Change-Id: Ibf82481539bcfddda8c61ccac13edac77a8f9720
2018-04-18 14:03:21 +00:00
Patrick Hiesel
16c0875d40 Implement a PerThreadCache
This commit implements a per-request cache as a per-thread cache.

PerThreadCache is intended to cache objects that have a high
instantiation cost, are specific to the current request and
potentially need to be instantiated multiple times while serving a
request.

The implementation uses a ThreadLocal which limits the cache to
the serving thread. Given that fan-outs (like ChangeJson's executor)
run with high parallelization, it seems acceptable to re-do some
instantiation there in lieu of this cache.

This commit uses a the new cache in DefaultPermissionBackend to cache
a ProjectControl instance.

Change-Id: Id5de21ed8941c7e08eb37197b16ce2cadece7aea
2018-04-18 13:55:19 +00:00
David Pursehouse
76846bdeba Merge "Make GitReferenceUpdated.Event public" 2018-04-18 13:53:31 +00:00
Han-Wen Nienhuys
4140ea3391 Drop trivial method AbstractDaemonTest#fail
Change-Id: I5efcd33f1253322a6e9c1f09eb084060f364740a
2018-04-18 15:47:32 +02:00
Han-Wen Nienhuys
37d913af63 Push down more AbstractDaemonTest methods
Change-Id: I3a15fce564381bdf4fbc453dace86524673753ba
2018-04-18 15:45:43 +02:00
Han-Wen Nienhuys
274cd4f071 Inline trivial helper method AbstractDaemonTest#previewSubmit
Change-Id: I4515240113524d0e3040be55b66da02aff20422e
2018-04-18 15:24:08 +02:00
Han-Wen Nienhuys
95f643fa52 Push down AbstractDaemonTest#grantTagPermissions to callers
Change-Id: I73285200bf7c8617faa56662b5a0c6c1255e7db1
2018-04-18 15:24:08 +02:00
Patrick Hiesel
86e01bca9b Use FanOutExectutor for ReviewerRecommender
The newly created thread pool should be used for parallelizing
work from request serving threads.

This is a fairly large pool that makes it less likely that
ReviewerRecommender uses a congested queue.

Change-Id: Ie229ef98c33ec475b94fc76d8fc2bd0c0872465a
2018-04-18 14:19:44 +02:00
Patrick Hiesel
b6e9a0661c Offer a parallelized way of getting accounts from AccountCache
Loading accounts from disk if they are not present in the cache
is OK for single accounts but creates significant latency burden
when multiple accounts need to be loaded.

This commit adds a method to AccountCache that allows for
parallelization and implements it on a new executor in
AccountCacheImpl.

Change-Id: I1b58b2f71c8b1c62e809314e6c50ac1b25a3a166
2018-04-18 14:19:44 +02:00
Patrick Hiesel
2ed3982ffb Parallelize ChangeJson#toChangeInfos
99%ile latency of QueryChanges is between 20 seconds and
80 seconds on googlesource.com depending on the time of
day. There are two main reasons for that:
1) Performing operations that require loading ChangeNotes
2) Performing operations that require opening the repo
3) Filling accounts with a cold AccountCache

This commit parallelizes formatting the individual results
to mitigate 1+2. 3 will be addressed by a different change
that will make the AccountFiller parallelize, too.

Parallelization is done on a newly introduced FanOutExectuor
that can be used whenever a serving thread wants to parallelize
work.

Change-Id: I36c6b92e31488ad001f5aea43efc837d31ba3021
2018-04-18 14:19:38 +02:00
Patrick Hiesel
867b2aa941 Add getCacheKey() to CurrentUser and its implementations
getCacheKey() can be used when grouping objects by user in caches. The
first user of this new method is a PerThreadCache added in a later commit.

Change-Id: I00a9c1528e2fdd2219f511ba749074efe9d967b6
2018-04-18 13:26:19 +02:00
hanwen
659a06d698 Merge changes I5270b683,I29a7c8d6
* changes:
  Provide a GET endpoint for CheckAccess
  introduce the viewAccess global capability.
2018-04-18 11:05:52 +00:00
hanwen
7df1990b31 Merge changes I79cfa5b6,I9e67bf58,If5441d01
* changes:
  Inline one-liner utility AbstractDaemonTest#changeIds
  Push AbstractDaemonTest#getActions and #getETag down
  Remove single-use method AbstractDaemonTest#blockForgeCommitter
2018-04-18 11:04:35 +00:00
Saša Živkov
fd56601396 Make GitReferenceUpdated.Event public
This is needed for the replication plugin which will need to serialize
GitReferenceUpdatedListener.Event objects into json and deserialize them
as the implementation class GitReferenceUpdated.Event.

Change-Id: I672fc34841d7da2c26067ec9eccd608a15f934a9
2018-04-18 12:02:44 +02:00
Maxime Guerreiro
be34e5cf1c Merge "Expose the submit requirements in the Change API" 2018-04-18 09:03:13 +00:00
Han-Wen Nienhuys
4c91005a15 Inline one-liner utility AbstractDaemonTest#changeIds
Change-Id: I79cfa5b66b6ea2ae91fd1ddd92a77376606b0be6
2018-04-18 10:53:51 +02:00
Han-Wen Nienhuys
ffd7001cd7 Push AbstractDaemonTest#getActions and #getETag down
They are used rarely so have to be in AbstractDaemonTest

Change-Id: I9e67bf585d555d4e7d8be2f6b9d16b474d8ad9ac
2018-04-18 10:51:51 +02:00
Han-Wen Nienhuys
990254c674 Remove single-use method AbstractDaemonTest#blockForgeCommitter
Change-Id: If5441d0156d3394bc1ea66c97aaf1d22d088e56d
2018-04-18 10:43:24 +02:00
David Pursehouse
6a1cfd20c9 Upgrade to google-java-format 1.5
Change-Id: I0c72f6c4bc0d0a93605240c2d427c80751b6ccc8
2018-04-18 10:36:46 +02:00
Han-Wen Nienhuys
5b089f2966 Provide a GET endpoint for CheckAccess
Practically speaking, the CheckAccess endpoint is annoying for use in
administrative actions, because it requires the right Content-Type
headers.

Philosophically, the endpoint only inspects, so it should not use the
POST method.

Change-Id: I5270b683f51f2876785ba84e762b7ecedbceea21
2018-04-18 08:33:44 +00:00
Han-Wen Nienhuys
a0ce3bb311 introduce the viewAccess global capability.
This capability can be assigned separately so users can run check.access
without having to be admin.

Change-Id: I29a7c8d637a606ce9f7cf353ba242b1c43c2682b
2018-04-18 08:33:35 +00:00
Dave Borowitz
cd80425b54 Merge changes I374e9081,I9310b58f,I6df14c97,I93d0c7cd
* changes:
  Update .mailmap
  InitPluginStepsLoader: Remove redundant warning suppression
  Upgrade auto-value to 1.6
  Merge branch 'stable-2.15'
2018-04-18 08:33:27 +00:00
Edwin Kempin
6fe4b90101 Merge "Document ChangeJson#lazyLoad and ChangeData#lazyLoad" 2018-04-18 08:32:13 +00:00