966 Commits

Author SHA1 Message Date
Dave Borowitz
0ecf8cf401 Avoid Multimap implementation create methods
Guava team recommends MultimapBuilder over specific Multimap
implementations, so callers don't have to know the specific key/value
behaviors of the individual implementations. The static factory
methods in the implementations will be removed in a later version.

LinkedListMultimap and LinkedHashMultimap are not affected, since
MultimapBuilder lacks support for specifying linked entries, and thus
the factory methods are not in immediate danger of deletion.

Change-Id: I7744db687da84a7beae31d1cb8953e782ed23c1d
2017-01-18 10:37:06 +09:00
Dave Borowitz
484da493b3 Prefer subtypes of Multimap
Guava team recommends using the subinterfaces of Multimap, for the
same reasons they recommend using Set and List rather than Collection:
it documents expectations about ordering, uniqueness, and behavior of
equals. Do this across the board in Gerrit.

Mostly this is straightforward and I tried to exactly match existing
behavior where possible. However, there were a few wrinkles, where
different callers passed different subtypes to the same method.

The main one is arguments to ParameterParser#parse and
splitQueryString, where some callers used SetMultimaps (perhaps
semi-intentionally, or perhaps misunderstanding the nature of
HashMultimap). For the purposes of parameter parsing, a ListMultimap
makes more sense, because it preserves argument order and repetition.

Another instance is a couple places in ReceiveCommits and downstream
where there were SetMultimap<?, Ref>. Since Refs do not implement
equals, this is effectively the same thing as a ListMultimap, and
changing the interface no longer misleads readers into thinking there
might be some deduplication happening.

Finally, this change includes a breaking API change to the return
type of ExternalIncludedIn#getIncludedIn.

Change-Id: I5f1d15e27a32e534a6aaefe204e7a31815f4c8d7
2017-01-18 10:36:38 +09:00
Jacek Centkowski
ae52c0450d Introduce "git-lfs-authenticate" SSH Command for LFS plugins
According to [1] Git LFS can use SSH protocol to either obtain Git LFS
endpoint or authorize the following Git LFS request (upload/download).
This patchset introduces Git LFS SSH command stub that forwards call to
implementation provided by Git LFS plugin. In case no plugin is loaded
it exits with 1.

Different attempts to solve this problem were submitted for review ([2],
[3]) but there were either too generic or simply to controversial to
address the problem correctly. I believe that this attempt is superior
as:
1. it is specific to the problem - doesn't open any Pandora's box
2. it comes along with existing solution that adds Git LFS HTTP Servlet
endpoint that transfers Git LFS HTTP calls to plugin

[1]
https://github.com/git-lfs/git-lfs/blob/master/docs/api/server-discovery.md
[2] https://gerrit-review.googlesource.com/93496
[3] https://gerrit-review.googlesource.com/93373
Change-Id: I5f9b95c4b29cd8c0d64053e986851500f147eb5a
Signed-off-by: Jacek Centkowski <geminica.programs@gmail.com>
2017-01-12 10:32:06 +01:00
Edwin Kempin
92b9a7c1b2 Index group on creation and whenever a group is evicted from cache
Change-Id: Ia6a8186f0fd91026d59b1337ffd012c53dbda1e9
Signed-off-by: Edwin Kempin <ekempin@google.com>
2017-01-11 11:56:02 +01:00
Luca Milanesio
9a52dd1dce Do not assume DB down if change to index not found
The Gerrit SSH command "index changes" may receive change numbers that
cannot be found or refer to repositories that have problems.

The assumption of "DB is down" when receiving an OrmException was wrong
because the JGit errors were simply wrapped as OrmException and
do not necessarily mean that you cannot continue to process the other
ones in the list.

Change-Id: I1098be16b682dce8d4e63120e7e50f937b19a7b8
2016-12-08 23:16:30 +00:00
David Ostrovsky
fa18907d7f Bazel: Reformat build files
Reformat the Bazel build files with the buildifier tool [1].

The style is different for Bazel files. Most notably, indentation level
is 4 spaces instead of 2, and " is used instead of '.

[1] https://github.com/bazelbuild/buildifier

Change-Id: I95c0c6f11b6d76572797853b4ebb5cee5ebd3c98
2016-12-07 11:33:07 +00:00
David Pursehouse
c89bc10e92 Merge branch 'stable-2.13'
* stable-2.13:
  Support move change via SSH

Change-Id: I342bc28410a39a189b5cee64be8c57253bd3d18d
2016-12-06 18:09:55 +09:00
Luca Milanesio
117536d7bf Allow to continue reindex despite failures
Display a warning when a change is not found but then
continue to try to reindex the rest of the changes, consistently
with the current behavior of the command.

Change-Id: Idbd3d450c34788a9f5dd304b052e5810593088fe
2016-11-30 12:10:52 +00:00
Orgad Shaneh
8ad3ae5469 Support move change via SSH
Change-Id: I64ec14bea8a81c439d51459462abd2fcfb7ae81d
2016-11-23 12:50:27 +02:00
David Pursehouse
ede1ff4004 Merge branch 'stable-2.13'
* stable-2.13:
  Fix NPE when requesting invalid Change-Id to index

Change-Id: Id909cd3585a5631bb0ae7947e6e4cb94117fc3a9
2016-11-17 14:51:29 -08:00
Luca Milanesio
650769c46b Fix NPE when requesting invalid Change-Id to index
When an invalid Change-Id containing a comma ',' is provided to the
SSH gerrit index changes command, it does not blow up and instead
returns an "Invalid change ID" error and exit code 2.

Change-Id: I97e888e372eb886b9ac9815791a7a725802d736f
Bug: Issue 4911
2016-11-14 05:04:14 -08:00
David Pursehouse
1a4ff2f233 Merge branch 'stable-2.13'
* stable-2.13:
  SetMembersCommand: Handle REST errors gracefully

Change-Id: I59cdf98ef4d24fa3f97fb4faaf7964b03b99b454
2016-11-04 19:24:42 +09:00
David Pursehouse
664a284bc6 SetMembersCommand: Handle REST errors gracefully
If a REST error occurs while adding a member or group to a group, the
command fails with "internal server error" which is not helpful to the
user.

Catch REST errors and allow them to be reported back to the user.

Other types of error will still result in "internal server error".

Change-Id: Ia21f7aa28212f6d93ec4e6ff74a9de9ee6428a00
2016-11-04 10:21:06 +09:00
Hugo Arès
18d8f00aeb Remove unused import
Change-Id: I5799cb58ec1cc2d01914a8633a3e6047fc6ff781
2016-11-03 09:34:30 -04:00
Luca Milanesio
e624bc00f5 ChangeArgumentParser: Replace FluentIterable with Java 8 stream
Change-Id: I34dbf535241a604089ddb5d3af570c4b97c5cc2e
2016-11-02 17:40:59 +09:00
David Pursehouse
34c0481f47 Merge branch 'stable-2.13'
* stable-2.13:
  Fix to reindex a change via SSH: get from DB/Notes instead of Lucene
  Update git submodules
  Fix Postgresql JDBC driver leaking memory
  Fix Schema_127 for mysql on case-sensitive FS
  Update git submodules

Change-Id: Ida6a895ccb8cab2c68bb0cdb83fe4d076f2b007c
2016-11-02 17:36:09 +09:00
Luca Milanesio
7bcb71def6 Fix to reindex a change via SSH: get from DB/Notes instead of Lucene
When reindexing a change via SSH, we need to avoid using the
Lucene index to decode the change-id into a ChangeControl.

The rationale stays in the typical use-case of the reindex itself:
if we need to recreate the Lucene index entry we cannot rely on
a Lucene query to find it.

Change-Id: I34dbf535241a604089ddb5d3af570c4b97c5cc2e
2016-11-02 08:05:43 +00:00
David Pursehouse
ef6398e9d0 Upgrade Guava to 20.0-rc1
The propagate and propagateIfPossible methods are deprecated. Replace
them with the recommended alternatives.

Change-Id: I9107dc51b968e2acc453502ece3c5ed5dd58b5de
2016-10-24 23:46:14 +09:00
Hugo Arès
cf4a2010d6 Merge branch 'stable-2.13'
* stable-2.13:
  Update 2.13.1 release notes
  Catch exceptions from StreamEvents JSON Serialization
  Set version to 2.13.1
  Add 2.13.1 release notes

Change-Id: Ida2a689a71c4f7b94a057bd53cc47c8329d9b015
2016-09-23 16:02:53 +02:00
Gustaf Lundh
e698965e20 Catch exceptions from StreamEvents JSON Serialization
Sometimes GSONs JSON serialization method would fail, allowing the
exception to travel up to the event dispatcher and deregister the
listening user, without any logs identicating an issue.

The user would still be connected over SSH, not knowing something
went wrong.

Catch the exception, log it, and allow the stream-event to continue
serving events.

Change-Id: I4b1d4db6934566f1213ebc2e416dd95c1b0b07e6
2016-09-23 15:44:53 +02:00
Dave Borowitz
a62dc20d7f Convert some Functions/Predicates to streams & lambdas (5)
Change-Id: I4eca94928d77a15a81678dcd91b8d6174b4f6ec3
2016-09-21 04:27:43 -04:00
Gustaf Lundh
aa5d14be3e BaseCommand: Removed unused checkExclusivity method
checkExclusivity is unused since change Ic4a6ea6ff2.

Change-Id: Idbfbb439b3d7a5177b52f042121dca3c357e988f
2016-09-21 07:47:31 +00:00
David Ostrovsky
0b774c478e Update Buck to latest version
This version fixed a major issue: [1] that was a reason of frustration
of many plugin developers: Not cache sources files under symbolic link.
Now for all such source files, the warning is issued:

"
Disabling caching for target //plugins/wip:wip__plugin, because one or
more input files are under a symbolic link
({plugins/wip=/home/davido/projects/wip}). This will severely impact
performance! To resolve this, use separate rules and declare
dependencies instead of using symbolic links.
"

To suppress this warning we add project.allow_symlink option. This
doesn't have any impact for gerrit core but silences the warning above
when plugins are built in gerrit tree mode.

As pointed out in this issue: [2], we are using some artifacts as source
to the java_library() rule as well as binary_jar for prebuilt_ja rule.
To avoid the warning, we rename sources to have "-sources.jar" suffix
and we rename *.zip to end with .jar in other places.

"
Assuming edit.src.zip is a JAR and renaming to edit.src.jar in
//gerrit-patch-jgit:edit_src. Change the extension of the binary_jar to
'.jar' to remove this warning.
"

source_under_test attribute was removed from java_test() rule.
Replication and cookbook-plugin are updated as well.

local.properties support was removed, but we use it only for download
process customization in our own python script, so that we can keep it
usage and not need to move it to .buckconfig.local.

[1] https://github.com/facebook/buck/issues/341
[2] https://github.com/facebook/buck/issues/855

Change-Id: Idf76cc71c21df43e808179b645f9175767b322a8
2016-09-20 13:19:15 +02:00
Stefan Beller
d96c3e5986 Move GetArchive.AllowedFormats to its own file
Change-Id: I999ed908d587ff9af91715c488b944de5254fac4
2016-09-14 10:05:11 -07:00
Edwin Kempin
407fca3cc2 Support notify option for delete reviewer REST endpoint
Some automated tools may want to suppress email notifications when
removing reviewers.

Change-Id: I26f66272d4f871aae9d36f3524faea9ffa86ea35
Signed-off-by: Edwin Kempin <ekempin@google.com>
2016-08-29 13:08:18 +02:00
Dave Borowitz
e1767e5ad7 Revert "Remove usage of to-be-deprecated Throwables"
Guava snapshot is gone.

This reverts commit 8df8a2489d95726a899a4642e72ff4dcce63ee37.

Change-Id: I1f6f40a824e39240d67da9b305bdc8144505e7bf
2016-08-25 00:09:01 +00:00
Scott Dial
7c707175cf Add configuration of key exchange algorithms for sshd
Add new config key "sshd.kex". The default and supported values are:

  1. ecdh-sha2-nistp521
  2. ecdh-sha2-nistp384
  3. ecdh-sha2-nistp256
  4. diffie-hellman-group-exchange-sha256
  5. diffie-hellman-group-exchange-sha1,
  6. diffie-hellman-group14-sha1
  7. diffie-hellman-group1-sha1

With Bouncy Castle installed, all of the above are supported (previously
only 6 and 7). With JCE, only 7 is available.

Bug: Issue 3517
Change-Id: I6b44e88dc4a0ff8f693f21510aba30546bf4cd99
(cherry picked from commit b4a04fa1c5d5ab6e87ac5cd2a046395090cf163f)
2016-08-24 11:06:30 +00:00
David Pursehouse
3f34c3fefe Merge "Add configuration of key exchange algorithms for sshd" 2016-08-24 04:02:59 +00:00
David Pursehouse
8df8a2489d Remove usage of to-be-deprecated Throwables
The propagate and propagateIfPossible methods will be deprecated
in Guava 20. Replace them with the recommended alternatives.

Change-Id: I1c8d19aac5434c938ee10af88766af14e5e4d023
2016-08-19 23:02:10 +09:00
Dave Borowitz
040c39bcb3 Fix auto-adding reviewers during push
When we converted ReceiveCommits to use BatchUpdate's parallel
functionality in I40545a4d, we lost the automatic request scope
propagation. This was mostly fine, with the notable exception of:
 - pushing a new patch set of an existing change, and
 - pushing multiple changes so work is in a background thread, and
 - mentioning a user in a footer (Signed-Off-By, etc.), and
 - not including an email address in that footer, and
 - not having the account index enabled.

This would cause AccountResolver to try to call its
Provider<ReviewDb>, which fails because it's not in request scope.

Fix this by passing a ReviewDb into AccountResolver methods. That was
the easy part; the hard part was figuring out how to write a test case
that triggered this. Since the account index is now enabled by
default, this means putting a test-only hack into LuceneIndexModule
to support disabling a specific index. (This could also be useful for
other tests, since we currently don't exercise the non-index
fallbacks.)

Change-Id: I7231be3ea4660c9ee27f09994706b39ee622488a
2016-08-15 09:58:14 +09:00
Edwin Kempin
447bcdfffc Merge "ShowConnections: Only show start and idle columns for mina backend" 2016-08-04 06:09:28 +00:00
David Pursehouse
a407a3fb5b ShowConnections: Only show start and idle columns for mina backend
The nio2 backend does not provide these data, so displaying dummy
values can confuse users.

Modify the command to only show those columns when the backend is
mina, which does support them.

Change-Id: I889339a4811e38c1b02211983bde6b6e017cd600
2016-08-04 05:55:37 +00:00
David Pursehouse
2b2c3dd55a Merge branch 'stable-2.12'
* stable-2.12:
  Set uploader for inline edit
  Update 2.12.4 release notes
  SshDaemon: Set up the session timeout to make sshd.idleTimeout work
  Attach sources for sshd-core and mina-core

Change-Id: I651e8901f96646660790a7ef6a14275f6d94343c
2016-08-04 11:18:01 +09:00
Hugo Arès
c66120af5a Merge "Add an optional flag to force an online reindex" 2016-08-03 13:18:35 +00:00
David Pursehouse
b150ce534a SshDaemon: Set up the session timeout to make sshd.idleTimeout work
SshServer#start() sets up the session timeout by calling:

  setupSessionTimeout(sessionFactory);

but SshDaemon overrides the start() method, does not call super(), and
does not call setupSessionTimeout.

Therefore, no session timeout is set up on ssh sessions, and sessions
continue to be kept open beyond the timeout specified in the configured
sshd.idleTimeout.

Change-Id: Ia72980d3f0f700386f62d301164f61f32405528e
2016-08-03 18:06:25 +09:00
Khai Do
b9a5d00e15 Add an optional flag to force an online reindex
As discussed on Gerrit ML[1] and in referenced issue, there are
situations where you may want to run an online reindex besides
the upgrade gerrit use case.  This change allows users to force an
online reindex.

[1] https://groups.google.com/d/msg/repo-discuss/pUn8fHVh58w/icsSFef2CAAJ

feature: issue 3987
Change-Id: Ie88b7effda08996f7c0f325543be703bcf09c794
2016-08-02 14:09:12 -07:00
Hugo Arès
41b4c0d3da Add extension point to register JGit PostUploadHooks
Plugins may register PostUploadHook instances in order to get notified
after JGit is done uploading a pack.

Now that PostUploadHook can be registered using the DynamicSet, use that
mechanism to register UploadPackMetricsHook.

Change-Id: If848bddc85ca8923a5d55f7dd448ac218c910e1c
2016-08-02 15:32:08 -04:00
Edwin Kempin
ece1d7db72 Merge "SSH show-queue: option to group output by queue and print queue info" 2016-07-14 07:15:15 +00:00
Saša Živkov
eaf64568a7 SSH show-queue: option to group output by queue and print queue info
When the new option --by-queue is used the output of the show-queue
command is grouped by the work queue. Queue name is printed on top of
the tasks list and the number of worker threads assigned to that thread
pool is printed in the summary line. This option should help Gerrit admins
to identify which thread pools need to be adjusted.

For comparison, here is an example output without the new --by-queue option:

  Task     State        StartTime         Command
  ------------------------------------------------------------------------------
  9d51b721              16:29:14.750      git-upload-pack '/gerrit' (admin)
  18fr4561              16:29:14.750      git-upload-pack '/gerrit' (admin)
  ad52b823              16:29:14.750      git-upload-pack '/gerrit' (admin)
  bd51bbcc              16:29:14.750      git-upload-pack '/gerrit' (admin)
  11223344              16:29:14.750      git-upload-pack '/gerrit' (admin)
  3d120b5c 17:28:20.169 16:28:20.169      Log File Compressor
  ------------------------------------------------------------------------------
    6 tasks

With the --by-queue option the output looks like:

  $ ssh -p29418 admin@localhost gerrit show-queue -w --by-queue
  Task     State        StartTime         Command
  ------------------------------------------------------------------------------
  Queue: SSH-Interactive-Worker
  9d51b721              16:29:14.750      git-upload-pack '/gerrit' (admin)
  18fr4561              16:29:14.750      git-upload-pack '/gerrit' (admin)
  ad52b823              16:29:14.750      git-upload-pack '/gerrit' (admin)
  bd51bbcc              16:29:14.750      git-upload-pack '/gerrit' (admin)
  11223344              16:29:14.750      git-upload-pack '/gerrit' (admin)
  ------------------------------------------------------------------------------
    5 tasks, 14 worker threads

  Queue: WorkQueue
  3d120b5c 17:28:20.169 16:28:20.169      Log File Compressor
  ------------------------------------------------------------------------------
    1 tasks, 1 worker threads

Change-Id: If82bdb50702f2001f887fe5004b0d80b61131a90
2016-07-13 18:24:55 +02:00
Dave Borowitz
d25fb550e4 Merge "Unwrap db in ssh.commands.QueryShell" 2016-07-12 16:15:13 +00:00
David Pursehouse
89634f70f3 ReviewInput: Split NotifyHandling out to a separate file
NotifyHandling is used for other purposes than adding a review, so
it doesn't really belong in ReviewInput.

Change-Id: I49ce1952eab0a75c8466a871fcc690a94b14e058
2016-07-12 17:24:35 +09:00
Björn Pedersen
484bf3cae1 Unwrap db in ssh.commands.QueryShell
If NoteDb reads are enabled then the gsql ssh commands fails with

fatal: internal server error

due to DisabledChangesReviewDbWrapper (see Issue 4247).

As not all tables are migrated yet, it makes sense to still allow
low-level access to the db.

Bug: Issue 4247
Change-Id: Icebe51f470010d913f387bb98f7319616dd89aba
2016-07-12 08:55:59 +02:00
Zhen Chen
10c815d2e1 Refactor ProjectControl#canReadCommmit to take repository as a parameter
Two reasons:
1. Reusing an existing RevWalk to call isMergedInto or similar on it can
corrupt the state in the caller, see I0d242362.
2. The repository is already open before calling this method.

Pass the Repo instead of RevWalk and create a new clean RevWalk from the
Repo.

Change-Id: I00a6d9061d4c690c236e739b29b66b79427cd47b
2016-07-11 12:37:23 -07:00
Eryk Szymanski
a26067f126 StreamEvents: Correctly serialize Project.NameKey elements in json
Add a ProjectNameKeySerializer so that the project attribute gets
serialized correctly in the json output, i.e.:

  "project":"project-name"

instead of:

  "project": { "name" : "project-name" }

Change-Id: I96dbb4c38f2da506165d5905a9edd8f01a87c899
Signed-off-by: Eryk Szymanski <eryksz@gmail.com>
2016-07-11 12:22:36 +09:00
Edwin Kempin
a62673ca77 Accounts API: Add methods to create an account
Change-Id: I798f98384fc513987c00aa1a6102c9967813b542
Signed-off-by: Edwin Kempin <ekempin@google.com>
2016-07-06 17:00:22 +02:00
Scott Dial
b4a04fa1c5 Add configuration of key exchange algorithms for sshd
Add new config key "sshd.kex". The default and supported values are:

  1. ecdh-sha2-nistp521
  2. ecdh-sha2-nistp384
  3. ecdh-sha2-nistp256
  4. diffie-hellman-group-exchange-sha256
  5. diffie-hellman-group-exchange-sha1,
  6. diffie-hellman-group14-sha1
  7. diffie-hellman-group1-sha1

With Bouncy Castle installed, all of the above are supported (previously
only 6 and 7). With JCE, only 7 is available.

Bug: Issue 3517
Change-Id: I6b44e88dc4a0ff8f693f21510aba30546bf4cd99
2016-07-05 15:29:04 +09:00
Edwin Kempin
b644ff8fcf Merge "Implement Bazel build" 2016-06-24 10:51:33 +00:00
Edwin Kempin
94732bfa19 Reindex account whenever account is evicted from cache
Change-Id: I025cabc9be98628777066cda7aa97186f5a0da15
Signed-off-by: Edwin Kempin <ekempin@google.com>
2016-06-22 16:55:32 +02:00
David Pursehouse
8219d818a7 PutConfig: move static inner Input class to extension API
Also rename the existing PutDescriptionInput to DescriptionInput.

Change-Id: I8291e970353b8ae6ed4f0cbe5dce746007e25291
2016-06-21 16:30:43 +09:00
David Ostrovsky
b81b4f75ae Implement Bazel build
To run the tests:

  bazel test //...

To build the Gerrit plugin API, run:

  bazel build gerrit-plugin-api:plugin-api_deploy.jar

To build the Gerrit extension API, run:

  bazel build gerrit-extension-api:extension-api_deploy.jar

TODOs:

Licenses
Reduce visibility (all public for now)
Generate HTML Documentation
Core plugins
gerrit_plugin() rule to build plugins in tree and standalone modes
GWT UI (only gwt_module() skylark rule is provided, no gwt_binary())
PolyGerrit UI
WAR
Publish artifacts to Maven Central
Ask Bazel team to add Gerrit to their CI on ci.bazel.io

Contributed-By: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I9a86e670882a44a5c966579cdeb8ed79b1590de3
2016-06-14 21:12:02 +02:00