36503 Commits

Author SHA1 Message Date
Matthias Sohn
79bec412a1 Update JGit to 5.1.13.202002110435-r
JGit 5.1.13.202002110435-r adds
- new WindowCache metrics
- and enables using strong references to reference cache pages in the
WindowCache. Set core.packedGitUseStrongRefs = true to enable using
strong references to reference packfile pages cached in the WindowCache.
If this option is set Java gc can no longer flush the WindowCache to
free memory if the used heap comes close to the maximum heap size. On
the other hand this provides more predictable performance.

Change-Id: I895369a8596dfc5adce76c65202ecf37e6a8386d
2020-02-14 13:38:59 +01:00
Matthias Sohn
32f2566275 BucketedCallback: fix prune() to remove unset sub-metrics from registry
BucketedCallback enables defining compound metrics having a set of
sub-metrics along a dimension. This is used e.g. in CallbackMetric1
where the additional type parameter F1 defines the type of this
dimension. For each of the values of this dimension present in the
current dataset of the compound metric BucketedCallback registers a
corresponding metric in DropWizard's metric registry which isn't aware
of the relationship between a compount metric in Gerrit and the
corresponding sub-metrics.

If another measurement yields a new metric dataset which no longer
contains some of the earlier registered sub-metrics doPrune() did remove
these stale sub-metrics from the "cells" collection but missed to remove
them from DropWizard's metric registry. This had the effect that such
stale metrics no longer present in the current dataset were still
reported via the used metrics reporter plugin albeit the underlying
monitoring dataset didn't contain that data anymore.

Fix this by removing the stale sub-metrics also from the DropWizard
metric registry.

Change-Id: I767530f57883379c976614705241cbd1d1597541
2020-02-14 13:38:59 +01:00
Youssef Elghareeb
e9ab0e6564 Merge "Modify draft ref updates commits to point to an empty parent" into stable-2.16 2020-02-14 11:53:14 +00:00
Youssef Elghareeb
4f77b29cb9 Modify draft ref updates commits to point to an empty parent
Draft comments refs are stored in the "All-Users" repo under a special
ref with the form refs/draft-comments/$change_id_short/$change/$user.

Previously, ref updates used fast forward updates for all operations
(add draft, delete draft, edit draft), and the new commit pointed to the
last commit of the ref. This behavior caused draft deletions to persist
in history.

In this change, the implementation of the change draft update is
modified so that new commits point to an empty parent list, causing
draft deletions to be garbage collected by git after a period of time.

Change-Id: Ifbb7ad2fe1b1370eac30feb3fe7b181cbb626257
2020-02-14 12:11:32 +01:00
David Pursehouse
03e0ff2a3e Merge "Fix Access when the only readable ref is a RegExp with a gerrit pattern" into stable-2.16 2020-02-13 22:14:35 +00:00
Guillaume Chauvel
80f75734e7 Fix Access when the only readable ref is a RegExp with a gerrit pattern
When no other ref than a RegExp with a gerrit username or shardeduserid
pattern, the permission is not properly validated and repository access
is denied.

With change Iedf3022f6c, calling shortestExample on regexp+parameters
refs produces a reference that won't match the username or the
shardeduserid.

For a reference like : ^refs/heads/users/${username}/.+
we will have the following:
'refs/heads/users/user1/my_branch' vs
'refs/heads/users/_PLACEHOLDER_/my_branch'

When we remove shortestExample, we have to modify Regexp and
ExpandParameters to skip the regexp start anchor caracter when comparing
against the prefix, or to compare regexps themselves.

Bug: Issue 3340
Change-Id: Id6a890a91c5d6d954dab74694c2d3fd53c15897c
2020-02-13 22:13:45 +00:00
Marco Miller
daf865d398 Merge "Fix refPattern when using username/shardeduserid pattern with regexp" into stable-2.16 2020-02-13 18:51:58 +00:00
Marco Miller
58b8f03671 Merge "Enable optional DEBUG level logs for query tests" into stable-2.16 2020-02-13 18:39:33 +00:00
Guillaume Chauvel
1b4b264adf Fix refPattern when using username/shardeduserid pattern with regexp
The special regexp caracters ('{', '}') from gerrit pattern are causing
an exception. As ${username} and ${shardeduserid} are only meaningful to
gerrit, we substitute them for placeholders.

Unlike what is done in RefPatternMatcher.ExpandParameters, the
placeholder does not contain ':' but '_' to be correctly evaluated by
Repository.isValidRefName later on.

Bug: Issue 3340
Change-Id: Iedf3022f6ce3068f7e9534f224cf05bb99b0f970
2020-02-13 10:39:33 +09:00
David Pursehouse
9a5141f0ad Consistently declare 'throws Exception' from @Test annotated methods
Change-Id: I1948607db496d0a9b8d933c1a870ecc97434dd13
2020-02-13 08:53:44 +09:00
Hamza Kaced
c54e6b5bfd Enable optional DEBUG level logs for query tests
Allow optionally enabling DEBUG level logs for Lucene and Elasticsearch
query tests. It occurs in the form of a command passed to the JVM before
tests start. Tests will only emit INFO level logs if not enabled.

Enable rootLogger default INFO log level after tests conclude.

Add missing log4j-config rule which includes log4j.properties in BUILD
files.

Update online Gerrit documentation to explain how to optionally enable
DEBUG log level for these specific tests.

Bug: Issue 10498
Change-Id: I75492de1c5ef8721ae745266e36b8cbfb0b67682
2020-02-12 11:58:05 -05:00
Edwin Kempin
d3f5bccb39 Merge "ProjectCacheImpl: Don't log cause when project is not found" into stable-2.16 2020-02-12 15:26:35 +00:00
David Pursehouse
3005e188c8 ProjectCacheImpl: Don't log cause when project is not found
This occurs very frequently and the cause is always the same, so
logging with the stack trace spams the logs making it difficult
to find stack traces that are caused by real issues.

Change-Id: I6acc32937ee647337337f99309947b0d414dffad
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 19:49:35 +09:00
Eryk Szymanski
936ed535a1 VersionedMetaData: Add debug log when saving metadata
Change-Id: I4ad514018bc54c99c48e6e9b6f2000ed61313f49
2020-02-12 19:48:59 +09:00
Tao Zhou
5a7a364ff2 Update web-component-tester to 6.5.1
This will fix the build failure as the sinon.js dependency issue
fixed in upstream (wct 6.5.1)

Change-Id: I871f08e59dcfc0717888aa9f0ade82fc3edc851d
2020-02-12 08:42:20 +01:00
Marco Miller
f386aa3de2 Add support for Elasticsearch version 7.6.*
- Upgrade elasticsearch-rest-client to 7.6.0.
- Add V7_6 to the version manager, and use this in all the V7 tests.
- Use the 7.6.0 image from blacktop in the test container; based on [1].
- Triggered by watching releases [2], which links to the release notes.

[1] https://github.com/blacktop/docker-elasticsearch-alpine/pull/27
[2] https://github.com/elastic/elasticsearch/releases/tag/v7.6.0

Change-Id: I63e6c396b8c159b4ebdfe9817518f239fd62928c
2020-02-12 12:02:02 +09:00
Matthias Sohn
73b8fd117c Write upload-pack metrics to sshd log
In order to enable detailed performance and problem analysis add
upload-pack metrics to the sshd log. The log format for upload-pack log
records is now enhanced to:
- timestamp
- session id
- thread name
- user name
- account id
- message
- command wait time
- command execution time
- time negotiating
- time searching for reuse
- time searching for sizes
- time counting
- time compressing
- time writing
- total time in UploadPack
- bitmap index misses (-1 means no bitmap index available)
- total deltas
- total objects
- total bytes transferred
- client agent

If statistics aren't available e.g. since an exception occurred they are
logged as -1.

Change-Id: I4ec08579cedfeb6a30eb41d2e2a110f4f8eee8fa
2020-02-12 00:34:34 +01:00
David Pursehouse
a13052bad8 Clarify documentation of container.javaOptions
The option may be specified multiple times, but this is not explicitly
clear in the documentation.

Change-Id: I33a6030b00a24f578c79cdd369bff246fd4470b0
2020-02-10 14:35:35 +09:00
David Pursehouse
a70a60cf1f Fix formatting of container.javaOptions section
Change-Id: I9e35dfc5b57bed862443e12fbeeb1e173e101bcd
2020-02-10 14:35:30 +09:00
David Ostrovsky
59ed05a6ed Run polylint tests with bazelisk if available
Change-Id: I342cfef83f812268f95dfb2cb30c52d63a31e622
2020-02-08 19:01:56 +01:00
David Pursehouse
f40935296e Bump required Bazel version to 2.1.0
Change-Id: Ic8662bd6f5a54a155a2ef07c1ea6c2b7371383b0
2020-02-08 10:55:06 +09:00
David Pursehouse
0e9fd39dc7 Bazel: Sort error prone check settings alphabetically
Change-Id: I044749d213e5a2b0b1a6c55c55515aa944e145ff
2020-02-07 13:21:56 +09:00
David Pursehouse
cd4e1d3fee Merge changes I6dd2035d,I6b6710d1,I43c5d7c6 into stable-2.16
* changes:
  Fix method name to register custom event types in plugins
  Don't send mails on publish change edits for WIP changes
  PatchSetInserter: allow to set "sendEmail" bit
2020-02-04 23:13:32 +00:00
David Ostrovsky
5edf70ec11 Fix method name to register custom event types in plugins
Change-Id: I6dd2035de403e3d6ce1d7c78634a3a0ffd8feff1
2020-02-04 22:58:29 +01:00
David Ostrovsky
b88351bc93 Don't send mails on publish change edits for WIP changes
Bug: Issue 10397
Change-Id: I6b6710d17138f277ac118924355311d192710f5c
2020-02-04 22:36:10 +01:00
Changcheng Xiao
47d74d2173 PatchSetInserter: allow to set "sendEmail" bit
With this new "sendEmail" bit, we will have more control
for whether emails are going to be sent out after a new
patch set is added. This is useful for avoiding duplicate
emails, e.g merge (already have an email to notify a change
is merged, no need to have another email to say a new patch
set is uploaded).

Change-Id: I43c5d7c6560316d5a6034dc8e711f7e87b7af932
2020-02-04 22:36:07 +01:00
Luca Milanesio
903e586b13 Set version to 2.16.17-SNAPSHOT
Change-Id: I0de189351cf696c14127f303bfbdcfcb9efe87c3
2020-02-04 16:27:38 +00:00
Luca Milanesio
dd780721ab Set version to 2.16.16
Change-Id: I3d0d1c1e89dd6bc2030056ab8a4ce09189403a06
v2.16.16
2020-02-04 16:01:59 +00:00
Saša Živkov
d4ba7fb691 Merge changes from topic "motd" into stable-2.16
* changes:
  Document MessageOfTheDay extension
  Add UI element to display messages of the day
  Add MessageOfTheDay-entries to ServerInfo
2020-02-03 14:15:54 +00:00
Sven Selberg
15753c282e Tests: Use helper method for config change
Use enableCreateNewChangeForAllNotInTarget() from AbstractDaemonTest
instead of duplicating the code.

Change-Id: Ic2fe137533a7b6ad33fc1705947473766730bf56
2020-02-03 11:49:13 +01:00
David Pursehouse
97331cc168 Merge "Generate Change-Ids randomly instead of computing them from commit content and timestamp" into stable-2.16 2020-02-03 01:59:16 +00:00
Edwin Kempin
79138831e5 Generate Change-Ids randomly instead of computing them from commit content and timestamp
If Change-Ids are computed from the commit content and timestamp it can
happen that 2 changes get the same Change-Id. E.g. this can happen if
the Create Change REST endpoint is used to create empty changes in
parallel. In this case the base commit, the commit content (empty) and
the commit timestamp are the same, so that the same Change-Id is
computed for several changes. This also means that the commit message
and hence the commit SHA1 can be identical between the changes. If in a
follow-up steps identical change edits are published in parallel on
these changes you can even get non-empty changes for the same commit
SHA1. Having multiple changes for the same commit SHA1 (and with the
same target branch) is an inconsistency in Gerrit that Gerrit cannot
deal well with (e.g. this causes issues during submit).

To prevent such duplicate Change-Ids we are now randomly generating
Change-Ids. This means 2 parallel Create Changes requests will use
different Change-Ids even if the commit content and timestamp is the
same. A new test does parallel calls to Create Change REST endpoint and
verifies that commit SHA1's of the created changes differ (they differ
because in the commit message we have different Change-Ids now). When
the Change-Ids were computed from the commit content and timestamp this
test failed.

Also cherry-pick Change-Id: I9de36332 for keeping the secure random
generation out of the GWT-related code and minimise the conflicts
with the merge upstream.

Bug: Issue 12246
Co-authored-by: Luca Milanesio <luca.milanesio@gmail.com>
Change-Id: Ie4f01e5a32b1efca58f8b443ff046623f0e5f188
(cherry picked from commit 73163490b75a8d39aaa2eb60ba2de0cb27f6ba70)
2020-02-02 22:47:20 +00:00
David Ostrovsky
11bce058a0 Highlight WORKSPACE as python
Change-Id: I2f46d6de7989f3951fcea49475c52ea216b25c94
2020-02-02 10:30:18 +01:00
David Ostrovsky
827c929cd5 Highlight BUILD.bazel as python
Bazel supports both BUILD and BUILD.bazel files and in this issue: [1]
there is a discussion to prefer BUILD.bazel over BUILD naming
convention. Detect BUILD.bazel files as Python code in addition to
BUILD files.

[1] https://github.com/bazelbuild/bazel/issues/4517

Change-Id: Iabc719b914fcd08ef6e73c314f09f924540c3794
2020-02-02 10:25:53 +01:00
Matthias Sohn
db1e78649e Move documentation for Jetty metrics to a separate section
Change-Id: Ibdb1516b091c81203c282384dbe8aa2276c838ff
2020-01-31 14:17:10 +01:00
David Pursehouse
c733e8aa38 Merge "Improve Jetty thread pool metrics" into stable-2.16 2020-01-31 12:58:49 +00:00
David Pursehouse
3b569a3fcb Merge "NoteDbMigrator: Totally skip migration for orphan changes" into stable-2.16 2020-01-31 10:15:04 +00:00
Thomas Draebing
cfff720610 Document MessageOfTheDay extension
Change-Id: I99894e9d9b0376e11ac47796cee4c9e2e0c9be5a
2020-01-31 10:45:23 +01:00
Matthias Sohn
ae4a73b87c Improve Jetty thread pool metrics
Follow-up change for be4184f7 which missed documentation. Also names of
newly added metrics didn't align with existing metrics for REST API.

- add documentation for the jetty metrics
- improve descriptions of the jetty metrics
- rename jetty thread pool metrics from httpd/jetty/* to
  http/server/jetty/* in order to align metric names with already
  existing REST API metrics

Change-Id: I1d4eabec91749f28d09df7d2d29606b133fefc4d
2020-01-31 10:42:21 +01:00
Thomas Draebing
1fbf53df60 Add UI element to display messages of the day
This change adds an UI element that displays messages registered with
the MessageOfTheDay extension point. The messages will be displayed
below the main navigation header. This functionality was already part
of the GWT-UI, but was never migrated to the new UI.

The messages can be dismissed by a button click, that will create a
cookie that will prevent the message from being displayed again until
the redisplay time configured with the message.

The content of the message can be HTML. Depending on where the message-
content is coming from, this might be harmful. In case of the
messageoftheday plugin, the message is added as a config from the Gerrit
server site, which can probably be considered save. However, in a future
change rendering HTML should be made configurable and switched off by
default. Text should then be rendered using `gr-formatted-text`.

Change-Id: I2cbbcbcd49734b645701b9fac62a016a269d37f2
2020-01-31 10:14:27 +01:00
Thomas Draebing
c77032cc74 Add MessageOfTheDay-entries to ServerInfo
The message of the day functionality was lost in the new PolyGerrit-UI,
since the original implementation relied on the server side addition of
the messages to the host page.

This change adds the messages registered to the MessageOfTheDay-
extension point to the ServerInfo, which can be fetched via Rest API.
This allows accessing this information from the new UI.

The UI elements that consume this information will be added in a
followup change.

Change-Id: I1deff5afbd812a1efaf6ad11e871544b36a5d88d
2020-01-31 10:14:23 +01:00
Thomas Draebing
5f948ecf1d Fix commentlinks with same prefix in pattern and link
If using the commentlinks.%NAME%.links option and having the same prefix
in the pattern and the url, the parsing did not work properly.
For example:

  [commentlink "test"]
    pattern = [Hh][Tt][Tt][Pp]example
    link = http://example.com

and the line in the message:

  httpexample 1234

would result in:

  http<a href="://example.com" ...>httpexample 1234</a>

This bug was introduced in a change implementing the possibility to
have overlapping patterns, e.g. allowing a comma-separated list of
issue ids. (commit: fb48902ef303d008c6f0dd38d654106c00313530). This
would have however anyway only worked for the `html`-variant of
comment links, which allow to directly provide html-elements as a
substitute for the text matching the pattern.

Thus, this change restricts the logic introduced in the above mentioned
commit to the html-variant.

Bug: Issue 12197
Change-Id: I9efd9308f2b40f3fec8405c93849d13d4576562a
2020-01-31 08:58:08 +00:00
David Pursehouse
720d0af16d Use bugs.chromium.org in gr-linked-text_test
The tests are using code.google.com issue tracker links but this
tracker has not been used for ages. Replace them with links to the
new Monorail tracker on chromium.org.

This makes no functional difference in the tests, it is just a
cosmetic change.

Change-Id: I33fbc1cdc1f28a585f4c6c12a8bbd88568a20fc4
2020-01-31 10:37:23 +09:00
David Pursehouse
ebb7a1a844 Merge "Fix support for python3 in download_bower.py" into stable-2.16 2020-01-31 01:02:45 +00:00
David Pursehouse
d3943129a2 Merge "Expose jetty threadpool metrics via dropwizard" into stable-2.16 2020-01-31 01:02:04 +00:00
David Pursehouse
022defe870 Merge changes Ic4e04936,I6462d87c into stable-2.16
* changes:
  Fix editing name & email for service user
  Fix accountBelongsToRealm implementation
2020-01-31 00:52:44 +00:00
Matthias Sohn
be4184f753 Expose jetty threadpool metrics via dropwizard
Change-Id: I91496c38683c4b41f11978d832eaacd1f60adcf0
2020-01-30 17:34:15 +01:00
David Pursehouse
fa51424daa Merge branch 'stable-2.15' into stable-2.16
* stable-2.15:
  Improve log if VersionedMetaData fails to update

Change-Id: Ic2635cf7319276dbf32d18c2c690bb3f38d0a3e7
2020-01-30 10:30:17 +09:00
Eryk Szymanski
8bf1d99854 Improve log if VersionedMetaData fails to update
In case of lock failure there is no hint on what commit has caused it.

This change adds ref log message, which is "commit:" + subject line
of commit message or "commit: meta data update" in case of no commit
message, to the exception message.

Also extract the exception message creation to a separate method, so
that the same info is available when update fails for other reasons.

Change-Id: I5686e079853807ba88dae6d8ccf184eb81b50013
2020-01-30 10:02:48 +09:00
Francois Ferrand
a3db8e117d Fix editing name & email for service user
A service-user (either created by SSH command or through ServiceUser
plugin) is actually not bound to the realm : so it makes no sense to
check if the realm supports editing FULL_USERNAME or EMAIL.

To overcome, we now check realm capabilities only for users which belong
to that realm.

Change-Id: Ic4e04936a5d2444bf6b4644a2f8a3838a2686256
2020-01-29 13:42:58 +01:00