* stable-2.16:
Switch PatchListCache to using legacy cache backend
Replace guava caches with caffeine
Add caffeine and caffeine-guava to Documentation/licenses.txt
which did not exist on earlier branches.
Change-Id: I40fb2798a8b6eda2e92e4409c1e57042a8e59ef4
Replacing Guava caches with Caffeine reduces the chances of having the
deadlocks and improves the cache performance.
This was already attempted in: I8d2b17a94d0, but got reverted in:
If65560b4a9b due to recursion in PatchListLoader. This recursion issue
is present on current master. While this change replaces all caches
with Caffeine backend, the follow-up change in this series will switch
back to using Guava backend for PatchListCache implementation.
For seamless integration, the caffeine-guava adapter library is used.
Given that the final artifact for the adapter is also called guava,
there is only the version number that differentiates that artifact from
the guava library itself so that we have a danger for naming collision.
To avoid potential naming collision risk, rename the library name to
caffeine-guava.jar during the fetch from Maven Central.
Alternatives considered is not to use the caffeine-guava adapter
library. But then the Cache and LoadingCache classes and friends would
change the package name from com.google.common.cache package to
com.github.benmanes.caffeine.cache package and this change would also
affect some gerrit plugins and thus considered to be a quite intrusive
change. Still we can consider to do this change in one of the future
gerrit releases.
Bug: Issue 7645
Bug: Issue 11484
Change-Id: I6af4c15d6c15f438becd62409b7d233c309be8de
(cherry picked from commit 0050a9b5f61da67a9f33fda0bb170f8b1df6080a)
* stable-2.16:
Load proto_library from @rules_proto//proto:defs.bzl
Update git submodules
Bazel: Add fixes for --incompatible_load_{java|python}_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Lucene index configuration and docs.
Change-Id: I401c192096047dd5069be37f17c4a62a8ed3afb7
* stable-2.15:
Bazel: Add fixes for --incompatible_load_{java|python}_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Lucene index configuration and docs.
Change-Id: I6c597cbc89fafece83c374e9b36c4c4c0126704f
This change is fixing "All Java build rules should be loaded from
Starlark" warning flagged by latest buildifier version: [1]. Python
rules are now also loaded from the Starlark.
Also extract codemirror library import to BUILD file. This is needed to
avoid cycle in the workspace file, after importing java rules from
Starlark.
[1] https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#native-java
Change-Id: I36192c9465d988b25cf09c250e110f15850910cd
* stable-2.16:
Error Prone: Enable DateFormatConstant at ERROR severity
Synchronize protobuf with version used in rules_closure
Update rules_closure to latest version
Upgrade elasticsearch-rest-client to 7.1.1
Change-Id: I00004567e675b67f498a2edfb93951a5aab140f5
* stable-2.15:
Synchronize protobuf with version used in rules_closure
Update rules_closure to latest version
Upgrade elasticsearch-rest-client to 7.1.1
Change-Id: If4cdcc44c53e90f3562bdff1b7a30dd09cd9764f
Avoid fetching a potentially different version of protobuf-java and use
version that rules_closure transitively depends on.
Change-Id: Ief63f06cb531cd76e2bcf8c5623d653a5c1b69c7
* stable-2.16:
lib: check that nongoogle.bzl doesn't grow new dependencies
tools/nongoogle.bzl: separate out dependencies exempt from library compliance
Change-Id: Idd5113d6ecc312a67f5390541abe5de715ec2e9b
This is the minimal set of files from gwtorm that are still needed by
Gerrit core. The long term plan is to migrate away from these, for
example by revamping our exception hierarchy (I9c8fac5f) and converting
the legacy mutable gwtorm-based types to immutable AutoValues.
In the short term, copying these files into core allows us to remove a
bunch of unused class files from the war, and prevents new dependencies
from creeping in. This change should be viewed as a starting point of
the larger project to eliminate gwtorm, not an end in itself.
Files copied from:
https://gerrit.googlesource.com/gwtorm/+/de62a8902ade3239e4a2db105bb24e4561e34df3
The whole gwtorm repo is under the same AOSP umbrella as Gerrit core, so
it's all under the same CLA and can be copied between repos on the
Gerrit hosts without any change to the license.
Change-Id: I0f09ed399180bc132fb0a28be07944b4aa9fe4a0
We no longer use GWT at all so this dep is not really appropriate. All
necessary code has been copied into Gerrit core.
Change-Id: I5e81f0c8fc37710101395301706bc7c3e09eeaec
ReviewDb is gone and the PostgreSQL lib is no longer needed.
Note that there is a PostgreSQL implementation of
AccountPatchReviewStore but this implementation is based on JDBC and
doesn't use the PostgreSQL lib.
Change-Id: I72a65efc99abbc6baa7ffc7dabe62b312a30e84c
Signed-off-by: Edwin Kempin <ekempin@google.com>
This upgrade fixes CVE-2018-10237 [1]:
Unbounded memory allocation in Google Guava 11.0 through 24.x before
24.1.1 allows remote attackers to conduct denial of service attacks
against servers that depend on this library and deserialize attacker-
provided data, because the AtomicDoubleArray class (when serialized
with Java serialization) and the CompoundOrdering class (when
serialized with GWT serialization) perform eager allocation without
appropriate checks on what a client has sent and whether the data size
is reasonable.
[1] https://nvd.nist.gov/vuln/detail/CVE-2018-10237
This also adds dependency on j2objc-annotations to prevent the following
warning during the build:
INFO: From Building java/com/google/gerrit/lucene/liblucene.jar (12 source files):
warning: unknown enum constant ReflectionSupport$Level.FULL
reason: class file for com.google.j2objc.annotations.ReflectionSupport$Level not found
Bug: Issue 9952
Change-Id: Iea79ee7d93c4b7c85479b5ec01ee07e19beed611
Bazel correctly emits warning when the classpath contains a reference to
com.google.j2objc.annotations.ReflectionSupport, which isn't on the
classpath:
warning: unknown enum constant ReflectionSupport$Level.FULL
reason: class file for com.google.j2objc.annotations.ReflectionSupport$Level not found
See this issue for more details: [1].
[1] https://github.com/bazelbuild/bazel/issues/6414
Bug: Issue 9779
Change-Id: I313549d5cb219cb144087dcdc85091f21faa6e94
There is pending upgrade for review for years now to version 2.33: [1],
that appears to be too introusive change. There are two issues:
* Collision with rules_closure's and closure-templates's args4j version
* Non trivial changes in gerrit core to adapt to newer args4j version
Dismantle those issues, by upgrading to intermediate version: 2.0.29.
Re-fetch args4j twice: one for rules_closure to work properly with
ancient args4j version, while renaming the internal one to the more
recent version: args4j-intern.
Needless to say, that we have uploaded these PRs to both projects:
rules_closure and closure-templates that were more or less rejected,
so that we don't hope that args4j is going to be upgraded upstream,
thus keeping this status quo of double fetching: [2],[3].
The reason we need this change is dependency of gitblit-plugin on more
recent version so that we are getting classpath collision. Another
alternative that was considered (but abandoned) is to downgrade args4j
version in gitblit to the same ancient version used by gerrit core.
[1] https://gerrit-review.googlesource.com/c/gerrit/+/67793
[2] https://github.com/bazelbuild/rules_closure/pull/262
[3] https://github.com/google/closure-templates/pull/155
Change-Id: I5ec6645b92d54fccc03a723b7b50c50a99393d57
The new version provides better support for Java 9 and improves
performance. Multibindings artifact is now included in the guice
core artifact. The full release notes is here: [1].
[1] https://github.com/google/guice/wiki/Guice42
Change-Id: I208d53ce1cf9b99ed8f03ae34470e95c3b8f71ce
(cherry picked from commit 1fa7774d237d0ea864b9f03b6551fd1ebef0c619)
All the commons-* libraries are defined in lib/commons/BUILD with
the exception of commons-io which is in lib/BUILD.
Move it to lib/commons/BUILD to make it consistent.
Change-Id: If2541eba2a4a9e9b5e7a777940bfce923478808b
Recent Bazel versions support dash character in external repository
names. Consistently use them with one exception: javax_inject. This is
needed to match the name in the rules_closure.
Change-Id: I1e75690fe1ee2ab32fffe07c0c30dbed84753960
Recent Bazel versions support dash character in external repository
names. Consistently use them with one exception: javax_inject. This is
needed to match the name in the rules_closure.
Change-Id: I1e75690fe1ee2ab32fffe07c0c30dbed84753960
The new version provides better support for Java 9 and improves
performance. Multibindings artifact in now included in the guice
core artifact. The full release notes is here: [1].
[1] https://github.com/google/guice/wiki/Guice42
Change-Id: I208d53ce1cf9b99ed8f03ae34470e95c3b8f71ce
This reverts commit 00fc15ac0073b86270e7c0f40d386f95dfe31e86.
With Caffeine, accessing the cache from the cache loader is causing an
infinite loop [1]. Merging that change up to stable-2.15 exposed that
there is still at least one cache loader (PatchListLoader) accessing its
cache (PatchListCacheImpl).
The test going into an infinite loop on stable-2.15 is
RevisionDiffIT.rebaseHunksOneLineApartFromRegularHunkAreIdentified.
Revert the change until circular dependency is removed or a solution is
found to access the cache from the cache loader.
[1]https://github.com/ben-manes/caffeine/issues/89
Bug: Issue 8464
Change-Id: If65560b4a9bfcf0a03decaedd83ad000c6b28f4f
This deadlock is not the typical deadlock where 2 threads locked a
resource and each one is waiting to lock a second resource already
locked by the other thread. The thread owning the account cache lock is
parked, which tell us that the locked was not released. I could not
determine the exact sequence of events leading to this deadlock making
it really hard to report/fix the problem.
While investigating, I realized that there quite a few reported issues
in Guava that could be major for Gerrit:
Our deadlock happening in account cache
https://bugs.chromium.org/p/gerrit/issues/detail?id=7645
Other deadlock
https://github.com/google/guava/issues/2976https://github.com/google/guava/issues/2863
Performance
https://github.com/google/guava/issues/2063
Race condition
https://github.com/google/guava/issues/2971
Because I could not reproduce the deadlock in a dev environment or in
a unit test making it almost impossible to fix, I considered other
options such as replacing Guava by something else.
The maintainer of Caffeine[1] cache claims that Caffeine is a high
performance[2], near optimal caching library designed/implemented base
on the experience of designing Guava's cache and ConcurrentLinkedHashMap.
I also did some benchmarks about spawning a lot of threads reading/writing
values from the caches. I ran those benchmarks on both Guava and Caffeine
and Guava was always taking at least double the time than Caffeine to
complete all operations.
Migrating to Caffeine is almost a drop-in replacement. Caffeine
interface are very similar to Guava cache and there is an adapter to
migrate to Caffeine and keep using Guava's interfaces. After migrating
to Caffeine, we saw that deadlock occurrence was reduced from once a day
to once every 2 weeks in our production server.
The maintainer of Caffeine, Ben Manes pointed us to the possible
cause[3] of this issue, a bug[4] in the kernel and its fix[5]. Our
kernel version is supposed to have the fix but we will try another OS
and kernel version.
Replacing Guava caches by Caffeine brings 2 things, it reduces the
chances of having the deadlock most likely caused by a kernel bug and
improve the cache performance.
[1]https://github.com/ben-manes/caffeine
[2]https://github.com/ben-manes/caffeine/wiki/Benchmarks
[3]https://groups.google.com/forum/#!topic/mechanical-sympathy/QbmpZxp6C64
[4]b0c29f79ec
[5]76835b0ebf
Bug: Issue 7645
Change-Id: I8d2b17a94d0e9daf9fa9cdda563316c5768b29ae
Remove the dependency on the velocity template library, and all code
related to reading and parsing velocity templates.
With this change, Gerrit no longer recognizes or handles Velocity
templates (*.vm) for emails. Site administrators must replace any
Velocity templates with the equivilant Soy templates before upgrading
to a Gerrit version that includes this change.
Update the mail config documentation to clarify that Velocity is no
longer supported.
Change-Id: I9942eb1a94adfa677c5a25634f3a1fee68c680fa
New release depends on safe-html-types that is released under Apache 2
license: [1].
[1] https://github.com/google/safe-html-types
Change-Id: If46fcf6dd2e7ad7e2c6eac0906e5df0fa401b6cc
This was presumably fixed in Ia5e6b9791 and in I8b735db4f. It turns
out, that labels map is not null in GWT UI, as it is the case in PG,
but also non empty, even in non-voting case: Code-Review: 0.
Fix the optimization check to account for zero votes.
Bug: Issue 4638
Change-Id: I6d9a2cc42ec51e6b1df13b96cf4bcdd082c87f60
Reformat the Bazel build files with the buildifier tool [1].
The style is different for Bazel files. Most notably, indentation level
is 4 spaces instead of 2, and " is used instead of '.
[1] https://github.com/bazelbuild/buildifier
Change-Id: I95c0c6f11b6d76572797853b4ebb5cee5ebd3c98
This lets us associate the jar and the src jar in the java_import()
declaration, so IDE plugins could understand how artifacts and source
are connected.
Classpath generation for Eclipse IDE was adjusted to reflect the changed
location of the source artifact.
Change-Id: I2cf9c6db3ad4b648457f2ac8d380c3771b4a3ee2
Using existing download_file.py from custom maven_jar Skylark rule
gives us the following advantages:
* fetching of sources (needed for GWT JSNI build and IDE integration)
* server support for proxies
* global, per-user cache of artifacts
* support for local maven repository (~/.m2)
Change-Id: Ic246dd9298775a6819f05453c601c688d8cb7ab9
Modify license_map rule to:
- Accept more than 1 targets
- Add '--asciidoctor' option to generate asciidoctor txt file
- Add support for multiple licenses for a single target (diffy_logo)
Also add test_license.sh for license_test rule to handle special
DO_NOT_DISTRIBUTE exceptions.
TESTED:
bazel build Documentation:licenses.txt
bazel test gerrit-pgm:pgm_license_test
bazel test gerrit-gwtui:ui_module_license_test
Change-Id: Ic64b227fea34882721e6e064b1520cd9a4d5d4a4
Example:
$ bazel build //Documentation:pgm-licenses.txt
..
$ cat bazel-genfiles/Documentation/pgm-licenses.txt
//lib:jsch //lib:LICENSE-jsch
//lib:servlet-api-3_1 //lib:LICENSE-Apache2.0
Still missing:
* check that there are dependencies on forbidden licenses
* check that there are no external dependencies except through the
//lib directory.
Change-Id: I8f0ed94de20f68e0dbc1fe87dfd56ce11e02fab7
After adding the soy lib to buck, it needs to be added to bazel as well.
Tested with:
bazel build //...
Change-Id: Ibc65a3d2d37a353a73a3f1e546dfc974b6e0b3c0
To run the tests:
bazel test //...
To build the Gerrit plugin API, run:
bazel build gerrit-plugin-api:plugin-api_deploy.jar
To build the Gerrit extension API, run:
bazel build gerrit-extension-api:extension-api_deploy.jar
TODOs:
Licenses
Reduce visibility (all public for now)
Generate HTML Documentation
Core plugins
gerrit_plugin() rule to build plugins in tree and standalone modes
GWT UI (only gwt_module() skylark rule is provided, no gwt_binary())
PolyGerrit UI
WAR
Publish artifacts to Maven Central
Ask Bazel team to add Gerrit to their CI on ci.bazel.io
Contributed-By: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I9a86e670882a44a5c966579cdeb8ed79b1590de3
