This is a recent snapshot built off JGit's master branch, including
the fixes that went into 4.1.2.201602141800-r which we have already
upgraded to on stable-2.12.
Change-Id: I9cb311906957f4cbc00e0d168abfe512c070dca1
I39f2d5d7 isolated jgit in its own cell, that is based on this JGit
Buck build implementation: [1]. Migration was done seamlessly, meant
that single BUCK file in lib/jgit represents JGit cell root location.
However, the real structure of JGit project is divided to number of
different sub-projects. To map between simplified JGit cell in gerrit
and real JGit project structure in JGit project, java_library() rules
were added to root BUCK file in JGit project that work like proxy to
real rules located in JGit sub-projects. For example //:jgit in JGit
tree was implemented as:
java_library(
name = 'jgit',
exported_deps = ['//org.eclipse.jgit:jgit'],
visibility = ['PUBLIC'],
)
Such proxies are needed for every artifact that is referenced from
gerrit build and make Buck build implementation unnecessary verbose.
Moreover this introduced some subtle issues, like using JGit
dependencies in context of java_doc rules, where $(location :foo) macro
is unable to resolve the underlying files because java_library with
exported dependencies only do not have association with output file.
An attempt to replace java_library with only exported dependenies with
prebuilt_jar: [2] that depends on the real artifact introduced another
problem with assembly of gerrit.war, because now jgit.jar is twice in
the classpath (because prebuilt_jar has output file association). To
fix this we would need to filter potential duplicates in the assembly
process of gerrit.war.
Instead of using proxy approach and to try to provide yet another
workaround to subtle problems, emulate the JGit project structure and
reference directly the same artifacts paths within gerrit JGit cell
in gerrit build:
deps = [
'@jgit//org.eclipse.jgit:jgit',
],
This simplifies JGit Buck build implementation, as we wouldn't need to
proxy all artifacts referenced from gerrit build from the root build
file. And this would fix all remaining issues.
This approach make gerrit build slightly more verbose. JGit upgrade
process would need to touch 4 files instead of only one. But given that
the Gerrit/JGit development integration is important feature, we would
like to support (as this integration attempt shows: [3]) in our build
toolchain, this overhead is justified.
With this change, the root build file in JGit project can be stripped:
[4].
[1] https://git.eclipse.org/r/61938
[2] https://git.eclipse.org/r/66547
[3] https://gerrit-review.googlesource.com/61892
[4] https://git.eclipse.org/r/66562
Change-Id: I2d278f80d0fedc4c5e9943804873f57145877dfe
Consume JGit as first third party library from its own cell. Normally
the cell is defined as lib/jgit directory. It can be easily replaced
with CLI:
buck build --config repositories.jgit=path/to/dev/jgit gerrit
or tweaking the .buckconfig:
[repositories]
jgit = path/to/dev/jgit
The former approach is sufficient to build and run the test from the
CLI, the latter is needed to generate eclipse project.
To isolate the JGit rules in its own cell some refactoring was needed.
JGit patch for GWT module was moved to gerrit-patch-jgit project, and
some symlinks were needed for maven machinery to work. include_defs()
doesn't work for now across cell boundaries, and native `buck fetch`
feature still has some limitations: [1]. Moreover, excluding paths,
unsigning JARs and license linking should be re-implemented on top
of it.
[1] https://github.com/facebook/buck/issues/602
Test Plan:
Normal gerrit build and the build with hijacked JGit cell should work
in both standalone (gerrit.war) and Eclipse environment. Note, that
to test --config repositories.jgit=path/to/dev/jgit use case, the most
recent JGit tree must be used, that contains Buck driven build
implementation.
Change-Id: I39f2d5d75bbac88804406d6242b5e714f4916926
This includes a new method on Repository, so our subclass needs to be
updated.
Also requires disabling atomic updates on InMemoryRepositoryMangers
used by acceptance tests, since a bug in the atomic implementation
causes BanCommitIT to fail:
FAILURE com.google.gerrit.acceptance.rest.project.BanCommitIT banCommit: Not true that null reference starts with <"contains banned commit">
java.lang.AssertionError: Not true that null reference starts with <"contains banned commit">
at com.google.common.truth.FailureStrategy.fail(FailureStrategy.java:24)
at com.google.common.truth.FailureStrategy.fail(FailureStrategy.java:20)
at com.google.common.truth.Subject.failWithRawMessage(Subject.java:381)
at com.google.common.truth.StringSubject.startsWith(StringSubject.java:167)
at com.google.gerrit.acceptance.rest.project.BanCommitIT.banCommit(BanCommitIT.java:49)
This is because ReceiveCommits rejects the command but still executes
the batch, and the relevant part of InMemoryRepository doesn't ensure
updates are NOT_ATTEMPTED before trying to apply them.
Change-Id: Icd2485c51723119c897406806e2f6e086831df3e
This release fixes the bug
https://bugs.eclipse.org/bugs/show_bug.cgi?id=478865
"exactRef behaves differently in InMemoryRepository when HEAD is linked
to a non-existent ref"
Change-Id: Iced94eeec58f51b981821d55c65b5c1a2ad26db4
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Includes Jacob Keller's fix to show submodule differences properly[1].
See the full release notes[2] for further details.
v2.12-rc0~528^2 (Use exactRef() when possible, 2015-06-20) changed the
expectation in ListBranchesIt#listBranchesOfEmptyProject to work
around a bug in exactRef where it would treat a symref pointing to an
unknown branch as missing. That bug has been fixed in both ref
database backends in JGit:
in DfsRepository, d0e47a99 (2015-07-16)
in FileRepository, 797f94d3 (2015-11-11)
(more details at https://bugs.eclipse.org/478865). So add HEAD back
to the expected list of branches.
[1] https://git.eclipse.org/r/#/c/56263/
[2] https://projects.eclipse.org/projects/technology.jgit/releases/4.1.0
Change-Id: I97f3be172de8cdd8ecf6a3eacb037d4a6332b50f
Includes the fix in 4.0.1.201506240215-r.65 release that was already
merged on stable-2.11:
6b65adc Add a grace period for packfiles during GC
Change-Id: I53692783e3f78931417e6b37af34f3aefadeb1f7
This should only appear in the server deps.
For UI code build //lib/jgit:edit_lib which
has only the tiny slice of code used in the UI.
Change-Id: Ia46c0559a7180f80fda30efb8884ca236997c92d
This JGit version includes the bugfix [1] which is an attempt to fix the
"Cannot read project" issue in Gerrit, as discussed in [2] and [3].
This version of JGit also removes the 'release()' method in many
interfaces/classes in favor or the implementing the AutoCloseable
interface. In stable-2.10 we just replace all usages of the release()
method with the close() method. Refactoring the code to make use of the
AutoCloseable in stable-2.10 would be a larger change which wouldn't
justify itself as we don't expect any major development in stable-2.10
and the usage of AutoCloseable in the master branch is already done.
[1] https://git.eclipse.org/r/48288
[2] https://groups.google.com/forum/#!topic/repo-discuss/ZeGWPyyJlrM
[3] https://groups.google.com/forum/#!topic/repo-discuss/CYYoHfDxCfA
Change-Id: Ie540296238e3bbaf453c9e29426825431e15d423
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
18c4ccd2c3 changed JGit version from the 3.7.0.201502260915-r.58-g65c379e
to the 3.7.1.201504261725-r. However, except for the one new commit
which the 3.7.1.201504261725-r brought, this was effectively a JGit
downgrade.
We need to upgrade JGit to a version which contains the fix
for the [1] and is a successor of the snapshot version
3.7.0.201502260915-r.58-g65c379e which was used in the 2.10.3.1.
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=465509
Change-Id: I7b5f21700c6cda20b000e1e55266015f081b66bf
This version fixed JGit regression, causing severe (>10x)
performance degradation on huge repositories (>2GB) on git
push and CPU consumption explosion during replication: [1].
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=465509
Bug: Issue 3300
Change-Id: I6b1fa985fa3738801d3fa27d690a1c02c1afc1db
This version of JGit logs IOExceptions caught while accessing pack files
and only removes affected packs from the pack list if we know that the
pack is corrupt. Other IOExceptions could be transient hence JGit
doesn't remove the affected pack from the list anymore to avoid the
problem reported on the Gerrit list [1]. It looks like in the reported
case the pack was removed from the pack list causing
MissingObjectExceptions which disappear when the server is restarted.
[1] https://groups.google.com/forum/#!topic/repo-discuss/Qdmbl-YZ4NU
Bug: issue 3094
Change-Id: I3cf36e1c2000f42652053ada712eccb955e99390
This JGit version fixes:
- Bug 420915 - jgit gc hangs in partitionTasks with a very small repo
- Bug 427107 - cannot push anymore
The latter was observed by CollabNet to break Gerrit replication if gc
created a bitmap index which may have induced PackWriterBitmapWalker.
findObjects() to throw a MissingObjectException.
This version of JGit also fixes the recursive merger on all storage
systems. Objects created during the virtual base construction of a
recursive merge must be written out somewhere and made available
through an ObjectReader for later passes to work on.
In both local filesystem and DFS implementations Gerrit was no-op'ing
the inserter in dry-run mode, causing these objects to be lost and
unavailable during the later processing stages of the merger. With a
virtual common ancestor tree or blob missing, the dry-run merger fails
and a spurious merge conflict is reported.
Instead build a non-flushing inserter wrapper around a real inserter
for the repository. On local disk (standard storage) this will allow
the virtual base to write loose objects, which may be reclaimed in
about two weeks by the standard `git prune` invoked by `git gc`.
On DFS systems this will create a new pack file and buffer a block of
data in memory before starting to store to persistent storage.
However with no flush() the DfsInserter will attempt to rollback the
pack, which may allow the DFS system to reclaim its storage quickly.
Some implementations of DFS may buffer even more deeply than one
block, making the discard even cheaper for smaller merges.
This update also fixes a potential infinite loop during object
inflation within both the WindowCursor or DfsReader versions of
ObjectReader. Inflation could get stuck if an object's compression
stream within a pack ended at a very precise alignment with the cache
block size. The alignment problem is very rare, as it has taken
several years to identify and track down.
Includes changes done in I9859bd073bd710424e12b8b091abb8278f4f9fcc
on master.
Change-Id: I898ad7d5e836ebae0f8f84b17d0ae74489479ef9
Since JGit 3.1 archive command was implemented. Add it to download
drop down as new line.
The following libraries are introduced in this change:
* jgit-archive (Apache 2)
* commons-compress (Apache 2)
* tukaani-xz (Public domain)
Change-Id: I5f61aac8c434414c73585a9320e84f4430dd111d
This fixes a long standing bug in the copy and rename detection
code where a copied file that is also updating its mode loses its
copy status and breaks the diff display.
Change-Id: I15c4d61ff1489e2c9b17e3b9ca76d9dcc98e26a8
After switching to Eclipse Maven repository for pulling JGit lib, we
have the problem that according to Eclipse release train the JARs have
to be signed. That collids with our jgit patch for diff deserialization.
To rectify that, add `unsign` parameter to maven_jar() function.
Change-Id: Ib7bfa5d16f980a64b887d61a4b4ec325e6ffb0a1
Implement a new build system using Buck[1], Facebook's
open source clone of Google's internal build system.
Pros:
- Concise build language
- Test and build output is concise
- Test failures and stack traces show on terminal
- Reliable incrementals; clean is unnecessary
- Extensible with simple blocks of Python
- Fast
buck: clean: 0.452s, full 1m21.083s [*], no-op: 7.145s,
mvn: clean: 4.596s, full 2m53.776s, no-op: 59.108s,
[*] full build includes downloading all dependencies,
time can vary due to remote server performance.
Cons:
- No Windows support
- No native Maven Central support (added by macros)
- No native GWT, Prolog, or WAR support (added by macros)
- Bootstrap of buck requires Ant
Getting started:
git clone https://gerrit.googlesource.com/buck
cd buck
ant
Mac OS X:
PATH="`pwd`/bin:/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands:$PATH"
Linux:
PATH="`pwd`/bin:$PATH"
Importing into Eclipse:
$ time buck build :eclipse
0m48.949s
Import existing project from `pwd`
Import 'gerrit' (do not import other Maven based projects)
Expand 'gerrit'
Right click 'buck-out' > Properties
Under Attributes check 'Derived'
If the code doesn't currently compile but an updated classpath
is needed, refresh the configs and obtain missing JARs:
$ buck build :eclipse_project :download
Running JUnit tests:
$ time buck test --all -e slow # skip slow tests
0m19.320s
$ time buck test --all # includes acceptance tests
5m17.517s
Building WAR:
$ buck build :gerrit
$ java -jar buck-out/gen/gerrit.war
Building release:
$ buck test --all && buck build :api :release
$ java -jar buck-out/gen/release.war
$ ls -lh buck-out/gen/{extension,plugin}-api.jar
Downloading dependencies:
Dependencies are normally downloaded automatically, but Buck can
inspect its graph and download missing dependencies so future
compiles can run without the network:
$ buck build :download
[1] http://facebook.github.io/buck/
Change-Id: I40853b108bd8e153cefa0896a5280a9a5ff81655
