520f29c16a
Switch deploying Gerrit extension and plugin API from Google storage bucket to Maven Central. To simplify the troubleshooting new trace option was added to underlying python script. When environment variable VERBOSE=1 is set, then the mvn command is printed to stderr: VERBOSE=1 buck build api_deploy_oss Change-Id: I9ebe44ec8aca91aa6b5514f561209e7c50377927
507 lines
12 KiB
Plaintext
507 lines
12 KiB
Plaintext
= Gerrit Code Review - Building with Buck
|
|
|
|
|
|
== Installation
|
|
|
|
There is currently no binary distribution of Buck, so it has to be manually
|
|
built and installed. Apache Ant is required. Currently only Linux and Mac
|
|
OS are supported. Gerrit's buck wrappers require Python version 2.6 or higher.
|
|
|
|
Clone the git and build it:
|
|
|
|
----
|
|
git clone https://gerrit.googlesource.com/buck
|
|
cd buck
|
|
ant
|
|
----
|
|
|
|
If you don't have a `bin/` directory in your home directory, create one:
|
|
|
|
----
|
|
mkdir ~/bin
|
|
----
|
|
|
|
Add the `~/bin` folder to the path:
|
|
|
|
----
|
|
PATH=~/bin:$PATH
|
|
----
|
|
|
|
Note that the buck executable needs to be available in all shell sessions,
|
|
so also make sure it is appended to the path globally.
|
|
|
|
Add a symbolic link in `~/bin` to the buck executable:
|
|
|
|
----
|
|
ln -s `pwd`/bin/buck ~/bin/
|
|
----
|
|
|
|
Verify that `buck` is accessible:
|
|
|
|
----
|
|
which buck
|
|
----
|
|
|
|
If you plan to use the link:#buck-daemon[Buck daemon] add a symbolic
|
|
link in `~/bin` to the buckd executable:
|
|
|
|
----
|
|
ln -s `pwd`/bin/buckd ~/bin/
|
|
----
|
|
|
|
To enable autocompletion of buck commands, install the autocompletion
|
|
script from `./scripts/bash_completion` in the buck project. Refer to
|
|
the script's header comments for installation instructions.
|
|
|
|
|
|
[[eclipse]]
|
|
== Eclipse Integration
|
|
|
|
|
|
=== Generating the Eclipse Project
|
|
|
|
Create the Eclipse project:
|
|
|
|
----
|
|
tools/eclipse/project.py
|
|
----
|
|
|
|
In Eclipse, choose 'Import existing project' and select the `gerrit` project
|
|
from the current working directory.
|
|
|
|
Expand the `gerrit` project, right-click on the `buck-out` folder, select
|
|
'Properties', and then under 'Attributes' check 'Derived'.
|
|
|
|
Note that if you make any changes in the project configuration
|
|
that get saved to the `.project` file, for example adding Resource
|
|
Filters on a folder, they will be overwritten the next time you run
|
|
`tools/eclipse/project.py`.
|
|
|
|
|
|
=== Refreshing the Classpath
|
|
|
|
If an updated classpath is needed, the Eclipse project can be
|
|
refreshed and missing dependency JARs can be downloaded:
|
|
|
|
----
|
|
tools/eclipse/project.py
|
|
----
|
|
|
|
|
|
=== Attaching Sources
|
|
|
|
To save time and bandwidth source JARs are only downloaded by the buck
|
|
build where necessary to compile Java source into JavaScript using the
|
|
GWT compiler. Additional sources may be obtained, allowing Eclipse to
|
|
show documentation or dive into the implementation of a library JAR:
|
|
|
|
----
|
|
tools/eclipse/project.py --src
|
|
----
|
|
|
|
|
|
[[build]]
|
|
== Building on the Command Line
|
|
|
|
|
|
=== Gerrit Development WAR File
|
|
|
|
To build the Gerrit web application:
|
|
|
|
----
|
|
buck build gerrit
|
|
----
|
|
|
|
The output executable WAR will be placed in:
|
|
|
|
----
|
|
buck-out/gen/gerrit.war
|
|
----
|
|
|
|
|
|
=== Extension and Plugin API JAR Files
|
|
|
|
To build the extension, plugin and GWT API JAR files:
|
|
|
|
----
|
|
buck build api
|
|
----
|
|
|
|
Java binaries, Java sources and Java docs are generated into corresponding
|
|
project directories in `buck-out/gen`, here as example for plugin API:
|
|
|
|
----
|
|
buck-out/gen/gerrit-plugin-api/plugin-api.jar
|
|
buck-out/gen/gerrit-plugin-api/plugin-api-src.jar
|
|
buck-out/gen/gerrit-plugin-api/plugin-api-javadoc.jar
|
|
----
|
|
|
|
Install {extension,plugin,gwt}-api to the local maven repository:
|
|
|
|
----
|
|
buck build api_install
|
|
----
|
|
|
|
Install gerrit.war to the local maven repository:
|
|
|
|
----
|
|
buck build war_install
|
|
----
|
|
|
|
=== Plugins
|
|
|
|
To build all core plugins:
|
|
|
|
----
|
|
buck build plugins:core
|
|
----
|
|
|
|
The output JAR files for individual plugins will be placed in:
|
|
|
|
----
|
|
buck-out/gen/plugins/<name>/<name>.jar
|
|
----
|
|
|
|
The JAR files will also be packaged in:
|
|
|
|
----
|
|
buck-out/gen/plugins/core.zip
|
|
----
|
|
|
|
To build a specific plugin:
|
|
|
|
----
|
|
buck build plugins/<name>:<name>
|
|
----
|
|
|
|
The output JAR file will be be placed in:
|
|
|
|
----
|
|
buck-out/gen/plugins/<name>/<name>.jar
|
|
----
|
|
|
|
Note that when building an individual plugin, the `core.zip` package
|
|
is not regenerated.
|
|
|
|
Additional plugins with BUCK files can be added to the build
|
|
environment by cloning the source repository into the plugins
|
|
subdirectory:
|
|
|
|
----
|
|
git clone https://gerrit.googlesource.com/plugins/<name> plugins/<name>
|
|
echo /plugins/<name> >>.git/info/exclude
|
|
----
|
|
|
|
Additional plugin sources will be automatically added to Eclipse the
|
|
next time project.py is run:
|
|
|
|
----
|
|
tools/eclipse/project.py
|
|
----
|
|
|
|
|
|
[[documentation]]
|
|
=== Documentation
|
|
|
|
To build only the documentation:
|
|
|
|
----
|
|
buck build docs
|
|
----
|
|
|
|
The generated html files will be placed in:
|
|
|
|
----
|
|
buck-out/gen/Documentation/html__tmp/Documentation
|
|
----
|
|
|
|
The html files will also be bundled into `html.zip` in this location:
|
|
|
|
----
|
|
buck-out/gen/Documentation/html.zip
|
|
----
|
|
|
|
To build the executable WAR with the documentation included:
|
|
|
|
----
|
|
buck build withdocs
|
|
----
|
|
|
|
The WAR file will be placed in:
|
|
|
|
----
|
|
buck-out/gen/withdocs.war
|
|
----
|
|
|
|
[[release]]
|
|
=== Gerrit Release WAR File
|
|
|
|
To build the release of the Gerrit web application, including documentation and
|
|
all core plugins:
|
|
|
|
----
|
|
buck build release
|
|
----
|
|
|
|
The output release WAR will be placed in:
|
|
|
|
----
|
|
buck-out/gen/release.war
|
|
----
|
|
|
|
[[tests]]
|
|
== Running Unit Tests
|
|
|
|
To run all tests including acceptance tests:
|
|
|
|
----
|
|
buck test --all
|
|
----
|
|
|
|
To exclude slow tests:
|
|
|
|
----
|
|
buck test --all --exclude slow
|
|
----
|
|
|
|
To run a specific test, e.g. the acceptance test
|
|
`com.google.gerrit.acceptance.git.HttpPushForReviewIT`:
|
|
|
|
----
|
|
buck test //gerrit-acceptance-tests/src/test/java/com/google/gerrit/acceptance/git:HttpPushForReviewIT
|
|
----
|
|
|
|
|
|
== Dependencies
|
|
|
|
Dependency JARs are normally downloaded automatically, but Buck can inspect
|
|
its graph and download any missing JAR files. This is useful to enable
|
|
subsequent builds to run without network access:
|
|
|
|
----
|
|
tools/download_all.py
|
|
----
|
|
|
|
When downloading from behind a proxy (which is common in some corporate
|
|
environments), it might be necessary to explicitly specify the proxy that
|
|
is then used by `curl`:
|
|
|
|
----
|
|
export http_proxy=http://<proxy_user_id>:<proxy_password>@<proxy_server>:<proxy_port>
|
|
----
|
|
|
|
Redirection to local mirrors of Maven Central and the Gerrit storage
|
|
bucket is supported by defining specific properties in
|
|
`local.properties`, a file that is not tracked by Git:
|
|
|
|
----
|
|
echo download.GERRIT = http://nexus.my-company.com/ >>local.properties
|
|
echo download.MAVEN_CENTRAL = http://nexus.my-company.com/ >>local.properties
|
|
----
|
|
|
|
The `local.properties` file may be placed in the root of the gerrit repository
|
|
being built, or in `~/.gerritcodereview/`. The file in the root of the gerrit
|
|
repository has precedence.
|
|
|
|
== Building against unpublished Maven JARs
|
|
|
|
To build against unpublished Maven JARs, like gwtorm or PrologCafe, the custom
|
|
JARs must be installed in the local Maven repository (`mvn clean install`) and
|
|
`maven_jar()` must be updated to point to the `MAVEN_LOCAL` Maven repository for
|
|
that artifact:
|
|
|
|
[source,python]
|
|
----
|
|
maven_jar(
|
|
name = 'gwtorm',
|
|
id = 'gwtorm:gwtorm:42',
|
|
license = 'Apache2.0',
|
|
repository = MAVEN_LOCAL,
|
|
)
|
|
----
|
|
|
|
== Building against unpublished JARs, that change frequently
|
|
|
|
If a dependent Gerrit library is undergoing active development it must be
|
|
recompiled and the change must be reflected in the Buck build process. For
|
|
example testing Gerrit against changed JGit snapshot version. After building
|
|
JGit library, the artifacts are created in local Maven build directory, e. g.:
|
|
|
|
----
|
|
mvn package
|
|
/home/<user>/projects/jgit/org.eclipse.jgit/target/org.eclipse.jgit-3.3.0-SNAPSHOT.jar
|
|
/home/<user>/projects/jgit/org.eclipse.jgit/target/org.eclipse.jgit-3.3.0-SNAPSHOT-sources.jar
|
|
----
|
|
|
|
If as usual, installation of the build artifacts takes place in local maven
|
|
repository, then the Buck build must fetch them from there with normal
|
|
`download_file.py` process. Disadvantage of this approach is that Buck cache
|
|
invalidation must occur to refresh the artifacts after next
|
|
change-compile-install round trip.
|
|
|
|
To shorten that workflow and take the installation of the artifacts to the
|
|
local Maven repository and fetching it again from there out of the picture,
|
|
`local_jar()` method is used instead of `maven_jar()`:
|
|
|
|
[source,python]
|
|
----
|
|
local_jar(
|
|
name = 'jgit',
|
|
jar = '/home/<user>/projects/jgit/org.eclipse.jgit/target/org.eclipse.jgit-3.3.0-SNAPSHOT.jar',
|
|
src = '/home/<user>/projects/jgit/org.eclipse.jgit/target/org.eclipse.jgit-3.3.0-SNAPSHOT-sources.jar',
|
|
deps = [':ewah']
|
|
)
|
|
----
|
|
|
|
This creates a symlink to the Buck targets direct against artifacts in
|
|
another project's Maven target directory:
|
|
|
|
----
|
|
buck-out/gen/lib/jgit/jgit.jar ->
|
|
/home/<user>/projects/jgit/org.eclipse.jgit/target/org.eclipse.jgit-3.3.0-SNAPSHOT.jar
|
|
----
|
|
|
|
== Building against artifacts from custom Maven repositories
|
|
|
|
To build against custom Maven repositories, two modes of operations are
|
|
supported: with rewrite in local.properties and without.
|
|
|
|
Without rewrite the URL of custom Maven repository can be directly passed
|
|
to the maven_jar() function:
|
|
|
|
[source,python]
|
|
----
|
|
GERRIT_FORGE = 'http://gerritforge.com/snapshot'
|
|
|
|
maven_jar(
|
|
name = 'gitblit',
|
|
id = 'com.gitblit:gitblit:1.4.0',
|
|
sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa',
|
|
license = 'Apache2.0',
|
|
repository = GERRIT_FORGE,
|
|
)
|
|
----
|
|
|
|
When the custom URL has to be rewritten, then the same logic as with Gerrit
|
|
known Maven repository is used: Repo name must be defined that matches an entry
|
|
in local.properties file:
|
|
|
|
----
|
|
download.GERRIT_FORGE = http://my.company.mirror/gerrit-forge
|
|
----
|
|
|
|
And corresponding BUCK excerpt:
|
|
|
|
[source,python]
|
|
----
|
|
GERRIT_FORGE = 'GERRIT_FORGE:'
|
|
|
|
maven_jar(
|
|
name = 'gitblit',
|
|
id = 'com.gitblit:gitblit:1.4.0',
|
|
sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa',
|
|
license = 'Apache2.0',
|
|
repository = GERRIT_FORGE,
|
|
)
|
|
----
|
|
|
|
=== Caching Build Results
|
|
|
|
Build results can be locally cached, saving rebuild time when
|
|
switching between Git branches. Buck's documentation covers
|
|
caching in link:http://facebook.github.io/buck/concept/buckconfig.html[buckconfig].
|
|
The trivial case using a local directory is:
|
|
|
|
----
|
|
cat >.buckconfig.local <<EOF
|
|
[cache]
|
|
mode = dir
|
|
dir = buck-cache
|
|
EOF
|
|
----
|
|
|
|
[[buck-daemon]]
|
|
=== Using Buck daemon
|
|
|
|
Buck ships with a daemon command `buckd`, which uses the
|
|
link:https://github.com/martylamb/nailgun[Nailgun] protocol for running
|
|
Java programs from the command line without incurring the JVM startup
|
|
overhead.
|
|
|
|
Using a Buck daemon can save significant amounts of time as it avoids the
|
|
overhead of starting a Java virtual machine, loading the buck class files
|
|
and parsing the build files for each command.
|
|
|
|
It is safe to run several buck daemons started from different project
|
|
directories and they will not interfere with each other. Buck's documentation
|
|
covers daemon in http://facebook.github.io/buck/command/buckd.html[buckd].
|
|
|
|
The trivial use case is to run `buckd` from the project's root directory and
|
|
run `buck` as usual:
|
|
|
|
----
|
|
buckd
|
|
buck build gerrit
|
|
Using buckd.
|
|
[-] PARSING BUILD FILES...FINISHED 0.6s
|
|
[-] BUILDING...FINISHED 0.2s
|
|
----
|
|
|
|
=== Override Buck's settings
|
|
|
|
Additional JVM args for Buck can be set in `.buckjavaargs` in the
|
|
project root directory. For example to override Buck's default 1GB
|
|
heap size:
|
|
|
|
----
|
|
cat > .buckjavaargs <<EOF
|
|
-XX:MaxPermSize=512m -Xms8000m -Xmx16000m
|
|
EOF
|
|
----
|
|
|
|
== Rerun unit tests
|
|
|
|
If for some reasons tests, that were already run must be repeated, unit test
|
|
cache must be removed fist. That's because the test execution results are
|
|
cached by Buck:
|
|
|
|
----
|
|
$ rm -rf buck-out/bin/gerrit-acceptance-tests/src/test/java/com/google/gerrit/acceptance/rest/group/.AddRemoveGroupMembersIT/
|
|
----
|
|
|
|
After clearing the cache test can be rerun again:
|
|
|
|
----
|
|
$ buck test //gerrit-acceptance-tests/src/test/java/com/google/gerrit/acceptance/rest/group:AddRemoveGroupMembersIT
|
|
TESTING //gerrit-acceptance-tests/src/test/java/com/google/gerrit/acceptance/rest/group:AddRemoveGroupMembersIT
|
|
PASS 14,9s 8 Passed 0 Failed com.google.gerrit.acceptance.rest.group.AddRemoveGroupMembersIT
|
|
TESTS PASSED
|
|
----
|
|
|
|
An alternative approach is to use a Buck feature:
|
|
--test-selectors (-filters, -f) option:
|
|
|
|
----
|
|
buck test --all -f 'com.google.gerrit.acceptance.rest.change.SubmitByMergeAlwaysIT'
|
|
TESTING SELECTED TESTS
|
|
PASS 14,5s 6 Passed 0 Failed com.google.gerrit.acceptance.rest.change.SubmitByMergeAlwaysIT
|
|
TESTS PASSED
|
|
----
|
|
|
|
When this option is used, cache is disabled per design and doesn't need to be deleted.
|
|
Note: when -f option is used, the whole unit test cache is dropped. As a consequence,
|
|
repeating the
|
|
|
|
----
|
|
buck test --all
|
|
----
|
|
|
|
would re-execute all tests again.
|
|
|
|
GERRIT
|
|
------
|
|
Part of link:index.html[Gerrit Code Review]
|
|
|
|
SEARCHBOX
|
|
---------
|