= Gerrit Code Review - Building with Bazel [[installation]] == Prerequisites To build Gerrit from source, you need: * A Linux or macOS system (Windows is not supported at this time) * A JDK for Java 8|9|10|11|... * Python 2 or 3 * Node.js * link:https://www.bazel.io/versions/master/docs/install.html[Bazel] * Maven * zip, unzip * gcc [[Java 10 and newer version support]] Java 10 (and newer is) supported through vanilla java toolchain link:https://docs.bazel.build/versions/master/toolchains.html[Bazel option]. To build Gerrit with Java 10 and newer, specify vanilla java toolchain and provide the path to JDK home: ``` $ bazel build \ --define=ABSOLUTE_JAVABASE= \ --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \ --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \ --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \ :release ``` To run the tests, `--javabase` option must be passed as well, because bazel test runs the test using the target javabase: ``` $ bazel test \ --define=ABSOLUTE_JAVABASE= \ --javabase=@bazel_tools//tools/jdk:absolute_javabase \ --host_javabase=@bazel_tools//tools/jdk:absolute_javabase \ --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \ --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \ //... ``` To avoid passing all those options on every Bazel build invocation, they could be added to ~/.bazelrc resource file: ``` $ cat << EOF > ~/.bazelrc > build --define=ABSOLUTE_JAVABASE= > build --javabase=@bazel_tools//tools/jdk:absolute_javabase > build --host_javabase=@bazel_tools//tools/jdk:absolute_javabase > build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla > build --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla > EOF ``` Now, invoking Bazel with just `bazel build :release` would include all those options. Note that the follow option must be added to `container.javaOptions` in `$gerrit_site/etc/gerrit.config` to run Gerrit with Java 10|11|...: ``` [container] javaOptions = --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED ``` [[Java 9 support]] Java 9 is supported through alternative java toolchain link:https://docs.bazel.build/versions/master/toolchains.html[Bazel option]. The Java 9 support is backwards compatible. Java 8 is still the default. To build Gerrit with Java 9, specify JDK 9 java toolchain: ``` $ bazel build \ --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java9 \ --java_toolchain=@bazel_tools//tools/jdk:toolchain_java9 \ :release ``` Note that the follow option must be added to `container.javaOptions` in `$gerrit_site/etc/gerrit.config` to run Gerrit with Java 9: ``` [container] javaOptions = --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED ``` [[build]] == Building on the Command Line === Gerrit Development WAR File To build the Gerrit web application: ---- bazel build gerrit ---- [NOTE] PolyGerrit UI may require additional tools (such as npm). Please read the polygerrit-ui/README.md for more info. The output executable WAR will be placed in: ---- bazel-bin/gerrit.war ---- [[release]] === Gerrit Release WAR File To build the Gerrit web application that includes the PolyGerrit UI, core plugins and documentation: ---- bazel build release ---- The output executable WAR will be placed in: ---- bazel-bin/release.war ---- === Headless Mode To build Gerrit in headless mode, i.e. without the PolyGerrit UI: Web UI: ---- bazel build headless ---- The output executable WAR will be placed in: ---- bazel-bin/headless.war ---- === Extension and Plugin API JAR Files To build the extension, plugin and acceptance-framework JAR files: ---- bazel build api ---- The output archive that contains Java binaries, Java sources and Java docs will be placed in: ---- bazel-genfiles/api.zip ---- Install {extension,plugin,acceptance-framework}-api to the local maven repository: ---- tools/maven/api.sh install ---- Install gerrit.war to the local maven repository: ---- tools/maven/api.sh war_install ---- === Plugins ---- bazel build plugins:core ---- The output JAR files for individual plugins will be placed in: ---- bazel-genfiles/plugins//.jar ---- The JAR files will also be packaged in: ---- bazel-genfiles/plugins/core.zip ---- To build a specific plugin: ---- bazel build plugins/ ---- The output JAR file will be be placed in: ---- bazel-genfiles/plugins//.jar ---- Note that when building an individual plugin, the `core.zip` package is not regenerated. To build with all Error Prone warnings activated, run: ---- bazel build --java_toolchain //tools:error_prone_warnings_toolchain //... ---- [[IDEs]] == Using an IDE. === IntelliJ The Gerrit build works with Bazel's link:https://ij.bazel.io[IntelliJ plugin]. Please follow the instructions on <>. === Eclipse ==== Generating the Eclipse Project Create the Eclipse project: ---- tools/eclipse/project.py ---- and then follow the link:dev-eclipse.html#setup[setup instructions]. ==== Refreshing the Classpath If an updated classpath is needed, the Eclipse project can be refreshed and missing dependency JARs can be downloaded by running `project.py` again. For IntelliJ, you need to click the `Sync Project with BUILD Files` button of link:https://ij.bazel.io[IntelliJ plugin]. [[documentation]] === Documentation To build only the documentation for testing or static hosting: ---- bazel build Documentation:searchfree ---- The html files will be bundled into `searchfree.zip` in this location: ---- bazel-bin/Documentation/searchfree.zip ---- To build the executable WAR with the documentation included: ---- bazel build withdocs ---- The WAR file will be placed in: ---- bazel-bin/withdocs.war ---- [[tests]] == Running Unit Tests ---- bazel test --build_tests_only //... ---- Debugging tests: ---- bazel test --test_output=streamed --test_filter=com.gerrit.TestClass.testMethod testTarget ---- Debug test example: ---- bazel test --test_output=streamed --test_filter=com.google.gerrit.acceptance.api.change.ChangeIT.getAmbiguous //javatests/com/google/gerrit/acceptance/api/change:api_change ---- To run a specific test group, e.g. the rest-account test group: ---- bazel test //javatests/com/google/gerrit/acceptance/rest/account:rest_account ---- The tests run with NoteDb fully enabled by default. To run the tests against NoteDb backend with write to NoteDb, but not read from it: ---- bazel test --test_env=GERRIT_NOTEDB=WRITE //... ---- Write and read from NoteDb: ---- bazel test --test_env=GERRIT_NOTEDB=READ_WRITE //... ---- Primary storage NoteDb: ---- bazel test --test_env=GERRIT_NOTEDB=PRIMARY //... ---- NoteDb entirely disabled: ---- bazel test --test_env=GERRIT_NOTEDB=OFF //... ---- To run only tests that do not use SSH: ---- bazel test --test_env=GERRIT_USE_SSH=NO //... ---- To exclude tests that have been marked as flaky: ---- bazel test --test_tag_filters=-flaky //... ---- To exclude tests that require a Docker host: ---- bazel test --test_tag_filters=-docker //... ---- To ignore cached test results: ---- bazel test --cache_test_results=NO //... ---- To run one or more specific groups of tests: ---- bazel test --test_tag_filters=api,git //... ---- The following values are currently supported for the group name: * annotation * api * docker * edit * elastic * git * notedb * pgm * rest * server * ssh [[elasticsearch]] === Elasticsearch Successfully running the Elasticsearch tests requires Docker, and may require setting the local link:https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html[virtual memory]. If Docker is not available, the Elasticsearch tests will be skipped. Note that Bazel currently does not show link:https://github.com/bazelbuild/bazel/issues/3476[the skipped tests]. == Dependencies Dependency JARs are normally downloaded as needed, but you can download everything upfront. This is useful to enable subsequent builds to run without network access: ---- bazel fetch //... ---- When downloading from behind a proxy (which is common in some corporate environments), it might be necessary to explicitly specify the proxy that is then used by `curl`: ---- export http_proxy=http://:@: ---- Redirection to local mirrors of Maven Central and the Gerrit storage bucket is supported by defining specific properties in `local.properties`, a file that is not tracked by Git: ---- echo download.GERRIT = http://nexus.my-company.com/ >>local.properties echo download.MAVEN_CENTRAL = http://nexus.my-company.com/ >>local.properties ---- The `local.properties` file may be placed in the root of the gerrit repository being built, or in `~/.gerritcodereview/`. The file in the root of the gerrit repository has precedence. == Building against unpublished Maven JARs To build against unpublished Maven JARs, like gwtorm or PrologCafe, the custom JARs must be installed in the local Maven repository (`mvn clean install`) and `maven_jar()` must be updated to point to the `MAVEN_LOCAL` Maven repository for that artifact: [source,python] ---- maven_jar( name = 'gwtorm', artifact = 'gwtorm:gwtorm:42', repository = MAVEN_LOCAL, ) ---- == Building against artifacts from custom Maven repositories To build against custom Maven repositories, two modes of operations are supported: with rewrite in local.properties and without. Without rewrite the URL of custom Maven repository can be directly passed to the maven_jar() function: [source,python] ---- GERRIT_FORGE = 'http://gerritforge.com/snapshot' maven_jar( name = 'gitblit', artifact = 'com.gitblit:gitblit:1.4.0', sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa', repository = GERRIT_FORGE, ) ---- When the custom URL has to be rewritten, then the same logic as with Gerrit known Maven repository is used: Repo name must be defined that matches an entry in local.properties file: ---- download.GERRIT_FORGE = http://my.company.mirror/gerrit-forge ---- And corresponding WORKSPACE excerpt: [source,python] ---- GERRIT_FORGE = 'GERRIT_FORGE:' maven_jar( name = 'gitblit', artifact = 'com.gitblit:gitblit:1.4.0', sha1 = '1b130dbf5578ace37507430a4a523f6594bf34fa', repository = GERRIT_FORGE, ) ---- [[consume-jgit-from-development-tree]] To consume the JGit dependency from the development tree, edit `lib/jgit/jgit.bzl` setting LOCAL_JGIT_REPO to a directory holding a JGit repository. [[bazel-local-caches]] To accelerate builds, several caches are activated per default: * ~/.gerritcodereview/bazel-cache/downloaded-artifacts * ~/.gerritcodereview/bazel-cache/repository * ~/.gerritcodereview/bazel-cache/cas Currently none of these caches have a maximum size limit. See link:https://github.com/bazelbuild/bazel/issues/5139[this bazel issue] for details. Users should watch the cache sizes and clean them manually if necessary. [[npm-binary]] == NPM Binaries Parts of the PolyGerrit build require running NPM-based JavaScript programs as "binaries". We don't attempt to resolve and download NPM dependencies at build time, but instead use pre-built bundles of the NPM binary along with all its dependencies. Some packages on link:https://docs.npmjs.com/misc/registry[registry.npmjs.org] come with their dependencies bundled, but this is the exception rather than the rule. More commonly, to add a new binary to this list, you will need to bundle the binary yourself. [NOTE] We can only use binaries that meet certain licensing requirements, and that do not include any native code. Start by checking that the license and file types of the bundle are acceptable: [source,bash] ---- gerrit_repo=/path/to/gerrit package=some-npm-package version=1.2.3 npm install -g license-checker && \ rm -rf /tmp/$package-$version && mkdir -p /tmp/$package-$version && \ cd /tmp/$package-$version && \ npm install $package@$version && \ license-checker | grep licenses: | sort -u ---- This will output a list of the different licenses used by the package and all its transitive dependencies. We can only legally distribute a bundle via our storage bucket if the licenses allow us to do so. As long as all of the listed license are allowed by link:https://opensource.google.com/docs/thirdparty/licenses/[Google's standards]. Any `by_exception_only`, commercial, prohibited, or unlisted licenses are not allowed; otherwise, it is ok to distribute the source. If in doubt, contact a maintainer who is a Googler. Next, check the file types: [source,bash] ---- cd /tmp/$package-$version find . -type f | xargs file | grep -v 'ASCII\|UTF-8\|empty$' ---- If you see anything that looks like a native library or binary, then we can't use the bundle. If everything looks good, create the bundle, and note the SHA-1: [source,bash] ---- $gerrit_repo/tools/js/npm_pack.py $package $version && \ sha1sum $package-$version.tgz ---- This creates a file named `$package-$version.tgz` in your working directory. Any project maintainer can upload this file to the link:https://console.cloud.google.com/storage/browser/gerrit-maven/npm-packages[storage bucket]. Finally, add the new binary to the build process: ---- # WORKSPACE npm_binary( name = "some-npm-package", repository = GERRIT, ) # lib/js/npm.bzl NPM_VERSIONS = { ... "some-npm-package": "1.2.3", } NPM_SHA1S = { ... "some-npm-package": "", } ---- To use the binary from the Bazel build, you need to use the `run_npm_binary.py` wrapper script. For an example, see the use of `crisper` in `tools/bzl/js.bzl`. GERRIT ------ Part of link:index.html[Gerrit Code Review] SEARCHBOX ---------