35 Commits

Author SHA1 Message Date
Shawn Pearce
41c2e73da2 Support Bazel 0.5.1
Bazel 0.5.1 is using "bazel-out/darwin_x86_64-fastbuild/", which
failed these extraction patterns.  Allow anything before the
-fastbuild part of the path.

Change-Id: I850271bb1c657a5ffe77946a6beaf54a3680af75
2017-06-12 08:53:05 +09:00
David Ostrovsky
596fff00cc Eclipse project generation: Support jgit from external repository
This change fixes Eclipse project generation when jgit is consumed from
external repository. Only path to the library is added, but not the
sourcepath.

Test Plan:

* Switch to using jgit from external repository by amending the path
  in lib/jgit/jgit.bzl
* Run tools/eclipse/project.py
* Verify that classpath entries are added to the .classpath file

Change-Id: I2bdadb74b3ef869805a96671b7882917967998d6
2017-04-07 07:44:10 +00:00
David Pursehouse
c26261acf9 Fix launching daemon in Eclipse on MacOS
Change I5fc96bf1f removed the generation of the .primary_build_tool
file and change I3d6b90320 removed the build system abstraction with
the motivation that we now only have one build tool.

However this broke launching the daemon from Eclipse on MacOS, which
now fails because bazel is not on the path and can't be found.

Restore the removed functionality, but rename the file to .bazel_path

Change-Id: Ibd0ec09d3bf47f383ca68a37cca4e81640960416
2017-03-17 14:10:41 +09:00
David Pursehouse
34b59f0ad3 Get rid of references to .primary_build_tool
This was introduced to support both buck and bazel build tools, and
is no longer necessary since we completely dropped buck in favor of
bazel.

Change-Id: I5fc96bf1fc27f20180737fea954e150070e5302f
2017-03-15 08:38:05 +09:00
David Pursehouse
7e43d1d1ad Rename project_bzl.py to project.py
Since the project.py script was removed along with the buck-based
build, it's not necessary to keep the _bzl suffix on the script for
the bazel build.

Change-Id: I3f97cab5cc0e855a5dc1865bb4cc00c18adea2e9
2017-01-24 08:45:16 +09:00
David Ostrovsky
fdbfcad77d Remove Buck based build
Bug: Issue 5302
Change-Id: I6e860446ef30ff0ad1c7c49fc0e39d39d921820b
2017-01-23 12:44:58 +00:00
Zhen Chen
a617c47f83 Add missing import for project.py
import os.makedirs

Change-Id: Ibc279c8b7f2bb5198f9fa3fd1dfa587ee75e88b0
2016-11-15 11:50:45 -08:00
David Ostrovsky
c69f360714 Bazel: Generate Eclipse classpath
To guess what build system is used, we create now .primary_build_tool
file in the root of the project during the eclipse classpath generation.

Change the working directory for GWT SDM session to be .gwt_work_dir.
The reason for that Bazel doesn't allow to write to bazel-out.

Change-Id: I984068350244ee9d66807e4bc8c6779b34a26bab
2016-11-15 10:12:44 -08:00
David Ostrovsky
19b60b5527 Bazel: Gracefully ignore eclipse-out directory
Bazel doesn't provide a way to exclude scanning directories in project
root: [1]. Add workaround, by teaching Eclipse to exclude BUILD file
instead. With this workaround in place Eclipse and Bazel build can
co-exist:

  $ tools/eclipse/project.py # generates .classpath and other files
  # Start Eclipse and rebuild gerrit, eclipse-out is created
  $ bazel test ... # works as expected now

[1] https://github.com/bazelbuild/bazel/issues/1083

Change-Id: I7e767d0768af8bba3acc84d5fc242cab0e9abfdb
2016-10-30 08:51:31 +01:00
David Ostrovsky
f8c0901db8 Fix Eclipse classpath generation for source jars
Bug: Issue 4646
Change-Id: I8f7e5fcb20a0a3c70f147abb0fe3bfc5c07e359c
2016-09-27 22:02:48 +02:00
David Ostrovsky
3931c16bdb Build with Java 8
Change-Id: I06f811fc46a606b719582c0724e8d0c89e823b37
2016-09-20 10:19:52 +02:00
David Pursehouse
6925f8a7e4 project.py: Allow to specify the name of the generated project
Eclipse won't allow to open multiple projects with the same name.

Add a --name option to the project.py to allow to specify the name
of the generated project. This will allow to have multiple Gerrit
workspaces in Eclipse at the same time.

Change-Id: I6f6809a0a42ee21262eadd5125beb344cbaf7dfe
2016-05-30 09:53:15 +00:00
David Ostrovsky
08ea694499 Buck: Remove jgit cell
Cross cell support in Buck is considered as experimental feature, with
number of open issues: [1], [2], [3].  Moreover, to make Maven Central
machinery work, it was needed to create symbolic links in source tree.
That broke `buck targets` feature.

Remove it for now, and re-consider to add it later.

[1] https://github.com/facebook/buck/issues/656
[2] https://github.com/facebook/buck/issues/658
[3] https://github.com/facebook/buck/issues/717

Bug: Issue 3954
Change-Id: Ic621a07771f926001df181b46b2169e214ce208a
2016-04-20 22:02:49 +02:00
David Pursehouse
cb7f914a20 project.py: always download sources
Instead of downloading the sources optionally, always download them.
This will help to catch problems introduced in the build that prevent
the sources from downloading (see [1] for example).

Keep the --src option but mark it as deprecated.

Add a new --no-src option to explicitly prevent download of sources.

[1] https://gerrit-review.googlesource.com/#/c/74859/

Change-Id: I9b6a9e210f83fe6a5663011dc5c0daeaa2f7fe46
2016-02-24 01:29:58 +00:00
David Ostrovsky
17bb3dbfe8 Buck: Allow to consume JGit from its own cell
Consume JGit as first third party library from its own cell. Normally
the cell is defined as lib/jgit directory. It can be easily replaced
with CLI:

  buck build --config repositories.jgit=path/to/dev/jgit gerrit

or tweaking the .buckconfig:

  [repositories]
    jgit = path/to/dev/jgit

The former approach is sufficient to build and run the test from the
CLI, the latter is needed to generate eclipse project.

To isolate the JGit rules in its own cell some refactoring was needed.
JGit patch for GWT module was moved to gerrit-patch-jgit project, and
some symlinks were needed for maven machinery to work. include_defs()
doesn't work for now across cell boundaries, and native `buck fetch`
feature still has some limitations: [1]. Moreover, excluding paths,
unsigning JARs and license linking should be re-implemented on top
of it.

[1] https://github.com/facebook/buck/issues/602

Test Plan:

Normal gerrit build and the build with hijacked JGit cell should work
in both standalone (gerrit.war) and Eclipse environment. Note, that
to test --config repositories.jgit=path/to/dev/jgit use case, the most
recent JGit tree must be used, that contains Buck driven build
implementation.

Change-Id: I39f2d5d75bbac88804406d6242b5e714f4916926
2016-02-09 00:40:08 +00:00
David Ostrovsky
feeb212308 Update Buck to newest version
Now, that cross-cell support is implemented: [1] we can share rules
between projects. Cell is a container with its own buck-out directory.
The Buck design allows to replace a cell with alternative location.
This is a preparation change to support cross-cell dependencies.

The prerequisite for the hijacked cell to work properly, is that the
classpaths are always represented as absolute paths. That was not the
case in Buck implementation and thus classpath resolution across cell
boundaries was broken. This got fixed in: [2], [3].

[1] https://github.com/facebook/buck/issues/116
[2] https://github.com/facebook/buck/issues/545
[3] https://github.com/facebook/buck/pull/558

Change-Id: I3b17129c0f7ca5801f03091fff5651fb63d1482f
2016-01-28 03:12:05 +00:00
Dave Borowitz
004bf2a928 Eclipse: List GWT deps after non-GWT deps
The stated purpose of separating libs and gwt_libs is to avoid
incorrect versions of classes used elsewhere in Gerrit (e.g. servlet
API 3.0 instead of 3.1). But we were iterating through gwt_libs first,
so this wasn't actually working. Swap the order.

Also change the prefix check for GWT deps to a contains check, as buck
audit at some point started returning absolute paths to jars.

Change-Id: Ic372b6bfaca5a8a1c1b9cc32aaf868a86a91c6ef
2015-12-02 09:30:25 -05:00
David Ostrovsky
58cb4b4881 Update Buck to newest version
The new Buck version fixed annoying stdout spamming bug on unit test
failures: [1]. Now we can revert our monkey patching hack to prevent
that.

Since [2] Buck interferes with files in buck-out directory: [3]. Switch
to using eclipse-out directory as Eclipse output directory instead. For
this change it's necessary to clean up buck-out directory, otherwise
`buck test` would fail.

This version also fixed "Python client lost connection" bug: [4].

This reverts commit 94e93aaad22e67b5956627cff1a9cb84d03a29ec.

[1] https://github.com/facebook/buck/issues/505
[2] 35cb495b57
[3] https://github.com/facebook/buck/issues/527
[4] https://github.com/facebook/buck/issues/534

Change-Id: I4cd1a99ce9d0615713c235d873e6cdd61b1854bb
2015-12-02 14:07:12 +00:00
David Ostrovsky
805ef02aa3 Buck: Tweak Eclipse project generation to include sources again
Ib3f22e70b7 upgraded to new Buck version but broke sources download
during Eclipse project creation. That's because the namespacing of
genrule() artifact. To rectify and to avoid querying the actual
location of source artifact from Buck for every source library with:

  $ buck targets --show_output //lib:foo__download_src | awk '{print $2}'

switch from using genrule() to prebuilt_jar() to induce the source from
binary artifact, because the location is stable for this rule.

Change-Id: I153d3e31b4f7098ddd5157b7b1bba17529c83b32
2015-10-15 08:11:19 +02:00
David Ostrovsky
f377aa96de Update buck to the latest version
This version includes a lot of new features and improvements,
including:

* Switched to top-down-building, which should generally make builds
  faster. The old behavior can be bypassed by passing --deep on the
  command line [1]

* New query command was added (inspired by Bazel) [2]

* Performance improvement in file globbing

Extend tools/eclipse/project.py to pass the --deep option to ask Buck
to execute bottom-up build when generating the Eclipse project. This is
needed otherwise after using the buck clean command only the gerrit.war
file would be fetched from the cache and the buck-out/gen/lib folder
would remain empty.

With [3] genrule output is now namespaced with the genrule name. Adapt
documentation and references in the code to the new location of Buck
artifacts. Because of this change, `buck clean` must be issued after
Buck upgrade, otherwise the build would fail. The same prolem exists
when switching between branches: `buck clean` must be issued, otherwise
the build would fail.

Test plan:

* buck build release
* buck build api_install
* buck test
* install and verify new gerrit site
* upgrade and verify existing gerrit site
* reindex existing gerrit site
* verify that tools/eclipse/project.py produces sane Eclipse project
* verify that unit test execution from Eclipse works
* verify that daemon started from Eclipse works
* verify that GWT SDM debug session started from Eclipe works

[1] 217cec33bc
[2] https://buckbuild.com/command/query.html
[3] c92ef212b5

Change-Id: Ib3f22e70b7cb9eb9349618f2bcc018bf799c40f8
2015-10-09 08:51:35 +02:00
David Ostrovsky
f3285e0280 Buck: Fix eclipse project generation
After refactoring done in Ia46c0559a lib/jgit is recognized as source
folder for the non existing Edit patch library:

  $ buck audit classpath //gerrit-gwtui:ui_module
  buck-out/gen/lib/jgit/lib__Edit__output/Edit.jar

As the consequence non existing path is induced from it and included
in .classpath file:

  path="/home/davido/projects/gerrit/lib/jgit/src/main/java"

Add a check for existence of source folder and include it only if the
folder exists.

Change-Id: Icef39196173722c9c8b760c041a4c9ec9d22ab7c
2015-06-26 05:41:48 +00:00
Björn Pedersen
c357cebdae tools: use python on path
Use the canonical syntax to specify the python executable to use.

Avoids a fatal error if /usr/bin/python is a too old version (< 2.7).

Change-Id: I3e8affb52be993d35c0dcf90774d962a59ef5635
2015-06-18 17:10:11 +02:00
David Pursehouse
686bfea7b3 Fix minor python code style issues
Fix some minor issues that were reported by pyflakes and pylint.

- Rename arguments to prevent redefinition of built-in 'dir'
- Rename method call to prevent redefinition of built-in 'help'
- Remove unused imports
- Prefix unused variable with underscore to prevent warning

Change-Id: Ia9e21c0f1a69af43f6e4566be46c31bdda540d2f
2014-12-18 02:51:06 +00:00
Dave Borowitz
08180de4de Support AutoValue
AutoValue[1] is a lightweight annotation-processor-based library for
implementing classes with simple, obvious value semantics.

Add support for AutoValue to build rules and Eclipse project
generation. Buck does not currently have an officially-supported
interface for specifying annotation processor dependencies[2], so we
have to take the slightly ugly approach of monkey-patching
java_library and java_test to add annotation processor arguments to
each rule that requires annotation processing; hopefully this ugliness
can be reduced in the future.

[1] https://github.com/google/auto/tree/master/value
[2] https://github.com/facebook/buck/issues/85

Change-Id: I8b49d6f9f25d61688b667d964848c6ce106ae4ec
2014-11-08 08:31:08 -08:00
Stefan Lay
e0368d860f Add test sources to .classpath files of eclipse plugin projects
Change-Id: I274e7d52648eff78f8cf218b8146bdedac7f8007
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
2014-10-02 09:43:05 +02:00
Saša Živkov
353f8752a8 Merge "Add an option to create eclipse projects also for plugins" 2014-09-24 07:57:18 +00:00
Stefan Lay
2e9a46e923 Add an option to create eclipse projects also for plugins
With the option --plugins project.py creates an own eclipse project
for each plugin. The plugin projects depend on the /gerrit project.
All libraries in the gerrit project are exported now in order to make
them available for the plugin projects.

The advantage of this is that now each eclipse project only contains
sources belonging to a single git repository. EGit assumes one git
repository per eclipse project. When the plugin sources are part of the
gerrit eclipse project, EGit cannot determine the git repository
correctly.

Change-Id: I154ca2e474e5a729d2adc2ec44fcfd9429387e51
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2014-09-23 14:40:32 +02:00
Khai Do
72b38e022a Fix generated classpath ordering
There is a bug with this script only when plugins are added to the plugins
folder. I found it when attempting to debug the its-bugzilla plugin.
It places the plugin dependent libraries ahead of the Gerrit core libraries
which can cause conflicts and failures with Gerrit core.  This change
switches the ordering.

Change-Id: Id39a9035928c1dcc3daf034f5fdd7d88d87bb024
2014-09-10 14:56:22 +02:00
Luca Milanesio
8f11e6c9fc Fix root path resolution for updated Buck
Since the last Buck upgraded this morning with [1]
the root path resolution is broken.

This is due to the different expansion of the __file__
macro into a deeper structure, including the python
relative path name inside a PEX directory structure.

Example:
gerrit/buck-out/bin/tools/maven/mvn.pex/gerrit-extension-api/pom.xml

In order to go back to the root package structure
and locate the pom.xml file correctly, we need now
to go back until the .buckconfig file is found
(instead of going back exactly 3 times as it was before).

P.S. Without this change the targets api_install
     and war_install are broken on master.

Change-Id: I1a99606ad304c245b21ba501f15c473142a89fc2
2014-06-13 07:15:03 +00:00
Shawn Pearce
9e4d5aa347 Isolate plugin class directories from each other
Configure Eclipse to compile/copy each plugin's classes and resources
into its own private classes folder.  This should fix the duplicate
resource warnings for about.md or other documentation and static image
assets.

Move test and utility classes into their own directories too. This
has no real impact on the runtime classpath at this time, but opens
the door to do something more creative later.

Change-Id: If2a048dfe0349f671a17f9e80d6f22a69a5c0ade
2013-11-30 13:17:18 -08:00
Shawn Pearce
10f09d3d52 Require Java 7
Jetty 9 was built with compiler settings producing class files that
are not recognized by Java 6 runtimes.

Gerrit 2.9 will require Java 7 as its minimum runtime version.

Change-Id: Ifed3f9e07bf77a7a737b16c20e6e400acd990649
2013-11-16 19:58:03 -08:00
David Pursehouse
7a16e859a4 Handle KeyboardInterrupt in Buck build wrapper scripts
Pressing Ctrl-C during the build results in an ugly Python
TraceBack.

Add handling of KeyboardInterrupt and print a useful message.

This fixes most cases, but there are still a few TraceBacks
coming from inside Buck.  These will need to be fixed upstream.

Change-Id: I3f0dc19f3be599460a2a6409642a70a195b50753
2013-10-18 15:10:25 +09:00
David Pursehouse
6c25ea8fdd Fix a few coding style nits in Buck build Python scripts
- Prefix regex strings with `r`
- Remove unused imports
- Rename variables and parameters to avoid redefinition of
  built-in types `dir` and `type`

Change-Id: Ia21667b89631c751f241dd6499a2e460e3435697
2013-09-26 16:50:07 +09:00
Shawn Pearce
0672a19c54 Create buck.properties during tools/eclipse/project.py
Change-Id: I7fdccee7d08e2177a5bb4a4ecbcf17be9d4ada39
2013-09-24 12:27:31 -07:00
Shawn Pearce
a7a3ee1e3a Update Buck to latest master
genrule() no longer supports running commands run only for the
side-effect with no output file. Actions like download_sources or
eclipse need to be handled using Python scripts executed from the
user's shell, otherwise Buck fails if caching is enabled.

Change-Id: I361fc20675f211e15e4ab7942ef52778d0a615c2
2013-09-20 19:41:26 -07:00