10 Commits

Author SHA1 Message Date
Shawn Pearce
6d9ad02f9b Merge "Support downloading from other Maven repositories" 2013-05-14 07:00:32 +00:00
David Pursehouse
72a1701927 Add missing import of symlink in download_jar.py
Missing import of `symlink` from the `os` module causes an error:

  Traceback (most recent call last):
    File "/work/gerrit/tools/download_jar.py", line 108, in <module>
      symlink(cache_ent, args.o)
  NameError: name 'symlink' is not defined

Change-Id: Ie0d6b6dbdd211f830a4443c98bd5ea3b3ab56e3e
2013-05-14 15:47:16 +09:00
Shawn Pearce
1b89f85874 Support downloading from other Maven repositories
Developers may now request buck to use a local mirror by setting
the URL into local.properties, a file already ignored under the
current buck build process, e.g.:

  download.GERRIT = http://nexus.my-company.com/
  download.MAVEN_CENTRAL = http://nexus.my-company.com/

To support this usage buck now passes to download_jar.py only the name
of the repository in the URL.  download_jar parses local.properties
(if present) and replaces the "MAVEN_CENTRAL:" prefix with the URL
supplied by the user.

Because the URL can vary cache entries in buck-cache are now using the
SHA-1 we expect/verify against, instead of the SHA-1 of the URL.  This
makes it easier to find a specific JAR in the cache.  The artifact,
version and SHA-1 as named in the maven_jar() rule are the strings
used in the file name in buck-cache.

If no SHA-1 verification happens (e.g.  source attachment JARs that
are not included into the build) then sha1(canonical_url) is used.
Here the canonical URL uses the repository id prefix string, e.g.:

  MAVEN_CENTRAL:org/apache/mina/mina-core/2.0.5/mina-core-2.0.5-sources.jar

Change-Id: I5a469ab15fd36b81bf76f6b51e1110dfdcf04c86
2013-05-13 23:45:13 -07:00
Shawn Pearce
7e2eff6e4d Use --proxy-anyauth when running curl
This option should permit curl to pick up proxy authentication
information from ~/.netrc rather than requiring it to be embedded into
the http_proxy environment variable.  This is important on shared user
systems where the environment of a process is typically readable by
any user, while ~/.netrc is usually set with a mode 0600.

Change-Id: I995e6c8e9677dbda14a4e79942eed6682e381c36
2013-05-13 19:47:49 -07:00
Dave Borowitz
f62ed18438 Fix broken print function in download_jar.py
Change-Id: I1b8459166115ae201bf20d3387e35b5db8a11225
2013-05-13 15:42:58 -07:00
Shawn Pearce
c676e446ce Merge "Python3 support" 2013-05-13 16:38:17 +00:00
Chirayu Desai
4c5ee48f47 Python3 support
* Mostly involves changing print to be a function, and adding
  "from __future__ import print_function" for copatibility.

Change-Id: I3129233726e0116a348753a2e2bb68806a08668c
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-05-13 14:07:42 +05:30
David Pursehouse
bae05fd5c1 Print the url of dependency JARs being downloaded in buck build
When the build is downloading a lot of dependency JARs, there is
no output on the console.

Print the URL of the file that is being downloaded.

Change-Id: Id1bbecf3c38818b6bffb23cf0bef13ee00002a47
2013-05-10 10:25:14 -07:00
David Pursehouse
2879877ef4 Catch CalledProcessError when downloading dependencies with buck
When calling curl with check_call, CalledProcessError can be
raised.  Catch it to avoid an ugly python traceback in the build
log.

Change-Id: Icd56b94391fc6fb51e4d84889b123e892d15bc5b
2013-05-09 15:49:12 +01:00
Shawn Pearce
fd6bb9f6a5 Build with Buck
Implement a new build system using Buck[1], Facebook's
open source clone of Google's internal build system.

  Pros:
  - Concise build language
  - Test and build output is concise
  - Test failures and stack traces show on terminal
  - Reliable incrementals; clean is unnecessary
  - Extensible with simple blocks of Python
  - Fast
      buck: clean: 0.452s, full 1m21.083s [*], no-op:  7.145s,
      mvn:  clean: 4.596s, full 2m53.776s,     no-op: 59.108s,

      [*] full build includes downloading all dependencies,
          time can vary due to remote server performance.

  Cons:
  - No Windows support
  - No native Maven Central support (added by macros)
  - No native GWT, Prolog, or WAR support (added by macros)
  - Bootstrap of buck requires Ant

Getting started:

  git clone https://gerrit.googlesource.com/buck
  cd buck
  ant

  Mac OS X:
    PATH="`pwd`/bin:/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands:$PATH"

  Linux:
    PATH="`pwd`/bin:$PATH"

Importing into Eclipse:

  $ time buck build :eclipse
  0m48.949s

  Import existing project from `pwd`
  Import 'gerrit' (do not import other Maven based projects)
  Expand 'gerrit'
  Right click 'buck-out' > Properties
  Under Attributes check 'Derived'

  If the code doesn't currently compile but an updated classpath
  is needed, refresh the configs and obtain missing JARs:

  $ buck build :eclipse_project :download

Running JUnit tests:

  $ time buck test --all -e slow  # skip slow tests
  0m19.320s

  $ time buck test --all          # includes acceptance tests
  5m17.517s

Building WAR:

  $ buck build :gerrit
  $ java -jar buck-out/gen/gerrit.war

Building release:

  $ buck test --all && buck build :api :release
  $ java -jar buck-out/gen/release.war
  $ ls -lh buck-out/gen/{extension,plugin}-api.jar

Downloading dependencies:

  Dependencies are normally downloaded automatically, but Buck can
  inspect its graph and download missing dependencies so future
  compiles can run without the network:

  $ buck build :download

[1] http://facebook.github.io/buck/

Change-Id: I40853b108bd8e153cefa0896a5280a9a5ff81655
2013-05-09 13:40:36 +00:00