Buck: Allow to trigger Maven deployment even when nothing changed

Buck extensively uses caching and storing metadata in buck-out
directory, so that it's not possible per design to re-trigger the
execution of custom rule, without wiping out the whole buck-out
directory. See also the discussion on this issue: [1].

The implementation of Maven deployment as a custom build rule with
a side effect is wrong approach to start with. It was only done as a
workaround, because buck doesn't offer `install` or `publish` command
that must not be free of side effects like it's the case with `build`
command. Having side effects with `build` command breaks bucks model.

As workaround for now add standalone Python script, that re-uses Buck
api_{deploy|install} targets, resolves $(location <target>) macros and
executes the deployment by calling mvn.py utility directly:

  $ tools/maven/api.py {deploy,install}

Dry run mode is supported as well:

  $ tools/maven/api.py -n {deploy,install}

[1] https://github.com/facebook/buck/issues/342

Change-Id: I7fb86ad6967a1fa1e7ac842ba5e0e8cf0103b773
This commit is contained in:
David Ostrovsky 2015-06-17 00:25:59 +02:00 committed by David Ostrovsky
parent 89e8457b2b
commit 66f3429c98
3 changed files with 102 additions and 0 deletions

View File

@ -670,6 +670,30 @@ Watchman can either be de-installed or disabled. See
link:#buck-daemon[Using Buck daemon] section above how to temporarily
disable `buckd`.
=== Re-triggering rule execution
There is no way to re-trigger custom rules with side effects, like
`api_{deploy|install}`. This is a `genrule()` that depends on Java sources
and is deploying the Plugin API through custom Python script to the local or
remote Maven repositories. When for some reasons the deployment was undone,
there is no supported way to re-trigger the execution of `api_{deploy|install}`
targets. That's because `--no-cache` option will ignore the `Buck` cache, but
there is no way to ignore `buck-out` directory. To overcome this Buck's design
limitation new `tools/maven/api.py` script was added, that always re-triggers
installation or deployment of Plugin API to local or Central Maven repository.
```
tools/maven/api.py {deploy|install}
```
Dry run mode is also supported:
```
tools/maven/api.py -n {deploy|install}
```
With this script the deployment would re-trigger on every invocation.
== Troubleshooting Buck
In some cases problems with Buck itself need to be investigated. See for example

78
tools/maven/api.py Executable file
View File

@ -0,0 +1,78 @@
#!/usr/bin/env python
# Copyright (C) 2015 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import print_function
from argparse import ArgumentParser
from json import loads
from os import environ, path, remove
from subprocess import check_call, check_output, Popen, PIPE
from sys import stderr
from tempfile import mkstemp
def locations():
d = Popen('buck audit dependencies api'.split(),
stdin=None, stdout=PIPE, stderr=PIPE)
t = Popen('xargs buck targets --show_output'.split(),
stdin=d.stdout, stdout=PIPE, stderr=PIPE)
out = t.communicate()[0]
d.wait()
targets = []
outs = []
for e in out.strip().split('\n'):
t, o = e.split()
targets.append(t)
outs.append(o)
return dict(zip(targets, outs))
parser = ArgumentParser()
parser.add_argument('-n', '--dryrun', action='store_true')
parser.add_argument('-v', '--verbose', action='store_true')
subparsers = parser.add_subparsers(help='action', dest='action')
subparsers.add_parser('deploy', help='Deploy to Maven (remote)')
subparsers.add_parser('install', help='Install to Maven (local)')
args = parser.parse_args()
root = path.abspath(__file__)
while not path.exists(path.join(root, '.buckconfig')):
root = path.dirname(root)
if not args.dryrun:
check_call('buck build api'.split())
target = check_output(('buck targets --json api_%s' % args.action).split())
s = loads(target)[0]['cmd']
fd, tempfile = mkstemp()
s = s.replace('$(exe //tools/maven:mvn)', path.join(root, 'tools/maven/mvn.py'))
s = s.replace('-o $OUT', '-o %s' % tempfile)
locations = locations()
while '$(location' in s:
start = s.index('$(location')
end = s.index(')', start)
target = s[start+11:end]
s = s.replace(s[start:end+1], locations[target])
try:
if args.verbose or args.dryrun or environ.get('VERBOSE'):
print(s, file=stderr)
if not args.dryrun:
check_call(s.split())
finally:
remove(tempfile)

0
tools/maven/mvn.py Normal file → Executable file
View File