tools/maven: generate Maven install scripts
Build actions can be cached, so they can't be used for Maven interactions. Instead, generate a script that calls buck and then runs the upload process. This obviates tools/maven/api.py. The script tools/maven/api.sh runs all deployment steps combined (generate script, generate artifacts, upload to maven.) Tested: Ran "python tools/maven/api.py -n deploy" before and "buck build //tools/maven:gen_api_deploy" afterwards. The old command line and generated script differ as expected: * paths in the script are absolute, because Buck expands $(location) to absolute paths. * the ordering of arguments differs, because Python dict iteration was random beforehand. * "-o" missing afterward, as expected. The same check was performed for gen_api_install. Change-Id: Ia246000f8b59e881c53265751e2ebcfe8ec7d433
This commit is contained in:
Han-Wen Nienhuys
parent
221415d298
commit
2e7f5ccb7c
@ -1,9 +1,5 @@
|
||||
[alias]
|
||||
api = //:api
|
||||
api_deploy = //tools/maven:api_deploy
|
||||
api_install = //tools/maven:api_install
|
||||
war_deploy = //tools/maven:war_deploy
|
||||
war_install = //tools/maven:war_install
|
||||
chrome = //:chrome
|
||||
docs = //Documentation:searchfree
|
||||
firefox = //:firefox
|
||||
@ -34,4 +30,3 @@
|
||||
|
||||
[test]
|
||||
excluded_labels = manual
|
||||
|
||||
|
||||
@ -161,13 +161,13 @@ project directories in `buck-out/gen`, here as example for plugin API:
|
||||
Install {extension,plugin,gwt}-api to the local maven repository:
|
||||
|
||||
----
|
||||
buck build api_install
|
||||
sh tools/maven/api.sh install
|
||||
----
|
||||
|
||||
Install gerrit.war to the local maven repository:
|
||||
|
||||
----
|
||||
buck build war_install
|
||||
sh tools/maven/api.sh war_install
|
||||
----
|
||||
|
||||
=== Plugins
|
||||
@ -613,7 +613,7 @@ buck test --no-results-cache
|
||||
The following tests should be executed, when Buck version is upgraded:
|
||||
|
||||
* buck build release
|
||||
* buck build api_install
|
||||
* tools/maven/api.sh install
|
||||
* buck test
|
||||
* buck build gerrit, change some sources in gerrit-server project,
|
||||
repeat buck build gerrit and verify that gerrit.war was updated
|
||||
@ -642,30 +642,6 @@ Watchman can either be de-installed or disabled. See
|
||||
link:#buck-daemon[Using Buck daemon] section above how to temporarily
|
||||
disable `buckd`.
|
||||
|
||||
=== Re-triggering rule execution
|
||||
|
||||
There is no way to re-trigger custom rules with side effects, like
|
||||
`api_{deploy|install}`. This is a `genrule()` that depends on Java sources
|
||||
and is deploying the Plugin API through custom Python script to the local or
|
||||
remote Maven repositories. When for some reasons the deployment was undone,
|
||||
there is no supported way to re-trigger the execution of `api_{deploy|install}`
|
||||
targets. That's because `--no-cache` option will ignore the `Buck` cache, but
|
||||
there is no way to ignore `buck-out` directory. To overcome this Buck's design
|
||||
limitation new `tools/maven/api.py` script was added, that always re-triggers
|
||||
installation or deployment of Plugin API to local or Central Maven repository.
|
||||
|
||||
```
|
||||
tools/maven/api.py {deploy|install}
|
||||
```
|
||||
|
||||
Dry run mode is also supported:
|
||||
|
||||
```
|
||||
tools/maven/api.py -n {deploy|install}
|
||||
```
|
||||
|
||||
With this script the deployment would re-trigger on every invocation.
|
||||
|
||||
== Troubleshooting Buck
|
||||
|
||||
In some cases problems with Buck itself need to be investigated. See for example
|
||||
|
||||
@ -191,26 +191,19 @@ Versions and Create Release Tag] section.
|
||||
* Push the WAR to Maven Central:
|
||||
+
|
||||
----
|
||||
buck build war_deploy
|
||||
sh tools/maven/api.sh war_deploy
|
||||
----
|
||||
|
||||
* Push the plugin artifacts to Maven Central:
|
||||
+
|
||||
----
|
||||
buck build api_deploy
|
||||
----
|
||||
+
|
||||
For troubleshooting, the environment variable `VERBOSE` can be set. This
|
||||
prints out the commands that are executed by the Buck build process:
|
||||
+
|
||||
----
|
||||
VERBOSE=1 buck build api_deploy
|
||||
sh tools/maven/api.sh deploy
|
||||
----
|
||||
+
|
||||
If no artifacts are uploaded, clean the `buck-out` folder and retry:
|
||||
+
|
||||
----
|
||||
rm -rf buck-out
|
||||
buck clean ; rm -rf buck-out
|
||||
----
|
||||
|
||||
* Push the plugin Maven archetypes to Maven Central:
|
||||
|
||||
@ -1 +1 @@
|
||||
Subproject commit 3e801bd7d488c0b750422b32e4d4729beafcc00c
|
||||
Subproject commit 97c7ccfd62028c0fd7cb88db5567a0f20fc9f09b
|
||||
@ -1 +1 @@
|
||||
Subproject commit fb05fb988e17b1a2eb7ef476b8613419312afabd
|
||||
Subproject commit 1f743e5a877a8ef97b1f3a9e72933c0d87e11d6e
|
||||
@ -1 +1 @@
|
||||
Subproject commit 8db7117d509498c7d88979034afbc67220d75f6c
|
||||
Subproject commit d5cd908c0d7938a5d253d49f68ed352bbc7449cf
|
||||
@ -1,78 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# Copyright (C) 2015 The Android Open Source Project
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import print_function
|
||||
from argparse import ArgumentParser
|
||||
from json import loads
|
||||
from os import environ, path, remove
|
||||
from subprocess import check_call, check_output, Popen, PIPE
|
||||
from sys import stderr
|
||||
from tempfile import mkstemp
|
||||
|
||||
|
||||
def locations():
|
||||
d = Popen('buck audit dependencies api'.split(),
|
||||
stdin=None, stdout=PIPE, stderr=PIPE)
|
||||
t = Popen('xargs buck targets --show_output'.split(),
|
||||
stdin=d.stdout, stdout=PIPE, stderr=PIPE)
|
||||
out = t.communicate()[0]
|
||||
d.wait()
|
||||
targets = []
|
||||
outs = []
|
||||
for e in out.strip().split('\n'):
|
||||
t, o = e.split()
|
||||
targets.append(t)
|
||||
outs.append(o)
|
||||
return dict(zip(targets, outs))
|
||||
|
||||
parser = ArgumentParser()
|
||||
parser.add_argument('-n', '--dryrun', action='store_true')
|
||||
parser.add_argument('-v', '--verbose', action='store_true')
|
||||
|
||||
subparsers = parser.add_subparsers(help='action', dest='action')
|
||||
subparsers.add_parser('deploy', help='Deploy to Maven (remote)')
|
||||
subparsers.add_parser('install', help='Install to Maven (local)')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
root = path.abspath(__file__)
|
||||
while not path.exists(path.join(root, '.buckconfig')):
|
||||
root = path.dirname(root)
|
||||
|
||||
if not args.dryrun:
|
||||
check_call('buck build api'.split())
|
||||
target = check_output(('buck targets --json api_%s' % args.action).split())
|
||||
|
||||
s = loads(target)[0]['cmd']
|
||||
|
||||
fd, tempfile = mkstemp()
|
||||
s = s.replace('$(exe //tools/maven:mvn)', path.join(root, 'tools/maven/mvn.py'))
|
||||
s = s.replace('-o $OUT', '-o %s' % tempfile)
|
||||
|
||||
locations = locations()
|
||||
|
||||
while '$(location' in s:
|
||||
start = s.index('$(location')
|
||||
end = s.index(')', start)
|
||||
target = s[start+11:end]
|
||||
s = s.replace(s[start:end+1], locations[target])
|
||||
|
||||
try:
|
||||
if args.verbose or args.dryrun or environ.get('VERBOSE'):
|
||||
print(s, file=stderr)
|
||||
if not args.dryrun:
|
||||
check_call(s.split())
|
||||
finally:
|
||||
remove(tempfile)
|
||||
69
tools/maven/api.sh
Executable file
69
tools/maven/api.sh
Executable file
@ -0,0 +1,69 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Copyright (C) 2015 The Android Open Source Project
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
if [[ "$#" == "0" ]] ; then
|
||||
cat <<EOF
|
||||
Usage: run "$0 COMMAND" from the top of your workspace, where
|
||||
COMMAND is one of
|
||||
|
||||
install
|
||||
deploy
|
||||
war_install
|
||||
war_deploy
|
||||
|
||||
Set VERBOSE in the environment to get more information.
|
||||
|
||||
EOF
|
||||
|
||||
exit 1
|
||||
fi
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
|
||||
|
||||
case "$1" in
|
||||
install)
|
||||
command="api_install"
|
||||
;;
|
||||
deploy)
|
||||
command="api_deploy"
|
||||
;;
|
||||
war_install)
|
||||
command="war_install"
|
||||
;;
|
||||
war_deploy)
|
||||
command="war_deploy"
|
||||
;;
|
||||
*)
|
||||
echo "unknown command $1"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
if [[ "${VERBOSE:-x}" != "x" ]]; then
|
||||
set -o xtrace
|
||||
fi
|
||||
|
||||
buck build //tools/maven:gen_${command} || \
|
||||
{ echo "buck failed to build gen_${command}. Use VERBOSE=1 for more info" ; exit 1 ; }
|
||||
|
||||
script="./buck-out/gen/tools/maven/gen_${command}/${command}.sh"
|
||||
|
||||
# The PEX wrapper does some funky exit handling, so even if the script
|
||||
# does "exit(0)", the return status is '1'. So we can't tell if the
|
||||
# following invocation was successful.
|
||||
${script}
|
||||
@ -69,7 +69,12 @@ for spec in args.s:
|
||||
file=stderr)
|
||||
exit(1)
|
||||
|
||||
with open(args.o, 'w') as fd:
|
||||
|
||||
out = stderr
|
||||
if args.o:
|
||||
out = open(args.o, 'w')
|
||||
|
||||
with out as fd:
|
||||
if args.repository:
|
||||
print('Repository: %s' % args.repository, file=fd)
|
||||
if args.url:
|
||||
|
||||
@ -12,6 +12,19 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
sh_bang_template = (' && '.join([
|
||||
"echo '#!/bin/sh -eu' > $OUT",
|
||||
'echo "# this script should run from the root of your workspace." >> $OUT',
|
||||
'echo "" >> $OUT',
|
||||
"echo 'if [[ -n \"$${VERBOSE:-}\" ]]; then set -x ; fi' >> $OUT",
|
||||
'echo "" >> $OUT',
|
||||
'echo %s >> $OUT',
|
||||
'echo "" >> $OUT',
|
||||
'echo %s >> $OUT',
|
||||
# This is supposed to be handled by executable=True, but it doesn't
|
||||
# work. Bug?
|
||||
'chmod +x $OUT' ]))
|
||||
|
||||
def maven_package(
|
||||
version,
|
||||
repository = None,
|
||||
@ -20,44 +33,61 @@ def maven_package(
|
||||
src = {},
|
||||
doc = {},
|
||||
war = {}):
|
||||
cmd = ['$(exe //tools/maven:mvn)', '-v', version, '-o', '$OUT']
|
||||
api_cmd = []
|
||||
|
||||
build_cmd = ['buck', 'build']
|
||||
|
||||
mvn_cmd = ['$(exe //tools/maven:mvn)', '-v', version]
|
||||
api_cmd = mvn_cmd[:]
|
||||
api_targets = [ '//tools/maven:mvn' ]
|
||||
for type,d in [('jar', jar), ('java-source', src), ('javadoc', doc)]:
|
||||
for a,t in d.iteritems():
|
||||
for a,t in sorted(d.iteritems()):
|
||||
api_cmd.append('-s %s:%s:$(location %s)' % (a,type,t))
|
||||
api_targets.append(t)
|
||||
|
||||
genrule(
|
||||
name = 'api_install',
|
||||
cmd = ' '.join(cmd + api_cmd + ['-a', 'install']),
|
||||
out = 'api_install.info',
|
||||
name = 'gen_api_install',
|
||||
cmd = sh_bang_template % (
|
||||
' '.join(build_cmd + api_targets),
|
||||
' '.join(api_cmd + ['-a', 'install'])),
|
||||
out = 'api_install.sh',
|
||||
executable = True,
|
||||
)
|
||||
|
||||
if repository and url:
|
||||
genrule(
|
||||
name = 'api_deploy',
|
||||
cmd = ' '.join(cmd + api_cmd + [
|
||||
'-a', 'deploy',
|
||||
'--repository', repository,
|
||||
'--url', url]),
|
||||
out = 'api_deploy.info',
|
||||
name = 'gen_api_deploy',
|
||||
cmd = sh_bang_template % (
|
||||
' '.join(build_cmd + api_targets),
|
||||
' '.join(api_cmd + ['-a', 'deploy',
|
||||
'--repository', repository,
|
||||
'--url', url])),
|
||||
out = 'api_deploy.sh',
|
||||
executable = True,
|
||||
)
|
||||
|
||||
war_cmd = []
|
||||
for a,t in war.iteritems():
|
||||
war_cmd = mvn_cmd[:]
|
||||
war_targets = [ '//tools/maven:mvn' ]
|
||||
for a,t in sorted(war.iteritems()):
|
||||
war_cmd.append('-s %s:war:$(location %s)' % (a,t))
|
||||
war_targets.append(t)
|
||||
|
||||
genrule(
|
||||
name = 'war_install',
|
||||
cmd = ' '.join(cmd + war_cmd + ['-a', 'install']),
|
||||
out = 'war_install.info',
|
||||
name = 'gen_war_install',
|
||||
cmd = sh_bang_template % (' '.join(build_cmd + war_targets),
|
||||
' '.join(war_cmd + ['-a', 'install'])),
|
||||
out = 'war_install.sh',
|
||||
executable = True,
|
||||
)
|
||||
|
||||
if repository and url:
|
||||
genrule(
|
||||
name = 'war_deploy',
|
||||
cmd = ' '.join(cmd + war_cmd + [
|
||||
name = 'gen_war_deploy',
|
||||
cmd = sh_bang_template % (
|
||||
' '.join(build_cmd + war_targets),
|
||||
' '.join(war_cmd + [
|
||||
'-a', 'deploy',
|
||||
'--repository', repository,
|
||||
'--url', url]),
|
||||
out = 'war_deploy.info',
|
||||
'--url', url])),
|
||||
out = 'war_deploy.sh',
|
||||
executable = True,
|
||||
)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user