From fa2486a39739dc103b3f57048f3f84ca8764779c Mon Sep 17 00:00:00 2001 From: "Shawn O. Pearce" Date: Wed, 11 Nov 2009 14:51:30 -0800 Subject: [PATCH] Embed Jetty and run it out of `java -jar gerrit.war daemon` We now include Jetty 7.x as part of our distribution WAR and allow administrators to launch the embedded copy using our Daemon command line program. This vastly simplifies server installation as you do not need to download the Jetty servlet container and configure it prior to using Gerrit Code Review. Configuration of Jetty is performed using our gerrit.config, rather than the Jetty XML configuration language. Jetty's language can be quite a bit more powerful for isoteric configurations, but we want to embed Jetty to simplify setup, not complicate it. Gerrit's own configuration file in the git syntax is simpler, so we stick with that and offer a limited subset of Jetty's configuration features through httpd.* properties. Logging currently goes only to stderr, which is fine for toy setups but probably not ideal for a production environment. Unfortunately we are still relying on GerritServer.properties to get us the database connection information, and this DataSource is not pooled. Thus the web handlers in our embedded container will run slightly slower then the web handlers in a standalone container, as the latter will be likely be using a connection pool configured out of the container's JNDI namespace. Bug: issue 202 Change-Id: Ia7956ba48cbc3a8d4241101db5ff493e16d26b9f Signed-off-by: Shawn O. Pearce --- Documentation/config-gerrit.txt | 139 ++++++++ Documentation/licenses.txt | 45 ++- Documentation/pgm-daemon.txt | 23 +- Documentation/pgm-index.txt | 2 +- gerrit-gwtdebug/pom.xml | 6 + .../httpd/HttpCanonicalWebUrlProvider.java | 4 +- .../com/google/gerrit/httpd/WebModule.java | 2 +- gerrit-pgm/pom.xml | 16 + .../google/gerrit/pgm/AbstractProgram.java | 17 +- .../java/com/google/gerrit/pgm/Daemon.java | 147 ++++++++- .../main/java/com/google/gerrit/pgm/Die.java | 25 ++ .../google/gerrit/pgm/RuntimeShutdown.java | 114 +++++++ .../gerrit/pgm/http/jetty/JettyEnv.java | 25 ++ .../gerrit/pgm/http/jetty/JettyModule.java | 38 +++ .../gerrit/pgm/http/jetty/JettyServer.java | 304 ++++++++++++++++++ .../gerrit/lifecycle/LifecycleManager.java | 3 + .../server/config/GerritGlobalModule.java | 28 -- gerrit-war/pom.xml | 3 +- .../gerrit/httpd/WebAppInitializer.java | 26 +- gerrit-war/src/main/java/log4j.properties | 6 + pom.xml | 7 + 21 files changed, 908 insertions(+), 72 deletions(-) create mode 100644 gerrit-pgm/src/main/java/com/google/gerrit/pgm/Die.java create mode 100644 gerrit-pgm/src/main/java/com/google/gerrit/pgm/RuntimeShutdown.java create mode 100644 gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyEnv.java create mode 100644 gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyModule.java create mode 100644 gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyServer.java diff --git a/Documentation/config-gerrit.txt b/Documentation/config-gerrit.txt index b14ea5fe6b..b55059a638 100644 --- a/Documentation/config-gerrit.txt +++ b/Documentation/config-gerrit.txt @@ -503,6 +503,145 @@ Gerrit appends any necessary query arguments onto the end of this URL. For example, "?p=$project.git;h=$commit". +[[httpd]] Section httpd +~~~~~~~~~~~~~~~~~~~~~~~ + +The httpd section configures the embedded servlet container. + +[[httpd.listenUrl]]httpd.listenUrl:: ++ +Specifies the URLs the internal HTTP daemon should listen for +connections on. The special hostname '\*' may be used to listen +on all local addresses. A context path may optionally be included, +placing Gerrit Code Review's web address within a subdirectory of +the server. ++ +Multiple protocol schemes are supported: ++ +* `http://`'hostname'`:`'port' ++ +Plain-text HTTP protocol. If port is not supplied, defaults to 80, +the standard HTTP port. ++ +* `https://`'hostname'`:`'port' ++ +SSL encrypted HTTP protocol. If port is not supplied, defaults to +443, the standard HTTPS port. ++ +Externally facing production sites are encouraged to use a reverse +proxy configuration and `proxy-https://` (below), rather than using +the embedded servlet container to implement the SSL processing. +The proxy server with SSL support is probably easier to configure, +provides more configuration options to control cipher usage, and +is likely using natively compiled encryption algorithms, resulting +in higher throughput. ++ +* `proxy-http://`'hostname'`:`'port' ++ +Plain-text HTTP relayed from a reverse proxy. If port is not +supplied, defaults to 8080. ++ +Like http, but additional header parsing features are +enabled to honor X-Forwarded-For, X-Forwarded-Host and +X-Forwarded-Server. These headers are typically set by Apache's +link:http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#x-headers[mod_proxy]. ++ +* `proxy-https://`'hostname'`:`'port' ++ +Plain text HTTP relayed from a reverse proxy that has already +handled the SSL encryption/decryption. If port is not supplied, +defaults to 8080. ++ +Behaves exactly like proxy-http, but also sets the scheme to assume +'https://' is the proper URL back to the server. + ++ +If multiple values are supplied, the daemon will listen on all +of them. ++ +By default, http://*:8080. + +[[httpd.reuseAddress]]httpd.reuseAddress:: ++ +If true, permits the daemon to bind to the port even if the port +is already in use. If false, the daemon ensures the port is not +in use before starting. Busy sites may need to set this to true +to permit fast restarts. ++ +By default, true. + +[[httpd.requestHeaderSize]]httpd.requestHeaderSize:: ++ +Size, in bytes, of the buffer used to parse the HTTP headers of an +incoming HTTP request. The entire request headers, including any +cookies sent by the browser, must fit within this buffer, otherwise +the server aborts with the response '413 Request Entity Too Large'. ++ +One buffer of this size is allocated per active connection. +Allocating a buffer that is too large wastes memory that cannot be +reclaimed, allocating a buffer that is too small may cause unexpected +errors caused by very long Referer URLs or large cookie values. ++ +By default, 16384 (16 K), which is sufficient for most OpenID and +other web-based single-sign-on integrations. + +[[httpd.sslKeyStore]]httpd.sslKeyStore:: ++ +Path of the Java keystore containing the server's SSL certificate +and private key. This keystore is required for `https://` in URL. ++ +To create a self-signed certificate for simple internal usage: ++ +==== + keytool -keystore keystore -alias jetty -genkey -keyalg RSA + chmod 600 keystore +==== ++ +If not absolute, the path is resolved relative to `$site_path`. ++ +By default, `$site_path/keystore`. + +[[httpd.sslKeyPassword]]httpd.sslKeyPassword:: ++ +Password used to decrypt the private portion of the sslKeyStore. +Java key stores require a password, even if the administrator +doesn't want to enable one. ++ +If set to the empty string the embedded server will prompt for the +password during startup. ++ +By default, `gerrit`. + +[[httpd.acceptorThreads]]httpd.acceptorThreads:: ++ +Number of worker threads dedicated to accepting new incoming TCP +connections and allocate them connection-specific resources. ++ +By default, 2, which should be suitable for most high-traffic sites. + +[[httpd.minThreads]]httpd.minThreads:: ++ +Minimum number of spare threads to keep in the worker thread pool. +This number must be at least 1 larger than httpd.acceptorThreads +multipled by the number of httpd.listenUrls configured. ++ +By default, 5, suitable for most lower-volume traffic sites. + +[[httpd.maxThreads]]httpd.maxThreads:: ++ +Maximum number of threads to permit in the worker thread pool. ++ +By default 25, suitable for most lower-volume traffic sites. + +[[httpd.maxQueued]]httpd.maxQueued:: ++ +Maximum number of client connections which can enter the worker +thread pool waiting for a worker thread to become available. +0 disables the queue and permits infinite number of connections. ++ +By default 50. + + [[ldap]]Section ldap ~~~~~~~~~~~~~~~~~~~~ diff --git a/Documentation/licenses.txt b/Documentation/licenses.txt index b7b889b52e..e2b138505f 100644 --- a/Documentation/licenses.txt +++ b/Documentation/licenses.txt @@ -1,15 +1,18 @@ Gerrit2 - Licenses ================== -Gerrit itself is licensed under the Apache License, 2.0. Gerrit -however also includes software covered by the Apache license, -the new style BSD license, and the MIT license. +Gerrit itself is licensed under the Apache License, 2.0. Gerrit's +executable distributions also include many other software components +covered by additional licenses. + +Included Components +------------------- [grid="all"] `---------------------------`------------------------------ Included Package License ----------------------------------------------------------- -Gerrit <> +Gerrit Code Review <> gwtexpui <> gwtjsonrpc <> gwtorm <> @@ -33,6 +36,8 @@ OpenXRI <> Neko HTML <> Ehcache <> mime-util <> +Jetty <>, or link:http://www.eclipse.org/legal/epl-v10.html[EPL] +Java Servlet API <> ICU4J <> JGit <> JSch <> @@ -81,10 +86,12 @@ and/or the link:http://www.bouncycastle.org/java.html[Bouncy Castle Crypto API] to be installed by the end-user. +Licenses +-------- [[apache2]] Apache License 2.0 ------------------- +~~~~~~~~~~~~~~~~~~ ---- Apache License @@ -290,9 +297,17 @@ Apache License 2.0 limitations under the License. ---- + +[[cddl]] +Common Development and Distribution License (CDDL) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* link:https://glassfish.dev.java.net/public/CDDLv1.0.html[Original] + + [[sshd]] Apache SSHD - Notice --------------------- +~~~~~~~~~~~~~~~~~~~~ * link:http://svn.apache.org/viewvc/mina/sshd/trunk/NOTICE.txt?view=markup[Original] @@ -356,7 +371,7 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. [[postgresql]] PostgreSQL JDBC Driver - New Style BSD --------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * link:http://jdbc.postgresql.org/license.html[Original] @@ -391,7 +406,7 @@ POSSIBILITY OF SUCH DAMAGE. [[h2]] H2 Database - EPL or modified MPL ---------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * link:http://www.h2database.com/html/license.html[Complete Terms] @@ -401,7 +416,7 @@ MPL 1.1 (Mozilla Public License) or under the (unmodified) EPL 1.0 [[asm]] ObjectWeb ASM - New Style BSD ------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * link:http://svn.forge.objectweb.org/cgi-bin/viewcvs.cgi/asm/trunk/asm/LICENSE.txt[Original] @@ -438,7 +453,7 @@ ObjectWeb ASM - New Style BSD [[antlr]] ANTLR - New Style BSD ---------------------- +~~~~~~~~~~~~~~~~~~~~~ * link:http://www.antlr.org/license.html[Original] @@ -476,7 +491,7 @@ POSSIBILITY OF SUCH DAMAGE. [[jgit]] JGit - New Style BSD --------------------- +~~~~~~~~~~~~~~~~~~~~ * link:http://repo.or.cz/w/egit.git?a=blob;f=org.spearce.jgit/LICENSE;hb=HEAD[Original] @@ -517,14 +532,14 @@ JGit - New Style BSD [[args4j]] args4j - MIT License --------------------- +~~~~~~~~~~~~~~~~~~~~ * link:https://args4j.dev.java.net/[Home] * link:http://www.opensource.org/licenses/mit-license.php[Canonical MIT License] [[slf4j]] SLF4J - MIT License -------------------- +~~~~~~~~~~~~~~~~~~~ * link:http://www.slf4j.org/license.html[Original] @@ -554,7 +569,7 @@ SLF4J - MIT License [[clippy]] Clippy - MIT License --------------------- +~~~~~~~~~~~~~~~~~~~~ * link:http://github.com/mojombo/clippy/tree/master[Site] @@ -585,7 +600,7 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. [[icu4j]] ICU4J License -------------- +~~~~~~~~~~~~~ * link:http://source.icu-project.org/repos/icu/icu4j/tags/release-3-4-4/license.html[Original] diff --git a/Documentation/pgm-daemon.txt b/Documentation/pgm-daemon.txt index 7a96735227..da2f04fb8a 100644 --- a/Documentation/pgm-daemon.txt +++ b/Documentation/pgm-daemon.txt @@ -3,17 +3,20 @@ daemon NAME ---- -daemon - Git enabled SSH daemon +daemon - Gerrit network server SYNOPSIS -------- [verse] -'java' -DGerritServer=PATH -jar gerrit.war 'daemon' [\--slave] +'java' -DGerritServer=PATH -jar gerrit.war 'daemon' + [\--enable-httpd | \--disable-httpd] + [\--enable-sshd | \--disable-sshd] + [\--slave] DESCRIPTION ----------- -Runs the Gerrit SSH daemon on the local system, configured as per -the local copy of link:config-gerrit.html[gerrit.config]. +Runs the Gerrit network daemon on the local system, configured as +per the local copy of link:config-gerrit.html[gerrit.config]. The path to gerrit.config is read from the metadata database, which requires that all slaves (and master) reading from the same @@ -34,12 +37,24 @@ OPTIONS java -jar gerrit.war \--cat extra/GerritServer.properties_example >GerritServer.properties ==== +\--enable-httpd:: +\--disable-httpd:: + Enable (or disable) the internal HTTP daemon, answering + web requests. Enabled by default. + +\--enable-sshd:: +\--disable-sshd:: + Enable (or disable) the internal SSH daemon, answering SSH + clients and remotely executed commands. Enabled by default. + \--slave:: Run in slave mode, permitting only read operations by clients. Commands which modify state such as link:cmd-receive-pack.html[recieve-pack] (creates new changes or updates existing ones) or link:cmd-approve.html[approve] (sets approve marks) are disabled. ++ +This option automatically implies '\--disable-httpd \--enable-sshd'. CONTEXT ------- diff --git a/Documentation/pgm-index.txt b/Documentation/pgm-index.txt index d89771c1f2..00246bb111 100644 --- a/Documentation/pgm-index.txt +++ b/Documentation/pgm-index.txt @@ -13,7 +13,7 @@ CreateSchema:: Initialize a new database schema. link:pgm-daemon.html[daemon]:: - Git enabled SSH daemon. + Gerrit HTTP, SSH network server. version:: Display the release version of Gerrit Code Review. diff --git a/gerrit-gwtdebug/pom.xml b/gerrit-gwtdebug/pom.xml index 7173e98e90..e868c4e7d6 100644 --- a/gerrit-gwtdebug/pom.xml +++ b/gerrit-gwtdebug/pom.xml @@ -47,6 +47,12 @@ limitations under the License. com.google.gerrit gerrit-war + + + com.google.gerrit + gerrit-pgm + + diff --git a/gerrit-httpd/src/main/java/com/google/gerrit/httpd/HttpCanonicalWebUrlProvider.java b/gerrit-httpd/src/main/java/com/google/gerrit/httpd/HttpCanonicalWebUrlProvider.java index a28abfe485..5df678b76b 100644 --- a/gerrit-httpd/src/main/java/com/google/gerrit/httpd/HttpCanonicalWebUrlProvider.java +++ b/gerrit-httpd/src/main/java/com/google/gerrit/httpd/HttpCanonicalWebUrlProvider.java @@ -27,7 +27,7 @@ import org.eclipse.jgit.lib.Config; import javax.servlet.http.HttpServletRequest; /** Sets {@link CanonicalWebUrl} to current HTTP request if not configured. */ -class HttpCanonicalWebUrlProvider extends CanonicalWebUrlProvider { +public class HttpCanonicalWebUrlProvider extends CanonicalWebUrlProvider { private Provider requestProvider; @Inject @@ -36,7 +36,7 @@ class HttpCanonicalWebUrlProvider extends CanonicalWebUrlProvider { } @Inject(optional = true) - void setHttpServletRequest(final Provider hsr) { + public void setHttpServletRequest(final Provider hsr) { requestProvider = hsr; } diff --git a/gerrit-httpd/src/main/java/com/google/gerrit/httpd/WebModule.java b/gerrit-httpd/src/main/java/com/google/gerrit/httpd/WebModule.java index f11d35c3ae..0558bb2741 100644 --- a/gerrit-httpd/src/main/java/com/google/gerrit/httpd/WebModule.java +++ b/gerrit-httpd/src/main/java/com/google/gerrit/httpd/WebModule.java @@ -42,7 +42,7 @@ import com.google.inject.servlet.ServletModule; import java.net.SocketAddress; -class WebModule extends FactoryModule { +public class WebModule extends FactoryModule { private final Provider sshInfoProvider; private final Provider sshKeyCacheProvider; private final AuthType loginType; diff --git a/gerrit-pgm/pom.xml b/gerrit-pgm/pom.xml index 3acf8b1d3d..1b7a39de65 100644 --- a/gerrit-pgm/pom.xml +++ b/gerrit-pgm/pom.xml @@ -48,6 +48,12 @@ limitations under the License. postgresql + + com.google.gerrit + gerrit-main + provided + + com.google.gerrit gerrit-util-cli @@ -62,5 +68,15 @@ limitations under the License. com.google.gerrit gerrit-sshd + + + com.google.gerrit + gerrit-httpd + + + + org.eclipse.jetty + jetty-servlet + diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/AbstractProgram.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/AbstractProgram.java index 4ec9756256..b78e587d9f 100644 --- a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/AbstractProgram.java +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/AbstractProgram.java @@ -68,13 +68,28 @@ public abstract class AbstractProgram { return 1; } - return run(); + try { + return run(); + } catch (Die err) { + System.err.println("fatal: " + err.getMessage()); + return 128; + } } private static Injector emptyInjector() { return Guice.createInjector(Collections. emptyList()); } + /** Create a new exception to indicate we won't continue. */ + protected static Die die(String why) { + return new Die(why); + } + + /** Create a new exception to indicate we won't continue. */ + protected static Die die(String why, Throwable cause) { + return new Die(why, cause); + } + /** Method that never returns, e.g. to keep a daemon running. */ protected int never() { synchronized (sleepLock) { diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Daemon.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Daemon.java index ebf582aa02..82b245637f 100644 --- a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Daemon.java +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Daemon.java @@ -14,46 +14,173 @@ package com.google.gerrit.pgm; +import static com.google.inject.Stage.PRODUCTION; + +import com.google.gerrit.httpd.HttpCanonicalWebUrlProvider; +import com.google.gerrit.httpd.WebModule; import com.google.gerrit.lifecycle.LifecycleManager; +import com.google.gerrit.pgm.http.jetty.JettyEnv; +import com.google.gerrit.pgm.http.jetty.JettyModule; +import com.google.gerrit.server.config.CanonicalWebUrlModule; +import com.google.gerrit.server.config.CanonicalWebUrlProvider; +import com.google.gerrit.server.config.DatabaseModule; +import com.google.gerrit.server.config.GerritConfigModule; import com.google.gerrit.server.config.GerritGlobalModule; import com.google.gerrit.server.config.MasterNodeStartup; import com.google.gerrit.sshd.SshModule; import com.google.gerrit.sshd.commands.MasterCommandModule; import com.google.gerrit.sshd.commands.SlaveCommandModule; +import com.google.inject.Guice; import com.google.inject.Injector; import com.google.inject.Module; +import com.google.inject.Provider; import org.kohsuke.args4j.Option; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.util.ArrayList; import java.util.List; -/** Run only the SSH daemon portions of Gerrit. */ +import javax.servlet.http.HttpServletRequest; + +/** Run SSH daemon portions of Gerrit. */ public class Daemon extends AbstractProgram { + private static final Logger log = LoggerFactory.getLogger(Daemon.class); + + @Option(name = "--enable-httpd", usage = "Enable the internal HTTP daemon") + private Boolean httpd; + + @Option(name = "--disable-httpd", usage = "Disable the internal HTTP daemon") + void setDisableHttpd(final boolean arg) { + httpd = false; + } + + @Option(name = "--enable-sshd", usage = "Enable the internal SSH daemon") + private boolean sshd = true; + + @Option(name = "--disable-sshd", usage = "Disable the internal SSH daemon") + void setDisableSshd(final boolean arg) { + sshd = false; + } @Option(name = "--slave", usage = "support fetch only") - boolean slave; + private boolean slave; + + private final LifecycleManager manager = new LifecycleManager(); + private Injector dbInjector; + private Injector cfgInjector; + private Injector sysInjector; + private Injector sshInjector; + private Injector webInjector; + private Injector httpdInjector; @Override public int run() throws Exception { - Injector sysInjector = GerritGlobalModule.createInjector(); - Injector sshInjector = createSshInjector(sysInjector); + if (httpd == null) { + httpd = !slave; + } - final LifecycleManager mgr = new LifecycleManager(); - mgr.add(sysInjector, sshInjector); - mgr.start(); - return never(); + if (!httpd && !sshd) { + throw die("No services enabled, nothing to do"); + } + if (slave && httpd) { + throw die("Cannot combine --slave and --enable-httpd"); + } + if (httpd && !sshd) { + // TODO Support HTTP without SSH. + throw die("--enable-httpd currently requires --enable-sshd"); + } + + dbInjector = Guice.createInjector(PRODUCTION, new DatabaseModule()); + cfgInjector = dbInjector.createChildInjector(new GerritConfigModule()); + sysInjector = createSysInjector(); + manager.add(dbInjector, cfgInjector, sysInjector); + + if (sshd) { + initSshd(); + } + + if (httpd) { + initHttpd(); + } + + manager.start(); + log.info("Gerrit Code Review " + myVersion() + " ready"); + RuntimeShutdown.add(new Runnable() { + public void run() { + log.info("caught shutdown, cleaning up"); + manager.stop(); + } + }); + RuntimeShutdown.waitFor(); + return 0; } - private Injector createSshInjector(final Injector sysInjector) { + private String myVersion() { + return com.google.gerrit.common.Version.getVersion(); + } + + private Injector createSysInjector() { + final List modules = new ArrayList(); + modules.add(cfgInjector.getInstance(GerritGlobalModule.class)); + if (httpd) { + modules.add(new CanonicalWebUrlModule() { + @Override + protected Class> provider() { + return HttpCanonicalWebUrlProvider.class; + } + }); + } else { + modules.add(new CanonicalWebUrlModule() { + @Override + protected Class> provider() { + return CanonicalWebUrlProvider.class; + } + }); + } + if (!slave) { + modules.add(new MasterNodeStartup()); + } + return cfgInjector.createChildInjector(modules); + } + + private void initSshd() { + sshInjector = createSshInjector(); + manager.add(sshInjector); + } + + private Injector createSshInjector() { final List modules = new ArrayList(); modules.add(new SshModule()); if (slave) { modules.add(new SlaveCommandModule()); } else { modules.add(new MasterCommandModule()); - modules.add(new MasterNodeStartup()); } return sysInjector.createChildInjector(modules); } + + private void initHttpd() { + webInjector = createWebInjector(); + + sysInjector.getInstance(HttpCanonicalWebUrlProvider.class) + .setHttpServletRequest( + webInjector.getProvider(HttpServletRequest.class)); + + httpdInjector = createHttpdInjector(); + manager.add(webInjector, httpdInjector); + } + + private Injector createWebInjector() { + final List modules = new ArrayList(); + modules.add(sshInjector.getInstance(WebModule.class)); + return sysInjector.createChildInjector(modules); + } + + private Injector createHttpdInjector() { + final List modules = new ArrayList(); + modules.add(new JettyModule(new JettyEnv(webInjector))); + return sysInjector.createChildInjector(modules); + } } diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Die.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Die.java new file mode 100644 index 0000000000..37886a6e02 --- /dev/null +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Die.java @@ -0,0 +1,25 @@ +// Copyright (C) 2009 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.google.gerrit.pgm; + +public class Die extends RuntimeException { + public Die(final String why) { + super(why); + } + + public Die(final String why, final Throwable cause) { + super(why, cause); + } +} diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/RuntimeShutdown.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/RuntimeShutdown.java new file mode 100644 index 0000000000..a97afa9741 --- /dev/null +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/RuntimeShutdown.java @@ -0,0 +1,114 @@ +// Copyright (C) 2009 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.google.gerrit.pgm; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.ArrayList; +import java.util.List; + +public class RuntimeShutdown { + private static final ShutdownCallback cb = new ShutdownCallback(); + + /** Add a task to be performed when graceful shutdown is requested. */ + public static void add(final Runnable task) { + if (!cb.add(task)) { + // If the shutdown has already begun we cannot enqueue a new + // task. Instead trigger the task in the caller, without any + // of our locks held. + // + task.run(); + } + } + + /** Wait for the JVM shutdown to occur. */ + public static void waitFor() { + cb.waitForShutdown(); + } + + private RuntimeShutdown() { + } + + private static class ShutdownCallback extends Thread { + private static final Logger log = + LoggerFactory.getLogger(ShutdownCallback.class); + + private final List tasks = new ArrayList(); + private boolean shutdownStarted; + private boolean shutdownComplete; + + ShutdownCallback() { + setName("ShutdownCallback"); + } + + boolean add(final Runnable newTask) { + synchronized (this) { + if (!shutdownStarted && !shutdownComplete) { + if (tasks.isEmpty()) { + Runtime.getRuntime().addShutdownHook(this); + } + tasks.add(newTask); + return true; + + } else { + // We don't permit adding a task once shutdown has started. + // + return false; + } + } + } + + @Override + public void run() { + log.debug("Graceful shutdown requested"); + + List taskList; + synchronized (this) { + shutdownStarted = true; + taskList = tasks; + } + + for (Runnable task : taskList) { + try { + task.run(); + } catch (Exception err) { + log.error("Cleanup task failed", err); + } + } + + log.debug("Shutdown complete"); + + synchronized (this) { + shutdownComplete = true; + notifyAll(); + } + } + + void waitForShutdown() { + synchronized (this) { + while (!shutdownComplete) { + try { + wait(); + } catch (InterruptedException e) { + log.warn("Thread " + Thread.currentThread().getName() + + " interrupted while waiting for graceful shutdown;" + + " ignoring interrupt request"); + } + } + } + } + } +} diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyEnv.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyEnv.java new file mode 100644 index 0000000000..ebca46700a --- /dev/null +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyEnv.java @@ -0,0 +1,25 @@ +// Copyright (C) 2009 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.google.gerrit.pgm.http.jetty; + +import com.google.inject.Injector; + +public class JettyEnv { + final Injector webInjector; + + public JettyEnv(final Injector webInjector) { + this.webInjector = webInjector; + } +} diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyModule.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyModule.java new file mode 100644 index 0000000000..1ae9355316 --- /dev/null +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyModule.java @@ -0,0 +1,38 @@ +// Copyright (C) 2009 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.google.gerrit.pgm.http.jetty; + +import com.google.gerrit.lifecycle.LifecycleModule; +import com.google.inject.AbstractModule; + +public class JettyModule extends AbstractModule { + private final JettyEnv env; + + public JettyModule(final JettyEnv env) { + this.env = env; + } + + @Override + protected void configure() { + bind(JettyEnv.class).toInstance(env); + bind(JettyServer.class); + install(new LifecycleModule() { + @Override + protected void configure() { + listener().to(JettyServer.Lifecycle.class); + } + }); + } +} diff --git a/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyServer.java b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyServer.java new file mode 100644 index 0000000000..f07b669f93 --- /dev/null +++ b/gerrit-pgm/src/main/java/com/google/gerrit/pgm/http/jetty/JettyServer.java @@ -0,0 +1,304 @@ +// Copyright (C) 2009 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.google.gerrit.pgm.http.jetty; + +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static java.util.concurrent.TimeUnit.SECONDS; + +import com.google.gerrit.lifecycle.LifecycleListener; +import com.google.gerrit.main.GerritLauncher; +import com.google.gerrit.server.config.GerritServerConfig; +import com.google.gerrit.server.config.SitePath; +import com.google.inject.Inject; +import com.google.inject.Injector; +import com.google.inject.Singleton; +import com.google.inject.servlet.GuiceFilter; +import com.google.inject.servlet.GuiceServletContextListener; + +import org.eclipse.jetty.io.EndPoint; +import org.eclipse.jetty.server.Connector; +import org.eclipse.jetty.server.Handler; +import org.eclipse.jetty.server.Request; +import org.eclipse.jetty.server.Server; +import org.eclipse.jetty.server.handler.ContextHandler; +import org.eclipse.jetty.server.handler.ContextHandlerCollection; +import org.eclipse.jetty.server.nio.SelectChannelConnector; +import org.eclipse.jetty.server.ssl.SslSelectChannelConnector; +import org.eclipse.jetty.servlet.DefaultServlet; +import org.eclipse.jetty.servlet.FilterMapping; +import org.eclipse.jetty.servlet.ServletContextHandler; +import org.eclipse.jetty.util.resource.Resource; +import org.eclipse.jetty.util.thread.QueuedThreadPool; +import org.eclipse.jetty.util.thread.ThreadPool; +import org.eclipse.jgit.lib.Config; + +import java.io.File; +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URI; +import java.net.URISyntaxException; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.Set; + +@Singleton +public class JettyServer { + static class Lifecycle implements LifecycleListener { + private final JettyServer server; + @Inject + Lifecycle(final JettyServer server) { + this.server=server; + } + + @Override + public void start() { + try { + server.httpd.start(); + } catch (Exception e) { + throw new IllegalStateException("Cannot start HTTP daemon", e); + } + } + + @Override + public void stop() { + try { + server.httpd.stop(); + server.httpd.join(); + } catch (Exception e) { + throw new IllegalStateException("Cannot stop HTTP daemon", e); + } + } + } + + private final File sitePath; + private final Server httpd; + + @Inject + JettyServer(@GerritServerConfig final Config cfg, + @SitePath final File sitePath, final JettyEnv env) + throws MalformedURLException, IOException { + this.sitePath = sitePath; + + Handler app = makeContext(env, cfg); + + httpd = new Server(); + httpd.setConnectors(listen(cfg)); + httpd.setThreadPool(threadPool(cfg)); + httpd.setHandler(app); + + httpd.setStopAtShutdown(false); + httpd.setSendDateHeader(true); + httpd.setSendServerVersion(false); + httpd.setGracefulShutdown((int) MILLISECONDS.convert(1, SECONDS)); + } + + private Connector[] listen(final Config cfg) { + // OpenID and certain web-based single-sign-on products can cause + // some very long headers, especially in the Referer header. We + // need to use a larger default header size to ensure we have + // the space required. + // + final int requestHeaderSize = + cfg.getInt("httpd", "requestheadersize", 16386); + final URI[] listenUrls = listenURLs(cfg); + final boolean reuseAddress = cfg.getBoolean("httpd", "reuseaddress", true); + final int acceptors = cfg.getInt("httpd", "acceptorThreads", 2); + + final Connector[] connectors = new Connector[listenUrls.length]; + for (int idx = 0; idx < listenUrls.length; idx++) { + final URI u = listenUrls[idx]; + final int defaultPort; + final SelectChannelConnector c; + + if ("http".equals(u.getScheme())) { + defaultPort = 80; + c = new SelectChannelConnector(); + + } else if ("https".equals(u.getScheme())) { + final SslSelectChannelConnector ssl = new SslSelectChannelConnector(); + final File keystore = getFile(cfg, "sslkeystore", "keystore"); + String password = cfg.getString("httpd", null, "sslkeypassword"); + if (password == null) { + password = "gerrit"; + } + ssl.setKeystore(keystore.getAbsolutePath()); + ssl.setTruststore(keystore.getAbsolutePath()); + ssl.setKeyPassword(password); + ssl.setTrustPassword(password); + + defaultPort = 443; + c = ssl; + + } else if ("proxy-http".equals(u.getScheme())) { + defaultPort = 8080; + c = new SelectChannelConnector(); + c.setForwarded(true); + + } else if ("proxy-https".equals(u.getScheme())) { + defaultPort = 8080; + c = new SelectChannelConnector() { + @Override + public void customize(EndPoint endpoint, Request request) + throws IOException { + request.setScheme("https"); + super.customize(endpoint, request); + } + }; + c.setForwarded(true); + + } else { + throw new IllegalArgumentException("Protocol '" + u.getScheme() + "' " + + " not supported in httpd.listenurl '" + u + "';" + + " only 'http', 'https', 'proxy-http, 'proxy-https'" + + " are supported"); + } + + try { + if (u.getHost() == null && (u.getAuthority().equals("*") // + || u.getAuthority().startsWith("*:"))) { + // Bind to all local addresses. Port wasn't parsed right by URI + // due to the illegal host of "*" so replace with a legal name + // and parse the URI. + // + final URI r = + new URI(u.toString().replace('*', 'A')).parseServerAuthority(); + c.setHost(null); + c.setPort(0 < r.getPort() ? r.getPort() : defaultPort); + } else { + final URI r = u.parseServerAuthority(); + c.setHost(r.getHost()); + c.setPort(0 < r.getPort() ? r.getPort() : defaultPort); + } + } catch (URISyntaxException e) { + throw new IllegalArgumentException("Invalid httpd.listenurl " + u, e); + } + + c.setRequestHeaderSize(requestHeaderSize); + c.setAcceptors(acceptors); + c.setReuseAddress(reuseAddress); + c.setStatsOn(false); + + connectors[idx] = c; + } + return connectors; + } + + private URI[] listenURLs(final Config cfg) { + String[] urls = cfg.getStringList("httpd", null, "listenurl"); + if (urls.length == 0) { + urls = new String[] {"http://*:8080/"}; + } + + final URI[] r = new URI[urls.length]; + for (int i = 0; i < r.length; i++) { + final String s = urls[i]; + try { + r[i] = new URI(s); + } catch (URISyntaxException e) { + throw new IllegalArgumentException("Invalid httpd.listenurl " + s, e); + } + } + return r; + } + + private File getFile(final Config cfg, final String name, final String def) { + String path = cfg.getString("httpd", null, name); + if (path == null || path.length() == 0) { + path = def; + } + + File loc = new File(path); + if (!loc.isAbsolute()) { + loc = new File(sitePath, path); + } + return loc; + } + + private ThreadPool threadPool(Config cfg) { + final QueuedThreadPool pool = new QueuedThreadPool(); + pool.setName("HTTP"); + pool.setMinThreads(cfg.getInt("httpd", null, "minthreads", 5)); + pool.setMaxThreads(cfg.getInt("httpd", null, "maxthreads", 25)); + pool.setMaxQueued(cfg.getInt("httpd", null, "maxqueued", 50)); + return pool; + } + + private Handler makeContext(final JettyEnv env, final Config cfg) + throws MalformedURLException, IOException { + final Set paths = new HashSet(); + for (URI u : listenURLs(cfg)) { + paths.add(u.getPath() != null ? u.getPath() : "/"); + } + + final List all = new ArrayList(); + for (String path : paths) { + all.add(makeContext(path, env)); + } + + if (all.size() == 1) { + // If we only have one context path in our web space, return it + // without any wrapping so Jetty has less work to do per-request. + // + return all.get(0); + } else { + // We have more than one path served out of this container so + // combine them in a handler which supports dispatching to the + // individual contexts. + // + final ContextHandlerCollection r = new ContextHandlerCollection(); + r.setHandlers(all.toArray(new Handler[0])); + return r; + } + } + + private ContextHandler makeContext(final String contextPath, + final JettyEnv env) throws MalformedURLException, IOException { + final ServletContextHandler app = new ServletContextHandler(); + + // This is the path we are accessed by clients within our domain. + // + app.setContextPath(contextPath); + + // Serve static resources directly from our JAR. This way we don't + // need to unpack them into yet another temporary directory prior to + // serving to clients. + // + final File war = GerritLauncher.getDistributionArchive(); + app.setBaseResource(Resource.newResource("jar:" + war.toURI() + "!/")); + + // Perform the same binding as our web.xml would do, but instead + // of using the listener to create the injector pass the one we + // already have built. + // + app.addFilter(GuiceFilter.class, "/*", FilterMapping.DEFAULT); + app.addEventListener(new GuiceServletContextListener() { + @Override + protected Injector getInjector() { + return env.webInjector; + } + }); + + // Jetty requires at least one servlet be bound before it will + // bother running the filter above. Since the filter has all + // of our URLs except the static resources, the only servlet + // we need to bind is the default static resource servlet from + // the Jetty container. + // + app.addServlet(DefaultServlet.class, "/"); + + return app; + } +} diff --git a/gerrit-server/src/main/java/com/google/gerrit/lifecycle/LifecycleManager.java b/gerrit-server/src/main/java/com/google/gerrit/lifecycle/LifecycleManager.java index 6c5697823c..ada124e6f7 100644 --- a/gerrit-server/src/main/java/com/google/gerrit/lifecycle/LifecycleManager.java +++ b/gerrit-server/src/main/java/com/google/gerrit/lifecycle/LifecycleManager.java @@ -77,6 +77,9 @@ public class LifecycleManager { new LinkedHashMap(); for (final Injector injector : injectors) { + if (injector == null) { + continue; + } for (final Binding binding : get(injector)) { found.put(binding.getProvider().get(), true); } diff --git a/gerrit-server/src/main/java/com/google/gerrit/server/config/GerritGlobalModule.java b/gerrit-server/src/main/java/com/google/gerrit/server/config/GerritGlobalModule.java index 89432b35ab..239ff533e8 100644 --- a/gerrit-server/src/main/java/com/google/gerrit/server/config/GerritGlobalModule.java +++ b/gerrit-server/src/main/java/com/google/gerrit/server/config/GerritGlobalModule.java @@ -15,7 +15,6 @@ package com.google.gerrit.server.config; import static com.google.inject.Scopes.SINGLETON; -import static com.google.inject.Stage.PRODUCTION; import com.google.gerrit.common.data.ApprovalTypes; import com.google.gerrit.lifecycle.LifecycleModule; @@ -60,40 +59,13 @@ import com.google.gerrit.server.patch.PatchSetInfoFactory; import com.google.gerrit.server.project.ProjectCacheImpl; import com.google.gerrit.server.project.ProjectState; import com.google.gerrit.server.workflow.FunctionState; -import com.google.inject.Guice; import com.google.inject.Inject; -import com.google.inject.Injector; -import com.google.inject.Module; -import com.google.inject.Provider; import org.eclipse.jgit.lib.Config; import org.eclipse.jgit.lib.PersonIdent; -import java.util.ArrayList; -import java.util.List; - /** Starts global state with standard dependencies. */ public class GerritGlobalModule extends FactoryModule { - public static Injector createInjector() { - final Injector db = Guice.createInjector(PRODUCTION, new DatabaseModule()); - final CanonicalWebUrlModule canonicalWebUrl = new CanonicalWebUrlModule() { - @Override - protected Class> provider() { - return CanonicalWebUrlProvider.class; - } - }; - return createInjector(db, canonicalWebUrl); - } - - public static Injector createInjector(final Injector db, - final CanonicalWebUrlModule canonicalWebUrl) { - final Injector cfg = db.createChildInjector(new GerritConfigModule()); - final List modules = new ArrayList(); - modules.add(cfg.getInstance(GerritGlobalModule.class)); - modules.add(canonicalWebUrl); - return cfg.createChildInjector(modules); - } - private final AuthType loginType; @Inject diff --git a/gerrit-war/pom.xml b/gerrit-war/pom.xml index 113214bf60..86730d491a 100644 --- a/gerrit-war/pom.xml +++ b/gerrit-war/pom.xml @@ -37,7 +37,8 @@ limitations under the License. javax.servlet servlet-api - provided + + compile diff --git a/gerrit-war/src/main/java/com/google/gerrit/httpd/WebAppInitializer.java b/gerrit-war/src/main/java/com/google/gerrit/httpd/WebAppInitializer.java index 03f4237e77..9e9aab89d6 100644 --- a/gerrit-war/src/main/java/com/google/gerrit/httpd/WebAppInitializer.java +++ b/gerrit-war/src/main/java/com/google/gerrit/httpd/WebAppInitializer.java @@ -19,6 +19,7 @@ import static com.google.inject.Stage.PRODUCTION; import com.google.gerrit.lifecycle.LifecycleManager; import com.google.gerrit.server.config.CanonicalWebUrlModule; import com.google.gerrit.server.config.DatabaseModule; +import com.google.gerrit.server.config.GerritConfigModule; import com.google.gerrit.server.config.GerritGlobalModule; import com.google.gerrit.server.config.MasterNodeStartup; import com.google.gerrit.sshd.SshModule; @@ -47,6 +48,7 @@ public class WebAppInitializer extends GuiceServletContextListener { LoggerFactory.getLogger(WebAppInitializer.class); private Injector dbInjector; + private Injector cfgInjector; private Injector sysInjector; private Injector webInjector; private Injector sshInjector; @@ -75,14 +77,8 @@ public class WebAppInitializer extends GuiceServletContextListener { throw new CreationException(Collections.singleton(first)); } - sysInjector = - GerritGlobalModule.createInjector(dbInjector, - new CanonicalWebUrlModule() { - @Override - protected Class> provider() { - return HttpCanonicalWebUrlProvider.class; - } - }); + cfgInjector = dbInjector.createChildInjector(new GerritConfigModule()); + sysInjector = createSysInjector(); sshInjector = createSshInjector(); webInjector = createWebInjector(); @@ -107,11 +103,23 @@ public class WebAppInitializer extends GuiceServletContextListener { } } + private Injector createSysInjector() { + final List modules = new ArrayList(); + modules.add(cfgInjector.getInstance(GerritGlobalModule.class)); + modules.add(new CanonicalWebUrlModule() { + @Override + protected Class> provider() { + return HttpCanonicalWebUrlProvider.class; + } + }); + modules.add(new MasterNodeStartup()); + return cfgInjector.createChildInjector(modules); + } + private Injector createSshInjector() { final List modules = new ArrayList(); modules.add(new SshModule()); modules.add(new MasterCommandModule()); - modules.add(new MasterNodeStartup()); return sysInjector.createChildInjector(modules); } diff --git a/gerrit-war/src/main/java/log4j.properties b/gerrit-war/src/main/java/log4j.properties index 183f3a8a81..2ef4ac9a78 100644 --- a/gerrit-war/src/main/java/log4j.properties +++ b/gerrit-war/src/main/java/log4j.properties @@ -18,6 +18,8 @@ log4j.appender.stderr.target=System.err log4j.appender.stderr.layout=org.apache.log4j.PatternLayout log4j.appender.stderr.layout.ConversionPattern=%d::%-5p: %c %x - %m%n +log4j.logger.com.google.gerrit=INFO + # Silence non-critical messages from MINA SSHD. # log4j.logger.org.apache.sshd.common=WARN @@ -25,6 +27,10 @@ log4j.logger.org.apache.sshd.server=WARN log4j.logger.org.apache.sshd.common.keyprovider.FileKeyPairProvider=INFO log4j.logger.com.google.gerrit.server.ssh.GerritServerSession=WARN +# Silence non-critical messages from Jetty. +# +log4j.logger.org.eclipse.jetty=INFO + # Silence non-critical messages from mime-util. # log4j.logger.eu.medsea.mimeutil=WARN diff --git a/pom.xml b/pom.xml index f69e0e5eb9..343182aefa 100644 --- a/pom.xml +++ b/pom.xml @@ -53,6 +53,7 @@ limitations under the License. 1.7.0 1.5.8 2.0 + 7.0.0.v20091005 OBF @@ -701,6 +702,12 @@ limitations under the License. 8.4-701.jdbc4 + + org.eclipse.jetty + jetty-servlet + ${jettyVersion} + + org.easymock easymock