Fix race in Lucene shutdown

After a ControlledRealTimeReopenThread is stopped, any NrtFutures
depending on that thread will never finish; get() will hang forever.
This means if there are any indexing operations happening in the
WorkQueue after indexes are closed, they will never complete, causing
a hang during WorkQueue#stop().

Swap the order that lifecycle listeners are registered between the
index module and the work queue, so that the work queue is shut down
first, and the reopen threads don't shut down and leave stranded
tasks.

Change-Id: I005315336178234a974cde0ac2e51f1e04fbd999
This commit is contained in:
Dave Borowitz
2015-12-16 13:58:35 -05:00
parent 679574697b
commit 18ae9168c0
3 changed files with 23 additions and 9 deletions

View File

@@ -333,6 +333,11 @@ public class Daemon extends SiteProgram {
modules.add(SchemaVersionCheck.module());
modules.add(new DropWizardMetricMaker.RestModule());
modules.add(new LogFileCompressor.Module());
// Index module shutdown must happen before work queue shutdown, otherwise
// work queue can get stuck waiting on index futures that will never return.
modules.add(createIndexModule());
modules.add(new WorkQueue.Module());
modules.add(new ChangeHookRunner.Module());
modules.add(new ReceiveCommitsExecutorModule());
@@ -351,7 +356,6 @@ public class Daemon extends SiteProgram {
modules.add(new PluginRestApiModule());
modules.add(new RestCacheAdminModule());
modules.add(new GpgModule(config));
modules.add(createIndexModule());
if (MoreObjects.firstNonNull(httpd, true)) {
modules.add(new CanonicalWebUrlModule() {
@Override