Support async change operations directly in BatchUpdate

This replaces the manual threadpool management in ReceiveCommits with
a more general approach. That support was added for slow NoSQL
backends, which may also have poor performance characteristics when
updating lots of NoteDb refs in parallel.

This approach substantially rewrites the BatchUpdate loop to stage all
NoteDb update operations in memory in separate threads, each with its
own copy of the ReviewDb and change repo, and then aggregates the
results together in the main thread with a single ObjectInserter and
BatchRefUpdate. One nice thing about this approach is it doesn't
require any changes to BatchUpdate.Op implementations: each Op's
methods are executed sequentially, in one thread at a time, with
proper barriers (an executor) to ensure later methods see the results
of writes in the background thread.

Use an in-memory implementation of ObjectInserter in each of the
background threads' NoteDbUpdateManagers to buffer writes completely
in memory until it's time for the caller to flush. This wastes a small
amount of memory for the buffer, but these are just NoteDb objects, so
they should be quite small. This implementation using immutable result
types is preferable to trying to share a Repository/ObjectReader/
ObjectInserter across threads, which requires manual locking. That is
not only painful but also produces deadlocks when mixing repo-level
locks and SQL-level locks (e.g. H2's transaction implementation)
across threads.

Change-Id: I40545a4d48fcfa892bd3e4c0cd9b72ab7fac9436
This commit is contained in:
Dave Borowitz
2016-06-06 16:51:40 -04:00
parent 9d2474e480
commit 8d3885d357
13 changed files with 618 additions and 177 deletions

View File

@@ -329,7 +329,6 @@ public class PushOneCommit {
public void assertMessage(String expectedMessage) {
RemoteRefUpdate refUpdate = result.getRemoteUpdate(ref);
assertThat(message(refUpdate).toLowerCase())
.named(message(refUpdate))
.contains(expectedMessage.toLowerCase());
}