MergeSuperSet: Expand topic first, then complete change set without topic
For computing the changes that should be submitted together with submitWholeTopic enabled we iteratively discover changes to be submitted by: 1. including predecessor changes 2. expanding the topics This change swaps this order to: 1. expand the topics 2. include predecessor changes In a follow-up change the second step (include predecessor changes) should become an extension point so that in a multi-master setup a plugin can decide to distribute the load for computing the predecessor changes across multiple nodes. In the multi-master setup at Google the load balancer distributes requests to different nodes based on the project in the URL. This means not all nodes have to load all repositories into cache which improves performance a lot. However we still have performance problems with the Submitted Together REST endpoint and for large topics which span several hundreds of projects loading the list of changes that should be submitted together can take several minutes. The problem is that the request to the Submitted Together REST endpoint is not specific to one project, but to include the predecessor change we have to open the repository for all projects that are touched in the topics. This means the one node node that handles the request has to open several hundreds of repositories which is slow and also polutes the repository caches. To fix this we want to implement the extension point for computing predecessor changes in such a way that it distributes the load over multiple nodes that each have the necessary repositories already cached. These nodes can do the computation in parallel which should speed up the operation further. In this context it makes sense to swap the order of the computation steps. The first step (expand the topics) doesn't require opening any repository since the topic is resolved via the change index and hence this step can be executed locally on the node that receives the request. This means we first do locally everything we can before distributing the work to compute predecessor changes across multiple nodes. Change-Id: I8ea6fc18a7cf94e16282fa53507c725d8aaae8f9 Signed-off-by: Edwin Kempin <ekempin@google.com>
This commit is contained in:
@@ -358,8 +358,8 @@ public class MergeSuperSet {
|
||||
do {
|
||||
oldSeen = seen;
|
||||
|
||||
changes = completeChangeSetWithoutTopic(db, changes, user);
|
||||
changes = topicClosure(db, changes, user, topicsSeen, visibleTopicsSeen);
|
||||
changes = completeChangeSetWithoutTopic(db, changes, user);
|
||||
|
||||
seen = topicsSeen.size() + visibleTopicsSeen.size();
|
||||
} while (seen != oldSeen);
|
||||
|
||||
Reference in New Issue
Block a user