opendev/gerrit
Code Issues Proposed changes

Migrate reviewed flags to local H2 database

Browse Source 
With NoteDb Gerrit will no longer have a database. Hence all data from
ReviewDb must be migrated away from it. This change migrates the
reviewed flags from ReviewDb to a local H2 database.

Reviewed flags are tuples of (patch set ID, file, account ID) and
record whether a user has reviewed a file in a patch set. Each user
can easily have thousands of reviewed flags and the number of reviewed
flags is growing without bound.

Most data from ReviewDb is migrated to be stored in git, but for
reviewed flags a git-based storage is not very well suitable because
there is a relatively high write latency [1]. Also depending on the
exact storage format updating reviewed flags may create a huge git
history which is not good for anything since an audit log for reviewed
flags is not needed [1].

Since reviewed flags are small per-user bits that are quickly read and
updated a database seems to be the best storage [2]. This is why with
this change the reviewed flags will be stored in a local H2 database.
This store implements the AccountPatchReviewStore extension point that
was added by change Iaaacd9f0a. Plugins can implement this extension
point and replace the local H2 database with another storage.

To support multi-master the local H2 database must be replaced by a
storage that supports replication, e.g. a MySQL replication system
[2]. This can be done by implementing the AccountPatchReviewStore
extension point.

A schema migration copies all reviewed flags from ReviewDb to the
local H2 database and the old ACCOUNT_PATCH_REVIEWS table in ReviewDb
is dropped.

Not using gwtorm to access the local H2 database is intentional. When
ReviewDb is gone, we want to drop the dependency to gwtorm, and hence
we don't want to introduce any new dependencies to it. Instead the
implementation is using raw SQL.

The local H2 database is stored in the review site in:
  $site_path/db/account_patch_reviews.h2.db

For the acceptence tests an in memory H2 database is used.

[1] https://groups.google.com/d/msg/repo-discuss/KhXKMKTJNMs/Tq3XaB8wCgAJ
[2] https://groups.google.com/d/msg/repo-discuss/KhXKMKTJNMs/hHoCa1_5CwAJ

Change-Id: I7cb77ff124e36709cc69ef547f2b4e5e1a1d7f62
Signed-off-by: Edwin Kempin <ekempin@google.com>
This commit is contained in:
Edwin Kempin
2016-06-15 17:58:58 +02:00
committed by Björn Pedersen
parent 9c3d23bb93
commit 15ca7b0bcb
15 changed files with 386 additions and 192 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
gerrit-war/src/main/java/com/google/gerrit/httpd/WebAppInitializer.java
View File

@@ -34,7 +34,6 @@ import com.google.gerrit.metrics.dropwizard.DropWizardMetricMaker;
import com.google.gerrit.reviewdb.client.AuthType;
import com.google.gerrit.server.account.InternalAccountDirectory;
import com.google.gerrit.server.cache.h2.DefaultCacheFactory;
import com.google.gerrit.server.change.AccountPatchReviewStoreImpl;
import com.google.gerrit.server.change.ChangeCleanupRunner;
import com.google.gerrit.server.config.AuthConfig;
import com.google.gerrit.server.config.AuthConfigModule;
@@ -63,6 +62,7 @@ import com.google.gerrit.server.schema.DataSourceModule;
import com.google.gerrit.server.schema.DataSourceProvider;
import com.google.gerrit.server.schema.DataSourceType;
import com.google.gerrit.server.schema.DatabaseModule;
import com.google.gerrit.server.schema.H2AccountPatchReviewStore;
import com.google.gerrit.server.schema.SchemaModule;
import com.google.gerrit.server.schema.SchemaVersionCheck;
import com.google.gerrit.server.securestore.SecureStoreClassName;
@@ -298,7 +298,7 @@ public class WebAppInitializer extends GuiceServletContextListener
final List<Module> modules = new ArrayList<>();
modules.add(new DropWizardMetricMaker.RestModule());
modules.add(new EventBroker.Module());
modules.add(new AccountPatchReviewStoreImpl.Module());
modules.add(new H2AccountPatchReviewStore.Module());
modules.add(cfgInjector.getInstance(GitRepositoryManagerModule.class));
modules.add(new ChangeHookApiListener.Module());
modules.add(new StreamEventsApiListener.Module());