Migrate external IDs to NoteDb (part 2)

This is the second part of migrating external IDs from ReviewDb to
NoteDb.

This change:
* migrates the external IDs from ReviewDb to NoteDb (for single
  instance Gerrit servers)
* adds a configuration parameter (user.readExternalIdsFromGit) that
  controls whether external IDs are read from ReviewDb or NoteDb

The new ExternalIds class provides access to external IDs. All code that
needs external IDs is adapted to use this class to retrieve external IDs
(instead of reading directly from the database). ExternalIds gets the
external IDs either directly from the storage backend (via
ExternalIdReader) or from a cache (via ExternalIdCache).

ExternalIdReader reads the external IDs from the storage backend.
Depending on the value of the user.readExternalIdsFromGit parameter the
external IDs are read from ReviewDB or NoteDb.

If reading external IDs from NoteDb is enabled, reading the external IDs
of an account requires parsing all Git notes. This is because external
IDs are keyed by external ID key ('<scheme>:<id>') and the account ID is
only contained in the Git note content. Since parsing all Git notes is
too expensive if it is done frequently, there is a new external ID cache
which makes external IDs accessible by account. This cache is populated
once by reading all external IDs from NoteDb and is then kept up to date
by informing it whenever an external ID is added, updated or deleted.
The external ID cache uses the revision of the refs/meta/external-ids
branch as key, so that all external IDs are reloaded when the
refs/meta/external-ids branch is changed behind Gerrit's back. This
makes it easy to use this cache in a multimaster setup, since an update
of the refs/meta/external-ids branch which is done due to replication
between nodes causes a reload of the external IDs in the receiving node.
The ExternalIdCache is an implementation detail of how external IDs are
read and written, which is why it is package private. Callers should
always use ExternalIds to access external IDs and ExternalIdsUpdate /
ExternalIdsBatchUpdate to update external IDs.

The LocalUsernamesToLowerCase program needs to access all external IDs
only once to update them. After the update they are not accessed again.
Hence the LocalUsernamesToLowerCase program doesn't benefit from caching
external IDs and the external ID cache can be disabled for it.

The external ID cache is defined by the ExternalIdCache interface. It is
implemented by ExternalIdCacheImpl and DisabledExternalIdCache.
DisabledExternalIdCache can be used when an external ID cache is not
needed, e.g. in the LocalUsernamesToLowerCase program or in tests.

Pushing to the refs/meta/external-ids branch, which would only update
the external IDs in NoteDb, is still prevented by a commit validator so
that the external IDs in ReviewDb and NoteDb do not go out of sync.

Change-Id: Ia1dae9306b7ee07388b6c5e1f3dc4a1a5eea4b08
Signed-off-by: Edwin Kempin <ekempin@google.com>

gerrit-server/src/main/java/com/google/gerrit/server/account/GetExternalIds.java

@@ -25,11 +25,13 @@ import com.google.gerrit.extensions.restapi.RestReadView;
 import com.google.gerrit.reviewdb.server.ReviewDb;
 import com.google.gerrit.server.CurrentUser;
 import com.google.gerrit.server.account.externalids.ExternalId;
+import com.google.gerrit.server.account.externalids.ExternalIds;
 import com.google.gerrit.server.config.AuthConfig;
 import com.google.gwtorm.server.OrmException;
 import com.google.inject.Inject;
 import com.google.inject.Provider;
 import com.google.inject.Singleton;
+import java.io.IOException;
 import java.util.Collection;
 import java.util.Collections;
 import java.util.List;
@@ -37,26 +39,30 @@ import java.util.List;
 @Singleton
 public class GetExternalIds implements RestReadView<AccountResource> {
   private final Provider<ReviewDb> db;
+  private final ExternalIds externalIds;
   private final Provider<CurrentUser> self;
   private final AuthConfig authConfig;
 
   @Inject
-  GetExternalIds(Provider<ReviewDb> db, Provider<CurrentUser> self, AuthConfig authConfig) {
+  GetExternalIds(
+      Provider<ReviewDb> db,
+      ExternalIds externalIds,
+      Provider<CurrentUser> self,
+      AuthConfig authConfig) {
     this.db = db;
+    this.externalIds = externalIds;
     this.self = self;
     this.authConfig = authConfig;
   }
 
   @Override
   public List<AccountExternalIdInfo> apply(AccountResource resource)
-      throws RestApiException, OrmException {
+      throws RestApiException, IOException, OrmException {
     if (self.get() != resource.getUser()) {
       throw new AuthException("not allowed to get external IDs");
     }
 
-    Collection<ExternalId> ids =
-        ExternalId.from(
-            db.get().accountExternalIds().byAccount(resource.getUser().getAccountId()).toList());
+    Collection<ExternalId> ids = externalIds.byAccount(db.get(), resource.getUser().getAccountId());
     if (ids.isEmpty()) {
       return ImmutableList.of();
     }