Allow to update external IDs with BatchRefUpdate
This change adds a VersionedMetaData subclass to update external IDs,
called ExternalIdNotes. Having this class has several advantages:
- External IDs can now be updated as part of a BatchRefUpdate:
This will allow us to update accounts and external IDs atomically
(implemented in follow-up changes).
- External ID updates can now easily be batched and committed
atomically:
The ExternalIdsBatchUpdate class which implemented replacing of
external IDs in a batch becomes unneeded and is removed.
- Reading and writing external IDs is now implemented in the same class
(ExternalIdNotes)
- External ID updates from init, schema migrations, standalone programs
and tests become easier.
- Using VersionedMetaData to update the external ID notes branch is
consistent with using VersionedMetaData to update the group names notes
branch.
Quite a lot of code had to be refactored for this. The most important
changes are:
- The code to parse external IDs is moved from ExternalIdReader to
ExternalIdNotes:
ExternalIdReader still takes care of opening the repository and
loading the external IDs. In addition it updates the metric for
reading all external IDs and allows tests to disable reading external
IDs.
- The code to update external IDs is moved from ExternalIdsUpdate to
ExternalIdNotes:
ExternalIdsUpdate now only takes care to retry external ID updates on
LockFailure. When external IDs are updated atomically with account
updates (follow-up changes) most methods will be removed. Basically
all external ID updates that touch external IDs of a single account
will be done through AccountsUpdate. ExternalIdsUpdate will then only
be used to update external IDs across multiple accounts (if we have
such updates).
- External ID updates from init, schema migrations, standalone programs
and tests are now done via ExternalIdNotes:
This removes the need for public static methods and reduces code
duplication.
How does ExternalIdNotes work?
- On load the note map from refs/meta/external-ids is read (but the
external IDs are not parsed yet).
- After loading the note map callers can access single or all external
IDs. Only now the requested external IDs are parsed.
- After loading the note map callers can stage various external ID
updates (insert, upsert, delete, replace).
- On save the staged external ID updates are performed.
- After committing the external IDs ExternalIdNotes can be asked to
update the ExternalIdCache and to evict affected accounts from the
account cache (which triggers a reindex of the accounts).
There are several modes of instantiating ExternalIdNotes:
- Through Factories
- ExternalIdNotes.Factory:
Used to get the caches injected.
- ExternalIdNotes.FactoryNoReindex:
Used to get the external IDs cache injected, the account cache is
not needed since accounts should not be reindex). Used by the
LocalUsernamesToLowerCase site program and internally at Google.
- Through static methods:
- loadReadOnly:
Used when external IDs are only read (since there are no updates the
caches are not needed)
- loadNoCacheUpdate:
Used when updates should be done but the caches don't need to be
updated (during init and schema migrations where the caches are
not bound)
- Through package-private constructor:
Only used by ExternalIdsUpdate that supports external ID updates with
and without reindex (hence we don't know which of the factories to
inject).
I tested the schema migrations, init and the LocalUsernamesToLowerCase
site program manually.
Change-Id: Ib643ae8072c61ec467c92f3d1672e595a9cdcbc8
Signed-off-by: Edwin Kempin <ekempin@google.com>
This commit is contained in:
@@ -718,7 +718,7 @@ public class CommitValidators {
|
||||
throw new CommitValidationException("invalid external IDs", msgs);
|
||||
}
|
||||
return msgs;
|
||||
} catch (IOException e) {
|
||||
} catch (IOException | ConfigInvalidException e) {
|
||||
String m = "error validating external IDs";
|
||||
log.warn(m, e);
|
||||
throw new CommitValidationException(m, e);
|
||||
|
||||
Reference in New Issue
Block a user