opendev/gerrit
Code Issues Proposed changes

Merge branch 'stable-3.1'

Browse Source 
* stable-3.1:
  Set version to 3.1.2-SNAPSHOT
  Set version to 3.1.1
  Set version to 3.0.6-SNAPSHOT
  Set version to 3.0.5
  Set version to 2.16.15-SNAPSHOT
  Set version to 2.16.14
  Rename "master" to "primary" in documentation and comments
  Add event interface to Gerrit

Change-Id: I53e83ccce28a03a58518d5c419dfc3fe56e826dc
This commit is contained in:
David Pursehouse
2019-12-11 17:19:40 +09:00
parent 278c8416c5 f2c5c36dcb
commit bfa270a360
18 changed files with 385 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
Documentation/backup.txt
View File

@@ -139,11 +139,12 @@ Use a file system supporting snapshots to keep the period where the gerrit
server is read-only or down as short as possible.
[#cons-backup-read-only]
=== Turn master read-only for backup
=== Turn primary server read-only for backup
Make the server read-only before taking the backup. This means read-access
is still available during backup, because only write operations have to be
stopped to ensure consistency. This can be implemented using the
Make the primary server handling write operations read-only before taking the
backup. This means read-access is still available from replica servers during
backup, because only write operations have to be stopped to ensure consistency.
This can be implemented using the
link:https://gerrit.googlesource.com/plugins/readonly/[_readonly_,role=external,window=_blank] plugin.
[#cons-backup-replicate]
@@ -180,11 +181,12 @@ If you are using Gerrit replica to offload read traffic you can use one of these
replica for creating backups.
[#cons-backup-offline]
=== Take master offline for backup
=== Take primary server offline for backup
Shutdown the server before taking a backup. This is simple but means downtime
for the users. Also crons and currently running cron jobs (e.g. repacking
repositories) which affect the repositories may need to be shut down.
Shut down the primary server handling write operations before taking a backup.
This is simple but means downtime for the users. Also crons and currently
running cron jobs (e.g. repacking repositories) which affect the repositories
may need to be shut down.
[#backup-methods]
== Backup methods

8
Documentation/config-accounts.txt
View File

@@ -386,10 +386,10 @@ stored in a local H2 database, but there is an extension point that
allows to plug in alternate implementations for storing the reviewed
flags. To replace the storage for reviewed flags a plugin needs to
implement the link:dev-plugins.html#account-patch-review-store[
AccountPatchReviewStore] interface. E.g. to support a multi-master
setup where reviewed flags should be replicated between the master
nodes one could implement a store for the reviewed flags that is
based on MySQL with replication.
AccountPatchReviewStore] interface. E.g. to support a cluster setup with
multiple primary servers handling write operations where reviewed flags should
be replicated between the primary nodes one could implement a store for the
reviewed flags that is based on MySQL with replication.
[[account-sequence]]
== Account Sequence

8
Documentation/config-gerrit.txt
View File

@@ -826,7 +826,8 @@ changes for up to 1024 projects can be held in the cache.
+
Default value is 0 (disabled). It is disabled by default due to the fact
that change updates are not communicated between Gerrit servers. Hence
this cache should be disabled in an multi-master/multi-replica setup.
this cache should be disabled in a cluster setup using multiple primary
or multiple replica nodes.
+
The cache should be flushed whenever the database changes table is modified
outside of Gerrit.
@@ -1575,7 +1576,8 @@ options.
+
Used on Gerrit replica installations. If set to true the Gerrit JVM is
called with the '--replica' switch, enabling replica mode. If no value is
set (or any other value), Gerrit defaults to master mode.
set (or any other value), Gerrit defaults to primary mode enabling write
operations.
[[container.slave]]container.slave::
+
@@ -4442,7 +4444,7 @@ SSH-compression since git does not compress the ref announcement during
handshake.
+
Compression can be especially useful when Gerrit replicas are being used
for the larger clones and fetches and the master server mostly takes
for the larger clones and fetches and the primary server mostly takes
small receive-packs.
+
By default, `false`.

8
Documentation/config-groups.txt
View File

@@ -103,7 +103,7 @@ Group] reindex the affected groups manually.
== Replication
In a replicated setting (eg. backups and or master/replica
configurations), all refs in the `All-Users` project must be copied
onto all replicas, including `refs/groups/*`, `refs/meta/group-names`
and `refs/sequences/groups`.
In a replicated setting (eg. backups and or primary/replica configurations), all
refs in the `All-Users` project on primary nodes must be copied onto all
replicas, including `refs/groups/*`, `refs/meta/group-names` and
`refs/sequences/groups`.

4
Documentation/config-plugins.txt
View File

@@ -795,8 +795,8 @@ Configuration,role=external,window=_blank]
This plugin replaces the built-in Gerrit H2 based websession cache with
a flatfile based implementation. This implementation is shareable
among multiple Gerrit servers, making it useful for multi-master
Gerrit installations.
among multiple Gerrit servers, making it useful for cluster
Gerrit installations having multiple primary Gerrit nodes.
link:https://gerrit-review.googlesource.com/admin/repos/plugins/websession-flatfile[
Project,role=external,window=_blank] |

3
Documentation/dev-plugins.txt
View File

@@ -2306,7 +2306,8 @@ flags is growing without bound. The store must be able handle this data
volume efficiently.
Gerrit implements this extension point, but plugins may bind another
implementation, e.g. one that supports multi-master.
implementation, e.g. one that supports cluster setup with multiple
primary Gerrit nodes handling write operations.
----
DynamicItem.bind(binder(), AccountPatchReviewStore.class)

4
Documentation/pgm-daemon.txt
View File

@@ -81,7 +81,7 @@ external log cleaning service to clean up the prior logs.
== KNOWN ISSUES
Replica daemon caches can quickly become out of date when modifications
are made on the master. The following configuration is suggested in
are made on the primary node. The following configuration is suggested in
a replica to reduce the maxAge for each cache entry, so that changes
are recognized in a reasonable period of time:
@@ -106,7 +106,7 @@ and if LDAP support was enabled, also include:
maxAge = 5 min
----
Automatic cache coherency between master and replica systems is
Automatic cache coherency between primary and replica systems is
planned to be implemented in a future version.
GERRIT

18
contrib/hooks/post-receive-move-tmp-refs
View File

@@ -15,10 +15,10 @@
# limitations under the License.
# --------------------------------------------------------
# Install this hook script as post-receive hook in replicated repositories
# hosted by a gerrit slave which are updated by push replication from the
# corresponding gerrit master.
# hosted by a gerrit replica which are updated by push replication from the
# corresponding gerrit primary node.
#
# In the gerrit master configure the replication plugin to push changes from
# In the gerrit primary node configure the replication plugin to push changes from
# refs/changes/ to refs/tmp/changes/
# remote.NAME.push = +refs/changes/*:refs/tmp/changes/*
# remote.NAME.push = +refs/heads/*:refs/heads/*
@@ -26,26 +26,26 @@
# And if it's a Gerrit mirror:
# remote.NAME.push = +refs/meta/*:refs/meta/*
#
# In the replicated repository in the gerrit slave configure
# In the replicated repository in the gerrit replica configure
# receive.hideRefs = refs/changes/
# in order to not advertise the big number of refs in this namespace when
# the gerrit master's replication plugin is pushing a change
# the gerrit primary's replication plugin is pushing a change
#
# Whenever a ref under refs/tmp/changes/ is arriving this hook will move it
# to refs/changes/. This helps to avoid the large overhead of advertising all
# refs/changes/ refs to the gerrit master when it replicates changes to the
# slave..
# refs/changes/ refs to the gerrit primary when it replicates changes to the
# replica.
#
# Make this script executable then link to it in the repository you would like
# to use it in.
# cd /path/to/your/repository.git
# ln -sf <shared hooks directory>/post-receive-move-tmp-refs hooks/post-receive
#
# If you want to use this by default for repositories on the Gerrit slave you
# If you want to use this by default for repositories on the Gerrit replica you
# can set up a git template directory $TEMPLATE_DIR/hooks/post-receive and
# configure init.templateDir in the ~/.gitconfig of the user that receives the
# replication on the mirror host. That way when a new repository is created on
# the master and hence on the mirror (if configured that way) it will
# the primary and hence on the mirror (if configured that way) it will
# automatically have the "tmp-refs" commit hook installed.
# See https://git-scm.com/docs/git-init#_template_directory for details.

10
java/com/google/gerrit/server/ExceptionHook.java
View File

@@ -20,11 +20,11 @@ import java.util.Optional;
/**
* Allows implementors to control how certain exceptions should be handled.
*
* <p>This interface is intended to be implemented for multi-master setups to control the behavior
* for handling exceptions that are thrown by a lower layer that handles the consensus and
* synchronization between different server nodes. E.g. if an operation fails because consensus for
* a Git update could not be achieved (e.g. due to slow responding server nodes) this interface can
* be used to retry the request instead of failing it immediately.
* <p>This interface is intended to be implemented for cluster setups with multiple primary nodes to
* control the behavior for handling exceptions that are thrown by a lower layer that handles the
* consensus and synchronization between different server nodes. E.g. if an operation fails because
* consensus for a Git update could not be achieved (e.g. due to slow responding server nodes) this
* interface can be used to retry the request instead of failing it immediately.
*/
@ExtensionPoint
public interface ExceptionHook {

3
java/com/google/gerrit/server/change/AccountPatchReviewStore.java
View File

@@ -30,7 +30,8 @@ import java.util.Optional;
* number of reviewed flags is growing without bound. The store must be able handle this data volume
* efficiently.
*
* <p>For a multi-master setup the store must replicate the data between the masters.
* <p>For a cluster setups with multiple primary nodes the store must replicate the data between the
* primary servers.
*/
public interface AccountPatchReviewStore {

6
java/com/google/gerrit/server/submit/MergeOp.java
View File

@@ -820,9 +820,9 @@ public class MergeOp implements AutoCloseable {
}
// The patch set ref is not found but we want to merge the change. We can't safely do that
// if the patch set ref is missing. In a multi-master setup this can indicate a replication
// lag (e.g. the change meta data was already replicated, but the replication of the patch
// set ref is still pending).
// if the patch set ref is missing. In a cluster setups with multiple primary nodes this can
// indicate a replication lag (e.g. the change meta data was already replicated, but the
// replication of the patch set ref is still pending).
commitStatus.logProblem(
changeId,
"Patch set ref "

7
java/com/google/gerrit/sshd/SshDaemon.java
View File

@@ -638,10 +638,9 @@ public class SshDaemon extends SshServer implements SshInfo, LifecycleListener {
// However, if there are CPU in abundance and the server is reachable through
// slow networks, gits with huge amount of refs can benefit from SSH-compression
// since git does not compress the ref announcement during the handshake.
//
// Compression can be especially useful when Gerrit slaves are being used
// for the larger clones and fetches and the master server mostly takes small
// receive-packs.
// Compression can be especially useful when Gerrit replica are being used
// for the larger clones and fetches and the primary server handling write
// operations mostly takes small receive-packs.
if (enableCompression) {
compressionFactories.add(BuiltinCompressions.zlib);

12
javatests/com/google/gerrit/acceptance/server/project/ProjectWatchIT.java
View File

@@ -306,7 +306,7 @@ public class ProjectWatchIT extends AbstractDaemonTest {
requestScopeOperations.setApiUser(user.id());
// watch keyword in project as user
watch(watchedProject, "multimaster");
watch(watchedProject, "multiprimary");
// push a change with keyword -> should trigger email notification
requestScopeOperations.setApiUser(admin.id());
@@ -314,7 +314,7 @@ public class ProjectWatchIT extends AbstractDaemonTest {
cloneProject(Project.nameKey(watchedProject), admin);
PushOneCommit.Result r =
pushFactory
.create(admin.newIdent(), watchedRepo, "Document multimaster setup", "a.txt", "a1")
.create(admin.newIdent(), watchedRepo, "Document multiprimary setup", "a.txt", "a1")
.to("refs/for/master");
r.assertOkStatus();
@@ -323,7 +323,7 @@ public class ProjectWatchIT extends AbstractDaemonTest {
assertThat(messages).hasSize(1);
Message m = messages.get(0);
assertThat(m.rcpt()).containsExactly(user.getEmailAddress());
assertThat(m.body()).contains("Change subject: Document multimaster setup\n");
assertThat(m.body()).contains("Change subject: Document multiprimary setup\n");
assertThat(m.body()).contains("Gerrit-PatchSet: 1\n");
sender.clear();
@@ -418,7 +418,7 @@ public class ProjectWatchIT extends AbstractDaemonTest {
requestScopeOperations.setApiUser(user.id());
// watch keyword in project as user
watch(allProjects.get(), "multimaster");
watch(allProjects.get(), "multiprimary");
// push a change with keyword to any project -> should trigger email
// notification
@@ -426,7 +426,7 @@ public class ProjectWatchIT extends AbstractDaemonTest {
TestRepository<InMemoryRepository> anyRepo = cloneProject(Project.nameKey(anyProject), admin);
PushOneCommit.Result r =
pushFactory
.create(admin.newIdent(), anyRepo, "Document multimaster setup", "a.txt", "a1")
.create(admin.newIdent(), anyRepo, "Document multiprimary setup", "a.txt", "a1")
.to("refs/for/master");
r.assertOkStatus();
@@ -435,7 +435,7 @@ public class ProjectWatchIT extends AbstractDaemonTest {
assertThat(messages).hasSize(1);
Message m = messages.get(0);
assertThat(m.rcpt()).containsExactly(user.getEmailAddress());
assertThat(m.body()).contains("Change subject: Document multimaster setup\n");
assertThat(m.body()).contains("Change subject: Document multiprimary setup\n");
assertThat(m.body()).contains("Gerrit-PatchSet: 1\n");
sender.clear();

142
polygerrit-ui/app/elements/shared/gr-event-interface/gr-event-interface.js Normal file
View File

@@ -0,0 +1,142 @@
/**
* @license
* Copyright (C) 2019 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
(function(window) {
'use strict';
// Avoid duplicate registeration
if (window.EventEmitter) return;
/**
* An lite implementation of
* https://nodejs.org/api/events.html#events_class_eventemitter.
*
* This is unrelated to the native DOM events, you should use it when you want
* to enable EventEmitter interface on any class.
*
* @example
*
* class YourClass extends EventEmitter {
* // now all instance of YourClass will have this EventEmitter interface
* }
*
*/
class EventEmitter {
constructor() {
/**
* Shared events map from name to the listeners.
* @type {!Object<string, Array<eventCallback>>}
*/
this._listenersMap = new Map();
}
/**
* Register an event listener to an event.
*
* @param {string} eventName
* @param {eventCallback} cb
* @returns {Function} Unsubscribe method
*/
addListener(eventName, cb) {
if (!eventName || !cb) {
console.warn('A valid eventname and callback is required!');
return;
}
const listeners = this._listenersMap.get(eventName) || [];
listeners.push(cb);
this._listenersMap.set(eventName, listeners);
return () => {
this.off(eventName, cb);
};
}
// Alias for addListener.
on(eventName, cb) {
return this.addListener(eventName, cb);
}
// Attach event handler only once. Automatically removed.
once(eventName, cb) {
const onceWrapper = (...args) => {
cb(...args);
this.off(eventName, onceWrapper);
};
return this.on(eventName, onceWrapper);
}
/**
* De-register an event listener to an event.
*
* @param {string} eventName
* @param {eventCallback} cb
*/
removeListener(eventName, cb) {
let listeners = this._listenersMap.get(eventName) || [];
listeners = listeners.filter(listener => listener !== cb);
this._listenersMap.set(eventName, listeners);
}
// Alias to removeListener
off(eventName, cb) {
this.removeListener(eventName, cb);
}
/**
* Synchronously calls each of the listeners registered for
* the event named eventName, in the order they were registered,
* passing the supplied detail to each.
*
* Returns true if the event had listeners, false otherwise.
*
* @param {string} eventName
* @param {*} detail
*/
emit(eventName, detail) {
const listeners = this._listenersMap.get(eventName) || [];
for (const listener of listeners) {
try {
listener(detail);
} catch (e) {
console.error(e);
}
}
return listeners.length !== 0;
}
// Alias to emit.
dispatch(eventName, detail) {
return this.emit(eventName, detail);
}
/**
* Remove listeners for a specific event or all.
*
* @param {string} eventName if not provided, will remove all
*/
removeAllListeners(eventName) {
if (eventName) {
this._listenersMap.set(eventName, []);
} else {
this._listenersMap = new Map();
}
}
}
window.EventEmitter = EventEmitter;
})(window);

148
polygerrit-ui/app/elements/shared/gr-event-interface/gr-event-interface_test.html Normal file
View File

@@ -0,0 +1,148 @@
<!DOCTYPE html>
<!--
@license
Copyright (C) 2019 The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<meta name="viewport" content="width=device-width, minimum-scale=1.0, initial-scale=1.0, user-scalable=yes">
<title>gr-api-interface</title>
<script src="../../../bower_components/webcomponentsjs/webcomponents-lite.min.js"></script>
<script src="../../../bower_components/web-component-tester/browser.js"></script>
<link rel="import" href="../../../test/common-test-setup.html"/>
<link rel="import" href="../gr-js-api-interface/gr-js-api-interface.html">
<script>void(0);</script>
<test-fixture id="basic">
<template>
<gr-js-api-interface></gr-js-api-interface>
</template>
</test-fixture>
<script>
suite('gr-event-interface tests', () => {
let sandbox;
setup(() => {
sandbox = sinon.sandbox.create();
});
teardown(() => {
sandbox.restore();
});
suite('test on Gerrit', () => {
setup(() => {
fixture('basic');
Gerrit.removeAllListeners();
});
test('communicate between plugin and Gerrit', done => {
const eventName = 'test-plugin-event';
let p;
Gerrit.on(eventName, e => {
assert.equal(e.value, 'test');
assert.equal(e.plugin, p);
done();
});
Gerrit.install(plugin => {
p = plugin;
Gerrit.emit(eventName, {value: 'test', plugin});
}, '0.1',
'http://test.com/plugins/testplugin/static/test.js');
});
test('listen on events from core', done => {
const eventName = 'test-plugin-event';
Gerrit.on(eventName, e => {
assert.equal(e.value, 'test');
done();
});
Gerrit.emit(eventName, {value: 'test'});
});
test('communicate across plugins', done => {
const eventName = 'test-plugin-event';
Gerrit.install(plugin => {
Gerrit.on(eventName, e => {
assert.equal(e.plugin.getPluginName(), 'testB');
done();
});
}, '0.1',
'http://test.com/plugins/testA/static/testA.js');
Gerrit.install(plugin => {
Gerrit.emit(eventName, {plugin});
}, '0.1',
'http://test.com/plugins/testB/static/testB.js');
});
});
suite('test on interfaces', () => {
let testObj;
class TestClass extends EventEmitter {
}
setup(() => {
testObj = new TestClass();
});
test('on', () => {
const cbStub = sinon.stub();
testObj.on('test', cbStub);
testObj.emit('test');
testObj.emit('test');
assert.isTrue(cbStub.calledTwice);
});
test('once', () => {
const cbStub = sinon.stub();
testObj.once('test', cbStub);
testObj.emit('test');
testObj.emit('test');
assert.isTrue(cbStub.calledOnce);
});
test('unsubscribe', () => {
const cbStub = sinon.stub();
const unsubscribe = testObj.on('test', cbStub);
testObj.emit('test');
unsubscribe();
testObj.emit('test');
assert.isTrue(cbStub.calledOnce);
});
test('off', () => {
const cbStub = sinon.stub();
testObj.on('test', cbStub);
testObj.emit('test');
testObj.off('test', cbStub);
testObj.emit('test');
assert.isTrue(cbStub.calledOnce);
});
test('removeAllListeners', () => {
const cbStub = sinon.stub();
testObj.on('test', cbStub);
testObj.removeAllListeners('test');
testObj.emit('test');
assert.isTrue(cbStub.notCalled);
});
});
});
</script>

1
polygerrit-ui/app/elements/shared/gr-js-api-interface/gr-js-api-interface.html
View File

@@ -39,6 +39,7 @@ limitations under the License.
3. gr-public-js-api depends on gr-plugin-rest-api
-->
<script src="gr-api-utils.js"></script>
<script src="../gr-event-interface/gr-event-interface.js"></script>
<script src="gr-annotation-actions-context.js"></script>
<script src="gr-annotation-actions-js-api.js"></script>
<script src="gr-change-actions-js-api.js"></script>

36
polygerrit-ui/app/elements/shared/gr-js-api-interface/gr-public-js-api.js
View File

@@ -381,4 +381,40 @@
};
window.Plugin = Plugin;
// TODO(taoalpha): List all internal supported event names.
// Also convert this to inherited class once we move Gerrit to class.
Gerrit._eventEmitter = new EventEmitter();
['addListener',
'dispatch',
'emit',
'off',
'on',
'once',
'removeAllListeners',
'removeListener',
].forEach(method => {
/**
* Enabling EventEmitter interface on Gerrit.
*
* This will enable to signal across different parts of js code without relying on DOM,
* including core to core, plugin to plugin and also core to plugin.
*
* @example
*
* // Emit this event from pluginA
* Gerrit.install(pluginA => {
* fetch("some-api").then(() => {
* Gerrit.on("your-special-event", {plugin: pluginA});
* });
* });
*
* // Listen on your-special-event from pluignB
* Gerrit.install(pluginB => {
* Gerrit.on("your-special-event", ({plugin}) => {
* // do something, plugin is pluginA
* });
* });
*/
Gerrit[method] = Gerrit._eventEmitter[method].bind(Gerrit._eventEmitter);
});
})(window);

1
polygerrit-ui/app/test/index.html
View File

@@ -154,6 +154,7 @@ limitations under the License.
'settings/gr-settings-view/gr-settings-view_test.html',
'settings/gr-ssh-editor/gr-ssh-editor_test.html',
'settings/gr-watched-projects-editor/gr-watched-projects-editor_test.html',
'shared/gr-event-interface/gr-event-interface_test.html',
'shared/gr-account-entry/gr-account-entry_test.html',
'shared/gr-account-label/gr-account-label_test.html',
'shared/gr-account-list/gr-account-list_test.html',