* Update plugins/replication from branch 'stable-2.16'
to 0c6bc832e9e36e1c3183dea6377423fbfcb77266
- Merge branch 'stable-2.15' into stable-2.16
* stable-2.15:
Fix issue with dropping events on start
Change-Id: I8d1b52bcf8f8c288892492b4375178756c784ec3
- Fix issue with dropping events on start
During the configuration reload ReplicationQueue dropped reference
updated events. Because replication scheduling is not created and
events are not preserved replication for these events never happened.
Solution is to add in memory queue which holds events until plugin
is up and running and then fire those events to make sure that all
replication happened.
Bug: Issue 11573
Change-Id: I3f9767c3879152acc8ff4ab73233abb278fb07a7
* Update plugins/replication from branch 'stable-2.16'
to df1008552b16d3ea74d8963728482431a127754c
- Revert "PushOneTest: Remove unused mock ReplicationQueue"
This reverts commit 04bd5274b5b9d852989f92ad42a3ceea3f917238.
Reason for revert: ReplicationQueue is still being used and cannot be
removed.
Change-Id: I1d573a579e714e71757c3483a98bad2125fd4865
* Update plugins/replication from branch 'stable-2.16'
to 04bd5274b5b9d852989f92ad42a3ceea3f917238
- PushOneTest: Remove unused mock ReplicationQueue
Change-Id: I162fb33059ef954b437d230cef05f732752dc137
- ReplicationStateTest: Remove unused mock ReplicationTasksStorage
Change-Id: I285805381512d76437ad740764697784b8f39c9e
- Wrap calls to createProject in a utility method
The createProject method is removed in stable-3.0, and creating a test
project must be done with the ProjectOperations class. This means that
when a new test is added in stable-2.16 and uses createProject, there
is a conflict when merging it up to stable-3.0.
Wrap all the calls to createProject in a new utility method and call
that instead in the tests. This commit will conflict on the next merge
up to stable-3.0, but then there will be no more conflicts in future.
Change-Id: I271d0d55cd07c25eb4c6f8acc47d144b47600356
- ReplicationIT: stop using EasyMock
Integration tests should never use mocks as they are always
supposed to use the real implementation and the integration
between components.
Remove the use of EasyMock as it has a very limited
contribution and impact on ReplicationIT.
Change-Id: I61d42c69164484486187f123a2fa26f8810c1924
* Update plugins/singleusergroup from branch 'stable-2.16'
to 2a38a8413c25fec910fb843932aa869647bdc1e7
- SingleUserGroupTest: Remove unnecessary groupBackend member
The GroupBackend is provided in the base class.
Change-Id: I61500178ca37af70e002f4d46fcdc534f8cde325
* Update plugins/singleusergroup from branch 'stable-2.16'
to 8a81e853a07b1bbd28766793e31d44787dfde1f6
- Fix the testAllNumericUserGroup all-numeric username
Bug: Issue 11498
Change-Id: Iee6dc050f8d1e97920c3e105af7bd2a69479dcea
- Merge branch 'stable-2.15' into stable-2.16
* stable-2.15:
Test all-numeric username
Change-Id: Ie659c4e19b9258dcb5b4ada96305d47b591cd5b7
- Test all-numeric username
Verify that a username with all numbers works as expected
and generates an equivalent group associated with it.
Change-Id: Ie98181fdecbf196c320b8b9d0ca0c7bd88b0d238
* Update plugins/replication from branch 'stable-2.16'
to 434edc81349b5c6efcbcadd809c71d13c787296d
- Do not synchronize replication config shutdown
Remove the synchronization of the AutoReloadConfigDecorator
shutdown because of potential deadlocks detected in the replication tests.
When replication events drainage is enabled, the shutdown operation may
deadlock when trying to access the configuration.
Change-Id: Ief6f2bee9124ee0f252e6671aa4a4829c6356b0d
* Update plugins/replication from branch 'stable-2.16'
to 41dcd7a9830830e0a1592462086455a2a6240e0f
- ReplicationIT: Replace ExpectedException with assertThrows
The ExpectedException is removed in later branches. Replace it
in stable-2.16 to avoid unnecessary rework when merging up.
Change-Id: I65b18bf77f9d5816e7d2c9533535d0c307f65a46
- ReplicationFileBasedConfig#EventQueueNotEmptyException: Add missing serialVersionUID
Change-Id: I4bc1d415f32ac5511f41bf9ecbd59aa4ff104d2e
* Update plugins/replication from branch 'stable-2.16'
to 628f4795d07c783720dee4ef51374810f3a41649
- ReplicationIT: speedup drain replication queue tests
Do not introduce artificial delays for the testing of the
replication queue drain.
Explicitly shutdown the replication config immediately
after the push operation, so that even a small 1s replication
delay is good enough to test the functionality.
Change-Id: I68caea21675875a947c2b520d265446ca2f7fe30
- ReplicationIT: Remove unused variables in tests
Change-Id: Ic652fd71748bf7899377f5e4b3ef5e3f9d7d819a
* Update plugins/replication from branch 'stable-2.16'
to 7927786d06cf0b8ae046e3ea8e8283e16f5db3e1
- Reformat with GJF
Change-Id: I5e476d79c33f3e2b9bb45f3f9ea0396b20072854
* Update plugins/replication from branch 'stable-2.16'
to bf66e9959da0b3b16881687e8ed1dacafbbb5d04
- Option to drain the event queue before shutdown
The node can be down for a while and the replication events could be
missing for all that time.
When the node is coming up again, the SHA1 should be then obsolete
and then effectively lost.
Allow the Gerrit administrator to avoid delaying replication events
by automatically making the shutdown of the replication queue more
graceful and wait for all in-flight replications tasks to end before
shutting down the system.
Bug: Issue 11145
Change-Id: I9d823cd64176b4987d7ba71d9a46e717d0f52b93
* stable-2.15:
Bazel: Add fixes for --incompatible_load_{java|python}_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Lucene index configuration and docs.
Change-Id: I6c597cbc89fafece83c374e9b36c4c4c0126704f
* Update plugins/replication from branch 'stable-2.16'
to 60f470c96232d44c5fd83c75ac99174b356adcf5
- Persist replication task when pushing all refs
Create a replication event associated with the replication of all
refs, so that the completion of the operation would find a corresponding
task and will not throw an exception.
Another benefit also is the ability to replay pending replication
events for a push of all refs upon plugin restarts and configuration
reloads.
Bug: Issue 11424
Change-Id: I63a82ff128c98e0ab7e58b76fa8f8548eb74e915
* stable-2.14:
Bazel: Add fixes for --incompatible_load_{java|python}_rules_from_bzl
Bazel: Bump minimum supported version to 0.29.0
Lucene index configuration and docs.
Change-Id: I0a8c17c853746ca7367cc4b723b18f8d0f2b6094
This change is fixing "All Java build rules should be loaded from
Starlark" warning flagged by latest buildifier version: [1]. Python
rules are now also loaded from the Starlark.
Also extract codemirror library import to BUILD file. This is needed to
avoid cycle in the workspace file, after importing java rules from
Starlark.
[1] https://github.com/bazelbuild/buildtools/blob/master/WARNINGS.md#native-java
Change-Id: I36192c9465d988b25cf09c250e110f15850910cd
* Update plugins/replication from branch 'stable-2.16'
to bfc9e0f05d1da46118723d846e9e450455d89b74
- ReplicationIT: fix flakiness
Fix the flakiness when checking for replication tasks stored
events on the filesystem and when replicating new projects.
Make use of the ReplicationTasksStorage for listing events
instead of listing the filesystem and allow disabling the deletion
so that the tests can have the time to verify the files existence.
Change-Id: Idfba7d177faed3c3ff45507b6f5b6aadb0bc5dd0
* Update plugins/replication from branch 'stable-2.16'
to a1b83633a72c46a9ad641342fc7fe1e30631fafb
- ReplicationIT: reduce execution time
Change-Id: I259aa073ae16234d2aaa556012cfa2f584aca748
* Update plugins/replication from branch 'stable-2.16'
to 53af4e7839eb4b386584446ff8c4a53a0b706180
- ReplicationIT: Split waitUntil to a separate class and fix it
The waitUntil method does not fail the test if the expected condition
is not satisfied within the timeout; it just returns.
Split it out to a separate class, to make it easier to test, fix
it, and add tests.
Change-Id: I6cbd6801bcfd07869dd56d3ab6b54c5ac7c8a691
* Update plugins/replication from branch 'stable-2.16'
to eef56952d3f5b05923ebcd07fdc1741a6bf9cc0e
- ReplicationIT: Fix typo in constant name
Change-Id: I349fc79f2b78d1d0bee23b516e37c29705f58a08
* Update plugins/replication from branch 'stable-2.16'
to ba9f04aece217d477e0ec3973aec3fd1b32bc6ab
- ReplicationIT: fix flakiness of shouldReplicateNewBranch
shouldReplicateNewBranch() should compare the target branch with
the source SHA1 and not with the target.
Comparing two SHA1 on the target repository is flaky because
the two replication events may come at different times.
Change-Id: Icfc4da705bc07805e881faea517984fb2c9abe09
* Update plugins/replication from branch 'stable-2.16'
to 729e1df4b4dbc73bcf7f97dfdb55301343628ef3
- ReplicationIT: remove dependency of ReviewDb/NoteDb
Tests need to be independent on the underlying changes storage
and thus make assertions of the expected replication tasks and
the associated refs regex.
Change-Id: I1e645339888deef609670ca3e58517a159e93b2b
* Update plugins/replication from branch 'stable-2.16'
to b48b29cc779020278f9a64b473d67f373cdf085b
- ReplicationIT: Add a test that HEAD update is replicated
Change-Id: I140a63933fe50bc6d5ec109fe6d46a2371a89f1e
- ReplicationIT#shouldReplicateNewBranch: Create branch on local project
The test creates the branch directly on the target project, which
defeats the point of the replication test since the branch is not
actually getting replicated there.
Fix it to create the branch on the local project, which was the
original intention when the test was added in change I67aa8b4ae.
Change-Id: I517f3f49241207861e2f5158c08f6193a63d8916
* Update plugins/replication from branch 'stable-2.16'
to b1b1e67465084779511d9117bb97528ae8e8b8af
- ReplicationIT: Don't swallow exceptions on failed repository operations
Change I02d8cda0a undid the improvements in the tests that were
done by change I0f0caca03.
Rework the tests to reintroduce those improvements.
Change-Id: I812674325bb6756fbcdc3d9674fc8ca69c8f3c82
* Update plugins/replication from branch 'stable-2.16'
to 29aa49702e85f922d367bd27719f31d09770207a
- ReplicationIT: Fix typo in method name
Change-Id: I5a4d468f18cd50009132c0c516f3475906e04c7f
* Update plugins/replication from branch 'stable-2.16'
to 236d9a6468c6e91f3d5135413adbf06071770625
- AdminApi: Reintroduce return value of methods
Prior to change Ie760bf3e1 all the methods of GerritSshApi returned
a boolean value. After that change, they all return void.
Removing the return value breaks classes that extend GerritSshApi and
expect to be able to invoke the superclass implementation and get the
status.
In change I6566f8671 the return value of createProject was changed
back to boolean, but the others were not.
Change the remaining methods back to return a boolean.
Change-Id: If4a9227f70bfcd85965e1febd45f4a75cc2505bd
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
* Update plugins/replication from branch 'stable-2.16'
to d4981a1921e48fe76d97f7cc6107ae074879bfbe
- Store replication tasks instead of ref-update events
Fix the granularity of persistent events on disk by storing
replication tasks rather than ref-update events.
Ref-updates are triggering the replication tasks,
however it is tricky to perform reference counting on the associated
tasks reliably, due to all the different conditions of where a
replication task can be in the replication queue.
Fix a recurring issue on the persistence of ref-update events.
The persisted ref-update events were either removed prematurely on the
filesystem or left forever orphan even after the replication was completed
on all remotes and target URLs.
The problem was caused by the different granularity of the incoming
ref-update events and the corresponding replication tasks queued
and executed.
Do not persist ref-update events anymore but include remote and
target URLs at the replication tasks and persist those instead.
The stored objects are replication tasks and not anymore ref-update
events.
Bug: Issue 11172
Change-Id: I02d8cda0a124e8e3d2b9bb01b7d44f98ba717fcd
* Update plugins/replication from branch 'stable-2.16'
to 896e67184e9b98262775ab1c71a0b76ac5baff21
- ReplicationIT: Don't swallow exceptions on failed repository operations
The methods getRepo and getRef catch exceptions and return null, which
will then cause NullPointerException on subsequent dereference, making
it less easy to track down the actual cause.
Remove the getRepo method and always open the repo in a try-with-resource
that will throw the exception and fail the test if the repository could
not be opened.
Some calls to getRef are done in a consumer which means the method can't
throw an exception. In these cases, it's OK for the method to return
null because then the test will fail anyway due to InterruptedException
being raised when the non-null was not returned within the timeout.
Other callers of getRef however should not get the null value as it will
cause NPE.
Make getRef throw the exception, thus failing the test early, and add a
variant method "checkedGetRef" which returns null and is used in the
consumer.
Also replace usage of printStackTrace with FluentLogger.
Change-Id: I0f0caca03386c4fa5710f4c8e8fd4f798acead2b
- ReplicationIT: Add explicit test for replication of new branch
Change-Id: I67aa8b4aec05ad23b3c286d56a3b894eede7cd7e
- ReplicationIT: Change test method name to reflect actual operation
The test shouldReplicateNewBranch creates a new change and then
verifies that the corresponding refs/changes/xx/yy/zzzzzz ref is
replicated to the destination.
Rename it to shouldReplicateNewChangeRef to match what it's actually
doing.
Change-Id: I32fe7237c8857907dd51d627a571f6ccb73741a3
- Allow AdminApiFactory to be replaced dynamically
Since change Ie760bf3e1 the GerritSshApi class is no longer a singleton,
and is instead instantiated on demand by a new AdminApiFactory.
This breaks implementations that consume the replication plugin as a
library and extend the GerritSshApi, since the extended class no longer
gets instantiated in place of GerritSshApi.
Refactor it so that AdminApiFactory is an interface with a default
implementation that gets bound as a dynamic item, which can be replaced
by derived implementations.
Change-Id: Ia150d6802e11015fa00ee9144b3dfbfa696c7a0d
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
* Update plugins/replication from branch 'stable-2.16'
to 86b3ac835c651cbaff0c0dd9b2540ceee4c32a78
- Fix test build rule to include *IT tests
The glob in the build rule only includes *Test.java files, thus the
recently added ReplicationIT was not being executed.
Update the rule to include *IT.java files.
Change-Id: I04a76723feb6e38dd441b41e9de22e5fdf1c4578
* Update plugins/replication from branch 'stable-2.16'
to 4ed1a3eaa052672422f5f19468736de42e4d4d27
- ReplicationIT: Do not use deprecated getRef()
Change-Id: Idf56bcb4d9e7ed514d1011f62d6738d124ba870a
* Update plugins/replication from branch 'stable-2.16'
to d5170016dbb9db03f686c2a6f4994295b685aeb8
- First ReplicationIT test for new project and change
Test end-to-end that a new project and a new change are propagated
to the replicated repository.
Change-Id: Iea4170aa881757c7e29bc6237fe4c230da8ae0ec
* Update plugins/replication from branch 'stable-2.16'
to 24ae8627fef3aef243e8e22c30901617f724ad4c
- Revert "Delete event file only after replication completed for all destinations"
This reverts commit d557ccc642c59a55750f560ce0d98870e1550d65.
Reason for revert: This fix did not solve the problem, see associated issue.
Bug: Issue 11172
Change-Id: Ifc2e7209bd86163e945f0595ff0ebd681932053f
* Update plugins/replication from branch 'stable-2.16'
to bc9c7eaca297578e2e86e961118336a72227ac87
- GerritSshApi: Consistently use logError method
Change-Id: I9aeb27ce9e535c177fcdab3bee417c77233e8310
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
- GerritSshApi: Make sshHelper and uri members protected
This allows them to be accessed by classes that extend GerritSshApi.
Change-Id: Ie2ca2da1303ed4a14ac6074e37696c5f509f3ebe
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
* Update plugins/replication from branch 'stable-2.16'
to 23037c51be039e2e94d9f4fd54d4d38cdc88ac82
- Revert "Revert "Do not reload config when queue is not ready""
This reverts commit bcb15e00b3bd8804417d7401cff12a941bdb8aee.
The test that was previously failing on stable-3.0 has been fixed
with Change-Id: I2bef00e701 and thus does not represent anymore an
impediment for getting this fix merged in stable-2.16 and stable-3.0.
Change-Id: I936e3547ff4533850d538fe6b9fada7437bf4e49
* Update plugins/replication from branch 'stable-2.16'
to 29875954b9e766620f31991f425020ae22ade485
- Add replication refs-filtering before push
Add ability to filter out refs before being pushed for replication to
remote instance. This will help to prevent split-brain issue by
allowing us to create filter in multi-site plugin to stop the out of sync
instance from overriding the changes of the instance that is up to date.
Bug: Issue 11175
Change-Id: I99a7cb4fb2f231626d50abc7d90bb3d79c23861a
* Update plugins/replication from branch 'stable-2.16'
to d33615ca4eae3f73b8b90e46edefc1584ebf458d
- Create missing repository only at the remote we are pushing to
When replication of a ref failed due to missing remote repository at one
remote site, we then initiated creation of the missing repository on all
remotes where this repository is replicated. The creation of the missing
repository is considered to be success if it succeeded for all remotes
and only in this case the ref replication was rescheduled. This logic
could cause a ref replication to not be rescheduled when creation of a
missing repository fails in a remote which is unrelated to the current
push operation. For example, if we replicate to two remote sites:
[remote "foo"]
url = https://foo.org/...
adminUrl = ssh://gerrit@foo.org/...
[remote "bar"]
url = https://bar.org/...
adminUrl = ssh://gerrit@bar.org/...
and if during a push to "foo" we find that the repository is missing
then we will try to create it on both "foo" and "bar". It could happen
that the creation of the missing repository succeeds on "foo" and then
fails on "bar" because "bar" is unreachable or down at that moment.
In this case the replication to "foo" will not be rescheduled but it
should be as we successfully created the missing repository in "foo".
Create missing repository only under the (admin) URL in the same remote.
Note that there will be one ref replication task for each remote which
ensures that the missing repository will be created in all remotes.
Change-Id: Idbc3f614df53bb3aabcc4cf6cb0d6540e7000e29
(cherry picked from commit abafb83c47eb50c0ec5ce9a4b8d04dc355d5a5bf)
* Update plugins/replication from branch 'stable-2.16'
to ee867cafde59ff6e02c2a1cef07a5fb82423e9ac
- Remove the NewProjectCreatedListener implementation
Initial version of this plugin implemented the NewProjectCreatedListener
in order to create missing repositories in the replication targets. This
was the only way how replication plugin created missing repositories.
Since I4e587cdfca09445c9b1c528b2f1edae0944aec68, if during the
replication of a ref it is found that the repository missing on the
remote site it will be automatically created. This means, that since
that change there are two ways a repository is created on the remote
site:
1: from the NewProjectCreatedListener.onNewProjectCreated
2: during ref replication when the repository is missing
There are two major differences in how cases 1 and 2 are invoked. In the
case 1 the remote repository creation is performed from the calling
thread, which means from the Gerrit core thread which invokes
NewProjectCreatedListener(s). In the case 2, the creation of the remote
repository is done from the replication queue thread which was
processing the ref replication. Note that replication tasks are created
with an additional child injector [1] which provides additional bindings
available for injection into the classes implementing the replication
and repository creation. These binding are, however, not available when
the repository creation is processed from the Gerrit core thread
invoking NewProjectCreatedListener(s).
Removing the NewProjectCreatedListener implementation removes this
asymmetry and makes sure that all replication relevant steps, including
repository creation, are processed from replication queue threads.
[1] 871da5aa02/src/main/java/com/googlesource/gerrit/plugins/replication/Destination.java (189)
Change-Id: I32e7aa632ffe0e94eb525f7f2c007fbb88569004
(cherry picked from commit c16fe9c5a6da6c23dbf45a533f788dcc527e209c)
* Update plugins/replication from branch 'stable-2.16'
to aba8c72aaf29d0ece59779dc59e4ed6e97c401a0
- Consistently handle remote repository creation failures
Failure to create remote repository was handled inconsistently in the
three existing AdminApi implementations. This issue was present in the
code even before the refactoring done in
Ie760bf3e143b1d143b6e81ac6cfa816ef1f8d016. For example, creation of
remote repository over SSH [1] didn't return a status back to the
caller, while the Gerrit+SSH implementation [2] did return a boolean
flag which was afterwards ignored by the caller [3].
Not handling remote repository creation failures had an ugly effect:
we log the "Missing repository created; ..." [4], even if the repository
creation failed.
Let all three implementations of the AdminApi return a flag indicating
the success/failure and make sure to handle it properly.
Further, remove some redundant and potentially confusing parts of the
log messages as they assume that failure to create missing repository
can only be caused by using wrong protocol.
[1] d557ccc642/src/main/java/com/googlesource/gerrit/plugins/replication/ReplicationQueue.java (290)
[2] d557ccc642/src/main/java/com/googlesource/gerrit/plugins/replication/GerritSshApi.java (43)
[3] d557ccc642/src/main/java/com/googlesource/gerrit/plugins/replication/ReplicationQueue.java (259)
[4] 092792edac/src/main/java/com/googlesource/gerrit/plugins/replication/PushOne.java (400)
Change-Id: I6566f867138ab73c81ff7a67630772c3734e501b
(cherry picked from commit b95d921d6db891dc724f619d957ab35e634f7e44)
* Update plugins/replication from branch 'stable-2.16'
to 902b60ed3ce34b63b41c454f4cdfeb61f0310a90
- Merge branch 'stable-2.15' into stable-2.16
* stable-2.15:
Fix creation of missing repository when replicating to a Gerrit server
Change-Id: I35712167582fc0a1765707ae13b6aabf85374f33
- Fix creation of missing repository when replicating to a Gerrit server
When replicating to a Gerrit server over http, non-existence of a
repository is reported as:
<repo-name> unavailable
This message wasn't among expected messages and creation of the missing
repository wasn't executed.
Bug: Issue 11204
Change-Id: I3d6c3e14573b638b17d54d1777a606b8f356f1f9
* Update plugins/replication from branch 'stable-2.15'
to 9b4b786ffdbbe855fe44f8f371e5660b64d94f3e
- Fix creation of missing repository when replicating to a Gerrit server
When replicating to a Gerrit server over http, non-existence of a
repository is reported as:
<repo-name> unavailable
This message wasn't among expected messages and creation of the missing
repository wasn't executed.
Bug: Issue 11204
Change-Id: I3d6c3e14573b638b17d54d1777a606b8f356f1f9
* Update plugins/replication from branch 'stable-2.16'
to bcb15e00b3bd8804417d7401cff12a941bdb8aee
- Revert "Do not reload config when queue is not ready"
This reverts commit a3cf8a61980a84ae710b15cb93d9a2a7423d93cf.
Breaks the test on stable-3.0. This should be re-done after the
tests have been backported to stable-2.16.
Change-Id: I147ee42cf3607aa1943a5f57070687a9f0948036
* Update plugins/replication from branch 'stable-2.16'
to a3cf8a61980a84ae710b15cb93d9a2a7423d93cf
- Do not reload config when queue is not ready
Do not stop/start the replication queue upon a new configuration
added when the queue is either not running yet or is replaying
incoming events.
Restarting a partially initialized queue would have unexpected
consequences as the objects that are needed to schedule and
serve the requests could be still null and generate random NPEs
all around.
The replay also is a very delicate phase because it is responsible
for the replay of the past missed replication events that should
be re-triggered with highest priority possible. All new events
and new configuration can wait to be processed once all the
past events have been replayed and have completed.
Bug: Issue 11055
Change-Id: Ib154532ec32a50f4cb9ca5553c1f77fa931ae2ac
* Update plugins/replication from branch 'stable-2.15'
to 3b1b656d09eb2dd73f94f50772bbeb39d167b540
- Revert "Remove replication event from pending when runway is allowed"
This reverts commit e780ae61cbbba3f88558a3620065d1fcdc0768c8.
Reason for revert: Creates potentially infinite Heap consumption and
JVM crash when a PushOp is denied to use the runway.
The problem is caused by the late removal of the PushOp from the
list of pending operations, which causes the subsequent reschedule
to find the same operation and double the number of replication states
associated with it. If the loop happens multiple times, the generated
consumption grows exponentially causing the JVM to enter a series
of continuous stop-the-world GC that eventually lead to complete
block of the JVM activity.
The method `requestRunway` is thus not telling the truth because
it is actually relying on the fact that the PushOne operation is
removed also from the pending list, independently from its runway
concession status.
Change-Id: I6097bc7ad16c8bcc86a7d30af7d2ad331728712d
* Update plugins/replication from branch 'stable-2.15'
to 0df288afccb53b7abbe4f2ee90f68c80e9dcd2c7
- ReplicationFileBasedConfig: Fix setting default sshConnectionTimeout
The default value did not get set when the replication config did not
exist, was empty, or was invalid. This is not really a problem since
if there is no config, no SSH connections are made anyway.
A more serious issue is that the value was read with minutes as the
default time unit, but then converted to milliseconds using seconds as
the time unit, which resulted in an incorrect value.
Change-Id: I64906e29acb56f0f53b432db61d2707dfe1963d3
* Update plugins/replication from branch 'stable-2.15'
to 7f94fed1b567713a629f7621ff2c511767e448ab
- SshHelper: Add class javadoc
Change-Id: I80ab0c2225ad02628b630889c299c474e0087d70
- Make more classes and fields public to ease extensibility
Make classes public, and their constructors protected, to allow
them to be extended.
Make event type names and fields public.
Change-Id: Iba275f99b7afbf87d57dd44851d043ad0f23fbe1
* Update plugins/replication from branch 'stable-2.15'
to 624ec820682e0d7b163e1de262134b7b94087441
- Allow to configure timeout for SSH connection and commands
The timeout for SSH command execution is hard-coded to 0 which means
no timeout, and the timeout for SSH connections is hard-coded to 2
minutes.
Introduce new configuration options to allow configuring each timeout
individually:
gerrit.sshCommandTimeout
gerrit.sshConnectionTimeout
When not set, both default to their previously hard-coded values,
i.e. no limit and 2 minutes respectively.
Change-Id: Ibd377d45543ef4631082e8d521aeeb99209003f7
Signed-off-by: Dariusz Luksza <dluksza@collab.net>
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
* Update plugins/replication from branch 'stable-2.15'
to 95dc833b63a1629608c64e343ab07fa6cafc84a9
- StartCommand: Fix synchronization on non-final field
When all error prone warnings are enabled the SynchronizeOnNonFinalField
bug pattern is reported:
plugins/replication/src/main/java/com/googlesource/gerrit/plugins/replication/StartCommand.java:103:
error: [SynchronizeOnNonFinalField] Synchronizing on non-final fields is not safe:
if the field is ever updated, different threads may end up locking on different objects.
synchronized (stdout) {
^
(see https://errorprone.info/bugpattern/SynchronizeOnNonFinalField)
Change-Id: Ib2df20aa28af4edd36ce5b9dfcf7d82c409dab84
* Update plugins/replication from branch 'stable-2.15'
to 404e5069adbdf7d9ab17b887894842ad3638042d
- Destination: Suppress FutureReturnValueIgnored warning
When all error prone warnings are enabled the FutureReturnValueIgnored
bug pattern is reported, for example:
plugins/replication/src/main/java/com/googlesource/gerrit/plugins/replication/Destination.java:367:
error: [FutureReturnValueIgnored] Return value of methods returning Future must be checked.
Ignoring returned Futures suppresses exceptions thrown from the code that completes the Future.
pool.schedule(e, now ? 0 : config.getDelay(), TimeUnit.SECONDS);
^
(see https://errorprone.info/bugpattern/FutureReturnValueIgnored)
Did you mean to remove this line?
Change-Id: I43b5c3c9f9bf8cda5f2d4ee701b2153e7e6f2807
* Update plugins/replication from branch 'stable-2.16'
to 57c0f0be81242067a01f6639a659bbdd3904e274
- Merge branch 'stable-2.15' into stable-2.16
* stable-2.15:
Cancel pending replications upon shutdown
Remove replication event from pending when runway is allowed
Change-Id: I81209708a42832cbe2be818e2dd50f625362b392
- Cancel pending replications upon shutdown
When the replication plugin is stopped or reloaded,
mark all the current replications as cancelled.
Having pending and unmanaged replication tasks is quite
dangerous for two reasons:
1. The result of the replication isn't accounted and
properly managed, because the associated objects won't
be there anymore (pending replications, runway, ...)
2. The destination configuration could have changed completely
(auto-reload use-case) or even not exist anymore: the
replication event would thus perform an unwanted and
unconfigured operation.
With regards to the replication events that are already
on the runway, there is no value in cancelling them. Just flag
them as cancelled so that, once they finish, can be clearly
recognized.
Change-Id: Iabc17d375011cbc61f8c642ae07d3d018b49fc69
- Remove replication event from pending when runway is allowed
Consider a replication event not pending anymore *only if* the runway
is free and allowed to run. Removing it too early would result in a
ghost event which isn't officially pending but that will be eventually
run when rescheduled.
Change-Id: Ib0ffa3bec7a6d5d5be56e78a677e8b2133f91e18
* Update plugins/replication from branch 'stable-2.15'
to 20bc91d0bd11dd2c9b77aedb6bebb02fa67f70f4
- Cancel pending replications upon shutdown
When the replication plugin is stopped or reloaded,
mark all the current replications as cancelled.
Having pending and unmanaged replication tasks is quite
dangerous for two reasons:
1. The result of the replication isn't accounted and
properly managed, because the associated objects won't
be there anymore (pending replications, runway, ...)
2. The destination configuration could have changed completely
(auto-reload use-case) or even not exist anymore: the
replication event would thus perform an unwanted and
unconfigured operation.
With regards to the replication events that are already
on the runway, there is no value in cancelling them. Just flag
them as cancelled so that, once they finish, can be clearly
recognized.
Change-Id: Iabc17d375011cbc61f8c642ae07d3d018b49fc69
- Remove replication event from pending when runway is allowed
Consider a replication event not pending anymore *only if* the runway
is free and allowed to run. Removing it too early would result in a
ghost event which isn't officially pending but that will be eventually
run when rescheduled.
Change-Id: Ib0ffa3bec7a6d5d5be56e78a677e8b2133f91e18
* Update plugins/replication from branch 'stable-2.16'
to d8cc3f9e293d4fea1131e23d7ad9fac51d63589b
- Merge branch 'stable-2.15' into stable-2.16
* stable-2.15:
Add missing log statement if replication was canceled
When rescheduling due to in-flight push log also the in-flight task ID
Make sure to always remove in-flight pushes
Change-Id: I938d5ad49e94865515d8e2fe4694da8a605fb53a
- Add missing log statement if replication was canceled
Change-Id: Id06bae985cfdb000c66743ccd3c5d9f56d34d681
- When rescheduling due to in-flight push log also the in-flight task ID
When a push operation was rescheduled due to an in-flight push we logged
a line like:
[79bda07a] Rescheduling replication to https://host/foo.git to avoid collision with an in-flight push.
This log was already useful but it didn't tell us which push task was
actually in-flight.
Add the ID of the in-flight push task to the log:
[79bda07a] Rescheduling replication to https://host/foo.git to avoid collision with the in-flight push [80ceb18b].
This information is especially useful when a task gets repeatedly
rescheduled and we need to identify the root cause of that.
(cherry picked from commit f585eba0d21be222404b871bfe7a2b11e578eb14)
Change-Id: I7866f964cab1e4f479b0d7d62ae4aac3019fab4b
- Make sure to always remove in-flight pushes
When a push is finished successfully or fails due to an error we have to
remove it from the map of in-flight pushes. Although this was done from
a finally block, the same block performed a Repository.close() operation
prior to the in-flight push removal. If the Repository.close() threw an
exception the pool.notifyFinished() wouldn't be called and we would end
up with a zombie in-flight push.
Bug: Issue 10852
Change-Id: I8d5918d5271ba74ce12153f054503adaef155c5c
(cherry picked from commit 6fcdcf98fe61e8c6258fb40967f0cff6b8c38fd4)
