opendev/gerrit
Code Issues Proposed changes

Move replication logic to replication plugin

Browse Source 
This splits all of the replication code out of the core server
and moves it into a standard plugin. A new listener API is used
inside of the core server to notify interested plugins of any Git
reference changes, which the replication code can hook into to
schedule its events.

Change-Id: I77ee4440a009c2ce1c62fb6a445c7e3c912245f9
This commit is contained in:
Shawn O. Pearce 2012-05-10 19:12:09 -07:00
parent 795167c05e
commit 7d2cb04d07
56 changed files with 306 additions and 2055 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
Documentation/access-control.txt
View File

@ -1142,7 +1142,8 @@ command, but also to the web UI results pagination size.
Start Replication
~~~~~~~~~~~~~~~~~
Allow access to execute link:cmd-replicate.html[the `gerrit replicate` command].
Allow access to execute `replication start` command, if the
replication plugin is installed on the server.
[[capability_viewCaches]]

8
Documentation/cmd-create-project.txt
View File

@ -163,7 +163,8 @@ the single quotes to delimit the value.
REPLICATION
-----------
The remote repository creation is performed by a Bourne shell script:
If the replication plugin is installed, the plugin will attempt to
perform remote repository creation by a Bourne shell script:
====
mkdir -p '/base/project.git' && cd '/base/project.git' && git init --bare && git update-ref HEAD refs/heads/master
@ -174,10 +175,13 @@ arbitrary shell scripts, and must have `git` in the user's PATH
environment variable. Administrators could also run this command line
by hand to establish a new empty repository.
A custom extension or plugin may also be developed to implement the
NewProjectCreatedListener extension point and handle custom logic
for remote repository creation.
SEE ALSO
--------
* link:config-replication.html[Git Replication/Mirroring]
* link:project-setup.html[Project Setup]
GERRIT

3
Documentation/cmd-index.txt
View File

@ -111,9 +111,6 @@ link:cmd-flush-caches.html[gerrit flush-caches]::
link:cmd-gsql.html[gerrit gsql]::
Administrative interface to active database.
link:cmd-replicate.html[gerrit replicate]::
Manually trigger replication, to recover a node.
link:cmd-set-project-parent.html[gerrit set-project-parent]::
Change the project permissions are inherited from.

103
Documentation/cmd-replicate.txt
View File

@ -1,103 +0,0 @@
gerrit replicate
================
NAME
----
gerrit replicate - Manually trigger replication, to recover a node
SYNOPSIS
--------
[verse]
'ssh' -p <port> <host> 'gerrit replicate'
[--url <PATTERN>]
{--all | <PROJECT> ...}
DESCRIPTION
-----------
Schedules replication of the specified projects to all configured
replication destinations, or only those whose URLs match the pattern
given on the command line.
Normally Gerrit automatically schedules replication whenever it
makes a change to a managed Git repository. However, there are
other reasons why an administrator may wish to trigger replication:
* Destination disappears, then later comes back online.
+
If a destination went offline for a period of time, when it comes
back, it may be missing commits that it should have. Triggering a
replication run for all projects against that URL will update it.
* After repacking locally, and using `rsync` to distribute the new
pack files to the destinations.
+
If the local server is repacked, and then the resulting pack files
are sent to remote peers using `rsync -a --delete-after`, there
is a chance that the rsync missed a change that was added during
the rsync data transfer, and the rsync will remove that changes's
data from the remote, even though the automatic replication pushed
it there in parallel to the rsync.
+
Its a good idea to run replicate with `--all` to ensure all
projects are consistent after the rsync is complete.
* After deleting a ref by hand.
+
If a ref must be removed (e.g. to purge a change or patch set
that shouldn't have been created, and that must be eradicated)
that delete must be done by direct git access on the local,
managed repository. Gerrit won't know about the delete, and is
unable to replicate it automatically. Triggering replication on
just the affected project can update the mirrors.
ACCESS
------
Caller must be a member of the privileged 'Administrators' group,
or have been granted
link:access-control.html#capability_startReplication[the 'Start Replication' global capability].
SCRIPTING
---------
This command is intended to be used in scripts.
OPTIONS
-------
--all::
Schedule replicating for all projects.
--url <PATTERN>::
Replicate only to replication destinations whose URL
contains the substring <PATTERN>. This can be useful to
replicate only to a previously down node, which has been
brought back online.
EXAMPLES
--------
Replicate every project, to every configured remote:
====
$ ssh -p 29418 review.example.com gerrit replicate --all
====
Replicate only to `srv2` now that it is back online:
====
$ ssh -p 29418 review.example.com gerrit replicate --url srv2 --all
====
Replicate only the `tools/gerrit` project, after deleting a ref
locally by hand:
====
$ git --git-dir=/home/git/tools/gerrit.git update-ref -d refs/changes/00/100/1
$ ssh -p 29418 review.example.com gerrit replicate tools/gerrit
====
SEE ALSO
--------
* link:config-replication.html[Git Replication/Mirroring]
GERRIT
------
Part of link:index.html[Gerrit Code Review]

15
Documentation/config-gerrit.txt
View File

@ -1076,11 +1076,6 @@ By default unset, as the HTTP daemon must be configured externally
by the system administrator, and might not even be running on the
same host as Gerrit.
[[gerrit.replicateOnStartup]]gerrit.replicateOnStartup::
+
If true, replicates to all remotes on startup to ensure they are
in-sync with this server. By default, true.
[[gitweb]]Section gitweb
~~~~~~~~~~~~~~~~~~~~~~~~
@ -2325,15 +2320,6 @@ Sample `etc/secure.config`:
password = s3kr3t
----
File `etc/replication.config`
-----------------------------
The optional file `'$site_path'/etc/replication.config` controls how
Gerrit automatically replicates changes it makes to any of the Git
repositories under its control.
* link:config-replication.html[Git Replication/Mirroring]
File `etc/peer_keys`
--------------------
@ -2369,7 +2355,6 @@ Files in this directory provide additional configuration.
Other files support site customization.
+
* link:config-headerfooter.html[Site Header/Footer]
* link:config-replication.html[Git Replication/Mirroring]
GERRIT
------

273
Documentation/config-replication.txt
View File

@ -1,273 +0,0 @@
Gerrit Code Review - Git Replication
====================================
Gerrit can automatically push any changes it makes to its managed Git
repositories to another system. Usually this would be configured to
provide mirroring of changes, for warm-standby backups, or a
load-balanced public mirror farm.
The replication runs on a short delay. This gives Gerrit a small
time window to batch updates going to the same project, such as
when a user uploads multiple changes at once.
Typically replication should be done over SSH, with a passwordless
public/private key pair. On a trusted network it is also possible to
use replication over the insecure (but much faster) git:// protocol,
by enabling the `receive-pack` service on the receiving system, but
this configuration is not recommended. It is also possible to
specify a local path as replication target. This makes e.g. sense if
a network share is mounted to which the repositories should be
replicated.
Enabling Replication
--------------------
If replicating over SSH (recommended), ensure the host key of the
remote system(s) is already in the Gerrit user's `~/.ssh/known_hosts`
file. The easiest way to add the host key is to connect once by hand
with the command line:
====
sudo su -c 'ssh mirror1.us.some.org echo' gerrit2
====
Next, create `'$site_path'/etc/replication.config` as a Git-style
config file, and restart Gerrit.
Example `replication.config` to replicate in parallel to four
different hosts:
====
[remote "host-one"]
url = gerrit2@host-one.example.com:/some/path/${name}.git
[remote "pubmirror"]
url = mirror1.us.some.org:/pub/git/${name}.git
url = mirror2.us.some.org:/pub/git/${name}.git
url = mirror3.us.some.org:/pub/git/${name}.git
push = +refs/heads/*:refs/heads/*
push = +refs/tags/*:refs/tags/*
threads = 3
authGroup = Public Mirror Group
authGroup = Second Public Mirror Group
====
To manually trigger replication at runtime, see
link:cmd-replicate.html[gerrit replicate].
[[replication_config]]File `replication.config`
-----------------------------------------------
The optional file `'$site_path'/etc/replication.config` is a
Git-style config file that controls the replication settings for
Gerrit.
The file is composed of one or more `remote` sections, each remote
section provides common configuration settings for one or more
destination URLs.
Each remote section uses its own thread pool. If pushing to
multiple remotes, over differing types of network connections
(e.g. LAN and also public Internet), its a good idea to put them
into different remote sections, so that replication to the slower
connection does not starve out the faster local one. The example
file above does this.
[[remote]]Section remote
~~~~~~~~~~~~~~~~~~~~~~~~
In the keys below, the <name> portion is unused by Gerrit, but must be
unique to distinguish the different sections if more than one remote
section appears in the file.
[[remote.name.url]]remote.<name>.url::
+
Address of the remote server to push to. Multiple URLs may
be specified within a single remote block, listing different
destinations which share the same settings. Assuming sufficient
threads in the thread pool, Gerrit pushes to all URLs in parallel,
using one thread per URL.
+
Within each URL value the magic placeholder `${name}` is replaced
with the Gerrit project name. This is a Gerrit specific extension
to the otherwise standard Git URL syntax and it must be included
in each URL so that Gerrit can figure out where each project needs
to be replicated.
+
See link:http://www.kernel.org/pub/software/scm/git/docs/git-push.html#URLS[GIT URLS]
for details on Git URL syntax.
[[remote.name.url]]remote.<name>.adminUrl::
+
Address of the alternative remote server only for repository creation. Multiple URLs may
be specified within a single remote block, listing different
destinations which share the same settings.
+
The adminUrl can be used as a ssh alternative to the url option, but only related to repository creation.
If not specified, the repository creation tries to follow the default way through the url value specified.
+
It is useful when remote.<name>.url protocols does not allow repository creation
although their usage are mandatory in the local environment.
In that case, an alternative ssh url could be specified to repository creation.
[[remote.name.receivepack]]remote.<name>.receivepack::
+
Path of the `git-receive-pack` executable on the remote system, if
using the SSH transport.
+
Defaults to `git-receive-pack`.
[[remote.name.uploadpack]]remote.<name>.uploadpack::
+
Path of the `git-upload-pack` executable on the remote system, if
using the SSH transport.
+
Defaults to `git-upload-pack`.
[[remote.name.push]]remote.<name>.push::
+
Standard Git refspec denoting what should be replicated. Setting this
to `+refs/heads/*:refs/heads/*` would mirror only the active
branches, but not the change refs under `refs/changes/`, or the tags
under `refs/tags/`.
+
Multiple push keys can be supplied, to specify multiple patterns
to match against. In the example file above, remote "pubmirror"
uses two push keys to match both `refs/heads/*` and `refs/tags/*`,
but excludes all others, including `refs/changes/*`.
+
Defaults to `+refs/*:refs/*` (all refs) if not specified.
[[remote.name.timeout]]remote.<name>.timeout::
+
Number of seconds to wait for a network read or write to complete
before giving up and declaring the remote side is not responding.
If 0, there is no timeout, and the push client waits indefinitely.
+
A timeout should be large enough to mostly transfer the objects to
the other side. 1 second may be too small for larger projects,
especially over a WAN link, while 10-30 seconds is a much more
reasonable timeout value.
+
Defaults to 0 seconds, wait indefinitely.
[[remote.name.replicationDelay]]remote.<name>.replicationDelay::
+
Number of seconds to wait before scheduling a remote push operation.
Setting the delay to 0 effectively disables the delay, causing the
push to start as soon as possible.
+
This is a Gerrit specific extension to the Git remote block.
+
By default, 15 seconds.
[[remote.name.replicationRetry]]remote.<name>.replicationRetry::
+
Number of minutes to wait before scheduling a remote push operation
previously failed due to an offline remote server.
+
If a remote push operation fails because a remote server was
offline, all push operations to the same destination URL are
blocked, and the remote push is continuously retried.
+
This is a Gerrit specific extension to the Git remote block.
+
By default, 1 minute.
[[remote.name.threads]]remote.<name>.threads::
+
Number of worker threads to dedicate to pushing to the repositories
described by this remote. Each thread can push one project at a
time, to one destination URL. Scheduling within the thread pool
is done on a per-project basis. If a remote block describes 4
URLs, allocating 4 threads in the pool will permit some level of
parallel pushing.
+
By default, 1 thread.
[[remote.name.authGroup]]remote.<name>.authGroup::
+
Specifies the name of a group that the remote should use to access
the repositories. Multiple authGroups may be specified within a
single remote block to signify a wider access right. In the project
administration web interface the read access can be specified for
this group to control if a project should be replicated or not to the
remote.
+
By default, replicates without group control, i.e replicates
everything to all remotes.
[[remote.name.replicatePermissions]]remote.<name>.replicatePermissions::
+
If true, permissions-only projects and the refs/meta/config branch
will also be replicated to the remote site. These projects and
branches may be needed to keep a backup or slave server current.
+
By default, true, replicating everything.
[[remote.name.mirror]]remote.<name>.mirror::
+
If true, replication will remove remote branches that absent locally
or invisible to the replication (i.e. read access denied via 'authGroup'
option).
+
By default, false, do not remove remote branches.
[[secure_config]]File `secure.config`
-----------------------------------------------
The optional file `'$site_path'/secure.config` is a Git-style config
file that provides secure values that should not be world-readable,
such as passwords. Passwords for HTTP remotes can be obtained from
this file.
[[remote.name.username]]remote.<name>.username::
+
Username to use for HTTP authentication on this remote, if not given
in the URL.
[[remote.name.password]]remote.<name>.password::
+
Password to use for HTTP authentication on this remote.
[[ssh_config]]File `~/.ssh/config`
----------------------------------
If present, Gerrit reads and caches `~/.ssh/config` at startup, and
supports most SSH configuration options. For example:
====
Host host-one.example.com:
IdentityFile ~/.ssh/id_hostone
PreferredAuthentications publickey
Host mirror*.us.some.org:
User mirror-updater
IdentityFile ~/.ssh/id_pubmirror
PreferredAuthentications publickey
====
Supported options:
* Host
* Hostname
* User
* Port
* IdentityFile
* PreferredAuthentications
* StrictHostKeyChecking
SSH authentication must be by passwordless public key, as there is
no facility to read passphases on startup or passwords during the
SSH connection setup, and SSH agents are not supported from Java.
Host keys for any destination SSH servers must appear in the user's
`~/.ssh/known_hosts` file, and must be added in advance, before
Gerrit starts. If a host key is not listed, Gerrit will be unable to
connect to that destination, and replication to that URL will fail.
GERRIT
------
Part of link:index.html[Gerrit Code Review]

11
Documentation/dev-design.txt
View File

@ -675,11 +675,12 @@ real time. Gerrit instances which care about reduduncy will setup
this feature of PostgreSQL or MySQL to ensure the warm-standby is
reasonably current should the master go offline.
Gerrit can be configured to replicate changes made to the local
Git repositories over any standard Git transports. This can be
configured in `'$site_path'/etc/replication.conf` to send copies
of all changes over SSH to other servers, or to the Amazon S3 blob
storage service.
Using the standard replication plugin, Gerrit can be configured
to replicate changes made to the local Git repositories over any
standard Git transports. After the plugin is installed, remote
destinations can be configured in `'$site_path'/etc/replication.conf`
to send copies of all changes over SSH to other servers, or to the
Amazon S3 blob storage service.
Logging Plan

1
Documentation/index.txt
View File

@ -35,7 +35,6 @@ Configuration
* link:config-gerrit.html[System Settings]
* link:config-contact.html[User Contact Information]
* link:config-replication.html[Git Replication/Mirroring]
* link:config-gitweb.html[Gitweb Integration]
* link:config-headerfooter.html[Site Header/Footer]
* link:config-sso.html[Single Sign-On Systems]

1
Documentation/install.txt
View File

@ -203,7 +203,6 @@ For more information, see the related topic in this manual:
* link:config-reverseproxy.html[Reverse Proxy]
* link:config-sso.html[Single Sign-On Systems]
* link:config-replication.html[Git Replication/Mirroring]
* link:config-headerfooter.html[Site Header/Footer]
* link:config-gitweb.html[Gitweb Integration]
* link:config-gerrit.html[Other System Settings]

3
gerrit-ehcache/.settings/org.eclipse.core.resources.prefs
View File

@ -1,3 +1,4 @@
#Tue Sep 02 16:59:24 PDT 2008
#Tue May 15 09:21:09 PDT 2012
eclipse.preferences.version=1
encoding//src/main/java=UTF-8
encoding/<project>=UTF-8

34
gerrit-extension-api/src/main/java/com/google/gerrit/extensions/events/GitReferenceUpdatedListener.java Normal file
View File

@ -0,0 +1,34 @@
// Copyright (C) 2012 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.extensions.events;
import com.google.gerrit.extensions.annotations.ExtensionPoint;
import java.util.List;
/** Notified when one or more references are modified. */
@ExtensionPoint
public interface GitReferenceUpdatedListener {
public interface Update {
String getRefName();
}
public interface Event {
String getProjectName();
List<Update> getUpdates();
}
void onGitReferenceUpdated(Event event);
}

29
gerrit-extension-api/src/main/java/com/google/gerrit/extensions/events/NewProjectCreatedListener.java Normal file
View File

@ -0,0 +1,29 @@
// Copyright (C) 2012 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.extensions.events;
import com.google.gerrit.extensions.annotations.ExtensionPoint;
/** Notified whenever a project is created on the master. */
@ExtensionPoint
public interface NewProjectCreatedListener {
public interface Event {
String getProjectName();
String getHeadName();
}
void onNewProjectCreated(Event event);
}

6
gerrit-httpd/src/main/java/com/google/gerrit/httpd/rpc/changedetail/DeleteDraftChange.java
View File

@ -19,8 +19,8 @@ import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.reviewdb.client.PatchSet;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.ChangeUtil;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.project.ChangeControl;
import com.google.gerrit.server.project.NoSuchChangeException;
import com.google.gwtjsonrpc.common.VoidResult;
@ -38,7 +38,7 @@ class DeleteDraftChange extends Handler<VoidResult> {
private final ChangeControl.Factory changeControlFactory;
private final ReviewDb db;
private final GitRepositoryManager gitManager;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final PatchSet.Id patchSetId;
@ -46,7 +46,7 @@ class DeleteDraftChange extends Handler<VoidResult> {
DeleteDraftChange(final ReviewDb db,
final ChangeControl.Factory changeControlFactory,
final GitRepositoryManager gitManager,
final ReplicationQueue replication,
final GitReferenceUpdated replication,
@Assisted final PatchSet.Id patchSetId) {
this.changeControlFactory = changeControlFactory;
this.db = db;

6
gerrit-httpd/src/main/java/com/google/gerrit/httpd/rpc/changedetail/RebaseChange.java
View File

@ -24,8 +24,8 @@ import com.google.gerrit.server.ApprovalsUtil;
import com.google.gerrit.server.ChangeUtil;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.mail.EmailException;
import com.google.gerrit.server.mail.RebasedPatchSetSender;
import com.google.gerrit.server.patch.PatchSetInfoFactory;
@ -54,7 +54,7 @@ class RebaseChange extends Handler<ChangeDetail> {
private final RebasedPatchSetSender.Factory rebasedPatchSetSenderFactory;
private final ChangeDetailFactory.Factory changeDetailFactory;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final PatchSet.Id patchSetId;
@ -75,7 +75,7 @@ class RebaseChange extends Handler<ChangeDetail> {
@Assisted final PatchSet.Id patchSetId, final ChangeHookRunner hooks,
final GitRepositoryManager gitManager,
final PatchSetInfoFactory patchSetInfoFactory,
final ReplicationQueue replication,
final GitReferenceUpdated replication,
@GerritPersonIdent final PersonIdent myIdent,
final ApprovalsUtil approvalsUtil) {
this.changeControlFactory = changeControlFactory;

6
gerrit-httpd/src/main/java/com/google/gerrit/httpd/rpc/changedetail/RevertChange.java
View File

@ -24,8 +24,8 @@ import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.ChangeUtil;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.mail.EmailException;
import com.google.gerrit.server.mail.RevertedSender;
import com.google.gerrit.server.patch.PatchSetInfoFactory;
@ -54,7 +54,7 @@ class RevertChange extends Handler<ChangeDetail> {
private final IdentifiedUser currentUser;
private final RevertedSender.Factory revertedSenderFactory;
private final ChangeDetailFactory.Factory changeDetailFactory;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final PatchSet.Id patchSetId;
@Nullable
@ -76,7 +76,7 @@ class RevertChange extends Handler<ChangeDetail> {
@Assisted @Nullable final String message, final ChangeHooks hooks,
final GitRepositoryManager gitManager,
final PatchSetInfoFactory patchSetInfoFactory,
final ReplicationQueue replication,
final GitReferenceUpdated replication,
@GerritPersonIdent final PersonIdent myIdent) {
this.changeControlFactory = changeControlFactory;
this.db = db;

10
gerrit-httpd/src/main/java/com/google/gerrit/httpd/rpc/project/AddBranch.java
View File

@ -22,8 +22,8 @@ import com.google.gerrit.httpd.rpc.Handler;
import com.google.gerrit.reviewdb.client.Branch;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.project.NoSuchProjectException;
import com.google.gerrit.server.project.ProjectControl;
import com.google.gerrit.server.project.RefControl;
@ -59,7 +59,7 @@ class AddBranch extends Handler<ListBranchesResult> {
private final ListBranches.Factory listBranchesFactory;
private final IdentifiedUser identifiedUser;
private final GitRepositoryManager repoManager;
private final ReplicationQueue replication;
private final GitReferenceUpdated referenceUpdated;
private final ChangeHooks hooks;
private final Project.NameKey projectName;
@ -71,7 +71,7 @@ class AddBranch extends Handler<ListBranchesResult> {
final ListBranches.Factory listBranchesFactory,
final IdentifiedUser identifiedUser,
final GitRepositoryManager repoManager,
final ReplicationQueue replication,
GitReferenceUpdated referenceUpdated,
final ChangeHooks hooks,
@Assisted Project.NameKey projectName,
@ -81,7 +81,7 @@ class AddBranch extends Handler<ListBranchesResult> {
this.listBranchesFactory = listBranchesFactory;
this.identifiedUser = identifiedUser;
this.repoManager = repoManager;
this.replication = replication;
this.referenceUpdated = referenceUpdated;
this.hooks = hooks;
this.projectName = projectName;
@ -144,7 +144,7 @@ class AddBranch extends Handler<ListBranchesResult> {
case FAST_FORWARD:
case NEW:
case NO_CHANGE:
replication.scheduleUpdate(name.getParentKey(), refname);
referenceUpdated.fire(name.getParentKey(), refname);
hooks.doRefUpdatedHook(name, u, identifiedUser.getAccount());
break;
default: {

8
gerrit-httpd/src/main/java/com/google/gerrit/httpd/rpc/project/DeleteBranches.java
View File

@ -20,8 +20,8 @@ import com.google.gerrit.reviewdb.client.Branch;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.project.NoSuchProjectException;
import com.google.gerrit.server.project.ProjectControl;
import com.google.gwtorm.server.OrmException;
@ -50,7 +50,7 @@ class DeleteBranches extends Handler<Set<Branch.NameKey>> {
private final ProjectControl.Factory projectControlFactory;
private final GitRepositoryManager repoManager;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final IdentifiedUser identifiedUser;
private final ChangeHooks hooks;
private final ReviewDb db;
@ -61,7 +61,7 @@ class DeleteBranches extends Handler<Set<Branch.NameKey>> {
@Inject
DeleteBranches(final ProjectControl.Factory projectControlFactory,
final GitRepositoryManager repoManager,
final ReplicationQueue replication,
final GitReferenceUpdated replication,
final IdentifiedUser identifiedUser,
final ChangeHooks hooks,
final ReviewDb db,
@ -121,7 +121,7 @@ class DeleteBranches extends Handler<Set<Branch.NameKey>> {
case FAST_FORWARD:
case FORCED:
deleted.add(branchKey);
replication.scheduleUpdate(projectName, refname);
replication.fire(projectName, refname);
hooks.doRefUpdatedHook(branchKey, u, identifiedUser.getAccount());
break;

2
gerrit-pgm/src/main/java/com/google/gerrit/pgm/Daemon.java
View File

@ -42,7 +42,6 @@ import com.google.gerrit.server.config.CanonicalWebUrlProvider;
import com.google.gerrit.server.config.GerritGlobalModule;
import com.google.gerrit.server.config.MasterNodeStartup;
import com.google.gerrit.server.contact.HttpContactStoreConnection;
import com.google.gerrit.server.git.PushReplication;
import com.google.gerrit.server.git.ReceiveCommitsExecutorModule;
import com.google.gerrit.server.git.WorkQueue;
import com.google.gerrit.server.mail.SignedTokenEmailTokenVerifier;
@ -212,7 +211,6 @@ public class Daemon extends SiteProgram {
modules.add(new EhcachePoolImpl.Module());
modules.add(new SmtpEmailSender.Module());
modules.add(new SignedTokenEmailTokenVerifier.Module());
modules.add(new PushReplication.Module());
modules.add(new PluginModule());
if (httpd) {
modules.add(new CanonicalWebUrlModule() {

4
gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/SitePathInitializer.java
View File

@ -81,10 +81,6 @@ public class SitePathInitializer {
savePublic(flags.cfg);
saveSecure(flags.sec);
if (!site.replication_config.exists()) {
site.replication_config.createNewFile();
}
extract(site.gerrit_sh, Init.class, "gerrit.sh");
chmod(0755, site.gerrit_sh);
chmod(0700, site.tmp_dir);

5
gerrit-server/src/main/java/com/google/gerrit/server/AccessPath.java
View File

@ -26,8 +26,5 @@ public enum AccessPath {
SSH_COMMAND,
/** Access from a Git client using any Git protocol. */
GIT,
/** Access through replication */
REPLICATION;
GIT;
}

16
gerrit-server/src/main/java/com/google/gerrit/server/ChangeUtil.java
View File

@ -29,9 +29,9 @@ import com.google.gerrit.reviewdb.client.TrackingId;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.config.TrackingFooter;
import com.google.gerrit.server.config.TrackingFooters;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MergeOp;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.mail.EmailException;
import com.google.gerrit.server.mail.RebasedPatchSetSender;
import com.google.gerrit.server.mail.ReplacePatchSetSender;
@ -240,7 +240,7 @@ public class ChangeUtil {
RebasedPatchSetSender.Factory rebasedPatchSetSenderFactory,
final ChangeHookRunner hooks, GitRepositoryManager gitManager,
final PatchSetInfoFactory patchSetInfoFactory,
final ReplicationQueue replication, PersonIdent myIdent,
final GitReferenceUpdated replication, PersonIdent myIdent,
final ChangeControl.Factory changeControlFactory,
final ApprovalsUtil approvalsUtil) throws NoSuchChangeException,
EmailException, OrmException, MissingObjectException,
@ -381,7 +381,7 @@ public class ChangeUtil {
+ ": " + ru.getResult());
}
replication.scheduleUpdate(change.getProject(), ru.getName());
replication.fire(change.getProject(), ru.getName());
List<PatchSetApproval> patchSetApprovals = approvalsUtil.copyVetosToLatestPatchSet(change);
@ -424,7 +424,7 @@ public class ChangeUtil {
final RevertedSender.Factory revertedSenderFactory,
final ChangeHooks hooks, GitRepositoryManager gitManager,
final PatchSetInfoFactory patchSetInfoFactory,
final ReplicationQueue replication, PersonIdent myIdent)
final GitReferenceUpdated replication, PersonIdent myIdent)
throws NoSuchChangeException, EmailException, OrmException,
MissingObjectException, IncorrectObjectTypeException, IOException,
PatchSetInfoNotAvailableException {
@ -495,7 +495,7 @@ public class ChangeUtil {
throw new IOException("Failed to create ref " + ps.getRefName()
+ " in " + git.getDirectory() + ": " + ru.getResult());
}
replication.scheduleUpdate(db.changes().get(changeId).getProject(),
replication.fire(db.changes().get(changeId).getProject(),
ru.getName());
final ChangeMessage cmsg =
@ -525,7 +525,7 @@ public class ChangeUtil {
public static void deleteDraftChange(final PatchSet.Id patchSetId,
GitRepositoryManager gitManager,
final ReplicationQueue replication, final ReviewDb db)
final GitReferenceUpdated replication, final ReviewDb db)
throws NoSuchChangeException, OrmException, IOException {
final Change.Id changeId = patchSetId.getParentKey();
final Change change = db.changes().get(changeId);
@ -546,7 +546,7 @@ public class ChangeUtil {
public static void deleteOnlyDraftPatchSet(final PatchSet patch,
final Change change, GitRepositoryManager gitManager,
final ReplicationQueue replication, final ReviewDb db)
final GitReferenceUpdated replication, final ReviewDb db)
throws NoSuchChangeException, OrmException, IOException {
final PatchSet.Id patchSetId = patch.getId();
if (patch == null || !patch.isDraft()) {
@ -569,7 +569,7 @@ public class ChangeUtil {
throw new IOException("Failed to delete ref " + patch.getRefName() +
" in " + repo.getDirectory() + ": " + update.getResult());
}
replication.scheduleUpdate(change.getProject(), update.getName());
replication.fire(change.getProject(), update.getName());
} finally {
repo.close();
}

36
gerrit-server/src/main/java/com/google/gerrit/server/ReplicationUser.java → gerrit-server/src/main/java/com/google/gerrit/server/InternalUser.java
View File

@ -1,4 +1,4 @@
// Copyright (C) 2009 The Android Open Source Project
// Copyright (C) 2012 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@ -14,40 +14,37 @@
package com.google.gerrit.server;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.client.AccountProjectWatch;
import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.server.account.CapabilityControl;
import com.google.gerrit.server.account.GroupMembership;
import com.google.gerrit.server.account.ListGroupMembership;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
import java.util.Collection;
import java.util.Collections;
import java.util.Set;
public class ReplicationUser extends CurrentUser {
/** Magic set of groups enabling read of any project and reference. */
public static final GroupMembership EVERYTHING_VISIBLE =
new ListGroupMembership(Collections.<AccountGroup.UUID>emptySet());
/**
* User identity for plugin code that needs an identity.
* <p>
* An InternalUser has no real identity, it acts as the server and can access
* anything it wants, anytime it wants, given the JVM's own direct access to
* data. Plugins may use this when they need to have a CurrentUser with read
* permission on anything.
*/
public class InternalUser extends CurrentUser {
public interface Factory {
ReplicationUser create(@Assisted GroupMembership authGroups);
InternalUser create();
}
private final GroupMembership effectiveGroups;
@Inject
protected ReplicationUser(CapabilityControl.Factory capabilityControlFactory,
@Assisted GroupMembership authGroups) {
super(capabilityControlFactory, AccessPath.REPLICATION);
effectiveGroups = authGroups;
protected InternalUser(CapabilityControl.Factory capabilityControlFactory) {
super(capabilityControlFactory, AccessPath.UNKNOWN);
}
@Override
public GroupMembership getEffectiveGroups() {
return effectiveGroups;
return GroupMembership.EMPTY;
}
@Override
@ -60,7 +57,8 @@ public class ReplicationUser extends CurrentUser {
return Collections.emptySet();
}
public boolean isEverythingVisible() {
return getEffectiveGroups() == EVERYTHING_VISIBLE;
@Override
public String toString() {
return "InternalUser";
}
}

6
gerrit-server/src/main/java/com/google/gerrit/server/changedetail/DeleteDraftPatchSet.java
View File

@ -20,8 +20,8 @@ import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.reviewdb.client.PatchSet;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.ChangeUtil;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.patch.PatchSetInfoFactory;
import com.google.gerrit.server.patch.PatchSetInfoNotAvailableException;
import com.google.gerrit.server.project.ChangeControl;
@ -44,7 +44,7 @@ public class DeleteDraftPatchSet implements Callable<ReviewResult> {
private final ChangeControl.Factory changeControlFactory;
private final ReviewDb db;
private final GitRepositoryManager gitManager;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final PatchSetInfoFactory patchSetInfoFactory;
private final PatchSet.Id patchSetId;
@ -52,7 +52,7 @@ public class DeleteDraftPatchSet implements Callable<ReviewResult> {
@Inject
DeleteDraftPatchSet(ChangeControl.Factory changeControlFactory,
ReviewDb db, GitRepositoryManager gitManager,
ReplicationQueue replication, PatchSetInfoFactory patchSetInfoFactory,
GitReferenceUpdated replication, PatchSetInfoFactory patchSetInfoFactory,
@Assisted final PatchSet.Id patchSetId) {
this.changeControlFactory = changeControlFactory;
this.db = db;

17
gerrit-server/src/main/java/com/google/gerrit/server/config/GerritGlobalModule.java
View File

@ -17,13 +17,16 @@ package com.google.gerrit.server.config;
import static com.google.inject.Scopes.SINGLETON;
import com.google.gerrit.common.data.ApprovalTypes;
import com.google.gerrit.extensions.events.GitReferenceUpdatedListener;
import com.google.gerrit.extensions.events.NewProjectCreatedListener;
import com.google.gerrit.extensions.registration.DynamicSet;
import com.google.gerrit.reviewdb.client.AuthType;
import com.google.gerrit.rules.PrologModule;
import com.google.gerrit.rules.RulesCache;
import com.google.gerrit.server.FileTypeRegistry;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.InternalUser;
import com.google.gerrit.server.MimeUtilFileTypeRegistry;
import com.google.gerrit.server.ReplicationUser;
import com.google.gerrit.server.account.AccountByEmailCacheImpl;
import com.google.gerrit.server.account.AccountCacheImpl;
import com.google.gerrit.server.account.AccountInfoCacheFactory;
@ -39,12 +42,11 @@ import com.google.gerrit.server.account.MaterializedGroupMembership;
import com.google.gerrit.server.account.Realm;
import com.google.gerrit.server.auth.ldap.LdapModule;
import com.google.gerrit.server.events.EventFactory;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.ChangeMergeQueue;
import com.google.gerrit.server.git.GitModule;
import com.google.gerrit.server.git.MergeQueue;
import com.google.gerrit.server.git.PushAllProjectsOp;
import com.google.gerrit.server.git.ReloadSubmitQueueOp;
import com.google.gerrit.server.git.SecureCredentialsProvider;
import com.google.gerrit.server.git.TagCache;
import com.google.gerrit.server.git.TransferConfig;
import com.google.gerrit.server.mail.FromAddressGenerator;
@ -119,6 +121,7 @@ public class GerritGlobalModule extends FactoryModule {
factory(AccountInfoCacheFactory.Factory.class);
factory(CapabilityControl.Factory.class);
factory(GroupInfoCacheFactory.Factory.class);
factory(InternalUser.Factory.class);
factory(ProjectNode.Factory.class);
factory(ProjectState.Factory.class);
factory(MaterializedGroupMembership.Factory.class);
@ -132,9 +135,6 @@ public class GerritGlobalModule extends FactoryModule {
bind(EventFactory.class);
bind(TransferConfig.class);
factory(SecureCredentialsProvider.Factory.class);
factory(PushAllProjectsOp.Factory.class);
bind(ChangeMergeQueue.class).in(SINGLETON);
bind(MergeQueue.class).to(ChangeMergeQueue.class).in(SINGLETON);
factory(ReloadSubmitQueueOp.Factory.class);
@ -150,6 +150,9 @@ public class GerritGlobalModule extends FactoryModule {
bind(ChangeControl.GenericFactory.class);
bind(ProjectControl.GenericFactory.class);
factory(FunctionState.Factory.class);
factory(ReplicationUser.Factory.class);
bind(GitReferenceUpdated.class);
DynamicSet.setOf(binder(), GitReferenceUpdatedListener.class);
DynamicSet.setOf(binder(), NewProjectCreatedListener.class);
}
}

16
gerrit-server/src/main/java/com/google/gerrit/server/config/MasterNodeStartup.java
View File

@ -16,12 +16,9 @@ package com.google.gerrit.server.config;
import com.google.gerrit.extensions.events.LifecycleListener;
import com.google.gerrit.lifecycle.LifecycleModule;
import com.google.gerrit.server.git.PushAllProjectsOp;
import com.google.gerrit.server.git.ReloadSubmitQueueOp;
import com.google.inject.Inject;
import org.eclipse.jgit.lib.Config;
import java.util.concurrent.TimeUnit;
/** Configuration for a master node in a cluster of servers. */
@ -32,26 +29,15 @@ public class MasterNodeStartup extends LifecycleModule {
}
static class OnStart implements LifecycleListener {
private final PushAllProjectsOp.Factory pushAll;
private final ReloadSubmitQueueOp.Factory submit;
private final boolean replicateOnStartup;
@Inject
OnStart(final PushAllProjectsOp.Factory pushAll,
final ReloadSubmitQueueOp.Factory submit,
final @GerritServerConfig Config cfg) {
this.pushAll = pushAll;
OnStart(final ReloadSubmitQueueOp.Factory submit) {
this.submit = submit;
replicateOnStartup = cfg.getBoolean("gerrit", "replicateOnStartup", true);
}
@Override
public void start() {
if (replicateOnStartup) {
pushAll.create(null).start(30, TimeUnit.SECONDS);
}
submit.create().start(15, TimeUnit.SECONDS);
}

2
gerrit-server/src/main/java/com/google/gerrit/server/config/SitePaths.java
View File

@ -41,7 +41,6 @@ public final class SitePaths {
public final File gerrit_config;
public final File secure_config;
public final File replication_config;
public final File contact_information_pub;
public final File ssl_keystore;
@ -78,7 +77,6 @@ public final class SitePaths {
gerrit_config = new File(etc_dir, "gerrit.config");
secure_config = new File(etc_dir, "secure.config");
replication_config = new File(etc_dir, "replication.config");
contact_information_pub = new File(etc_dir, "contact_information.pub");
ssl_keystore = new File(etc_dir, "keystore");

73
gerrit-server/src/main/java/com/google/gerrit/server/extensions/events/GitReferenceUpdated.java Normal file
View File

@ -0,0 +1,73 @@
// Copyright (C) 2012 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.extensions.events;
import com.google.common.collect.ImmutableList;
import com.google.gerrit.extensions.events.GitReferenceUpdatedListener;
import com.google.gerrit.extensions.registration.DynamicSet;
import com.google.gerrit.reviewdb.client.Project;
import com.google.inject.Inject;
import java.util.Collections;
import java.util.List;
public class GitReferenceUpdated {
public static final GitReferenceUpdated DISABLED = new GitReferenceUpdated(
Collections.<GitReferenceUpdatedListener> emptyList());
private final Iterable<GitReferenceUpdatedListener> listeners;
@Inject
GitReferenceUpdated(DynamicSet<GitReferenceUpdatedListener> listeners) {
this.listeners = listeners;
}
GitReferenceUpdated(Iterable<GitReferenceUpdatedListener> listeners) {
this.listeners = listeners;
}
public void fire(Project.NameKey project, String ref) {
Event event = new Event(project, ref);
for (GitReferenceUpdatedListener l : listeners) {
l.onGitReferenceUpdated(event);
}
}
private static class Event implements GitReferenceUpdatedListener.Event {
private final String projectName;
private final String ref;
Event(Project.NameKey project, String ref) {
this.projectName = project.get();
this.ref = ref;
}
@Override
public String getProjectName() {
return projectName;
}
@Override
public List<GitReferenceUpdatedListener.Update> getUpdates() {
GitReferenceUpdatedListener.Update update =
new GitReferenceUpdatedListener.Update() {
public String getRefName() {
return ref;
}
};
return ImmutableList.of(update);
}
}
}

14
gerrit-server/src/main/java/com/google/gerrit/server/git/LocalDiskRepositoryManager.java
View File

@ -23,6 +23,8 @@ import com.google.inject.AbstractModule;
import com.google.inject.Inject;
import com.google.inject.Singleton;
import com.jcraft.jsch.Session;
import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.ConfigConstants;
@ -34,6 +36,9 @@ import org.eclipse.jgit.lib.StoredConfig;
import org.eclipse.jgit.storage.file.LockFile;
import org.eclipse.jgit.storage.file.WindowCache;
import org.eclipse.jgit.storage.file.WindowCacheConfig;
import org.eclipse.jgit.transport.JschConfigSessionFactory;
import org.eclipse.jgit.transport.OpenSshConfig;
import org.eclipse.jgit.transport.SshSessionFactory;
import org.eclipse.jgit.util.FS;
import org.eclipse.jgit.util.IO;
import org.eclipse.jgit.util.RawParseUtils;
@ -82,6 +87,15 @@ public class LocalDiskRepositoryManager implements GitRepositoryManager {
@Override
public void start() {
// Install our own factory which always runs in batch mode, as we
// have no UI available for interactive prompting.
SshSessionFactory.setInstance(new JschConfigSessionFactory() {
@Override
protected void configure(OpenSshConfig.Host hc, Session session) {
// Default configuration is batch mode.
}
});
final WindowCacheConfig c = new WindowCacheConfig();
c.fromConfig(cfg);
WindowCache.reconfigure(c);

10
gerrit-server/src/main/java/com/google/gerrit/server/git/MergeOp.java
View File

@ -37,6 +37,7 @@ import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.account.AccountCache;
import com.google.gerrit.server.config.CanonicalWebUrl;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.mail.MergeFailSender;
import com.google.gerrit.server.mail.MergedSender;
import com.google.gerrit.server.patch.PatchSetInfoFactory;
@ -134,7 +135,7 @@ public class MergeOp {
private final SchemaFactory<ReviewDb> schemaFactory;
private final ProjectCache projectCache;
private final FunctionState.Factory functionState;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final MergedSender.Factory mergedSenderFactory;
private final MergeFailSender.Factory mergeFailSenderFactory;
private final Provider<String> urlProvider;
@ -170,7 +171,7 @@ public class MergeOp {
@Inject
MergeOp(final GitRepositoryManager grm, final SchemaFactory<ReviewDb> sf,
final ProjectCache pc, final FunctionState.Factory fs,
final ReplicationQueue rq, final MergedSender.Factory msf,
final GitReferenceUpdated rq, final MergedSender.Factory msf,
final MergeFailSender.Factory mfsf,
@CanonicalWebUrl @Nullable final Provider<String> cwu,
final ApprovalTypes approvalTypes, final PatchSetInfoFactory psif,
@ -1029,8 +1030,7 @@ public class MergeOp {
ps.getProject().getDescription());
}
replication.scheduleUpdate(destBranch.getParentKey(), branchUpdate
.getName());
replication.fire(destBranch.getParentKey(), branchUpdate.getName());
Account account = null;
final PatchSetApproval submitter = getSubmitter(db, mergeTip.patchsetId);
@ -1125,7 +1125,7 @@ public class MergeOp {
} catch (CodeReviewNoteCreationException e) {
log.error(e.getMessage());
}
replication.scheduleUpdate(destBranch.getParentKey(),
replication.fire(destBranch.getParentKey(),
GitRepositoryManager.REFS_NOTES_REVIEW);
}

9
gerrit-server/src/main/java/com/google/gerrit/server/git/MetaDataUpdate.java
View File

@ -17,6 +17,7 @@ package com.google.gerrit.server.git;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
@ -86,13 +87,13 @@ public class MetaDataUpdate {
@Assisted Repository db);
}
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final Project.NameKey projectName;
private final Repository db;
private final CommitBuilder commit;
@Inject
public MetaDataUpdate(ReplicationQueue replication,
public MetaDataUpdate(GitReferenceUpdated replication,
@Assisted Project.NameKey projectName, @Assisted Repository db) {
this.replication = replication;
this.projectName = projectName;
@ -123,8 +124,6 @@ public class MetaDataUpdate {
}
void replicate(String ref) {
if (replication.isEnabled()) {
replication.scheduleUpdate(projectName, ref);
}
replication.fire(projectName, ref);
}
}

37
gerrit-server/src/main/java/com/google/gerrit/server/git/NoReplication.java
View File

@ -1,37 +0,0 @@
// Copyright (C) 2011 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import com.google.gerrit.reviewdb.client.Project;
/** A disabled {@link ReplicationQueue}. */
public final class NoReplication implements ReplicationQueue {
@Override
public boolean isEnabled() {
return false;
}
@Override
public void scheduleUpdate(Project.NameKey project, String ref) {
}
@Override
public void scheduleFullSync(Project.NameKey project, String urlMatch) {
}
@Override
public void replicateNewProject(Project.NameKey project, String head) {
}
}

8
gerrit-server/src/main/java/com/google/gerrit/server/git/PerThreadRequestScope.java
View File

@ -24,7 +24,7 @@ import com.google.inject.Scope;
import java.util.HashMap;
import java.util.Map;
class PerThreadRequestScope {
public class PerThreadRequestScope {
static class Propagator
extends ThreadLocalRequestScopePropagator<PerThreadRequestScope> {
Propagator() {
@ -49,13 +49,13 @@ class PerThreadRequestScope {
return ctx;
}
static PerThreadRequestScope set(PerThreadRequestScope ctx) {
public static PerThreadRequestScope set(PerThreadRequestScope ctx) {
PerThreadRequestScope old = current.get();
current.set(ctx);
return old;
}
static final Scope REQUEST = new Scope() {
public static final Scope REQUEST = new Scope() {
public <T> Provider<T> scope(final Key<T> key, final Provider<T> creator) {
return new Provider<T>() {
public T get() {
@ -81,7 +81,7 @@ class PerThreadRequestScope {
final RequestCleanup cleanup;
private final Map<Key<?>, Object> map;
PerThreadRequestScope() {
public PerThreadRequestScope() {
cleanup = new RequestCleanup();
map = new HashMap<Key<?>, Object>();
map.put(RC_KEY, cleanup);

75
gerrit-server/src/main/java/com/google/gerrit/server/git/PushAllProjectsOp.java
View File

@ -1,75 +0,0 @@
// Copyright (C) 2009 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.project.ProjectCache;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.concurrent.TimeUnit;
import javax.annotation.Nullable;
public class PushAllProjectsOp extends DefaultQueueOp {
public interface Factory {
PushAllProjectsOp create(String urlMatch);
}
private static final Logger log =
LoggerFactory.getLogger(PushAllProjectsOp.class);
private final ProjectCache projectCache;
private final ReplicationQueue replication;
private final String urlMatch;
@Inject
public PushAllProjectsOp(final WorkQueue wq, final ProjectCache projectCache,
final ReplicationQueue rq, @Assisted @Nullable final String urlMatch) {
super(wq);
this.projectCache = projectCache;
this.replication = rq;
this.urlMatch = urlMatch;
}
@Override
public void start(final int delay, final TimeUnit unit) {
if (replication.isEnabled()) {
super.start(delay, unit);
}
}
public void run() {
try {
for (final Project.NameKey nameKey : projectCache.all()) {
replication.scheduleFullSync(nameKey, urlMatch);
}
} catch (RuntimeException e) {
log.error("Cannot enumerate known projects", e);
}
}
@Override
public String toString() {
String s = "Replicate All Projects";
if (urlMatch != null) {
s = s + " to " + urlMatch;
}
return s;
}
}

433
gerrit-server/src/main/java/com/google/gerrit/server/git/PushOp.java
View File

@ -1,433 +0,0 @@
// Copyright (C) 2009 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.client.Project.NameKey;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.project.NoSuchProjectException;
import com.google.gerrit.server.project.ProjectControl;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.SchemaFactory;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
import com.jcraft.jsch.JSchException;
import org.eclipse.jgit.errors.NoRemoteRepositoryException;
import org.eclipse.jgit.errors.NotSupportedException;
import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.errors.TransportException;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.NullProgressMonitor;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.transport.CredentialsProvider;
import org.eclipse.jgit.transport.FetchConnection;
import org.eclipse.jgit.transport.PushResult;
import org.eclipse.jgit.transport.RefSpec;
import org.eclipse.jgit.transport.RemoteConfig;
import org.eclipse.jgit.transport.RemoteRefUpdate;
import org.eclipse.jgit.transport.Transport;
import org.eclipse.jgit.transport.URIish;
import org.slf4j.Logger;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* A push to remote operation started by {@link ReplicationQueue}.
* <p>
* Instance members are protected by the lock within PushQueue. Callers must
* take that lock to ensure they are working with a current view of the object.
*/
class PushOp implements ProjectRunnable {
interface Factory {
PushOp create(Project.NameKey d, URIish u);
}
private static final Logger log = PushReplication.log;
static final String ALL_REFS = "..all..";
private final GitRepositoryManager repoManager;
private final SchemaFactory<ReviewDb> schema;
private final PushReplication.ReplicationConfig pool;
private final RemoteConfig config;
private final CredentialsProvider credentialsProvider;
private final TagCache tagCache;
private final Set<String> delta = new HashSet<String>();
private final Project.NameKey projectName;
private final URIish uri;
private boolean pushAllRefs;
private Repository db;
/**
* It indicates if the current instance is in fact retrying to push.
*/
private boolean retrying;
private boolean canceled;
@Inject
PushOp(final GitRepositoryManager grm, final SchemaFactory<ReviewDb> s,
final PushReplication.ReplicationConfig p, final RemoteConfig c,
final SecureCredentialsProvider.Factory cpFactory,
final TagCache tc,
@Assisted final Project.NameKey d, @Assisted final URIish u) {
repoManager = grm;
schema = s;
pool = p;
config = c;
credentialsProvider = cpFactory.create(c.getName());
tagCache = tc;
projectName = d;
uri = u;
}
public boolean isRetrying() {
return retrying;
}
public void setToRetry() {
retrying = true;
}
public void cancel() {
canceled = true;
}
public boolean wasCanceled() {
return canceled;
}
URIish getURI() {
return uri;
}
void addRef(final String ref) {
if (ALL_REFS.equals(ref)) {
delta.clear();
pushAllRefs = true;
} else if (!pushAllRefs) {
delta.add(ref);
}
}
public Set<String> getRefs() {
final Set<String> refs;
if (pushAllRefs) {
refs = new HashSet<String>(1);
refs.add(ALL_REFS);
} else {
refs = delta;
}
return refs;
}
public void addRefs(Set<String> refs) {
if (!pushAllRefs) {
for (String ref : refs) {
addRef(ref);
}
}
}
public void run() {
PerThreadRequestScope ctx = new PerThreadRequestScope();
PerThreadRequestScope old = PerThreadRequestScope.set(ctx);
try {
runPushOperation();
} finally {
PerThreadRequestScope.set(old);
}
}
private void runPushOperation() {
// Lock the queue, and remove ourselves, so we can't be modified once
// we start replication (instead a new instance, with the same URI, is
// created and scheduled for a future point in time.)
//
pool.notifyStarting(this);
// It should only verify if it was canceled after calling notifyStarting,
// since the canceled flag would be set locking the queue.
if (!canceled) {
try {
db = repoManager.openRepository(projectName);
runImpl();
} catch (RepositoryNotFoundException e) {
log.error("Cannot replicate " + projectName + "; " + e.getMessage());
} catch (NoRemoteRepositoryException e) {
log.error("Cannot replicate to " + uri + "; repository not found");
} catch (NotSupportedException e) {
log.error("Cannot replicate to " + uri, e);
} catch (TransportException e) {
final Throwable cause = e.getCause();
if (cause instanceof JSchException
&& cause.getMessage().startsWith("UnknownHostKey:")) {
log.error("Cannot replicate to " + uri + ": " + cause.getMessage());
} else {
log.error("Cannot replicate to " + uri, e);
}
// The remote push operation should be retried.
pool.reschedule(this);
} catch (IOException e) {
log.error("Cannot replicate to " + uri, e);
} catch (RuntimeException e) {
log.error("Unexpected error during replication to " + uri, e);
} catch (Error e) {
log.error("Unexpected error during replication to " + uri, e);
} finally {
if (db != null) {
db.close();
}
}
}
}
@Override
public String toString() {
return "push " + uri;
}
private void runImpl() throws IOException {
final Transport tn = Transport.open(db, uri);
final PushResult res;
try {
res = pushVia(tn);
} finally {
try {
tn.close();
} catch (Throwable e2) {
log.warn("Unexpected error while closing " + uri, e2);
}
}
for (final RemoteRefUpdate u : res.getRemoteUpdates()) {
switch (u.getStatus()) {
case OK:
case UP_TO_DATE:
case NON_EXISTING:
break;
case NOT_ATTEMPTED:
case AWAITING_REPORT:
case REJECTED_NODELETE:
case REJECTED_NONFASTFORWARD:
case REJECTED_REMOTE_CHANGED:
log.error("Failed replicate of " + u.getRemoteName() + " to " + uri
+ ": status " + u.getStatus().name());
break;
case REJECTED_OTHER_REASON:
if ("non-fast-forward".equals(u.getMessage())) {
log.error("Failed replicate of " + u.getRemoteName() + " to " + uri
+ ", remote rejected non-fast-forward push."
+ " Check receive.denyNonFastForwards variable in config file"
+ " of destination repository.");
} else {
log.error("Failed replicate of " + u.getRemoteName() + " to " + uri
+ ", reason: " + u.getMessage());
}
break;
}
}
}
private PushResult pushVia(final Transport tn) throws IOException,
NotSupportedException, TransportException {
tn.applyConfig(config);
tn.setCredentialsProvider(credentialsProvider);
final List<RemoteRefUpdate> todo = generateUpdates(tn);
if (todo.isEmpty()) {
// If we have no commands selected, we have nothing to do.
// Calling JGit at this point would just redo the work we
// already did, and come up with the same answer. Instead
// send back an empty result.
//
return new PushResult();
}
return tn.push(NullProgressMonitor.INSTANCE, todo);
}
private List<RemoteRefUpdate> generateUpdates(final Transport tn)
throws IOException {
final ProjectControl pc;
try {
pc = pool.controlFor(projectName);
} catch (NoSuchProjectException e) {
return Collections.emptyList();
}
Map<String, Ref> local = db.getAllRefs();
if (!pc.allRefsAreVisible()) {
if (!pushAllRefs) {
// If we aren't mirroring, reduce the space we need to filter
// to only the references we will update during this operation.
//
Map<String, Ref> n = new HashMap<String, Ref>();
for (String src : delta) {
Ref r = local.get(src);
if (r != null) {
n.put(src, r);
}
}
local = n;
}
final ReviewDb meta;
try {
meta = schema.open();
} catch (OrmException e) {
log.error("Cannot read database to replicate to " + projectName, e);
return Collections.emptyList();
}
try {
local = new VisibleRefFilter(tagCache, db, pc, meta, true).filter(local, true);
} finally {
meta.close();
}
}
final boolean noPerms = !pool.isReplicatePermissions();
final List<RemoteRefUpdate> cmds = new ArrayList<RemoteRefUpdate>();
if (pushAllRefs) {
final Map<String, Ref> remote = listRemote(tn);
for (final Ref src : local.values()) {
if (noPerms && GitRepositoryManager.REF_CONFIG.equals(src.getName())) {
continue;
}
final RefSpec spec = matchSrc(src.getName());
if (spec != null) {
final Ref dst = remote.get(spec.getDestination());
if (dst == null || !src.getObjectId().equals(dst.getObjectId())) {
// Doesn't exist yet, or isn't the same value, request to push.
//
send(cmds, spec, src);
}
}
}
if (config.isMirror()) {
for (final Ref ref : remote.values()) {
if (!Constants.HEAD.equals(ref.getName())) {
final RefSpec spec = matchDst(ref.getName());
if (spec != null && !local.containsKey(spec.getSource())) {
// No longer on local side, request removal.
//
delete(cmds, spec);
}
}
}
}
} else {
for (final String src : delta) {
final RefSpec spec = matchSrc(src);
if (spec != null) {
// If the ref still exists locally, send it, otherwise delete it.
//
Ref srcRef = local.get(src);
if (srcRef != null &&
!(noPerms && GitRepositoryManager.REF_CONFIG.equals(src))) {
send(cmds, spec, srcRef);
} else if (config.isMirror()) {
delete(cmds, spec);
}
}
}
}
return cmds;
}
private Map<String, Ref> listRemote(final Transport tn)
throws NotSupportedException, TransportException {
final FetchConnection fc = tn.openFetch();
try {
return fc.getRefsMap();
} finally {
fc.close();
}
}
private RefSpec matchSrc(final String ref) {
for (final RefSpec s : config.getPushRefSpecs()) {
if (s.matchSource(ref)) {
return s.expandFromSource(ref);
}
}
return null;
}
private RefSpec matchDst(final String ref) {
for (final RefSpec s : config.getPushRefSpecs()) {
if (s.matchDestination(ref)) {
return s.expandFromDestination(ref);
}
}
return null;
}
private void send(final List<RemoteRefUpdate> cmds, final RefSpec spec,
final Ref src) throws IOException {
final String dst = spec.getDestination();
final boolean force = spec.isForceUpdate();
cmds.add(new RemoteRefUpdate(db, src, dst, force, null, null));
}
private void delete(final List<RemoteRefUpdate> cmds, final RefSpec spec)
throws IOException {
final String dst = spec.getDestination();
final boolean force = spec.isForceUpdate();
cmds.add(new RemoteRefUpdate(db, (Ref) null, dst, force, null, null));
}
@Override
public NameKey getProjectNameKey() {
return projectName;
}
@Override
public String getRemoteName() {
return config.getName();
}
@Override
public boolean hasCustomizedPrint() {
return true;
}
}

684
gerrit-server/src/main/java/com/google/gerrit/server/git/PushReplication.java
View File

@ -1,684 +0,0 @@
// Copyright (C) 2009 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import com.google.common.collect.ImmutableSet;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.ReplicationUser;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.account.GroupMembership;
import com.google.gerrit.server.account.ListGroupMembership;
import com.google.gerrit.server.config.FactoryModule;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.project.NoSuchProjectException;
import com.google.gerrit.server.project.PerRequestProjectControlCache;
import com.google.gerrit.server.project.ProjectControl;
import com.google.gwtorm.server.SchemaFactory;
import com.google.inject.AbstractModule;
import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Singleton;
import com.google.inject.servlet.RequestScoped;
import com.jcraft.jsch.Session;
import org.eclipse.jgit.errors.ConfigInvalidException;
import org.eclipse.jgit.errors.RepositoryNotFoundException;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.Ref;
import org.eclipse.jgit.lib.RefUpdate;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.storage.file.FileBasedConfig;
import org.eclipse.jgit.storage.file.FileRepository;
import org.eclipse.jgit.transport.JschConfigSessionFactory;
import org.eclipse.jgit.transport.OpenSshConfig;
import org.eclipse.jgit.transport.RefSpec;
import org.eclipse.jgit.transport.RemoteConfig;
import org.eclipse.jgit.transport.RemoteSession;
import org.eclipse.jgit.transport.SshSessionFactory;
import org.eclipse.jgit.transport.URIish;
import org.eclipse.jgit.util.FS;
import org.eclipse.jgit.util.QuotedString;
import org.eclipse.jgit.util.io.StreamCopyThread;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.net.URISyntaxException;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
/** Manages automatic replication to remote repositories. */
@Singleton
public class PushReplication implements ReplicationQueue {
static final Logger log = LoggerFactory.getLogger(PushReplication.class);
public static class Module extends AbstractModule {
@Override
protected void configure() {
bind(ReplicationQueue.class).to(PushReplication.class);
}
}
static {
// Install our own factory which always runs in batch mode, as we
// have no UI available for interactive prompting.
//
SshSessionFactory.setInstance(new JschConfigSessionFactory() {
@Override
protected void configure(OpenSshConfig.Host hc, Session session) {
// Default configuration is batch mode.
}
});
}
private final Injector injector;
private final WorkQueue workQueue;
private final List<ReplicationConfig> configs;
private final SchemaFactory<ReviewDb> database;
private final ReplicationUser.Factory replicationUserFactory;
private final GitRepositoryManager gitRepositoryManager;
private final GroupCache groupCache;
@Inject
PushReplication(final Injector i, final WorkQueue wq, final SitePaths site,
final ReplicationUser.Factory ruf, final SchemaFactory<ReviewDb> db,
final GitRepositoryManager grm, GroupCache gc)
throws ConfigInvalidException, IOException {
injector = i;
workQueue = wq;
database = db;
replicationUserFactory = ruf;
gitRepositoryManager = grm;
groupCache = gc;
configs = allConfigs(site);
}
@Override
public boolean isEnabled() {
return configs.size() > 0;
}
@Override
public void scheduleFullSync(final Project.NameKey project,
final String urlMatch) {
for (final ReplicationConfig cfg : configs) {
for (final URIish uri : cfg.getURIs(project, urlMatch)) {
cfg.schedule(project, PushOp.ALL_REFS, uri);
}
}
}
@Override
public void scheduleUpdate(final Project.NameKey project, final String ref) {
for (final ReplicationConfig cfg : configs) {
if (cfg.wouldPushRef(ref)) {
for (final URIish uri : cfg.getURIs(project, null)) {
cfg.schedule(project, ref, uri);
}
}
}
}
private static String replace(final String pat, final String key,
final String val) {
final int n = pat.indexOf("${" + key + "}");
if (n != -1) {
return pat.substring(0, n) + val + pat.substring(n + 3 + key.length());
} else {
return null;
}
}
private List<ReplicationConfig> allConfigs(final SitePaths site)
throws ConfigInvalidException, IOException {
final FileBasedConfig cfg =
new FileBasedConfig(site.replication_config, FS.DETECTED);
if (!cfg.getFile().exists()) {
log.warn("No " + cfg.getFile() + "; not replicating");
return Collections.emptyList();
}
if (cfg.getFile().length() == 0) {
log.info("Empty " + cfg.getFile() + "; not replicating");
return Collections.emptyList();
}
try {
cfg.load();
} catch (ConfigInvalidException e) {
throw new ConfigInvalidException("Config file " + cfg.getFile()
+ " is invalid: " + e.getMessage(), e);
} catch (IOException e) {
throw new IOException("Cannot read " + cfg.getFile() + ": "
+ e.getMessage(), e);
}
final List<ReplicationConfig> r = new ArrayList<ReplicationConfig>();
for (final RemoteConfig c : allRemotes(cfg)) {
if (c.getURIs().isEmpty()) {
continue;
}
for (final URIish u : c.getURIs()) {
if (u.getPath() == null || !u.getPath().contains("${name}")) {
throw new ConfigInvalidException("remote." + c.getName() + ".url"
+ " \"" + u + "\" lacks ${name} placeholder in " + cfg.getFile());
}
}
// In case if refspec destination for push is not set then we assume it is
// equal to source
for (RefSpec ref : c.getPushRefSpecs()) {
if (ref.getDestination() == null) {
ref.setDestination(ref.getSource());
}
}
if (c.getPushRefSpecs().isEmpty()) {
RefSpec spec = new RefSpec();
spec = spec.setSourceDestination("refs/*", "refs/*");
spec = spec.setForceUpdate(true);
c.addPushRefSpec(spec);
}
r.add(new ReplicationConfig(injector, workQueue, c, cfg, database,
replicationUserFactory, gitRepositoryManager, groupCache));
}
return Collections.unmodifiableList(r);
}
private List<RemoteConfig> allRemotes(final FileBasedConfig cfg)
throws ConfigInvalidException {
List<String> names = new ArrayList<String>(cfg.getSubsections("remote"));
Collections.sort(names);
final List<RemoteConfig> result = new ArrayList<RemoteConfig>(names.size());
for (final String name : names) {
try {
result.add(new RemoteConfig(cfg, name));
} catch (URISyntaxException e) {
throw new ConfigInvalidException("remote " + name
+ " has invalid URL in " + cfg.getFile());
}
}
return result;
}
@Override
public void replicateNewProject(Project.NameKey projectName, String head) {
if (!isEnabled()) {
return;
}
for (ReplicationConfig config : configs) {
List<URIish> uriList = config.getURIs(projectName, "*");
String[] adminUrls = config.getAdminUrls();
boolean adminURLUsed = false;
for (String url : adminUrls) {
URIish adminURI = null;
try {
if (url != null && !url.isEmpty()) {
adminURI = new URIish(url);
}
} catch (URISyntaxException e) {
log.error("The URL '" + url + "' is invalid");
}
if (adminURI != null) {
final String replacedPath =
replace(adminURI.getPath(), "name", projectName.get());
if (replacedPath != null) {
adminURI = adminURI.setPath(replacedPath);
if (usingSSH(adminURI)) {
replicateProject(adminURI, head);
adminURLUsed = true;
} else {
log.error("The adminURL '" + url
+ "' is non-SSH which is not allowed");
}
}
}
}
if (!adminURLUsed) {
for (URIish uri : uriList) {
replicateProject(uri, head);
}
}
}
}
private void replicateProject(final URIish replicateURI, final String head) {
if (!replicateURI.isRemote()) {
replicateProjectLocally(replicateURI, head);
} else if (usingSSH(replicateURI)) {
replicateProjectOverSsh(replicateURI, head);
} else {
log.warn("Cannot create new project on remote site since neither the "
+ "connection method is SSH nor the replication target is local: "
+ replicateURI.toString());
return;
}
}
private void replicateProjectLocally(final URIish replicateURI,
final String head) {
try {
final Repository repo = new FileRepository(replicateURI.getPath());
try {
repo.create(true /* bare */);
final RefUpdate u = repo.updateRef(Constants.HEAD);
u.disableRefLog();
u.link(head);
} finally {
repo.close();
}
} catch (IOException e) {
log.error("Failed to replicate project locally: "
+ replicateURI.getPath());
}
}
private void replicateProjectOverSsh(final URIish replicateURI,
final String head) {
SshSessionFactory sshFactory = SshSessionFactory.getInstance();
RemoteSession sshSession;
String projectPath = QuotedString.BOURNE.quote(replicateURI.getPath());
OutputStream errStream = createErrStream();
String cmd =
"mkdir -p " + projectPath + "&& cd " + projectPath
+ "&& git init --bare" + "&& git symbolic-ref HEAD "
+ QuotedString.BOURNE.quote(head);
try {
sshSession = sshFactory.getSession(replicateURI, null, FS.DETECTED, 0);
Process proc = sshSession.exec(cmd, 0);
proc.getOutputStream().close();
StreamCopyThread out = new StreamCopyThread(proc.getInputStream(), errStream);
StreamCopyThread err = new StreamCopyThread(proc.getErrorStream(), errStream);
out.start();
err.start();
try {
proc.waitFor();
out.halt();
err.halt();
} catch (InterruptedException interrupted) {
// Don't wait, drop out immediately.
}
sshSession.disconnect();
} catch (IOException e) {
log.error("Communication error when trying to replicate to: "
+ replicateURI.toString() + "\n" + "Error reported: "
+ e.getMessage() + "\n" + "Error in communication: "
+ errStream.toString());
}
}
private OutputStream createErrStream() {
return new OutputStream() {
private StringBuilder all = new StringBuilder();
private StringBuilder sb = new StringBuilder();
@Override
public String toString() {
String r = all.toString();
while (r.endsWith("\n"))
r = r.substring(0, r.length() - 1);
return r;
}
@Override
public synchronized void write(final int b) {
if (b == '\r') {
return;
}
sb.append((char) b);
if (b == '\n') {
all.append(sb);
sb.setLength(0);
}
}
};
}
private boolean usingSSH(final URIish uri) {
final String scheme = uri.getScheme();
if (!uri.isRemote()) return false;
if (scheme != null && scheme.toLowerCase().contains("ssh")) return true;
if (scheme == null && uri.getHost() != null && uri.getPath() != null)
return true;
return false;
}
static class ReplicationConfig {
private final RemoteConfig remote;
private final String[] adminUrls;
private final int delay;
private final int retryDelay;
private final WorkQueue.Executor pool;
private final Map<URIish, PushOp> pending = new HashMap<URIish, PushOp>();
private final PushOp.Factory opFactory;
private final ProjectControl.Factory projectControlFactory;
private final GitRepositoryManager mgr;
private final boolean replicatePermissions;
ReplicationConfig(final Injector injector, final WorkQueue workQueue,
final RemoteConfig rc, final Config cfg, SchemaFactory<ReviewDb> db,
final ReplicationUser.Factory replicationUserFactory,
final GitRepositoryManager gitRepositoryManager,
GroupCache groupCache) {
remote = rc;
delay = Math.max(0, getInt(rc, cfg, "replicationdelay", 15));
retryDelay = Math.max(0, getInt(rc, cfg, "replicationretry", 1));
final int poolSize = Math.max(0, getInt(rc, cfg, "threads", 1));
final String poolName = "ReplicateTo-" + rc.getName();
pool = workQueue.createQueue(poolSize, poolName);
String[] authGroupNames =
cfg.getStringList("remote", rc.getName(), "authGroup");
final GroupMembership authGroups;
if (authGroupNames.length > 0) {
ImmutableSet.Builder<AccountGroup.UUID> builder = ImmutableSet.builder();
for (String name : authGroupNames) {
AccountGroup g = groupCache.get(new AccountGroup.NameKey(name));
if (g != null) {
builder.add(g.getGroupUUID());
} else {
log.warn("Group \"{0}\" not in database, removing from authGroup", name);
}
}
authGroups = new ListGroupMembership(builder.build());
} else {
authGroups = ReplicationUser.EVERYTHING_VISIBLE;
}
adminUrls = cfg.getStringList("remote", rc.getName(), "adminUrl");
replicatePermissions = cfg.getBoolean("remote", rc.getName(),
"replicatePermissions", true);
mgr = gitRepositoryManager;
final ReplicationUser remoteUser =
replicationUserFactory.create(authGroups);
projectControlFactory =
injector.createChildInjector(new AbstractModule() {
@Override
protected void configure() {
bindScope(RequestScoped.class, PerThreadRequestScope.REQUEST);
bind(PerRequestProjectControlCache.class).in(RequestScoped.class);
bind(CurrentUser.class).toInstance(remoteUser);
}
}).getInstance(ProjectControl.Factory.class);
opFactory = injector.createChildInjector(new FactoryModule() {
@Override
protected void configure() {
bind(PushReplication.ReplicationConfig.class).toInstance(ReplicationConfig.this);
bind(RemoteConfig.class).toInstance(remote);
factory(PushOp.Factory.class);
}
}).getInstance(PushOp.Factory.class);
}
private int getInt(final RemoteConfig rc, final Config cfg,
final String name, final int defValue) {
return cfg.getInt("remote", rc.getName(), name, defValue);
}
void schedule(final Project.NameKey project, final String ref,
final URIish uri) {
PerThreadRequestScope ctx = new PerThreadRequestScope();
PerThreadRequestScope old = PerThreadRequestScope.set(ctx);
try {
try {
if (!controlFor(project).isVisible()) {
return;
}
} catch (NoSuchProjectException e1) {
log.error("Internal error: project " + project
+ " not found during replication");
return;
}
} finally {
PerThreadRequestScope.set(old);
}
if (!replicatePermissions) {
PushOp e;
synchronized (pending) {
e = pending.get(uri);
}
if (e == null) {
Repository git;
try {
git = mgr.openRepository(project);
} catch (RepositoryNotFoundException err) {
log.error("Internal error: project " + project
+ " not found during replication", err);
return;
} catch (IOException err) {
log.error("Internal error: unable to open project " + project
+ " during replication", err);
return;
}
try {
Ref head = git.getRef(Constants.HEAD);
if (head != null
&& head.isSymbolic()
&& GitRepositoryManager.REF_CONFIG.equals(head.getLeaf().getName())) {
return;
}
} catch (IOException err) {
log.error("Internal error: cannot check type of project " + project
+ " during replication", err);
return;
} finally {
git.close();
}
}
}
synchronized (pending) {
PushOp e = pending.get(uri);
if (e == null) {
e = opFactory.create(project, uri);
pool.schedule(e, delay, TimeUnit.SECONDS);
pending.put(uri, e);
}
e.addRef(ref);
}
}
/**
* It schedules again a PushOp instance.
* <p>
* It is assumed to be previously scheduled and found a
* transport exception. It will schedule it as a push
* operation to be retried after the minutes count
* determined by class attribute retryDelay.
* <p>
* In case the PushOp instance to be scheduled has same
* URI than one also pending for retry, it adds to the one
* pending the refs list of the parameter instance.
* <p>
* In case the PushOp instance to be scheduled has same
* URI than one pending, but not pending for retry, it
* indicates the one pending should be canceled when it
* starts executing, removes it from pending list, and
* adds its refs to the parameter instance. The parameter
* instance is scheduled for retry.
* <p>
* Notice all operations to indicate a PushOp should be
* canceled, or it is retrying, or remove/add it from/to
* pending Map should be protected by the lock on pending
* Map class instance attribute.
*
* @param pushOp The PushOp instance to be scheduled.
*/
void reschedule(final PushOp pushOp) {
// It locks access to pending variable.
synchronized (pending) {
URIish uri = pushOp.getURI();
PushOp pendingPushOp = pending.get(uri);
if (pendingPushOp != null) {
// There is one PushOp instance already pending to same URI.
if (pendingPushOp.isRetrying()) {
// The one pending is one already retrying, so it should
// maintain it and add to it the refs of the one passed
// as parameter to the method.
// This scenario would happen if a PushOp has started running
// and then before it failed due transport exception, another
// one to same URI started. The first one would fail and would
// be rescheduled, being present in pending list. When the
// second one fails, it will also be rescheduled and then,
// here, find out replication to its URI is already pending
// for retry (blocking).
pendingPushOp.addRefs(pushOp.getRefs());
} else {
// The one pending is one that is NOT retrying, it was just
// scheduled believing no problem would happen. The one pending
// should be canceled, and this is done by setting its canceled
// flag, removing it from pending list, and adding its refs to
// the pushOp instance that should then, later, in this method,
// be scheduled for retry.
// Notice that the PushOp found pending will start running and,
// when notifying it is starting (with pending lock protection),
// it will see it was canceled and then it will do nothing with
// pending list and it will not execute its run implementation.
pendingPushOp.cancel();
pending.remove(uri);
pushOp.addRefs(pendingPushOp.getRefs());
}
}
if (pendingPushOp == null || !pendingPushOp.isRetrying()) {
// The PushOp method param instance should be scheduled for retry.
// Remember when retrying it should be used different delay.
pushOp.setToRetry();
pending.put(uri, pushOp);
pool.schedule(pushOp, retryDelay, TimeUnit.MINUTES);
}
}
}
ProjectControl controlFor(final Project.NameKey project)
throws NoSuchProjectException {
return projectControlFactory.controlFor(project);
}
void notifyStarting(final PushOp op) {
synchronized (pending) {
if (!op.wasCanceled()) {
pending.remove(op.getURI());
}
}
}
boolean wouldPushRef(final String ref) {
if (!replicatePermissions && GitRepositoryManager.REF_CONFIG.equals(ref)) {
return false;
}
for (final RefSpec s : remote.getPushRefSpecs()) {
if (s.matchSource(ref)) {
return true;
}
}
return false;
}
boolean isReplicatePermissions() {
return replicatePermissions;
}
List<URIish> getURIs(final Project.NameKey project, final String urlMatch) {
final List<URIish> r = new ArrayList<URIish>(remote.getURIs().size());
for (URIish uri : remote.getURIs()) {
if (matches(uri, urlMatch)) {
String name = project.get();
if (needsUrlEncoding(uri)) {
name = encode(name);
}
String replacedPath = replace(uri.getPath(), "name", name);
if (replacedPath != null) {
uri = uri.setPath(replacedPath);
r.add(uri);
}
}
}
return r;
}
static boolean needsUrlEncoding(URIish uri) {
return "http".equalsIgnoreCase(uri.getScheme())
|| "https".equalsIgnoreCase(uri.getScheme())
|| "amazon-s3".equalsIgnoreCase(uri.getScheme());
}
static String encode(String str) {
try {
// Some cleanup is required. The '/' character is always encoded as %2F
// however remote servers will expect it to be not encoded as part of the
// path used to the repository. Space is incorrectly encoded as '+' for this
// context. In the path part of a URI space should be %20, but in form data
// space is '+'. Our cleanup replace fixes these two issues.
return URLEncoder.encode(str, "UTF-8")
.replaceAll("%2[fF]", "/")
.replace("+", "%20");
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
String[] getAdminUrls() {
return this.adminUrls;
}
private boolean matches(URIish uri, final String urlMatch) {
if (urlMatch == null || urlMatch.equals("") || urlMatch.equals("*")) {
return true;
}
return uri.toString().contains(urlMatch);
}
}
}

11
gerrit-server/src/main/java/com/google/gerrit/server/git/ReceiveCommits.java
View File

@ -41,6 +41,7 @@ import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.account.AccountResolver;
import com.google.gerrit.server.config.CanonicalWebUrl;
import com.google.gerrit.server.config.TrackingFooters;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.MultiProgressMonitor.Task;
import com.google.gerrit.server.mail.CreateChangeSender;
import com.google.gerrit.server.mail.MergedSender;
@ -205,7 +206,7 @@ public class ReceiveCommits {
private final CreateChangeSender.Factory createChangeSenderFactory;
private final MergedSender.Factory mergedSenderFactory;
private final ReplacePatchSetSender.Factory replacePatchSetFactory;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final PatchSetInfoFactory patchSetInfoFactory;
private final ChangeHooks hooks;
private final ApprovalsUtil approvalsUtil;
@ -254,7 +255,7 @@ public class ReceiveCommits {
final CreateChangeSender.Factory createChangeSenderFactory,
final MergedSender.Factory mergedSenderFactory,
final ReplacePatchSetSender.Factory replacePatchSetFactory,
final ReplicationQueue replication,
final GitReferenceUpdated replication,
final PatchSetInfoFactory patchSetInfoFactory,
final ChangeHooks hooks,
final ApprovalsUtil approvalsUtil,
@ -509,7 +510,7 @@ public class ReceiveCommits {
// We only schedule direct refs updates for replication.
// Change refs are scheduled when they are created.
//
replication.scheduleUpdate(project.getNameKey(), c.getRefName());
replication.fire(project.getNameKey(), c.getRefName());
Branch.NameKey destBranch = new Branch.NameKey(project.getNameKey(), c.getRefName());
hooks.doRefUpdatedHook(destBranch, c.getOldId(), c.getNewId(), currentUser.getAccount());
commandProgress.update(1);
@ -1155,7 +1156,7 @@ public class ReceiveCommits {
throw new IOException("Failed to create ref " + ps.getRefName() + " in "
+ repo.getDirectory() + ": " + ru.getResult());
}
replication.scheduleUpdate(project.getNameKey(), ru.getName());
replication.fire(project.getNameKey(), ru.getName());
allNewChanges.add(change);
@ -1457,7 +1458,7 @@ public class ReceiveCommits {
throw new IOException("Failed to create ref " + ps.getRefName() + " in "
+ repo.getDirectory() + ": " + ru.getResult());
}
replication.scheduleUpdate(project.getNameKey(), ru.getName());
replication.fire(project.getNameKey(), ru.getName());
hooks.doPatchsetCreatedHook(result.change, ps, db);
request.cmd.setResult(ReceiveCommand.Result.OK);

59
gerrit-server/src/main/java/com/google/gerrit/server/git/ReplicationQueue.java
View File

@ -1,59 +0,0 @@
// Copyright (C) 2009 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import com.google.gerrit.reviewdb.client.Project;
/** Manages replication to other nodes. */
public interface ReplicationQueue {
/** Is replication to one or more other destinations configured? */
boolean isEnabled();
/**
* Schedule a full replication for a single project.
* <p>
* All remote URLs are checked to verify the are current with regards to the
* local project state. If not, they are updated by pushing new refs, updating
* existing ones which don't match, and deleting stale refs which have been
* removed from the local repository.
*
* @param project identity of the project to replicate.
* @param urlMatch substring that must appear in a URI to support replication.
*/
void scheduleFullSync(Project.NameKey project, String urlMatch);
/**
* Schedule update of a single ref.
* <p>
* This method automatically tries to batch together multiple requests in the
* same project, to take advantage of Git's native ability to update multiple
* refs during a single push operation.
*
* @param project identity of the project to replicate.
* @param ref unique name of the ref; must start with {@code refs/}.
*/
void scheduleUpdate(Project.NameKey project, String ref);
/**
* Create new empty project at the remote sites.
* <p>
* When a new project has been created locally call this method to make sure
* that the project will be created at the remote sites as well.
*
* @param project of the project to be created.
* @param head name HEAD should point at (must be {@code refs/heads/...}).
*/
void replicateNewProject(Project.NameKey project, String head);
}

92
gerrit-server/src/main/java/com/google/gerrit/server/git/SecureCredentialsProvider.java
View File

@ -1,92 +0,0 @@
// Copyright (C) 2011 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import com.google.gerrit.server.config.GerritServerConfig;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
import org.eclipse.jgit.errors.UnsupportedCredentialItem;
import org.eclipse.jgit.lib.Config;
import org.eclipse.jgit.transport.CredentialItem;
import org.eclipse.jgit.transport.CredentialsProvider;
import org.eclipse.jgit.transport.URIish;
/** Looks up a remote's password in secure.config. */
public class SecureCredentialsProvider extends CredentialsProvider {
public interface Factory {
SecureCredentialsProvider create(String remoteName);
}
private final String cfgUser;
private final String cfgPass;
@Inject
SecureCredentialsProvider(@GerritServerConfig Config cfg,
@Assisted String remoteName) {
cfgUser = cfg.getString("remote", remoteName, "username");
cfgPass = cfg.getString("remote", remoteName, "password");
}
@Override
public boolean isInteractive() {
return false;
}
@Override
public boolean supports(CredentialItem... items) {
for (CredentialItem i : items) {
if (i instanceof CredentialItem.Username) {
continue;
} else if (i instanceof CredentialItem.Password) {
continue;
} else {
return false;
}
}
return true;
}
@Override
public boolean get(URIish uri, CredentialItem... items)
throws UnsupportedCredentialItem {
String username = uri.getUser();
if (username == null) {
username = cfgUser;
}
if (username == null) {
return false;
}
String password = uri.getPass();
if (password == null) {
password = cfgPass;
}
if (password == null) {
return false;
}
for (CredentialItem i : items) {
if (i instanceof CredentialItem.Username) {
((CredentialItem.Username) i).setValue(username);
} else if (i instanceof CredentialItem.Password) {
((CredentialItem.Password) i).setValue(password.toCharArray());
} else {
throw new UnsupportedCredentialItem(uri, i.getPromptText());
}
}
return true;
}
}

7
gerrit-server/src/main/java/com/google/gerrit/server/git/SubmoduleOp.java
View File

@ -21,6 +21,7 @@ import com.google.gerrit.reviewdb.client.SubmoduleSubscription;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.config.CanonicalWebUrl;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.util.SubmoduleSectionParser;
import com.google.gwtorm.server.OrmException;
import com.google.gwtorm.server.SchemaFactory;
@ -84,7 +85,7 @@ public class SubmoduleOp {
private final Map<Change.Id, CodeReviewCommit> commits;
private final PersonIdent myIdent;
private final GitRepositoryManager repoManager;
private final ReplicationQueue replication;
private final GitReferenceUpdated replication;
private final SchemaFactory<ReviewDb> schemaFactory;
private final Set<Branch.NameKey> updatedSubscribers;
@ -96,7 +97,7 @@ public class SubmoduleOp {
@Assisted Project destProject, @Assisted List<Change> submitted,
@Assisted final Map<Change.Id, CodeReviewCommit> commits,
@GerritPersonIdent final PersonIdent myIdent,
GitRepositoryManager repoManager, ReplicationQueue replication) {
GitRepositoryManager repoManager, GitReferenceUpdated replication) {
this.destBranch = destBranch;
this.mergeTip = mergeTip;
this.rw = rw;
@ -331,7 +332,7 @@ public class SubmoduleOp {
switch (rfu.update()) {
case NEW:
case FAST_FORWARD:
replication.scheduleUpdate(subscriber.getParentKey(), rfu.getName());
replication.fire(subscriber.getParentKey(), rfu.getName());
// TODO since this is performed "in the background" no mail will be
// sent to inform users about the updated branch
break;

4
gerrit-server/src/main/java/com/google/gerrit/server/git/WorkQueue.java
View File

@ -172,6 +172,10 @@ public class WorkQueue {
);
}
public void unregisterWorkQueue() {
queues.remove(this);
}
@Override
protected <V> RunnableScheduledFuture<V> decorateTask(
final Runnable runnable, RunnableScheduledFuture<V> r) {

33
gerrit-server/src/main/java/com/google/gerrit/server/project/CreateProject.java
View File

@ -19,6 +19,8 @@ import com.google.gerrit.common.data.GroupReference;
import com.google.gerrit.common.data.Permission;
import com.google.gerrit.common.data.PermissionRule;
import com.google.gerrit.common.errors.ProjectCreationFailedException;
import com.google.gerrit.extensions.events.NewProjectCreatedListener;
import com.google.gerrit.extensions.registration.DynamicSet;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.server.ReviewDb;
@ -26,10 +28,10 @@ import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.account.GroupCache;
import com.google.gerrit.server.config.ProjectOwnerGroups;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MetaDataUpdate;
import com.google.gerrit.server.git.ProjectConfig;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.git.RepositoryCaseMismatchException;
import com.google.inject.Inject;
import com.google.inject.assistedinject.Assisted;
@ -64,7 +66,8 @@ public class CreateProject {
private final Set<AccountGroup.UUID> projectOwnerGroups;
private final IdentifiedUser currentUser;
private final GitRepositoryManager repoManager;
private final ReplicationQueue replication;
private final GitReferenceUpdated referenceUpdated;
private final DynamicSet<NewProjectCreatedListener> createdListener;
private final PersonIdent serverIdent;
private CreateProjectArgs createProjectArgs;
private ProjectCache projectCache;
@ -74,14 +77,17 @@ public class CreateProject {
@Inject
CreateProject(@ProjectOwnerGroups Set<AccountGroup.UUID> pOwnerGroups,
IdentifiedUser identifiedUser, GitRepositoryManager gitRepoManager,
ReplicationQueue replicateq, ReviewDb db,
GitReferenceUpdated referenceUpdated,
DynamicSet<NewProjectCreatedListener> createdListener,
ReviewDb db,
@GerritPersonIdent PersonIdent personIdent, final GroupCache groupCache,
final MetaDataUpdate.User metaDataUpdateFactory,
@Assisted CreateProjectArgs createPArgs, ProjectCache pCache) {
this.projectOwnerGroups = pOwnerGroups;
this.currentUser = identifiedUser;
this.repoManager = gitRepoManager;
this.replication = replicateq;
this.referenceUpdated = referenceUpdated;
this.createdListener = createdListener;
this.serverIdent = personIdent;
this.createProjectArgs = createPArgs;
this.projectCache = pCache;
@ -98,7 +104,20 @@ public class CreateProject {
: createProjectArgs.branch;
final Repository repo = repoManager.createRepository(nameKey);
try {
replication.replicateNewProject(nameKey, head);
NewProjectCreatedListener.Event event = new NewProjectCreatedListener.Event() {
@Override
public String getProjectName() {
return nameKey.get();
}
@Override
public String getHeadName() {
return head;
}
};
for (NewProjectCreatedListener l : createdListener) {
l.onNewProjectCreated(event);
}
final RefUpdate u = repo.updateRef(Constants.HEAD);
u.disableRefLog();
@ -186,7 +205,7 @@ public class CreateProject {
projectCache.onCreateProject(createProjectArgs.getProject());
repoManager.setProjectDescription(createProjectArgs.getProject(),
createProjectArgs.projectDescription);
replication.scheduleUpdate(createProjectArgs.getProject(),
referenceUpdated.fire(createProjectArgs.getProject(),
GitRepositoryManager.REF_CONFIG);
}
@ -246,7 +265,7 @@ public class CreateProject {
final Result result = ru.update();
switch (result) {
case NEW:
replication.scheduleUpdate(project, ref);
referenceUpdated.fire(project, ref);
break;
default: {
throw new IOException(result.name());

12
gerrit-server/src/main/java/com/google/gerrit/server/project/ProjectControl.java
View File

@ -29,7 +29,7 @@ import com.google.gerrit.reviewdb.client.Change;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.ReplicationUser;
import com.google.gerrit.server.InternalUser;
import com.google.gerrit.server.config.CanonicalWebUrl;
import com.google.gerrit.server.config.GitReceivePackGroups;
import com.google.gerrit.server.config.GitUploadPackGroups;
@ -185,7 +185,7 @@ public class ProjectControl {
/** Can this user see this project exists? */
public boolean isVisible() {
return (visibleForReplication()
return (user instanceof InternalUser
|| canPerformOnAnyRef(Permission.READ)) && !isHidden();
}
@ -196,16 +196,10 @@ public class ProjectControl {
/** Can this user see all the refs in this projects? */
public boolean allRefsAreVisible() {
return visibleForReplication()
return user instanceof InternalUser
|| canPerformOnAllRefs(Permission.READ);
}
/** Is this project completely visible for replication? */
boolean visibleForReplication() {
return user instanceof ReplicationUser
&& ((ReplicationUser) user).isEverythingVisible();
}
/** Is this user a project owner? Ownership does not imply {@link #isVisible()} */
public boolean isOwner() {
return isDeclaredOwner()

3
gerrit-server/src/main/java/com/google/gerrit/server/project/RefControl.java
View File

@ -23,6 +23,7 @@ import com.google.gerrit.common.errors.InvalidNameException;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.InternalUser;
import com.google.gerrit.server.git.GitRepositoryManager;
import dk.brics.automaton.RegExp;
@ -101,7 +102,7 @@ public class RefControl {
/** Can this user see this reference exists? */
public boolean isVisible() {
return (projectControl.visibleForReplication() || canPerform(Permission.READ))
return (getCurrentUser() instanceof InternalUser || canPerform(Permission.READ))
&& canRead();
}

5
gerrit-server/src/main/java/com/google/gerrit/server/schema/SchemaCreator.java
View File

@ -32,9 +32,9 @@ import com.google.gerrit.server.account.GroupUUID;
import com.google.gerrit.server.config.AllProjectsName;
import com.google.gerrit.server.config.SitePath;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MetaDataUpdate;
import com.google.gerrit.server.git.NoReplication;
import com.google.gerrit.server.git.ProjectConfig;
import com.google.gwtorm.jdbc.JdbcExecutor;
import com.google.gwtorm.jdbc.JdbcSchema;
@ -219,7 +219,8 @@ public class SchemaCreator {
}
}
try {
MetaDataUpdate md = new MetaDataUpdate(new NoReplication(), allProjectsName, git);
MetaDataUpdate md =
new MetaDataUpdate(GitReferenceUpdated.DISABLED, allProjectsName, git);
md.getCommitBuilder().setAuthor(serverUser);
md.getCommitBuilder().setCommitter(serverUser);

4
gerrit-server/src/main/java/com/google/gerrit/server/schema/Schema_53.java
View File

@ -37,9 +37,9 @@ import com.google.gerrit.reviewdb.client.SystemConfig;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.account.GroupUUID;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MetaDataUpdate;
import com.google.gerrit.server.git.NoReplication;
import com.google.gerrit.server.git.ProjectConfig;
import com.google.gwtorm.jdbc.JdbcSchema;
import com.google.gwtorm.server.OrmException;
@ -166,7 +166,7 @@ class Schema_53 extends SchemaVersion {
}
try {
MetaDataUpdate md =
new MetaDataUpdate(new NoReplication(), nameKey, git);
new MetaDataUpdate(GitReferenceUpdated.DISABLED, nameKey, git);
md.getCommitBuilder().setAuthor(serverUser);
md.getCommitBuilder().setCommitter(serverUser);

5
gerrit-server/src/main/java/com/google/gerrit/server/schema/Schema_57.java
View File

@ -26,9 +26,9 @@ import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.config.AllProjectsNameProvider;
import com.google.gerrit.server.config.SitePaths;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.LocalDiskRepositoryManager;
import com.google.gerrit.server.git.MetaDataUpdate;
import com.google.gerrit.server.git.NoReplication;
import com.google.gerrit.server.git.ProjectConfig;
import com.google.gwtorm.server.OrmException;
import com.google.inject.Inject;
@ -81,7 +81,8 @@ public class Schema_57 extends SchemaVersion {
try {
Repository git = mgr.openRepository(allProjects);
try {
MetaDataUpdate md = new MetaDataUpdate(new NoReplication(), allProjects, git);
MetaDataUpdate md =
new MetaDataUpdate(GitReferenceUpdated.DISABLED, allProjects, git);
md.getCommitBuilder().setAuthor(serverUser);
md.getCommitBuilder().setCommitter(serverUser);

4
gerrit-server/src/main/java/com/google/gerrit/server/schema/Schema_64.java
View File

@ -23,9 +23,9 @@ import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.config.AllProjectsName;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MetaDataUpdate;
import com.google.gerrit.server.git.NoReplication;
import com.google.gerrit.server.git.ProjectConfig;
import com.google.gwtorm.jdbc.JdbcSchema;
import com.google.gwtorm.server.OrmException;
@ -92,7 +92,7 @@ public class Schema_64 extends SchemaVersion {
}
try {
MetaDataUpdate md =
new MetaDataUpdate(new NoReplication(), allProjects, git);
new MetaDataUpdate(GitReferenceUpdated.DISABLED, allProjects, git);
md.getCommitBuilder().setAuthor(serverUser);
md.getCommitBuilder().setCommitter(serverUser);

4
gerrit-server/src/main/java/com/google/gerrit/server/schema/Schema_65.java
View File

@ -34,9 +34,9 @@ import com.google.gerrit.server.GerritPersonIdent;
import com.google.gerrit.server.account.GroupUUID;
import com.google.gerrit.server.config.AllProjectsName;
import com.google.gerrit.server.config.AnonymousCowardName;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gerrit.server.git.GitRepositoryManager;
import com.google.gerrit.server.git.MetaDataUpdate;
import com.google.gerrit.server.git.NoReplication;
import com.google.gerrit.server.git.ProjectConfig;
import com.google.gerrit.server.git.VersionedMetaData.BatchMetaDataUpdate;
import com.google.gwtorm.jdbc.JdbcSchema;
@ -92,7 +92,7 @@ public class Schema_65 extends SchemaVersion {
}
try {
MetaDataUpdate md =
new MetaDataUpdate(new NoReplication(), allProjects, git);
new MetaDataUpdate(GitReferenceUpdated.DISABLED, allProjects, git);
ProjectConfig config = ProjectConfig.read(md);
Map<Integer, ContributorAgreement> agreements = getAgreementToAdd(db, config);
if (agreements.isEmpty()) {

3
gerrit-server/src/main/java/gerrit/PRED_current_user_1.java
View File

@ -20,7 +20,6 @@ import com.google.gerrit.server.AnonymousUser;
import com.google.gerrit.server.CurrentUser;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.PeerDaemonUser;
import com.google.gerrit.server.ReplicationUser;
import com.google.gerrit.server.project.ChangeControl;
import com.googlecode.prolog_cafe.lang.EvaluationException;
@ -61,8 +60,6 @@ public class PRED_current_user_1 extends Predicate.P1 {
resultTerm = anonymous;
} else if (curUser instanceof PeerDaemonUser) {
resultTerm = peerDaemon;
} else if (curUser instanceof ReplicationUser) {
resultTerm = replication;
} else {
throw new EvaluationException("Unknown user type");
}

6
gerrit-server/src/test/java/com/google/gerrit/server/git/ProjectConfigTest.java
View File

@ -28,6 +28,7 @@ import com.google.gerrit.common.data.Permission;
import com.google.gerrit.common.data.PermissionRule;
import com.google.gerrit.reviewdb.client.AccountGroup;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import org.eclipse.jgit.errors.ConfigInvalidException;
import org.eclipse.jgit.errors.IncorrectObjectTypeException;
@ -201,8 +202,9 @@ public class ProjectConfigTest extends LocalDiskRepositoryTestCase {
private RevCommit commit(ProjectConfig cfg) throws IOException,
MissingObjectException, IncorrectObjectTypeException {
MetaDataUpdate md = new MetaDataUpdate(new NoReplication(), //
cfg.getProject().getNameKey(), //
MetaDataUpdate md = new MetaDataUpdate(
GitReferenceUpdated.DISABLED,
cfg.getProject().getNameKey(),
db);
util.tick(5);
util.setAuthorAndCommitter(md.getCommitBuilder());

44
gerrit-server/src/test/java/com/google/gerrit/server/git/PushReplicationTest.java
View File

@ -1,44 +0,0 @@
// Copyright (C) 2011 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.server.git;
import static com.google.gerrit.server.git.PushReplication.ReplicationConfig.encode;
import static com.google.gerrit.server.git.PushReplication.ReplicationConfig.needsUrlEncoding;
import junit.framework.TestCase;
import org.eclipse.jgit.transport.URIish;
import java.net.URISyntaxException;
public class PushReplicationTest extends TestCase {
public void testNeedsUrlEncoding() throws URISyntaxException {
assertTrue(needsUrlEncoding(new URIish("http://host/path")));
assertTrue(needsUrlEncoding(new URIish("https://host/path")));
assertTrue(needsUrlEncoding(new URIish("amazon-s3://config/bucket/path")));
assertFalse(needsUrlEncoding(new URIish("host:path")));
assertFalse(needsUrlEncoding(new URIish("user@host:path")));
assertFalse(needsUrlEncoding(new URIish("git://host/path")));
assertFalse(needsUrlEncoding(new URIish("ssh://host/path")));
}
public void testUrlEncoding() {
assertEquals("foo/bar/thing", encode("foo/bar/thing"));
assertEquals("--%20All%20Projects%20--", encode("-- All Projects --"));
assertEquals("name/with%20a%20space", encode("name/with a space"));
assertEquals("name%0Awith-LF", encode("name\nwith-LF"));
}
}

9
gerrit-server/src/test/java/com/google/gerrit/server/git/SubmoduleOpTest.java
View File

@ -26,6 +26,7 @@ import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.reviewdb.client.SubmoduleSubscription;
import com.google.gerrit.reviewdb.server.ReviewDb;
import com.google.gerrit.reviewdb.server.SubmoduleSubscriptionAccess;
import com.google.gerrit.server.extensions.events.GitReferenceUpdated;
import com.google.gwtorm.client.KeyUtil;
import com.google.gwtorm.server.ListResultSet;
import com.google.gwtorm.server.ResultSet;
@ -71,7 +72,7 @@ public class SubmoduleOpTest extends LocalDiskRepositoryTestCase {
private ReviewDb schema;
private Provider<String> urlProvider;
private GitRepositoryManager repoManager;
private ReplicationQueue replication;
private GitReferenceUpdated replication;
@SuppressWarnings("unchecked")
@Override
@ -84,7 +85,7 @@ public class SubmoduleOpTest extends LocalDiskRepositoryTestCase {
subscriptions = createStrictMock(SubmoduleSubscriptionAccess.class);
urlProvider = createStrictMock(Provider.class);
repoManager = createStrictMock(GitRepositoryManager.class);
replication = createStrictMock(ReplicationQueue.class);
replication = createStrictMock(GitReferenceUpdated.class);
}
private void doReplay() {
@ -638,7 +639,7 @@ public class SubmoduleOpTest extends LocalDiskRepositoryTestCase {
expect(repoManager.openRepository(targetBranchNameKey.getParentKey()))
.andReturn(targetRepository);
replication.scheduleUpdate(targetBranchNameKey.getParentKey(),
replication.fire(targetBranchNameKey.getParentKey(),
targetBranchNameKey.get());
expect(schema.submoduleSubscriptions()).andReturn(subscriptions);
@ -739,7 +740,7 @@ public class SubmoduleOpTest extends LocalDiskRepositoryTestCase {
expect(repoManager.openRepository(targetBranchNameKey.getParentKey()))
.andReturn(targetRepository);
replication.scheduleUpdate(targetBranchNameKey.getParentKey(),
replication.fire(targetBranchNameKey.getParentKey(),
targetBranchNameKey.get());
expect(schema.submoduleSubscriptions()).andReturn(subscriptions);

4
gerrit-sshd/.settings/org.eclipse.core.resources.prefs
View File

@ -1,4 +1,6 @@
#Thu Jul 28 11:02:36 PDT 2011
#Tue May 15 09:21:12 PDT 2012
eclipse.preferences.version=1
encoding//src/main/java=UTF-8
encoding//src/main/resources=UTF-8
encoding//src/test/java=UTF-8
encoding/<project>=UTF-8

1
gerrit-sshd/src/main/java/com/google/gerrit/sshd/commands/MasterCommandModule.java
View File

@ -33,7 +33,6 @@ public class MasterCommandModule extends CommandModule {
command(gerrit, "gsql").to(AdminQueryShell.class);
command(gerrit, "set-reviewers").to(SetReviewersCommand.class);
command(gerrit, "receive-pack").to(Receive.class);
command(gerrit, "replicate").to(Replicate.class);
command(gerrit, "set-project-parent").to(AdminSetParent.class);
command(gerrit, "review").to(ReviewCommand.class);
command(gerrit, "set-account").to(SetAccountCommand.class);

82
gerrit-sshd/src/main/java/com/google/gerrit/sshd/commands/Replicate.java
View File

@ -1,82 +0,0 @@
// Copyright (C) 2009 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package com.google.gerrit.sshd.commands;
import com.google.gerrit.common.data.GlobalCapability;
import com.google.gerrit.reviewdb.client.Project;
import com.google.gerrit.server.IdentifiedUser;
import com.google.gerrit.server.git.PushAllProjectsOp;
import com.google.gerrit.server.git.ReplicationQueue;
import com.google.gerrit.server.project.ProjectCache;
import com.google.gerrit.sshd.RequiresCapability;
import com.google.gerrit.sshd.SshCommand;
import com.google.inject.Inject;
import org.kohsuke.args4j.Argument;
import org.kohsuke.args4j.Option;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
/** Force a project to replicate, again. */
@RequiresCapability(GlobalCapability.START_REPLICATION)
final class Replicate extends SshCommand {
@Option(name = "--all", usage = "push all known projects")
private boolean all;
@Option(name = "--url", metaVar = "PATTERN", usage = "pattern to match URL on")
private String urlMatch;
@Argument(index = 0, multiValued = true, metaVar = "PROJECT", usage = "project name")
private List<String> projectNames = new ArrayList<String>(2);
@Inject
IdentifiedUser currentUser;
@Inject
private PushAllProjectsOp.Factory pushAllOpFactory;
@Inject
private ReplicationQueue replication;
@Inject
private ProjectCache projectCache;
@Override
protected void run() throws Failure {
if (all && projectNames.size() > 0) {
throw new UnloggedFailure(1, "error: cannot combine --all and PROJECT");
}
if (!replication.isEnabled()) {
throw new Failure(1, "error: replication not enabled");
}
if (all) {
pushAllOpFactory.create(urlMatch).start(0, TimeUnit.SECONDS);
} else {
for (final String name : projectNames) {
final Project.NameKey key = new Project.NameKey(name);
if (projectCache.get(key) != null) {
replication.scheduleFullSync(key, urlMatch);
} else {
throw new UnloggedFailure(1, "error: '" + name + "': not a Gerrit project");
}
}
}
}
}

2
gerrit-war/src/main/java/com/google/gerrit/httpd/WebAppInitializer.java
View File

@ -33,7 +33,6 @@ import com.google.gerrit.server.config.MasterNodeStartup;
import com.google.gerrit.server.config.SitePath;
import com.google.gerrit.server.contact.HttpContactStoreConnection;
import com.google.gerrit.server.git.LocalDiskRepositoryManager;
import com.google.gerrit.server.git.PushReplication;
import com.google.gerrit.server.git.ReceiveCommitsExecutorModule;
import com.google.gerrit.server.git.WorkQueue;
import com.google.gerrit.server.mail.SignedTokenEmailTokenVerifier;
@ -204,7 +203,6 @@ public class WebAppInitializer extends GuiceServletContextListener {
modules.add(new EhcachePoolImpl.Module());
modules.add(new SmtpEmailSender.Module());
modules.add(new SignedTokenEmailTokenVerifier.Module());
modules.add(new PushReplication.Module());
modules.add(new PluginModule());
modules.add(new CanonicalWebUrlModule() {
@Override