27 Commits

Author SHA1 Message Date
Logan Hanks
aa0baaecb4 Trim leading slash from redirect after oauth login
Bug: Issue 6059
Change-Id: I35899fa0c76263a4b732c08b4781752c5aca8f4c
2017-04-24 22:56:25 +00:00
Edwin Kempin
744d2b8967 Migrate external IDs to NoteDb (part 1)
In NoteDb external IDs are stored in the All-Users repository in a Git
Notes branch called refs/meta/external-ids where the sha1 of the
external ID is used as note name. Each note content is a Git config
file that contains an external ID. It has exactly one externalId
subsection with an accountId and optionally email and password:

  [externalId "username:jdoe"]
     accountId = 1003407
     email = jdoe@example.com
     password = bcrypt:4:LCbmSBDivK/hhGVQMfkDpA==:XcWn0pKYSVU/UJgOvhidkEtmqCp6oKB7

Storing the external IDs in a Git Notes branch with using the sha1 of
the external ID as note name ensures that external IDs are unique and
are only assigned to a single account. If it is tried to assign the
same external ID concurrently to different accounts, only one Git
update succeeds while the other Git updates fail with LOCK_FAILURE.
This means assigning external IDs is also safe in a multimaster setup
if a consensus algorithm for updating Git refs is implemented (which
is needed for multimaster in any case). Alternatively it was
considered to store the external IDs per account as Git config file in
the refs/users/<sharded-id> user branches in the All-Users repository
(see abandoned change 9f9f07ef). This approach was given up because in
race conditions it allowed to assign the same external ID to different
accounts by updating different branches in Git.

To support a live migration on a multi-master Gerrit installation, the
migration of external IDs from ReviewDb to NoteDb is done in 2 steps:

- part 1 (this change):
  * always write to both backends (ReviewDb and NoteDb)
  * always read external IDs from ReviewDb
  * upgraded instances write to both backends, old instances only
    write to ReviewDb
  * after upgrading all instances (all still read from ReviewDb)
    run a batch to copy all external IDs from the ReviewDb to NoteDb
- part 2 (next change):
  * bump the database schema version
  * migrate the external IDs from ReviewDb to NoteDb (for single instance
    Gerrit servers)
  * read external IDs from NoteDb
  * delete the database table

With this change reading external IDs from NoteDb is not implemented
yet. This is because the storage format of external IDs in NoteDb
doesn't support efficient lookup of external IDs by account and this
problem is only addressed in the follow-up change (it adds a cache for
external IDs, but this cache uses the revision of the notes branch as
key, and hence can be only implemented once the external IDs are fully
migrated to NoteDb and storing external IDs in ReviewDb is dropped).

The ExternalIdsUpdate class implements updating of external IDs in
both NoteDb and ReviewDb. It provides various methods to update
external IDs (e.g. insert, upsert, delete, replace). For NoteDb each
method invocation leads to one commit in the Git notes branch.
ExternalIdsUpdate has two factories, User and Server. This allows to
record either the calling user or the Gerrit server identity as
committer for an update of the external Ids.

External IDs are now represented by a new AutoValue class called
ExternalId. This class replaces the usage of the old gwtorm entity
AccountExternalId class. For ExternalId scheme names are the same as for
AccountExternalId but no longer include the trailing ':'.

The class ExternalIdsOnInit makes it possible to update external IDs
during the init phase. This is required for inserting external IDs for
the initial admin user which is created by InitAdminUser. We need a
special class for this since not all dependencies of ExternalIdsUpdate
are available during init.

The class ExternalIdsBatchUpdate allows to do batch updates to
external IDs. For NoteDb all updates will result in a single commit to
the refs/meta/external-ids Git notes branch.

LocalUsernamesToLowerCase is now always converting the usernames in a
single thread only. This allows us to get a single commit for the
username convertion in NoteDb (this would not be possible if workers
do updates in parallel). Since LocalUsernamesToLowerCase is rather
light-weight being able to parallelize work is not really needed and
removing the workers simplifies the code significantly.

To protect the refs/meta/external-ids Git notes branch in the All-Users
repository read access for this ref is only allowed to users that have
the 'Access Database' global capability assigned. In addition
there is a commit validator that disallows updating the
refs/meta/external-ids branch by push. This is to prevent that the
external IDs in NoteDb diverge from the external IDs in ReviewDb while
the migration to NoteDb is not fully done yet.

Change-Id: Ic9bd5791e84ee8d332ccb1f709970b59ee66b308
Signed-off-by: Edwin Kempin <ekempin@google.com>
2017-02-28 09:09:39 +01:00
Dave Borowitz
292fa154c1 Format all Java files with google-java-format
Having a standard tool for formatting saves reviewers' valuable time.
google-java-format is Google's standard formatter and is somewhat
inspired by gofmt[1]. This commit formats everything using
google-java-format version 1.2.

The downside of this one-off formatting is breaking blame. This can be
somewhat hacked around with a tool like git-hyper-blame[2], but it's
definitely not optimal until/unless this kind of feature makes its way
to git core.

Not in this change:
* Tool support, e.g. Eclipse. The command must be run manually [3].
* Documentation of best practice, e.g. new 100-column default.

[1] https://talks.golang.org/2015/gofmt-en.slide#3
[2] https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/git-hyper-blame.html
[3] git ls-files | grep java$ | xargs google-java-format -i

Change-Id: Id5f3c6de95ce0b68b41f0a478b5c99a93675aaa3
Signed-off-by: David Pursehouse <dpursehouse@collab.net>
2017-02-07 10:04:39 +09:00
David Ostrovsky
fdbfcad77d Remove Buck based build
Bug: Issue 5302
Change-Id: I6e860446ef30ff0ad1c7c49fc0e39d39d921820b
2017-01-23 12:44:58 +00:00
Edwin Kempin
45814b1fe3 Return Optional<Account.Id> from AccountManager.lookup(String)
This makes it more clear that a lookup may find no account.

Change-Id: Ia0ffe7547c365be1526eb959ff4b5e25cfa1d37e
Signed-off-by: Edwin Kempin <ekempin@google.com>
2016-12-28 16:14:15 +01:00
David Ostrovsky
fa18907d7f Bazel: Reformat build files
Reformat the Bazel build files with the buildifier tool [1].

The style is different for Bazel files. Most notably, indentation level
is 4 spaces instead of 2, and " is used instead of '.

[1] https://github.com/bazelbuild/buildifier

Change-Id: I95c0c6f11b6d76572797853b4ebb5cee5ebd3c98
2016-12-07 11:33:07 +00:00
David Ostrovsky
d5b21545d7 OAuthTokenCache: Use account id as key
Current implementation assumes that OAuth provider always exposes
user name by using it for key in OAuth token cache. For some OAuth
providers (most notably Google OAuth provider) this is not the case.

Move from using user name to account id for token cache key. Postpone
populating the cache to the point when the authentication already took
place and use returned account id as cache key.

Bug: Issue 4627
Change-Id: I59f15b7c5ca8be6d52b59d21fac58cba88ba7fe3
2016-10-18 23:37:24 +02:00
David Ostrovsky
b81b4f75ae Implement Bazel build
To run the tests:

  bazel test //...

To build the Gerrit plugin API, run:

  bazel build gerrit-plugin-api:plugin-api_deploy.jar

To build the Gerrit extension API, run:

  bazel build gerrit-extension-api:extension-api_deploy.jar

TODOs:

Licenses
Reduce visibility (all public for now)
Generate HTML Documentation
Core plugins
gerrit_plugin() rule to build plugins in tree and standalone modes
GWT UI (only gwt_module() skylark rule is provided, no gwt_binary())
PolyGerrit UI
WAR
Publish artifacts to Maven Central
Ask Bazel team to add Gerrit to their CI on ci.bazel.io

Contributed-By: Han-Wen Nienhuys <hanwen@google.com>
Change-Id: I9a86e670882a44a5c966579cdeb8ed79b1590de3
2016-06-14 21:12:02 +02:00
David Pursehouse
323247227d Enable and fix 'Statement unnecessarily nested within else clause' warnings
Change-Id: Ida6df4593fc2ab3c11581309b2b4a638229ea093
2016-06-02 10:20:37 +09:00
Michael Ochmann
524faceb97 Cache for OAuth access tokens
OAuth access tokens retrieved during login in the web UI are stored
privately in OAuthSession. There is no possibility for a user to
obtain that token, e.g. to authenticate with a native Git client.

This patch adds a persistent cache for OAuth tokens and modifies
OAuthSession to store tokens received during the login handshake
with the OAuth provider in this cache.

Since access tokens must be kept secret, the cache defines a new
extension point OAuthTokenEncrypter. If an encrypter is provided,
access tokens are encrypted before storing them in the cache, and
decrypted when reading from the cache. By default, no encryption
is applied.

In subsequent patches a REST API for retrieving OAuth tokens will
be added as well as a corresponding settings page.

Change-Id: I751dd5f70dd30823bd2f531e1ac1da0759f98976
Signed-off-by: Michael Ochmann <michael.ochmann@sap.com>
2016-05-10 13:47:34 +02:00
David Pursehouse
c5cc9ddd12 Import StandardCharsets.UTF_8 as static
Change-Id: I066770f2eddfa7e9c1a67e91544006d33062d6d0
2015-10-19 14:59:46 +09:00
David Pursehouse
885ad8a7ef Merge branch 'stable-2.10' into stable-2.11
* stable-2.10:
  OAuth: Simplify protocol implementation
  Allow to link user identity to another OAuth provider
  Update JGit to the 4.0.0.201505050340-m2 version
  Hybrid OpenID/OAuth: Support switching identities
  Hybrid OpenID/OAuth: Allow to link identity accross protocols
  OAuth: Check for session validity during logout

Change-Id: I9da0073a72d8c4327313405b11c66cd253ff640b
2015-05-07 10:37:48 +09:00
Saša Živkov
bad310ec3f Merge "OAuth: Check for session validity during logout" into stable-2.10 2015-05-06 13:00:24 +00:00
David Ostrovsky
f74bd4d8d0 OAuth: Simplify protocol implementation
Change-Id: Ia713593c57d9f68f6fcac8ff3978428052aee5fb
2015-05-05 22:35:07 +02:00
David Ostrovsky
6269edfc68 Allow to link user identity to another OAuth provider
GitHub-Bug: https://github.com/davido/gerrit-oauth-provider/issues/12
Change-Id: I9507d15983cd021ba883afbdf4e526091d55c517
2015-05-05 21:39:50 +02:00
David Ostrovsky
8573bed76a OAuth: Check for session validity during logout
GitHub-Bug: https://github.com/davido/gerrit-oauth-provider/issues/9
Change-Id: Id25792cdf6e28ba8d0f97bcc41d8c6409558314e
2015-04-25 12:39:28 +02:00
David Pursehouse
e6af899893 Merge branch 'stable-2.10' into stable-2.11
* stable-2.10:
  OAuth: Respect servlet context path in URL for login token
  Invalidate OAuth session after web_sessions cache expiration

Change-Id: Id66e2ef07c164c17fa7839ea5c5ae19f26abd621
2015-04-14 14:29:36 +09:00
Shawn Pearce
3cab540024 Merge "OAuth: Respect servlet context path in URL for login token" into stable-2.10 2015-04-14 05:14:40 +00:00
David Ostrovsky
87d15d972d OAuth: Respect servlet context path in URL for login token
Due to a limitation in Jetty [1] we cannot rely on getPathInfo() from
web filter and need to strip the context path manually.

[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=28323

GitHub-Bug: https://github.com/davido/gerrit-oauth-provider/issues/6
Change-Id: Ie5e82abfc1b03b5be72769e05665ecd6099d2897
2015-04-13 05:45:09 +00:00
David Ostrovsky
6b18e3a003 Invalidate OAuth session after web_sessions cache expiration
When web_sessions cache is expired, OAuth session preserves it
logged in state. This makes new sign-in impossible.

Rectify it by checking the states mismatch and invalidating OAuth
session when web_sessions cache was expired.

GitHub-Bug: https://github.com/davido/gerrit-oauth-provider/issues/5
Change-Id: I3d57193c5af29561fd1fac0804dd19c08a0e9dbe
2015-04-12 10:43:25 +02:00
David Pursehouse
245dc68a01 Merge branch 'stable-2.10' into stable-2.11
* stable-2.10:
  Update version to 2.10.2
  Release notes for Gerrit 2.10.2
  Do not return 403 when clicking on Gitweb breadcrumb
  Add log messages to troubleshoot OAuth/OpenID linking
  Remove unused OAuthToken in authorisation URL
  OnlineReindexer: log the success/failure numbers on exit
  Update replication plugin
  OAuth: Allow to link claimed identity to existing accounts
  OAuth: Allow to change username

Change-Id: Ia9fc371b9f957c8e0fc3e215084baa3d31dadd41
2015-03-29 20:49:13 +01:00
Saša Živkov
ca29f4292f Merge "Remove unused OAuthToken in authorisation URL" into stable-2.10 2015-03-26 12:33:24 +00:00
David Ostrovsky
87b782b16b Add log messages to troubleshoot OAuth/OpenID linking
Change-Id: Ic8e13eb570d66e144520c29cd65308ce1f1d15c1
2015-03-26 09:40:53 +00:00
Luca Milanesio
3408d4deb8 Remove unused OAuthToken in authorisation URL
When the user needs to be redirected to the OAuth authentication URL
for entering their credentials, the session is not active yet and
there is no OAuthToken available. There is no value then in having
a RequestToken parameter that will always be null anyway.

Change-Id: I00fdbd32923a51e0c92e6bc0efff551936ec344f
2015-03-25 13:19:14 +00:00
David Ostrovsky
043c85728a OAuth: Allow to link claimed identity to existing accounts
One of use cases OAuth plugin based authentication scheme is aiming
to support is switch from deprecated OpenID provider to OAuth scheme
offered by the same povider. In this specific case the database is
already pre-populated with OpenID accounts. After switching the auth
scheme to OAuth all existing accounts must be linked to the new OAuth
identity.

To support linking new OAuth identity to existing accounts, user info
extension point is extended with claimed identity attribute. When
passed, the account for this identity is looked up and when found new
OAuth identity is linked to it.

Change-Id: Ia6489762dd370bfbbaa16a7418cd3106d2d1112a
2015-03-20 07:31:40 +01:00
David Pursehouse
21b83da225 Merge branch 'stable-2.10' into stable-2.11
* stable-2.10:
  Set version to 2.10.1
  Release notes for Gerrit 2.10.1
  Expose extension point for generic OAuth providers

Change-Id: I33006b0ea12611590979f376816d066883177d4f
2015-03-16 14:10:54 +09:00
David Ostrovsky
e9707d8f85 Expose extension point for generic OAuth providers
Further development of OAuth authentication scheme support suggested in
I86fb8fab3 is to restrict the core to expose only the OAuth extension
point and use Gerrit plugin concept for OAuth provider implementations.

When multiple OAuth providers are deployed on Gerrit site (from one or
multiple plugins) selection page is shown to select OAuth provider per
user base (as it's known for OpenID authentication scheme). The only
difference is that the user can only select between deployed providers.

OAuth logo was borrowed from:

http://en.wikipedia.org/wiki/OAuth and
http://en.wikipedia.org/wiki/File:Oauth_logo.svg
The OAuth logo, designed by Chris Messina
Creative Commons Attribution-Share Alike 3.0 Unported license

Converted as base64 using: http://www.base64-image.de
Source for OAuth protocol description: the same link as above.

Bug: issue 2677
Bug: issue 2715
Contributed-by: Luca Milanesio <luca.milanesio@gmail.com>
Change-Id: I7da0a6b3f2a99b6188bd14cf2818f673a3ddd680
2015-03-06 01:10:40 +01:00