Commit Graph

744 Commits

Author SHA1 Message Date
Lance Bragstad
bb0393623c Write a symptom for checking memcache connections
This makes it easier for operators to troubleshoot connection issues to
Memcached.

Related-Bug: 1332058

Change-Id: I6e67363822480314b93608bb1eae3514f1480f6d
2020-08-26 10:28:39 -05:00
Zuul
63a4d95ea2 Merge "Remove an assignment from domain and project" 2020-08-25 22:54:38 +00:00
Lance Bragstad
8bf222ac5d Properly handle octet (byte) strings when converting LDAP responses
If LDAP returns a UUID as an octet string the LDAP driver will fail to
convert it to something meaningful. The error usually looks something
like:

  ID attribute objectGUID not found in LDAP object

Microsoft AD's `objectGUID` parameter is stored and transmitted as an
octet string [0]. If you attempt to use the `objectGUID` to generate
user or group IDs, you'll get an HTTP 404 because keystone can't decode
it properly. This is unfortunate because `objectGUID` are a fixed
length, UUID format, and ideal for generating IDs in keystone. As
opposed to using the object's CN, which is variable length, and can
generate hashes that are larger than keystone's database table limit for
user IDs.

[0] https://docs.microsoft.com/en-us/windows/win32/ad/reading-an-objectampaposs-objectguid-and-creating-a-string-representation-of-the-guid

Change-Id: Id80b17bdff015e10340e636102576b7435bd564f
Closes-Bug: 1889936
2020-08-05 14:25:18 -05:00
Zuul
3da5eb8501 Merge "Fix "allow expired" feature for JWT" 2020-07-29 12:01:25 +00:00
Zuul
dc68ee4816 Merge "Support regexes in whitelists/blacklists" 2020-07-16 20:03:47 +00:00
Vishakha Agarwal
2707498474 Fix "allow expired" feature for JWT
GET /v3/auth/tokens?allow_expired=1 works fine with fernet tokens
returning the expired token data, whereas it returns exception
TokenNotFound for JWT. This patch fixes the same.

Change-Id: I03f6c58dce7d140d62055a97063aeb480498e5e6
Closes-Bug: #1886017
2020-07-08 17:30:36 +05:30
Pavlo Shchelokovskyy
c9c655a1e1 Add ignore_user_inactivity user option
this option allows to override the
[security_compliance]disable_user_account_days_inactive setting from
config on per-user basis.

Co-Authored-By: Vishakha Agarwal <agarwalvishakha18@gmail.com>

Change-Id: Ida360e215426184195687bee2a800877af33af04
Closes-Bug: #1827431
2020-07-07 20:40:52 +05:30
Zuul
e3bd1d747d Merge "New config option 'user_limit' in credentials" 2020-06-23 20:22:29 +00:00
Vishakha Agarwal
28faa24e68 Remove an assignment from domain and project
When you setup a user with a role assignment on a domain
and then a role assignment on a project "acting as a domain",
you can't actually remove them. The database throws you the
error "Multiple rows were found for one()" since it gets two
results for "actor_id" with the same "target_id".

This patch fixes this problem by filtering the database query
by "type" field to determine whether it is a user domain relation
or a user project and then removing the assignment.

Change-Id: Ife92a3c9e0982baafb4224882681c0855f573580
Closes-Bug: #1754677
2020-06-22 14:43:22 +05:30
OpenStack Proposal Bot
a0346effc8 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I8910eccab6d14c2ce6b6496b18b4987aff64fc9b
2020-06-19 07:42:43 +00:00
Zuul
104437c1c5 Merge "Disable EC2 credentials access_id update" 2020-06-18 11:14:46 +00:00
Vishakha Agarwal
a49ee620fa New config option 'user_limit' in credentials
This patch allows adds new config option 'user_limit'
to credentials to set maximum number of credentials a
user is permitted to create.
Closes-Bug: #1872732

Change-Id: Ic9dc9a4a9ec1ecbf01842c865e19a7a100e5041d
2020-06-12 13:52:45 +05:30
Jason Anderson
feaf034438
Support regexes in whitelists/blacklists
This adds support for the "regex" flag for both the "whitelist" and
"blacklist" conditional types. Before, only the "any_one_of" and
"not_any_of" conditionals supported this. Similar to the pre-existing
regex logic, the patterns are matched from the beginning of the string,
meaning you may need prefix them with ".*" if you do not care about the
first characters of the match.

Closes-Bug: #1880252
Change-Id: Ia51f47a58712c7230753f2cfa0c87b83a7339bf9
2020-05-26 11:00:21 -05:00
Andreas Jaeger
7820cafffe Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* parallelizing building of documents

Update Sphinx version as well.

openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.

Set openstackdocs_auto_name to use project as name.
Set openstackdocs_pdf_link to link to PDF file.

Remove docs requirements from lower-constraints, they are not installed.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I320a69816b4101bb76b88448881f3177c892ea92
2020-05-21 13:43:22 +00:00
Vishakha Agarwal
252c23b1b8 Disable EC2 credentials access_id update
Without this patch user can alter EC2 credential access_id and user
cannot use it anymore as an ec2 auth token since EC2 credential
access ID is used to calculate an ID of the "credential" [1] and it
doesn't update the EC2 credential ID with new access ID. This leads
to unwanted EC2 credentials stored in database.

As per the discussion of keystone team [2] we decided to block patching
of "access_id" attribute.

[1] 7bb6314e40/keystone/api/users.py (L363)
[2]http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2020-05-12.log.html#t2020-05-12T17:45:20

Closes-Bug: #1872753
Change-Id: I1f6ce3927c2881d9a2d7dcda3ccd29e0a82e45a9
2020-05-19 17:35:05 +05:30
Zuul
241fa9fe12 Merge "Fix security issues with EC2 credentials" 2020-05-08 16:14:46 +00:00
Zuul
79d7d929e1 Merge "Ensure OAuth1 authorized roles are respected" 2020-05-07 04:37:51 +00:00
Zuul
963cc3a692 Merge "Update master for stable/ussuri" 2020-05-02 20:02:59 +00:00
Colleen Murphy
37e9907a17 Fix security issues with EC2 credentials
This change addresses several issues in the creation and use of EC2/S3
credentials with keystone tokens.

1. Disable altering credential owner attributes or metadata

Without this patch, an authenticated user can create an EC2 credential
for themself for a project they have a role on, then update the
credential to target a user and project completely unrelated to them. In
the worst case, this could be the admin user and a project the admin
user has a role assignment on. A token granted for an altered credential
like this would allow the user to masquerade as the victim user. This
patch ensures that when updating a credential, the new form of the
credential is one the acting user has access to: if the system admin
user is changing the credential, the new user ID or project ID could be
anything, but regular users may only change the credential to be one
that they still own.

Relatedly, when a user uses an application credential or a trust to
create an EC2 credential, keystone automatically adds the trust ID or
application credential ID as metadata in the EC2 access blob so that it
knows how the token can be scoped when it is used. Without this patch, a
user who has created a credential in this way can update the access blob
to remove or alter this metadata and escalate their privileges to be
fully authorized for the trustor's, application credential creator's, or
OAuth1 access token authorizor's privileges on the project. This patch
fixes the issue by simply disallowing updates to keystone-controlled
metadata in the credential.

2. Respect token roles when creating EC2 credentials

Without this patch, a trustee, an application credential user, or an
OAuth1 access token holder could create an EC2 credential or an
application credential using any roles the trustor, application
credential creator, or access token authorizor had on the project,
regardless of whether the creator had delegated only a limited subset of
roles. This was because the trust_id attribute of the EC2 access blob
was ignored, and no metadata for the application credential or access
token was recorded either. This change ensures that the access
delegation resource is recorded in the metadata of the EC2 credential
when created and passed to the token provider when used for
authentication so that the token provider can look up the correct roles
for the request.

Change-Id: I39d0d705839fbe31ac518ac9a82959e108cb7c1d
Closes-bug: #1872733
Closes-bug: #1872755
Closes-bug: #1872735
2020-05-02 12:34:20 -07:00
Colleen Murphy
6c73690f77 Ensure OAuth1 authorized roles are respected
Without this patch, when an OAuth1 request token is authorized with a
limited set of roles, the roles for the access token are ignored when
the user uses it to request a keystone token. This means that user of an
access token can use it to escallate their role assignments beyond what
was authorized by the creator. This patch fixes the issue by ensuring
the token model accounts for an OAuth1-scoped token and correctly
populating the roles for it.

Change-Id: I02f9836fbd4d7e629653977fc341476cfd89859e
Closes-bug: #1873290
2020-05-01 15:48:58 -07:00
Colleen Murphy
ab89ea7490 Check timestamp of signed EC2 token request
EC2 token requests contain a signature that signs the entire request,
including the access timestamp. While the signature is checked, the
timestamp is not, and so these signed requests remain valid
indefinitely, leaving the token API vulnerable to replay attacks. This
change introduces a configurable TTL for signed token requests and
ensures that the timestamp is actually validated against it.

The check will work for either an AWS Signature v1/v2 'Timestamp'
parameter[1] or the AWS Signature v4 'X-Aws-Date' header or
parameter[2].

Although this technically adds a new feature and the default value of
the feature changes behavior, this change is required to protect
credential holders and therefore must be backported to all supported
branches.

[1] https://docs.aws.amazon.com/general/latest/gr/signature-version-2.html
[2] https://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html

Change-Id: Idb10267338b4204b435df233c636046a1ce5711f
Closes-bug: #1872737
2020-04-28 11:45:24 -07:00
0b26e64f45 Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: Ia08507c8c15ff556af254e9eb1cf9b49e9cd04d6
Sem-Ver: feature
2020-04-24 08:42:57 +00:00
Zuul
84d564582a Merge "Stop adding entry in local_user while updating ephemerals" 2020-04-20 20:34:43 +00:00
Kristi Nikolla
d23965aaf1 Update api-ref for federated objects in user
Also includes a release note.

Change-Id: I72a5d461488b50f20b59d1288016514a2b8f71e5
Closes-Bug: 1816076
2020-04-10 00:49:35 +00:00
Zuul
b3d3735138 Merge "Add expiring user group memberships on mapped authentication" 2020-04-10 00:18:37 +00:00
Kristi Nikolla
8153a9d592 Add expiring user group memberships on mapped authentication
When a federated user authenticates, they are added to their
mapped groups during shadowing.

Closes-Bug: 1809116

Change-Id: I19dc400b2a7aa46709b242cdeef82beaca975ff3
2020-04-07 19:30:57 -04:00
Zuul
9f9040257f Merge "Add openstack_groups to assertion" 2020-03-23 19:24:38 +00:00
Vishakha Agarwal
dda426b61a Add openstack_groups to assertion
Currently, a keystone IdP does not provide the
groups to which user belong when generating SAML
assertions.This patch adds an additional attribute
called "openstack_groups" in the assertion.

Change-Id: I205e8bbf9a4579b16177f57e29e363f4205a2b48
Closes-Bug: #1641625
2020-03-19 20:14:41 +05:30
Andreas Jaeger
e715a4bbd0 Doc Cleanup
Remove unused git_cmd from api-ref.
Remove html_last_updated_fmt and latex_engine setting,
these are done by openstackdocstheme nowadays.

Change-Id: I1c63f83b3fa074f9fa136e0b89bba0586756bc56
2020-02-23 18:16:01 +01:00
Zuul
cdab9284d1 Merge "Use inspect instead of Inspector.from_engine()" 2020-02-21 23:04:11 +00:00
Colleen Murphy
da28046944 Default to bootstrapping roles as immutable
In the previous cycle, the ``--immutable-roles`` option was added to the
bootstrap command as an optional way to opt-in to making the default
roles immutable. Following step 4 of the spec[1], we now make that
behavior the default and additionally offer a way to opt out of it.

[1] http://specs.openstack.org/openstack/keystone-specs/specs/keystone/train/immutable-resources.html#proposed-change

Change-Id: I6b680efb2c87c1d7559ddcc989bbce68456b9a5f
Closes-Bug: #1823258
2020-02-04 14:21:13 -08:00
Mike Bayer
527b1587e1 Use inspect instead of Inspector.from_engine()
SQLAlchemy is likely going to deprecate
Inspector.from_engine() which will start emitting a deprecation
warning in 1.4.   Use sqlalchemy.inspect() instead.

Change-Id: I287ba10f2e3308950b9caf6f51f3ee1db29a6448
2020-02-04 16:24:31 -05:00
Zuul
99733f172f Merge "Always have username in CADF initiator" 2020-01-29 23:53:46 +00:00
Zuul
7bb26ffd15 Merge "Fix role_assignments role.id filter" 2020-01-27 20:00:05 +00:00
Gage Hugo
95edaaab06 Always have username in CADF initiator
The current initiator object for CADF notifications does not include
the username of the user who initiated the action, which leads to
issues when using an LDAP backend and not having a direct way to
map a username to a user id.

This change makes it so that the initiator object for CADF
notifications always contains the username for a user as well
as the user id. This follows along with the CADF standard
for OpenStack[0].

[0] https://www.dmtf.org/sites/default/files/standards/documents/DSP2038_1.1.0.pdf#page=12

Closes-Bug: #1856904

Change-Id: I833e6e0d7792acf49f816050ad7a63e8ea4f702f
2020-01-09 15:55:48 +00:00
Zuul
28a1da6c74 Merge "Fix release note link formatting" 2020-01-08 19:30:24 +00:00
Zuul
406a7fb222 Merge "Ensure bootstrap handles multiple roles with the same name" 2020-01-08 00:20:08 +00:00
Lance Bragstad
25cf359e5f Ensure bootstrap handles multiple roles with the same name
The bootstrap logic doesn't take into consideration multiple roles
with the same name. If bootstrap is unable to determine which role to
use and accidentally uses a domain-specific role with the same name
as a default role, bootstrap will fail in unexpected ways.

Closes-Bug: 1856881
Change-Id: Iddc364d8c934b6e54d1e8c75b8b159faadbf865d
2020-01-02 13:44:23 -06:00
Colleen Murphy
c2d8830662 Fix role_assignments role.id filter
Without this patch, if there are multiple role assignments on the system
and they are not all the same role, querying for role assignments with
/v3/role_assignments?role.id={role_id} may leak some role assignments
that don't match the role_id, making the returned results incorrect.
This patch fixes the issue by using a list comprehension instead of a
for loop over a list that was being modified within the loop.

Change-Id: Icfce3b14abb55c6fef3de1b314cee22fc8b1d08c
Closes-bug: #1858012
2020-01-02 09:24:54 -08:00
Colleen Murphy
150d3ef8b0 Fix release note link formatting
Corrects the RST link formatting for a bugfix release note. This note
was added during the current cycle so it is fine to modify it on the
master branch.

Change-Id: Id552e936b780f8bc31523a771937b9f9307cbda1
2019-12-30 08:57:50 -08:00
shenjiatong
f0d964e666 Fix token auth error if federated_groups_id is empty list
`federation_group_ids` could be zero length list, so deciding whether
a token is federated by checking if it is none.

Change-Id: I0f4b9e24d949aa4838ee721a165999b29c684d32
Closes-Bug: #1856962
2019-12-30 08:49:24 -08:00
Pedro Martins
7597ecc135 Stop adding entry in local_user while updating ephemerals
Problem description
===================
Today we have a consistency problem when updating federated
users via OpenStack. When I update a ephemeral user via OpenStack,
a registry in the local_user table is created, making this user
having entries in user, local_user and federated_user tables in
the database.

Furthermore, if I try to do some operations using this user
(that has entries in all three tables), I get a "More than one
user exists with the name ..." error from the OpenStack
Keystone API. It happens because the user has an entry in both
local_user and federated_user tables.

I fix the persistence in the local_user table for ephemeral
users when doing updates.

Proposal
========
I fix the problem with creating an entry in the
local_user table while updating an ephemeral user

Closes-Bug: #1848342

Change-Id: I2ac6e90f24b94dc5c0d9c0758f008a388597036c
2019-12-11 16:07:06 -03:00
Colleen Murphy
17c337dbdb Fix credential list for project members
Without this patch, project members and readers can list any credentials
with the /v3/credentials API when enforce_scope is false. enforce_scope
is only applicable to project admins due to the admin-ness problem[1],
and this policy is not meant to allow project admins any access to users'
credentials (only system admins should be able to access them). However,
when enforce_scope is false, we need to preserve the old behavior of
project admins being able to list all credentials. This change mitigates
the problem by running the identity:get_credential policy check to
filter out credentials the user does not have access to. This will
impact performance.

Closes-bug: #1855080

[1] https://bugs.launchpad.net/keystone/+bug/968696

Change-Id: I5dd85a6b8368373a27aef2942a64499d020662ef
2019-12-04 16:42:17 -08:00
Zuul
af1c1a822a Merge "Stop testing Python 2" 2019-11-14 10:28:52 +00:00
Zuul
17e518f1ba Merge "Revert "Resource backend is SQL only now"" 2019-11-14 01:11:04 +00:00
Zuul
09ba263fb4 Merge "Drop project.id foreign keys" 2019-11-12 20:42:23 +00:00
Arthur Dayne
a92885a98b Stop testing Python 2
In Ussuri, Drop support for Python 2 according to [1] and [2]

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010356.html
[2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html

Change-Id: I35e871992dd21e96e7e812e440e6cafdb61d26ac
2019-11-01 08:57:08 +08:00
Sami MAKKI
d6977a0e9b Remove group deletion for non-sql driver when removing domains.
As LDAP is now read-only, trying to remove it was throwing an error.
We now only try to delete it when the driver is sql-based.

Change-Id: I15b92b35b31d0e5d735a629e7c154ddd7bdda03d
Closes-bug: #1848238
2019-10-29 12:19:51 -07:00
Zuul
e4e3b0cdfb Merge "Update master for stable/train" 2019-10-14 20:56:19 +00:00
Colleen Murphy
9607ed3266 Revert "Resource backend is SQL only now"
This reverts commit 3d46c8a5d9.

In the last commit, the foreign key constraints between the project
table and other tables were dropped, which allows us to restore the
configurability of the resource driver.

Change-Id: Iba4951e2d3965be5acec705385967d312456f1c7
2019-10-11 14:13:04 -07:00