Browse Source
With the new config option `keystone_auth_default_policy`, cloud admin can set a default keystone auth policy for k8s cluster when the keystone auth is enabled. As a result, user can use their current keystone user to access k8s cluster as long as they're assigned correct roles, and they will get the pre-defined permissions set by the cloud provider. The default policy now is based on the v2 format recently introduced in k8s-keystone-auth which is getting more useful now. For example, in v1 it doesn't support a policy for user to access resources from all namespaces but kube-system, but v2 can do that. NOTE: Now we're using openstackmagnum dockerhub repo until CPO team fixing their image release issue. Task: 30069 Story: 1755770 Change-Id: I2425e957bd99edc92482b6f11ca0b1f91fe59ff6changes/25/643225/22
Feilong Wang
3 years ago
12 changed files with 349 additions and 24 deletions
@ -0,0 +1,145 @@ |
|||
Currently, there are several ways to access the Kubernetes API, such as RBAC, |
|||
ABAC, Webhook, etc. Though RBAC is the best way for most of the cases, Webhook |
|||
provides a good approach for Kubernetes to query an outside REST service when |
|||
determining user privileges. In other words, we can use a Webhook to integrate |
|||
other IAM service into Kubernetes. In our case, under the OpenStack context, |
|||
we're introducing the intergration with Keystone auth for Kubernetes. |
|||
|
|||
Since Rocky release, we introduced a new label named `keystone_auth_enabled`, |
|||
by default it's True, which means user can get this very nice feature out of |
|||
box. |
|||
|
|||
Create roles |
|||
------------ |
|||
|
|||
As cloud provider, necessary Keystone roles for Kubernetes cluster operations |
|||
need to be created for different users, e.g. k8s_admin, k8s_developer, |
|||
k8s_viewer |
|||
|
|||
- k8s_admin role can create/update/delete Kubernetes cluster, can also |
|||
associate roles to other normal users within the tenant |
|||
- k8s_developer can create/update/delete/watch Kubernetes cluster resources |
|||
- k8s_viewer can only have read access to Kubernetes cluster resources |
|||
|
|||
NOTE: Those roles will be created automatically in devstack. Below is the |
|||
samples commands about how to create them. |
|||
|
|||
.. code-block:: bash |
|||
|
|||
source ~/openstack_admin_credentials |
|||
for role in "k8s_admin" "k8s_developer" "k8s_viewer"; do openstack role create $role; done |
|||
|
|||
openstack user create demo_viewer --project demo --password password |
|||
openstack role add --user demo_viewer --project demo k8s_viewer |
|||
|
|||
openstack user create demo_editor --project demo --password password |
|||
openstack role add --user demo_developer --project demo k8s_developer |
|||
|
|||
openstack user create demo_admin --project demo --password password |
|||
openstack role add --user demo_admin --project demo k8s_admin |
|||
|
|||
Those roles should be public and can be accessed by any project so that user |
|||
can configure their cluster's role policies with those roles. |
|||
|
|||
Setup configmap for authorization policies |
|||
------------------------------------------ |
|||
|
|||
Given the k8s Keystone auth has been enable by default, user can get the |
|||
authentication support by default without doing anything. However, user can't |
|||
do anything actually before setup a default authorization policies. |
|||
|
|||
The authorization policy can be specified using an existing configmap name in |
|||
the cluster, by doing this, the policy could be changed dynamically without |
|||
the k8s-keystone-auth service restart. |
|||
|
|||
Or the policy can be read from a default policy file. In devstack, the policy |
|||
file will be created automatically. |
|||
|
|||
Currently, k8s-keystone-auth service supports four types of policies: |
|||
|
|||
- user. The Keystone user ID or name. |
|||
- roject. The Keystone project ID or name. |
|||
- role. The user role defined in Keystone. |
|||
- group. The group is not a Keystone concept actually, it’s supported for |
|||
backward compatibility, you can use group as project ID. |
|||
|
|||
For example, in the following configmap, we only allow the users in |
|||
project demo with k8s-viewer role in OpenStack to query the pod information |
|||
from all the namespaces. So we need to update the configmap |
|||
`k8s-keystone-auth-policy` which has been created in kube-system namespace. |
|||
|
|||
.. code-block:: bash |
|||
|
|||
cat <<EOF | kubectl apply -f - |
|||
apiVersion: v1 |
|||
kind: ConfigMap |
|||
metadata: |
|||
name: k8s-keystone-auth-policy |
|||
namespace: kube-system |
|||
data: |
|||
policies: | |
|||
[ |
|||
{ |
|||
"resource": { |
|||
"verbs": ["get", "list", "watch"], |
|||
"resources": ["pods"], |
|||
"version": "*", |
|||
"namespace": "default" |
|||
}, |
|||
"match": [ |
|||
{ |
|||
"type": "role", |
|||
"values": ["k8s-viewer"] |
|||
}, |
|||
{ |
|||
"type": "project", |
|||
"values": ["demo"] |
|||
} |
|||
] |
|||
} |
|||
] |
|||
EOF |
|||
|
|||
Please note that the default configmap name is `k8s-keystone-auth-policy`, user |
|||
can change it, but they have to change the config of the k8s keystone auth |
|||
service configuration as well and restart the service. |
|||
|
|||
Now user need to get a token from Keystone to have a kubeconfig for kubectl, |
|||
user can also get the config with Magnum python client. |
|||
|
|||
Here is a sample of the kubeconfig: |
|||
|
|||
.. code-block:: bash |
|||
|
|||
apiVersion: v1 |
|||
clusters: |
|||
- cluster: |
|||
certificate-authority-data: CERT-DATA== |
|||
server: https://172.24.4.25:6443 |
|||
name: k8s-2 |
|||
contexts: |
|||
- context: |
|||
cluster: k8s-2 |
|||
user: openstackuser |
|||
name: openstackuser@kubernetes |
|||
current-context: openstackuser@kubernetes |
|||
kind: Config |
|||
preferences: {} |
|||
users: |
|||
- name: openstackuser |
|||
user: |
|||
exec: |
|||
command: /bin/bash |
|||
apiVersion: client.authentication.k8s.io/v1alpha1 |
|||
args: |
|||
- -c |
|||
- > |
|||
if [ -z ${OS_TOKEN} ]; then |
|||
echo 'Error: Missing OpenStack credential from environment variable $OS_TOKEN' > /dev/stderr |
|||
exit 1 |
|||
else |
|||
echo '{ "apiVersion": "client.authentication.k8s.io/v1alpha1", "kind": "ExecCredential", "status": { "token": "'"${OS_TOKEN}"'"}}' |
|||
fi |
|||
|
|||
Now after export the Keystone token to OS_TOKEN, user should be able to list |
|||
pods with kubectl. |
|||
@ -0,0 +1,76 @@ |
|||
[ |
|||
{ |
|||
"users":{ |
|||
"roles":[ |
|||
"k8s_admin" |
|||
], |
|||
"projects":[ |
|||
"$PROJECT_ID" |
|||
] |
|||
}, |
|||
"resource_permissions":{ |
|||
"*/*":[ |
|||
"*" |
|||
] |
|||
}, |
|||
"nonresource_permissions":{ |
|||
"/healthz":[ |
|||
"get", |
|||
"post" |
|||
] |
|||
} |
|||
}, |
|||
{ |
|||
"users":{ |
|||
"roles":[ |
|||
"k8s_developer" |
|||
], |
|||
"projects":[ |
|||
"$PROJECT_ID" |
|||
] |
|||
}, |
|||
"resource_permissions":{ |
|||
"!kube-system/['apiServices', 'bindings', 'componentstatuses', 'configmaps', 'cronjobs', 'customResourceDefinitions', 'deployments', 'endpoints', 'events', 'horizontalPodAutoscalers', 'ingresses', 'initializerConfigurations', 'jobs', 'limitRanges', 'localSubjectAccessReviews', 'namespaces', 'networkPolicies', 'persistentVolumeClaims', 'persistentVolumes', 'podDisruptionBudgets', 'podPresets', 'podTemplates', 'pods', 'replicaSets', 'replicationControllers', 'resourceQuotas', 'secrets', 'selfSubjectAccessReviews', 'serviceAccounts', 'services', 'statefulSets', 'storageClasses', 'subjectAccessReviews', 'tokenReviews']":[ |
|||
"*" |
|||
], |
|||
"*/['clusterrolebindings', 'clusterroles', 'rolebindings', 'roles', 'controllerrevisions', 'nodes', 'podSecurityPolicies']":[ |
|||
"get", |
|||
"list", |
|||
"watch" |
|||
], |
|||
"*/['certificateSigningRequests']":[ |
|||
"create", |
|||
"delete", |
|||
"get", |
|||
"list", |
|||
"watch", |
|||
"update" |
|||
] |
|||
} |
|||
}, |
|||
{ |
|||
"users":{ |
|||
"roles":[ |
|||
"k8s_viewer" |
|||
], |
|||
"projects":[ |
|||
"$PROJECT_ID" |
|||
] |
|||
}, |
|||
"resource_permissions":{ |
|||
"!kube-system/['tokenReviews']":[ |
|||
"*" |
|||
], |
|||
"!kube-system/['apiServices', 'bindings', 'componentstatuses', 'configmaps', 'cronjobs', 'customResourceDefinitions', 'deployments', 'endpoints', 'events', 'horizontalPodAutoscalers', 'ingresses', 'initializerConfigurations', 'jobs', 'limitRanges', 'localSubjectAccessReviews', 'namespaces', 'networkPolicies', 'persistentVolumeClaims', 'persistentVolumes', 'podDisruptionBudgets', 'podPresets', 'podTemplates', 'pods', 'replicaSets', 'replicationControllers', 'resourceQuotas', 'secrets', 'selfSubjectAccessReviews', 'serviceAccounts', 'services', 'statefulSets', 'storageClasses', 'subjectAccessReviews']":[ |
|||
"get", |
|||
"list", |
|||
"watch" |
|||
], |
|||
"*/['clusterrolebindings', 'clusterroles', 'rolebindings', 'roles', 'controllerrevisions', 'nodes', 'podSecurityPolicies']":[ |
|||
"get", |
|||
"list", |
|||
"watch" |
|||
] |
|||
} |
|||
} |
|||
] |
|||
@ -0,0 +1,36 @@ |
|||
# Licensed under the Apache License, Version 2.0 (the "License"); you may not |
|||
# use this file except in compliance with the License. You may obtain a copy |
|||
# of the License at |
|||
# |
|||
# http://www.apache.org/licenses/LICENSE-2.0 |
|||
# |
|||
# Unless required by applicable law or agreed to in writing, software |
|||
# distributed under the License is distributed on an "AS IS" BASIS, |
|||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|||
# See the License for the specific language governing permissions and |
|||
# limitations under the License. |
|||
|
|||
from oslo_config import cfg |
|||
|
|||
kubernetes_group = cfg.OptGroup(name='kubernetes', |
|||
title='Options for the Kubernetes addons') |
|||
|
|||
kubernetes_opts = [ |
|||
cfg.StrOpt('keystone_auth_default_policy', |
|||
default="/etc/magnum/keystone_auth_default_policy.json", |
|||
help='Explicitly specify the path to the file defined default ' |
|||
'Keystone auth policy for Kubernetes cluster when ' |
|||
'the Keystone auth is enabled. Vendors can put their ' |
|||
'specific default policy here'), |
|||
] |
|||
|
|||
|
|||
def register_opts(conf): |
|||
conf.register_group(kubernetes_group) |
|||
conf.register_opts(kubernetes_opts, group=kubernetes_group) |
|||
|
|||
|
|||
def list_opts(): |
|||
return { |
|||
kubernetes_group: kubernetes_opts |
|||
} |
|||
@ -0,0 +1,9 @@ |
|||
--- |
|||
issues: |
|||
- | |
|||
With the new config option keystone_auth_default_policy, cloud admin |
|||
can set a default keystone auth policy for k8s cluster when the |
|||
keystone auth is enabled. As a result, user can use their current |
|||
keystone user to access k8s cluster as long as they're assigned |
|||
correct roles, and they will get the pre-defined permissions |
|||
defined by the cloud provider. |
|||
Loading…
Reference in new issue