Merge "Fix usage of the trustee user in K8S Cinder plugin"
This commit is contained in:
commit
ff58a66b08
@ -78,57 +78,7 @@ neutron_lbaas.conf::
|
||||
|
||||
To configure LBaaS v1 or v2, refer to the Neutron documentation.
|
||||
|
||||
To enable the load balancer, log into each master node of your cluster and
|
||||
perform the following steps:
|
||||
|
||||
1. Configure kube-apiserver::
|
||||
|
||||
sudo vi /etc/kubernetes/apiserver
|
||||
|
||||
Comment out the line::
|
||||
|
||||
#KUBE_API_ARGS="--runtime_config=api/all=true"
|
||||
|
||||
Uncomment the line::
|
||||
|
||||
KUBE_API_ARGS="--runtime_config=api/all=true --cloud_config=/etc/sysconfig/kube_openstack_config --cloud_provider=openstack"""
|
||||
|
||||
2. Configure kube-controller-manager::
|
||||
|
||||
sudo vi /etc/kubernetes/manifests/kube-controller-manager.yaml
|
||||
|
||||
Immediately after the lines::
|
||||
|
||||
- controller-manager
|
||||
- --master=http://127.0.0.1:8080
|
||||
- --service-account-private-key-file=/etc/kubernetes/ssl/server.key
|
||||
- --root-ca-file=/etc/kubernetes/ssl/ca.crt
|
||||
|
||||
Add the following lines::
|
||||
|
||||
- --cloud_config=/etc/sysconfig/kube_openstack_config
|
||||
- --cloud_provider=openstack
|
||||
|
||||
When the file is saved, the pod will automatically restart the
|
||||
kube-controller-manager container to pick up the change.
|
||||
|
||||
3. Enter OpenStack user credential::
|
||||
|
||||
sudo vi /etc/sysconfig/kube_openstack_config
|
||||
|
||||
The username and tenant-name entries have been filled in with the
|
||||
Keystone values of the user who created the cluster. Enter the password
|
||||
of this user on the entry for password::
|
||||
|
||||
password=ChangeMe
|
||||
|
||||
4. Restart the Kubernetes API server::
|
||||
|
||||
sudo service kube-apiserver restart
|
||||
service kube-apiserver status
|
||||
|
||||
This only needs to be done once. The steps can be reversed to disable the
|
||||
load balancer feature. Before deleting the Kubernetes cluster, make sure to
|
||||
Before deleting the Kubernetes cluster, make sure to
|
||||
delete all the services that created load balancers. Because the Neutron
|
||||
objects created by Kubernetes are not managed by Heat, they will not be
|
||||
deleted by Heat and this will cause the cluster-delete operation to fail. If
|
||||
@ -138,6 +88,9 @@ lb-healthmonitor) and then run cluster-delete again.
|
||||
Steps for the users
|
||||
===================
|
||||
|
||||
This feature requires the OpenStack cloud provider to be enabled.
|
||||
To do so, enable the cinder support (--volume-driver cinder).
|
||||
|
||||
For the user, publishing the service endpoint externally involves the following
|
||||
2 steps:
|
||||
|
||||
|
@ -2212,12 +2212,8 @@ Following are some examples for using Cinder as persistent storage.
|
||||
Using Cinder in Kubernetes
|
||||
++++++++++++++++++++++++++
|
||||
|
||||
**NOTE:** This feature requires Kubernetes version 1.1.1 or above and
|
||||
Docker version 1.8.3 or above. The public Fedora image from Atomic
|
||||
currently meets this requirement.
|
||||
|
||||
**NOTE:** The following steps are a temporary workaround, and Magnum's
|
||||
development team is working on a long term solution to automate these steps.
|
||||
**NOTE:** This feature requires Kubernetes version 1.5.0 or above.
|
||||
The public Fedora image from Atomic currently meets this requirement.
|
||||
|
||||
1. Create the ClusterTemplate.
|
||||
|
||||
@ -2240,49 +2236,6 @@ development team is working on a long term solution to automate these steps.
|
||||
--cluster-template k8s-cluster-template \
|
||||
--node-count 1
|
||||
|
||||
|
||||
3. Configure kubelet.
|
||||
|
||||
To allow Kubernetes to interface with Cinder, log into each minion
|
||||
node of your cluster and perform step 4 through 6::
|
||||
|
||||
sudo vi /etc/kubernetes/kubelet
|
||||
|
||||
Comment out the line::
|
||||
|
||||
#KUBELET_ARGS=--config=/etc/kubernetes/manifests --cadvisor-port=4194
|
||||
|
||||
Uncomment the line::
|
||||
|
||||
#KUBELET_ARGS="--config=/etc/kubernetes/manifests --cadvisor-port=4194 --cloud-provider=openstack --cloud-config=/etc/kubernetes/kube_openstack_config"
|
||||
|
||||
|
||||
4. Enter OpenStack user credential::
|
||||
|
||||
sudo vi /etc/kubernetes/kube_openstack_config
|
||||
|
||||
The username, tenant-name and region entries have been filled in with the
|
||||
Keystone values of the user who created the cluster. Enter the password
|
||||
of this user on the entry for password::
|
||||
|
||||
password=ChangeMe
|
||||
|
||||
5. Restart Kubernetes services::
|
||||
|
||||
sudo systemctl restart kubelet
|
||||
|
||||
On restart, the new configuration enables the Kubernetes cloud provider
|
||||
plugin for OpenStack, along with the necessary credential for kubelet
|
||||
to authenticate with Keystone and to make request to OpenStack services.
|
||||
|
||||
6. Install nsenter::
|
||||
|
||||
sudo docker run -v /usr/local/bin:/target jpetazzo/nsenter
|
||||
|
||||
The nsenter utility is used by Kubernetes to run new processes within
|
||||
existing kernel namespaces. This allows the kubelet agent to manage storage
|
||||
for pods.
|
||||
|
||||
Kubernetes is now ready to use Cinder for persistent storage.
|
||||
Following is an example illustrating how Cinder is used in a pod.
|
||||
|
||||
|
@ -4,11 +4,6 @@
|
||||
|
||||
echo "configuring kubernetes (master)"
|
||||
|
||||
if [ -z "$KUBE_NODE_IP" ]; then
|
||||
# FIXME(yuanying): Set KUBE_NODE_IP correctly
|
||||
KUBE_NODE_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
|
||||
fi
|
||||
|
||||
sed -i '
|
||||
/^KUBE_ALLOW_PRIV=/ s/=.*/="--allow-privileged='"$KUBE_ALLOW_PRIV"'"/
|
||||
' /etc/kubernetes/config
|
||||
@ -31,6 +26,10 @@ if [ -n "${ADMISSION_CONTROL_LIST}" ] && [ "${TLS_DISABLED}" == "False" ]; then
|
||||
KUBE_ADMISSION_CONTROL="--admission-control=${ADMISSION_CONTROL_LIST}"
|
||||
fi
|
||||
|
||||
if [ -n "$TRUST_ID" ]; then
|
||||
KUBE_API_ARGS="$KUBE_API_ARGS --cloud-config=/etc/sysconfig/kube_openstack_config --cloud-provider=openstack"
|
||||
fi
|
||||
|
||||
sed -i '
|
||||
/^KUBE_API_ADDRESS=/ s/=.*/="'"${KUBE_API_ADDRESS}"'"/
|
||||
/^KUBE_SERVICE_ADDRESSES=/ s|=.*|="--service-cluster-ip-range='"$PORTAL_NETWORK_CIDR"'"|
|
||||
@ -39,10 +38,7 @@ sed -i '
|
||||
/^KUBE_ADMISSION_CONTROL=/ s/=.*/="'"${KUBE_ADMISSION_CONTROL}"'"/
|
||||
' /etc/kubernetes/apiserver
|
||||
cat << _EOC_ >> /etc/kubernetes/apiserver
|
||||
#Uncomment the following line to disable Load Balancer feature
|
||||
KUBE_API_ARGS="$KUBE_API_ARGS"
|
||||
#Uncomment the following line to enable Load Balancer feature
|
||||
#KUBE_API_ARGS="$KUBE_API_ARGS --cloud-config=/etc/sysconfig/kube_openstack_config --cloud-provider=openstack"
|
||||
_EOC_
|
||||
|
||||
# Add controller manager args
|
||||
@ -50,16 +46,18 @@ KUBE_CONTROLLER_MANAGER_ARGS=""
|
||||
if [ -n "${ADMISSION_CONTROL_LIST}" ] && [ "${TLS_DISABLED}" == "False" ]; then
|
||||
KUBE_CONTROLLER_MANAGER_ARGS="--service-account-private-key-file=/srv/kubernetes/server.key --root-ca-file=/srv/kubernetes/ca.crt"
|
||||
fi
|
||||
|
||||
if [ -n "$TRUST_ID" ]; then
|
||||
KUBE_CONTROLLER_MANAGER_ARGS="$KUBE_CONTROLLER_MANAGER_ARGS --cloud-config=/etc/sysconfig/kube_openstack_config --cloud-provider=openstack"
|
||||
fi
|
||||
|
||||
sed -i '
|
||||
/^KUBELET_ADDRESSES=/ s/=.*/="--machines='""'"/
|
||||
/^KUBE_CONTROLLER_MANAGER_ARGS=/ s#\(KUBE_CONTROLLER_MANAGER_ARGS\).*#\1="'"${KUBE_CONTROLLER_MANAGER_ARGS}"'"#
|
||||
' /etc/kubernetes/controller-manager
|
||||
cat << _EOC_ >> /etc/kubernetes/controller-manager
|
||||
#Uncomment the following line to enable Kubernetes Load Balancer feature
|
||||
#KUBE_CONTROLLER_MANAGER_ARGS="\$KUBE_CONTROLLER_MANAGER_ARGS --cloud-config=/etc/sysconfig/kube_openstack_config --cloud-provider=openstack"
|
||||
_EOC_
|
||||
|
||||
KUBELET_ARGS="--register-node=true --register-schedulable=false --config=/etc/kubernetes/manifests --hostname-override=$KUBE_NODE_IP"
|
||||
HOSTNAME_OVERRIDE=$(hostname --short | sed 's/\.novalocal//')
|
||||
KUBELET_ARGS="--register-node=true --register-schedulable=false --config=/etc/kubernetes/manifests --hostname-override=${HOSTNAME_OVERRIDE}"
|
||||
|
||||
if [ -n "${INSECURE_REGISTRY_URL}" ]; then
|
||||
KUBELET_ARGS="${KUBELET_ARGS} --pod-infra-container-image=${INSECURE_REGISTRY_URL}/google_containers/pause\:0.8.0"
|
||||
|
@ -4,11 +4,6 @@
|
||||
|
||||
echo "configuring kubernetes (minion)"
|
||||
|
||||
if [ -z "$KUBE_NODE_IP" ]; then
|
||||
# FIXME(yuanying): Set KUBE_NODE_IP correctly
|
||||
KUBE_NODE_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
|
||||
fi
|
||||
|
||||
CERT_DIR=/srv/kubernetes
|
||||
PROTOCOL=https
|
||||
FLANNEL_OPTIONS="-etcd-cafile $CERT_DIR/ca.crt \
|
||||
@ -52,8 +47,17 @@ sed -i '
|
||||
# The hostname of the node is set to be the Nova name of the instance, and
|
||||
# the option --hostname-override for kubelet uses the hostname to register the node.
|
||||
# Using any other name will break the load balancer and cinder volume features.
|
||||
HOSTNAME=$(hostname --short | sed 's/\.novalocal//')
|
||||
KUBELET_ARGS="--config=/etc/kubernetes/manifests --cadvisor-port=4194 ${KUBE_CONFIG} --hostname-override=${HOSTNAME}"
|
||||
HOSTNAME_OVERRIDE=$(hostname --short | sed 's/\.novalocal//')
|
||||
KUBELET_ARGS="--config=/etc/kubernetes/manifests --cadvisor-port=4194 ${KUBE_CONFIG} --hostname-override=${HOSTNAME_OVERRIDE}"
|
||||
|
||||
if [ -n "$TRUST_ID" ]; then
|
||||
KUBELET_ARGS="$KUBELET_ARGS --cloud-provider=openstack --cloud-config=/etc/sysconfig/kube_openstack_config"
|
||||
fi
|
||||
|
||||
# Workaround for Cinder support (fixed in k8s >= 1.6)
|
||||
if [ ! -f /usr/bin/udevadm ]; then
|
||||
ln -s /sbin/udevadm /usr/bin/udevadm
|
||||
fi
|
||||
|
||||
if [ -n "${INSECURE_REGISTRY_URL}" ]; then
|
||||
KUBELET_ARGS="${KUBELET_ARGS} --pod-infra-container-image=${INSECURE_REGISTRY_URL}/google_containers/pause\:0.8.0"
|
||||
@ -86,32 +90,6 @@ if [ "$NETWORK_DRIVER" = "flannel" ]; then
|
||||
done
|
||||
fi
|
||||
|
||||
if [ "$VOLUME_DRIVER" = "cinder" ]; then
|
||||
CLOUD_CONFIG=/etc/kubernetes/kube_openstack_config
|
||||
KUBERNETES=/etc/kubernetes
|
||||
if [ ! -d ${KUBERNETES} -o ! -f ${CLOUD_CONFIG} ]; then
|
||||
mkdir -p $KUBERNETES
|
||||
fi
|
||||
AUTH_URL=${AUTH_URL/v3/v2.0}
|
||||
cat > $CLOUD_CONFIG <<EOF
|
||||
[Global]
|
||||
auth-url=$AUTH_URL
|
||||
username=$USERNAME
|
||||
password=$PASSWORD
|
||||
region=$REGION_NAME
|
||||
tenant-name=$TENANT_NAME
|
||||
EOF
|
||||
|
||||
cat << _EOC_ >> /etc/kubernetes/kubelet
|
||||
#KUBELET_ARGS="$KUBELET_ARGS --cloud-provider=openstack --cloud-config=/etc/kubernetes/kube_openstack_config"
|
||||
_EOC_
|
||||
|
||||
if [ ! -f /usr/bin/udevadm ]; then
|
||||
ln -s /sbin/udevadm /usr/bin/udevadm
|
||||
fi
|
||||
|
||||
fi
|
||||
|
||||
cat >> /etc/environment <<EOF
|
||||
KUBERNETES_MASTER=$KUBE_MASTER_URI
|
||||
EOF
|
||||
|
@ -30,6 +30,7 @@ write_files:
|
||||
TLS_DISABLED="$TLS_DISABLED"
|
||||
CLUSTER_UUID="$CLUSTER_UUID"
|
||||
MAGNUM_URL="$MAGNUM_URL"
|
||||
VOLUME_DRIVER="$VOLUME_DRIVER"
|
||||
HTTP_PROXY="$HTTP_PROXY"
|
||||
HTTPS_PROXY="$HTTPS_PROXY"
|
||||
NO_PROXY="$NO_PROXY"
|
||||
|
@ -38,8 +38,6 @@ write_files:
|
||||
WAIT_CURL="$WAIT_CURL"
|
||||
KUBE_VERSION="$KUBE_VERSION"
|
||||
TRUSTEE_USER_ID="$TRUSTEE_USER_ID"
|
||||
TRUSTEE_USERNAME="$TRUSTEE_USERNAME"
|
||||
TRUSTEE_PASSWORD="$TRUSTEE_PASSWORD"
|
||||
TRUSTEE_DOMAIN_ID="$TRUSTEE_DOMAIN_ID"
|
||||
TRUST_ID="$TRUST_ID"
|
||||
INSECURE_REGISTRY_URL="$INSECURE_REGISTRY_URL"
|
||||
|
@ -4,17 +4,14 @@
|
||||
|
||||
KUBE_OS_CLOUD_CONFIG=/etc/sysconfig/kube_openstack_config
|
||||
|
||||
# kubernetes backend only support keystone v2 at this point
|
||||
AUTH_URL=$(echo "$AUTH_URL" | sed 's/v3/v2.0/')
|
||||
|
||||
# Generate a the configuration for Kubernetes services
|
||||
# to talk to OpenStack Neutron
|
||||
cat > $KUBE_OS_CLOUD_CONFIG <<EOF
|
||||
[Global]
|
||||
auth-url=$AUTH_URL
|
||||
username=$USERNAME
|
||||
password=$PASSWORD
|
||||
tenant-name=$TENANT_NAME
|
||||
user-id=$TRUSTEE_USER_ID
|
||||
password=$TRUSTEE_PASSWORD
|
||||
trust-id=$TRUST_ID
|
||||
[LoadBalancer]
|
||||
subnet-id=$CLUSTER_SUBNET
|
||||
create-monitor=yes
|
||||
|
@ -245,16 +245,9 @@ class BaseTemplateDefinition(TemplateDefinition):
|
||||
extra_params['trustee_username'] = cluster.trustee_username
|
||||
extra_params['trustee_password'] = cluster.trustee_password
|
||||
|
||||
# Only pass trust ID into the template when it is needed.
|
||||
if (cluster_template.volume_driver == 'rexray' or
|
||||
cluster_template.registry_enabled):
|
||||
# Only pass trust ID into the template if allowed by the config file
|
||||
if CONF.trust.cluster_user_trust:
|
||||
extra_params['trust_id'] = cluster.trust_id
|
||||
else:
|
||||
missing_setting = ('trust/cluster_user_trust = True')
|
||||
msg = ('This cluster can only be created with %s in '
|
||||
'magnum.conf')
|
||||
raise exception.ConfigInvalid(msg % missing_setting)
|
||||
else:
|
||||
extra_params['trust_id'] = ""
|
||||
|
||||
|
@ -453,6 +453,7 @@ resources:
|
||||
discovery_url: {get_param: discovery_url}
|
||||
cluster_uuid: {get_param: cluster_uuid}
|
||||
magnum_url: {get_param: magnum_url}
|
||||
volume_driver: {get_param: volume_driver}
|
||||
fixed_network: {get_attr: [network, fixed_network]}
|
||||
fixed_subnet: {get_attr: [network, fixed_subnet]}
|
||||
api_pool_id: {get_attr: [api_lb, pool_id]}
|
||||
|
@ -48,6 +48,10 @@ parameters:
|
||||
constraints:
|
||||
- allowed_values: ["devicemapper", "overlay"]
|
||||
|
||||
volume_driver:
|
||||
type: string
|
||||
description: volume driver to use for container storage
|
||||
|
||||
flannel_network_cidr:
|
||||
type: string
|
||||
description: network range for flannel overlay network
|
||||
@ -276,6 +280,7 @@ resources:
|
||||
"$TLS_DISABLED": {get_param: tls_disabled}
|
||||
"$CLUSTER_UUID": {get_param: cluster_uuid}
|
||||
"$MAGNUM_URL": {get_param: magnum_url}
|
||||
"$VOLUME_DRIVER": {get_param: volume_driver}
|
||||
"$HTTP_PROXY": {get_param: http_proxy}
|
||||
"$HTTPS_PROXY": {get_param: https_proxy}
|
||||
"$NO_PROXY": {get_param: no_proxy}
|
||||
|
@ -255,9 +255,7 @@ resources:
|
||||
$NO_PROXY: {get_param: no_proxy}
|
||||
$KUBE_VERSION: {get_param: kube_version}
|
||||
$WAIT_CURL: {get_attr: [minion_wait_handle, curl_cli]}
|
||||
$TRUSTEE_DOMAIN_ID: {get_param: trustee_domain_id}
|
||||
$TRUSTEE_USER_ID: {get_param: trustee_user_id}
|
||||
$TRUSTEE_USERNAME: {get_param: trustee_username}
|
||||
$TRUSTEE_PASSWORD: {get_param: trustee_password}
|
||||
$TRUST_ID: {get_param: trust_id}
|
||||
$AUTH_URL: {get_param: auth_url}
|
||||
@ -269,6 +267,12 @@ resources:
|
||||
group: ungrouped
|
||||
config: {get_file: ../../common/templates/kubernetes/fragments/write-kubeconfig.yaml}
|
||||
|
||||
write_kube_os_config:
|
||||
type: OS::Heat::SoftwareConfig
|
||||
properties:
|
||||
group: ungrouped
|
||||
config: {get_file: ../../common/templates/kubernetes/fragments/write-kube-os-config.sh}
|
||||
|
||||
make_cert:
|
||||
type: OS::Heat::SoftwareConfig
|
||||
properties:
|
||||
@ -364,6 +368,7 @@ resources:
|
||||
- config: {get_resource: disable_selinux}
|
||||
- config: {get_resource: write_heat_params}
|
||||
- config: {get_resource: write_kubeconfig}
|
||||
- config: {get_resource: write_kube_os_config}
|
||||
- config: {get_resource: make_cert}
|
||||
- config: {get_resource: kube_examples}
|
||||
- config: {get_resource: configure_docker_storage}
|
||||
|
@ -286,7 +286,7 @@ class TestClusterConductorWithSwarm(base.TestCase):
|
||||
'trustee_username': 'fake_trustee',
|
||||
'trustee_password': 'fake_trustee_password',
|
||||
'trustee_user_id': '7b489f04-b458-4541-8179-6a48a553e656',
|
||||
'trust_id': '',
|
||||
'trust_id': 'bd11efc5-d4e2-4dac-bbce-25e348ddf7de',
|
||||
'auth_url': 'http://192.168.10.10:5000/v3',
|
||||
'swarm_version': 'fake-version',
|
||||
'swarm_strategy': u'spread',
|
||||
|
Loading…
Reference in New Issue
Block a user