diff --git a/doc/source/admin/install/config_kubernetes.rst b/doc/source/admin/install/config_kubernetes.rst index 4686c1d4..7c332628 100644 --- a/doc/source/admin/install/config_kubernetes.rst +++ b/doc/source/admin/install/config_kubernetes.rst @@ -12,64 +12,89 @@ License for the specific language governing permissions and limitations under the License. -Config Qinling with existing Kubernetes cluster -=============================================== +Config Qinling with existing kubernetes/etcd cluster +==================================================== -In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for -Qinling. The component which works with Kubernetes cluster in Qinling is the -``qinling-engine``. Follow the steps below to configure Qinling to work with an -existing Kubernetes cluster, and make Qinling access the Kubernetes API with -authentication and authorization. +In most cases, it's not ideal to set up a new dedicated kubernetes cluster for +Qinling. The component which works with kubernetes cluster in Qinling is the +``qinling-engine``. Follow the steps below to configure Qinling to work with +existing kubernetes/etcd cluster, and make Qinling access the kubernetes/etcd +service with authentication and authorization. -Qinling Configurations +Prerequisites +~~~~~~~~~~~~~ + +* You know the kubernetes API address and etcd service address, for example: + + .. code-block:: console + + K8S_ADDRESS=10.0.0.5 + ETCD_ADDRESS=10.0.0.6 + + .. end + +* You have CA certificates of the kubernetes and etcd respectively and store on + the host that ``qinling-engine`` is running. + + .. code-block:: console + + K8S_CA_CERT=$HOME/ca.crt + K8S_CA_KEY=$HOME/ca.key + ETCD_CA_CERT=$HOME/etcd_ca.crt + ETCD_CA_KEY=$HOME/etcd_ca.key + + .. end + +* This guide assumes + `RBAC `_ is enabled in + the kubernetes cluster. + +Qinling configurations ~~~~~~~~~~~~~~~~~~~~~~ -Below are the options that relate to accessing the Kubernetes API in Qinling's -configuration file, all of them are under the ``kubernetes`` section. +Below are the options and their default values that relate to accessing the +Kubernetes API and etcd in Qinling's configuration file. .. code-block:: ini [kubernetes] - kube_host = http://127.0.0.1:8001 + kube_host = https://127.0.0.1:8001 use_api_certificate = True ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt cert_file = /etc/qinling/pki/kubernetes/qinling.crt key_file = /etc/qinling/pki/kubernetes/qinling.key -For now, just update the ``kube_host`` to the URI which the Kubernetes API -serves for HTTPS connections with authentication and authorization, for -example, ``kube_host = https://kube-api.example.com:6443``. We will cover the -other options in the following sections. + [etcd] + host = 127.0.0.1 + port = 2379 + protocol = https + ca_cert = /etc/qinling/pki/etcd/ca.crt + cert_file = /etc/qinling/pki/etcd/qinling-etcd-client.crt + cert_key = /etc/qinling/pki/etcd/qinling-etcd-client.key -Authentication and Authorization -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. end -The access to the Kubernetes API is controlled by several modules, refer to -`Controlling Access to the Kubernetes API `_ -for more details. +Change the kubernetes and etcd service addresses: -By default, Qinling engine is configured to access the Kubernetes API with -a client certificate for authentication(``use_api_certificate`` is set to -``True``), so make sure that the Kubernetes API server is running with the -``--client-ca-file=SOMEFILE`` option for client certificate authentication to -be enabled. The common name of the subject in the client certificate is used as -the user name for the requests that Qinling engine makes to the Kubernetes API -server. Refer to -`Authentication in Kubernetes `_. +.. code-block:: ini -If `RBAC Authorization `_ -is enabled in the Kubernetes API, we will also have to grant access to -resources in Kubernetes for the specific user that Qinling uses to make -requests to the Kubernetes API. Using RBAC Authorization can ensure that -Qinling access the Kubernetes API with only the permission that it needs. + [kubernetes] + kube_host = https://${K8S_ADDRESS}:6443 + ... + [etcd] + host = ${ETCD_ADDRESS} + ... -Generate Client Certificate for Qinling -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. end -See `Managing Certificates `_ -for how to generate a client cert. We use ``cfssl`` as the example here. +Generate and config client certificates for Qinling +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -#) Download and prepare the command line tools. +There are a lot of +`tools `_ +out there for certificate generation. We use ``cfssl`` as the example here. + +#) Download and prepare the command line tools as needed. .. code-block:: console @@ -78,76 +103,75 @@ for how to generate a client cert. We use ``cfssl`` as the example here. curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson chmod +x /tmp/cfssljson -#) Generate the client ceritificate for Qinling. Note that the common name - of the subject is set to ``qinling`` in the example CSR located at - ``example/kubernetes/cfssl-client-csr.json``. + .. end + +#) Generate the kubernetes and etcd client certificates for Qinling. .. code-block:: console - mkdir /tmp/certs; cd /tmp/certs - /tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt \ - -ca-key=/path/to/kubernetes_ca_key \ - -config=QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-ca-config.json \ + mkdir -p /tmp/certs; cd /tmp/certs + curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-ca-config.json -o /tmp/certs/cfssl-ca-config.json + curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-client-csr.json -o /tmp/certs/cfssl-client-csr.json + /tmp/cfssl gencert -ca=${K8S_CA_CERT} \ + -ca-key=${K8S_CA_KEY} \ + -config=/tmp/certs/cfssl-ca-config.json \ -profile=client \ - QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client + /tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare k8s-client + /tmp/cfssl gencert -ca=${ETCD_CA_CERT} \ + -ca-key=${ETCD_CA_KEY} \ + -config=/tmp/certs/cfssl-ca-config.json \ + -profile=client \ + /tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare etcd-client -#) Copy the needed files to the locations. The command above generates two - files named ``client-key.pem`` and ``client.pem``, the former is the key - file of the client certificate, and the latter is the certificate file - itself. + .. end - .. note:: +#) Move the certificates to the appropriate folders and ensure the qinling + service user has permission to those folders. - Remember to backup the existing files in /etc/qinling/pki/kubernetes - folder first. + .. code-block:: console - .. code-block:: console + mkdir -p /etc/qinling/pki/{kubernetes,etcd} + cp k8s-client-key.pem /etc/qinling/pki/kubernetes/qinling.key + cp k8s-client.pem /etc/qinling/pki/kubernetes/qinling.crt + cp etcd-client-key.pem /etc/qinling/pki/etcd/qinling-etcd-client.key + cp etcd-client.pem /etc/qinling/pki/etcd/qinling-etcd-client.crt + cp ${K8S_CA_CERT} /etc/qinling/pki/kubernetes/ca.crt + cp ${ETCD_CA_CERT} /etc/qinling/pki/etcd/ca.crt + chown -R qinling:qinling /etc/qinling/pki - mkdir -p /etc/qinling/pki/kubernetes - mv client-key.pem /etc/qinling/pki/kubernetes/qinling.key - mv client.pem /etc/qinling/pki/kubernetes/qinling.crt - mv /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt - - .. note:: - - Make sure both ``/etc/qinling/pki/kubernetes`` and ``/etc/qinling/pki`` - belong to Qinling service user. You can set the permissions with - ``chown -R qinling:qinling /etc/qinling/pki`` + .. end Create Role and RoleBinding in Kubernetes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If RBAC Authorization is enabled, we can limit the permissions that Qinling -access the Kubernetes API. Before you procceed the steps in this section, -make sure that the Kubernetes API server is running with the -``--authorization-mode=RBAC`` option. - -Qinling provides a single file located at -``example/kubernetes/k8s_qinling_role.yaml`` for users to -create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling -needs, and bind the roles to the user named ``qinling``, which is from -the common name of the subject in the client certificate. The role is defined -within a namespace named ``qinling``, which is the default namespace that -Qinling uses and the name is configurable. +According least privilege principle, the operation permission of qinling user +in kubernetes cluster should be limited, this could be easily achieved by +applying the pre-defined authorization manifest file. .. code-block:: console curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f - +.. end -Restart Qinling Engine -~~~~~~~~~~~~~~~~~~~~~~ +Restart qinling-engine service +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Restart the qinling-engine service after the steps above are done, and now -Qinling is accessing the Kubernetes API with itself authenticated by a client -certificate. The requests that Qinling makes to the Kubernetes API are also -authorized. +Restart the ``qinling-engine`` service after the steps above are done, and now +Qinling is accessing the Kubernetes API and etcd service using TLS. The +requests that Qinling makes to the Kubernetes API are also authorized. + +.. code-block:: console + + systemctl restart devstack@qinling-engine.service + +.. end Access the Kubernetes API Insecurely (For testing purpose ONLY) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API -server serves for insecure connections. However this is not recommended and +server serves for insecure connections. However, this is not recommended and should be used for testing purpose only. In the configuration file, under the ``kubernetes`` section, set ``kube_host`` @@ -155,3 +179,11 @@ to the URI which the Kubernetes API serves for insecure HTTP connections, for example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate`` to ``False`` to disable Qinling using a client certificate to access the Kubernetes API. + +.. code-block:: ini + + [kubernetes] + kube_host = http://localhost:8080 + use_api_certificate = False + +.. end diff --git a/doc/source/contributor/development-environment-devstack.rst b/doc/source/contributor/development-environment-devstack.rst index a8f7e19f..6cb3de6c 100644 --- a/doc/source/contributor/development-environment-devstack.rst +++ b/doc/source/contributor/development-environment-devstack.rst @@ -15,7 +15,7 @@ Setting up a development environment with devstack ================================================== -This page describes how to setup a working development +This page describes how to set up a working development environment that can be used in deploying qinling on latest releases of Ubuntu. These instructions assume you are already familiar with git. Refer to `Getting the code`_ for additional information. @@ -67,15 +67,14 @@ configuration: LOG_COLOR=False LOGDAYS=1 - ENABLED_SERVICES=rabbit,mysql,key,tempest + ENABLED_SERVICES=rabbit,mysql,key .. end -.. note:: +Here are several things you could customize: - For multiple network cards, you need to update the Kubernetes apiserver's advertise address - to the address on the interface which is used to get to the default gateway by adding one - environment variable. +* For multiple network cards, you need to specify the kubernetes API server's + advertise address manually. .. code-block:: console @@ -83,6 +82,17 @@ configuration: .. end +* Devstack will set up a new kubernetes cluster and re-use etcd service inside + the cluster for Qinling services, which means you don't need to add etcd to + the enabled services list in the ``local.conf`` file. +* If you already have an existing kubernetes cluster, add + ``QINLING_INSTALL_K8S=False`` to the ``local.conf`` file. Go to + `Config Qinling with existing Kubernetes cluster `_ + for more details. +* If you want to interact with Qinling in Horizon, add + ``enable_plugin qinling-dashboard https://git.openstack.org/openstack/qinling-dashboard`` + in the ``local.conf`` file. + Running devstack ----------------