Document how to make Qinling work with existing k8s cluster

Change-Id: I8578939e164c2dfb6c672b5c4e2628ef58e115f4
This commit is contained in:
Hunt Xu 2018-05-04 17:25:12 +08:00
parent e22faa69fd
commit 61dc02f105
1 changed files with 131 additions and 3 deletions

View File

@ -24,6 +24,134 @@ Refer to
Config Qinling using existing Kubernetes cluster Config Qinling using existing Kubernetes cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In most cases, it's not ideal to set up a new dedicated kubernetes cluster for In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for
Qinling. Follow the steps below to config Qinling work with the existing Qinling. The component which works with Kubernetes cluster in Qinling is the
kubernetes cluster ``qinling-engine``. Follow the steps below to configure Qinling to work with an
existing Kubernetes cluster, and make Qinling access the Kubernetes API with
authentication and authorization.
Configurations
^^^^^^^^^^^^^^
Below are the options that relate to accessing the Kubernetes API in Qinling's
configuration file, all of them are under the ``kubernetes`` section.
.. code-block:: ini
[kubernetes]
kube_host = http://127.0.0.1:8001
use_api_certificate = True
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
cert_file = /etc/qinling/pki/kubernetes/qinling.crt
key_file = /etc/qinling/pki/kubernetes/qinling.key
For now, just update the ``kube_host`` to the URI which the Kubernetes API
serves for HTTPS connections with authentication and authorization, for
example, ``kube_host = https://kube-api.example.com:6443``. We will cover the
other options in the following sections.
Authentication and Authorization
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The access to the Kubernetes API is controlled by several modules, refer to
`Controlling Access to the Kubernetes API <https://kubernetes.io/docs/admin/accessing-the-api/>`_
for more details.
By default, Qinling engine is configured to access the Kubernetes API with
a client certificate for authentication(``use_api_certificate`` is set to
``True``), so make sure that the Kubernetes API server is running with the
``--client-ca-file=SOMEFILE`` option for client certificate authentication to
be enabled. The common name of the subject in the client certificate is used as
the user name for the requests that Qinling engine makes to the Kubernetes API
server. Refer to
`Authentication in Kubernetes <https://kubernetes.io/docs/admin/authentication/>`_.
If `RBAC Authorization <https://kubernetes.io/docs/admin/authorization/rbac/>`_
is enabled in the Kubernetes API, we will also have to grant access to resources
in Kubernetes for the specific user that Qinling uses to make requests to the
Kubernetes API. Using RBAC Authorization can ensure that Qinling access the
Kubernetes API with only the permission that it needs.
Generate Client Certificate for Qinling
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
See `Managing Certificates <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
for how to generate a client cert. We use ``cfssl`` as the example here.
#) Download and prepare the command line tools.
.. code-block:: console
curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /tmp/cfssl
chmod +x /tmp/cfssl
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
chmod +x /tmp/cfssljson
#) Generate the client ceritificate for Qinling. Note that the common name
of the subject is set to ``qinling`` in the example CSR located at
``QINLING_SOURCE/example/kubernetes/cfssl-client-csr.json``.
.. code-block:: console
mkdir cert
cd cert
/tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt -ca-key=/path/to/kubernetes_ca_key -config=QINLING_SOURCE/example/kubernetes/cfssl-ca-config.json -profile=client QINLING_SOURCE/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client
#) Copy the needed files to the locations. The command above generates two
files named client-key.pem and client.pem, the former is the key file of
the client certificate, and the latter is the certificate file itself.
.. code-block:: console
cp client-key.pem /etc/qinling/pki/kubernetes/qinling.key
cp client.pem /etc/qinling/pki/kubernetes/qinling.crt
cp /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt
Create Role and RoleBinding in Kubernetes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If RBAC Authorization is enabled, we can limit the permissions that Qinling
access the Kubernetes API. Before you procceed the steps in this section,
make sure that the Kubernetes API server is running with the
``--authorization-mode=RBAC`` option.
Qinling provides a single file located at
``QINLING_SOURCE/example/kubernetes/k8s_qinling_role.yaml`` for users to
create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling
needs, and bind the roles to the user named ``qinling``, which is from
the common name of the subject in the client certificate. The role is defined
within a namespace named ``qinling``, which is the default namespace that
Qinling uses and the name is configurable.
#) Grant access to the resources in the Kubernetes cluster for Qinling.
.. code-block:: console
kubectl create -f QINLING_SOURCE/example/kubernetes/k8s_qinling_role.yaml
The command above creates a ``ClusterRole`` named ``qinling`` with the
cluster-wide permissions that Qinling needs and binds it to the ``qinling``
user. It also creates a ``Role`` named ``qinling`` within a newly created
``qinling`` namespace and binds it to the specific user. So the access to
resources within that namespace is also granted.
Start Qinling Engine
^^^^^^^^^^^^^^^^^^^^
Start the qinling-engine service after the steps above are done. And now
Qinling is accessing the Kubernetes API with itself authenticated by a client
certificate. And the requests that Qinling makes to the Kubernetes API
are also authorized.
Access the Kubernetes API Insecurely (For Testing ONLY)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
server serves for insecure connections. However this is not recommended and
should be used for testing purpose only.
In the configuration file, under the ``kubernetes`` section, set ``kube_host``
to the URI which the Kubernetes API serves for insecure HTTP connections, for
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
to ``False`` to disable Qinling using a client certificate to access the
Kubernetes API.