Document how to make Qinling work with existing k8s cluster
Change-Id: I8578939e164c2dfb6c672b5c4e2628ef58e115f4
This commit is contained in:
Hunt Xu
parent
e22faa69fd
commit
61dc02f105
@ -24,6 +24,134 @@ Refer to
|
||||
Config Qinling using existing Kubernetes cluster
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In most cases, it's not ideal to set up a new dedicated kubernetes cluster for
|
||||
Qinling. Follow the steps below to config Qinling work with the existing
|
||||
kubernetes cluster
|
||||
In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for
|
||||
Qinling. The component which works with Kubernetes cluster in Qinling is the
|
||||
``qinling-engine``. Follow the steps below to configure Qinling to work with an
|
||||
existing Kubernetes cluster, and make Qinling access the Kubernetes API with
|
||||
authentication and authorization.
|
||||
|
||||
Configurations
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
Below are the options that relate to accessing the Kubernetes API in Qinling's
|
||||
configuration file, all of them are under the ``kubernetes`` section.
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[kubernetes]
|
||||
kube_host = http://127.0.0.1:8001
|
||||
use_api_certificate = True
|
||||
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
|
||||
cert_file = /etc/qinling/pki/kubernetes/qinling.crt
|
||||
key_file = /etc/qinling/pki/kubernetes/qinling.key
|
||||
|
||||
For now, just update the ``kube_host`` to the URI which the Kubernetes API
|
||||
serves for HTTPS connections with authentication and authorization, for
|
||||
example, ``kube_host = https://kube-api.example.com:6443``. We will cover the
|
||||
other options in the following sections.
|
||||
|
||||
Authentication and Authorization
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The access to the Kubernetes API is controlled by several modules, refer to
|
||||
`Controlling Access to the Kubernetes API <https://kubernetes.io/docs/admin/accessing-the-api/>`_
|
||||
for more details.
|
||||
|
||||
By default, Qinling engine is configured to access the Kubernetes API with
|
||||
a client certificate for authentication(``use_api_certificate`` is set to
|
||||
``True``), so make sure that the Kubernetes API server is running with the
|
||||
``--client-ca-file=SOMEFILE`` option for client certificate authentication to
|
||||
be enabled. The common name of the subject in the client certificate is used as
|
||||
the user name for the requests that Qinling engine makes to the Kubernetes API
|
||||
server. Refer to
|
||||
`Authentication in Kubernetes <https://kubernetes.io/docs/admin/authentication/>`_.
|
||||
|
||||
If `RBAC Authorization <https://kubernetes.io/docs/admin/authorization/rbac/>`_
|
||||
is enabled in the Kubernetes API, we will also have to grant access to resources
|
||||
in Kubernetes for the specific user that Qinling uses to make requests to the
|
||||
Kubernetes API. Using RBAC Authorization can ensure that Qinling access the
|
||||
Kubernetes API with only the permission that it needs.
|
||||
|
||||
Generate Client Certificate for Qinling
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
See `Managing Certificates <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
|
||||
for how to generate a client cert. We use ``cfssl`` as the example here.
|
||||
|
||||
#) Download and prepare the command line tools.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /tmp/cfssl
|
||||
chmod +x /tmp/cfssl
|
||||
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
|
||||
chmod +x /tmp/cfssljson
|
||||
|
||||
#) Generate the client ceritificate for Qinling. Note that the common name
|
||||
of the subject is set to ``qinling`` in the example CSR located at
|
||||
``QINLING_SOURCE/example/kubernetes/cfssl-client-csr.json``.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
mkdir cert
|
||||
cd cert
|
||||
/tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt -ca-key=/path/to/kubernetes_ca_key -config=QINLING_SOURCE/example/kubernetes/cfssl-ca-config.json -profile=client QINLING_SOURCE/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client
|
||||
|
||||
#) Copy the needed files to the locations. The command above generates two
|
||||
files named client-key.pem and client.pem, the former is the key file of
|
||||
the client certificate, and the latter is the certificate file itself.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
cp client-key.pem /etc/qinling/pki/kubernetes/qinling.key
|
||||
cp client.pem /etc/qinling/pki/kubernetes/qinling.crt
|
||||
cp /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt
|
||||
|
||||
Create Role and RoleBinding in Kubernetes
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If RBAC Authorization is enabled, we can limit the permissions that Qinling
|
||||
access the Kubernetes API. Before you procceed the steps in this section,
|
||||
make sure that the Kubernetes API server is running with the
|
||||
``--authorization-mode=RBAC`` option.
|
||||
|
||||
Qinling provides a single file located at
|
||||
``QINLING_SOURCE/example/kubernetes/k8s_qinling_role.yaml`` for users to
|
||||
create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling
|
||||
needs, and bind the roles to the user named ``qinling``, which is from
|
||||
the common name of the subject in the client certificate. The role is defined
|
||||
within a namespace named ``qinling``, which is the default namespace that
|
||||
Qinling uses and the name is configurable.
|
||||
|
||||
#) Grant access to the resources in the Kubernetes cluster for Qinling.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
kubectl create -f QINLING_SOURCE/example/kubernetes/k8s_qinling_role.yaml
|
||||
|
||||
The command above creates a ``ClusterRole`` named ``qinling`` with the
|
||||
cluster-wide permissions that Qinling needs and binds it to the ``qinling``
|
||||
user. It also creates a ``Role`` named ``qinling`` within a newly created
|
||||
``qinling`` namespace and binds it to the specific user. So the access to
|
||||
resources within that namespace is also granted.
|
||||
|
||||
Start Qinling Engine
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Start the qinling-engine service after the steps above are done. And now
|
||||
Qinling is accessing the Kubernetes API with itself authenticated by a client
|
||||
certificate. And the requests that Qinling makes to the Kubernetes API
|
||||
are also authorized.
|
||||
|
||||
Access the Kubernetes API Insecurely (For Testing ONLY)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
|
||||
server serves for insecure connections. However this is not recommended and
|
||||
should be used for testing purpose only.
|
||||
|
||||
In the configuration file, under the ``kubernetes`` section, set ``kube_host``
|
||||
to the URI which the Kubernetes API serves for insecure HTTP connections, for
|
||||
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
|
||||
to ``False`` to disable Qinling using a client certificate to access the
|
||||
Kubernetes API.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user