Improve the documentation

Change-Id: Ib63e487ea3f920a8a6bee22e1553accc13198d81
This commit is contained in:
Lingxian Kong 2018-10-31 01:15:52 +13:00
parent 2e0770dc56
commit ee7d2f338c
2 changed files with 131 additions and 89 deletions

View File

@ -12,64 +12,89 @@
License for the specific language governing permissions and limitations
under the License.
Config Qinling with existing Kubernetes cluster
===============================================
Config Qinling with existing kubernetes/etcd cluster
====================================================
In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for
Qinling. The component which works with Kubernetes cluster in Qinling is the
``qinling-engine``. Follow the steps below to configure Qinling to work with an
existing Kubernetes cluster, and make Qinling access the Kubernetes API with
authentication and authorization.
In most cases, it's not ideal to set up a new dedicated kubernetes cluster for
Qinling. The component which works with kubernetes cluster in Qinling is the
``qinling-engine``. Follow the steps below to configure Qinling to work with
existing kubernetes/etcd cluster, and make Qinling access the kubernetes/etcd
service with authentication and authorization.
Qinling Configurations
Prerequisites
~~~~~~~~~~~~~
* You know the kubernetes API address and etcd service address, for example:
.. code-block:: console
K8S_ADDRESS=10.0.0.5
ETCD_ADDRESS=10.0.0.6
.. end
* You have CA certificates of the kubernetes and etcd respectively and store on
the host that ``qinling-engine`` is running.
.. code-block:: console
K8S_CA_CERT=$HOME/ca.crt
K8S_CA_KEY=$HOME/ca.key
ETCD_CA_CERT=$HOME/etcd_ca.crt
ETCD_CA_KEY=$HOME/etcd_ca.key
.. end
* This guide assumes
`RBAC <https://kubernetes.io/docs/admin/authorization/rbac/>`_ is enabled in
the kubernetes cluster.
Qinling configurations
~~~~~~~~~~~~~~~~~~~~~~
Below are the options that relate to accessing the Kubernetes API in Qinling's
configuration file, all of them are under the ``kubernetes`` section.
Below are the options and their default values that relate to accessing the
Kubernetes API and etcd in Qinling's configuration file.
.. code-block:: ini
[kubernetes]
kube_host = http://127.0.0.1:8001
kube_host = https://127.0.0.1:8001
use_api_certificate = True
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
cert_file = /etc/qinling/pki/kubernetes/qinling.crt
key_file = /etc/qinling/pki/kubernetes/qinling.key
For now, just update the ``kube_host`` to the URI which the Kubernetes API
serves for HTTPS connections with authentication and authorization, for
example, ``kube_host = https://kube-api.example.com:6443``. We will cover the
other options in the following sections.
[etcd]
host = 127.0.0.1
port = 2379
protocol = https
ca_cert = /etc/qinling/pki/etcd/ca.crt
cert_file = /etc/qinling/pki/etcd/qinling-etcd-client.crt
cert_key = /etc/qinling/pki/etcd/qinling-etcd-client.key
Authentication and Authorization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. end
The access to the Kubernetes API is controlled by several modules, refer to
`Controlling Access to the Kubernetes API <https://kubernetes.io/docs/admin/accessing-the-api/>`_
for more details.
Change the kubernetes and etcd service addresses:
By default, Qinling engine is configured to access the Kubernetes API with
a client certificate for authentication(``use_api_certificate`` is set to
``True``), so make sure that the Kubernetes API server is running with the
``--client-ca-file=SOMEFILE`` option for client certificate authentication to
be enabled. The common name of the subject in the client certificate is used as
the user name for the requests that Qinling engine makes to the Kubernetes API
server. Refer to
`Authentication in Kubernetes <https://kubernetes.io/docs/admin/authentication/>`_.
.. code-block:: ini
If `RBAC Authorization <https://kubernetes.io/docs/admin/authorization/rbac/>`_
is enabled in the Kubernetes API, we will also have to grant access to
resources in Kubernetes for the specific user that Qinling uses to make
requests to the Kubernetes API. Using RBAC Authorization can ensure that
Qinling access the Kubernetes API with only the permission that it needs.
[kubernetes]
kube_host = https://${K8S_ADDRESS}:6443
...
[etcd]
host = ${ETCD_ADDRESS}
...
Generate Client Certificate for Qinling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. end
See `Managing Certificates <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
for how to generate a client cert. We use ``cfssl`` as the example here.
Generate and config client certificates for Qinling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#) Download and prepare the command line tools.
There are a lot of
`tools <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
out there for certificate generation. We use ``cfssl`` as the example here.
#) Download and prepare the command line tools as needed.
.. code-block:: console
@ -78,76 +103,75 @@ for how to generate a client cert. We use ``cfssl`` as the example here.
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
chmod +x /tmp/cfssljson
#) Generate the client ceritificate for Qinling. Note that the common name
of the subject is set to ``qinling`` in the example CSR located at
``example/kubernetes/cfssl-client-csr.json``.
.. end
#) Generate the kubernetes and etcd client certificates for Qinling.
.. code-block:: console
mkdir /tmp/certs; cd /tmp/certs
/tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt \
-ca-key=/path/to/kubernetes_ca_key \
-config=QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-ca-config.json \
mkdir -p /tmp/certs; cd /tmp/certs
curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-ca-config.json -o /tmp/certs/cfssl-ca-config.json
curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-client-csr.json -o /tmp/certs/cfssl-client-csr.json
/tmp/cfssl gencert -ca=${K8S_CA_CERT} \
-ca-key=${K8S_CA_KEY} \
-config=/tmp/certs/cfssl-ca-config.json \
-profile=client \
QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare k8s-client
/tmp/cfssl gencert -ca=${ETCD_CA_CERT} \
-ca-key=${ETCD_CA_KEY} \
-config=/tmp/certs/cfssl-ca-config.json \
-profile=client \
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare etcd-client
#) Copy the needed files to the locations. The command above generates two
files named ``client-key.pem`` and ``client.pem``, the former is the key
file of the client certificate, and the latter is the certificate file
itself.
.. end
.. note::
#) Move the certificates to the appropriate folders and ensure the qinling
service user has permission to those folders.
Remember to backup the existing files in /etc/qinling/pki/kubernetes
folder first.
.. code-block:: console
.. code-block:: console
mkdir -p /etc/qinling/pki/{kubernetes,etcd}
cp k8s-client-key.pem /etc/qinling/pki/kubernetes/qinling.key
cp k8s-client.pem /etc/qinling/pki/kubernetes/qinling.crt
cp etcd-client-key.pem /etc/qinling/pki/etcd/qinling-etcd-client.key
cp etcd-client.pem /etc/qinling/pki/etcd/qinling-etcd-client.crt
cp ${K8S_CA_CERT} /etc/qinling/pki/kubernetes/ca.crt
cp ${ETCD_CA_CERT} /etc/qinling/pki/etcd/ca.crt
chown -R qinling:qinling /etc/qinling/pki
mkdir -p /etc/qinling/pki/kubernetes
mv client-key.pem /etc/qinling/pki/kubernetes/qinling.key
mv client.pem /etc/qinling/pki/kubernetes/qinling.crt
mv /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt
.. note::
Make sure both ``/etc/qinling/pki/kubernetes`` and ``/etc/qinling/pki``
belong to Qinling service user. You can set the permissions with
``chown -R qinling:qinling /etc/qinling/pki``
.. end
Create Role and RoleBinding in Kubernetes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If RBAC Authorization is enabled, we can limit the permissions that Qinling
access the Kubernetes API. Before you procceed the steps in this section,
make sure that the Kubernetes API server is running with the
``--authorization-mode=RBAC`` option.
Qinling provides a single file located at
``example/kubernetes/k8s_qinling_role.yaml`` for users to
create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling
needs, and bind the roles to the user named ``qinling``, which is from
the common name of the subject in the client certificate. The role is defined
within a namespace named ``qinling``, which is the default namespace that
Qinling uses and the name is configurable.
According least privilege principle, the operation permission of qinling user
in kubernetes cluster should be limited, this could be easily achieved by
applying the pre-defined authorization manifest file.
.. code-block:: console
curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f -
.. end
Restart Qinling Engine
~~~~~~~~~~~~~~~~~~~~~~
Restart qinling-engine service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Restart the qinling-engine service after the steps above are done, and now
Qinling is accessing the Kubernetes API with itself authenticated by a client
certificate. The requests that Qinling makes to the Kubernetes API are also
authorized.
Restart the ``qinling-engine`` service after the steps above are done, and now
Qinling is accessing the Kubernetes API and etcd service using TLS. The
requests that Qinling makes to the Kubernetes API are also authorized.
.. code-block:: console
systemctl restart devstack@qinling-engine.service
.. end
Access the Kubernetes API Insecurely (For testing purpose ONLY)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
server serves for insecure connections. However this is not recommended and
server serves for insecure connections. However, this is not recommended and
should be used for testing purpose only.
In the configuration file, under the ``kubernetes`` section, set ``kube_host``
@ -155,3 +179,11 @@ to the URI which the Kubernetes API serves for insecure HTTP connections, for
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
to ``False`` to disable Qinling using a client certificate to access the
Kubernetes API.
.. code-block:: ini
[kubernetes]
kube_host = http://localhost:8080
use_api_certificate = False
.. end

View File

@ -15,7 +15,7 @@
Setting up a development environment with devstack
==================================================
This page describes how to setup a working development
This page describes how to set up a working development
environment that can be used in deploying qinling on latest releases
of Ubuntu. These instructions assume you are already familiar
with git. Refer to `Getting the code`_ for additional information.
@ -67,15 +67,14 @@ configuration:
LOG_COLOR=False
LOGDAYS=1
ENABLED_SERVICES=rabbit,mysql,key,tempest
ENABLED_SERVICES=rabbit,mysql,key
.. end
.. note::
Here are several things you could customize:
For multiple network cards, you need to update the Kubernetes apiserver's advertise address
to the address on the interface which is used to get to the default gateway by adding one
environment variable.
* For multiple network cards, you need to specify the kubernetes API server's
advertise address manually.
.. code-block:: console
@ -83,6 +82,17 @@ configuration:
.. end
* Devstack will set up a new kubernetes cluster and re-use etcd service inside
the cluster for Qinling services, which means you don't need to add etcd to
the enabled services list in the ``local.conf`` file.
* If you already have an existing kubernetes cluster, add
``QINLING_INSTALL_K8S=False`` to the ``local.conf`` file. Go to
`Config Qinling with existing Kubernetes cluster <https://docs.openstack.org/qinling/latest/admin/install/config_kubernetes.html>`_
for more details.
* If you want to interact with Qinling in Horizon, add
``enable_plugin qinling-dashboard https://git.openstack.org/openstack/qinling-dashboard``
in the ``local.conf`` file.
Running devstack
----------------