Improve the documentation
Change-Id: Ib63e487ea3f920a8a6bee22e1553accc13198d81
This commit is contained in:
Lingxian Kong
parent
2e0770dc56
commit
ee7d2f338c
@ -12,64 +12,89 @@
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
Config Qinling with existing Kubernetes cluster
|
||||
===============================================
|
||||
Config Qinling with existing kubernetes/etcd cluster
|
||||
====================================================
|
||||
|
||||
In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for
|
||||
Qinling. The component which works with Kubernetes cluster in Qinling is the
|
||||
``qinling-engine``. Follow the steps below to configure Qinling to work with an
|
||||
existing Kubernetes cluster, and make Qinling access the Kubernetes API with
|
||||
authentication and authorization.
|
||||
In most cases, it's not ideal to set up a new dedicated kubernetes cluster for
|
||||
Qinling. The component which works with kubernetes cluster in Qinling is the
|
||||
``qinling-engine``. Follow the steps below to configure Qinling to work with
|
||||
existing kubernetes/etcd cluster, and make Qinling access the kubernetes/etcd
|
||||
service with authentication and authorization.
|
||||
|
||||
Qinling Configurations
|
||||
Prerequisites
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
* You know the kubernetes API address and etcd service address, for example:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
K8S_ADDRESS=10.0.0.5
|
||||
ETCD_ADDRESS=10.0.0.6
|
||||
|
||||
.. end
|
||||
|
||||
* You have CA certificates of the kubernetes and etcd respectively and store on
|
||||
the host that ``qinling-engine`` is running.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
K8S_CA_CERT=$HOME/ca.crt
|
||||
K8S_CA_KEY=$HOME/ca.key
|
||||
ETCD_CA_CERT=$HOME/etcd_ca.crt
|
||||
ETCD_CA_KEY=$HOME/etcd_ca.key
|
||||
|
||||
.. end
|
||||
|
||||
* This guide assumes
|
||||
`RBAC <https://kubernetes.io/docs/admin/authorization/rbac/>`_ is enabled in
|
||||
the kubernetes cluster.
|
||||
|
||||
Qinling configurations
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Below are the options that relate to accessing the Kubernetes API in Qinling's
|
||||
configuration file, all of them are under the ``kubernetes`` section.
|
||||
Below are the options and their default values that relate to accessing the
|
||||
Kubernetes API and etcd in Qinling's configuration file.
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[kubernetes]
|
||||
kube_host = http://127.0.0.1:8001
|
||||
kube_host = https://127.0.0.1:8001
|
||||
use_api_certificate = True
|
||||
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
|
||||
cert_file = /etc/qinling/pki/kubernetes/qinling.crt
|
||||
key_file = /etc/qinling/pki/kubernetes/qinling.key
|
||||
|
||||
For now, just update the ``kube_host`` to the URI which the Kubernetes API
|
||||
serves for HTTPS connections with authentication and authorization, for
|
||||
example, ``kube_host = https://kube-api.example.com:6443``. We will cover the
|
||||
other options in the following sections.
|
||||
[etcd]
|
||||
host = 127.0.0.1
|
||||
port = 2379
|
||||
protocol = https
|
||||
ca_cert = /etc/qinling/pki/etcd/ca.crt
|
||||
cert_file = /etc/qinling/pki/etcd/qinling-etcd-client.crt
|
||||
cert_key = /etc/qinling/pki/etcd/qinling-etcd-client.key
|
||||
|
||||
Authentication and Authorization
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
.. end
|
||||
|
||||
The access to the Kubernetes API is controlled by several modules, refer to
|
||||
`Controlling Access to the Kubernetes API <https://kubernetes.io/docs/admin/accessing-the-api/>`_
|
||||
for more details.
|
||||
Change the kubernetes and etcd service addresses:
|
||||
|
||||
By default, Qinling engine is configured to access the Kubernetes API with
|
||||
a client certificate for authentication(``use_api_certificate`` is set to
|
||||
``True``), so make sure that the Kubernetes API server is running with the
|
||||
``--client-ca-file=SOMEFILE`` option for client certificate authentication to
|
||||
be enabled. The common name of the subject in the client certificate is used as
|
||||
the user name for the requests that Qinling engine makes to the Kubernetes API
|
||||
server. Refer to
|
||||
`Authentication in Kubernetes <https://kubernetes.io/docs/admin/authentication/>`_.
|
||||
.. code-block:: ini
|
||||
|
||||
If `RBAC Authorization <https://kubernetes.io/docs/admin/authorization/rbac/>`_
|
||||
is enabled in the Kubernetes API, we will also have to grant access to
|
||||
resources in Kubernetes for the specific user that Qinling uses to make
|
||||
requests to the Kubernetes API. Using RBAC Authorization can ensure that
|
||||
Qinling access the Kubernetes API with only the permission that it needs.
|
||||
[kubernetes]
|
||||
kube_host = https://${K8S_ADDRESS}:6443
|
||||
...
|
||||
[etcd]
|
||||
host = ${ETCD_ADDRESS}
|
||||
...
|
||||
|
||||
Generate Client Certificate for Qinling
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
.. end
|
||||
|
||||
See `Managing Certificates <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
|
||||
for how to generate a client cert. We use ``cfssl`` as the example here.
|
||||
Generate and config client certificates for Qinling
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
#) Download and prepare the command line tools.
|
||||
There are a lot of
|
||||
`tools <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
|
||||
out there for certificate generation. We use ``cfssl`` as the example here.
|
||||
|
||||
#) Download and prepare the command line tools as needed.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -78,76 +103,75 @@ for how to generate a client cert. We use ``cfssl`` as the example here.
|
||||
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
|
||||
chmod +x /tmp/cfssljson
|
||||
|
||||
#) Generate the client ceritificate for Qinling. Note that the common name
|
||||
of the subject is set to ``qinling`` in the example CSR located at
|
||||
``example/kubernetes/cfssl-client-csr.json``.
|
||||
.. end
|
||||
|
||||
#) Generate the kubernetes and etcd client certificates for Qinling.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
mkdir /tmp/certs; cd /tmp/certs
|
||||
/tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt \
|
||||
-ca-key=/path/to/kubernetes_ca_key \
|
||||
-config=QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-ca-config.json \
|
||||
mkdir -p /tmp/certs; cd /tmp/certs
|
||||
curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-ca-config.json -o /tmp/certs/cfssl-ca-config.json
|
||||
curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-client-csr.json -o /tmp/certs/cfssl-client-csr.json
|
||||
/tmp/cfssl gencert -ca=${K8S_CA_CERT} \
|
||||
-ca-key=${K8S_CA_KEY} \
|
||||
-config=/tmp/certs/cfssl-ca-config.json \
|
||||
-profile=client \
|
||||
QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client
|
||||
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare k8s-client
|
||||
/tmp/cfssl gencert -ca=${ETCD_CA_CERT} \
|
||||
-ca-key=${ETCD_CA_KEY} \
|
||||
-config=/tmp/certs/cfssl-ca-config.json \
|
||||
-profile=client \
|
||||
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare etcd-client
|
||||
|
||||
#) Copy the needed files to the locations. The command above generates two
|
||||
files named ``client-key.pem`` and ``client.pem``, the former is the key
|
||||
file of the client certificate, and the latter is the certificate file
|
||||
itself.
|
||||
.. end
|
||||
|
||||
.. note::
|
||||
#) Move the certificates to the appropriate folders and ensure the qinling
|
||||
service user has permission to those folders.
|
||||
|
||||
Remember to backup the existing files in /etc/qinling/pki/kubernetes
|
||||
folder first.
|
||||
.. code-block:: console
|
||||
|
||||
.. code-block:: console
|
||||
mkdir -p /etc/qinling/pki/{kubernetes,etcd}
|
||||
cp k8s-client-key.pem /etc/qinling/pki/kubernetes/qinling.key
|
||||
cp k8s-client.pem /etc/qinling/pki/kubernetes/qinling.crt
|
||||
cp etcd-client-key.pem /etc/qinling/pki/etcd/qinling-etcd-client.key
|
||||
cp etcd-client.pem /etc/qinling/pki/etcd/qinling-etcd-client.crt
|
||||
cp ${K8S_CA_CERT} /etc/qinling/pki/kubernetes/ca.crt
|
||||
cp ${ETCD_CA_CERT} /etc/qinling/pki/etcd/ca.crt
|
||||
chown -R qinling:qinling /etc/qinling/pki
|
||||
|
||||
mkdir -p /etc/qinling/pki/kubernetes
|
||||
mv client-key.pem /etc/qinling/pki/kubernetes/qinling.key
|
||||
mv client.pem /etc/qinling/pki/kubernetes/qinling.crt
|
||||
mv /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt
|
||||
|
||||
.. note::
|
||||
|
||||
Make sure both ``/etc/qinling/pki/kubernetes`` and ``/etc/qinling/pki``
|
||||
belong to Qinling service user. You can set the permissions with
|
||||
``chown -R qinling:qinling /etc/qinling/pki``
|
||||
.. end
|
||||
|
||||
Create Role and RoleBinding in Kubernetes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If RBAC Authorization is enabled, we can limit the permissions that Qinling
|
||||
access the Kubernetes API. Before you procceed the steps in this section,
|
||||
make sure that the Kubernetes API server is running with the
|
||||
``--authorization-mode=RBAC`` option.
|
||||
|
||||
Qinling provides a single file located at
|
||||
``example/kubernetes/k8s_qinling_role.yaml`` for users to
|
||||
create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling
|
||||
needs, and bind the roles to the user named ``qinling``, which is from
|
||||
the common name of the subject in the client certificate. The role is defined
|
||||
within a namespace named ``qinling``, which is the default namespace that
|
||||
Qinling uses and the name is configurable.
|
||||
According least privilege principle, the operation permission of qinling user
|
||||
in kubernetes cluster should be limited, this could be easily achieved by
|
||||
applying the pre-defined authorization manifest file.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f -
|
||||
|
||||
.. end
|
||||
|
||||
Restart Qinling Engine
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
Restart qinling-engine service
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Restart the qinling-engine service after the steps above are done, and now
|
||||
Qinling is accessing the Kubernetes API with itself authenticated by a client
|
||||
certificate. The requests that Qinling makes to the Kubernetes API are also
|
||||
authorized.
|
||||
Restart the ``qinling-engine`` service after the steps above are done, and now
|
||||
Qinling is accessing the Kubernetes API and etcd service using TLS. The
|
||||
requests that Qinling makes to the Kubernetes API are also authorized.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
systemctl restart devstack@qinling-engine.service
|
||||
|
||||
.. end
|
||||
|
||||
Access the Kubernetes API Insecurely (For testing purpose ONLY)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
|
||||
server serves for insecure connections. However this is not recommended and
|
||||
server serves for insecure connections. However, this is not recommended and
|
||||
should be used for testing purpose only.
|
||||
|
||||
In the configuration file, under the ``kubernetes`` section, set ``kube_host``
|
||||
@ -155,3 +179,11 @@ to the URI which the Kubernetes API serves for insecure HTTP connections, for
|
||||
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
|
||||
to ``False`` to disable Qinling using a client certificate to access the
|
||||
Kubernetes API.
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[kubernetes]
|
||||
kube_host = http://localhost:8080
|
||||
use_api_certificate = False
|
||||
|
||||
.. end
|
||||
|
||||
@ -15,7 +15,7 @@
|
||||
Setting up a development environment with devstack
|
||||
==================================================
|
||||
|
||||
This page describes how to setup a working development
|
||||
This page describes how to set up a working development
|
||||
environment that can be used in deploying qinling on latest releases
|
||||
of Ubuntu. These instructions assume you are already familiar
|
||||
with git. Refer to `Getting the code`_ for additional information.
|
||||
@ -67,15 +67,14 @@ configuration:
|
||||
LOG_COLOR=False
|
||||
LOGDAYS=1
|
||||
|
||||
ENABLED_SERVICES=rabbit,mysql,key,tempest
|
||||
ENABLED_SERVICES=rabbit,mysql,key
|
||||
|
||||
.. end
|
||||
|
||||
.. note::
|
||||
Here are several things you could customize:
|
||||
|
||||
For multiple network cards, you need to update the Kubernetes apiserver's advertise address
|
||||
to the address on the interface which is used to get to the default gateway by adding one
|
||||
environment variable.
|
||||
* For multiple network cards, you need to specify the kubernetes API server's
|
||||
advertise address manually.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -83,6 +82,17 @@ configuration:
|
||||
|
||||
.. end
|
||||
|
||||
* Devstack will set up a new kubernetes cluster and re-use etcd service inside
|
||||
the cluster for Qinling services, which means you don't need to add etcd to
|
||||
the enabled services list in the ``local.conf`` file.
|
||||
* If you already have an existing kubernetes cluster, add
|
||||
``QINLING_INSTALL_K8S=False`` to the ``local.conf`` file. Go to
|
||||
`Config Qinling with existing Kubernetes cluster <https://docs.openstack.org/qinling/latest/admin/install/config_kubernetes.html>`_
|
||||
for more details.
|
||||
* If you want to interact with Qinling in Horizon, add
|
||||
``enable_plugin qinling-dashboard https://git.openstack.org/openstack/qinling-dashboard``
|
||||
in the ``local.conf`` file.
|
||||
|
||||
Running devstack
|
||||
----------------
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user