Merge "Improve the documentation"

This commit is contained in:
Zuul 2018-11-01 01:36:05 +00:00 committed by Gerrit Code Review
commit 448cf1c90a
2 changed files with 131 additions and 89 deletions

View File

@ -12,64 +12,89 @@
License for the specific language governing permissions and limitations License for the specific language governing permissions and limitations
under the License. under the License.
Config Qinling with existing Kubernetes cluster Config Qinling with existing kubernetes/etcd cluster
=============================================== ====================================================
In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for In most cases, it's not ideal to set up a new dedicated kubernetes cluster for
Qinling. The component which works with Kubernetes cluster in Qinling is the Qinling. The component which works with kubernetes cluster in Qinling is the
``qinling-engine``. Follow the steps below to configure Qinling to work with an ``qinling-engine``. Follow the steps below to configure Qinling to work with
existing Kubernetes cluster, and make Qinling access the Kubernetes API with existing kubernetes/etcd cluster, and make Qinling access the kubernetes/etcd
authentication and authorization. service with authentication and authorization.
Qinling Configurations Prerequisites
~~~~~~~~~~~~~
* You know the kubernetes API address and etcd service address, for example:
.. code-block:: console
K8S_ADDRESS=10.0.0.5
ETCD_ADDRESS=10.0.0.6
.. end
* You have CA certificates of the kubernetes and etcd respectively and store on
the host that ``qinling-engine`` is running.
.. code-block:: console
K8S_CA_CERT=$HOME/ca.crt
K8S_CA_KEY=$HOME/ca.key
ETCD_CA_CERT=$HOME/etcd_ca.crt
ETCD_CA_KEY=$HOME/etcd_ca.key
.. end
* This guide assumes
`RBAC <https://kubernetes.io/docs/admin/authorization/rbac/>`_ is enabled in
the kubernetes cluster.
Qinling configurations
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
Below are the options that relate to accessing the Kubernetes API in Qinling's Below are the options and their default values that relate to accessing the
configuration file, all of them are under the ``kubernetes`` section. Kubernetes API and etcd in Qinling's configuration file.
.. code-block:: ini .. code-block:: ini
[kubernetes] [kubernetes]
kube_host = http://127.0.0.1:8001 kube_host = https://127.0.0.1:8001
use_api_certificate = True use_api_certificate = True
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
cert_file = /etc/qinling/pki/kubernetes/qinling.crt cert_file = /etc/qinling/pki/kubernetes/qinling.crt
key_file = /etc/qinling/pki/kubernetes/qinling.key key_file = /etc/qinling/pki/kubernetes/qinling.key
For now, just update the ``kube_host`` to the URI which the Kubernetes API [etcd]
serves for HTTPS connections with authentication and authorization, for host = 127.0.0.1
example, ``kube_host = https://kube-api.example.com:6443``. We will cover the port = 2379
other options in the following sections. protocol = https
ca_cert = /etc/qinling/pki/etcd/ca.crt
cert_file = /etc/qinling/pki/etcd/qinling-etcd-client.crt
cert_key = /etc/qinling/pki/etcd/qinling-etcd-client.key
Authentication and Authorization .. end
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The access to the Kubernetes API is controlled by several modules, refer to Change the kubernetes and etcd service addresses:
`Controlling Access to the Kubernetes API <https://kubernetes.io/docs/admin/accessing-the-api/>`_
for more details.
By default, Qinling engine is configured to access the Kubernetes API with .. code-block:: ini
a client certificate for authentication(``use_api_certificate`` is set to
``True``), so make sure that the Kubernetes API server is running with the
``--client-ca-file=SOMEFILE`` option for client certificate authentication to
be enabled. The common name of the subject in the client certificate is used as
the user name for the requests that Qinling engine makes to the Kubernetes API
server. Refer to
`Authentication in Kubernetes <https://kubernetes.io/docs/admin/authentication/>`_.
If `RBAC Authorization <https://kubernetes.io/docs/admin/authorization/rbac/>`_ [kubernetes]
is enabled in the Kubernetes API, we will also have to grant access to kube_host = https://${K8S_ADDRESS}:6443
resources in Kubernetes for the specific user that Qinling uses to make ...
requests to the Kubernetes API. Using RBAC Authorization can ensure that [etcd]
Qinling access the Kubernetes API with only the permission that it needs. host = ${ETCD_ADDRESS}
...
Generate Client Certificate for Qinling .. end
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See `Managing Certificates <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_ Generate and config client certificates for Qinling
for how to generate a client cert. We use ``cfssl`` as the example here. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#) Download and prepare the command line tools. There are a lot of
`tools <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
out there for certificate generation. We use ``cfssl`` as the example here.
#) Download and prepare the command line tools as needed.
.. code-block:: console .. code-block:: console
@ -78,76 +103,75 @@ for how to generate a client cert. We use ``cfssl`` as the example here.
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
chmod +x /tmp/cfssljson chmod +x /tmp/cfssljson
#) Generate the client ceritificate for Qinling. Note that the common name .. end
of the subject is set to ``qinling`` in the example CSR located at
``example/kubernetes/cfssl-client-csr.json``. #) Generate the kubernetes and etcd client certificates for Qinling.
.. code-block:: console .. code-block:: console
mkdir /tmp/certs; cd /tmp/certs mkdir -p /tmp/certs; cd /tmp/certs
/tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt \ curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-ca-config.json -o /tmp/certs/cfssl-ca-config.json
-ca-key=/path/to/kubernetes_ca_key \ curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-client-csr.json -o /tmp/certs/cfssl-client-csr.json
-config=QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-ca-config.json \ /tmp/cfssl gencert -ca=${K8S_CA_CERT} \
-ca-key=${K8S_CA_KEY} \
-config=/tmp/certs/cfssl-ca-config.json \
-profile=client \ -profile=client \
QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client /tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare k8s-client
/tmp/cfssl gencert -ca=${ETCD_CA_CERT} \
-ca-key=${ETCD_CA_KEY} \
-config=/tmp/certs/cfssl-ca-config.json \
-profile=client \
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare etcd-client
#) Copy the needed files to the locations. The command above generates two .. end
files named ``client-key.pem`` and ``client.pem``, the former is the key
file of the client certificate, and the latter is the certificate file
itself.
.. note:: #) Move the certificates to the appropriate folders and ensure the qinling
service user has permission to those folders.
Remember to backup the existing files in /etc/qinling/pki/kubernetes .. code-block:: console
folder first.
.. code-block:: console mkdir -p /etc/qinling/pki/{kubernetes,etcd}
cp k8s-client-key.pem /etc/qinling/pki/kubernetes/qinling.key
cp k8s-client.pem /etc/qinling/pki/kubernetes/qinling.crt
cp etcd-client-key.pem /etc/qinling/pki/etcd/qinling-etcd-client.key
cp etcd-client.pem /etc/qinling/pki/etcd/qinling-etcd-client.crt
cp ${K8S_CA_CERT} /etc/qinling/pki/kubernetes/ca.crt
cp ${ETCD_CA_CERT} /etc/qinling/pki/etcd/ca.crt
chown -R qinling:qinling /etc/qinling/pki
mkdir -p /etc/qinling/pki/kubernetes .. end
mv client-key.pem /etc/qinling/pki/kubernetes/qinling.key
mv client.pem /etc/qinling/pki/kubernetes/qinling.crt
mv /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt
.. note::
Make sure both ``/etc/qinling/pki/kubernetes`` and ``/etc/qinling/pki``
belong to Qinling service user. You can set the permissions with
``chown -R qinling:qinling /etc/qinling/pki``
Create Role and RoleBinding in Kubernetes Create Role and RoleBinding in Kubernetes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If RBAC Authorization is enabled, we can limit the permissions that Qinling According least privilege principle, the operation permission of qinling user
access the Kubernetes API. Before you procceed the steps in this section, in kubernetes cluster should be limited, this could be easily achieved by
make sure that the Kubernetes API server is running with the applying the pre-defined authorization manifest file.
``--authorization-mode=RBAC`` option.
Qinling provides a single file located at
``example/kubernetes/k8s_qinling_role.yaml`` for users to
create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling
needs, and bind the roles to the user named ``qinling``, which is from
the common name of the subject in the client certificate. The role is defined
within a namespace named ``qinling``, which is the default namespace that
Qinling uses and the name is configurable.
.. code-block:: console .. code-block:: console
curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f - curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f -
.. end
Restart Qinling Engine Restart qinling-engine service
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Restart the qinling-engine service after the steps above are done, and now Restart the ``qinling-engine`` service after the steps above are done, and now
Qinling is accessing the Kubernetes API with itself authenticated by a client Qinling is accessing the Kubernetes API and etcd service using TLS. The
certificate. The requests that Qinling makes to the Kubernetes API are also requests that Qinling makes to the Kubernetes API are also authorized.
authorized.
.. code-block:: console
systemctl restart devstack@qinling-engine.service
.. end
Access the Kubernetes API Insecurely (For testing purpose ONLY) Access the Kubernetes API Insecurely (For testing purpose ONLY)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
server serves for insecure connections. However this is not recommended and server serves for insecure connections. However, this is not recommended and
should be used for testing purpose only. should be used for testing purpose only.
In the configuration file, under the ``kubernetes`` section, set ``kube_host`` In the configuration file, under the ``kubernetes`` section, set ``kube_host``
@ -155,3 +179,11 @@ to the URI which the Kubernetes API serves for insecure HTTP connections, for
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate`` example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
to ``False`` to disable Qinling using a client certificate to access the to ``False`` to disable Qinling using a client certificate to access the
Kubernetes API. Kubernetes API.
.. code-block:: ini
[kubernetes]
kube_host = http://localhost:8080
use_api_certificate = False
.. end

View File

@ -15,7 +15,7 @@
Setting up a development environment with devstack Setting up a development environment with devstack
================================================== ==================================================
This page describes how to setup a working development This page describes how to set up a working development
environment that can be used in deploying qinling on latest releases environment that can be used in deploying qinling on latest releases
of Ubuntu. These instructions assume you are already familiar of Ubuntu. These instructions assume you are already familiar
with git. Refer to `Getting the code`_ for additional information. with git. Refer to `Getting the code`_ for additional information.
@ -67,15 +67,14 @@ configuration:
LOG_COLOR=False LOG_COLOR=False
LOGDAYS=1 LOGDAYS=1
ENABLED_SERVICES=rabbit,mysql,key,tempest ENABLED_SERVICES=rabbit,mysql,key
.. end .. end
.. note:: Here are several things you could customize:
For multiple network cards, you need to update the Kubernetes apiserver's advertise address * For multiple network cards, you need to specify the kubernetes API server's
to the address on the interface which is used to get to the default gateway by adding one advertise address manually.
environment variable.
.. code-block:: console .. code-block:: console
@ -83,6 +82,17 @@ configuration:
.. end .. end
* Devstack will set up a new kubernetes cluster and re-use etcd service inside
the cluster for Qinling services, which means you don't need to add etcd to
the enabled services list in the ``local.conf`` file.
* If you already have an existing kubernetes cluster, add
``QINLING_INSTALL_K8S=False`` to the ``local.conf`` file. Go to
`Config Qinling with existing Kubernetes cluster <https://docs.openstack.org/qinling/latest/admin/install/config_kubernetes.html>`_
for more details.
* If you want to interact with Qinling in Horizon, add
``enable_plugin qinling-dashboard https://git.openstack.org/openstack/qinling-dashboard``
in the ``local.conf`` file.
Running devstack Running devstack
---------------- ----------------