Merge "Improve the documentation"
This commit is contained in:
commit
448cf1c90a
|
@ -12,64 +12,89 @@
|
||||||
License for the specific language governing permissions and limitations
|
License for the specific language governing permissions and limitations
|
||||||
under the License.
|
under the License.
|
||||||
|
|
||||||
Config Qinling with existing Kubernetes cluster
|
Config Qinling with existing kubernetes/etcd cluster
|
||||||
===============================================
|
====================================================
|
||||||
|
|
||||||
In most cases, it's not ideal to set up a new dedicated Kubernetes cluster for
|
In most cases, it's not ideal to set up a new dedicated kubernetes cluster for
|
||||||
Qinling. The component which works with Kubernetes cluster in Qinling is the
|
Qinling. The component which works with kubernetes cluster in Qinling is the
|
||||||
``qinling-engine``. Follow the steps below to configure Qinling to work with an
|
``qinling-engine``. Follow the steps below to configure Qinling to work with
|
||||||
existing Kubernetes cluster, and make Qinling access the Kubernetes API with
|
existing kubernetes/etcd cluster, and make Qinling access the kubernetes/etcd
|
||||||
authentication and authorization.
|
service with authentication and authorization.
|
||||||
|
|
||||||
Qinling Configurations
|
Prerequisites
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
* You know the kubernetes API address and etcd service address, for example:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
K8S_ADDRESS=10.0.0.5
|
||||||
|
ETCD_ADDRESS=10.0.0.6
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
* You have CA certificates of the kubernetes and etcd respectively and store on
|
||||||
|
the host that ``qinling-engine`` is running.
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
K8S_CA_CERT=$HOME/ca.crt
|
||||||
|
K8S_CA_KEY=$HOME/ca.key
|
||||||
|
ETCD_CA_CERT=$HOME/etcd_ca.crt
|
||||||
|
ETCD_CA_KEY=$HOME/etcd_ca.key
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
|
* This guide assumes
|
||||||
|
`RBAC <https://kubernetes.io/docs/admin/authorization/rbac/>`_ is enabled in
|
||||||
|
the kubernetes cluster.
|
||||||
|
|
||||||
|
Qinling configurations
|
||||||
~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Below are the options that relate to accessing the Kubernetes API in Qinling's
|
Below are the options and their default values that relate to accessing the
|
||||||
configuration file, all of them are under the ``kubernetes`` section.
|
Kubernetes API and etcd in Qinling's configuration file.
|
||||||
|
|
||||||
.. code-block:: ini
|
.. code-block:: ini
|
||||||
|
|
||||||
[kubernetes]
|
[kubernetes]
|
||||||
kube_host = http://127.0.0.1:8001
|
kube_host = https://127.0.0.1:8001
|
||||||
use_api_certificate = True
|
use_api_certificate = True
|
||||||
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
|
ssl_ca_cert = /etc/qinling/pki/kubernetes/ca.crt
|
||||||
cert_file = /etc/qinling/pki/kubernetes/qinling.crt
|
cert_file = /etc/qinling/pki/kubernetes/qinling.crt
|
||||||
key_file = /etc/qinling/pki/kubernetes/qinling.key
|
key_file = /etc/qinling/pki/kubernetes/qinling.key
|
||||||
|
|
||||||
For now, just update the ``kube_host`` to the URI which the Kubernetes API
|
[etcd]
|
||||||
serves for HTTPS connections with authentication and authorization, for
|
host = 127.0.0.1
|
||||||
example, ``kube_host = https://kube-api.example.com:6443``. We will cover the
|
port = 2379
|
||||||
other options in the following sections.
|
protocol = https
|
||||||
|
ca_cert = /etc/qinling/pki/etcd/ca.crt
|
||||||
|
cert_file = /etc/qinling/pki/etcd/qinling-etcd-client.crt
|
||||||
|
cert_key = /etc/qinling/pki/etcd/qinling-etcd-client.key
|
||||||
|
|
||||||
Authentication and Authorization
|
.. end
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
The access to the Kubernetes API is controlled by several modules, refer to
|
Change the kubernetes and etcd service addresses:
|
||||||
`Controlling Access to the Kubernetes API <https://kubernetes.io/docs/admin/accessing-the-api/>`_
|
|
||||||
for more details.
|
|
||||||
|
|
||||||
By default, Qinling engine is configured to access the Kubernetes API with
|
.. code-block:: ini
|
||||||
a client certificate for authentication(``use_api_certificate`` is set to
|
|
||||||
``True``), so make sure that the Kubernetes API server is running with the
|
|
||||||
``--client-ca-file=SOMEFILE`` option for client certificate authentication to
|
|
||||||
be enabled. The common name of the subject in the client certificate is used as
|
|
||||||
the user name for the requests that Qinling engine makes to the Kubernetes API
|
|
||||||
server. Refer to
|
|
||||||
`Authentication in Kubernetes <https://kubernetes.io/docs/admin/authentication/>`_.
|
|
||||||
|
|
||||||
If `RBAC Authorization <https://kubernetes.io/docs/admin/authorization/rbac/>`_
|
[kubernetes]
|
||||||
is enabled in the Kubernetes API, we will also have to grant access to
|
kube_host = https://${K8S_ADDRESS}:6443
|
||||||
resources in Kubernetes for the specific user that Qinling uses to make
|
...
|
||||||
requests to the Kubernetes API. Using RBAC Authorization can ensure that
|
[etcd]
|
||||||
Qinling access the Kubernetes API with only the permission that it needs.
|
host = ${ETCD_ADDRESS}
|
||||||
|
...
|
||||||
|
|
||||||
Generate Client Certificate for Qinling
|
.. end
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
See `Managing Certificates <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
|
Generate and config client certificates for Qinling
|
||||||
for how to generate a client cert. We use ``cfssl`` as the example here.
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
#) Download and prepare the command line tools.
|
There are a lot of
|
||||||
|
`tools <https://kubernetes.io/docs/concepts/cluster-administration/certificates/>`_
|
||||||
|
out there for certificate generation. We use ``cfssl`` as the example here.
|
||||||
|
|
||||||
|
#) Download and prepare the command line tools as needed.
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
|
@ -78,76 +103,75 @@ for how to generate a client cert. We use ``cfssl`` as the example here.
|
||||||
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
|
curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /tmp/cfssljson
|
||||||
chmod +x /tmp/cfssljson
|
chmod +x /tmp/cfssljson
|
||||||
|
|
||||||
#) Generate the client ceritificate for Qinling. Note that the common name
|
.. end
|
||||||
of the subject is set to ``qinling`` in the example CSR located at
|
|
||||||
``example/kubernetes/cfssl-client-csr.json``.
|
#) Generate the kubernetes and etcd client certificates for Qinling.
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
mkdir /tmp/certs; cd /tmp/certs
|
mkdir -p /tmp/certs; cd /tmp/certs
|
||||||
/tmp/cfssl gencert -ca=/path/to/kubernetes_ca_crt \
|
curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-ca-config.json -o /tmp/certs/cfssl-ca-config.json
|
||||||
-ca-key=/path/to/kubernetes_ca_key \
|
curl -SL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/cfssl-client-csr.json -o /tmp/certs/cfssl-client-csr.json
|
||||||
-config=QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-ca-config.json \
|
/tmp/cfssl gencert -ca=${K8S_CA_CERT} \
|
||||||
|
-ca-key=${K8S_CA_KEY} \
|
||||||
|
-config=/tmp/certs/cfssl-ca-config.json \
|
||||||
-profile=client \
|
-profile=client \
|
||||||
QINLING_SOURCE_FOLDER/example/kubernetes/cfssl-client-csr.json | /tmp/cfssljson -bare client
|
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare k8s-client
|
||||||
|
/tmp/cfssl gencert -ca=${ETCD_CA_CERT} \
|
||||||
|
-ca-key=${ETCD_CA_KEY} \
|
||||||
|
-config=/tmp/certs/cfssl-ca-config.json \
|
||||||
|
-profile=client \
|
||||||
|
/tmp/certs/cfssl-client-csr.json | /tmp/cfssljson -bare etcd-client
|
||||||
|
|
||||||
#) Copy the needed files to the locations. The command above generates two
|
.. end
|
||||||
files named ``client-key.pem`` and ``client.pem``, the former is the key
|
|
||||||
file of the client certificate, and the latter is the certificate file
|
|
||||||
itself.
|
|
||||||
|
|
||||||
.. note::
|
#) Move the certificates to the appropriate folders and ensure the qinling
|
||||||
|
service user has permission to those folders.
|
||||||
|
|
||||||
Remember to backup the existing files in /etc/qinling/pki/kubernetes
|
.. code-block:: console
|
||||||
folder first.
|
|
||||||
|
|
||||||
.. code-block:: console
|
mkdir -p /etc/qinling/pki/{kubernetes,etcd}
|
||||||
|
cp k8s-client-key.pem /etc/qinling/pki/kubernetes/qinling.key
|
||||||
|
cp k8s-client.pem /etc/qinling/pki/kubernetes/qinling.crt
|
||||||
|
cp etcd-client-key.pem /etc/qinling/pki/etcd/qinling-etcd-client.key
|
||||||
|
cp etcd-client.pem /etc/qinling/pki/etcd/qinling-etcd-client.crt
|
||||||
|
cp ${K8S_CA_CERT} /etc/qinling/pki/kubernetes/ca.crt
|
||||||
|
cp ${ETCD_CA_CERT} /etc/qinling/pki/etcd/ca.crt
|
||||||
|
chown -R qinling:qinling /etc/qinling/pki
|
||||||
|
|
||||||
mkdir -p /etc/qinling/pki/kubernetes
|
.. end
|
||||||
mv client-key.pem /etc/qinling/pki/kubernetes/qinling.key
|
|
||||||
mv client.pem /etc/qinling/pki/kubernetes/qinling.crt
|
|
||||||
mv /path/to/kubernetes_ca_crt /etc/qinling/pki/kubernetes/ca.crt
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
Make sure both ``/etc/qinling/pki/kubernetes`` and ``/etc/qinling/pki``
|
|
||||||
belong to Qinling service user. You can set the permissions with
|
|
||||||
``chown -R qinling:qinling /etc/qinling/pki``
|
|
||||||
|
|
||||||
Create Role and RoleBinding in Kubernetes
|
Create Role and RoleBinding in Kubernetes
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
If RBAC Authorization is enabled, we can limit the permissions that Qinling
|
According least privilege principle, the operation permission of qinling user
|
||||||
access the Kubernetes API. Before you procceed the steps in this section,
|
in kubernetes cluster should be limited, this could be easily achieved by
|
||||||
make sure that the Kubernetes API server is running with the
|
applying the pre-defined authorization manifest file.
|
||||||
``--authorization-mode=RBAC`` option.
|
|
||||||
|
|
||||||
Qinling provides a single file located at
|
|
||||||
``example/kubernetes/k8s_qinling_role.yaml`` for users to
|
|
||||||
create a ``Role`` and a ``ClusterRole`` with the permissions that Qinling
|
|
||||||
needs, and bind the roles to the user named ``qinling``, which is from
|
|
||||||
the common name of the subject in the client certificate. The role is defined
|
|
||||||
within a namespace named ``qinling``, which is the default namespace that
|
|
||||||
Qinling uses and the name is configurable.
|
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f -
|
curl -sSL https://raw.githubusercontent.com/openstack/qinling/master/example/kubernetes/k8s_qinling_role.yaml | kubectl apply -f -
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
Restart Qinling Engine
|
Restart qinling-engine service
|
||||||
~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Restart the qinling-engine service after the steps above are done, and now
|
Restart the ``qinling-engine`` service after the steps above are done, and now
|
||||||
Qinling is accessing the Kubernetes API with itself authenticated by a client
|
Qinling is accessing the Kubernetes API and etcd service using TLS. The
|
||||||
certificate. The requests that Qinling makes to the Kubernetes API are also
|
requests that Qinling makes to the Kubernetes API are also authorized.
|
||||||
authorized.
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
systemctl restart devstack@qinling-engine.service
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
||||||
Access the Kubernetes API Insecurely (For testing purpose ONLY)
|
Access the Kubernetes API Insecurely (For testing purpose ONLY)
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
|
Qinling can also connect to the Kubernetes API insecurely if the Kubernetes API
|
||||||
server serves for insecure connections. However this is not recommended and
|
server serves for insecure connections. However, this is not recommended and
|
||||||
should be used for testing purpose only.
|
should be used for testing purpose only.
|
||||||
|
|
||||||
In the configuration file, under the ``kubernetes`` section, set ``kube_host``
|
In the configuration file, under the ``kubernetes`` section, set ``kube_host``
|
||||||
|
@ -155,3 +179,11 @@ to the URI which the Kubernetes API serves for insecure HTTP connections, for
|
||||||
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
|
example, ``kube_host = http://localhost:8080``, and set ``use_api_certificate``
|
||||||
to ``False`` to disable Qinling using a client certificate to access the
|
to ``False`` to disable Qinling using a client certificate to access the
|
||||||
Kubernetes API.
|
Kubernetes API.
|
||||||
|
|
||||||
|
.. code-block:: ini
|
||||||
|
|
||||||
|
[kubernetes]
|
||||||
|
kube_host = http://localhost:8080
|
||||||
|
use_api_certificate = False
|
||||||
|
|
||||||
|
.. end
|
||||||
|
|
|
@ -15,7 +15,7 @@
|
||||||
Setting up a development environment with devstack
|
Setting up a development environment with devstack
|
||||||
==================================================
|
==================================================
|
||||||
|
|
||||||
This page describes how to setup a working development
|
This page describes how to set up a working development
|
||||||
environment that can be used in deploying qinling on latest releases
|
environment that can be used in deploying qinling on latest releases
|
||||||
of Ubuntu. These instructions assume you are already familiar
|
of Ubuntu. These instructions assume you are already familiar
|
||||||
with git. Refer to `Getting the code`_ for additional information.
|
with git. Refer to `Getting the code`_ for additional information.
|
||||||
|
@ -67,15 +67,14 @@ configuration:
|
||||||
LOG_COLOR=False
|
LOG_COLOR=False
|
||||||
LOGDAYS=1
|
LOGDAYS=1
|
||||||
|
|
||||||
ENABLED_SERVICES=rabbit,mysql,key,tempest
|
ENABLED_SERVICES=rabbit,mysql,key
|
||||||
|
|
||||||
.. end
|
.. end
|
||||||
|
|
||||||
.. note::
|
Here are several things you could customize:
|
||||||
|
|
||||||
For multiple network cards, you need to update the Kubernetes apiserver's advertise address
|
* For multiple network cards, you need to specify the kubernetes API server's
|
||||||
to the address on the interface which is used to get to the default gateway by adding one
|
advertise address manually.
|
||||||
environment variable.
|
|
||||||
|
|
||||||
.. code-block:: console
|
.. code-block:: console
|
||||||
|
|
||||||
|
@ -83,6 +82,17 @@ configuration:
|
||||||
|
|
||||||
.. end
|
.. end
|
||||||
|
|
||||||
|
* Devstack will set up a new kubernetes cluster and re-use etcd service inside
|
||||||
|
the cluster for Qinling services, which means you don't need to add etcd to
|
||||||
|
the enabled services list in the ``local.conf`` file.
|
||||||
|
* If you already have an existing kubernetes cluster, add
|
||||||
|
``QINLING_INSTALL_K8S=False`` to the ``local.conf`` file. Go to
|
||||||
|
`Config Qinling with existing Kubernetes cluster <https://docs.openstack.org/qinling/latest/admin/install/config_kubernetes.html>`_
|
||||||
|
for more details.
|
||||||
|
* If you want to interact with Qinling in Horizon, add
|
||||||
|
``enable_plugin qinling-dashboard https://git.openstack.org/openstack/qinling-dashboard``
|
||||||
|
in the ``local.conf`` file.
|
||||||
|
|
||||||
Running devstack
|
Running devstack
|
||||||
----------------
|
----------------
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue