starlingx/docs
Code Issues Proposed changes

HTTPS cert updates

Browse Source 
General update to Security/HTTPS and Certificates Management:
- reorganization
- content updates
Implement patchset 1 review comments
Implement patchset 2 review comments

Closes-Bug: 2028184

Signed-off-by: Ron Stone <ronald.stone@windriver.com>
Change-Id: Iae75785e479c96751fb50a097eba8ed5e6069e94
This commit is contained in:
Ron Stone 2023-07-14 11:31:15 +00:00
parent bc2f3c07ed
commit f843d3daa4
11 changed files with 433 additions and 361 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
doc/source/security/kubernetes/configure-docker-registry-certificate-after-installation-c519edbfe90a.rst
View File

@ -21,9 +21,7 @@ create a certificate named ``system-registry-local-certificate`` in the
spec must also be named ``system-registry-local-certificate``.
See the example procedure below for creating the certificate for the local
Docker registry. This example assumes you have configured a
``system-local-ca`` ClusterIssuer as described in
:ref:`starlingx-rest-api-applications-and-the-web-admin-server-cert-9196c5794834`.
Docker registry.
Update the following fields:
@ -83,9 +81,9 @@ Update the following fields:
.. code-block::
~(keystone_admin)]$ kubectl get certificate system-registry-local-certificate –n deployment
~(keystone_admin)]$ kubectl get certificate system-registry-local-certificate -n deployment
If configuration was successful, the certificate’s Ready status will be
If configuration was successful, the certificate's Ready status will be
``True``.
#. Update the platform's trusted certificates (i.e. ``ssl_ca``) with the Root

4
doc/source/security/kubernetes/configure-rest-api-applications-and-web-administration-server-certificates-after-installation-6816457ab95f.rst
View File

@ -20,9 +20,7 @@ in the ``deployment`` namespace. The ``secretName`` attribute of this
certificate's spec must also be named ``system-restapi-gui-certificate``.
See the example procedure below for creating the certificate for the StarlingX
REST API and Web Server endpoints. This example assumes you have configured a
``system-local-ca`` ClusterIssuer as described in
:ref:`starlingx-rest-api-applications-and-the-web-admin-server-cert-9196c5794834`.
REST API and Web Server endpoints.
Update the following fields:

11
doc/source/security/kubernetes/create-certificates-locally-using-cert-manager-on-the-controller.rst
View File

@ -9,6 +9,17 @@ Create Certificates Locally using cert-manager on the Controller
You can use :command:`cert-manager` to locally create certificates suitable
for use in a lab environment.
.. note::
Ensure the certificates have RSA key length >= 2048 bits. The
|prod-long| Release |this-ver| provides a new version of ``openssl`` which
requires a minimum of 2048-bit keys for RSA for better security / encryption
strength.
You can check the key length by running ``openssl x509 -in <the-certificate-file>
-noout -text`` and looking for the "Public-Key" in the output. For more
information see :ref:`Create Certificates Locally using openssl <create-certificates-locally-using-openssl>`.
.. rubric:: |proc|
#. Create a Root |CA| Certificate and Key.

109
doc/source/security/kubernetes/https-access-overview.rst
View File

@ -6,12 +6,18 @@
HTTPS and Certificates Management Overview
==========================================
Certificates are heavily used for secure HTTPS access and authentication on
|prod| platform. This table lists the major certificates being used in the
system, and indicates which certificates are automatically created/renewed by
the system versus which certificates must be manually created/renewed by the
system administrator. Details on manual management of certificates can be found
in the following sections.
Certificates are required for secure HTTPS access and authentication on |prod|
platform.
This table lists all the platform certificates, and indicates which
certificates are automatically created/renewed by the system versus which
certificates must be manually created/renewed by the system administrator.
Platform certificates that are associated with optional platform components are
only present if the optional platform component is configured (e.g. |OIDC|).\
Platform certificates that are associated with Distributed Cloud are only
present on |DC| SystemController systems or |DC| Subclouds.
.. table::
:widths: auto
@ -19,6 +25,18 @@ in the following sections.
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Certificate | Auto Created | Renewal Status |
+===========================================================+=============================================================================+========================================================================================================+
| **Etcd:** |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| etcd Root CA certificate | Yes | NOT AUTO-RENEWED; Default expiry is set at 10 years |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| etcd server certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| etcd client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| kube-apiserver's etcd client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| **Kubernetes:** |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Kubernetes Root CA Certificate | Yes | NOT AUTO-RENEWED; Default expiry is set at 10 years; MUST be renewed via CLI. |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Cluster Admin client certificate used by kubectl | Yes | auto-renewed by cron job |
@ -31,58 +49,44 @@ in the following sections.
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| kube-apiserver's kubelet client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| kubelet client certificate | Yes | auto-renewed by kubelet feature enabled by default |
| kubelet client certificate | Yes | auto-renewed by kubelet. Feature enabled by default |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| front-proxy-client | Yes | front-proxy-client: auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| front-proxy-ca | Yes | front-proxy-ca: NOT AUTO-RENEWED; Default expiry is set at 10 years |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
| **system-local-ca** | Yes | NOT AUTO-RENEWED. MUST be renewed via CLI |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| etcd Root CA certificate | Yes | NOT AUTO-RENEWED; Default expiry is set at 10 years |
| **OpenLDAP Server Certificate** | Yes | auto-renewed by system |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| etcd server certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| etcd client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| kube-apiserver's etcd client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| StarlingX REST API & HORIZON Server Certificate | Yes (But the auto-created certificate is self-signed and should be changed) | auto-renewed if configured with cert-manager; |
| **StarlingX REST API & HORIZON Server Certificate** | Yes (But the auto-created certificate is self-signed and should be changed) | auto-renewed if configured with cert-manager; |
| | | NOT AUTO-RENEWED if configured with :command:`system certificate-install ..`, must be renewed via CLI |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Local Registry Server Certificate | Yes (But the auto-created certificate is self-signed and should be changed) | auto-renewed if configured with cert-manager; |
| **Local Registry Server Certificate** | Yes (But the auto-created certificate is self-signed and should be changed) | auto-renewed if configured with cert-manager; |
| | | NOT AUTO-RENEWED if configured with :command:`system certificate-install ..`, must be renewed via CLI |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| OpenLDAP Server Certificate | Yes | auto-renewed by system |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
| **OIDC:** |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| OIDC Client and Dex Server Server Certificate | No | auto-renewed if configured with cert-manager; |
| | | NOT AUTO-RENEWED if configured with an externally generated certificate, CUSTOMER MUST renew via CLI. |
| | | NOT AUTO-RENEWED if configured with an externally generated certificate. MUST be renewed via CLI. |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| OIDC Client and Dex Server CA certificate | No | NOT AUTO-RENEWED. CUSTOMER MUST renew via CLIs |
| OIDC Client and Dex Server CA certificate | No | NOT AUTO-RENEWED. MUST be renewed via CLI. |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| OIDC Remote WAD CA Certificate | No | NOT AUTO-RENEWED. CUSTOMER MUST renew via CLIs |
| OIDC Remote WAD CA Certificate | No | NOT AUTO-RENEWED. MUST be renewed via CLI. |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
| **Vault:** |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Vault Server Certificate | Yes | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
| Vault Server Certificate | Yes | NOT AUTO-RENEWED; MUST be renewed via CLI. |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Vault Root CA certificate | Yes | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
| Vault Root CA certificate | Yes | NOT AUTO-RENEWED; MUST be renewed via CLI. |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
| **Portieris:** |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Portieris Server Certificate | Yes | Auto renewed by cert-manager; BUT CUSTOMER MUST restart Portieris after the certificate is renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Portieris remote registry and notary server CA Certificate| No | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
| **DC Admin Endpoints:** |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| Root CA DC Admin Endpoint CA Certificate | Yes | auto-renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
@ -90,9 +94,7 @@ in the following sections.
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| DC Admin Endpoint Server Certificate | Yes | auto-renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
| System trusted CA Certificates | No | NOT AUTO-RENEWED as these are certificates that are not necessarily owned by the platform |
| **System trusted CA Certificates** | No | NOT AUTO-RENEWED as these are certificates that are not necessarily owned by the platform |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+
Where:
@ -103,30 +105,13 @@ Where:
- Renewal Status: whether the certificate is renewed automatically by the system
when expiry date approaches.
The following sections provide details on managing these certificates.
The specific certificates, and details such as expiration date, that are
present on a |prod| system can be displayed with a local script, :command:`sudo
show-certs.sh`, see :ref:`utility-script-to-display-certificates`.
- :ref:`StarlingX REST API Applications and the Web Administration Server Certificate <starlingx-rest-api-applications-and-the-web-administration-server>`
|prod| monitors the installed certificates on the system by raising alarms for
expired certificates and certificates that will expire soon, see
:ref:`alarm-expiring-soon-and-expired-certificates-baf5b8f73009`.
- :ref:`Kubernetes Certificates <kubernetes-certificates-f4196d7cae9c>`
- :ref:`Local Registry Server Certificates <security-install-update-the-docker-registry-certificate>`
- :ref:`System Trusted CA Certificates <add-a-trusted-ca>`
For further information about certificates expiration date or other certificates
information, see :ref:`Display Certificates Installed on a System <utility-script-to-display-certificates>`.
In addition, |prod| monitors the installed certificates on the system by raising
alarms for expire-soon certificates and for expired certificates on the system,
see :ref:`Expiring-Soon and Expired Certificate Alarms
<alarm-expiring-soon-and-expired-certificates-baf5b8f73009>`.
.. note::
Ensure the certificates have RSA key length >= 2048 bits. The
|prod-long| Release |this-ver| provides a new version of ``openssl`` which
requires a minimum of 2048-bit keys for RSA for better security / encryption
strength.
You can check the key length by running ``openssl x509 -in <the certificate file> -noout -text``
and looking for the "Public-Key" in the output. For more information see
:ref:`Create Certificates Locally using openssl <create-certificates-locally-using-openssl>`.
The following sections also provide details on creating and/or renewing the
|prod| platform certificates.

6
doc/source/security/kubernetes/index-security-kub-81153c1254c3.rst
View File

@ -115,10 +115,9 @@ HTTPS Certificate Management
https-access-overview
utility-script-to-display-certificates
starlingx-rest-api-applications-and-the-web-admin-server-cert-9196c5794834
kubernetes-certificates-f4196d7cae9c
etcd-certificates-c1fc943e4a9c
kubernetes-root-ca-certificate
kubernetes-certificates-f4196d7cae9c
starlingx-rest-api-applications-and-the-web-admin-server-cert-9196c5794834
configure-rest-api-applications-and-web-administration-server-certificates-after-installation-6816457ab95f
configure-docker-registry-certificate-after-installation-c519edbfe90a
oidc-client-dex-server-certificates-dc174462d51a
@ -127,7 +126,6 @@ HTTPS Certificate Management
vault-server-certificate-8573125eeea6
dc-admin-endpoint-certificates-8fe7adf3f932
add-a-trusted-ca
one-single-root-ca-multiple-server-client-certificates-0692df6ce16d
alarm-expiring-soon-and-expired-certificates-baf5b8f73009
************

117
doc/source/security/kubernetes/kubernetes-certificates-f4196d7cae9c.rst
View File

@ -9,39 +9,12 @@ endpoints.
Kubernetes automatically creates all of its client and server certificates, and
signs them with a Kubernetes Root |CA|. This includes the server certificate
for the external ``kube-apiserver`` API endpoint. By default, the Kubernetes
Root |CA| is automatically generated at install time.
If desired, you can externally generate a Root |CA| certificate and key, and
configure it as the Kubernetes Root |CA| during installation. Currently,
StarlingX supports only Internal |CA| mode with Kubernetes, which only supports
a Root |CA| for the Kubernetes Root |CA|, not an Intermediate |CA|.
The public certificate of the Kubernetes Root |CA|, whether auto-generated or
specified, needs to be configured as a trusted |CA| by external servers
connecting to |prod|'s Kubernetes API endpoint (e.g. via a remotely installed
``kubectl`` client).
.. note::
Some platform services (sysinv, cert-mon and VIM for example) also use X509
certificates to access Kubernetes by HTTPS.
It is optional that you update the Kubernetes Root |CA| with a custom Root CA
certificate and key, generated by yourself, and trusted by your external
servers connecting to |prod|’s Kubernetes API endpoint. The |prod|’s Kubernetes
Root |CA| certificate and key are configured as part of the bootstrap during
installation.
.. note::
You must use a Root |CA| certificate; Intermediate |CA| certificates
are not supported by upstream Kubernetes.
for the external ``kube-apiserver`` API endpoint.
Kubernetes certificates include:
- Kubernetes Root |CA| Certificate
- Cluster admin client certificate used by ``kubectl``
- Cluster admin client certificate
- ``kube-controller-manager`` client certificate
- ``kube-scheduler`` client certificate
- ``kube-apiserver`` server certificate
@ -54,8 +27,29 @@ The Kubernetes Root |CA| certificate signs all the other Kubernetes
certificates. This is also the |CA| certificate various components use to
verify server and client certificates signed by the Kubernetes Root |CA|
certificate. For example, applications running in pods use Kubernetes Root |CA|
certificate embedded in service account token to verify the ``kube-apiserver``'s
server certificate when it makes calls to the kube-apiserver.
certificate embedded in service account token to verify the
``kube-apiserver``'s server certificate when it makes calls to the
kube-apiserver.
By default, the Kubernetes Root |CA| is automatically generated at install time.
If desired, you can externally generate a Root |CA| certificate and key, and
configure it as the Kubernetes Root |CA| during installation, see
:ref:`kubernetes-root-ca-certificate`.
.. warning::
This must be a Root |CA|, not an Intermediate |CA|.
In a Distributed Cloud system, by default, the Subclouds are deployed with the
same Kubernetes Root CA as the SystemController.
The public certificate of the Kubernetes Root |CA|, whether auto-generated or
specified, needs to be configured as a trusted |CA| by external servers
connecting to Cloud Platform’s Kubernetes API endpoint (e.g. via a remotely
installed kubectl client). The Kubernetes Root |CA| public certificate can be
found at ``/etc/kubernetes/pki/ca.crt``.
Kubernetes Root |CA| certificate and corresponding private key are stored in
file system:
@ -64,16 +58,42 @@ file system:
- ``/etc/kubernetes/pki/ca.key``
.. note::
Updating/Renewing the Kubernetes Root |CA| is a complex process, because it is
not only the Root |CA| certificate that needs to be updated, but also all the
other Kubernetes certificates signed by it need to be regenerated and updated.
See :ref:`manual-kubernetes-root-ca-certificate-update-8e9df2cd7fb9` or
:ref:`kubernetes-root-ca-certificate-update-cloud-orchestration-a627f9d02d6d`
for information on how to update the Kubernetes Root |CA| certificate.
Kubernetes Root |CA| certificate is also embedded in various
configuration files and service account token.
**Cluster admin client certificate used by kubectl**
**Cluster admin client certificate**
This is the client certificate signed by Kubernetes Root |CA| and embedded in
``/etc/kubernetes/admin.conf``. It is used by kubectl command to identify
itself to the ``kube-apiserver``.
the ``/etc/kubernetes/admin.conf`` KUBECONFIG file for kubectl. This is used by
internal StarlingX services and users logged in via SSH with sys_protected
group permissions, to access kubernetes-admin credentials for kubernetes API or
CLI commands.
This certificate is monitored and auto-renewed by a cronjob, which runs every
day at midnight to check if the expiry date is approaching, and renew the
certificate if the expiry date is within 15 days.
.. note::
If the cronjob certificate renewal fails, a 250.003 alarm will be raised:
* Kubernetes certificates have been renewed but not all services have been
updated.
For this alarm, controller nodes need to lock/unlock for the services to
take the new certificates.
* Kubernetes certificates renewal failed.
For this alarm, the Kubernetes certificates need to be renewed manually,
during which services need to restart.
If this alarm is raised, the administrator should follow the recommended
action for the specific alarm.
**kube-controller-manager client certificate**
@ -81,12 +101,20 @@ This is the client certificate signed by Kubernetes Root |CA| and embedded in
``/etc/kubernetes/controller-manager.conf``. It is used by
``kube-controller-manager`` pod to identify itself to ``kube-apiserver``.
This certificate is monitored and auto-renewed by a cronjob, which runs every
day at midnight to check if the expiry date is approaching, and renew the
certificate if the expiry date is within 15 days.
**kube-scheduler client certificate**
This is the client certificate signed by Kubernetes Root |CA| and embedded in
``/etc/kubernetes/scheduler.conf``. It is used by ``kube-scheduler`` pod to
identify itself to the ``kube-apiserver``.
This certificate is monitored and auto-renewed by a cronjob, which runs every
day at midnight to check if the expiry date is approaching, and renew the
certificate if the expiry date is within 15 days.
**kube-apiserver server certificate**
This is the kube-apiserver's serving certificate. Clients connecting to the
@ -98,6 +126,10 @@ file system:
- ``/etc/kubernetes/pki/apiserver.key``
This certificate is monitored and auto-renewed by a cronjob, which runs every
day at midnight to check if the expiry date is approaching, and renew the
certificate if the expiry date is within 15 days.
**kube-apiserver's kubelet client certificate**
``kube-apiserver``'s client certificate for communications with ``kubelet``.
@ -109,6 +141,10 @@ file system:
- ``/etc/kubernetes/pki/apiserver-kubelet-client.key``
This certificate is monitored and auto-renewed by a cronjob, which runs every
day at midnight to check if the expiry date is approaching, and renew the
certificate if the expiry date is within 15 days.
**kubelet client certificate**
This is the ``kubelet``’s client certificate (with private key in it).
@ -121,7 +157,8 @@ one file:
- ``/var/lib/kubelet/pki/kubelet-client-current.pem``
This certificate is configured to auto renew.
Kubelet is configured to auto-renew this certificate.
**front-proxy-client certificate**
@ -136,7 +173,9 @@ required only if you run ``kube-proxy`` to support an extension API server.
.. toctree::
:maxdepth: 1
:hidden:
kubernetes-root-ca-certificate
update-renew-kubernetes-certificates-52b00bd0bdae
manual-kubernetes-root-ca-certificate-update-8e9df2cd7fb9
kubernetes-root-ca-certificate-update-cloud-orchestration-a627f9d02d6d

130
doc/source/security/kubernetes/migrate-platform-certificates-to-use-cert-manager-c0b1727e4e5d.rst
View File

@ -4,47 +4,49 @@
.. _migrate-platform-certificates-to-use-cert-manager-c0b1727e4e5d:
========================================================
Migrate/Update Platform Certificates to use Cert Manager
========================================================
===========================================================================
Update system-local-ca or Migrate Platform Certificates to use Cert Manager
===========================================================================
Platform Certificates from the legacy certificate management APIs/CLIs, which
will be deprecated in a future release, to the new method of configuring
Platform Certificates using Cert-Manager which enables auto-renewals of
Platform Certificates.
The playbook described in this section can be used to either update
``system-local-ca`` or migrate platform Certificates to use Cert Manager. For
updating ``system-local-ca``, this playbook will update the ``system-local-ca``
Secret and Issuer, re-sign all the Platform certificates using this issuer, and
in a Distributed Cloud environment, iterate thru all of the subclouds and do
the same updates and re-signing on each Subcloud. In the migration use case,
this playbook can be used to switch from using Platform certificates generated
from the legacy certificate management APIs/CLIs, which will be deprecated in a
future release, to the new method of configuring Platform certificates using
Cert-Manager which enables auto-renewals of Platform certificates. And again
in a Distributed Cloud environment will iterate thru all of the Subclouds.
.. rubric:: |context|
This migration script can be configured to execute on any
:ref:`deployment configuration <deployment-options>` supported by |prod|
(|AIO|, standard, and distributed cloud configurations), allowing you to migrate
the certificates at scale. The script will replace old certificates and a backup
of the original certificates will be retained for reference. The certificates
that will be migrated/updated with this playbook are:
This playbook can be configured to execute on any :ref:`deployment
configuration <deployment-options>` supported by |prod| (|AIO|, standard, and
distributed cloud configurations), allowing you to update system-local-ca and
re-sign certificates or migrate certificates at scale.
* REST API & Web Server certificate
* Docker Registry certificate
* OIDC-Auth-Apps certificate
The certificates (if they exist) that will be updated / migrated with this
playbook are:
The |CA| against which the server certificates will be validated can be generated
on-platform (self-signed) or use an external Root |CA| and |ICA|. Using an external
Root |CA| and |ICA| is recommended. Note that this ansible-playbook will use the same
|ICA| cert & key to create the Issuers and/or ClusterIssuers for all
controllers/subclouds.
.. note::
This playbook can also be used to update certificates, which is useful for
situations such as |ICA| approaching expiry.
* system-local-ca
* system-openldap-local-certificate
* REST API & Web Server certificate ( system-restapi-gui-certificate signed by system-local-ca)
* Docker Registry certificate (system-registry-local-certificate signed by system-local-ca)
* OIDC-Auth-Apps certificate (<user-specified> signed by system-local-ca)
.. rubric:: |proc|
#. Create an inventory file using Ansible-Vault.
You must create an inventory file to specify the playbook parameters. Using
ansible-vault is highly recommended for enhanced security. Ansible vault
will ask for a password in this step, which is used for subsequent
ansible-vault access and ansible-playbook commands.
ansible-vault is highly recommended in order to securely store the contents
of the inventory file which includes the ``system-local-ca`` public
certificate and private key, and the Root |CA| public certificate for
``system-local-ca``. Ansible vault will ask for a password in this step,
which is used for subsequent ansible-vault access and ansible-playbook
commands.
For example:
@ -88,27 +90,23 @@ controllers/subclouds.
The inventory parameters have the following meanings:
``system_local_ca_cert`` and ``system_local_ca_key``
You may choose to generate a certificate & key on the platform
(self-signed, internal Root |CA|) or use an external Root
|CA| that would make this an Intermediate |CA|.
Both values being the single-line base64 encoding of the corresponding
pem file; i.e. the output of :command:`base64 -w0 <pem-file>`.
.. note::
Ensure the certificates have RSA key length >= 2048 bits before
migrating to |prod-long| Release |this-ver|. The |prod-long| Release
|this-ver| provides a new version of ``openssl`` which requires a
minimum of 2048-bit keys for RSA for better security / encryption
strength.
You can check the key length by running ``openssl x509 -in <the certificate file> -noout -text``
and looking for the "Public-Key" in the output. For more information see
:ref:`Create Certificates Locally using openssl <create-certificates-locally-using-openssl>`.
It is highly recommended that you use an Intermediate |CA|
``system-local-ca``, where the ``system-local-ca``'s certificate and
key are generated and signed by an external trusted Root |CA|. Refer
to the documentation for the external trusted Root |CA| that you are
using, on how to create a public certificate and private key pair, for
use in an Intermediate |CA|.
The duration of the Intermediate CA public certificate and private key
pair should be at least 3 years. See *ca_duration* to modify this
semantic check.
``system_root_ca_cert``
The Root |CA| that signed ``system_local_ca_cert``. If
``system_local_ca_cert`` is a self-signed, internal Root |CA|
certificate, duplicate the value of ``system_local_ca_cert`` in this
field.
The public certificate of the Root |CA| that signed
``system_local_ca_cert``.
``ca_duration``
|CA| duration validation parameter. This will be used against
@ -118,19 +116,20 @@ controllers/subclouds.
renewed manually. Only override if necessary.
``system_platform_certificate.dns_domain``
The |DNS| domain that will be used to build the full dns name for the
The |DNS| domain that will be used to build a full DNS name for the
|SANs| List of the Platform Certificates. E.g.
``starlingx-restapi-gui.<dns_domain>`` would appear in the |SANs| list
of the REST API & Web Server certificate. in the server certificates.
``system_platform_certificate.duration``
The duration of certificate validity to use in all Platform
Certificates, in hours. The Platform Server Certificates will be
auto-renewed by Cert-Manager.
Certificates, in hours; defaults to 2160h (or 90 days). The Platform
Server Certificates will be auto-renewed by Cert-Manager.
``system_platform_certificate.renewBefore``
The number of hours before certificate expiry that the Platform
Certificate should be auto-renewed by Cert-Manager.
Certificate should be auto-renewed by Cert-Manager; defaults to 360h
(or 15 days).
``system_platform_certificate.subject_*fields``
Subject related fields that will be added to all platform certificates:
@ -147,9 +146,7 @@ controllers/subclouds.
- ``system_platform_certificate.subject_CN``: Common Name
- ``system_platform_certificate.subject_prefix``: An optional field
to add a prefix to further identify the certificate, such as |prod|
for instance
- ``system_platform_certificate.subject_prefix``: An optional field to add a prefix to further identify the certificate, such as |prod| for instance.
``ansible_ssh_user``
The username to use to connect to the target system using ``ssh``.
@ -158,7 +155,7 @@ controllers/subclouds.
The password to use to connect to the target system using ``ssh``.
``ansible_become_pass``
The target system's sudo password.
The ``ansible_ssh_user``'s sudo password.
If a separate set of overrides are required for a group of hosts,
``children`` groups can be added under ``target_group``.
@ -204,13 +201,19 @@ controllers/subclouds.
~(keystone_admin)]$ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/migrate_platform_certificates_to_certmanager.yml -i migration-inventory.yml --extra-vars "target_list=subcloud1 mode=update ignore_alarms=yes" --ask-vault-pass
The behavior of the migration can be customized using the following
.. note::
In |prod-dc| systems, the playbook must be executed from the System
Controller, and the ``target_list`` parameter should be used to target
the desired subclouds.
The behavior of the update/migration can be customized using the following
``--extra-vars`` parameter options:
``mode``
* ``update``: Creates or updates platform certificates. Also supports
ongoing updates, which is useful for operations such as such as
replacing the |ICA| or changing other parameters.
ongoing updates, which is useful for operations such as replacing the
|ICA| or changing other parameters.
* ``check``: Gathers certificates from all subclouds and prints them on
the system controller
@ -226,11 +229,8 @@ controllers/subclouds.
retrieve a list of online subclouds to target.
``ignore_alarms``
``yes``/``no``: When not specified defaults to no.
``yes``/``no``: When not specified defaults to no. Using
``ignore_alarms=yes`` should be avoided as much as possible. Only use
it after a careful analysis of the alarm in question and for specific
hosts.
.. note::
The ``ignore_alarms`` extra-var should be avoided as much as possible.
Only use it after a careful analysis of the alarm in question and for
specific hosts.

0
doc/source/security/kubernetes/one-single-root-ca-multiple-server-client-certificates-0692df6ce16d.rst → doc/source/security/kubernetes/one-single-root-ca-multiple-server-client-certificates-0692df6ce16d.unused
View File

12
doc/source/security/kubernetes/security-configure-container-backed-remote-clis-and-clients.rst
View File

@ -183,7 +183,8 @@ CLIs and Clients for an admin user with cluster-admin clusterrole.
to the remote workstation, and extract its content.
- The tarball is available from the |prod| area on the |prod| CENGEN build servers.
- The tarball is available from the |prod| area on the |prod| CENGE N
build servers.
- You can extract the tarball's contents anywhere on your client system.
@ -209,17 +210,18 @@ CLIs and Clients for an admin user with cluster-admin clusterrole.
.. note::
For a Distributed Cloud system, navigate to **Project** \> **Central Cloud Regions** \> **RegionOne** \>
and download the **Openstack RC file**.
For a Distributed Cloud system, navigate to **Project** \> **Central
Cloud Regions** \> **RegionOne** \> and download the **Openstack RC
file**.
#. If HTTPS has been enabled for the |prod| RESTAPI Endpoints on your
|prod| system, add the following line to the bottom of admin-openrc.sh:
|prod| system, add the following line to the bottom of ``admin-openrc.sh``:
.. code-block:: none
OS_CACERT=<path_to_ca_>
where ``<path_to_ca>`` is the full filename of the PEM file for the CA
where ``<path_to_ca>`` is the full filename of the |PEM| file for the |CA|
Certificate that signed the |prod| REST APIs Endpoint Certificate.
Copy the file ``admin-openrc.sh`` to the remote workstation. This example
assumes it is copied to the location of the extracted tarball.

355
doc/source/security/kubernetes/starlingx-rest-api-applications-and-the-web-admin-server-cert-9196c5794834.rst
View File

@ -1,174 +1,193 @@
.. _starlingx-rest-api-applications-and-the-web-admin-server-cert-9196c5794834:
========================
Create a local CA Issuer
========================
======================
System Local CA Issuer
======================
|org| recommends that a single local |CA| Issuer be created on the platform to
create, sign, and anchor all of your platform certificates. This |CA| can be
either a stand-alone local Root |CA| or a local Intermediate |CA| (whose
certificate is signed by an external Root |CA|). This simplifies your overall
platform certificate configuration, and means that external clients interfacing
with the platform's HTTPS endpoints, need only be given a single Root |CA|
certificate to add to their trusted |CAs|.
At installation time, a ``system-local-ca`` ClusterIssuer is created. The
intent is that the ``system-local-ca`` can be the single root of trust for
Platform Certificates, such that external clients, using Platform APIs, need
only add the single ``system-local-ca`` public certificate to their list of
trusted |CAs| for the purpose of validating Platform server certificates.
At installation time, the ``cert-manager/system-local-ca`` TLS Secret, which is
used for CA Signing by the ``system-local-ca`` ClusterIssuer, is initially set
to the Kubernetes Root CA. At installation time, the Kubernetes Root |CA| is
either auto-generated or explicitly set thru bootstrap playbook overrides (see
:ref:`kubernetes-root-ca-certificate`).
In a Distributed Cloud System, by default, the Subclouds are deployed with the
same Kubernetes Root |CA| and the same ``system-local-ca`` as the
SystemController.
.. note::
In order to change or renew the ``system-local-ca`` Secret for signing, the
``migrate_platform_certificates_to_certmanager.yml`` playbook MUST BE USED,
see :ref:`migrate-platform-certificates-to-use-cert-manager-c0b1727e4e5d`.
This playbook will update the ``system-local-ca`` Secret and Issuer, re-sign
all of the Platform Certificates using this issuer, and in a Distributed
Cloud environment iterate through all of the Subclouds and do the same
updates and re-signing on each Subcloud.
Create the ClusterIssuer
========================
Create a local Root CA
----------------------
The following sample procedure illustrates how to create a unique standalone
local Root |CA| (``system-local-ca``) that can be used to create, sign, and
anchor all of your platform certificates.
Update the ``subject`` fields to identify your particular system.
It is recommended that a 3-5 year duration be used for operational simplicity
since, although the certificate will automatically renew locally, when it does
renew, you will need to re-distribute the |CA|'s new public certificate to all
external clients using the platform's HTTPS endpoints.
The created ``system-local-ca`` Root |CA| is cluster-wide, so it can be used to
create all platform certificates and can also be used for hosted applications'
certificates.
.. rubric:: |proc|
#. Create a cluster issuer yaml configuration file.
.. code-block:: none
~(keystone_admin)]$ cat <<EOF > cluster-issuer.yaml
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: system-selfsigning
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: system-local-ca
namespace: cert-manager
spec:
subject:
organizationalUnits:
- StarlingX-system-local-ca
secretName: system-local-ca
commonName: system-local-ca
isCA: true
duration: 43800h # 5 year
renewBefore: 720h # 30 days
issuerRef:
name: system-selfsigning
kind: ClusterIssuer
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: system-local-ca
spec:
ca:
secretName: system-local-ca
EOF
For more information on supported parameters, see
`https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1
<https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1>`__.
#. Apply the configuration.
.. code-block:: none
~(keystone_admin)]$ kubectl apply –f cluster-issuer.yaml
#. Verify the configuration.
.. code-block::
~(keystone_admin)]$ kubectl get clusterissuer
#. Write the public certificate of this |CA| to a ``.pem`` file that can be
distributed to external clients using the platform's HTTPS endpoints.
.. code-block::
~(keystone_admin)]$ kubectl get secret <secretname> -n <namespacename> -o=jsonpath='{.data.tls\.crt}' | base64 --decode > <pemfilename>
Create a local Intermediate CA
------------------------------
Alternatively, if you are using an external RootCA, the following procedure is
an example of how to create a local Intermediate |CA| (whose certificate is
signed by an external Intermediate or Root |CA|) that can be used to
create, sign, and anchor all of your platform certificates. Refer to the
documentation for your external Intermediate or Root |CA| for information on
how to create a public certificate and private key for an intermediate |CA|.
It is recommended that a 3-5 year duration be used for operational simplicity
since this certificate will need to be manually renewed with the externally
generated certificate and key, and then referenced via the ClusterIssuer's
``spec.ca.secretName``. The |TLS| secret must be created in the Cluster
Resource Namespace, which defaults to ``cert-manager`` on the platform.
The ``system-local-ca`` Root |CA| is cluster-wide, so it can be used to create
all platform certificates and can also be used for hosted applications'
certificates.
#. Copy the |PEM| encoded certificate and key from the externally generated
|CA| to the controller host.
.. note::
Ensure the certificates have RSA key length >= 2048 bits. The
|prod-long| Release |this-ver| provides a new version of ``openssl``
which requires a minimum of 2048-bit keys for RSA for better
security / encryption strength.
You can check the key length by running ``openssl x509 -in <the certificate file> -noout -text``
and looking for the "Public-Key" in the output. For more information see
:ref:`Create Certificates Locally using openssl <create-certificates-locally-using-openssl>`.
#. Create a |TLS| secret in ``cert-manager`` namespace with the certificate/Key
files:
.. code-block:: none
~(keystone_admin)]$ kubectl -n cert-manager create secret tls system-local-ca --cert=./cert.pem --key=./key.pem
#. Create ClusterIssuer and the |CA| certificate.
.. code-block:: none
~(keystone_admin)]$ cat <<EOF > cluster-issuer.yaml
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: system-local-ca
spec:
ca:
secretName: system-local-ca
EOF
#. Apply the configuration.
.. code-block:: none
~(keystone_admin)]$ kubectl apply –f cluster-issuer.yaml
#. Verify the configuration.
.. code-block::
~(keystone_admin)]$ kubectl get clusterissuer
If the configuration is successful, the clusterissuer for
``system-local-ca`` will have Ready status of ``True``.
The clusterissuer is now ready to issue certificates on the platform.
.. Create the ClusterIssuer
.. ========================
..
.. Create a local Root CA
.. ----------------------
..
.. The following sample procedure illustrates how to create a unique standalone
.. local Root |CA| (``system-local-ca``) that can be used to create, sign, and
.. anchor all of your platform certificates.
..
.. Update the ``subject`` fields to identify your particular system.
..
.. It is recommended that a 3-5 year duration be used for operational simplicity
.. since, although the certificate will automatically renew locally, when it does
.. renew, you will need to re-distribute the |CA|'s new public certificate to all
.. external clients using the platform's HTTPS endpoints.
..
.. The created ``system-local-ca`` Root |CA| is cluster-wide, so it can be used to
.. create all platform certificates and can also be used for hosted applications'
.. certificates.
..
.. .. rubric:: |proc|
..
.. #. Create a cluster issuer yaml configuration file.
..
.. .. code-block:: none
..
.. ~(keystone_admin)]$ cat <<EOF > cluster-issuer.yaml
.. ---
.. apiVersion: cert-manager.io/v1
.. kind: ClusterIssuer
.. metadata:
.. name: system-selfsigning
.. spec:
.. selfSigned: {}
.. ---
.. apiVersion: cert-manager.io/v1
.. kind: Certificate
.. metadata:
.. name: system-local-ca
.. namespace: cert-manager
.. spec:
.. subject:
.. organizationalUnits:
.. - StarlingX-system-local-ca
.. secretName: system-local-ca
.. commonName: system-local-ca
.. isCA: true
.. duration: 43800h # 5 year
.. renewBefore: 720h # 30 days
.. issuerRef:
.. name: system-selfsigning
.. kind: ClusterIssuer
.. ---
.. apiVersion: cert-manager.io/v1
.. kind: ClusterIssuer
.. metadata:
.. name: system-local-ca
.. spec:
.. ca:
.. secretName: system-local-ca
.. EOF
..
.. For more information on supported parameters, see
.. `https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1
.. <https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1>`__.
..
..
.. #. Apply the configuration.
..
.. .. code-block:: none
..
.. ~(keystone_admin)]$ kubectl apply –f cluster-issuer.yaml
..
.. #. Verify the configuration.
..
.. .. code-block::
..
.. ~(keystone_admin)]$ kubectl get clusterissuer
..
.. #. Write the public certificate of this |CA| to a ``.pem`` file that can be
.. distributed to external clients using the platform's HTTPS endpoints.
..
.. .. code-block::
..
.. ~(keystone_admin)]$ kubectl get secret <secretname> -n <namespacename> -o=jsonpath='{.data.tls\.crt}' | base64 --decode > <pemfilename>
..
.. Create a local Intermediate CA
.. ------------------------------
..
.. Alternatively, if you are using an external RootCA, the following procedure is
.. an example of how to create a local Intermediate |CA| (whose certificate is
.. signed by an external Intermediate or Root |CA|) that can be used to
.. create, sign, and anchor all of your platform certificates. Refer to the
.. documentation for your external Intermediate or Root |CA| for information on
.. how to create a public certificate and private key for an intermediate |CA|.
.. It is recommended that a 3-5 year duration be used for operational simplicity
.. since this certificate will need to be manually renewed with the externally
.. generated certificate and key, and then referenced via the ClusterIssuer's
.. ``spec.ca.secretName``. The |TLS| secret must be created in the Cluster
.. Resource Namespace, which defaults to ``cert-manager`` on the platform.
..
.. The ``system-local-ca`` Root |CA| is cluster-wide, so it can be used to create
.. all platform certificates and can also be used for hosted applications'
.. certificates.
..
.. #. Copy the |PEM| encoded certificate and key from the externally generated
.. CA to the controller host.
..
.. .. note::
..
.. Ensure the certificates have RSA key length >= 2048 bits. The
.. prod-long Release this-ver provides a new version of ``openssl``
.. which requires a minimum of 2048-bit keys for RSA for better
.. security / encryption strength.
..
.. You can check the key length by running ``openssl x509 -in <the certificate file> -noout -text``
.. and looking for the "Public-Key" in the output. For more information see
.. :ref:`Create Certificates Locally using openssl <create-certificates-locally-using-openssl>`.
..
.. #. Create a |TLS| secret in ``cert-manager`` namespace with the certificate/Key
.. files:
..
.. .. code-block:: none
..
.. ~(keystone_admin)]$ kubectl -n cert-manager create secret tls system-local-ca --cert=./cert.pem --key=./key.pem
..
.. #. Create ClusterIssuer and the |CA| certificate.
..
.. .. code-block:: none
..
.. ~(keystone_admin)]$ cat <<EOF > cluster-issuer.yaml
.. ---
.. apiVersion: cert-manager.io/v1
.. kind: ClusterIssuer
.. metadata:
.. name: system-local-ca
.. spec:
.. ca:
.. secretName: system-local-ca
.. EOF
..
.. #. Apply the configuration.
..
.. .. code-block:: none
..
.. ~(keystone_admin)]$ kubectl apply –f cluster-issuer.yaml
..
.. #. Verify the configuration.
..
.. .. code-block::
..
.. ~(keystone_admin)]$ kubectl get clusterissuer
..
.. If the configuration is successful, the clusterissuer for
.. ``system-local-ca`` will have Ready status of ``True``.
..
.. The clusterissuer is now ready to issue certificates on the platform.
..

42
doc/source/security/kubernetes/utility-script-to-display-certificates.rst
View File

@ -6,8 +6,10 @@
Display Certificates Installed on a System
------------------------------------------
The utility script **show-certs.sh** can be used to display an overview of the
various certificates that exist in the system along with their expiry date.
The script **show-certs.sh** can be used to display a list of the specific
certificates present on a |prod| system with details such as expiry
date, residual time, subject, issuer and renewal behaviour (manual or
automatic).
The :command:`show-certs.sh` command has the following options:
@ -21,13 +23,18 @@ renewal, and certificates expiring within 90 days.
options:
-k displays certificates found in any Kubernetes SECRETS;
this may include platform certificates and end-users' certificates
``-k`` displays certificates found in any Kubernetes SECRETS; this may include
platform certificates and end-users' certificates.
-e <number-of-days> changes to highlight (in red) certificates within
<number-of-days> of expiry
``-e`` <number-of-days>. Changes to highlight (in red) certificates within
<number-of-days> of expiry.
-h displays help
``-h`` displays help
.. note::
This command can only be run locally on the active controller, in an SSH
shell.
For example:
@ -46,9 +53,24 @@ For example:
Residual Time : 341d
-----------------------------------------------------
For scalability in a Distributed cloud system, the Subcloud ICA certificates
are redirected to a file. The script displays the path to the file with a note
at the end of the displayed output.
local-openldap / deployment / system-openldap-local-certificate CERTIFICATE:
------------------------------------------
Renewal : Automatic [Managed by Cert-Manager]
Namespace : deployment
Secret : system-openldap-local-certificate
Subject : CN = system-openldap
Issuer : CN = starlingx
Issue Date : Jul 6 16:15:30 2023 GMT
Expiry Date : Oct 4 16:15:30 2023 GMT
Residual Time : 89d
… etc
For scalability reasons, in a Distributed cloud system, the Subcloud ICA
certificates that are present on a SystemController are redirected to a file.
The script displays the path to the file with a note at the end of the
displayed output.
.. code-block:: none