From 1c68e4ef0869c1c6b10bca39ce96ffc6973c19f3 Mon Sep 17 00:00:00 2001 From: MCamp859 Date: Tue, 1 Jun 2021 14:35:52 -0400 Subject: [PATCH] Add R5 install guides to latest branch Replace the R5 install guides folder in latest branch. Review 793093 (which moved the R5 install guides) broke the downstream doc publishing process. Made some path changes in R5 Release Notes. Change-Id: Ia574c3da47e31036b2a2cbea06d43f0d1493945a Signed-off-by: MCamp859 --- doc/source/deploy_install_guides/index.rst | 32 +- .../r5_release/ansible_bootstrap_configs.rst | 422 ++++++++ .../r5_release/ansible_install_time_only.txt | 7 + ...erver-files-for-a-custom-configuration.rst | 60 ++ .../bare_metal/adding-hosts-in-bulk.rst | 61 ++ ...dding-hosts-using-the-host-add-command.rst | 164 +++ .../r5_release/bare_metal/aio_duplex.rst | 26 + .../bare_metal/aio_duplex_extend.rst | 311 ++++++ .../bare_metal/aio_duplex_hardware.rst | 58 ++ .../aio_duplex_install_kubernetes.rst | 953 ++++++++++++++++++ .../r5_release/bare_metal/aio_simplex.rst | 21 + .../bare_metal/aio_simplex_hardware.rst | 58 ++ .../aio_simplex_install_kubernetes.rst | 607 +++++++++++ ...rapping-from-a-private-docker-registry.rst | 52 + .../bare_metal/bulk-host-xml-file-format.rst | 135 +++ .../configuring-a-pxe-boot-server.rst | 200 ++++ .../bare_metal/controller_storage.rst | 22 + .../controller_storage_hardware.rst | 56 + .../controller_storage_install_kubernetes.rst | 799 +++++++++++++++ .../bare_metal/dedicated_storage.rst | 22 + .../bare_metal/dedicated_storage_hardware.rst | 61 ++ .../dedicated_storage_install_kubernetes.rst | 498 +++++++++ .../exporting-host-configurations.rst | 53 + .../inc-install-software-on-controller.rest | 43 + .../inc-openstack-specific-host-config.rest | 109 ++ .../r5_release/bare_metal/ironic.rst | 72 ++ .../r5_release/bare_metal/ironic_hardware.rst | 51 + .../r5_release/bare_metal/ironic_install.rst | 392 +++++++ .../r5_release/bare_metal/prep_servers.txt | 17 + .../reinstalling-a-system-or-a-host.rst | 39 + ...ng-an-exported-host-configuration-file.rst | 45 + .../r5_release/bare_metal/rook_storage.rst | 22 + .../bare_metal/rook_storage_hardware.rst | 62 ++ .../rook_storage_install_kubernetes.rst | 731 ++++++++++++++ .../r5_release/desc_aio_duplex.txt | 29 + .../r5_release/desc_aio_simplex.txt | 24 + .../r5_release/desc_controller_storage.txt | 28 + .../r5_release/desc_dedicated_storage.txt | 24 + .../r5_release/desc_rook_storage.txt | 22 + .../r5_release/distributed_cloud/index.rst | 310 ++++++ .../install_virtualbox_configparms.png | Bin 0 -> 13447 bytes .../figures/install_virtualbox_guiscreen.png | Bin 0 -> 23175 bytes .../starlingx-access-openstack-command.png | Bin 0 -> 366267 bytes .../starlingx-access-openstack-flavorlist.png | Bin 0 -> 447605 bytes ...-deployment-options-controller-storage.png | Bin 0 -> 98773 bytes ...x-deployment-options-dedicated-storage.png | Bin 0 -> 111169 bytes ...x-deployment-options-distributed-cloud.png | Bin 0 -> 320078 bytes ...ngx-deployment-options-duplex-extended.png | Bin 0 -> 101986 bytes .../starlingx-deployment-options-duplex.png | Bin 0 -> 103883 bytes .../starlingx-deployment-options-ironic.png | Bin 0 -> 129791 bytes .../starlingx-deployment-options-simplex.png | Bin 0 -> 72126 bytes .../r5_release/index.rst | 128 +++ .../r5_release/ipv6_note.txt | 14 + .../r5_release/kubernetes_access.rst | 188 ++++ .../r5_release/kubernetes_install_next.txt | 7 + .../r5_release/openstack/access.rst | 329 ++++++ .../r5_release/openstack/index.rst | 16 + .../r5_release/openstack/install.rst | 98 ++ .../r5_release/openstack/uninstall_delete.rst | 33 + .../r5_release/virtual/aio_duplex.rst | 21 + .../r5_release/virtual/aio_duplex_environ.rst | 57 ++ .../virtual/aio_duplex_install_kubernetes.rst | 578 +++++++++++ .../r5_release/virtual/aio_simplex.rst | 21 + .../virtual/aio_simplex_environ.rst | 55 + .../aio_simplex_install_kubernetes.rst | 414 ++++++++ .../virtual/config_virtualbox_netwk.rst | 161 +++ .../r5_release/virtual/controller_storage.rst | 21 + .../virtual/controller_storage_environ.rst | 59 ++ .../controller_storage_install_kubernetes.rst | 593 +++++++++++ .../r5_release/virtual/dedicated_storage.rst | 21 + .../virtual/dedicated_storage_environ.rst | 61 ++ .../dedicated_storage_install_kubernetes.rst | 395 ++++++++ .../r5_release/virtual/install_virtualbox.rst | 366 +++++++ .../r5_release/virtual/physical_host_req.txt | 72 ++ .../r5_release/virtual/rook_storage.rst | 21 + .../virtual/rook_storage_environ.rst | 61 ++ .../rook_storage_install_kubernetes.rst | 539 ++++++++++ doc/source/releasenotes/r5_release.rst | 5 +- 78 files changed, 11018 insertions(+), 14 deletions(-) create mode 100644 doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst create mode 100644 doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/accessing-pxe-boot-server-files-for-a-custom-configuration.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-in-bulk.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-using-the-host-add-command.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/bootstrapping-from-a-private-docker-registry.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/bulk-host-xml-file-format.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/configuring-a-pxe-boot-server.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/exporting-host-configurations.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/inc-install-software-on-controller.rest create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/inc-openstack-specific-host-config.rest create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-or-a-host.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-using-an-exported-host-configuration-file.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_hardware.rst create mode 100644 doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt create mode 100644 doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt create mode 100644 doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt create mode 100644 doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt create mode 100644 doc/source/deploy_install_guides/r5_release/desc_rook_storage.txt create mode 100644 doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst create mode 100644 doc/source/deploy_install_guides/r5_release/figures/install_virtualbox_configparms.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/install_virtualbox_guiscreen.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-access-openstack-command.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-access-openstack-flavorlist.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-controller-storage.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-dedicated-storage.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-distributed-cloud.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-duplex-extended.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-duplex.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-ironic.png create mode 100644 doc/source/deploy_install_guides/r5_release/figures/starlingx-deployment-options-simplex.png create mode 100644 doc/source/deploy_install_guides/r5_release/index.rst create mode 100644 doc/source/deploy_install_guides/r5_release/ipv6_note.txt create mode 100644 doc/source/deploy_install_guides/r5_release/kubernetes_access.rst create mode 100644 doc/source/deploy_install_guides/r5_release/kubernetes_install_next.txt create mode 100644 doc/source/deploy_install_guides/r5_release/openstack/access.rst create mode 100644 doc/source/deploy_install_guides/r5_release/openstack/index.rst create mode 100644 doc/source/deploy_install_guides/r5_release/openstack/install.rst create mode 100644 doc/source/deploy_install_guides/r5_release/openstack/uninstall_delete.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/aio_duplex.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/aio_duplex_environ.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/aio_duplex_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/aio_simplex.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/aio_simplex_environ.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/aio_simplex_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/config_virtualbox_netwk.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/controller_storage.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/controller_storage_environ.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/controller_storage_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage_environ.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage_install_kubernetes.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/install_virtualbox.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/physical_host_req.txt create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/rook_storage.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/rook_storage_environ.rst create mode 100644 doc/source/deploy_install_guides/r5_release/virtual/rook_storage_install_kubernetes.rst diff --git a/doc/source/deploy_install_guides/index.rst b/doc/source/deploy_install_guides/index.rst index 4ef7ea5e7..506b8588e 100755 --- a/doc/source/deploy_install_guides/index.rst +++ b/doc/source/deploy_install_guides/index.rst @@ -18,15 +18,25 @@ StarlingX R6.0 is under development. r6_release/index -------------------------------- -Supported and archived releases -------------------------------- +----------------- +Supported release +----------------- StarlingX R5.0 is the most recent supported release of StarlingX. -To view the R5.0 documentation, use the **Version** selector in the upper right -or go directly to `Installation guides for R5.0 and older releases -`_. +.. toctree:: + :maxdepth: 1 + + r5_release/index + + +----------------- +Archived releases +----------------- + +To view the archived installation guides, see `Installation guides for R5.0 and +older releases `_. + @@ -40,13 +50,13 @@ or go directly to `Installation guides for R5.0 and older releases nvme_config .. Docs note: Starting with R5 (May 2021), team agreed that the latest/working - branch will include the current install guides only. The archived releases - will only be available in a release-specific branch. The instructions below - are modified to reflect this change. + branch will include the current & supported install guides only. The archived + releases will only be available in a release-specific branch. The + instructions below are modified to reflect this change. .. Making a new release -.. 1. Make the previous 'upcoming' release the new 'supported' release. - Copy the folder to the release-specific branch. +.. 1. Copy the previous 'upcoming' release to the 'supported' release section. + Copy the old 'supported' folder to the release-specific branch. Copy the toctree link into the Supported section of install landing page. Update intro text for the Supported release section to use the latest version. diff --git a/doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst b/doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst new file mode 100644 index 000000000..9db0a54c1 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/ansible_bootstrap_configs.rst @@ -0,0 +1,422 @@ +.. _ansible_bootstrap_configs_r5: + +================================ +Ansible Bootstrap Configurations +================================ + +This section describes Ansible bootstrap configuration options. + +.. contents:: + :local: + :depth: 1 + + +.. _install-time-only-params-r5: + +---------------------------- +Install-time-only parameters +---------------------------- + +Some Ansible bootstrap parameters can not be changed or are very difficult to +change after installation is complete. + +Review the set of install-time-only parameters before installation and confirm +that your values for these parameters are correct for the desired installation. + +.. note:: + + If you notice an incorrect install-time-only parameter value *before you + unlock controller-0 for the first time*, you can re-run the Ansible bootstrap + playbook with updated override values and the updated values will take effect. + +**************************** +Install-time-only parameters +**************************** + +**System Properties** + +* ``system_mode`` +* ``distributed_cloud_role`` + +**Network Properties** + +* ``pxeboot_subnet`` +* ``pxeboot_start_address`` +* ``pxeboot_end_address`` +* ``management_subnet`` +* ``management_start_address`` +* ``management_end_address`` +* ``cluster_host_subnet`` +* ``cluster_host_start_address`` +* ``cluster_host_end_address`` +* ``cluster_pod_subnet`` +* ``cluster_pod_start_address`` +* ``cluster_pod_end_address`` +* ``cluster_service_subnet`` +* ``cluster_service_start_address`` +* ``cluster_service_end_address`` +* ``management_multicast_subnet`` +* ``management_multicast_start_address`` +* ``management_multicast_end_address`` + +**Docker Proxies** + +* ``docker_http_proxy`` +* ``docker_https_proxy`` +* ``docker_no_proxy`` + +**Docker Registry Overrides** + +* ``docker_registries`` + + * ``k8s.gcr.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``gcr.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``quay.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``docker.io`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``docker.elastic.co`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + + * ``defaults`` + + * ``url`` + * ``username`` + * ``password`` + * ``secure`` + +**Certificates** + +* ``k8s_root_ca_cert`` +* ``k8s_root_ca_key`` + +**Kubernetes Parameters** + +* ``apiserver_oidc`` + +---- +IPv6 +---- + +If you are using IPv6, provide IPv6 configuration overrides for the Ansible +bootstrap playbook. Note that all addressing, except pxeboot_subnet, should be +updated to IPv6 addressing. + +Example IPv6 override values are shown below: + +:: + + dns_servers: + ‐ 2001:4860:4860::8888 + ‐ 2001:4860:4860::8844 + pxeboot_subnet: 169.254.202.0/24 + management_subnet: 2001:db8:2::/64 + cluster_host_subnet: 2001:db8:3::/64 + cluster_pod_subnet: 2001:db8:4::/64 + cluster_service_subnet: 2001:db8:4::/112 + external_oam_subnet: 2001:db8:1::/64 + external_oam_gateway_address: 2001:db8::1 + external_oam_floating_address: 2001:db8::2 + external_oam_node_0_address: 2001:db8::3 + external_oam_node_1_address: 2001:db8::4 + management_multicast_subnet: ff08::1:1:0/124 + +.. note:: + + The `external_oam_node_0_address`, and `external_oam_node_1_address` parameters + are not required for the AIO‐SX installation. + +---------------- +Private registry +---------------- + +To bootstrap StarlingX you must pull container images for multiple system +services. By default these container images are pulled from public registries: +k8s.gcr.io, gcr.io, quay.io, and docker.io. + +It may be required (or desired) to copy the container images to a private +registry and pull the images from the private registry (instead of the public +registries) as part of the StarlingX bootstrap. For example, a private registry +would be required if a StarlingX system was deployed in an air-gapped network +environment. + +Use the `docker_registries` structure in the bootstrap overrides file to specify +alternate registry(s) for the public registries from which container images are +pulled. These alternate registries are used during the bootstrapping of +controller-0, and on :command:`system application-apply` of application packages. + +The `docker_registries` structure is a map of public registries and the +alternate registry values for each public registry. For each public registry the +key is a fully scoped registry name of a public registry (for example "k8s.gcr.io") +and the alternate registry URL and username/password (if authenticated). + +url + The fully scoped registry name (and optionally namespace/) for the alternate + registry location where the images associated with this public registry + should now be pulled from. + + Valid formats for the `url` value are: + + * Domain. For example: + + :: + + example.domain + + * Domain with port. For example: + + :: + + example.domain:5000 + + * IPv4 address. For example: + + :: + + 1.2.3.4 + + * IPv4 address with port. For example: + + :: + + 1.2.3.4:5000 + + * IPv6 address. For example: + + :: + + FD01::0100 + + * IPv6 address with port. For example: + + :: + + [FD01::0100]:5000 + +username + The username for logging into the alternate registry, if authenticated. + +password + The password for logging into the alternate registry, if authenticated. + + +Additional configuration options in the `docker_registries` structure are: + +defaults + A special public registry key which defines common values to be applied to + all overrideable public registries. If only the `defaults` registry + is defined, it will apply `url`, `username`, and `password` for all + registries. + + If values under specific registries are defined, they will override the + values defined in the defaults registry. + + .. note:: + + The `defaults` key was formerly called `unified`. It was renamed + in StarlingX R3.0 and updated semantics were applied. + + This change affects anyone with a StarlingX installation prior to R3.0 that + specifies alternate Docker registries using the `unified` key. + +secure + Specifies whether the registry(s) supports HTTPS (secure) or HTTP (not secure). + Applies to all alternate registries. A boolean value. The default value is + True (secure, HTTPS). + +.. note:: + + The ``secure`` parameter was formerly called ``is_secure_registry``. It was + renamed in StarlingX R3.0. + +If an alternate registry is specified to be secure (using HTTPS), the certificate +used by the registry may not be signed by a well-known Certificate Authority (CA). +This results in the :command:`docker pull` of images from this registry to fail. +Use the `ssl_ca_cert` override to specify the public certificate of the CA that +signed the alternate registry’s certificate. This will add the CA as a trusted +CA to the StarlingX system. + +ssl_ca_cert + The `ssl_ca_cert` value is the absolute path of the certificate file. The + certificate must be in PEM format and the file may contain a single CA + certificate or multiple CA certificates in a bundle. + +The following example will apply `url`, `username`, and `password` to all +registries. + +:: + + docker_registries: + defaults: + url: my.registry.io + username: myreguser + password: myregP@ssw0rd + +The next example applies `username` and `password` from the defaults registry +to all public registries. `url` is different for each public registry. It +additionally specifies an alternate CA certificate. + +:: + + docker_registries: + k8s.gcr.io: + url: my.k8sregistry.io + gcr.io: + url: my.gcrregistry.io + quay.io: + url: my.quayregistry.io + docker.io: + url: my.dockerregistry.io + defaults: + url: my.registry.io + username: myreguser + password: myregP@ssw0rd + + ssl_ca_cert: /path/to/ssl_ca_cert_file + +------------ +Docker proxy +------------ + +If the StarlingX OAM interface or network is behind a http/https proxy, relative +to the Docker registries used by StarlingX or applications running on StarlingX, +then Docker within StarlingX must be configured to use these http/https proxies. + +Use the following configuration overrides to configure your Docker proxy settings. + +docker_http_proxy + Specify the HTTP proxy URL to use. For example: + + :: + + docker_http_proxy: http://my.proxy.com:1080 + +docker_https_proxy + Specify the HTTPS proxy URL to use. For example: + + :: + + docker_https_proxy: https://my.proxy.com:1443 + +docker_no_proxy + A no-proxy address list can be provided for registries not on the other side + of the proxies. This list will be added to the default no-proxy list derived + from localhost, loopback, management, and OAM floating addresses at run time. + Each address in the no-proxy list must neither contain a wildcard nor have + subnet format. For example: + + :: + + docker_no_proxy: + - 1.2.3.4 + - 5.6.7.8 + +.. _k8s-root-ca-cert-key-r5: + +-------------------------------------- +Kubernetes root CA certificate and key +-------------------------------------- + +By default the Kubernetes Root CA Certificate and Key are auto-generated and +result in the use of self-signed certificates for the Kubernetes API server. In +the case where self-signed certificates are not acceptable, use the bootstrap +override values `k8s_root_ca_cert` and `k8s_root_ca_key` to specify the +certificate and key for the Kubernetes root CA. + +k8s_root_ca_cert + Specifies the certificate for the Kubernetes root CA. The `k8s_root_ca_cert` + value is the absolute path of the certificate file. The certificate must be + in PEM format and the value must be provided as part of a pair with + `k8s_root_ca_key`. The playbook will not proceed if only one value is provided. + +k8s_root_ca_key + Specifies the key for the Kubernetes root CA. The `k8s_root_ca_key` + value is the absolute path of the certificate file. The certificate must be + in PEM format and the value must be provided as part of a pair with + `k8s_root_ca_cert`. The playbook will not proceed if only one value is provided. + +.. important:: + + The default length for the generated Kubernetes root CA certificate is 10 + years. Replacing the root CA certificate is an involved process so the custom + certificate expiry should be as long as possible. We recommend ensuring root + CA certificate has an expiry of at least 5-10 years. + +The administrator can also provide values to add to the Kubernetes API server +certificate Subject Alternative Name list using the 'apiserver_cert_sans` +override parameter. + +apiserver_cert_sans + Specifies a list of Subject Alternative Name entries that will be added to the + Kubernetes API server certificate. Each entry in the list must be an IP address + or domain name. For example: + + :: + + apiserver_cert_sans: + - hostname.domain + - 198.51.100.75 + +StarlingX automatically updates this parameter to include IP records for the OAM +floating IP and both OAM unit IP addresses. + +---------------------------------------------------- +OpenID Connect authentication for Kubernetes cluster +---------------------------------------------------- + +The Kubernetes cluster can be configured to use an external OpenID Connect +:abbr:`IDP (identity provider)`, such as Azure Active Directory, Salesforce, or +Google, for Kubernetes API authentication. + +By default, OpenID Connect authentication is disabled. To enable OpenID Connect, +use the following configuration values in the Ansible bootstrap overrides file +to specify the IDP for OpenID Connect: + +:: + + apiserver_oidc: + client_id: + issuer_url: + username_claim: + +When the three required fields of the `apiserver_oidc` parameter are defined, +OpenID Connect is considered active. The values will be used to configure the +Kubernetes cluster to use the specified external OpenID Connect IDP for +Kubernetes API authentication. + +In addition, you will need to configure the external OpenID Connect IDP and any +required OpenID client application according to the specific IDP's documentation. + +If not configuring OpenID Connect, all values should be absent from the +configuration file. + +.. note:: + + Default authentication via service account tokens is always supported, + even when OpenID Connect authentication is configured. \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt b/doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt new file mode 100644 index 000000000..a6cacd159 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/ansible_install_time_only.txt @@ -0,0 +1,7 @@ +.. important:: + + Some Ansible bootstrap parameters cannot be changed or are very difficult to change after installation is complete. + + Review the set of install-time-only parameters before installation and confirm that your values for these parameters are correct for the desired installation. + + Refer to :ref:`Ansible install-time-only parameters ` for details. \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/accessing-pxe-boot-server-files-for-a-custom-configuration.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/accessing-pxe-boot-server-files-for-a-custom-configuration.rst new file mode 100644 index 000000000..7210022c6 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/accessing-pxe-boot-server-files-for-a-custom-configuration.rst @@ -0,0 +1,60 @@ + +.. jow1442253584837 +.. _accessing-pxe-boot-server-files-for-a-custom-configuration: + +======================================================= +Access PXE Boot Server Files for a Custom Configuration +======================================================= + +If you prefer, you can create a custom |PXE| boot configuration using the +installation files provided with |prod|. + +.. rubric:: |context| + +You can use the setup script included with the ISO image to copy the boot +configuration files and distribution content to a working directory. You can +use the contents of the working directory to construct a |PXE| boot environment +according to your own requirements or preferences. + +For more information about using a |PXE| boot server, see :ref:`Configure a +PXE Boot Server `. + +.. rubric:: |proc| + +.. _accessing-pxe-boot-server-files-for-a-custom-configuration-steps-www-gcz-3t: + +#. Copy the ISO image from the source \(product DVD, USB device, or + |dnload-loc|\) to a temporary location on the |PXE| boot server. + + This example assumes that the copied image file is + tmp/TS-host-installer-1.0.iso. + +#. Mount the ISO image and make it executable. + + .. code-block:: none + + $ mount -o loop /tmp/TS-host-installer-1.0.iso /media/iso + $ mount -o remount,exec,dev /media/iso + +#. Create and populate a working directory. + + Use a command of the following form: + + .. code-block:: none + + $ /media/iso/pxeboot_setup.sh -u http:/// <-w > + + where: + + **ip-addr** + is the Apache listening address. + + **symlink** + is a name for a symbolic link to be created under the Apache document + root directory, pointing to the directory specified by . + + **working-dir** + is the path to the working directory. + +#. Copy the required files from the working directory to your custom |PXE| + boot server directory. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-in-bulk.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-in-bulk.rst new file mode 100644 index 000000000..19377ec3c --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-in-bulk.rst @@ -0,0 +1,61 @@ + +.. ulc1552927930507 +.. _adding-hosts-in-bulk: + +================= +Add Hosts in Bulk +================= + +You can add an arbitrary number of hosts using a single CLI command. + +.. rubric:: |proc| + +#. Prepare an XML file that describes the hosts to be added. + + For more information, see :ref:`Bulk Host XML File Format + `. + + You can also create the XML configuration file from an existing, running + configuration using the :command:`system host-bulk-export` command. + +#. Run the :command:`system host-bulk-add` utility. + + The command syntax is: + + .. code-block:: none + + ~[keystone_admin]$ system host-bulk-add + + where is the name of the prepared XML file. + +#. Power on the hosts to be added, if required. + + .. note:: + Hosts can be powered on automatically from board management controllers + using settings in the XML file. + +.. rubric:: |result| + +The hosts are configured. The utility provides a summary report, as shown in +the following example: + +.. code-block:: none + + Success: + worker-0 + worker-1 + Error: + controller-1: Host-add Rejected: Host with mgmt_mac 08:00:28:A9:54:19 already exists + +.. rubric:: |postreq| + +After adding the host, you must provision it according to the requirements of +the personality. + +.. xbooklink For more information, see :ref:`Installing, Configuring, and + Unlocking Nodes `, for your system, + and follow the *Configure* steps for the appropriate node personality. + +.. seealso:: + + :ref:`Bulk Host XML File Format ` diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-using-the-host-add-command.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-using-the-host-add-command.rst new file mode 100644 index 000000000..bc8099f5a --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/adding-hosts-using-the-host-add-command.rst @@ -0,0 +1,164 @@ + +.. pyp1552927946441 +.. _adding-hosts-using-the-host-add-command: + +==================================== +Add Hosts Using the host-add Command +==================================== + +You can add hosts to the system inventory using the command line. + +.. rubric:: |context| + +There are several ways to add hosts to |prod|; for an overview, see the +StarlingX Installation Guides, +`https://docs.starlingx.io/deploy_install_guides/index.html +`__ for your +system. Instead of powering up each host and then defining its personality and +other characteristics interactively, you can use the :command:`system host-add` +command to define hosts before you power them up. This can be useful for +scripting an initial setup. + +.. note:: + On systems that use static IP address assignment on the management network, + new hosts must be added to the inventory manually and assigned an IP + address using the :command:`system host-add` command. If a host is not + added successfully, the host console displays the following message at + power-on: + + .. code-block:: none + + This system has been configured with static management + and infrastructure IP address allocation. This requires + that the node be manually provisioned in System + Inventory using the 'system host-add' CLI, GUI, or + stx API equivalent. + +.. rubric:: |proc| + +#. Add the host to the system inventory. + + .. note:: + The host must be added to the system inventory before it is powered on. + + On **controller-0**, acquire Keystone administrative privileges: + + .. code-block:: none + + $ source /etc/platform/openrc + + Use the :command:`system host-add` command to add a host and specify its + personality. You can also specify the device used to display messages + during boot. + + .. note:: + The hostname parameter is required for worker hosts. For controller and + storage hosts, it is ignored. + + .. code-block:: none + + ~(keystone_admin)]$ system host-add -n \ + -p [-s ] \ + [-l ] [-o [-c ]] [-b ] \ + [-r ] [-m ] [-i ] [-D ] \ + [-T -I -U -P ] + + + where + + **** + is a name to assign to the host. This is used for worker nodes only. + Controller and storage node names are assigned automatically and + override user input. + + **** + is the host type. The following are valid values: + + - controller + + - worker + + - storage + + **** + are the host personality subfunctions \(used only for a worker host\). + + For a worker host, the only valid value is worker,lowlatency to enable + a low-latency performance profile. For a standard performance profile, + omit this option. + + For more information about performance profiles, see |deploy-doc|: + :ref:`Worker Function Performance Profiles + `. + + **** + is a string describing the location of the host + + **** + is the output device to use for message display on the host \(for + example, tty0\). The default is ttys0, 115200. + + **** + is the format for console output on the host \(text or graphical\). The + default is text. + + .. note:: + The graphical option currently has no effect. Text-based + installation is used regardless of this setting. + + **** + is the host device for boot partition, relative to /dev. The default is + sda. + + **** + is the host device for rootfs partition, relative to/dev. The default + is sda. + + **** + is the |MAC| address of the port connected to the internal management + or |PXE| boot network. + + **** + is the IP address of the port connected to the internal management or + |PXE| boot network, if static IP address allocation is used. + + .. note:: + The option is not used for a controller node. + + **** + is set to **True** to have any active console session automatically + logged out when the serial console cable is disconnected, or **False** + to disable this behavior. The server must support data carrier detect + on the serial console port. + + **** + is the board management controller type. Use bmc. + + **** + is the board management controller IP address \(used for external + access to board management controllers over the |OAM| network\) + + **** + is the username for board management controller access + + **** + is the password for board management controller access + + For example: + + .. code-block:: none + + ~(keystone_admin)]$ system host-add -n compute-0 -p worker -I 10.10.10.100 + +#. With **controller-0** running, start the host. + + The host is booted and configured with a personality. + +.. rubric:: |postreq| + +After adding the host, you must provision it according to the requirements of +the personality. + +.. xbooklink For more information, see :ref:`Install, Configure, and Unlock + Nodes ` and follow the *Configure* + steps for the appropriate node personality. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst new file mode 100644 index 000000000..376decf6f --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex.rst @@ -0,0 +1,26 @@ +============================================== +Bare metal All-in-one Duplex Installation R5.0 +============================================== + +-------- +Overview +-------- + +.. include:: ../desc_aio_duplex.txt + +The bare metal AIO-DX deployment configuration may be extended with up to four +worker nodes (not shown in the diagram). Installation instructions for +these additional nodes are described in :doc:`aio_duplex_extend`. + +.. include:: ../ipv6_note.txt + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + aio_duplex_hardware + aio_duplex_install_kubernetes + aio_duplex_extend \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst new file mode 100644 index 000000000..0fbc56636 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst @@ -0,0 +1,311 @@ +================================= +Extend Capacity with Worker Nodes +================================= + +This section describes the steps to extend capacity with worker nodes on a +|prod| All-in-one Duplex deployment configuration. + +.. contents:: + :local: + :depth: 1 + +-------------------------------- +Install software on worker nodes +-------------------------------- + +#. Power on the worker node servers and force them to network boot with the + appropriate BIOS boot options for your particular server. + +#. As the worker nodes boot, a message appears on their console instructing + you to configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered worker + node hosts (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-0 | controller | unlocked | enabled | available | + | 3 | None | None | locked | disabled | offline | + | 4 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'worker': + + :: + + system host-update 3 personality=worker hostname=worker-0 + system host-update 4 personality=worker hostname=worker-1 + + This initiates the install of software on worker nodes. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. Wait for the install of software on the worker nodes to complete, for the + worker nodes to reboot, and for both to show as locked/disabled/online in + 'system host-list'. + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + | 3 | worker-0 | worker | locked | disabled | online | + | 4 | worker-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure worker nodes +---------------------- + +#. The MGMT interfaces are partially set up by the network install procedure; + configuring the port used for network install as the MGMT port and + specifying the attached network of "mgmt". + + Complete the MGMT interface configuration of the worker nodes by specifying + the attached network of "cluster-host". + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + + #. **For OpenStack only:** Configure the host settings for the vSwitch. + + **If using OVS-DPDK vswitch, run the following commands:** + + Default recommendation for worker node is to use a single core on each + numa-node for |OVS|-|DPDK| vswitch. This should have been automatically + configured, if not run the following command. + + :: + + for NODE in worker-0 worker-1; do + + # assign 1 core on processor/numa-node 0 on worker-node to vswitch + system host-cpu-modify -f vswitch -p0 1 $NODE + + # assign 1 core on processor/numa-node 1 on worker-node to vswitch + system host-cpu-modify -f vswitch -p1 1 $NODE + + done + + + When using |OVS|-|DPDK|, configure 1x 1G huge page for vSwitch memory on + each |NUMA| node where vswitch is running on this host, with the + following command: + + :: + + for NODE in worker-0 worker-1; do + + # assign 1x 1G huge page on processor/numa-node 0 on worker-node to vswitch + system host-memory-modify -f vswitch -1G 1 $NODE 0 + + # assign 1x 1G huge page on processor/numa-node 0 on worker-node to vswitch + system host-memory-modify -f vswitch -1G 1 $NODE 1 + + done + + + .. important:: + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment for + this host with the command: + + :: + + for NODE in worker-0 worker-1; do + + # assign 10x 1G huge page on processor/numa-node 0 on worker-node to applications + system host-memory-modify -f application -1G 10 $NODE 0 + + # assign 10x 1G huge page on processor/numa-node 1 on worker-node to applications + system host-memory-modify -f application -1G 10 $NODE 1 + + done + + #. **For OpenStack only:** Setup disk partition for nova-local volume group, + needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + + #. **For OpenStack only:** Configure data interfaces for worker nodes. + Data class interfaces are vswitch interfaces used by vswitch to provide + VM virtio vNIC connectivity to OpenStack Neutron Tenant Networks on the + underlying assigned Data Network. + + .. important:: + + A compute-labeled worker host **MUST** have at least one Data class interface. + + * Configure the data interfaces for worker nodes. + + :: + + # Execute the following lines with + export NODE=worker-0 + # and then repeat with + export NODE=worker-1 + + # List inventoried host’s ports and identify ports to be used as ‘data’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘data’ class interfaces, MTU of 1500 and named data# + system host-if-modify -m 1500 -n data0 -c data ${NODE} + system host-if-modify -m 1500 -n data1 -c data ${NODE} + + # Create Data Networks that vswitch 'data' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to Data Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + +***************************************** +Optionally Configure PCI-SRIOV Interfaces +***************************************** + +#. **Optionally**, configure pci-sriov interfaces for worker nodes. + + This step is **optional** for Kubernetes. Do this step if using |SRIOV| + network attachments in hosted application containers. + + .. only:: openstack + + This step is **optional** for OpenStack. Do this step if using |SRIOV| + vNICs in hosted application VMs. Note that pci-sriov interfaces can + have the same Data Networks assigned to them as vswitch data interfaces. + + + * Configure the pci-sriov interfaces for worker nodes. + + :: + + # Execute the following lines with + export NODE=worker-0 + # and then repeat with + export NODE=worker-1 + + # List inventoried host’s ports and identify ports to be used as ‘pci-sriov’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘pci-sriov’ class interfaces, MTU of 1500 and named sriov# + system host-if-modify -m 1500 -n sriov0 -c pci-sriov ${NODE} + system host-if-modify -m 1500 -n sriov1 -c pci-sriov ${NODE} + + # Create Data Networks that the 'pci-sriov' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to PCI-SRIOV Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + + * To enable using |SRIOV| network attachments for the above interfaces in + Kubernetes hosted application containers: + + * Configure the Kubernetes |SRIOV| device plugin. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE sriovdp=enabled + done + + * If planning on running |DPDK| in Kubernetes hosted application + containers on this host, configure the number of 1G Huge pages required + on both |NUMA| nodes. + + :: + + for NODE in worker-0 worker-1; do + + # assign 10x 1G huge page on processor/numa-node 0 on worker-node to applications + system host-memory-modify -f application $NODE 0 -1G 10 + + # assign 10x 1G huge page on processor/numa-node 1 on worker-node to applications + system host-memory-modify -f application $NODE 1 -1G 10 + + done + + +------------------- +Unlock worker nodes +------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst new file mode 100644 index 000000000..5b8f062d9 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_hardware.rst @@ -0,0 +1,58 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal All-in-one Duplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++-------------------------+-----------------------------------------------------------+ +| Minimum Requirement | All-in-one Controller Node | ++=========================+===========================================================+ +| Number of servers | 2 | ++-------------------------+-----------------------------------------------------------+ +| Minimum processor class | - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) | +| | 8 cores/socket | +| | | +| | or | +| | | +| | - Single-CPU Intel® Xeon® D-15xx family, 8 cores | +| | (low-power/low-cost option) | ++-------------------------+-----------------------------------------------------------+ +| Minimum memory | 64 GB | ++-------------------------+-----------------------------------------------------------+ +| Primary disk | 500 GB SSD or NVMe (see :doc:`../../nvme_config`) | ++-------------------------+-----------------------------------------------------------+ +| Additional disks | - 1 or more 500 GB (min. 10K RPM) for Ceph OSD | +| | - Recommended, but not required: 1 or more SSDs or NVMe | +| | drives for Ceph journals (min. 1024 MiB per OSD journal)| +| | - For OpenStack, recommend 1 or more 500 GB (min. 10K RPM)| +| | for VM local ephemeral storage | ++-------------------------+-----------------------------------------------------------+ +| Minimum network ports | - Mgmt/Cluster: 1x10GE | +| | - OAM: 1x1GE | +| | - Data: 1 or more x 10GE | ++-------------------------+-----------------------------------------------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++-------------------------+-----------------------------------------------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst new file mode 100644 index 000000000..6af949b87 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_install_kubernetes.rst @@ -0,0 +1,953 @@ + +.. _aio_duplex_install_kubernetes: + +================================================ +Install Kubernetes Platform on All-in-one Duplex +================================================ + +.. only:: partner + + .. include:: /_includes/install-kubernetes-null-labels.rest + +.. only:: starlingx + + This section describes the steps to install the StarlingX Kubernetes + platform on a **StarlingX R5.0 All-in-one Duplex** deployment + configuration. + + .. contents:: + :local: + :depth: 1 + + --------------------- + Create a bootable USB + --------------------- + + Refer to :ref:`Bootable USB ` for instructions on how + to create a bootable USB with the StarlingX ISO on your system. + + -------------------------------- + Install software on controller-0 + -------------------------------- + + .. include:: inc-install-software-on-controller.rest + :start-after: incl-install-software-controller-0-aio-start + :end-before: incl-install-software-controller-0-aio-end + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +#. Login using the username / password of "sysadmin" / "sysadmin". + When logging in for the first time, you will be forced to change the password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will |DHCP| out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a |DHCP| server + is present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for + Ansible configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: + ``$HOME/.yml``. + + .. only:: starlingx + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file + comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment configuration + as shown in the example below. Use the OAM IP SUBNET and IP ADDRESSing + applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: duplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + external_oam_node_0_address: + external_oam_node_1_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + EOF + + + .. only:: starlingx + + In either of the above options, the bootstrap playbook’s default values + will pull all container images required for the |prod-p| from Docker hub. + + If you have setup a private Docker registry to use for bootstrapping + then you will need to add the following lines in $HOME/localhost.yml: + + .. only:: partner + + .. include:: /_includes/install-kubernetes-bootstrap-playbook.rest + :start-after: docker-reg-begin + :end-before: docker-reg-end + + .. code-block:: + + docker_registries: + quay.io: + url: myprivateregistry.abc.com:9001/quay.io + docker.elastic.co: + url: myprivateregistry.abc.com:9001/docker.elastic.co + gcr.io: + url: myprivateregistry.abc.com:9001/gcr.io + k8s.gcr.io: + url: myprivateregistry.abc.com:9001/k8s.gcr.io + docker.io: + url: myprivateregistry.abc.com:9001/docker.io + defaults: + type: docker + username: + password: + + # Add the CA Certificate that signed myprivateregistry.abc.com’s + # certificate as a Trusted CA + ssl_ca_cert: /home/sysadmin/myprivateregistry.abc.com-ca-cert.pem + + See :ref:`Use a Private Docker Registry ` + for more information. + + .. only:: starlingx + + If a firewall is blocking access to Docker hub or your private + registry from your StarlingX deployment, you will need to add the + following lines in $HOME/localhost.yml (see :ref:`Docker Proxy + Configuration ` for more details about Docker + proxy settings): + + .. only:: partner + + .. include:: /_includes/install-kubernetes-bootstrap-playbook.rest + :start-after: firewall-begin + :end-before: firewall-end + + .. code-block:: + + # Add these lines to configure Docker to use a proxy server + docker_http_proxy: http://my.proxy.com:1080 + docker_https_proxy: https://my.proxy.com:1443 + docker_no_proxy: + - 1.2.3.4 + + + Refer to :ref:`Ansible Bootstrap Configurations ` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. This can take 5-10 minutes, + depending on the performance of the host machine. + +---------------------- +Configure controller-0 +---------------------- + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the |OAM| interface of controller-0 and specify the + attached network as "oam". + + Use the |OAM| port name that is applicable to your deployment environment, + for example eth0: + + :: + + OAM_IF= + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + +#. Configure the MGMT interface of controller-0 and specify the attached + networks of both "mgmt" and "cluster-host". + + Use the MGMT port name that is applicable to your deployment environment, + for example eth1: + + :: + + MGMT_IF= + system host-if-modify controller-0 lo -c none + IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}') + for UUID in $IFNET_UUIDS; do + system interface-network-remove ${UUID} + done + system host-if-modify controller-0 $MGMT_IF -c platform + system interface-network-assign controller-0 $MGMT_IF mgmt + system interface-network-assign controller-0 $MGMT_IF cluster-host + +#. Configure |NTP| servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + system host-label-assign controller-0 openstack-compute-node=enabled + system host-label-assign controller-0 openvswitch=enabled + system host-label-assign controller-0 sriov=enabled + + #. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has |OVS| (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, |OVS|-|DPDK| (|OVS| with the Data + Plane Development Kit, which is supported only on bare metal hardware) + should be used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + **To deploy the default containerized OVS:** + + :: + + system modify --vswitch_type none + + This does not run any vSwitch directly on the host, instead, it uses the + containerized |OVS| defined in the helm charts of stx-openstack + manifest. + + **To deploy OVS-DPDK, run the following command:** + + :: + + system modify --vswitch_type ovs-dpdk + + Default recommendation for an |AIO|-controller is to use a single core + for |OVS|-|DPDK| vswitch. + + :: + + + # assign 1 core on processor/numa-node 0 on controller-0 to vswitch + system host-cpu-modify -f vswitch -p0 1 controller-0 + + Once vswitch_type is set to |OVS|-|DPDK|, any subsequent nodes created + will default to automatically assigning 1 vSwitch core for |AIO| + controllers and 2 vSwitch cores for compute-labeled worker nodes. + + + When using |OVS|-|DPDK|, configure 1x 1G huge page for vSwitch memory on + each |NUMA| node where vswitch is running on this host, with the + following command: + + :: + + # assign 1x 1G huge page on processor/numa-node 0 on controller-0 to vswitch + system host-memory-modify -f vswitch -1G 1 controller-0 0 + + + .. important:: + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment on this host with + the commands: + + :: + + + # assign 10x 1G huge page on processor/numa-node 0 on controller-0 to applications + system host-memory-modify -f application -1G 10 controller-0 0 + + # assign 10x 1G huge page on processor/numa-node 1 on controller-0 to applications + system host-memory-modify -f application -1G 10 controller-0 1 + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking controller-0 to apply the change. + + #. **For OpenStack only:** Set up disk partition for nova-local volume + group, which is needed for stx-openstack nova ephemeral disks. + + :: + + export NODE=controller-0 + + echo ">>> Getting root disk info" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + echo "Root disk: $ROOT_DISK, UUID: $ROOT_DISK_UUID" + + echo ">>>> Configuring nova-local" + NOVA_SIZE=34 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${NOVA_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + sleep 2 + + #. **For OpenStack only:** Configure data interfaces for controller-0. + Data class interfaces are vswitch interfaces used by vswitch to provide + VM virtio vNIC connectivity to OpenStack Neutron Tenant Networks on the + underlying assigned Data Network. + + .. important:: + + A compute-labeled All-in-one controller host **MUST** have at least one Data class interface. + + * Configure the data interfaces for controller-0. + + :: + + export NODE=controller-0 + + # List inventoried host’s ports and identify ports to be used as ‘data’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘data’ class interfaces, MTU of 1500 and named data# + system host-if-modify -m 1500 -n data0 -c data ${NODE} + system host-if-modify -m 1500 -n data1 -c data ${NODE} + + # Create Data Networks that vswitch 'data' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to Data Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + +***************************************** +Optionally Configure PCI-SRIOV Interfaces +***************************************** + +#. **Optionally**, configure pci-sriov interfaces for controller-0. + + This step is **optional** for Kubernetes. Do this step if using |SRIOV| + network attachments in hosted application containers. + + .. only:: openstack + + This step is **optional** for OpenStack. Do this step if using |SRIOV| + vNICs in hosted application VMs. Note that pci-sriov interfaces can + have the same Data Networks assigned to them as vswitch data interfaces. + + + * Configure the pci-sriov interfaces for controller-0. + + :: + + export NODE=controller-0 + + # List inventoried host’s ports and identify ports to be used as ‘pci-sriov’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘pci-sriov’ class interfaces, MTU of 1500 and named sriov# + system host-if-modify -m 1500 -n sriov0 -c pci-sriov ${NODE} + system host-if-modify -m 1500 -n sriov1 -c pci-sriov ${NODE} + + # Create Data Networks that the 'pci-sriov' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to PCI-SRIOV Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + + * To enable using |SRIOV| network attachments for the above interfaces in + Kubernetes hosted application containers: + + * Configure the Kubernetes |SRIOV| device plugin. + + :: + + system host-label-assign controller-0 sriovdp=enabled + + * If planning on running |DPDK| in Kubernetes hosted application + containers on this host, configure the number of 1G Huge pages required + on both |NUMA| nodes. + + :: + + # assign 10x 1G huge page on processor/numa-node 0 on controller-0 to applications + system host-memory-modify -f application controller-0 0 -1G 10 + + # assign 10x 1G huge page on processor/numa-node 1 on controller-0 to applications + system host-memory-modify -f application controller-0 1 -1G 10 + +*************************************************************** +If required, initialize a Ceph-based Persistent Storage Backend +*************************************************************** + +A persistent storage backend is required if your application requires |PVCs|. + +.. only:: openstack + + .. important:: + + The StarlingX OpenStack application **requires** |PVCs|. + +.. only:: starlingx + + There are two options for persistent storage backend: the host-based Ceph + solution and the Rook container-based Ceph solution. + +For host-based Ceph: + +#. Initialize with add ceph backend: + + :: + + system storage-backend-add ceph --confirmed + +#. Add an |OSD| on controller-0 for host-based Ceph: + + .. code-block:: bash + + # List host’s disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list controller-0 + + # Add disk as an OSD storage + system host-stor-add controller-0 osd + + # List OSD storage devices + system host-stor-list controller-0 + + + # Add disk as an OSD storage + system host-stor-add controller-0 osd + +.. only:: starlingx + + For Rook container-based Ceph: + + #. Initialize with add ceph-rook backend: + + :: + + system storage-backend-add ceph-rook --confirmed + + #. Assign Rook host labels to controller-0 in support of installing the + rook-ceph-apps manifest/helm-charts later: + + :: + + system host-label-assign controller-0 ceph-mon-placement=enabled + system host-label-assign controller-0 ceph-mgr-placement=enabled + + +------------------- +Unlock controller-0 +------------------- + +.. include:: aio_simplex_install_kubernetes.rst + :start-after: incl-unlock-controller-0-aio-simplex-start: + :end-before: incl-unlock-controller-0-aio-simplex-end: + +------------------------------------- +Install software on controller-1 node +------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered controller-1 + host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + +#. Wait for the software installation on controller-1 to complete, for + controller-1 to reboot, and for controller-1 to show as + locked/disabled/online in 'system host-list'. + + This can take 5-10 minutes, depending on the performance of the host machine. + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +#. Configure the |OAM| interface of controller-1 and specify the + attached network of "oam". + + Use the |OAM| port name that is applicable to your deployment environment, + for example eth0: + + :: + + OAM_IF= + system host-if-modify controller-1 $OAM_IF -c platform + system interface-network-assign controller-1 $OAM_IF oam + +#. The MGMT interface is partially set up by the network install procedure; + configuring the port used for network install as the MGMT port and + specifying the attached network of "mgmt". + + Complete the MGMT interface configuration of controller-1 by specifying the + attached network of "cluster-host". + + :: + + system interface-network-assign controller-1 mgmt0 cluster-host + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to controller-1 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-1 openstack-control-plane=enabled + system host-label-assign controller-1 openstack-compute-node=enabled + system host-label-assign controller-1 openvswitch=enabled + system host-label-assign controller-1 sriov=enabled + + #. **For OpenStack only:** Configure the host settings for the vSwitch. + + **If using OVS-DPDK vswitch, run the following commands:** + + Default recommendation for an AIO-controller is to use a single core + for |OVS|-|DPDK| vswitch. This should have been automatically configured, + if not run the following command. + + :: + + # assign 1 core on processor/numa-node 0 on controller-1 to vswitch + system host-cpu-modify -f vswitch -p0 1 controller-1 + + + When using |OVS|-|DPDK|, configure 1x 1G huge page for vSwitch memory on + each |NUMA| node where vswitch is running on this host, with the + following command: + + :: + + # assign 1x 1G huge page on processor/numa-node 0 on controller-1 to vswitch + system host-memory-modify -f vswitch -1G 1 controller-1 0 + + + .. important:: + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment for + this host with the command: + + :: + + # assign 10x 1G huge page on processor/numa-node 0 on controller-1 to applications + system host-memory-modify -f application -1G 10 controller-1 0 + + # assign 10x 1G huge page on processor/numa-node 1 on controller-1 to applications + system host-memory-modify -f application -1G 10 controller-1 1 + + + #. **For OpenStack only:** Set up disk partition for nova-local volume group, + which is needed for stx-openstack nova ephemeral disks. + + :: + + export NODE=controller-1 + + echo ">>> Getting root disk info" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + echo "Root disk: $ROOT_DISK, UUID: $ROOT_DISK_UUID" + + echo ">>>> Configuring nova-local" + NOVA_SIZE=34 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${NOVA_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + sleep 2 + + #. **For OpenStack only:** Configure data interfaces for controller-1. + Data class interfaces are vswitch interfaces used by vswitch to provide + VM virtio vNIC connectivity to OpenStack Neutron Tenant Networks on the + underlying assigned Data Network. + + .. important:: + + A compute-labeled All-in-one controller host **MUST** have at least one Data class interface. + + * Configure the data interfaces for controller-1. + + :: + + export NODE=controller-1 + + # List inventoried host’s ports and identify ports to be used as ‘data’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘data’ class interfaces, MTU of 1500 and named data# + system host-if-modify -m 1500 -n data0 -c data ${NODE} + system host-if-modify -m 1500 -n data1 -c data ${NODE} + + # Create Data Networks that vswitch 'data' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to Data Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + +***************************************** +Optionally Configure PCI-SRIOV Interfaces +***************************************** + +#. **Optionally**, configure pci-sriov interfaces for controller-1. + + This step is **optional** for Kubernetes. Do this step if using |SRIOV| + network attachments in hosted application containers. + + .. only:: openstack + + This step is **optional** for OpenStack. Do this step if using |SRIOV| + vNICs in hosted application VMs. Note that pci-sriov interfaces can + have the same Data Networks assigned to them as vswitch data interfaces. + + + * Configure the pci-sriov interfaces for controller-1. + + :: + + export NODE=controller-1 + + # List inventoried host’s ports and identify ports to be used as ‘pci-sriov’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘pci-sriov’ class interfaces, MTU of 1500 and named sriov# + system host-if-modify -m 1500 -n sriov0 -c pci-sriov ${NODE} + system host-if-modify -m 1500 -n sriov1 -c pci-sriov ${NODE} + + # Create Data Networks that the 'pci-sriov' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to PCI-SRIOV Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + + * To enable using |SRIOV| network attachments for the above interfaces in + Kubernetes hosted application containers: + + * Configure the Kubernetes |SRIOV| device plugin. + + :: + + system host-label-assign controller-1 sriovdp=enabled + + * If planning on running |DPDK| in Kubernetes hosted application + containers on this host, configure the number of 1G Huge pages required + on both |NUMA| nodes. + + :: + + # assign 10x 1G huge page on processor/numa-node 0 on controller-1 to applications + system host-memory-modify -f application controller-1 0 -1G 10 + + # assign 10x 1G huge page on processor/numa-node 1 on controller-1 to applications + system host-memory-modify -f application controller-1 1 -1G 10 + + +*************************************************************************************** +If configuring a Ceph-based Persistent Storage Backend, configure host-specific details +*************************************************************************************** + +For host-based Ceph: + +#. Add an |OSD| on controller-1 for host-based Ceph: + + :: + + # List host’s disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list controller-1 + + # Add disk as an OSD storage + system host-stor-add controller-1 osd + + # List OSD storage devices + system host-stor-list controller-1 + + # Add disk as an OSD storage + system host-stor-add controller-1 osd + + +.. only:: starlingx + + For Rook container-based Ceph: + + #. Assign Rook host labels to controller-1 in support of installing the + rook-ceph-apps manifest/helm-charts later: + + :: + + system host-label-assign controller-1 ceph-mon-placement=enabled + system host-label-assign controller-1 ceph-mgr-placement=enabled + + +------------------- +Unlock controller-1 +------------------- + +Unlock controller-1 in order to bring it into service: + +:: + + system host-unlock controller-1 + +Controller-1 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +.. only:: starlingx + + ----------------------------------------------------------------------------------------------- + If using Rook container-based Ceph, finish configuring the ceph-rook Persistent Storage Backend + ----------------------------------------------------------------------------------------------- + + For Rook container-based Ceph: + + On active controller: + + #. Wait for the ``rook-ceph-apps`` application to be uploaded + + :: + + $ source /etc/platform/openrc + $ system application-list + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + | application | version | manifest name | manifest file | status | progress | + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + | oidc-auth-apps | 1.0-0 | oidc-auth-manifest | manifest.yaml | uploaded | completed | + | platform-integ-apps | 1.0-8 | platform-integration-manifest | manifest.yaml | uploaded | completed | + | rook-ceph-apps | 1.0-1 | rook-ceph-manifest | manifest.yaml | uploaded | completed | + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + + #. Configure Rook to use /dev/sdb on controller-0 and controller-1 as a ceph + |OSD|. + + :: + + $ system host-disk-wipe -s --confirm controller-0 /dev/sdb + $ system host-disk-wipe -s --confirm controller-1 /dev/sdb + + values.yaml for rook-ceph-apps. + :: + + cluster: + storage: + nodes: + - name: controller-0 + devices: + - name: /dev/disk/by-path/pci-0000:00:03.0-ata-2.0 + - name: controller-1 + devices: + - name: /dev/disk/by-path/pci-0000:00:03.0-ata-2.0 + + :: + + system helm-override-update rook-ceph-apps rook-ceph kube-system --values values.yaml + + #. Apply the rook-ceph-apps application. + + :: + + system application-apply rook-ceph-apps + + #. Wait for |OSDs| pod to be ready. + + :: + + kubectl get pods -n kube-system + rook-ceph-crashcollector-controller-0-f984688ff-jsr8t 1/1 Running 0 4m9s + rook-ceph-crashcollector-controller-1-7f9b6f55b6-699bb 1/1 Running 0 2m5s + rook-ceph-mgr-a-7f9d588c5b-49cbg 1/1 Running 0 3m5s + rook-ceph-mon-a-75bcbd8664-pvq99 1/1 Running 0 4m27s + rook-ceph-mon-b-86c67658b4-f4snf 1/1 Running 0 4m10s + rook-ceph-mon-c-7f48b58dfb-4nx2n 1/1 Running 0 3m30s + rook-ceph-operator-77b64588c5-bhfg7 1/1 Running 0 7m6s + rook-ceph-osd-0-6949657cf7-dkfp2 1/1 Running 0 2m6s + rook-ceph-osd-1-5d4b58cf69-kdg82 1/1 Running 0 2m4s + rook-ceph-osd-prepare-controller-0-wcvsn 0/1 Completed 0 2m27s + rook-ceph-osd-prepare-controller-1-98h76 0/1 Completed 0 2m26s + rook-ceph-tools-5778d7f6c-2h8s8 1/1 Running 0 5m55s + rook-discover-xc22t 1/1 Running 0 6m2s + rook-discover-xndld 1/1 Running 0 6m2s + storage-init-rook-ceph-provisioner-t868q 0/1 Completed 0 108s + + +.. include:: /_includes/bootstrapping-and-deploying-starlingx.rest + +.. only:: starlingx + + ---------- + Next steps + ---------- + + .. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst new file mode 100644 index 000000000..b704e0316 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex.rst @@ -0,0 +1,21 @@ +=============================================== +Bare metal All-in-one Simplex Installation R5.0 +=============================================== + +-------- +Overview +-------- + +.. include:: ../desc_aio_simplex.txt + +.. include:: ../ipv6_note.txt + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + aio_simplex_hardware + aio_simplex_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst new file mode 100644 index 000000000..18b6cec54 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_hardware.rst @@ -0,0 +1,58 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal All-in-one Simplex** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++-------------------------+-----------------------------------------------------------+ +| Minimum Requirement | All-in-one Controller Node | ++=========================+===========================================================+ +| Number of servers | 1 | ++-------------------------+-----------------------------------------------------------+ +| Minimum processor class | - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) | +| | 8 cores/socket | +| | | +| | or | +| | | +| | - Single-CPU Intel® Xeon® D-15xx family, 8 cores | +| | (low-power/low-cost option) | ++-------------------------+-----------------------------------------------------------+ +| Minimum memory | 64 GB | ++-------------------------+-----------------------------------------------------------+ +| Primary disk | 500 GB SSD or NVMe (see :doc:`../../nvme_config`) | ++-------------------------+-----------------------------------------------------------+ +| Additional disks | - 1 or more 500 GB (min. 10K RPM) for Ceph OSD | +| | - Recommended, but not required: 1 or more SSDs or NVMe | +| | drives for Ceph journals (min. 1024 MiB per OSD | +| | journal) | +| | - For OpenStack, recommend 1 or more 500 GB (min. 10K | +| | RPM) for VM local ephemeral storage | ++-------------------------+-----------------------------------------------------------+ +| Minimum network ports | - OAM: 1x1GE | +| | - Data: 1 or more x 10GE | ++-------------------------+-----------------------------------------------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++-------------------------+-----------------------------------------------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst new file mode 100644 index 000000000..0e60ff217 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_simplex_install_kubernetes.rst @@ -0,0 +1,607 @@ + +.. _aio_simplex_install_kubernetes: + +================================================= +Install Kubernetes Platform on All-in-one Simplex +================================================= + +.. only:: partner + + .. include:: /_includes/install-kubernetes-null-labels.rest + +.. only:: starlingx + + This section describes the steps to install the StarlingX Kubernetes + platform on a **StarlingX R5.0 All-in-one Simplex** deployment + configuration. + + .. contents:: + :local: + :depth: 1 + + --------------------- + Create a bootable USB + --------------------- + + Refer to :ref:`Bootable USB ` for instructions on how + to create a bootable USB with the StarlingX ISO on your system. + + -------------------------------- + Install software on controller-0 + -------------------------------- + + .. include:: inc-install-software-on-controller.rest + :start-after: incl-install-software-controller-0-aio-start + :end-before: incl-install-software-controller-0-aio-end + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +#. Login using the username / password of "sysadmin" / "sysadmin". + When logging in for the first time, you will be forced to change the + password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will |DHCP| out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a |DHCP| server + is present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for + Ansible configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: + ``$HOME/.yml``. + + .. only:: starlingx + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file + comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment + configuration as shown in the example below. Use the OAM IP SUBNET and IP + ADDRESSing applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: simplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + EOF + + .. only:: starlingx + + In either of the above options, the bootstrap playbook’s default values + will pull all container images required for the |prod-p| from Docker hub. + + If you have setup a private Docker registry to use for bootstrapping + then you will need to add the following lines in $HOME/localhost.yml: + + .. only:: partner + + .. include:: /_includes/install-kubernetes-bootstrap-playbook.rest + :start-after: docker-reg-begin + :end-before: docker-reg-end + + .. code-block:: + + docker_registries: + quay.io: + url: myprivateregistry.abc.com:9001/quay.io + docker.elastic.co: + url: myprivateregistry.abc.com:9001/docker.elastic.co + gcr.io: + url: myprivateregistry.abc.com:9001/gcr.io + k8s.gcr.io: + url: myprivateregistry.abc.com:9001/k8s.gcr.io + docker.io: + url: myprivateregistry.abc.com:9001/docker.io + defaults: + type: docker + username: + password: + + # Add the CA Certificate that signed myprivateregistry.abc.com’s + # certificate as a Trusted CA + ssl_ca_cert: /home/sysadmin/myprivateregistry.abc.com-ca-cert.pem + + See :ref:`Use a Private Docker Registry ` + for more information. + + + .. only:: starlingx + + If a firewall is blocking access to Docker hub or your private + registry from your StarlingX deployment, you will need to add the + following lines in $HOME/localhost.yml (see :ref:`Docker Proxy + Configuration ` for more details about Docker + proxy settings): + + .. only:: partner + + .. include:: /_includes/install-kubernetes-bootstrap-playbook.rest + :start-after: firewall-begin + :end-before: firewall-end + + .. code-block:: + + # Add these lines to configure Docker to use a proxy server + docker_http_proxy: http://my.proxy.com:1080 + docker_https_proxy: https://my.proxy.com:1443 + docker_no_proxy: + - 1.2.3.4 + + + Refer to :ref:`Ansible Bootstrap Configurations ` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. This can take 5-10 minutes, + depending on the performance of the host machine. + +---------------------- +Configure controller-0 +---------------------- + +The newly installed controller needs to be configured. + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the |OAM| interface of controller-0 and specify the attached network + as "oam". Use the |OAM| port name that is applicable to your deployment + environment, for example eth0: + + :: + + OAM_IF= + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + +#. Configure |NTP| servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. incl-config-controller-0-openstack-specific-aio-simplex-start: + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + system host-label-assign controller-0 openstack-compute-node=enabled + system host-label-assign controller-0 openvswitch=enabled + system host-label-assign controller-0 sriov=enabled + + #. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has |OVS| (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, |OVS|-|DPDK| (|OVS| with the Data Plane + Development Kit, which is supported only on bare metal hardware) should be + used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + **To deploy the default containerized OVS:** + + :: + + system modify --vswitch_type none + + This does not run any vSwitch directly on the host, instead, it uses the + containerized |OVS| defined in the helm charts of stx-openstack + manifest. + + **To deploy OVS-DPDK, run the following command:** + + :: + + system modify --vswitch_type ovs-dpdk + + Default recommendation for an AIO-controller is to use a single core + for |OVS|-|DPDK| vswitch. + + :: + + # assign 1 core on processor/numa-node 0 on controller-0 to vswitch + system host-cpu-modify -f vswitch -p0 1 controller-0 + + When using |OVS|-|DPDK|, configure 1x 1G huge page for vSwitch memory on each |NUMA| node + where vswitch is running on this host, with the following command: + + :: + + # assign 1x 1G huge page on processor/numa-node 0 on controller-0 to vswitch + system host-memory-modify -f vswitch -1G 1 controller-0 0 + + .. important:: + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment on this host with + the commands: + + :: + + # assign 1x 1G huge page on processor/numa-node 0 on controller-0 to applications + system host-memory-modify -f application -1G 10 controller-0 0 + + # assign 1x 1G huge page on processor/numa-node 1 on controller-0 to applications + system host-memory-modify -f application -1G 10 controller-0 1 + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking controller-0 to apply the change. + + #. **For OpenStack only:** Set up disk partition for nova-local volume + group, which is needed for stx-openstack nova ephemeral disks. + + .. code-block:: bash + + export NODE=controller-0 + + echo ">>> Getting root disk info" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + echo "Root disk: $ROOT_DISK, UUID: $ROOT_DISK_UUID" + + echo ">>>> Configuring nova-local" + NOVA_SIZE=34 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${NOVA_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + sleep 2 + + #. **For OpenStack only:** Configure data interfaces for controller-0. + Data class interfaces are vswitch interfaces used by vswitch to provide + VM virtio vNIC connectivity to OpenStack Neutron Tenant Networks on the + underlying assigned Data Network. + + .. important:: + + A compute-labeled worker host **MUST** have at least one Data class interface. + + * Configure the data interfaces for controller-0. + + :: + + export NODE=controller-0 + + # List inventoried host’s ports and identify ports to be used as ‘data’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘data’ class interfaces, MTU of 1500 and named data# + system host-if-modify -m 1500 -n data0 -c data ${NODE} + system host-if-modify -m 1500 -n data1 -c data ${NODE} + + # Create Data Networks that vswitch 'data' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to Data Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + +***************************************** +Optionally Configure PCI-SRIOV Interfaces +***************************************** + +#. **Optionally**, configure pci-sriov interfaces for controller-0. + + This step is **optional** for Kubernetes. Do this step if using |SRIOV| + network attachments in hosted application containers. + + .. only:: openstack + + This step is **optional** for OpenStack. Do this step if using |SRIOV| + vNICs in hosted application VMs. Note that pci-sriov interfaces can + have the same Data Networks assigned to them as vswitch data interfaces. + + + * Configure the pci-sriov interfaces for controller-0. + + :: + + export NODE=controller-0 + + # List inventoried host’s ports and identify ports to be used as ‘pci-sriov’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘pci-sriov’ class interfaces, MTU of 1500 and named sriov# + system host-if-modify -m 1500 -n sriov0 -c pci-sriov ${NODE} + system host-if-modify -m 1500 -n sriov1 -c pci-sriov ${NODE} + + # Create Data Networks that the 'pci-sriov' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to PCI-SRIOV Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + + * To enable using |SRIOV| network attachments for the above interfaces in + Kubernetes hosted application containers: + + * Configure the Kubernetes |SRIOV| device plugin. + + :: + + system host-label-assign controller-0 sriovdp=enabled + + * If planning on running |DPDK| in Kubernetes hosted application + containers on this host, configure the number of 1G Huge pages required + on both |NUMA| nodes. + + :: + + # assign 10x 1G huge page on processor/numa-node 0 on controller-0 to applications + system host-memory-modify -f application controller-0 0 -1G 10 + + # assign 10x 1G huge page on processor/numa-node 1 on controller-0 to applications + system host-memory-modify -f application controller-0 1 -1G 10 + + +*************************************************************** +If required, initialize a Ceph-based Persistent Storage Backend +*************************************************************** + +A persistent storage backend is required if your application requires +|PVCs|. + +.. only:: openstack + + .. important:: + + The StarlingX OpenStack application **requires** |PVCs|. + +.. only:: starlingx + + There are two options for persistent storage backend: the host-based Ceph + solution and the Rook container-based Ceph solution. + +For host-based Ceph: + +#. Add host-based Ceph backend: + + :: + + system storage-backend-add ceph --confirmed + +#. Add an |OSD| on controller-0 for host-based Ceph: + + .. code-block:: bash + + # List host’s disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list controller-0 + + # Add disk as an OSD storage + system host-stor-add controller-0 osd + + # List OSD storage devices + system host-stor-list controller-0 + + +.. only:: starlingx + + For Rook container-based Ceph: + + #. Add Rook container-based backend: + + :: + + system storage-backend-add ceph-rook --confirmed + + #. Assign Rook host labels to controller-0 in support of installing the + rook-ceph-apps manifest/helm-charts later: + + :: + + system host-label-assign controller-0 ceph-mon-placement=enabled + system host-label-assign controller-0 ceph-mgr-placement=enabled + + + .. incl-config-controller-0-openstack-specific-aio-simplex-end: + + +------------------- +Unlock controller-0 +------------------- + +.. incl-unlock-controller-0-aio-simplex-start: + +Unlock controller-0 to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +.. incl-unlock-controller-0-aio-simplex-end: + +.. only:: starlingx + + ----------------------------------------------------------------------------------------------- + If using Rook container-based Ceph, finish configuring the ceph-rook Persistent Storage Backend + ----------------------------------------------------------------------------------------------- + + On controller-0: + + #. Wait for application rook-ceph-apps to be uploaded + + :: + + $ source /etc/platform/openrc + $ system application-list + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + | application | version | manifest name | manifest file | status | progress | + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + | oidc-auth-apps | 1.0-0 | oidc-auth-manifest | manifest.yaml | uploaded | completed | + | platform-integ-apps | 1.0-8 | platform-integration-manifest | manifest.yaml | uploaded | completed | + | rook-ceph-apps | 1.0-1 | rook-ceph-manifest | manifest.yaml | uploaded | completed | + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + + #. Configure rook to use /dev/sdb disk on controller-0 as a ceph |OSD|. + + :: + + system host-disk-wipe -s --confirm controller-0 /dev/sdb + + values.yaml for rook-ceph-apps. + :: + + cluster: + storage: + nodes: + - name: controller-0 + devices: + - name: /dev/disk/by-path/pci-0000:00:03.0-ata-2.0 + + :: + + system helm-override-update rook-ceph-apps rook-ceph kube-system --values values.yaml + + #. Apply the rook-ceph-apps application. + + :: + + system application-apply rook-ceph-apps + + #. Wait for |OSDs| pod to be ready. + + :: + + kubectl get pods -n kube-system + rook--ceph-crashcollector-controller-0-764c7f9c8-bh5c7 1/1 Running 0 62m + rook--ceph-mgr-a-69df96f57-9l28p 1/1 Running 0 63m + rook--ceph-mon-a-55fff49dcf-ljfnx 1/1 Running 0 63m + rook--ceph-operator-77b64588c5-nlsf2 1/1 Running 0 66m + rook--ceph-osd-0-7d5785889f-4rgmb 1/1 Running 0 62m + rook--ceph-osd-prepare-controller-0-cmwt5 0/1 Completed 0 2m14s + rook--ceph-tools-5778d7f6c-22tms 1/1 Running 0 64m + rook--discover-kmv6c 1/1 Running 0 65m + + ---------- + Next steps + ---------- + + .. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/bootstrapping-from-a-private-docker-registry.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/bootstrapping-from-a-private-docker-registry.rst new file mode 100644 index 000000000..4672dbec5 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/bootstrapping-from-a-private-docker-registry.rst @@ -0,0 +1,52 @@ + +.. vqr1569420650576 +.. _bootstrapping-from-a-private-docker-registry: + +============================================ +Bootstrapping from a Private Docker Registry +============================================ + +You can bootstrap controller-0 from a private Docker registry in the event that +your server is isolated from the public Internet. + +.. rubric:: |proc| + +#. Update your /home/sysadmin/localhost.yml bootstrap overrides file with the + following lines to use a Private Docker Registry pre-populated from the + |org| Docker Registry: + + .. code-block:: none + + docker_registries: + k8s.gcr.io: + url: /k8s.gcr.io + gcr.io: + url: /gcr.io + quay.io: + url: /quay.io + docker.io: + url: /docker.io + docker.elastic.co: + url: /docker.elastic.co + defaults: + type: docker + username: + password: + + Where ```` and + ```` are your login credentials for the + ```` private Docker registry. + + .. note:: + ```` must be a DNS name resolvable by the dns servers + configured in the ``dns\_servers:`` structure of the ansible bootstrap + override file /home/sysadmin/localhost.yml. + +#. For any additional local registry images required, use the full image name + as shown below. + + .. code-block:: none + + additional_local_registry_images: + docker.io/wind-river/: + diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/bulk-host-xml-file-format.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/bulk-host-xml-file-format.rst new file mode 100644 index 000000000..78a208635 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/bulk-host-xml-file-format.rst @@ -0,0 +1,135 @@ + +.. hzf1552927866550 +.. _bulk-host-xml-file-format: + +========================= +Bulk Host XML File Format +========================= + +Hosts for bulk addition are described using an XML document. + +The document root is **hosts**. Within the root, each host is described using a +**host** node. To provide details, child elements are used, corresponding to +the parameters for the :command:`host-add` command. + +The following elements are accepted. Each element takes a text string. For +valid values, refer to the CLI documentation. + + +.. _bulk-host-xml-file-format-simpletable-tc3-w15-ht: + + +.. table:: + :widths: auto + + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Element | Remarks | + +=========================================================================================================================================================================================+=========================================================================================================================================================================================+ + | hostname | A unique name for the host. | + | | | + | | .. note:: | + | | Controller and storage node names are assigned automatically and override user input. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | personality | The type of host. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | subfunctions | For a worker host, an optional element to enable a low-latency performance profile. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | mgmt\_mac | The MAC address of the management interface. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | mgmt\_ip | The IP address of the management interface. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bm\_ip | The IP address of the board management controller. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bm\_type | The board management controller type. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bm\_username | The username for board management controller authentication. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bm\_password | The password for board management controller authentication. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | power\_on | An empty element. If present, powers on the host automatically using the specified board management controller. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | install\_output | The display mode to use during installation \(text or graphical\). The default is **text**. | + | | | + | | .. note:: | + | | The graphical option currently has no effect. Text-based installation is used regardless of this setting. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | console | If present, this element specifies the port, and if applicable the baud, for displaying messages. If the element is empty or not present, the default setting **ttyS0,115200** is used. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | rootfs\_device | The device to use for the rootfs partition, relative to /dev. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | boot\_device | The device to use for the boot partition, relative to /dev. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | location | A description of the host location. | + +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +The following sample describes a controller, three worker nodes, and two storage nodes: + +.. code-block:: none + + + + + controller + 08:00:27:19:b0:c5 + 10.10.10.100 + bmc + tsmith1 + mypass1 + text + System12/A4 + + + worker-0 + worker + 08:00:27:dc:42:46 + 192.168.204.50 + 10.10.10.101 + tsmith1 + mypass1 + bmc + text + + + + worker-1 + worker + 08:00:27:87:82:3E + 192.168.204.51 + 10.10.10.102 + bmc + tsmith1 + mypass1 + sda + text + + + worker-2 + worker + 08:00:27:b9:16:0d + 192.168.204.52 + sda + text + + + 10.10.10.103 + bmc + tsmith1 + mypass1 + + + storage + 08:00:27:dd:e3:3f + 10.10.10.104 + bmc + tsmith1 + mypass1 + + + storage + 08:00:27:8e:f1:b8 + 10.10.10.105 + bmc + tsmith1 + mypass1 + + diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/configuring-a-pxe-boot-server.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/configuring-a-pxe-boot-server.rst new file mode 100644 index 000000000..b2dc1bb68 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/configuring-a-pxe-boot-server.rst @@ -0,0 +1,200 @@ + +.. jow1440534908675 +.. _configuring-a-pxe-boot-server: + +=========================== +Configure a PXE Boot Server +=========================== + +You can optionally set up a |PXE| Boot Server to support **controller-0** +initialization. + +.. rubric:: |context| + +|prod| includes a setup script to simplify configuring a |PXE| boot server. If +you prefer, you can manually apply a custom configuration; for more +information, see :ref:`Access PXE Boot Server Files for a Custom Configuration +`. + +The |prod| setup script accepts a path to the root TFTP directory as a +parameter, and copies all required files for BIOS and |UEFI| clients into this +directory. + +The |PXE| boot server serves a boot loader file to the requesting client from a +specified path on the server. The path depends on whether the client uses BIOS +or |UEFI|. The appropriate path is selected by conditional logic in the |DHCP| +configuration file. + +The boot loader runs on the client, and reads boot parameters, including the +location of the kernel and initial ramdisk image files, from a boot file +contained on the server. To find the boot file, the boot loader searches a +known directory on the server. This search directory can contain more than one +entry, supporting the use of separate boot files for different clients. + +The file names and locations depend on the BIOS or |UEFI| implementation. + +.. _configuring-a-pxe-boot-server-table-mgq-xlh-2cb: + +.. table:: Table 1. |PXE| boot server file locations for BIOS and |UEFI| implementations + :widths: auto + + +------------------------------------------+------------------------+-------------------------------+ + | Resource | BIOS | UEFI | + +==========================================+========================+===============================+ + | **boot loader** | ./pxelinux.0 | ./EFI/grubx64.efi | + +------------------------------------------+------------------------+-------------------------------+ + | **boot file search directory** | ./pxelinux.cfg | ./ or ./EFI | + | | | | + | | | \(system-dependent\) | + +------------------------------------------+------------------------+-------------------------------+ + | **boot file** and path | ./pxelinux.cfg/default | ./grub.cfg and ./EFI/grub.cfg | + +------------------------------------------+------------------------+-------------------------------+ + | \(./ indicates the root TFTP directory\) | + +------------------------------------------+------------------------+-------------------------------+ + +.. rubric:: |prereq| + +Use a Linux workstation as the |PXE| Boot server. + + +.. _configuring-a-pxe-boot-server-ul-mrz-jlj-dt: + +- On the workstation, install the packages required to support |DHCP|, TFTP, + and Apache. + +- Configure |DHCP|, TFTP, and Apache according to your system requirements. + For details, refer to the documentation included with the packages. + +- Additionally, configure |DHCP| to support both BIOS and |UEFI| client + architectures. For example: + + .. code-block:: none + + option arch code 93 unsigned integer 16; # ref RFC4578 + # ... + subnet 192.168.1.0 netmask 255.255.255.0 { + if option arch = 00:07 { + filename "EFI/grubx64.efi"; + # NOTE: substitute the full tftp-boot-dir specified in the setup script + } + else { + filename "pxelinux.0"; + } + # ... + } + + +- Start the |DHCP|, TFTP, and Apache services. + +- Connect the |PXE| boot server to the |prod| management or |PXE| boot + network. + + +.. rubric:: |proc| + + +.. _configuring-a-pxe-boot-server-steps-qfb-kyh-2cb: + +#. Copy the ISO image from the source \(product DVD, USB device, or WindShare + `http://windshare.windriver.com `__\) to a + temporary location on the PXE boot server. + + This example assumes that the copied image file is tmp/TS-host-installer-1.0.iso. + +#. Mount the ISO image and make it executable. + + .. code-block:: none + + $ mount -o loop /tmp/TS-host-installer-1.0.iso /media/iso + $ mount -o remount,exec,dev /media/iso + +#. Set up the |PXE| boot configuration. + + The ISO image includes a setup script, which you can run to complete the + configuration. + + .. code-block:: none + + $ /media/iso/pxeboot_setup.sh -u http:/// \ + -t + + where + + ``ip-addr`` + is the Apache listening address. + + ``symlink`` + is the name of a user-created symbolic link under the Apache document + root directory, pointing to the directory specified by . + + ``tftp-boot-dir`` + is the path from which the boot loader is served \(the TFTP root + directory\). + + The script creates the directory specified by . + + For example: + + .. code-block:: none + + $ /media/iso/pxeboot_setup.sh -u http://192.168.100.100/BIOS-client -t /export/pxeboot + +#. To serve a specific boot file to a specific controller, assign a special + name to the file. + + The boot loader searches for a file name that uses a string based on the + client interface |MAC| address. The string uses lower case, substitutes + dashes for colons, and includes the prefix "01-". + + + - For a BIOS client, use the |MAC| address string as the file name: + + .. code-block:: none + + $ cd /pxelinux.cfg/ + $ cp pxeboot.cfg + + where: + + ```` + is the path from which the boot loader is served. + + ```` + is a lower-case string formed from the |MAC| address of the client + |PXE| boot interface, using dashes instead of colons, and prefixed + by "01-". + + For example, to represent the |MAC| address ``08:00:27:dl:63:c9``, + use the string ``01-08-00-27-d1-63-c9`` in the file name. + + For example: + + .. code-block:: none + + $ cd /export/pxeboot/pxelinux.cfg/ + $ cp pxeboot.cfg 01-08-00-27-d1-63-c9 + + If the boot loader does not find a file named using this convention, it + looks for a file with the name default. + + - For a |UEFI| client, use the |MAC| address string prefixed by + "grub.cfg-". To ensure the file is found, copy it to both search + directories used by the |UEFI| convention. + + .. code-block:: none + + $ cd + $ cp grub.cfg grub.cfg- + $ cp grub.cfg ./EFI/grub.cfg- + + For example: + + .. code-block:: none + + $ cd /export/pxeboot + $ cp grub.cfg grub.cfg-01-08-00-27-d1-63-c9 + $ cp grub.cfg ./EFI/grub.cfg-01-08-00-27-d1-63-c9 + + .. note:: + Alternatively, you can use symlinks in the search directories to + ensure the file is found. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst new file mode 100644 index 000000000..8139e3e95 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage.rst @@ -0,0 +1,22 @@ +============================================================= +Bare metal Standard with Controller Storage Installation R5.0 +============================================================= + +-------- +Overview +-------- + +.. include:: ../desc_controller_storage.txt + +.. include:: ../ipv6_note.txt + + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + controller_storage_hardware + controller_storage_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst new file mode 100644 index 000000000..c79456547 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_hardware.rst @@ -0,0 +1,56 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Standard with Controller Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++-------------------------+-----------------------------+-----------------------------+ +| Minimum Requirement | Controller Node | Worker Node | ++=========================+=============================+=============================+ +| Number of servers | 2 | 2-10 | ++-------------------------+-----------------------------+-----------------------------+ +| Minimum processor class | - Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) | +| | 8 cores/socket | ++-------------------------+-----------------------------+-----------------------------+ +| Minimum memory | 64 GB | 32 GB | ++-------------------------+-----------------------------+-----------------------------+ +| Primary disk | 500 GB SSD or NVMe (see | 120 GB (Minimum 10k RPM) | +| | :doc:`../../nvme_config`) | | ++-------------------------+-----------------------------+-----------------------------+ +| Additional disks | - 1 or more 500 GB (min. | - For OpenStack, recommend | +| | 10K RPM) for Ceph OSD | 1 or more 500 GB (min. | +| | - Recommended, but not | 10K RPM) for VM local | +| | required: 1 or more SSDs | ephemeral storage | +| | or NVMe drives for Ceph | | +| | journals (min. 1024 MiB | | +| | per OSD journal) | | ++-------------------------+-----------------------------+-----------------------------+ +| Minimum network ports | - Mgmt/Cluster: 1x10GE | - Mgmt/Cluster: 1x10GE | +| | - OAM: 1x1GE | - Data: 1 or more x 10GE | ++-------------------------+-----------------------------+-----------------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++-------------------------+-----------------------------+-----------------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst new file mode 100644 index 000000000..7b3f070eb --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst @@ -0,0 +1,799 @@ + +.. _controller_storage_install_kubernetes: + +=============================================================== +Install Kubernetes Platform on Standard with Controller Storage +=============================================================== + +.. contents:: + :local: + :depth: 1 + +.. only:: starlingx + + This section describes the steps to install the StarlingX Kubernetes + platform on a **StarlingX R5.0 Standard with Controller Storage** + deployment configuration. + + ------------------- + Create bootable USB + ------------------- + + Refer to :ref:`Bootable USB ` for instructions on how to + create a bootable USB with the StarlingX ISO on your system. + + -------------------------------- + Install software on controller-0 + -------------------------------- + + .. include:: inc-install-software-on-controller.rest + :start-after: incl-install-software-controller-0-standard-start + :end-before: incl-install-software-controller-0-standard-end + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +.. incl-bootstrap-sys-controller-0-standard-start: + +#. Login using the username / password of "sysadmin" / "sysadmin". + + When logging in for the first time, you will be forced to change the + password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will |DHCP| out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a |DHCP| server + is present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for + Ansible configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: + ``$HOME/.yml``. + + .. only:: starlingx + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file + comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment + configuration as shown in the example below. Use the OAM IP SUBNET and IP + ADDRESSing applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: duplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + external_oam_node_0_address: + external_oam_node_1_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + EOF + + .. only:: starlingx + + In either of the above options, the bootstrap playbook’s default values + will pull all container images required for the |prod-p| from Docker hub. + + If you have setup a private Docker registry to use for bootstrapping + then you will need to add the following lines in $HOME/localhost.yml: + + .. only:: partner + + .. include:: /_includes/install-kubernetes-bootstrap-playbook.rest + :start-after: docker-reg-begin + :end-before: docker-reg-end + + .. code-block:: + + docker_registries: + quay.io: + url: myprivateregistry.abc.com:9001/quay.io + docker.elastic.co: + url: myprivateregistry.abc.com:9001/docker.elastic.co + gcr.io: + url: myprivateregistry.abc.com:9001/gcr.io + k8s.gcr.io: + url: myprivateregistry.abc.com:9001/k8s.gcr.io + docker.io: + url: myprivateregistry.abc.com:9001/docker.io + defaults: + type: docker + username: + password: + + # Add the CA Certificate that signed myprivateregistry.abc.com’s + # certificate as a Trusted CA + ssl_ca_cert: /home/sysadmin/myprivateregistry.abc.com-ca-cert.pem + + See :ref:`Use a Private Docker Registry ` + for more information. + + .. only:: starlingx + + If a firewall is blocking access to Docker hub or your private + registry from your StarlingX deployment, you will need to add the + following lines in $HOME/localhost.yml (see :ref:`Docker Proxy + Configuration ` for more details about Docker + proxy settings): + + .. only:: partner + + .. include:: /_includes/install-kubernetes-bootstrap-playbook.rest + :start-after: firewall-begin + :end-before: firewall-end + + .. code-block:: + + # Add these lines to configure Docker to use a proxy server + docker_http_proxy: http://my.proxy.com:1080 + docker_https_proxy: https://my.proxy.com:1443 + docker_no_proxy: + - 1.2.3.4 + + Refer to :ref:`Ansible Bootstrap Configurations ` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. + This can take 5-10 minutes, depending on the performance of the host machine. + +.. incl-bootstrap-sys-controller-0-standard-end: + + +---------------------- +Configure controller-0 +---------------------- + +.. incl-config-controller-0-storage-start: + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the |OAM| interface of controller-0 and specify the + attached network as "oam". + + Use the |OAM| port name that is applicable to your deployment environment, + for example eth0: + + :: + + OAM_IF= + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + +#. Configure the MGMT interface of controller-0 and specify the attached + networks of both "mgmt" and "cluster-host". + + Use the MGMT port name that is applicable to your deployment environment, + for example eth1: + + :: + + MGMT_IF= + system host-if-modify controller-0 lo -c none + IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}') + for UUID in $IFNET_UUIDS; do + system interface-network-remove ${UUID} + done + system host-if-modify controller-0 $MGMT_IF -c platform + system interface-network-assign controller-0 $MGMT_IF mgmt + system interface-network-assign controller-0 $MGMT_IF cluster-host + +#. Configure |NTP| servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +#. If required, configure Ceph storage backend: + + A persistent storage backend is required if your application requires |PVCs|. + + .. only:: openstack + + .. important:: + + The StarlingX OpenStack application **requires** |PVCs|. + + :: + + system storage-backend-add ceph --confirmed + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + + #. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has |OVS| (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, |OVS|-|DPDK| (OVS with the Data Plane + Development Kit, which is supported only on bare metal hardware) should + be used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + **To deploy the default containerized OVS:** + + :: + + system modify --vswitch_type none + + This does not run any vSwitch directly on the host, instead, it uses the + containerized |OVS| defined in the helm charts of stx-openstack manifest. + + **To deploy OVS-DPDK, run the following command:** + + :: + + system modify --vswitch_type ovs-dpdk + + Once vswitch_type is set to OVS-|DPDK|, any subsequent AIO-controller or + worker nodes created will default to automatically assigning 1 vSwitch + core for |AIO| controllers and 2 vSwitch cores for compute-labeled worker + nodes. + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking all compute-labeled worker nodes (and/or |AIO| + controllers) to apply the change. + + .. incl-config-controller-0-storage-end: + +------------------- +Unlock controller-0 +------------------- + +Unlock controller-0 in order to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +------------------------------------------------- +Install software on controller-1 and worker nodes +------------------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered + controller-1 host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + + This initiates the install of software on controller-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the worker nodes. + Set the personality to 'worker' and assign a unique hostname for each. + + For example, power on worker-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 3 personality=worker hostname=worker-0 + + Repeat for worker-1. Power on worker-1 and wait for the new host + (hostname=None) to be discovered by checking 'system host-list': + + :: + + system host-update 4 personality=worker hostname=worker-1 + +#. Wait for the software installation on controller-1, worker-0, and worker-1 + to complete, for all servers to reboot, and for all to show as + locked/disabled/online in 'system host-list'. + + :: + + system host-list + + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | worker-0 | worker | locked | disabled | online | + | 4 | worker-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +.. incl-config-controller-1-start: + +#. Configure the |OAM| interface of controller-1 and specify the + attached network of "oam". + + Use the |OAM| port name that is applicable to your deployment environment, + for example eth0: + + :: + + OAM_IF= + system host-if-modify controller-1 $OAM_IF -c platform + system interface-network-assign controller-1 $OAM_IF oam + +#. The MGMT interface is partially set up by the network install procedure; + configuring the port used for network install as the MGMT port and + specifying the attached network of "mgmt". + + Complete the MGMT interface configuration of controller-1 by specifying the + attached network of "cluster-host". + + :: + + system interface-network-assign controller-1 mgmt0 cluster-host + + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + **For OpenStack only:** Assign OpenStack host labels to controller-1 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-1 openstack-control-plane=enabled + +.. incl-config-controller-1-end: + +------------------- +Unlock controller-1 +------------------- + +.. incl-unlock-controller-1-start: + +Unlock controller-1 in order to bring it into service: + +:: + + system host-unlock controller-1 + +Controller-1 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +.. incl-unlock-controller-1-end: + +.. include:: /_includes/bootstrapping-and-deploying-starlingx.rest + +---------------------- +Configure worker nodes +---------------------- + +#. Add the third Ceph monitor to a worker node: + + (The first two Ceph monitors are automatically assigned to controller-0 and + controller-1.) + + :: + + system ceph-mon-add worker-0 + +#. Wait for the worker node monitor to complete configuration: + + :: + + system ceph-mon-list + +--------------------------------------+-------+--------------+------------+------+ + | uuid | ceph_ | hostname | state | task | + | | mon_g | | | | + | | ib | | | | + +--------------------------------------+-------+--------------+------------+------+ + | 64176b6c-e284-4485-bb2a-115dee215279 | 20 | controller-1 | configured | None | + | a9ca151b-7f2c-4551-8167-035d49e2df8c | 20 | controller-0 | configured | None | + | f76bc385-190c-4d9a-aa0f-107346a9907b | 20 | worker-0 | configured | None | + +--------------------------------------+-------+--------------+------------+------+ + +#. Assign the cluster-host network to the MGMT interface for the worker nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + + #. **For OpenStack only:** Configure the host settings for the vSwitch. + + **If using OVS-DPDK vswitch, run the following commands:** + + Default recommendation for worker node is to use a single core on each + numa-node for |OVS|-|DPDK| vswitch. This should have been automatically + configured, if not run the following command. + + :: + + for NODE in worker-0 worker-1; do + + # assign 1 core on processor/numa-node 0 on worker-node to vswitch + system host-cpu-modify -f vswitch -p0 1 $NODE + + # assign 1 core on processor/numa-node 1 on worker-node to vswitch + system host-cpu-modify -f vswitch -p1 1 $NODE + + done + + + When using |OVS|-|DPDK|, configure 1x 1G huge page for vSwitch memory on + each |NUMA| node where vswitch is running on this host, with the + following command: + + :: + + for NODE in worker-0 worker-1; do + + # assign 1x 1G huge page on processor/numa-node 0 on worker-node to vswitch + system host-memory-modify -f vswitch -1G 1 $NODE 0 + + # assign 1x 1G huge page on processor/numa-node 0 on worker-node to vswitch + system host-memory-modify -f vswitch -1G 1 $NODE 1 + + done + + + .. important:: + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment for + this host with the command: + + :: + + for NODE in worker-0 worker-1; do + + # assign 10x 1G huge page on processor/numa-node 0 on worker-node to applications + system host-memory-modify -f application -1G 10 $NODE 0 + + # assign 10x 1G huge page on processor/numa-node 1 on worker-node to applications + system host-memory-modify -f application -1G 10 $NODE 1 + + done + + #. **For OpenStack only:** Setup disk partition for nova-local volume group, + needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + + #. **For OpenStack only:** Configure data interfaces for worker nodes. + Data class interfaces are vswitch interfaces used by vswitch to provide + VM virtio vNIC connectivity to OpenStack Neutron Tenant Networks on the + underlying assigned Data Network. + + .. important:: + + A compute-labeled worker host **MUST** have at least one Data class interface. + + * Configure the data interfaces for worker nodes. + + :: + + # Execute the following lines with + export NODE=worker-0 + # and then repeat with + export NODE=worker-1 + + # List inventoried host’s ports and identify ports to be used as ‘data’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘data’ class interfaces, MTU of 1500 and named data# + system host-if-modify -m 1500 -n data0 -c data ${NODE} + system host-if-modify -m 1500 -n data1 -c data ${NODE} + + # Create Data Networks that vswitch 'data' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to Data Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + +***************************************** +Optionally Configure PCI-SRIOV Interfaces +***************************************** + +#. **Optionally**, configure pci-sriov interfaces for worker nodes. + + This step is **optional** for Kubernetes. Do this step if using |SRIOV| + network attachments in hosted application containers. + + .. only:: openstack + + This step is **optional** for OpenStack. Do this step if using |SRIOV| + vNICs in hosted application VMs. Note that pci-sriov interfaces can + have the same Data Networks assigned to them as vswitch data interfaces. + + + * Configure the pci-sriov interfaces for worker nodes. + + :: + + # Execute the following lines with + export NODE=worker-0 + # and then repeat with + export NODE=worker-1 + + # List inventoried host’s ports and identify ports to be used as ‘pci-sriov’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘pci-sriov’ class interfaces, MTU of 1500 and named sriov# + system host-if-modify -m 1500 -n sriov0 -c pci-sriov ${NODE} + system host-if-modify -m 1500 -n sriov1 -c pci-sriov ${NODE} + + # Create Data Networks that the 'pci-sriov' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to PCI-SRIOV Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + + * To enable using |SRIOV| network attachments for the above interfaces in + Kubernetes hosted application containers: + + * Configure the Kubernetes |SRIOV| device plugin. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE sriovdp=enabled + done + + * If planning on running |DPDK| in Kubernetes hosted application + containers on this host, configure the number of 1G Huge pages required + on both |NUMA| nodes. + + :: + + for NODE in worker-0 worker-1; do + + # assign 10x 1G huge page on processor/numa-node 0 on worker-node to applications + system host-memory-modify -f application $NODE 0 -1G 10 + + # assign 10x 1G huge page on processor/numa-node 1 on worker-node to applications + system host-memory-modify -f application $NODE 1 -1G 10 + + done + + +-------------------- +Unlock worker nodes +-------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +----------------------------------------------------------------- +If configuring Ceph Storage Backend, Add Ceph OSDs to controllers +----------------------------------------------------------------- + +#. Add |OSDs| to controller-0. The following example adds |OSDs| to the `sdb` disk: + + :: + + HOST=controller-0 + + # List host's disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list ${HOST} + + # Add disk as an OSD storage + system host-stor-add ${HOST} osd + + # List OSD storage devices and wait for configuration of newly added OSD to complete. + system host-stor-list ${HOST} + +#. Add |OSDs| to controller-1. The following example adds |OSDs| to the `sdb` disk: + + :: + + HOST=controller-1 + + # List host's disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list ${HOST} + + # Add disk as an OSD storage + system host-stor-add ${HOST} osd + + # List OSD storage devices and wait for configuration of newly added OSD to complete. + system host-stor-list ${HOST} + +.. only:: starlingx + + ---------- + Next steps + ---------- + + .. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst new file mode 100644 index 000000000..11cb22c39 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage.rst @@ -0,0 +1,22 @@ + +============================================================ +Bare metal Standard with Dedicated Storage Installation R5.0 +============================================================ + +-------- +Overview +-------- + +.. include:: ../desc_dedicated_storage.txt + +.. include:: ../ipv6_note.txt + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + dedicated_storage_hardware + dedicated_storage_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst new file mode 100644 index 000000000..6866f3b85 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_hardware.rst @@ -0,0 +1,61 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Standard with Dedicated Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum Requirement | Controller Node | Storage Node | Worker Node | ++=====================+===========================+=======================+=======================+ +| Number of servers | 2 | 2-9 | 2-100 | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum processor | Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket | +| class | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum memory | 64 GB | 64 GB | 32 GB | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Primary disk | 500 GB SSD or NVMe (see | 120 GB (min. 10k RPM) | 120 GB (min. 10k RPM) | +| | :doc:`../../nvme_config`) | | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Additional disks | None | - 1 or more 500 GB | - For OpenStack, | +| | | (min. 10K RPM) for | recommend 1 or more | +| | | Ceph OSD | 500 GB (min. 10K | +| | | - Recommended, but | RPM) for VM | +| | | not required: 1 or | ephemeral storage | +| | | more SSDs or NVMe | | +| | | drives for Ceph | | +| | | journals (min. 1024 | | +| | | MiB per OSD | | +| | | journal) | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum network | - Mgmt/Cluster: | - Mgmt/Cluster: | - Mgmt/Cluster: | +| ports | 1x10GE | 1x10GE | 1x10GE | +| | - OAM: 1x1GE | | - Data: 1 or more | +| | | | x 10GE | ++---------------------+---------------------------+-----------------------+-----------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++---------------------+---------------------------+-----------------------+-----------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst new file mode 100644 index 000000000..f747dfafe --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst @@ -0,0 +1,498 @@ + + +.. _dedicated_storage_install_kubernetes: + +.. only:: partner + + .. include:: /_includes/install-kubernetes-null-labels.rest + +============================================================== +Install Kubernetes Platform on Standard with Dedicated Storage +============================================================== + +This section describes the steps to install the |prod| Kubernetes platform on a +**Standard with Dedicated Storage** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +.. only:: starlingx + + ------------------- + Create bootable USB + ------------------- + + Refer to :ref:`Bootable USB ` for instructions on how to + create a bootable USB with the StarlingX ISO on your system. + + -------------------------------- + Install software on controller-0 + -------------------------------- + + .. include:: inc-install-software-on-controller.rest + :start-after: incl-install-software-controller-0-standard-start + :end-before: incl-install-software-controller-0-standard-end + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-bootstrap-sys-controller-0-standard-start: + :end-before: incl-bootstrap-sys-controller-0-standard-end: + +---------------------- +Configure controller-0 +---------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-config-controller-0-storage-start: + :end-before: incl-config-controller-0-storage-end: + +------------------- +Unlock controller-0 +------------------- + +.. important:: + + Make sure the Ceph storage backend is configured. If it is + not configured, you will not be able to configure storage + nodes. + +Unlock controller-0 in order to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +----------------------------------------------------------------- +Install software on controller-1, storage nodes, and worker nodes +----------------------------------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered controller-1 + host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + + This initiates the install of software on controller-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the storage-0 and + storage-1 servers. Set the personality to 'storage' and assign a unique + hostname for each. + + For example, power on storage-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 3 personality=storage + + Repeat for storage-1. Power on storage-1 and wait for the new host + (hostname=None) to be discovered by checking 'system host-list': + + :: + + system host-update 4 personality=storage + + This initiates the software installation on storage-0 and storage-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the worker nodes. + Set the personality to 'worker' and assign a unique hostname for each. + + For example, power on worker-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 5 personality=worker hostname=worker-0 + + Repeat for worker-1. Power on worker-1 and wait for the new host + (hostname=None) to be discovered by checking 'system host-list': + + :: + + system host-update 6 personality=worker hostname=worker-1 + + This initiates the install of software on worker-0 and worker-1. + +#. Wait for the software installation on controller-1, storage-0, storage-1, + worker-0, and worker-1 to complete, for all servers to reboot, and for all to + show as locked/disabled/online in 'system host-list'. + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | storage-0 | storage | locked | disabled | online | + | 4 | storage-1 | storage | locked | disabled | online | + | 5 | worker-0 | worker | locked | disabled | online | + | 6 | worker-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-config-controller-1-start: + :end-before: incl-config-controller-1-end: + +------------------- +Unlock controller-1 +------------------- + +.. include:: controller_storage_install_kubernetes.rst + :start-after: incl-unlock-controller-1-start: + :end-before: incl-unlock-controller-1-end: + +.. include:: /_includes/bootstrapping-and-deploying-starlingx.rest + +----------------------- +Configure storage nodes +----------------------- + +#. Assign the cluster-host network to the MGMT interface for the storage nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in storage-0 storage-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. Add |OSDs| to storage-0. + + :: + + HOST=storage-0 + + # List host’s disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list ${HOST} + + # Add disk as an OSD storage + system host-stor-add ${HOST} osd + + # List OSD storage devices and wait for configuration of newly added OSD to complete. + system host-stor-list ${HOST} + +#. Add |OSDs| to storage-1. + + :: + + HOST=storage-1 + + # List host’s disks and identify disks you want to use for CEPH OSDs, taking note of their UUID + # By default, /dev/sda is being used as system disk and can not be used for OSD. + system host-disk-list ${HOST} + + # Add disk as an OSD storage + system host-stor-add ${HOST} osd + + # List OSD storage devices and wait for configuration of newly added OSD to complete. + system host-stor-list ${HOST} + +-------------------- +Unlock storage nodes +-------------------- + +Unlock storage nodes in order to bring them into service: + +:: + + for STORAGE in storage-0 storage-1; do + system host-unlock $STORAGE + done + +The storage nodes will reboot in order to apply configuration changes and come +into service. This can take 5-10 minutes, depending on the performance of the +host machine. + +---------------------- +Configure worker nodes +---------------------- + +#. The MGMT interfaces are partially set up by the network install procedure; + configuring the port used for network install as the MGMT port and + specifying the attached network of "mgmt". + + Complete the MGMT interface configuration of the worker nodes by specifying + the attached network of "cluster-host". + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +.. only:: openstack + + ************************************* + OpenStack-specific host configuration + ************************************* + + .. important:: + + **These steps are required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + + #. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + + #. **For OpenStack only:** Configure the host settings for the vSwitch. + + **If using OVS-DPDK vswitch, run the following commands:** + + Default recommendation for worker node is to use a single core on each + numa-node for |OVS|-|DPDK| vswitch. This should have been automatically + configured, if not run the following command. + + :: + + for NODE in worker-0 worker-1; do + + # assign 1 core on processor/numa-node 0 on worker-node to vswitch + system host-cpu-modify -f vswitch -p0 1 $NODE + + # assign 1 core on processor/numa-node 1 on worker-node to vswitch + system host-cpu-modify -f vswitch -p1 1 $NODE + + done + + + When using |OVS|-|DPDK|, configure 1x 1G huge page for vSwitch memory on + each |NUMA| node where vswitch is running on this host, with the + following command: + + :: + + for NODE in worker-0 worker-1; do + + # assign 1x 1G huge page on processor/numa-node 0 on worker-node to vswitch + system host-memory-modify -f vswitch -1G 1 $NODE 0 + + # assign 1x 1G huge page on processor/numa-node 0 on worker-node to vswitch + system host-memory-modify -f vswitch -1G 1 $NODE 1 + + done + + + .. important:: + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment for + this host with the command: + + :: + + for NODE in worker-0 worker-1; do + + # assign 10x 1G huge page on processor/numa-node 0 on worker-node to applications + system host-memory-modify -f application -1G 10 $NODE 0 + + # assign 10x 1G huge page on processor/numa-node 1 on worker-node to applications + system host-memory-modify -f application -1G 10 $NODE 1 + + done + + #. **For OpenStack only:** Setup disk partition for nova-local volume group, + needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + + #. **For OpenStack only:** Configure data interfaces for worker nodes. + Data class interfaces are vswitch interfaces used by vswitch to provide + VM virtio vNIC connectivity to OpenStack Neutron Tenant Networks on the + underlying assigned Data Network. + + .. important:: + + A compute-labeled worker host **MUST** have at least one Data class interface. + + * Configure the data interfaces for worker nodes. + + :: + + # Execute the following lines with + export NODE=worker-0 + # and then repeat with + export NODE=worker-1 + + # List inventoried host’s ports and identify ports to be used as ‘data’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘data’ class interfaces, MTU of 1500 and named data# + system host-if-modify -m 1500 -n data0 -c data ${NODE} + system host-if-modify -m 1500 -n data1 -c data ${NODE} + + # Create Data Networks that vswitch 'data' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to Data Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + +***************************************** +Optionally Configure PCI-SRIOV Interfaces +***************************************** + +#. **Optionally**, configure pci-sriov interfaces for worker nodes. + + This step is **optional** for Kubernetes. Do this step if using |SRIOV| + network attachments in hosted application containers. + + .. only:: openstack + + This step is **optional** for OpenStack. Do this step if using |SRIOV| + vNICs in hosted application VMs. Note that pci-sriov interfaces can + have the same Data Networks assigned to them as vswitch data interfaces. + + + * Configure the pci-sriov interfaces for worker nodes. + + :: + + # Execute the following lines with + export NODE=worker-0 + # and then repeat with + export NODE=worker-1 + + # List inventoried host’s ports and identify ports to be used as ‘pci-sriov’ interfaces, + # based on displayed linux port name, pci address and device type. + system host-port-list ${NODE} + + # List host’s auto-configured ‘ethernet’ interfaces, + # find the interfaces corresponding to the ports identified in previous step, and + # take note of their UUID + system host-if-list -a ${NODE} + + # Modify configuration for these interfaces + # Configuring them as ‘pci-sriov’ class interfaces, MTU of 1500 and named sriov# + system host-if-modify -m 1500 -n sriov0 -c pci-sriov ${NODE} + system host-if-modify -m 1500 -n sriov1 -c pci-sriov ${NODE} + + # Create Data Networks that the 'pci-sriov' interfaces will be connected to + DATANET0='datanet0' + DATANET1='datanet1' + system datanetwork-add ${DATANET0} vlan + system datanetwork-add ${DATANET1} vlan + + # Assign Data Networks to PCI-SRIOV Interfaces + system interface-datanetwork-assign ${NODE} ${DATANET0} + system interface-datanetwork-assign ${NODE} ${DATANET1} + + + * To enable using |SRIOV| network attachments for the above interfaces in + Kubernetes hosted application containers: + + * Configure the Kubernetes |SRIOV| device plugin. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE sriovdp=enabled + done + + * If planning on running |DPDK| in Kubernetes hosted application + containers on this host, configure the number of 1G Huge pages required + on both |NUMA| nodes. + + :: + + for NODE in worker-0 worker-1; do + + # assign 10x 1G huge page on processor/numa-node 0 on worker-node to applications + system host-memory-modify -f application $NODE 0 -1G 10 + + # assign 10x 1G huge page on processor/numa-node 1 on worker-node to applications + system host-memory-modify -f application $NODE 1 -1G 10 + + done + + +------------------- +Unlock worker nodes +------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot in order to apply configuration changes and come +into service. This can take 5-10 minutes, depending on the performance of the +host machine. + +---------- +Next steps +---------- + +.. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/exporting-host-configurations.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/exporting-host-configurations.rst new file mode 100644 index 000000000..ddcc63d79 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/exporting-host-configurations.rst @@ -0,0 +1,53 @@ + +.. fdm1552927801987 +.. _exporting-host-configurations: + +========================== +Export Host Configurations +========================== + +You can generate a host configuration file from an existing system for +re-installation, upgrade, or maintenance purposes. + +.. rubric:: |context| + +You can generate a host configuration file using the :command:`system +host-bulk-export` command, and then use this file with the :command:`system +host-bulk-add` command to re-create the system. If required, you can modify the +file before using it. + +The configuration settings \(management |MAC| address, BM IP address, and so +on\) for all nodes except **controller-0** are written to the file. + +.. note:: + To ensure that the hosts are not powered on unexpectedly, the **power-on** + element for each host is commented out by default. + +.. rubric:: |prereq| + +To perform this procedure, you must be logged in as the **admin** user. + +.. rubric:: |proc| + +.. _exporting-host-configurations-steps-unordered-ntw-nw1-c2b: + +- Run the :command:`system host-bulk-export` command to create the host + configuration file. + + .. code-block:: none + + system host-bulk-export [--filename + + + - where is the path and name of the output file. If the + ``--filename`` option is not present, the default path ./hosts.xml is + used. + +.. rubric:: |postreq| + +To use the host configuration file, see :ref:`Reinstall a System Using an +Exported Host Configuration File +`. + +For details on the structure and elements of the file, see :ref:`Bulk Host XML +File Format `. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/inc-install-software-on-controller.rest b/doc/source/deploy_install_guides/r5_release/bare_metal/inc-install-software-on-controller.rest new file mode 100644 index 000000000..1fb1b3558 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/inc-install-software-on-controller.rest @@ -0,0 +1,43 @@ +.. incl-install-software-controller-0-aio-start + +#. Insert the bootable USB into a bootable USB port on the host you are + configuring as controller-0. + +#. Power on the host. + +#. Attach to a console, ensure the host boots from the USB, and wait for the + StarlingX Installer Menus. + +#. Make the following menu selections in the installer: + + #. First menu: Select 'All-in-one Controller Configuration' + + #. Second menu: Select 'Graphical Console' or 'Textual Console' depending on + your terminal access to the console port + +#. Wait for non-interactive install of software to complete and server to reboot. + This can take 5-10 minutes, depending on the performance of the server. + +.. incl-install-software-controller-0-aio-end + +.. incl-install-software-controller-0-standard-start + +#. Insert the bootable USB into a bootable USB port on the host you are + configuring as controller-0. + +#. Power on the host. + +#. Attach to a console, ensure the host boots from the USB, and wait for the + StarlingX Installer Menus. + +#. Make the following menu selections in the installer: + + #. First menu: Select 'Standard Controller Configuration' + + #. Second menu: Select 'Graphical Console' or 'Textual Console' depending on + your terminal access to the console port. + +#. Wait for non-interactive install of software to complete and server to reboot. + This can take 5-10 minutes, depending on the performance of the server. + +.. incl-install-software-controller-0-standard-end \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/inc-openstack-specific-host-config.rest b/doc/source/deploy_install_guides/r5_release/bare_metal/inc-openstack-specific-host-config.rest new file mode 100644 index 000000000..d5612745c --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/inc-openstack-specific-host-config.rest @@ -0,0 +1,109 @@ +.. incl-config-controller-0-openstack-specific-aio-simplex-start: + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + system host-label-assign controller-0 openstack-compute-node=enabled + system host-label-assign controller-0 openvswitch=enabled + system host-label-assign controller-0 sriov=enabled + +#. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has |OVS| (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, |OVS|-|DPDK| (|OVS| with the Data Plane + Development Kit, which is supported only on bare metal hardware) should be + used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + To deploy the default containerized |OVS|: + + :: + + system modify --vswitch_type none + + Do not run any vSwitch directly on the host, instead, use the containerized + |OVS| defined in the helm charts of stx-openstack manifest. + + To deploy |OVS|-|DPDK|, run the following command: + + :: + + system modify --vswitch_type ovs-dpdk + system host-cpu-modify -f vswitch -p0 1 controller-0 + + Once vswitch_type is set to |OVS|-|DPDK|, any subsequent nodes created + will default to automatically assigning 1 vSwitch core for |AIO| + controllers and 2 vSwitch cores for compute-labeled worker nodes. + + When using |OVS|-|DPDK|, configure vSwitch memory per |NUMA| node with + the following command: + + :: + + system host-memory-modify -f -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify -f vswitch -1G 1 worker-0 0 + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use + huge pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for |VMs| in an |OVS|-|DPDK| environment with + the command: + + :: + + system host-memory-modify -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify worker-0 0 -1G 10 + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking all compute-labeled worker nodes (and/or AIO + controllers) to apply the change. + + #. **For OpenStack only:** Set up disk partition for nova-local volume + group, which is needed for stx-openstack nova ephemeral disks. + + :: + + export NODE=controller-0 + + echo ">>> Getting root disk info" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + echo "Root disk: $ROOT_DISK, UUID: $ROOT_DISK_UUID" + + echo ">>>> Configuring nova-local" + NOVA_SIZE=34 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${NOVA_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + sleep 2 + +.. incl-config-controller-0-openstack-specific-aio-simplex-end: \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst new file mode 100644 index 000000000..5a1bcf3b5 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic.rst @@ -0,0 +1,72 @@ +==================================== +Bare metal Standard with Ironic R5.0 +==================================== + +-------- +Overview +-------- + +Ironic is an OpenStack project that provisions bare metal machines. For +information about the Ironic project, see +`Ironic Documentation `__. + +End user applications can be deployed on bare metal servers (instead of +virtual machines) by configuring OpenStack Ironic and deploying a pool of 1 or +more bare metal servers. + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-ironic.png + :scale: 50% + :alt: Standard with Ironic deployment configuration + + *Figure 1: Standard with Ironic deployment configuration* + +Bare metal servers must be connected to: + +* IPMI for OpenStack Ironic control +* ironic-provisioning-net tenant network via their untagged physical interface, + which supports PXE booting + +As part of configuring OpenStack Ironic in StarlingX: + +* An ironic-provisioning-net tenant network must be identified as the boot + network for bare metal nodes. +* An additional untagged physical interface must be configured on controller + nodes and connected to the ironic-provisioning-net tenant network. The + OpenStack Ironic tftpboot server will PXE boot the bare metal servers over + this interface. + +.. note:: + + Bare metal servers are NOT: + + * Running any OpenStack / StarlingX software; they are running end user + applications (for example, Glance Images). + * To be connected to the internal management network. + +------------ +Installation +------------ + +StarlingX currently supports only a bare metal installation of Ironic with a +standard configuration, either: + +* :doc:`controller_storage` + +* :doc:`dedicated_storage` + + +This guide assumes that you have a standard deployment installed and configured +with 2x controllers and at least 1x compute-labeled worker node, with the +StarlingX OpenStack application (stx-openstack) applied. + +.. toctree:: + :maxdepth: 1 + + ironic_hardware + ironic_install diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst new file mode 100644 index 000000000..6a59232a6 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_hardware.rst @@ -0,0 +1,51 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Ironic** deployment configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +* One or more bare metal hosts as Ironic nodes as well as tenant instance node. + +* BMC support on bare metal host and controller node connectivity to the BMC IP + address of bare metal hosts. + +For controller nodes: + +* Additional NIC port on both controller nodes for connecting to the + ironic-provisioning-net. + +For worker nodes: + +* If using a flat data network for the Ironic provisioning network, an additional + NIC port on one of the worker nodes is required. + +* Alternatively, use a VLAN data network for the Ironic provisioning network and + simply add the new data network to an existing interface on the worker node. + +* Additional switch ports / configuration for new ports on controller, worker, + and Ironic nodes, for connectivity to the Ironic provisioning network. + +----------------------------------- +BMC configuration of Ironic node(s) +----------------------------------- + +Enable BMC and allocate a static IP, username, and password in the BIOS settings. +For example, set: + +IP address + 10.10.10.126 + +username + root + +password + test123 diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst new file mode 100644 index 000000000..8e2a64e23 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/ironic_install.rst @@ -0,0 +1,392 @@ +================================ +Install Ironic on StarlingX R5.0 +================================ + +This section describes the steps to install Ironic on a standard configuration, +either: + +* **StarlingX R5.0 bare metal Standard with Controller Storage** deployment + configuration + +* **StarlingX R5.0 bare metal Standard with Dedicated Storage** deployment + configuration + +.. contents:: + :local: + :depth: 1 + +--------------------- +Enable Ironic service +--------------------- + +This section describes the pre-configuration required to enable the Ironic service. +All the commands in this section are for the StarlingX platform. + +First acquire administrative privileges: + +:: + + source /etc/platform/openrc + +******************************** +Download Ironic deployment image +******************************** + +The Ironic service requires a deployment image (kernel and ramdisk) which is +used to clean Ironic nodes and install the end-user's image. The cleaning done +by the deployment image wipes the disks and tests connectivity to the Ironic +conductor on the controller nodes via the Ironic Python Agent (IPA). + +The latest Ironic deployment image (**Ironic-kernel** and **Ironic-ramdisk**) +can be found here: + +* `Ironic-kernel ipa-centos8-master.kernel + `__ +* `Ironic-ramdisk ipa-centos8.initramfs + `__ + + +******************************************************* +Configure Ironic network on deployed standard StarlingX +******************************************************* + +#. Add an address pool for the Ironic network. This example uses `ironic-pool`: + + :: + + system addrpool-add --ranges 10.10.20.1-10.10.20.100 ironic-pool 10.10.20.0 24 + +#. Add the Ironic platform network. This example uses `ironic-net`: + + :: + + system addrpool-list | grep ironic-pool | awk '{print$2}' | xargs system network-add ironic-net ironic false + +#. Add the Ironic tenant network. This example uses `ironic-data`: + + .. note:: + + The tenant network is not the same as the platform network described in + the previous step. You can specify any name for the tenant network other + than ‘ironic’. If the name 'ironic' is used, a user override must be + generated to indicate the tenant network name. + + Refer to section `Generate user Helm overrides`_ for details. + + :: + + system datanetwork-add ironic-data flat + +#. Configure the new interfaces (for Ironic) on controller nodes and assign + them to the platform network. Host must be locked. This example uses the + platform network `ironic-net` that was named in a previous step. + + These new interfaces to the controllers are used to connect to the Ironic + provisioning network: + + **controller-0** + + :: + + system interface-network-assign controller-0 enp2s0 ironic-net + system host-if-modify -n ironic -c platform \ + --ipv4-mode static --ipv4-pool ironic-pool controller-0 enp2s0 + + # Apply the OpenStack Ironic node labels + system host-label-assign controller-0 openstack-ironic=enabled + + # Unlock the node to apply changes + system host-unlock controller-0 + + + **controller-1** + + :: + + system interface-network-assign controller-1 enp2s0 ironic-net + system host-if-modify -n ironic -c platform \ + --ipv4-mode static --ipv4-pool ironic-pool controller-1 enp2s0 + + # Apply the OpenStack Ironic node labels + system host-label-assign controller-1 openstack-ironic=enabled + + # Unlock the node to apply changes + system host-unlock controller-1 + +#. Configure the new interface (for Ironic) on one of the compute-labeled worker + nodes and assign it to the Ironic data network. This example uses the data + network `ironic-data` that was named in a previous step. + + :: + + system interface-datanetwork-assign worker-0 eno1 ironic-data + system host-if-modify -n ironicdata -c data worker-0 eno1 + +**************************** +Generate user Helm overrides +**************************** + +Ironic Helm Charts are included in the stx-openstack application. By default, +Ironic is disabled. + +To enable Ironic, update the following Ironic Helm Chart attributes: + +:: + + system helm-override-update stx-openstack ironic openstack \ + --set network.pxe.neutron_subnet_alloc_start=10.10.20.10 \ + --set network.pxe.neutron_subnet_gateway=10.10.20.1 \ + --set network.pxe.neutron_provider_network=ironic-data + +:command:`network.pxe.neutron_subnet_alloc_start` sets the DHCP start IP to +Neutron for Ironic node provision, and reserves several IPs for the platform. + +If the data network name for Ironic is changed, modify +:command:`network.pxe.neutron_provider_network` to the command above: + +:: + + --set network.pxe.neutron_provider_network=ironic-data + +******************************* +Apply stx-openstack application +******************************* + +Re-apply the stx-openstack application to apply the changes to Ironic: + +:: + + system helm-chart-attribute-modify stx-openstack ironic openstack \ + --enabled true + + system application-apply stx-openstack + +-------------------- +Start an Ironic node +-------------------- + +All the commands in this section are for the OpenStack application with +administrative privileges. + +From a new shell as a root user, without sourcing ``/etc/platform/openrc``: + +:: + + mkdir -p /etc/openstack + + tee /etc/openstack/clouds.yaml << EOF + clouds: + openstack_helm: + region_name: RegionOne + identity_api_version: 3 + endpoint_type: internalURL + auth: + username: 'admin' + password: 'Li69nux*' + project_name: 'admin' + project_domain_name: 'default' + user_domain_name: 'default' + auth_url: 'http://keystone.openstack.svc.cluster.local/v3' + EOF + + export OS_CLOUD=openstack_helm + +******************** +Create Glance images +******************** + +#. Create the **ironic-kernel** image: + + :: + + openstack image create \ + --file ~/coreos_production_pxe-stable-stein.vmlinuz \ + --disk-format aki \ + --container-format aki \ + --public \ + ironic-kernel + +#. Create the **ironic-ramdisk** image: + + :: + + openstack image create \ + --file ~/coreos_production_pxe_image-oem-stable-stein.cpio.gz \ + --disk-format ari \ + --container-format ari \ + --public \ + ironic-ramdisk + +#. Create the end user application image (for example, CentOS): + + :: + + openstack image create \ + --file ~/CentOS-7-x86_64-GenericCloud-root.qcow2 \ + --public --disk-format \ + qcow2 --container-format bare centos + +********************* +Create an Ironic node +********************* + +#. Create a node: + + :: + + openstack baremetal node create --driver ipmi --name ironic-test0 + +#. Add IPMI information: + + :: + + openstack baremetal node set \ + --driver-info ipmi_address=10.10.10.126 \ + --driver-info ipmi_username=root \ + --driver-info ipmi_password=test123 \ + --driver-info ipmi_terminal_port=623 ironic-test0 + +#. Set `ironic-kernel` and `ironic-ramdisk` images driver information, + on this bare metal node: + + :: + + openstack baremetal node set \ + --driver-info deploy_kernel=$(openstack image list | grep ironic-kernel | awk '{print$2}') \ + --driver-info deploy_ramdisk=$(openstack image list | grep ironic-ramdisk | awk '{print$2}') \ + ironic-test0 + +#. Set resource properties on this bare metal node based on actual Ironic node + capacities: + + :: + + openstack baremetal node set \ + --property cpus=4 \ + --property cpu_arch=x86_64\ + --property capabilities="boot_option:local" \ + --property memory_mb=65536 \ + --property local_gb=400 \ + --resource-class bm ironic-test0 + +#. Add pxe_template location: + + :: + + openstack baremetal node set --driver-info \ + pxe_template='/var/lib/openstack/lib64/python2.7/site-packages/ironic/drivers/modules/ipxe_config.template' \ + ironic-test0 + +#. Create a port to identify the specific port used by the Ironic node. + Substitute **a4:bf:01:2b:3b:c8** with the MAC address for the Ironic node + port which connects to the Ironic network: + + :: + + openstack baremetal port create \ + --node $(openstack baremetal node list | grep ironic-test0 | awk '{print$2}') \ + --pxe-enabled true a4:bf:01:2b:3b:c8 + +#. Change node state to `manage`: + + :: + + openstack baremetal node manage ironic-test0 + +#. Make node available for deployment: + + :: + + openstack baremetal node provide ironic-test0 + +#. Wait for ironic-test0 provision-state: available: + + :: + + openstack baremetal node show ironic-test0 + +--------------------------------- +Deploy an instance on Ironic node +--------------------------------- + +All the commands in this section are for the OpenStack application, but this +time with *tenant* specific privileges. + +#. From a new shell as a root user, without sourcing ``/etc/platform/openrc``: + + :: + + mkdir -p /etc/openstack + + tee /etc/openstack/clouds.yaml << EOF + clouds: + openstack_helm: + region_name: RegionOne + identity_api_version: 3 + endpoint_type: internalURL + auth: + username: 'joeuser' + password: 'mypasswrd' + project_name: 'intel' + project_domain_name: 'default' + user_domain_name: 'default' + auth_url: 'http://keystone.openstack.svc.cluster.local/v3' + EOF + + export OS_CLOUD=openstack_helm + +#. Create flavor. + + Set resource CUSTOM_BM corresponding to **--resource-class bm**: + + :: + + openstack flavor create --ram 4096 --vcpus 4 --disk 400 \ + --property resources:CUSTOM_BM=1 \ + --property resources:VCPU=0 \ + --property resources:MEMORY_MB=0 \ + --property resources:DISK_GB=0 \ + --property capabilities:boot_option='local' \ + bm-flavor + + See `Adding scheduling information + `__ + and `Configure Nova flavors + `__ + for more information. + +#. Enable service + + List the compute services: + + :: + + openstack compute service list + + Set compute service properties: + + :: + + openstack compute service set --enable controller-0 nova-compute + +#. Create instance + + .. note:: + + The :command:`keypair create` command is optional. It is not required to + enable a bare metal instance. + + :: + + openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey + + + Create 2 new servers, one bare metal and one virtual: + + :: + + openstack server create --image centos --flavor bm-flavor \ + --network baremetal --key-name mykey bm + + openstack server create --image centos --flavor m1.small \ + --network baremetal --key-name mykey vm diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt b/doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt new file mode 100644 index 000000000..61a686201 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/prep_servers.txt @@ -0,0 +1,17 @@ +Prior to starting the StarlingX installation, the bare metal servers must be in +the following condition: + +* Physically installed + +* Cabled for power + +* Cabled for networking + + * Far-end switch ports should be properly configured to realize the networking + shown in Figure 1. + +* All disks wiped + + * Ensures that servers will boot from either the network or USB storage (if present) + +* Powered off \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-or-a-host.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-or-a-host.rst new file mode 100644 index 000000000..816d5ae03 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-or-a-host.rst @@ -0,0 +1,39 @@ + +.. deo1552927844327 +.. _reinstalling-a-system-or-a-host: + +============================ +Reinstall a System or a Host +============================ + +You can reinstall individual hosts or the entire system if necessary. +Reinstalling host software or deleting and re-adding a host node may be +required to complete certain configuration changes. + +.. rubric:: |context| + +For a summary of changes that require system or host reinstallation, see +|node-doc|: :ref:`Configuration Changes Requiring Re-installation +`. + +To reinstall an entire system, refer to the Installation Guide for your system +type \(for example, Standard or All-in-one\). + +.. note:: + To simplify system reinstallation, you can export and reuse an existing + system configuration. For more information, see :ref:`Reinstalling a System + Using an Exported Host Configuration File + `. + +To reinstall the software on a host using the Host Inventory controls, see +|node-doc|: :ref:`Host Inventory `. In some cases, you must delete +the host instead, and then re-add it using the standard host installation +procedure. This applies if the system inventory record must be corrected to +complete the configuration change \(for example, if the |MAC| address of the +management interface has changed\). + +- :ref:`Reinstalling a System Using an Exported Host Configuration File + ` + +- :ref:`Exporting Host Configurations ` + diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-using-an-exported-host-configuration-file.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-using-an-exported-host-configuration-file.rst new file mode 100644 index 000000000..5b1700614 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/reinstalling-a-system-using-an-exported-host-configuration-file.rst @@ -0,0 +1,45 @@ + +.. wuh1552927822054 +.. _reinstalling-a-system-using-an-exported-host-configuration-file: + +============================================================ +Reinstall a System Using an Exported Host Configuration File +============================================================ + +You can reinstall a system using the host configuration file that is generated +using the :command:`host-bulk-export` command. + +.. rubric:: |prereq| + +For the following procedure, **controller-0** must be the active controller. + +.. rubric:: |proc| + +#. Create a host configuration file using the :command:`system + host-bulk-export` command, as described in :ref:`Exporting Host + Configurations `. + +#. Copy the host configuration file to a USB drive or somewhere off the + controller hard disk. + +#. Edit the host configuration file as needed, for example to specify power-on + or |BMC| information. + +#. Delete all the hosts except **controller-0** from the inventory. + +#. Reinstall the |prod| software on **controller-0**, which must be the active + controller. + +#. Run :command:`Ansible Bootstrap playbook`. + +#. Follow the instructions for using the :command:`system host-bulk-add` + command, as detailed in :ref:`Adding Hosts in Bulk `. + +.. rubric:: |postreq| + +After adding the host, you must provision it according to the requirements of +the personality. + +.. xbooklink For more information, see :ref:`Installing, Configuring, and + Unlocking Nodes `, for your system, + and follow the *Configure* steps for the appropriate node personality. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage.rst new file mode 100644 index 000000000..f3f377863 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage.rst @@ -0,0 +1,22 @@ +======================================================= +Bare metal Standard with Rook Storage Installation R5.0 +======================================================= + +-------- +Overview +-------- + +.. include:: ../desc_rook_storage.txt + +.. include:: ../ipv6_note.txt + + +------------ +Installation +------------ + +.. toctree:: + :maxdepth: 1 + + rook_storage_hardware + rook_storage_install_kubernetes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_hardware.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_hardware.rst new file mode 100644 index 000000000..0ee2dd69d --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_hardware.rst @@ -0,0 +1,62 @@ +===================== +Hardware Requirements +===================== + +This section describes the hardware requirements and server preparation for a +**StarlingX R5.0 bare metal Standard with Rook Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +----------------------------- +Minimum hardware requirements +----------------------------- + +The recommended minimum hardware requirements for bare metal servers for various +host types are: + ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum Requirement | Controller Node | Worker Node for rook | Worker Node for | +| | | storage | application | ++=====================+===========================+=======================+=======================+ +| Number of servers | 2 | 2-9 | 2-100 | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum processor | Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket | +| class | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum memory | 64 GB | 64 GB | 32 GB | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Primary disk | 500 GB SSD or NVMe (see | 120 GB (min. 10k RPM) | 120 GB (min. 10k RPM) | +| | :doc:`../../nvme_config`) | | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Additional disks | None | - 1 or more 500 GB | - For OpenStack, | +| | | (min. 10K RPM) for | recommend 1 or more | +| | | Ceph OSD | 500 GB (min. 10K | +| | | - Recommended, but | RPM) for VM | +| | | not required: 1 or | ephemeral storage | +| | | more SSDs or NVMe | | +| | | drives for Ceph | | +| | | journals (min. 1024 | | +| | | MiB per OSD | | +| | | journal) | | ++---------------------+---------------------------+-----------------------+-----------------------+ +| Minimum network | - Mgmt/Cluster: | - Mgmt/Cluster: | - Mgmt/Cluster: | +| ports | 1x10GE | 1x10GE | 1x10GE | +| | - OAM: 1x1GE | | - Data: 1 or more | +| | | | x 10GE | ++---------------------+---------------------------+-----------------------+-----------------------+ +| BIOS settings | - Hyper-Threading technology enabled | +| | - Virtualization technology enabled | +| | - VT for directed I/O enabled | +| | - CPU power and performance policy set to performance | +| | - CPU C state control disabled | +| | - Plug & play BMC detection disabled | ++---------------------+---------------------------+-----------------------+-----------------------+ + +-------------------------- +Prepare bare metal servers +-------------------------- + +.. include:: prep_servers.txt diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst new file mode 100644 index 000000000..d5aca828e --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst @@ -0,0 +1,731 @@ +===================================================================== +Install StarlingX Kubernetes on Bare Metal Standard with Rook Storage +===================================================================== + +This section describes the steps to install the StarlingX Kubernetes platform +on a **StarlingX R5.0 bare metal Standard with Rook Storage** deployment +configuration. + +.. contents:: + :local: + :depth: 1 + +------------------- +Create bootable USB +------------------- + +Refer to :doc:`/deploy_install_guides/bootable_usb` for instructions on how to +create a bootable USB with the StarlingX ISO on your system. + +-------------------------------- +Install software on controller-0 +-------------------------------- + +.. incl-install-software-controller-0-standard-start: + +#. Insert the bootable USB into a bootable USB port on the host you are + configuring as controller-0. + +#. Power on the host. + +#. Attach to a console, ensure the host boots from the USB, and wait for the + StarlingX Installer Menus. + +#. Make the following menu selections in the installer: + + #. First menu: Select 'Standard Controller Configuration' + #. Second menu: Select 'Graphical Console' or 'Textual Console' depending on + your terminal access to the console port + +#. Wait for non-interactive install of software to complete and server to reboot. + This can take 5-10 minutes, depending on the performance of the server. + +.. incl-install-software-controller-0-standard-end: + +-------------------------------- +Bootstrap system on controller-0 +-------------------------------- + +.. incl-bootstrap-sys-controller-0-standard-start: + +#. Login using the username / password of "sysadmin" / "sysadmin". + + When logging in for the first time, you will be forced to change the password. + + :: + + Login: sysadmin + Password: + Changing password for sysadmin. + (current) UNIX Password: sysadmin + New Password: + (repeat) New Password: + +#. Verify and/or configure IP connectivity. + + External connectivity is required to run the Ansible bootstrap playbook. The + StarlingX boot image will DHCP out all interfaces so the server may have + obtained an IP address and have external IP connectivity if a DHCP server is + present in your environment. Verify this using the :command:`ip addr` and + :command:`ping 8.8.8.8` commands. + + Otherwise, manually configure an IP address and default IP route. Use the + PORT, IP-ADDRESS/SUBNET-LENGTH and GATEWAY-IP-ADDRESS applicable to your + deployment environment. + + :: + + sudo ip address add / dev + sudo ip link set up dev + sudo ip route add default via dev + ping 8.8.8.8 + +#. Specify user configuration overrides for the Ansible bootstrap playbook. + + Ansible is used to bootstrap StarlingX on controller-0. Key files for Ansible + configuration are: + + ``/etc/ansible/hosts`` + The default Ansible inventory file. Contains a single host: localhost. + + ``/usr/share/ansible/stx-ansible/playbooks/bootstrap.yml`` + The Ansible bootstrap playbook. + + ``/usr/share/ansible/stx-ansible/playbooks/host_vars/bootstrap/default.yml`` + The default configuration values for the bootstrap playbook. + + ``sysadmin home directory ($HOME)`` + The default location where Ansible looks for and imports user + configuration override files for hosts. For example: ``$HOME/.yml``. + + .. include:: ../ansible_install_time_only.txt + + Specify the user configuration override file for the Ansible bootstrap + playbook using one of the following methods: + + #. Use a copy of the default.yml file listed above to provide your overrides. + + The default.yml file lists all available parameters for bootstrap + configuration with a brief description for each parameter in the file comments. + + To use this method, copy the default.yml file listed above to + ``$HOME/localhost.yml`` and edit the configurable values as desired. + + #. Create a minimal user configuration override file. + + To use this method, create your override file at ``$HOME/localhost.yml`` + and provide the minimum required parameters for the deployment configuration + as shown in the example below. Use the OAM IP SUBNET and IP ADDRESSing + applicable to your deployment environment. + + :: + + cd ~ + cat < localhost.yml + system_mode: duplex + + dns_servers: + - 8.8.8.8 + - 8.8.4.4 + + external_oam_subnet: / + external_oam_gateway_address: + external_oam_floating_address: + external_oam_node_0_address: + external_oam_node_1_address: + + admin_username: admin + admin_password: + ansible_become_pass: + + # Add these lines to configure Docker to use a proxy server + # docker_http_proxy: http://my.proxy.com:1080 + # docker_https_proxy: https://my.proxy.com:1443 + # docker_no_proxy: + # - 1.2.3.4 + + EOF + + Refer to :doc:`/deploy_install_guides/r5_release/ansible_bootstrap_configs` + for information on additional Ansible bootstrap configurations for advanced + Ansible bootstrap scenarios, such as Docker proxies when deploying behind a + firewall, etc. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +#. Run the Ansible bootstrap playbook: + + :: + + ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml + + Wait for Ansible bootstrap playbook to complete. + This can take 5-10 minutes, depending on the performance of the host machine. + +.. incl-bootstrap-sys-controller-0-standard-end: + + +---------------------- +Configure controller-0 +---------------------- + +.. incl-config-controller-0-storage-start: + +#. Acquire admin credentials: + + :: + + source /etc/platform/openrc + +#. Configure the OAM and MGMT interfaces of controller-0 and specify the + attached networks. Use the OAM and MGMT port names, for example eth0, that are + applicable to your deployment environment. + + :: + + OAM_IF= + MGMT_IF= + system host-if-modify controller-0 lo -c none + IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}') + for UUID in $IFNET_UUIDS; do + system interface-network-remove ${UUID} + done + system host-if-modify controller-0 $OAM_IF -c platform + system interface-network-assign controller-0 $OAM_IF oam + system host-if-modify controller-0 $MGMT_IF -c platform + system interface-network-assign controller-0 $MGMT_IF mgmt + system interface-network-assign controller-0 $MGMT_IF cluster-host + +#. Configure NTP servers for network time synchronization: + + :: + + system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org + +#. If required, and not already done as part of bootstrap, configure Docker to + use a proxy server. + + #. List Docker proxy parameters: + + :: + + system service-parameter-list platform docker + + #. Refer to :doc:`/../../configuration/docker_proxy_config` for + details about Docker proxy settings. + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to controller-0 in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + system host-label-assign controller-0 openstack-control-plane=enabled + +#. **For OpenStack only:** Configure the system setting for the vSwitch. + + StarlingX has |OVS| (kernel-based) vSwitch configured as default: + + * Runs in a container; defined within the helm charts of stx-openstack + manifest. + * Shares the core(s) assigned to the platform. + + If you require better performance, |OVS|-|DPDK| (|OVS| with the Data Plane + Development Kit, which is supported only on bare metal hardware) should be + used: + + * Runs directly on the host (it is not containerized). + * Requires that at least 1 core be assigned/dedicated to the vSwitch function. + + To deploy the default containerized |OVS|: + + :: + + system modify --vswitch_type none + + Do not run any vSwitch directly on the host, instead, use the containerized + |OVS| defined in the helm charts of stx-openstack manifest. + + To deploy |OVS|-|DPDK|, run the following command: + + :: + + system modify --vswitch_type ovs-dpdk + system host-cpu-modify -f vswitch -p0 1 controller-0 + + Once vswitch_type is set to |OVS|-|DPDK|, any subsequent nodes created will + default to automatically assigning 1 vSwitch core for AIO controllers and 2 + vSwitch cores for compute-labeled worker nodes. + + When using |OVS|-|DPDK|, configure vSwitch memory per NUMA node with the + following + command: + + :: + + system host-memory-modify -f -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify -f vswitch -1G 1 worker-0 0 + + |VMs| created in an |OVS|-|DPDK| environment must be configured to use huge + pages to enable networking and must use a flavor with property: + hw:mem_page_size=large + + Configure the huge pages for VMs in an |OVS|-|DPDK| environment with the + command: + + :: + + system host-memory-modify -1G <1G hugepages number> + + For example: + + :: + + system host-memory-modify worker-0 0 -1G 10 + + .. note:: + + After controller-0 is unlocked, changing vswitch_type requires + locking and unlocking all compute-labeled worker nodes (and/or AIO + controllers) to apply the change. + +.. incl-config-controller-0-storage-end: + +******************************** +Rook-specific host configuration +******************************** + +.. important:: + + **This step is required only if the StarlingX Rook application will be + installed.** + +**For Rook only:** Assign Rook host labels to controller-0 in support of +installing the rook-ceph-apps manifest/helm-charts later and add ceph-rook +as storage backend: + +:: + + system host-label-assign controller-0 ceph-mon-placement=enabled + system host-label-assign controller-0 ceph-mgr-placement=enabled + system storage-backend-add ceph-rook --confirmed + +------------------- +Unlock controller-0 +------------------- + +Unlock controller-0 in order to bring it into service: + +:: + + system host-unlock controller-0 + +Controller-0 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +------------------------------------------------- +Install software on controller-1 and worker nodes +------------------------------------------------- + +#. Power on the controller-1 server and force it to network boot with the + appropriate BIOS boot options for your particular server. + +#. As controller-1 boots, a message appears on its console instructing you to + configure the personality of the node. + +#. On the console of controller-0, list hosts to see newly discovered controller-1 + host (hostname=None): + + :: + + system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Using the host id, set the personality of this host to 'controller': + + :: + + system host-update 2 personality=controller + + This initiates the install of software on controller-1. + This can take 5-10 minutes, depending on the performance of the host machine. + +#. While waiting for the previous step to complete, power on the worker nodes. + Set the personality to 'worker' and assign a unique hostname for each. + + For example, power on worker-0 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 3 personality=worker hostname=worker-0 + + Repeat for worker-1. Power on worker-1 and wait for the new host (hostname=None) to + be discovered by checking 'system host-list': + + :: + + system host-update 4 personality=worker hostname=worker-1 + + For rook storage, there is no storage personality. Some hosts with worker personality + providers storage service. Here we still named these worker host storage-x. + Repeat for storage-0 and storage-1. Power on storage-0, storage-1 and wait for the + new host (hostname=None) to be discovered by checking 'system host-list': + + :: + + system host-update 5 personality=worker hostname=storage-0 + system host-update 6 personality=worker hostname=storage-1 + +#. Wait for the software installation on controller-1, worker-0, and worker-1 to + complete, for all servers to reboot, and for all to show as locked/disabled/online in + 'system host-list'. + + :: + + system host-list + + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | locked | disabled | online | + | 3 | worker-0 | worker | locked | disabled | online | + | 4 | worker-1 | worker | locked | disabled | online | + | 5 | storage-0 | worker | locked | disabled | online | + | 6 | storage-1 | worker | locked | disabled | online | + +----+--------------+-------------+----------------+-------------+--------------+ + +---------------------- +Configure controller-1 +---------------------- + +.. incl-config-controller-1-start: + +Configure the OAM and MGMT interfaces of controller-0 and specify the attached +networks. Use the OAM and MGMT port names, for example eth0, that are applicable +to your deployment environment. + +(Note that the MGMT interface is partially set up automatically by the network +install procedure.) + +:: + + OAM_IF= + MGMT_IF= + system host-if-modify controller-1 $OAM_IF -c platform + system interface-network-assign controller-1 $OAM_IF oam + system interface-network-assign controller-1 $MGMT_IF cluster-host + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +**For OpenStack only:** Assign OpenStack host labels to controller-1 in support +of installing the stx-openstack manifest and helm-charts later. + +:: + + system host-label-assign controller-1 openstack-control-plane=enabled + +.. incl-config-controller-1-end: + +******************************** +Rook-specific host configuration +******************************** + +.. important:: + + **This step is required only if the StarlingX Rook application will be + installed.** + +**For Rook only:** Assign Rook host labels to controller-1 in support of +installing the rook-ceph-apps manifest/helm-charts later: + +:: + + system host-label-assign controller-1 ceph-mon-placement=enabled + system host-label-assign controller-1 ceph-mgr-placement=enabled + +------------------- +Unlock controller-1 +------------------- + +.. incl-unlock-controller-1-start: + +Unlock controller-1 in order to bring it into service: + +:: + + system host-unlock controller-1 + +Controller-1 will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host +machine. + +.. incl-unlock-controller-1-end: + +---------------------- +Configure worker nodes +---------------------- + +#. Assign the cluster-host network to the MGMT interface for the worker nodes: + + (Note that the MGMT interfaces are partially set up automatically by the + network install procedure.) + + :: + + for NODE in worker-0 worker-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. Configure data interfaces for worker nodes. Use the DATA port names, for + example eth0, that are applicable to your deployment environment. + + .. important:: + + This step is **required** for OpenStack. + + This step is optional for Kubernetes: Do this step if using SRIOV network + attachments in hosted application containers. + + For Kubernetes SRIOV network attachments: + + * Configure SRIOV device plug in: + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign ${NODE} sriovdp=enabled + done + + * If planning on running DPDK in containers on this host, configure the number + of 1G Huge pages required on both NUMA nodes: + + :: + + for NODE in worker-0 worker-1; do + system host-memory-modify ${NODE} 0 -1G 100 + system host-memory-modify ${NODE} 1 -1G 100 + done + + For both Kubernetes and OpenStack: + + :: + + DATA0IF= + DATA1IF= + PHYSNET0='physnet0' + PHYSNET1='physnet1' + SPL=/tmp/tmp-system-port-list + SPIL=/tmp/tmp-system-host-if-list + + # configure the datanetworks in sysinv, prior to referencing it + # in the ``system host-if-modify`` command'. + system datanetwork-add ${PHYSNET0} vlan + system datanetwork-add ${PHYSNET1} vlan + + for NODE in worker-0 worker-1; do + echo "Configuring interface for: $NODE" + set -ex + system host-port-list ${NODE} --nowrap > ${SPL} + system host-if-list -a ${NODE} --nowrap > ${SPIL} + DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}') + DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}') + DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}') + DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}') + DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}') + DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}') + DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}') + DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}') + system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID} + system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID} + system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0} + system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1} + set +ex + done + +************************************* +OpenStack-specific host configuration +************************************* + +.. important:: + + **This step is required only if the StarlingX OpenStack application + (stx-openstack) will be installed.** + +#. **For OpenStack only:** Assign OpenStack host labels to the worker nodes in + support of installing the stx-openstack manifest and helm-charts later. + + :: + + for NODE in worker-0 worker-1; do + system host-label-assign $NODE openstack-compute-node=enabled + system host-label-assign $NODE openvswitch=enabled + system host-label-assign $NODE sriov=enabled + done + +#. **For OpenStack only:** Set up disk partition for nova-local volume group, + which is needed for stx-openstack nova ephemeral disks. + + :: + + for NODE in worker-0 worker-1; do + echo "Configuring Nova local for: $NODE" + ROOT_DISK=$(system host-show ${NODE} | grep rootfs | awk '{print $4}') + ROOT_DISK_UUID=$(system host-disk-list ${NODE} --nowrap | grep ${ROOT_DISK} | awk '{print $2}') + PARTITION_SIZE=10 + NOVA_PARTITION=$(system host-disk-partition-add -t lvm_phys_vol ${NODE} ${ROOT_DISK_UUID} ${PARTITION_SIZE}) + NOVA_PARTITION_UUID=$(echo ${NOVA_PARTITION} | grep -ow "| uuid | [a-z0-9\-]* |" | awk '{print $4}') + system host-lvg-add ${NODE} nova-local + system host-pv-add ${NODE} nova-local ${NOVA_PARTITION_UUID} + done + +-------------------- +Unlock worker nodes +-------------------- + +Unlock worker nodes in order to bring them into service: + +:: + + for NODE in worker-0 worker-1; do + system host-unlock $NODE + done + +The worker nodes will reboot in order to apply configuration changes and come into +service. This can take 5-10 minutes, depending on the performance of the host machine. + +----------------------- +Configure storage nodes +----------------------- + +#. Assign the cluster-host network to the MGMT interface for the storage nodes. + + Note that the MGMT interfaces are partially set up by the network install procedure. + + :: + + for NODE in storage-0 storage-1; do + system interface-network-assign $NODE mgmt0 cluster-host + done + +#. **For Rook only:** Assign Rook host labels to storage-0 in support + of installing the rook-ceph-apps manifest/helm-charts later: + + :: + + system host-label-assign storage-0 ceph-mon-placement=enabled + +-------------------- +Unlock storage nodes +-------------------- + +Unlock storage nodes in order to bring them into service: + +:: + + for STORAGE in storage-0 storage-1; do + system host-unlock $STORAGE + done + +The storage nodes will reboot in order to apply configuration changes and come +into service. This can take 5-10 minutes, depending on the performance of the host machine. + +------------------------------------------------- +Install Rook application manifest and helm-charts +------------------------------------------------- + +On host storage-0 and storage-1: + +#. Erase gpt header of disk sdb. + + :: + + $ system host-disk-wipe -s --confirm storage-0 /dev/sdb + $ system host-disk-wipe -s --confirm storage-1 /dev/sdb + +#. Wait for application "rook-ceph-apps" uploaded + + :: + + $ source /etc/platform/openrc + $ system application-list + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + | application | version | manifest name | manifest file | status | progress | + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + | oidc-auth-apps | 1.0-0 | oidc-auth-manifest | manifest.yaml | uploaded | completed | + | platform-integ-apps | 1.0-8 | platform-integration-manifest | manifest.yaml | uploaded | completed | + | rook-ceph-apps | 1.0-1 | rook-ceph-manifest | manifest.yaml | uploaded | completed | + +---------------------+---------+-------------------------------+---------------+----------+-----------+ + +#. Edit values.yaml for rook-ceph-apps. + + :: + + cluster: + storage: + nodes: + - name: storage-0 + devices: + - name: /dev/disk/by-path/pci-0000:00:03.0-ata-2.0 + - name: storage-1 + devices: + - name: /dev/disk/by-path/pci-0000:00:03.0-ata-2.0 + +#. Update rook-ceph-apps override value. + + :: + + system helm-override-update rook-ceph-apps rook-ceph kube-system --values values.yaml + +#. Apply the rook-ceph-apps application. + + :: + + system application-apply rook-ceph-apps + +#. Wait for OSDs pod ready. + + :: + + kubectl get pods -n kube-system + rook-ceph-mgr-a-ddffc8fbb-zkvln 1/1 Running 0 66s + rook-ceph-mon-a-c67fdb6c8-tlbvk 1/1 Running 0 2m11s + rook-ceph-mon-b-76969d8685-wcq62 1/1 Running 0 2m2s + rook-ceph-mon-c-5bc47c6cb9-vm4j8 1/1 Running 0 97s + rook-ceph-operator-6fc8cfb68b-bb57z 1/1 Running 1 7m9s + rook-ceph-osd-0-689b6f65b-2nvcx 1/1 Running 0 12s + rook-ceph-osd-1-7bfd69fdf9-vjqmp 1/1 Running 0 4s + rook-ceph-osd-prepare-rook-storage-0-hf28p 0/1 Completed 0 50s + rook-ceph-osd-prepare-rook-storage-1-r6lsd 0/1 Completed 0 50s + rook-ceph-tools-84c7fff88c-x5trx 1/1 Running 0 6m11s + +---------- +Next steps +---------- + +.. include:: ../kubernetes_install_next.txt diff --git a/doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt b/doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt new file mode 100644 index 000000000..ec00437cf --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_aio_duplex.txt @@ -0,0 +1,29 @@ +The All-in-one Duplex (AIO-DX) deployment option provides a pair of high +availability (HA) servers with each server providing all three cloud functions +(controller, worker, and storage). + +An AIO-DX configuration provides the following benefits: + +* Only a small amount of cloud processing and storage power is required +* Application consolidation using multiple containers or virtual machines on a + single pair of physical servers +* High availability (HA) services run on the controller function across two + physical servers in either active/active or active/standby mode +* A storage back end solution using a two-node CEPH deployment across two servers +* Containers or virtual machines scheduled on both worker functions +* Protection against overall server hardware fault, where + + * All controller HA services go active on the remaining healthy server + * All virtual machines are recovered on the remaining healthy server + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-duplex.png + :scale: 50% + :alt: All-in-one Duplex deployment configuration + + *Figure 1: All-in-one Duplex deployment configuration* \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt b/doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt new file mode 100644 index 000000000..56b22e94b --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_aio_simplex.txt @@ -0,0 +1,24 @@ +The All-in-one Simplex (AIO-SX) deployment option provides all three cloud +functions (controller, worker, and storage) on a single server with the +following benefits: + +* Requires only a small amount of cloud processing and storage power +* Application consolidation using multiple containers or virtual machines on a + single pair of physical servers +* A storage backend solution using a single-node CEPH deployment + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-simplex.png + :scale: 50% + :alt: All-in-one Simplex deployment configuration + + *Figure 1: All-in-one Simplex deployment configuration* + +An AIO-SX deployment gives no protection against overall server hardware fault. +Hardware component protection can be enabled with, for example, a hardware RAID +or 2x Port LAG in the deployment. \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt b/doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt new file mode 100644 index 000000000..3d606d532 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_controller_storage.txt @@ -0,0 +1,28 @@ +The Standard with Controller Storage deployment option provides two high +availability (HA) controller nodes and a pool of up to 10 worker nodes. + +A Standard with Controller Storage configuration provides the following benefits: + +* A pool of up to 10 worker nodes +* High availability (HA) services run across the controller nodes in either + active/active or active/standby mode +* A storage back end solution using a two-node CEPH deployment across two + controller servers +* Protection against overall controller and worker node failure, where + + * On overall controller node failure, all controller HA services go active on + the remaining healthy controller node + * On overall worker node failure, virtual machines and containers are + recovered on the remaining healthy worker nodes + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-controller-storage.png + :scale: 50% + :alt: Standard with Controller Storage deployment configuration + + *Figure 1: Standard with Controller Storage deployment configuration* \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt b/doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt new file mode 100644 index 000000000..79371f446 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_dedicated_storage.txt @@ -0,0 +1,24 @@ +The Standard with Dedicated Storage deployment option is a standard installation +with independent controller, worker, and storage nodes. + +A Standard with Dedicated Storage configuration provides the following benefits: + +* A pool of up to 100 worker nodes +* A 2x node high availability (HA) controller cluster with HA services running + across the controller nodes in either active/active or active/standby mode +* A storage back end solution using a two-to-9x node HA CEPH storage cluster + that supports a replication factor of two or three +* Up to four groups of 2x storage nodes, or up to three groups of 3x storage + nodes + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-dedicated-storage.png + :scale: 50% + :alt: Standard with Dedicated Storage deployment configuration + + *Figure 1: Standard with Dedicated Storage deployment configuration* \ No newline at end of file diff --git a/doc/source/deploy_install_guides/r5_release/desc_rook_storage.txt b/doc/source/deploy_install_guides/r5_release/desc_rook_storage.txt new file mode 100644 index 000000000..eb5ebc287 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/desc_rook_storage.txt @@ -0,0 +1,22 @@ +The Standard with Rook Storage deployment option is a standard installation +with independent controller and worker nodes. + +A Standard with Rook Storage configuration provides the following benefits: + +* A pool of up to 100 worker nodes +* A 2x node high availability (HA) controller cluster with HA services running + across the controller nodes in either active/active or active/standby mode +* A Rook storage back end solution using a two-to-9x node HA Rook storage + cluster that supports a replication factor of two or three + +.. note:: + + If you are behind a corporate firewall or proxy, you need to set proxy + settings. Refer to :doc:`/../../configuration/docker_proxy_config` for + details. + +.. figure:: ../figures/starlingx-deployment-options-controller-storage.png + :scale: 50% + :alt: Standard with Rook Storage deployment configuration + + *Figure 1: Standard with Rook Storage deployment configuration* diff --git a/doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst b/doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst new file mode 100644 index 000000000..26ec39518 --- /dev/null +++ b/doc/source/deploy_install_guides/r5_release/distributed_cloud/index.rst @@ -0,0 +1,310 @@ +=================================== +Distributed Cloud Installation R5.0 +=================================== + +This section describes how to install and configure the StarlingX distributed +cloud deployment. + +.. contents:: + :local: + :depth: 1 + +-------- +Overview +-------- + +Distributed cloud configuration supports an edge computing solution by +providing central management and orchestration for a geographically +distributed network of StarlingX Kubernetes edge systems/clusters. + +The StarlingX distributed cloud implements the OpenStack Edge Computing +Groups's MVP `Edge Reference Architecture +`_, +specifically the "Distributed Control Plane" scenario. + +The StarlingX distributed cloud deployment is designed to meet the needs of +edge-based data centers with centralized orchestration and independent control +planes, and in which Network Function Cloudification (NFC) worker resources +are localized for maximum responsiveness. The architecture features: + +- Centralized orchestration of edge cloud control planes. +- Full synchronized control planes at edge clouds (that is, Kubernetes cluster + master and nodes), with greater benefits for local services, such as: + + - Reduced network latency. + - Operational availability, even if northbound connectivity + to the central cloud is lost. + +The system supports a scalable number of StarlingX Kubernetes edge +systems/clusters, which are centrally managed and synchronized over L3 +networks from a central cloud. Each edge system is also highly scalable, from +a single node StarlingX Kubernetes deployment to a full standard cloud +configuration with controller, worker and storage nodes. + +------------------------------ +Distributed cloud architecture +------------------------------ + +A distributed cloud system consists of a central cloud, and one or more +subclouds connected to the SystemController region central cloud over L3 +networks, as shown in Figure 1. + +- **Central cloud** + + The central cloud provides a *RegionOne* region for managing the physical + platform of the central cloud and the *SystemController* region for managing + and orchestrating over the subclouds. + + - **RegionOne** + + In the Horizon GUI, RegionOne is the name of the access mode, or region, + used to manage the nodes in the central cloud. + + - **SystemController** + + In the Horizon GUI, SystemController is the name of the access mode, or + region, used to manage the subclouds. + + You can use the SystemController to add subclouds, synchronize select + configuration data across all subclouds and monitor subcloud operations + and alarms. System software updates for the subclouds are also centrally + managed and applied from the SystemController. + + DNS, NTP, and other select configuration settings are centrally managed + at the SystemController and pushed to the subclouds in parallel to + maintain synchronization across the distributed cloud. + +- **Subclouds** + + The subclouds are StarlingX Kubernetes edge systems/clusters used to host + containerized applications. Any type of StarlingX Kubernetes configuration, + (including simplex, duplex, or standard with or without storage nodes), can + be used for a subcloud. The two edge clouds shown in Figure 1 are subclouds. + + Alarms raised at the subclouds are sent to the SystemController for + central reporting. + +.. figure:: ../figures/starlingx-deployment-options-distributed-cloud.png + :scale: 45% + :alt: Distributed cloud deployment configuration + + *Figure 1: Distributed cloud deployment configuration* + + +-------------------- +Network requirements +-------------------- + +Subclouds are connected to the SystemController through both the OAM and the +Management interfaces. Because each subcloud is on a separate L3 subnet, the +OAM, Management and PXE boot L2 networks are local to the subclouds. They are +not connected via L2 to the central cloud, they are only connected via L3 +routing. The settings required to connect a subcloud to the SystemController +are specified when a subcloud is defined. A gateway router is required to +complete the L3 connections, which will provide IP routing between the +subcloud Management and OAM IP subnet and the SystemController Management and +OAM IP subnet, respectively. The SystemController bootstraps the subclouds via +the OAM network, and manages them via the management network. For more +information, see the `Install a Subcloud`_ section later in this guide. + +.. note:: + + All messaging between SystemControllers and Subclouds uses the ``admin`` + REST API service endpoints which, in this distributed cloud environment, + are all configured for secure HTTPS. Certificates for these HTTPS + connections are managed internally by StarlingX. + +--------------------------------------- +Install and provision the central cloud +--------------------------------------- + +Installing the central cloud is similar to installing a standard +StarlingX Kubernetes system. The central cloud supports either an AIO-duplex +deployment configuration or a standard with dedicated storage nodes deployment +configuration. + +To configure controller-0 as a distributed cloud central controller, you must +set certain system parameters during the initial bootstrapping of +controller-0. Set the system parameter *distributed_cloud_role* to +*systemcontroller* in the Ansible bootstrap override file. Also, set the +management network IP address range to exclude IP addresses reserved for +gateway routers providing routing to the subclouds' management subnets. + +Procedure: + +- Follow the StarlingX R5.0 installation procedures with the extra step noted below: + + - AIO-duplex: + `Bare metal All-in-one Duplex Installation R5.0 `_ + + - Standard with dedicated storage nodes: + `Bare metal Standard with Dedicated Storage Installation R5.0 `_ + +- For the step "Bootstrap system on controller-0", add the following + parameters to the Ansible bootstrap override file. + + .. code:: yaml + + distributed_cloud_role: systemcontroller + management_start_address: + management_end_address: + +------------------ +Install a subcloud +------------------ + +At the subcloud location: + +1. Physically install and cable all subcloud servers. +2. Physically install the top of rack switch and configure it for the + required networks. +3. Physically install the gateway routers which will provide IP routing + between the subcloud OAM and Management subnets and the SystemController + OAM and management subnets. +4. On the server designated for controller-0, install the StarlingX + Kubernetes software from USB or a PXE Boot server. + +5. Establish an L3 connection to the SystemController by enabling the OAM + interface (with OAM IP/subnet) on the subcloud controller using the + ``config_management`` script. This step is for subcloud ansible bootstrap + preparation. + + .. note:: This step should **not** use an interface that uses the MGMT + IP/subnet because the MGMT IP subnet will get moved to the loopback + address by the Ansible bootstrap playbook during installation. + + Be prepared to provide the following information: + + - Subcloud OAM interface name (for example, enp0s3). + - Subcloud OAM interface address, in CIDR format (for example, 10.10.10.12/24). + + .. note:: This must match the *external_oam_floating_address* supplied in + the subcloud's ansible bootstrap override file. + + - Subcloud gateway address on the OAM network + (for example, 10.10.10.1). A default value is shown. + - System Controller OAM subnet (for example, 10,10.10.0/24). + + .. note:: To exit without completing the script, use ``CTRL+C``. Allow a few minutes for + the script to finish. + + .. note:: The `config_management` in the code snippet configures the OAM + interface/address/gateway. + + .. code:: sh + + $ sudo config_management + Enabling interfaces... DONE + Waiting 120 seconds for LLDP neighbor discovery... Retrieving neighbor details... DONE + Available interfaces: + local interface remote port + --------------- ---------- + enp0s3 08:00:27:c4:6c:7a + enp0s8 08:00:27:86:7a:13 + enp0s9 unknown + + Enter management interface name: enp0s3 + Enter management address CIDR: 10.10.10.12/24 + Enter management gateway address [10.10.10.1]: + Enter System Controller subnet: 10.10.10.0/24 + Disabling non-management interfaces... DONE + Configuring management interface... DONE + RTNETLINK answers: File exists + Adding route to System Controller... DONE + +At the System Controller: + +1. Create a ``bootstrap-values.yml`` override file for the subcloud. For + example: + + .. code:: yaml + + system_mode: duplex + name: "subcloud1" + description: "Ottawa Site" + location: "YOW" + + management_subnet: 192.168.101.0/24 + management_start_address: 192.168.101.2 + management_end_address: 192.168.101.50 + management_gateway_address: 192.168.101.1 + + external_oam_subnet: 10.10.10.0/24 + external_oam_gateway_address: 10.10.10.1 + external_oam_floating_address: 10.10.10.12 + + systemcontroller_gateway_address: 192.168.204.101 + + .. important:: The `management_*` entries in the override file are required + for all installation types, including AIO-Simplex. + + .. important:: The `management_subnet` must not overlap with any other subclouds. + + .. note:: The `systemcontroller_gateway_address` is the address of central + cloud management network gateway. + +2. Add the subcloud using the CLI command below: + + .. code:: sh + + dcmanager subcloud add --bootstrap-address + --bootstrap-values + + Where: + + - ** is the OAM interface address set earlier on the subcloud. + - ** is the Ansible override configuration file, ``bootstrap-values.yml``, + created earlier in step 1. + + You will be prompted for the Linux password of the subcloud. This command + will take 5- 10 minutes to complete. You can monitor the progress of the + subcloud bootstrap through logs: + + .. code:: sh + + tail –f /var/log/dcmanager/_bootstrap_