starlingx/docs
Code Issues Proposed changes

Deployment Configuration

Browse Source 
Address review comments from previous patch. Also implemented abbreviations as substitutions
in docs/doc/source/shared/strings.txt. NB: This doc still to be reconciled against existing
install/deploy content.

Restored fault/index version of FM toctree in top-level index.

Removed Fault Management from top-level index.

Initial review submission.

Signed-off-by: Stone <ronald.stone@windriver.com>
Change-Id: I76c2e99bd96bb1d58d6daf67ac46e6e48e4396d7
Signed-off-by: Stone <ronald.stone@windriver.com>
This commit is contained in:
Stone
2020-11-27 14:26:36 -05:00
parent f63f0912c6
commit b250326272
12 changed files with 577 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

159
doc/source/deploy/common-components.rst Normal file
View File

@@ -0,0 +1,159 @@
.. hck1565197982784
.. _common-components:
=================
Common Components
=================
A number of components are common to most |prod| deployment configurations.
.. image:: figures/kubernetes-cluster-deployment-configuration-options.png
:width: 800
.. xreflink .. note::
Ceph is not configured by default. For more information, see the
|stor-doc|: :ref:`Configuring the Internal Ceph Storage Backend
<configuring-the-internal-ceph-storage-backend>`.
**Controller Node / Function**
Controller nodes or functions run the cloud control functions for managing
cloud resources; that is, they run all Kubernetes control functions, such
as for managing images, pods, services, etc.
For standard with controller storage deployment configurations, the
controller nodes/functions run a small-scale Ceph cluster using one or more
disks \(|SATA|, |SAS|, |SSD| and/or |NVMe|\) as the ceph |OSDs|. This
cluster provides the storage backend for Kubernetes' |PVCs|.
In most configurations, the controller nodes/functions are part of a two
node HA controller node cluster for running control functions in either
Active/Active or Active/Standby mode.
**Worker Node / Function**
Worker nodes or functions run the hosted containerized applications.
**Storage Node / Function**
For Standard with Dedicated Storage deployment configurations, the storage
nodes run a large scale Ceph cluster using disks \(|SATA|, |SAS|, |SSD| and
/or |NVMe|\) across 2-9 storage nodes as Ceph |OSDs|. This provides the
storage backend for Kubernetes' |PVCs|.
In most configurations the storage nodes/functions are part of a HA
multi-node Ceph storage cluster supporting a replication factor of 2 or 3,
journal caching and class tiering.
**All-In-One (AIO) Controller Node**
A single physical node which provides a controller function, worker
function and storage function.
**L2 Switches and L2 Networks**
A single physical switch may support multiple L2 networks.
**Operations, Administration and Management (OAM) Network \(Controller Nodes Only\)**
The network on which all external StarlingX platform APIs are exposed,
including platform REST APIs \(Keystone, StarlingX, Kubernetes\), the
Horizon Web interface, |SSH| and |SNMP|.
This is typically a 1GE network.
**Management Network \(All Nodes\)**
A private network \(i.e. not connected externally\) used for internal
StarlingX monitoring and control, and container access to storage cluster.
This is typically a 10GE network.
**Cluster Host Network \(All Nodes\)**
The cluster host network is used for Kubernetes management and control, as
well as private container networking. The |CNI| service, Calico, provides
private tunneled networking between hosted containers on the cluster host
network.
The cluster host network is internal by default and shares the same
interface as the management network. However, it can be configured on a
dedicated interface/network during initial configuration if required.
The cluster host network can be used as the network for external
connectivity or it can be deployed as an internal network.
**External Network Connectivity with EXTERNAL cluster host network**
The |CNI| service, Calico,
provides private tunneled networking between hosted containers on the
external cluster host network.
Containers' network endpoints can be exposed externally with 'NodePort'
Kubernetes services, exposing selected application containers' network
ports on *all* interfaces \(e.g. external cluster host interfaces\) of
both controller nodes and *all* worker nodes. This would typically be
done either directly to the application containers service or through
an ingress controller service. HA would be achieved through either an
external HA load balancer across two or more worker nodes or simply
using multiple records \(two or more destination worker node IPs\) for
the application's external DNS Entry.
Containers' network endpoints can also be exposed through |BGP| within
the Calico |CNI| service. The Calico |BGP| configuration can be
modified to advertise selected application container services or the
ingress controller service to a |BGP| peer, specifying the available
next hop worker nodes' cluster host IP addresses.
**External Network Connectivity with INTERNAL cluster host network**
If the cluster host network is INTERNAL, then either the OAM port or
additionally configured ports on both controller and worker nodes will
be used for connectivity to external networks.
As with the INTERNAL cluster host network, containers' network
endpoints can be exposed externally with NodePort Kubernetes services,
exposing selected application containers' network ports on *all*
interfaces of both controller nodes and *all* worker nodes. In this
scenario they are exposed on either the OAM interface or the
additionally configured interfaces for external connectivity on all
nodes. This is typically done either directly to the application
containers service or through an ingress controller service. HA can be
achieved through either an external HA load balancer across two or more
worker nodes or simply using multiple records \(two or more destination
worker node IP addresses\) for the application's external DNS Entry.
The use of Container Networking Calico |BGP| to advertise containers'
network endpoints is not available in this scenario.
**Additional External Network\(s\) \(Worker & AIO Nodes Only\)**
Networks on which ingress controllers and/or hosted application containers
expose their Kubernetes service, for example, through a NodePort service.
Node interfaces to these networks are configured as platform class
interfaces on nodes.
This can also refer to data networks attached to node interfaces configured
as 'pci-sriov' class interfaces; i.e. as part of the capability to support
hosted application containers to have interfaces directly connected to the
host's interface via pci-passthru or |SRIOV|.
**IPMI Network \(All Nodes\)**
An optional network on which |IPMI| interfaces of all nodes are connected.
The |IPMI| network must be L3/IP reachable from the controller's |OAM|
interfaces.
**PxeBoot Network \(All Nodes\)**
An *optional* network over which nodes net boot from controllers.
By default, controllers network boot other nodes over the management
network. This network is required for a variety of special case situations
where the management network cannot be used to boot the other nodes:
- The management network must be IPv6. IPv4 pxeboot network must be
configured since IPv6 does not support pxeboot.
- The management network must be vlan tagged. Most servers' BIOS do not
support pxebooting over a tagged network, so an untagged pxeboot
network must be configured.
**Node Interfaces**
In general, node network interfaces can optionally be:
- Untagged single port
- Untagged two port |LAG|, optionally split
between redundant L2 switches.
- VLAN on either single port or two port |LAG|

46
doc/source/deploy/deployment-and-configuration-options-standard-configuration-with-controller-storage.rst Normal file
View File

@@ -0,0 +1,46 @@
.. rde1565203741901
.. _deployment-and-configuration-options-standard-configuration-with-controller-storage:
==============================================
Standard Configuration with Controller Storage
==============================================
|prod| supports a small-scale deployment option using a small Ceph cluster as a
back-end for Kubernetes |PVCs| deployed on the
controller nodes instead of using dedicated storage nodes.
.. image:: ../deploy_install_guides/r5_release/figures/starlingx-deployment-options-controller-storage.png
:width: 800
.. note::
Physical L2 switches are not shown in the deployment diagram in subsequent
chapters. Only the L2 networks they support are shown.
See :ref:`Common Components <common-components>` for a description of common
components of this deployment configuration.
This deployment configuration consists of a two node HA controller+storage
cluster managing up to 200 worker nodes. The limit on the size of the worker
node pool is due to the performance and latency characteristics of the small
integrated Ceph cluster on the controller+storage nodes.
This configuration uses dedicated physical disks configured on each
controller+storage node as Ceph |OSDs|. The
primary disk is used by the platform for system purposes and subsequent disks
are used for Ceph |OSDs|.
On worker nodes, the primary disk is used for system requirements and for
container local ephemeral storage.
HA controller services run across the two controller+storage nodes in either
Active/Active or Active/Standby mode. The two node Ceph cluster on the
controller+storage nodes provides HA storage through |OSD| replication between
the two nodes.
In the event of an overall controller+storage node failure, all controller HA
services become active on the remaining healthy controller+storage node, and
the above mentioned nodal Ceph replication protects the Kubernetes |PVCs|.
On overall worker node failure, hosted application containers on the failed
worker node are recovered on the remaining healthy worker nodes.

105
doc/source/deploy/deployment-config-options-all-in-one-duplex-configuration.rst Normal file
View File

@@ -0,0 +1,105 @@
.. nex1565202435470
.. _deployment-config-options-all-in-one-duplex-configuration:
=====================================
All-in-one (AIO) Duplex Configuration
=====================================
|prod| AIO Duplex provides a scaled-down |prod| deployment
option that combines controller, storage, and worker functionality on a
redundant pair of hosts.
.. contents::
:local:
:depth: 1
.. image:: ../deploy_install_guides/r5_release/figures/starlingx-deployment-options-duplex.png
:width: 800
See :ref:`Common Components <common-components>` for a description of common
components of this deployment configuration.
This deployment configuration provides a solution for situations in which
protection against overall server hardware faults and only a small amount of
cloud processing / storage power is required.
HA services run on the controller function across the two physical servers in
either Active/Active or Active/Standby mode.
The storage function is provided by a small-scale two node Ceph cluster using
one or more disks/|OSDs| from each server, and
provides the backend for Kubernetes' |PVCs|.
The solution requires two or more disks per server; one for system
requirements and container ephemeral storage, and one or more for Ceph
|OSDs|.
Hosted application containers are scheduled on both worker functions.
In the event of an overall server hardware fault:
.. _deployment-config-options-all-in-one-duplex-configuration-ul-jr3-tcy-q3b:
- all controller HA services enter active mode on the remaining healthy
server, and
- all hosted application containers impacted by the failed server are
recovered on the remaining healthy server.
On an All-in-one Duplex system, two modes of connection are supported for the
management and cluster host network.
**Direct**
This mode uses direct peer-to-peer connections between the two nodes for
the management and an internal cluster host network connection,
eliminating the need for a |ToR| switch port for these
networks.
|org| recommends a 10GE shared management and cluster host network with
|LAG| for direct connections. If the management
network must be 1GE \(to support PXE booting\), then a separate 10GE
cluster host network with |LAG| is also
recommended. The use of |LAG| addresses failover
considerations unique to peer-to-peer connections.
**Switch-based**
This mode uses a |ToR| switch to complete the
management and cluster host network connections.
.. _deployment-config-options-all-in-one-duplex-configuration-section-N10099-N1001C-N10001:
----------------------
Intel Xeon D Processor
----------------------
In addition to regular all-in-one deployments, |prod| Simplex and Duplex
provide support for small scale deployments on the Intel Xeon D family of
processors using a smaller memory and CPU footprint than the standard Simplex
configuration.
For low-cost or low-power applications with minimal performance demands \(40
containers or fewer\), |prod| Simplex can be deployed on a server with a
single Intel Xeon D class processor. The platform-reserved memory and the
maximum number of worker threads are reduced by default, but can be
reconfigured if required.
.. _deployment-config-options-all-in-one-duplex-configuration-section-N100AA-N1001C-N10001:
---------------------------------------------------
Extending the capacity of the AIO-Duplex Deployment
---------------------------------------------------
Up to fifty worker/compute nodes can be added to the All-in-one Duplex
deployment, allowing a capacity growth path if starting with an AIO-Duplex
deployment.
.. image:: ../deploy_install_guides/r5_release/figures/starlingx-deployment-options-duplex-extended.png
:width: 800
The extended capacity is limited up to fifty worker/compute nodes as the
controller/worker function on the AIO controllers has only a portion of the
processing power of the overall server.
Hosted application containers can be scheduled on either of the AIO controller
nodes and/or the worker nodes.

66
doc/source/deploy/deployment-config-optionsall-in-one-simplex-configuration.rst Normal file
View File

@@ -0,0 +1,66 @@
.. egy1565201808746
.. _deployment-config-optionsall-in-one-simplex-configuration:
======================================
All-in-one (AIO) Simplex Configuration
======================================
The AIO Simplex deployment configuration provides a scaled-down |prod| that
combines controller, storage, and worker functionality on a single
non-redundant host.
.. image:: ../deploy_install_guides/r5_release/figures/starlingx-deployment-options-simplex.png
:width: 800
.. note::
Physical L2 switches are not shown in the deployment diagram in subsequent
chapters. Only the L2 networks they support are shown.
See :ref:`Common Components <common-components>` for a description of common
components of this deployment configuration.
This deployment configuration provides no protection against an overall server
hardware fault. However, there may be hardware component protection if, for
example, HW RAID or 2x Port |LAG| is used in the
deployment.
Typically, this solution is used where only a small amount of cloud processing
/ storage power is required, and protection against overall server hardware
faults is either not required or done at a higher level.
Ceph is deployed in this configuration using one or more disks for |OSDs|, and
provides the backend for Kubernetes' |PVCs|.
The solution requires two or more disks, one for system requirements and
container ephemeral storage, and one or more for Ceph |OSDs|.
.. xreflink .. note::
A storage backend is not configured by default. You can use either
internal Ceph or an external Netapp Trident backend. For more information,
see the :ref:`|stor-doc| <storage-configuration-storage-resources>` guide.
The internal management network is not required in this scenario as it is
configured on the loopback interface. The cluster host network is internal by
default and also configured on the loopback interface with application
container external connectivity being achieved through either the external OAM
network or optional additional external networks. However, the cluster host
network can be re-configured on a physical interface depending on external
connectivity requirements.
.. _deployment-config-optionsall-in-one-simplex-configuration-section-N1004C-N1001C-N10001:
----------------------
Intel Xeon D Processor
----------------------
In addition to regular all-in-one deployments, |prod| Simplex and Duplex
provide support for small scale deployments on the Intel Xeon D family of
processors using a smaller memory and CPU footprint than the standard Simplex
configuration.
For low-cost or low-power applications with minimal performance demands \(40
Containers or fewer\), |prod| Simplex can be deployed on a server with a
single Intel Xeon D class processor. The platform-reserved memory and the
maximum number of worker threads are reduced by default, but can be
reconfigured as required.

45
doc/source/deploy/deployment-options.rst Normal file
View File

@@ -0,0 +1,45 @@
.. dqo1565196952042
.. _deployment-options:
==================
Deployment Options
==================
A variety of |prod-long| deployment configuration options are supported.
**All-in-one Simplex**
A single physical server providing all three cloud functions \(controller,
worker and storage\).
**All-in-one Duplex \(up to 50 worker nodes\)**
Two HA-protected physical servers, both running all three cloud functions
\(controller, worker and storage\), optionally with up to 50 worker nodes
added to the cluster.
**Standard with Storage Cluster on Controller Nodes**
A two node HA controller + storage node cluster, managing up to 200 worker
nodes.
**Standard with Storage Cluster on dedicated Storage Nodes**
A two node HA controller node cluster with a 2-9 node Ceph storage
cluster, managing up to 200 worker nodes.
.. note::
A storage backend is not configured by default. You can use either
internal Ceph or an external Netapp Trident backend.
.. xreflink For more
information, see the :ref:`Storage
<storage-configuration-storage-resources>` guide.
All |prod| systems can use worker platforms \(worker hosts, or the worker
function on a simplex or duplex system\) configured for either standard or
low-latency performance.
.. seealso::
:ref:`Worker Function Performance Profiles
<worker-function-performance-profiles>`
The Ceph storage backend is configured by default.

BIN
doc/source/deploy/figures/kubernetes-cluster-deployment-configuration-options.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 109 KiB

15
doc/source/deploy/index.rs1 Normal file
View File

@@ -0,0 +1,15 @@
============================================
|prod-long| Deployment Configuration Options
============================================
- Deployment Options
- :ref:`Deployment Options <deployment-options>`
- :ref:`Common Components <common-components>`
- :ref:`All-in-one Simplex Configuration <deployment-config-optionsall-in-one-simplex-configuration>`
- :ref:`All-in-one Duplex Configuration <deployment-config-options-all-in-one-duplex-configuration>`
- :ref:`Standard Configuration with Controller Storage <deployment-and-configuration-options-standard-configuration-with-controller-storage>`
- :ref:`Standard Configuration with Dedicated Storage <standard-configuration-with-dedicated-storage>`
- :ref:`Worker Function Performance Profiles <worker-function-performance-profiles>`

19
doc/source/deploy/index.rst Normal file
View File

@@ -0,0 +1,19 @@
.. Deployment Configurations file, created by
sphinx-quickstart on Thu Sep 3 15:14:59 2020.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
=========================
Deployment Configurations
=========================
.. toctree::
:maxdepth: 1
deployment-options
common-components
deployment-config-optionsall-in-one-simplex-configuration
deployment-config-options-all-in-one-duplex-configuration
standard-configuration-with-dedicated-storage
deployment-and-configuration-options-standard-configuration-with-controller-storage
worker-function-performance-profiles

53
doc/source/deploy/standard-configuration-with-dedicated-storage.rst Normal file
View File

@@ -0,0 +1,53 @@
.. gzi1565204095452
.. _standard-configuration-with-dedicated-storage:
=============================================
Standard Configuration with Dedicated Storage
=============================================
Deployment of |prod| with dedicated storage nodes provides the highest capacity
\(single region\), performance, and scalability.
.. image:: ../deploy_install_guides/r5_release/figures/starlingx-deployment-options-dedicated-storage.png
:width: 800
.. note::
Physical L2 switches are not shown in the deployment diagram in subsequent
chapters. Only the L2 networks they realize are shown.
See :ref:`Common Components <common-components>` for a description of common
components of this deployment configuration.
The differentiating physical feature of this model is that the controller,
storage, and worker functionalities are deployed on separate physical hosts
allowing controller nodes, storage nodes, and worker nodes to scale
independently from each other.
The controller nodes provide the master function for the system. Two controller
nodes are required to provide redundancy. The controller nodes' server and
peripheral resources such as CPU cores/speed, memory, storage, and network
interfaces can be scaled to meet requirements.
Storage nodes provide a large scale Ceph cluster for the storage backend for
Kubernetes |PVCs|. They are deployed in replication groups of either two or
three for redundancy. For a system configured to use two storage hosts per
replication group, a maximum of eight storage hosts \(four replication groups\)
are supported. For a system with three storage hosts per replication group, up
to nine storage hosts \(three replication groups\) are supported. The system
provides redundancy and scalability through the number of Ceph |OSDs| installed
in a storage node group, with more |OSDs| providing more capacity and better
storage performance. The scalability and performance of the storage function is
affected by the |OSD| size and speed, optional |SSD| or |NVMe| Ceph journals,
CPU cores and speeds, memory, disk controllers, and networking. |OSDs| can be
grouped into storage tiers according to their performance characteristics.
.. note::
A storage backend is not configured by default. You can use either
internal Ceph or an external Netapp Trident backend.
.. xreflink For more information,
see the :ref:`|stor-doc| <storage-configuration-storage-resources>` guide.
On worker nodes, the primary disk is used for system requirements and for
container local ephemeral storage.

35
doc/source/deploy/worker-function-performance-profiles.rst Normal file
View File

@@ -0,0 +1,35 @@
.. dle1565204388710
.. _worker-function-performance-profiles:
====================================
Worker Function Performance Profiles
====================================
|prod| offers a choice of two performance profiles for worker hosts
configurable at the Kubernetes platform level.
**Standard**
This is the default profile, offering an optimal balance between throughput,
latency and functionality. The standard profile is suitable for most
applications.
**Low Latency**
This profile is optimized for low-latency performance. Characteristics of a
low-latency worker host include:
- lower throughput for most configurations
- higher power consumption at low CPU utilization
- disabled per-CPU scheduler watchdogs. This can result in increased
fault detection time for some fault scenarios.
|org| recommends the low-latency profile only for applications with
stringent latency constraints such as |MEC|.
You can create a heterogeneous configuration in which standard and low-latency
worker hosts are mixed in the same standard deployment. Standard and
low-latency worker nodes are identified with node labels. When launching a
container, you can choose the required performance profile by specifying the
appropriate label in the node-selector.

11
doc/source/index.rst
View File

@@ -39,6 +39,15 @@ Learn more about StarlingX:
introduction/index
---------------
Planning guides
---------------
.. toctree::
:maxdepth: 2
deploy/index
-------------------
Installation guides
-------------------
@@ -67,7 +76,7 @@ Data Network Configuration and Management Guides
datanet/index
----------------
Fault management
Fault Management
----------------
.. toctree::

24
doc/source/shared/strings.txt
View File

@@ -44,3 +44,27 @@
.. File name prefix, as in stx-remote-cli-<version>.tgz. May also be
.. used in sample domain names etc.
.. |prefix| replace:: stx
.. Common and domain-specific sbbreviations.
.. Plural forms must be defined seperately from singular as
.. replacements like |PVC|s won't work.
.. |BGP| replace:: :abbr:`BGP (Border Gateway Protocol)`
.. |CNI| replace:: :abbr:`CNI (Container Networking Interface)`
.. |IPMI| replace:: :abbr:`IPMI (Intelligent Platform Management Interface)`
.. |LAG| replace:: :abbr:`LAG (Link Aggregation)`
.. |MEC| replace:: :abbr:`MEC (Multi-access Edge Computing)`
.. |NVMe| replace:: :abbr:`NVMe (Non Volatile Memory express)`
.. |OAM| replace:: :abbr:`OAM (Operations, administration and management)`
.. |OSD| replace:: :abbr:`OSD (Object Storage Device)`
.. |OSDs| replace:: :abbr:`OSDs (Object Storage Devices)`
.. |PVC| replace:: :abbr:`PVC (Persistent Volume Claim)`
.. |PVCs| replace:: :abbr:`PVCs (Persistent Volume Claims)`
.. |SAS| replace:: :abbr:`SAS (Serial Attached SCSI)`
.. |SATA| replace:: :abbr:`SATA (Serial AT Attachment)`
.. |SNMP| replace:: :abbr:`SNMP (Simple Network Management Protocol)`
.. |SRIOV| replace:: :abbr:`SR-IOV (Single Root I/O Virtualization)`
.. |SSD| replace:: :abbr:`SSD (Solid State Drive)`
.. |SSH| replace:: :abbr:`SSH (Secure Shell)`
.. |ToR| replace:: :abbr:`ToR (Top-of-Rack)`