docs/doc/source/system_configuration/kubernetes/dual-stack-support-318550fd91b5.rst

.. WARNING: Add no lines of text between the label immediately following
.. and the title.

.. _dual-stack-support-318550fd91b5:

============================
IPv4/IPv6 Dual-Stack Network
============================

Dual-stack networking facilitates the simultaneous use of both IPv4 and IPv6
addresses, or continue to use each IP version independently. To accomplish
this, platform networks can be associated with 1 or 2 address pools, one for
each IP version (IPv4 or IPv6). The first pool is linked to the network
upon creation and cannot be subsequently removed. The second pool can be added or
removed to transition the system between dual-stack and single-stack modes.

The |PXE| boot network is an exception, as it currently only supports IPv4. Other
platform networks can be configured as either single-stack or dual-stack based
on specific requirements. For internal management communication among
controllers, workers, and storage nodes, the primary address pool is used as
encryption is currently only available for the primary pool on the management
network.

Once created, a network's primary address pool family cannot be modified.
Reinstalling the system is necessary to change it. While it is possible to
edit address pool addresses for management, |OAM|, and admin networks, all
addresses within a pool must belong to the same address family.

API and Command Line Interface Considerations
=============================================

The following system APIs handle the association between network and address pool:

* network-addrpool-assign: creates the association between a network and address pool
* network-addrpool-remove: removes the association between a network and address pool
* network-addrpool-list: lists all associations
* network-addrpool-show: shows a specific association

The first association is done internally when the network is created using the
command format:

.. code-block::

    ~(keystone_admin)]$ system network-add <network_name> <network type> <dynamic> <pool_uuid>

The ``addrpool-modify`` command allows to edit all its parameters with the CLI.
Follows the command format:

.. code-block::

    ~(keystone_admin)]$ system addrpool-modify [--name <name>] [--network <network address>] [--prefix <network prefix>] [--ranges <ranges>] [--order <sequential | random>] [--floating-address <floating address>] [--controller0-address <controller0 address>] [--controller1-address <controller1 address>] [--gateway-address <gateway address>] <address_pool uuid>


Install a System in Dual-Stack
==============================

It is possible to install a system in dual-stack by adding the secondary
subnets into the bootstrap variables with comma separated values as shown in
the following example:

.. code-block::

    pxeboot_subnet: 198.51.100.0/24

    management_subnet: fd01::/64,198.51.0.0/24
    management_start_address: fd01::2,198.51.0.2
    management_end_address: fd01::ffff,198.51.0.200
    management_gateway_address: fd01::1,198.51.0.1

    external_oam_subnet: fd00::/64,10.20.5.0/24
    external_oam_gateway_address: fd00::1,10.20.5.1
    external_oam_floating_address: fd00::3,10.20.5.3
    external_oam_node_0_address: fd00::4,10.20.5.4
    external_oam_node_1_address: fd00::5,10.20.5.5

    cluster_host_subnet: aefd:100::/64,198.51.100.0/24
    cluster_pod_subnet: aefd:206::/64,203.0.113.0/24
    cluster_service_subnet: aefd:207::/112,10.96.0.0/12


The order in which networks are listed determines the primary and secondary
address pools. It is important to note that all primary address pools must use
the same address family. For example, if the primary address pool for one
subnet is IPv6, all other subnet’s primary pools must also be IPv6 (as in the
example above).

Configure DNS Server
====================

It is optional to configure |DNS| servers with both IPv4 and IPv6 addresses.
This can be achieved using the command format:

.. code-block::

    ~(keystone_admin)]$ system dns-modify nameservers=<IPv6 DNS server>,<IPv4 DNS server>
    or
    ~(keystone_admin)]$ system dns-modify nameservers=<IPv4 DNS server>,<IPv6 DNS server>

If the selected |DNS| servers support both  ``A`` and ``AAAA`` records, specifying an
address for each address family is unnecessary.

Distributed Cloud Operations
============================

Subclouds can be installed in a dual-stack configuration, if its version
supports the feature. All operational communication between the system
controller and subclouds uses the primary address pool. While the system
controller and subclouds can operate in different network modes (single-stack
or dual-stack), they must share the same primary address family. Geo redundancy
uses the primary address pools to communicate.

Public Endpoint Considerations
==============================

All available public |OAM| endpoints can be accessed through the secondary
address using the same L4 port. |HA| Proxy is used to map these external
requests to the corresponding internal endpoints.

Modify Network Addresses
========================

**Supported Networks**
    Only the |OAM|, Admin, and Management networks can be modified using the ``addrpool-modify`` command during runtime.

**Other Networks**
    For other networks, reinstallation is required to make changes.

External API and Command Line Interface Considerations
======================================================

**Deprecated**
    The ``external-OAM`` API is marked as deprecated but can be used to modify the |OAM| network primary pool.

**CLI**
    The corresponding CLIs are:

    .. code-block::

        ~(keystone_admin)]$ system oam-modify <path=value> [<path=value> ...]
        ~(keystone_admin)]$ system oam-show

Enable Kubernetes in Dual-stack
===============================

To enable dual-stack functionality in Kubernetes, the |OAM|, cluster-host,
cluster-service, and cluster-pod networks must be configured to support it.
Making these changes at runtime triggers a quick restart for the ``kube-API-server``
and ``kube-controller-manager`` pods.

If converted during runtime, newly created pods will automatically receive
both primary and secondary addresses. Existing pods retain their current
primary addresses but will not acquire a secondary address until they are
restarted. The same happens from a dual-stack to single-stack configuration,
previously existing pods will retain their secondary address until restart.

Runtime Configuration
=====================

To add dual-stack in a running system the following sequence is suggested
(in the example below the system was installed as IPv6). The network's primary
family can be seen with:

.. code-block::

    ~(keystone_admin)]$ system network-list
    +----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
    | id | uuid...  | name            | type            | dynamic | pool_uuid                            | primary_pool_family |
    +----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
    | 4  | 196d...3 | multicast       | multicast       | False   | 7c445f38-067c-4b3c-a511-d8e00da5791c | IPv6                |
    | 5  | 43fe...3 | cluster-host    | cluster-host    | True    | 6250edb8-15f5-4204-80f1-8e54b9e28a5a | IPv6                |
    | 3  | 9996...9 | oam             | oam             | False   | b46512d7-5404-4daa-a64d-fc510e0c5864 | IPv6                |
    | 6  | a374...7 | cluster-pod     | cluster-pod     | False   | f4c9560c-47e5-46bd-aff5-18642831b1da | IPv6                |
    | 7  | afc1...d | cluster-service | cluster-service | False   | a6366aab-b3c1-4947-97e5-f5171e0e2f3e | IPv6                |
    | 1  | b565...9 | mgmt            | mgmt            | True    | 412aebff-9a86-40b1-a379-752f00a0c3a0 | IPv6                |
    | 2  | bbb1...2 | pxeboot         | pxeboot         | True    | 05fde56d-f26a-4ea4-8b32-1ebf868743e2 | IPv4                |
    +----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+

Configure OAM Network
=====================

Add an address pool for |OAM|:

.. code-block::

    ~(keystone_admin)]$ system addrpool-add oam-ipv4 171.168.204.0 24 --order random --ranges 171.168.204.1-171.168.204.254 --floating-address 171.168.204.1 --controller0-address 171.168.204.2 --controller1-address 171.168.204.3

Then assign the newly created pool to the |OAM| network:

.. code-block::

    ~(keystone_admin)]$ system network-addrpool-assign oam oam-ipv4

If the system is AIO-SX, the new configuration is applied immediately,
otherwise it is necessary to lock/unlock both controllers.

Configure Cluster (pod/service/host) Network
============================================

The cluster networks in dual-stack converts kubernetes to dual-stack operation,
in this case, first make sure the |OAM| network is already configured in
dual-stack and then start by adding the correspondent pools:

.. code-block::

    ~(keystone_admin)]$ system addrpool-add cluster-pod-subnet-ipv4 172.16.0.0 16 --order random --ranges 172.16.0.1-172.16.254.254
    ~(keystone_admin)]$ system addrpool-add cluster-service-subnet-ipv4 10.96.0.0 12 --order random --ranges 10.96.0.1-10.96.254.254
    ~(keystone_admin)]$ system addrpool-add cluster-host-subnet-ipv4 192.168.204.0 24 --order random --ranges 192.168.204.1-192.168.204.254 --floating-address 192.168.204.1 --controller0-address 192.168.204.2 --controller1-address 192.168.204.3

Then associate the new pools to each network (there is no preferred order
among the three networks):

.. code-block::

    ~(keystone_admin)]$ system network-addrpool-assign cluster-service cluster-service-subnet-ipv4
    ~(keystone_admin)]$ system network-addrpool-assign cluster-pod cluster-pod-subnet-ipv4
    ~(keystone_admin)]$ system network-addrpool-assign cluster-host cluster-host-subnet-ipv4

After the third cluster network receives dual-stack kubernetes and
calico will be reconfigured with the ``kube-apiserver-controller`` and
``kube-controller-manager-controller`` restarts. The entire operation will
be performed without the need of a node lock/unlock cycle.

Configure Management Network
============================

As stated, the internal communication is done through the primary pool, but it
is possible to add dual-stack configuration by first adding the new pool:

.. code-block::

    ~(keystone_admin)]$ system addrpool-add management-ipv4 20.20.20.0 24 --order random --ranges 20.20.20.1-20.20.20.254 --floating-address 20.20.20.1 --controller0-address 20.20.20.2 --controller1-address 20.20.20.3

Then create the association:

.. code-block::

    ~(keystone_admin)]$ system network-addrpool-assign management management-ipv4

A ``Configuration Out-Of-Date`` alarm is raised for the affected nodes and a
node lock/unlock cycle will clean the alarm.

Configure Admin Network
=======================

This network is used by subclouds to communicate with its system-controller
and that is done through the primary pool. To add a dual-stack configuration
start with a new pool:

.. code-block::

    ~(keystone_admin)]$ system addrpool-add admin-ipv4 30.30.30.0 24 --order random --ranges 30.30.30.1-30.30.30.254 --floating-address 30.30.30.1 --controller0-address 30.30.30.2 --controller1-address 30.30.30.3

Then create the association:

.. code-block::

    ~(keystone_admin)]$ system network-addrpool-assign admin admin-ipv4

This is done in runtime on the affected controllers, no lock/unlock cycle is
required.

Revert to Single-stack
=======================

By removing the network association with the address pool the single-stack
configuration operates in a similar fashion that was done to configure
dual-stack. If the configuration was done at runtime, or if a node lock/unlock
cycle was required to configure dual-stack, the same happens when configuring
single-stack.

To remove a address pool association with a network use ``network-addrpool-remove``,
for example:

.. code-block::

    ~(keystone_admin)]$ ADDR_POOL_NAME=”cluster-pod-ipv6"
    ~(keystone_admin)]$ DEL=$(system network-addrpool-list | awk '$6 == $ADDR_POOL_NAME { print $2 }') && system network-addrpool-remove $DEL