==================================================== Installation guide for Tricircle work with Container ==================================================== Introduction ^^^^^^^^^^^^ In the `Multi-pod Installation with DevStack `_ , we have discussed how to deploy Tricircle in multi-region scenario with DevStack. However, the previous installation guides have been on how to manage virtual machines using tricircle and Nova in cross-region openstack cloud environments. So, multi-region container management is not supported in Tricircle. Meanwhile, OpenStack uses Zun component to provide container management service, OpenStack also use kuyr component and kuryr-libnetwork component to provide container network. In view of the Tricircle Central_Neutron-Local_Neutron fashion, Tricircle work with zun and kuryr will provide a cross-region container management solution. This guide is to describe how tricircle work with container management and how to deploy a multi-region container environment. Prerequisite ^^^^^^^^^^^^ In this guide, we need specific versions of the zun project and kuryr project source code. The source code versions of both projects must be the Train version and upper. If not, we need to manually change the source code for both projects. The modification example is as follows: - 1 Zun Source Code Modification: For Zun project, we need modify the **neutron** function in /zun/zun/common/clients.py file. (The '+' sign represents the added line) .. code-block:: console def neutron(self): if self._neutron: return self._neutron session = self.keystone().session session.verify = self._get_client_option('neutron', 'ca_file') or True if self._get_client_option('neutron', 'insecure'): session.verify = False endpoint_type = self._get_client_option('neutron', 'endpoint_type') + region_name = self._get_client_option('neutron', 'region_name') self._neutron = neutronclient.Client(session=session, endpoint_type=endpoint_type, + region_name=region_name) return self._neutron - 2 Kuryr Source Code Modification: For kuryr project, we need modify the **get_neutron_client** function in /kuryr/kuryr/lib/utils.py file. (The '+' sign represents the added line) .. code-block:: console def get_neutron_client(*args, **kwargs): conf_group = kuryr_config.neutron_group.name auth_plugin = get_auth_plugin(conf_group) session = get_keystone_session(conf_group, auth_plugin) endpoint_type = getattr(getattr(cfg.CONF, conf_group), 'endpoint_type') + region_name = getattr(getattr(cfg.CONF, conf_group), 'region_name') return client.Client(session=session, auth=auth_plugin, endpoint_type=endpoint_type, + region_name=region_name) Setup ^^^^^ In this guide we take two nodes deployment as an example, the node1 run as RegionOne and Central Region, the node2 run as RegionTwo. - 1 For the node1 in RegionOne and the node2 in RegionTwo, clone the code from Zun repository and Kuryr repository to /opt/stack/ . If the code does not meet the requirements described in the Prerequisite Section, modify it with reference to the modification example of the Prerequisite Section. - 2 Follow "Multi-pod Installation with DevStack" document `Multi-pod Installation with DevStack `_ to prepare your local.conf for the node1 in RegionOne and the node12 in RegionTwo, and add the following lines before installation. Start DevStack in node1 and node2. .. code-block:: console enable_plugin zun https://git.openstack.org/openstack/zun enable_plugin zun-tempest-plugin https://git.openstack.org/openstack/zun-tempest-plugin enable_plugin devstack-plugin-container https://git.openstack.org/openstack/devstack-plugin-container enable_plugin kuryr-libnetwork https://git.openstack.org/openstack/kuryr-libnetwork KURYR_CAPABILITY_SCOPE=local KURYR_PROCESS_EXTERNAL_CONNECTIVITY=False - 3 After DevStack successfully started and finished, we need make some configuration changes to Zun component and Kuryr component in node1 and node2. - For Zun in node1, modify the /etc/zun/zun.conf .. csv-table:: :header: "Group", "Option", "Value" [neutron_client], region_name, RegionOne - Restart all the services of Zun in node1. .. code-block:: console $ sudo systemctl restart devstack@zun* - For Kuryr in node1, modify the /etc/kuryr/kuryr.conf .. csv-table:: :header: "Group", "Option", "Value" [neutron], region_name, RegionOne - Restart all the services of Kuryr in node1. .. code-block:: console $ sudo systemctl restart devstack@kur* - For Zun in node2, modify the /etc/zun/zun.conf .. csv-table:: :header: "Group", "Option", "Value" [neutron_client], region_name, RegionTwo - Restart all the services of Zun in node2. .. code-block:: console $ sudo systemctl restart devstack@zun* - For Kuryr in node2, modify the /etc/kuryr/kuryr.conf .. csv-table:: :header: "Group", "Option", "Value" [neutron], region_name, RegionTwo - Restart all the services of Zun in node2. .. code-block:: console $ sudo systemctl restart devstack@kur* - 4 Then, we must create environment variables for the admin user and use the admin project. .. code-block:: console $ source openrc admin admin $ unset OS_REGION_NAME - 5 Finally, use tricircle client to create pods for multi-region. .. code-block:: console $ openstack --os-region-name CentralRegion multiregion networking pod create --region-name CentralRegion $ openstack --os-region-name CentralRegion multiregion networking pod create --region-name RegionOne --availability-zone az1 $ openstack --os-region-name CentralRegion multiregion networking pod create --region-name RegionTwo --availability-zone az2 How to play ^^^^^^^^^^^ - 1 Create container glance image in RegionOne and RegionTwo. - Get docker image from Docker Hub. Run these command in the node1 and the node2. .. code-block:: console $ docker pull cirros $ docker save cirros -o /opt/stack/container_cirros - Use glance client to create container image. .. code-block:: console $ glance --os-region-name=RegionOne image-create --file /opt/stack/container_cirros --container-format=docker --disk-format=raw --name container_cirros --progress $ glance --os-region-name=RegionTwo image-create --file /opt/stack/container_cirros --container-format=docker --disk-format=raw --name container_cirros --progress $ openstack --os-region-name RegionOne image list +--------------------------------------+--------------------------+--------+ | ID | Name | Status | +--------------------------------------+--------------------------+--------+ | 11186baf-4381-4e52-956c-22878b0642df | cirros-0.4.0-x86_64-disk | active | | 87864205-4352-4a2c-b9b1-ca95df52c93c | container_cirros | active | +--------------------------------------+--------------------------+--------+ $ openstack --os-region-name RegionTwo image list +--------------------------------------+--------------------------+--------+ | ID | Name | Status | +--------------------------------------+--------------------------+--------+ | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | container_cirros | active | | cf4a2dc7-6d6e-4b7e-a772-44247246e1ff | cirros-0.4.0-x86_64-disk | active | +--------------------------------------+--------------------------+--------+ - 2 Create container network in CentralRegion. - Create a net in CentralRegion. .. code-block:: console $ openstack --os-region-name CentralRegion network create container-net +---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | None | | created_at | None | | description | None | | dns_domain | None | | id | 5e73dda5-902b-4322-b5b6-4121437fde26 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | location | cloud='', project.domain_id='default', project.domain_name=, project.id='2f314a39de10467bb62745bd96c5fe4d', project.name='admin', region_name='CentralRegion', zone= | | mtu | None | | name | container-net | | port_security_enabled | False | | project_id | 2f314a39de10467bb62745bd96c5fe4d | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 1070 | | qos_policy_id | None | | revision_number | None | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | | | tags | | | updated_at | None | +---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - Create a subnet in container-net .. code-block:: console $ openstack --os-region-name CentralRegion subnet create --subnet-range 10.0.60.0/24 --network container-net container-subnet +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | allocation_pools | 10.0.60.2-10.0.60.254 | | cidr | 10.0.60.0/24 | | created_at | 2019-12-10T07:13:21Z | | description | | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 10.0.60.1 | | host_routes | | | id | b7a7adbd-afd3-4449-9cbc-fbce16c7a2e7 | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | location | cloud='', project.domain_id='default', project.domain_name=, project.id='2f314a39de10467bb62745bd96c5fe4d', project.name='admin', region_name='CentralRegion', zone= | | name | container-subnet | | network_id | 5e73dda5-902b-4322-b5b6-4121437fde26 | | prefix_length | None | | project_id | 2f314a39de10467bb62745bd96c5fe4d | | revision_number | 0 | | segment_id | None | | service_types | None | | subnetpool_id | None | | tags | | | updated_at | 2019-12-10T07:13:21Z | +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - 3 Create container in RegionOne and RegionTwo. .. note:: We can give container a specific command to run it continually, e.g. "sudo nc -l -p 5000" . .. code-block:: console $ openstack --os-region-name RegionOne appcontainer run --name container01 --net network=$container_net_id --image-driver glance $RegionTwo_container_cirros_id sudo nc -l -p 5000 +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tty | False | | addresses | None | | links | [{u'href': u'http://192.168.1.81/v1/containers/ca67055c-635d-4603-9b0b-19c16eed7ef9', u'rel': u'self'}, {u'href': u'http://192.168.1.81/containers/ca67055c-635d-4603-9b0b-19c16eed7ef9', u'rel': u'bookmark'}] | | image | 87864205-4352-4a2c-b9b1-ca95df52c93c | | labels | {} | | disk | 0 | | security_groups | None | | image_pull_policy | None | | user_id | 57df611fd8c7415dad6d2530bf962ecd | | uuid | ca67055c-635d-4603-9b0b-19c16eed7ef9 | | hostname | None | | auto_heal | False | | environment | {} | | memory | 0 | | project_id | 2f314a39de10467bb62745bd96c5fe4d | | privileged | False | | status | Creating | | workdir | None | | healthcheck | None | | auto_remove | False | | status_detail | None | | cpu_policy | shared | | host | None | | image_driver | glance | | task_state | None | | status_reason | None | | name | container01 | | restart_policy | None | | ports | None | | command | [u'sudo', u'nc', u'-l', u'-p', u'5000'] | | runtime | None | | registry_id | None | | cpu | 0.0 | | interactive | False | +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ $ openstack --os-region-name RegionOne appcontainer list +--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+ | uuid | name | image | status | task_state | addresses | ports | +--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+ | ca67055c-635d-4603-9b0b-19c16eed7ef9 | container01 | 87864205-4352-4a2c-b9b1-ca95df52c93c | Running | None | 10.0.60.62 | [] | +--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+ $ openstack --os-region-name RegionTwo appcontainer run --name container02 --net network=$container_net_id --image-driver glance $RegionTwo_container_cirros_id sudo nc -l -p 5000 +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | tty | False | | addresses | None | | links | [{u'href': u'http://192.168.1.82/v1/containers/c359e48c-7637-4d9f-8219-95a4577683c3', u'rel': u'self'}, {u'href': u'http://192.168.1.82/containers/c359e48c-7637-4d9f-8219-95a4577683c3', u'rel': u'bookmark'}] | | image | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | | labels | {} | | disk | 0 | | security_groups | None | | image_pull_policy | None | | user_id | 57df611fd8c7415dad6d2530bf962ecd | | uuid | c359e48c-7637-4d9f-8219-95a4577683c3 | | hostname | None | | auto_heal | False | | environment | {} | | memory | 0 | | project_id | 2f314a39de10467bb62745bd96c5fe4d | | privileged | False | | status | Creating | | workdir | None | | healthcheck | None | | auto_remove | False | | status_detail | None | | cpu_policy | shared | | host | None | | image_driver | glance | | task_state | None | | status_reason | None | | name | container02 | | restart_policy | None | | ports | None | | command | [u'sudo', u'nc', u'-l', u'-p', u'5000'] | | runtime | None | | registry_id | None | | cpu | 0.0 | | interactive | False | +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ $ openstack --os-region-name RegionTwo appcontainer list +--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+ | uuid | name | image | status | task_state | addresses | ports | +--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+ | c359e48c-7637-4d9f-8219-95a4577683c3 | container02 | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | Running | None | 10.0.60.134 | [] | +--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+ - 4 Execute container in RegionOne and RegionTwo. .. code-block:: console $ openstack --os-region-name RegionOne appcontainer exec --interactive container01 /bin/sh $ openstack --os-region-name RegionTwo appcontainer exec --interactive container02 /bin/sh - 5 By now, we successfully created multi-region container scenario. So we can do something on cross-region container, e.g. 1) RegionOne container ping RegionTwo container 2) Cross-Region Container Load Balancing.