RETIRED, Tricircle is to provide networking automation across Neutron.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

41 KiB

Installation guide for Tricircle work with Container

Introduction

In the Multi-pod Installation with DevStack , we have discussed how to deploy Tricircle in multi-region scenario with DevStack. However, the previous installation guides have been on how to manage virtual machines using tricircle and Nova in cross-region openstack cloud environments. So, multi-region container management is not supported in Tricircle. Meanwhile, OpenStack uses Zun component to provide container management service, OpenStack also use kuyr component and kuryr-libnetwork component to provide container network. In view of the Tricircle Central_Neutron-Local_Neutron fashion, Tricircle work with zun and kuryr will provide a cross-region container management solution. This guide is to describe how tricircle work with container management and how to deploy a multi-region container environment.

Prerequisite

In this guide, we need specific versions of the zun project and kuryr project source code. The source code versions of both projects must be the Train version and upper. If not, we need to manually change the source code for both projects. The modification example is as follows:

  • 1 Zun Source Code Modification:

    For Zun project, we need modify the neutron function in /zun/zun/common/clients.py file. (The '+' sign represents the added line)

    def neutron(self):
        if self._neutron:
            return self._neutron
    
        session = self.keystone().session
        session.verify = self._get_client_option('neutron', 'ca_file') or True
        if self._get_client_option('neutron', 'insecure'):
            session.verify = False
        endpoint_type = self._get_client_option('neutron', 'endpoint_type')
    +   region_name = self._get_client_option('neutron', 'region_name')
        self._neutron = neutronclient.Client(session=session,
                                             endpoint_type=endpoint_type,
    +                                        region_name=region_name)
    
        return self._neutron
  • 2 Kuryr Source Code Modification:

    For kuryr project, we need modify the get_neutron_client function in /kuryr/kuryr/lib/utils.py file. (The '+' sign represents the added line)

    def get_neutron_client(*args, **kwargs):
        conf_group = kuryr_config.neutron_group.name
        auth_plugin = get_auth_plugin(conf_group)
        session = get_keystone_session(conf_group, auth_plugin)
        endpoint_type = getattr(getattr(cfg.CONF, conf_group), 'endpoint_type')
    +   region_name = getattr(getattr(cfg.CONF, conf_group), 'region_name')
    
        return client.Client(session=session,
                             auth=auth_plugin,
                             endpoint_type=endpoint_type,
    +                        region_name=region_name)

Setup

In this guide we take two nodes deployment as an example, the node1 run as RegionOne and Central Region, the node2 run as RegionTwo.

  • 1 For the node1 in RegionOne and the node2 in RegionTwo, clone the code from Zun repository and Kuryr repository to /opt/stack/ . If the code does not meet the requirements described in the Prerequisite Section, modify it with reference to the modification example of the Prerequisite Section.

  • 2 Follow "Multi-pod Installation with DevStack" document Multi-pod Installation with DevStack to prepare your local.conf for the node1 in RegionOne and the node12 in RegionTwo, and add the following lines before installation. Start DevStack in node1 and node2.

    enable_plugin zun https://git.openstack.org/openstack/zun
    enable_plugin zun-tempest-plugin https://git.openstack.org/openstack/zun-tempest-plugin
    enable_plugin devstack-plugin-container https://git.openstack.org/openstack/devstack-plugin-container
    enable_plugin kuryr-libnetwork https://git.openstack.org/openstack/kuryr-libnetwork
    
    KURYR_CAPABILITY_SCOPE=local
    KURYR_PROCESS_EXTERNAL_CONNECTIVITY=False
  • 3 After DevStack successfully started and finished, we need make some configuration changes to Zun component and Kuryr component in node1 and node2.

    • For Zun in node1, modify the /etc/zun/zun.conf

      Group Option Value
      [neutron_client] region_name RegionOne
    • Restart all the services of Zun in node1.

      $ sudo systemctl restart devstack@zun*
    • For Kuryr in node1, modify the /etc/kuryr/kuryr.conf

      Group Option Value
      [neutron] region_name RegionOne
    • Restart all the services of Kuryr in node1.

      $ sudo systemctl restart devstack@kur*
    • For Zun in node2, modify the /etc/zun/zun.conf

      Group Option Value
      [neutron_client] region_name RegionTwo
    • Restart all the services of Zun in node2.

      $ sudo systemctl restart devstack@zun*
    • For Kuryr in node2, modify the /etc/kuryr/kuryr.conf

      Group Option Value
      [neutron] region_name RegionTwo
    • Restart all the services of Zun in node2.

      $ sudo systemctl restart devstack@kur*
  • 4 Then, we must create environment variables for the admin user and use the admin project.

    $ source openrc admin admin
    $ unset OS_REGION_NAME
  • 5 Finally, use tricircle client to create pods for multi-region.

    $ openstack --os-region-name CentralRegion multiregion networking pod create --region-name CentralRegion
    $ openstack --os-region-name CentralRegion multiregion networking pod create --region-name RegionOne --availability-zone az1
    $ openstack --os-region-name CentralRegion multiregion networking pod create --region-name RegionTwo --availability-zone az2

How to play

  • 1 Create container glance image in RegionOne and RegionTwo.

    • Get docker image from Docker Hub. Run these command in the node1 and the node2.

      $ docker pull cirros
      $ docker save cirros -o /opt/stack/container_cirros
    • Use glance client to create container image.

      $ glance --os-region-name=RegionOne image-create --file /opt/stack/container_cirros --container-format=docker --disk-format=raw --name container_cirros --progress
      $ glance --os-region-name=RegionTwo image-create --file /opt/stack/container_cirros --container-format=docker --disk-format=raw --name container_cirros --progress
      
      $ openstack --os-region-name RegionOne image list
      
      +--------------------------------------+--------------------------+--------+
      | ID                                   | Name                     | Status |
      +--------------------------------------+--------------------------+--------+
      | 11186baf-4381-4e52-956c-22878b0642df | cirros-0.4.0-x86_64-disk | active |
      | 87864205-4352-4a2c-b9b1-ca95df52c93c | container_cirros         | active |
      +--------------------------------------+--------------------------+--------+
      
      $ openstack --os-region-name RegionTwo image list
      
      +--------------------------------------+--------------------------+--------+
      | ID                                   | Name                     | Status |
      +--------------------------------------+--------------------------+--------+
      | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | container_cirros         | active |
      | cf4a2dc7-6d6e-4b7e-a772-44247246e1ff | cirros-0.4.0-x86_64-disk | active |
      +--------------------------------------+--------------------------+--------+
  • 2 Create container network in CentralRegion.

    • Create a net in CentralRegion.

      $ openstack --os-region-name CentralRegion network create container-net
      
      +---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
      | Field                     | Value                                                                                                                                                                |
      +---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
      | admin_state_up            | UP                                                                                                                                                                   |
      | availability_zone_hints   |                                                                                                                                                                      |
      | availability_zones        | None                                                                                                                                                                 |
      | created_at                | None                                                                                                                                                                 |
      | description               | None                                                                                                                                                                 |
      | dns_domain                | None                                                                                                                                                                 |
      | id                        | 5e73dda5-902b-4322-b5b6-4121437fde26                                                                                                                                 |
      | ipv4_address_scope        | None                                                                                                                                                                 |
      | ipv6_address_scope        | None                                                                                                                                                                 |
      | is_default                | None                                                                                                                                                                 |
      | is_vlan_transparent       | None                                                                                                                                                                 |
      | location                  | cloud='', project.domain_id='default', project.domain_name=, project.id='2f314a39de10467bb62745bd96c5fe4d', project.name='admin', region_name='CentralRegion', zone= |
      | mtu                       | None                                                                                                                                                                 |
      | name                      | container-net                                                                                                                                                        |
      | port_security_enabled     | False                                                                                                                                                                |
      | project_id                | 2f314a39de10467bb62745bd96c5fe4d                                                                                                                                     |
      | provider:network_type     | vxlan                                                                                                                                                                |
      | provider:physical_network | None                                                                                                                                                                 |
      | provider:segmentation_id  | 1070                                                                                                                                                                 |
      | qos_policy_id             | None                                                                                                                                                                 |
      | revision_number           | None                                                                                                                                                                 |
      | router:external           | Internal                                                                                                                                                             |
      | segments                  | None                                                                                                                                                                 |
      | shared                    | False                                                                                                                                                                |
      | status                    | ACTIVE                                                                                                                                                               |
      | subnets                   |                                                                                                                                                                      |
      | tags                      |                                                                                                                                                                      |
      | updated_at                | None                                                                                                                                                                 |
      +---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    • Create a subnet in container-net

      $ openstack --os-region-name CentralRegion subnet create --subnet-range 10.0.60.0/24 --network container-net container-subnet
      
      +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
      | Field             | Value                                                                                                                                                                |
      +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
      | allocation_pools  | 10.0.60.2-10.0.60.254                                                                                                                                                |
      | cidr              | 10.0.60.0/24                                                                                                                                                         |
      | created_at        | 2019-12-10T07:13:21Z                                                                                                                                                 |
      | description       |                                                                                                                                                                      |
      | dns_nameservers   |                                                                                                                                                                      |
      | enable_dhcp       | True                                                                                                                                                                 |
      | gateway_ip        | 10.0.60.1                                                                                                                                                            |
      | host_routes       |                                                                                                                                                                      |
      | id                | b7a7adbd-afd3-4449-9cbc-fbce16c7a2e7                                                                                                                                 |
      | ip_version        | 4                                                                                                                                                                    |
      | ipv6_address_mode | None                                                                                                                                                                 |
      | ipv6_ra_mode      | None                                                                                                                                                                 |
      | location          | cloud='', project.domain_id='default', project.domain_name=, project.id='2f314a39de10467bb62745bd96c5fe4d', project.name='admin', region_name='CentralRegion', zone= |
      | name              | container-subnet                                                                                                                                                     |
      | network_id        | 5e73dda5-902b-4322-b5b6-4121437fde26                                                                                                                                 |
      | prefix_length     | None                                                                                                                                                                 |
      | project_id        | 2f314a39de10467bb62745bd96c5fe4d                                                                                                                                     |
      | revision_number   | 0                                                                                                                                                                    |
      | segment_id        | None                                                                                                                                                                 |
      | service_types     | None                                                                                                                                                                 |
      | subnetpool_id     | None                                                                                                                                                                 |
      | tags              |                                                                                                                                                                      |
      | updated_at        | 2019-12-10T07:13:21Z                                                                                                                                                 |
      +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  • 3 Create container in RegionOne and RegionTwo.

    Note

    We can give container a specific command to run it continually, e.g. "sudo nc -l -p 5000" .

    $ openstack --os-region-name RegionOne appcontainer run --name container01 --net network=$container_net_id --image-driver glance $RegionTwo_container_cirros_id sudo nc -l -p 5000
    
    +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field             | Value                                                                                                                                                                                                           |
    +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | tty               | False                                                                                                                                                                                                           |
    | addresses         | None                                                                                                                                                                                                            |
    | links             | [{u'href': u'http://192.168.1.81/v1/containers/ca67055c-635d-4603-9b0b-19c16eed7ef9', u'rel': u'self'}, {u'href': u'http://192.168.1.81/containers/ca67055c-635d-4603-9b0b-19c16eed7ef9', u'rel': u'bookmark'}] |
    | image             | 87864205-4352-4a2c-b9b1-ca95df52c93c                                                                                                                                                                            |
    | labels            | {}                                                                                                                                                                                                              |
    | disk              | 0                                                                                                                                                                                                               |
    | security_groups   | None                                                                                                                                                                                                            |
    | image_pull_policy | None                                                                                                                                                                                                            |
    | user_id           | 57df611fd8c7415dad6d2530bf962ecd                                                                                                                                                                                |
    | uuid              | ca67055c-635d-4603-9b0b-19c16eed7ef9                                                                                                                                                                            |
    | hostname          | None                                                                                                                                                                                                            |
    | auto_heal         | False                                                                                                                                                                                                           |
    | environment       | {}                                                                                                                                                                                                              |
    | memory            | 0                                                                                                                                                                                                               |
    | project_id        | 2f314a39de10467bb62745bd96c5fe4d                                                                                                                                                                                |
    | privileged        | False                                                                                                                                                                                                           |
    | status            | Creating                                                                                                                                                                                                        |
    | workdir           | None                                                                                                                                                                                                            |
    | healthcheck       | None                                                                                                                                                                                                            |
    | auto_remove       | False                                                                                                                                                                                                           |
    | status_detail     | None                                                                                                                                                                                                            |
    | cpu_policy        | shared                                                                                                                                                                                                          |
    | host              | None                                                                                                                                                                                                            |
    | image_driver      | glance                                                                                                                                                                                                          |
    | task_state        | None                                                                                                                                                                                                            |
    | status_reason     | None                                                                                                                                                                                                            |
    | name              | container01                                                                                                                                                                                                     |
    | restart_policy    | None                                                                                                                                                                                                            |
    | ports             | None                                                                                                                                                                                                            |
    | command           | [u'sudo', u'nc', u'-l', u'-p', u'5000']                                                                                                                                                                         |
    | runtime           | None                                                                                                                                                                                                            |
    | registry_id       | None                                                                                                                                                                                                            |
    | cpu               | 0.0                                                                                                                                                                                                             |
    | interactive       | False                                                                                                                                                                                                           |
    +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    
    $ openstack --os-region-name RegionOne appcontainer list
    
    +--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+
    | uuid                                 | name        | image                                | status  | task_state | addresses  | ports |
    +--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+
    | ca67055c-635d-4603-9b0b-19c16eed7ef9 | container01 | 87864205-4352-4a2c-b9b1-ca95df52c93c | Running | None       | 10.0.60.62 | []    |
    +--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+
    
    
    $ openstack --os-region-name RegionTwo appcontainer run --name container02 --net network=$container_net_id --image-driver glance $RegionTwo_container_cirros_id sudo nc -l -p 5000
    
    +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field             | Value                                                                                                                                                                                                           |
    +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | tty               | False                                                                                                                                                                                                           |
    | addresses         | None                                                                                                                                                                                                            |
    | links             | [{u'href': u'http://192.168.1.82/v1/containers/c359e48c-7637-4d9f-8219-95a4577683c3', u'rel': u'self'}, {u'href': u'http://192.168.1.82/containers/c359e48c-7637-4d9f-8219-95a4577683c3', u'rel': u'bookmark'}] |
    | image             | cd062c19-bb3a-4f60-b5ef-9688eb67b3da                                                                                                                                                                            |
    | labels            | {}                                                                                                                                                                                                              |
    | disk              | 0                                                                                                                                                                                                               |
    | security_groups   | None                                                                                                                                                                                                            |
    | image_pull_policy | None                                                                                                                                                                                                            |
    | user_id           | 57df611fd8c7415dad6d2530bf962ecd                                                                                                                                                                                |
    | uuid              | c359e48c-7637-4d9f-8219-95a4577683c3                                                                                                                                                                            |
    | hostname          | None                                                                                                                                                                                                            |
    | auto_heal         | False                                                                                                                                                                                                           |
    | environment       | {}                                                                                                                                                                                                              |
    | memory            | 0                                                                                                                                                                                                               |
    | project_id        | 2f314a39de10467bb62745bd96c5fe4d                                                                                                                                                                                |
    | privileged        | False                                                                                                                                                                                                           |
    | status            | Creating                                                                                                                                                                                                        |
    | workdir           | None                                                                                                                                                                                                            |
    | healthcheck       | None                                                                                                                                                                                                            |
    | auto_remove       | False                                                                                                                                                                                                           |
    | status_detail     | None                                                                                                                                                                                                            |
    | cpu_policy        | shared                                                                                                                                                                                                          |
    | host              | None                                                                                                                                                                                                            |
    | image_driver      | glance                                                                                                                                                                                                          |
    | task_state        | None                                                                                                                                                                                                            |
    | status_reason     | None                                                                                                                                                                                                            |
    | name              | container02                                                                                                                                                                                                     |
    | restart_policy    | None                                                                                                                                                                                                            |
    | ports             | None                                                                                                                                                                                                            |
    | command           | [u'sudo', u'nc', u'-l', u'-p', u'5000']                                                                                                                                                                         |
    | runtime           | None                                                                                                                                                                                                            |
    | registry_id       | None                                                                                                                                                                                                            |
    | cpu               | 0.0                                                                                                                                                                                                             |
    | interactive       | False                                                                                                                                                                                                           |
    +-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    
    $ openstack --os-region-name RegionTwo appcontainer list
    
    +--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+
    | uuid                                 | name        | image                                | status  | task_state | addresses   | ports |
    +--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+
    | c359e48c-7637-4d9f-8219-95a4577683c3 | container02 | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | Running | None       | 10.0.60.134 | []    |
    +--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+
  • 4 Execute container in RegionOne and RegionTwo.

    $ openstack --os-region-name RegionOne appcontainer exec --interactive container01 /bin/sh
    $ openstack --os-region-name RegionTwo appcontainer exec --interactive container02 /bin/sh
  • 5 By now, we successfully created multi-region container scenario. So we can do something on cross-region container, e.g. 1) RegionOne container ping RegionTwo container 2) Cross-Region Container Load Balancing.