Browse Source
1.What is the problem ? 1) Lack of installation doc for multi-region container management. 2.What is the solution for the problem? 1) Use zun component to provide container computation and use kuryr component to provide container network.In a word, make zun and kuryr compatible with tricircle. Change-Id: I7dd2ee04409eb20f2e4ad1e919d13cda1f7775a5 Signed-off-by: XuShimin <948026069@qq.com>changes/98/699098/2
XuShimin
2 years ago
committed by
Zhang Chi
2 changed files with 396 additions and 1 deletions
@ -0,0 +1,394 @@
|
||||
==================================================== |
||||
Installation guide for Tricircle work with Container |
||||
==================================================== |
||||
|
||||
Introduction |
||||
^^^^^^^^^^^^ |
||||
|
||||
In the `Multi-pod Installation with DevStack <https://docs.openstack.org/tricircle/latest/install/installation-guide.html#multi-pod-installation-with-devstack>`_ , |
||||
we have discussed how to deploy Tricircle in multi-region scenario with DevStack. |
||||
However, the previous installation guides have been on how to |
||||
manage virtual machines using tricircle and Nova in cross-region |
||||
openstack cloud environments. So, multi-region container management |
||||
is not supported in Tricircle. Meanwhile, OpenStack uses Zun |
||||
component to provide container management service, OpenStack also use |
||||
kuyr component and kuryr-libnetwork component to provide container network. |
||||
In view of the Tricircle Central_Neutron-Local_Neutron fashion, Tricircle work |
||||
with zun and kuryr will provide a cross-region container management solution. |
||||
This guide is to describe how tricircle work with container management and how |
||||
to deploy a multi-region container environment. |
||||
|
||||
|
||||
Prerequisite |
||||
^^^^^^^^^^^^ |
||||
|
||||
In this guide, we need specific versions of the zun project and |
||||
kuryr project source code. The source code versions of both projects |
||||
must be the Train version and upper. If not, we need to manually change |
||||
the source code for both projects. The modification example is as follows: |
||||
|
||||
- 1 Zun Source Code Modification: |
||||
For Zun project, we need modify the **neutron** function |
||||
in /zun/zun/common/clients.py file. |
||||
(The '+' sign represents the added line) |
||||
|
||||
.. code-block:: console |
||||
|
||||
def neutron(self): |
||||
if self._neutron: |
||||
return self._neutron |
||||
|
||||
session = self.keystone().session |
||||
session.verify = self._get_client_option('neutron', 'ca_file') or True |
||||
if self._get_client_option('neutron', 'insecure'): |
||||
session.verify = False |
||||
endpoint_type = self._get_client_option('neutron', 'endpoint_type') |
||||
+ region_name = self._get_client_option('neutron', 'region_name') |
||||
self._neutron = neutronclient.Client(session=session, |
||||
endpoint_type=endpoint_type, |
||||
+ region_name=region_name) |
||||
|
||||
return self._neutron |
||||
|
||||
- 2 Kuryr Source Code Modification: |
||||
For kuryr project, we need modify the **get_neutron_client** function |
||||
in /kuryr/kuryr/lib/utils.py file. |
||||
(The '+' sign represents the added line) |
||||
|
||||
.. code-block:: console |
||||
|
||||
def get_neutron_client(*args, **kwargs): |
||||
conf_group = kuryr_config.neutron_group.name |
||||
auth_plugin = get_auth_plugin(conf_group) |
||||
session = get_keystone_session(conf_group, auth_plugin) |
||||
endpoint_type = getattr(getattr(cfg.CONF, conf_group), 'endpoint_type') |
||||
+ region_name = getattr(getattr(cfg.CONF, conf_group), 'region_name') |
||||
|
||||
return client.Client(session=session, |
||||
auth=auth_plugin, |
||||
endpoint_type=endpoint_type, |
||||
+ region_name=region_name) |
||||
|
||||
|
||||
Setup |
||||
^^^^^ |
||||
|
||||
In this guide we take two nodes deployment as an example, the node1 run as RegionOne and |
||||
Central Region, the node2 run as RegionTwo. |
||||
|
||||
- 1 For the node1 in RegionOne and the node2 in RegionTwo, clone the code from Zun repository |
||||
and Kuryr repository to /opt/stack/ . If the code does not meet the requirements described |
||||
in the Prerequisite Section, modify it with reference to the modification example of the Prerequisite Section. |
||||
|
||||
- 2 Follow "Multi-pod Installation with DevStack" document `Multi-pod Installation with DevStack <https://docs.openstack.org/tricircle/latest/install/installation-guide.html#multi-pod-installation-with-devstack>`_ |
||||
to prepare your local.conf for the node1 in RegionOne and the node12 in RegionTwo, and add the |
||||
following lines before installation. Start DevStack in node1 and node2. |
||||
|
||||
.. code-block:: console |
||||
|
||||
enable_plugin zun https://git.openstack.org/openstack/zun |
||||
enable_plugin zun-tempest-plugin https://git.openstack.org/openstack/zun-tempest-plugin |
||||
enable_plugin devstack-plugin-container https://git.openstack.org/openstack/devstack-plugin-container |
||||
enable_plugin kuryr-libnetwork https://git.openstack.org/openstack/kuryr-libnetwork |
||||
|
||||
KURYR_CAPABILITY_SCOPE=local |
||||
KURYR_PROCESS_EXTERNAL_CONNECTIVITY=False |
||||
|
||||
- 3 After DevStack successfully started and finished, we need make some configuration changes to |
||||
Zun component and Kuryr component in node1 and node2. |
||||
|
||||
- For Zun in node1, modify the /etc/zun/zun.conf |
||||
|
||||
.. csv-table:: |
||||
:header: "Group", "Option", "Value" |
||||
|
||||
[neutron_client], region_name, RegionOne |
||||
|
||||
- Restart all the services of Zun in node1. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ sudo systemctl restart devstack@zun* |
||||
|
||||
- For Kuryr in node1, modify the /etc/kuryr/kuryr.conf |
||||
|
||||
.. csv-table:: |
||||
:header: "Group", "Option", "Value" |
||||
|
||||
[neutron], region_name, RegionOne |
||||
|
||||
- Restart all the services of Kuryr in node1. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ sudo systemctl restart devstack@kur* |
||||
|
||||
- For Zun in node2, modify the /etc/zun/zun.conf |
||||
|
||||
.. csv-table:: |
||||
:header: "Group", "Option", "Value" |
||||
|
||||
[neutron_client], region_name, RegionTwo |
||||
|
||||
- Restart all the services of Zun in node2. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ sudo systemctl restart devstack@zun* |
||||
|
||||
- For Kuryr in node2, modify the /etc/kuryr/kuryr.conf |
||||
|
||||
.. csv-table:: |
||||
:header: "Group", "Option", "Value" |
||||
|
||||
[neutron], region_name, RegionTwo |
||||
|
||||
- Restart all the services of Zun in node2. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ sudo systemctl restart devstack@kur* |
||||
|
||||
- 4 Then, we must create environment variables for the admin user and use the admin project. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ source openrc admin admin |
||||
$ unset OS_REGION_NAME |
||||
|
||||
- 5 Finally, use tricircle client to create pods for multi-region. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ openstack --os-region-name CentralRegion multiregion networking pod create --region-name CentralRegion |
||||
$ openstack --os-region-name CentralRegion multiregion networking pod create --region-name RegionOne --availability-zone az1 |
||||
$ openstack --os-region-name CentralRegion multiregion networking pod create --region-name RegionTwo --availability-zone az2 |
||||
|
||||
|
||||
How to play |
||||
^^^^^^^^^^^ |
||||
|
||||
- 1 Create container glance image in RegionOne and RegionTwo. |
||||
|
||||
- Get docker image from Docker Hub. Run these command in the node1 and the node2. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ docker pull cirros |
||||
$ docker save cirros -o /opt/stack/container_cirros |
||||
|
||||
- Use glance client to create container image. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ glance --os-region-name=RegionOne image-create --file /opt/stack/container_cirros --container-format=docker --disk-format=raw --name container_cirros --progress |
||||
$ glance --os-region-name=RegionTwo image-create --file /opt/stack/container_cirros --container-format=docker --disk-format=raw --name container_cirros --progress |
||||
|
||||
$ openstack --os-region-name RegionOne image list |
||||
|
||||
+--------------------------------------+--------------------------+--------+ |
||||
| ID | Name | Status | |
||||
+--------------------------------------+--------------------------+--------+ |
||||
| 11186baf-4381-4e52-956c-22878b0642df | cirros-0.4.0-x86_64-disk | active | |
||||
| 87864205-4352-4a2c-b9b1-ca95df52c93c | container_cirros | active | |
||||
+--------------------------------------+--------------------------+--------+ |
||||
|
||||
$ openstack --os-region-name RegionTwo image list |
||||
|
||||
+--------------------------------------+--------------------------+--------+ |
||||
| ID | Name | Status | |
||||
+--------------------------------------+--------------------------+--------+ |
||||
| cd062c19-bb3a-4f60-b5ef-9688eb67b3da | container_cirros | active | |
||||
| cf4a2dc7-6d6e-4b7e-a772-44247246e1ff | cirros-0.4.0-x86_64-disk | active | |
||||
+--------------------------------------+--------------------------+--------+ |
||||
|
||||
- 2 Create container network in CentralRegion. |
||||
|
||||
- Create a net in CentralRegion. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ openstack --os-region-name CentralRegion network create container-net |
||||
|
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| Field | Value | |
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| admin_state_up | UP | |
||||
| availability_zone_hints | | |
||||
| availability_zones | None | |
||||
| created_at | None | |
||||
| description | None | |
||||
| dns_domain | None | |
||||
| id | 5e73dda5-902b-4322-b5b6-4121437fde26 | |
||||
| ipv4_address_scope | None | |
||||
| ipv6_address_scope | None | |
||||
| is_default | None | |
||||
| is_vlan_transparent | None | |
||||
| location | cloud='', project.domain_id='default', project.domain_name=, project.id='2f314a39de10467bb62745bd96c5fe4d', project.name='admin', region_name='CentralRegion', zone= | |
||||
| mtu | None | |
||||
| name | container-net | |
||||
| port_security_enabled | False | |
||||
| project_id | 2f314a39de10467bb62745bd96c5fe4d | |
||||
| provider:network_type | vxlan | |
||||
| provider:physical_network | None | |
||||
| provider:segmentation_id | 1070 | |
||||
| qos_policy_id | None | |
||||
| revision_number | None | |
||||
| router:external | Internal | |
||||
| segments | None | |
||||
| shared | False | |
||||
| status | ACTIVE | |
||||
| subnets | | |
||||
| tags | | |
||||
| updated_at | None | |
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
|
||||
- Create a subnet in container-net |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ openstack --os-region-name CentralRegion subnet create --subnet-range 10.0.60.0/24 --network container-net container-subnet |
||||
|
||||
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| Field | Value | |
||||
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| allocation_pools | 10.0.60.2-10.0.60.254 | |
||||
| cidr | 10.0.60.0/24 | |
||||
| created_at | 2019-12-10T07:13:21Z | |
||||
| description | | |
||||
| dns_nameservers | | |
||||
| enable_dhcp | True | |
||||
| gateway_ip | 10.0.60.1 | |
||||
| host_routes | | |
||||
| id | b7a7adbd-afd3-4449-9cbc-fbce16c7a2e7 | |
||||
| ip_version | 4 | |
||||
| ipv6_address_mode | None | |
||||
| ipv6_ra_mode | None | |
||||
| location | cloud='', project.domain_id='default', project.domain_name=, project.id='2f314a39de10467bb62745bd96c5fe4d', project.name='admin', region_name='CentralRegion', zone= | |
||||
| name | container-subnet | |
||||
| network_id | 5e73dda5-902b-4322-b5b6-4121437fde26 | |
||||
| prefix_length | None | |
||||
| project_id | 2f314a39de10467bb62745bd96c5fe4d | |
||||
| revision_number | 0 | |
||||
| segment_id | None | |
||||
| service_types | None | |
||||
| subnetpool_id | None | |
||||
| tags | | |
||||
| updated_at | 2019-12-10T07:13:21Z | |
||||
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
|
||||
- 3 Create container in RegionOne and RegionTwo. |
||||
|
||||
.. note:: We can give container a specific command to run it continually, e.g. "sudo nc -l -p 5000" . |
||||
|
||||
|
||||
.. code-block:: console |
||||
|
||||
$ openstack --os-region-name RegionOne appcontainer run --name container01 --net network=$container_net_id --image-driver glance $RegionTwo_container_cirros_id sudo nc -l -p 5000 |
||||
|
||||
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| Field | Value | |
||||
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| tty | False | |
||||
| addresses | None | |
||||
| links | [{u'href': u'http://192.168.1.81/v1/containers/ca67055c-635d-4603-9b0b-19c16eed7ef9', u'rel': u'self'}, {u'href': u'http://192.168.1.81/containers/ca67055c-635d-4603-9b0b-19c16eed7ef9', u'rel': u'bookmark'}] | |
||||
| image | 87864205-4352-4a2c-b9b1-ca95df52c93c | |
||||
| labels | {} | |
||||
| disk | 0 | |
||||
| security_groups | None | |
||||
| image_pull_policy | None | |
||||
| user_id | 57df611fd8c7415dad6d2530bf962ecd | |
||||
| uuid | ca67055c-635d-4603-9b0b-19c16eed7ef9 | |
||||
| hostname | None | |
||||
| auto_heal | False | |
||||
| environment | {} | |
||||
| memory | 0 | |
||||
| project_id | 2f314a39de10467bb62745bd96c5fe4d | |
||||
| privileged | False | |
||||
| status | Creating | |
||||
| workdir | None | |
||||
| healthcheck | None | |
||||
| auto_remove | False | |
||||
| status_detail | None | |
||||
| cpu_policy | shared | |
||||
| host | None | |
||||
| image_driver | glance | |
||||
| task_state | None | |
||||
| status_reason | None | |
||||
| name | container01 | |
||||
| restart_policy | None | |
||||
| ports | None | |
||||
| command | [u'sudo', u'nc', u'-l', u'-p', u'5000'] | |
||||
| runtime | None | |
||||
| registry_id | None | |
||||
| cpu | 0.0 | |
||||
| interactive | False | |
||||
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
|
||||
$ openstack --os-region-name RegionOne appcontainer list |
||||
|
||||
+--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+ |
||||
| uuid | name | image | status | task_state | addresses | ports | |
||||
+--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+ |
||||
| ca67055c-635d-4603-9b0b-19c16eed7ef9 | container01 | 87864205-4352-4a2c-b9b1-ca95df52c93c | Running | None | 10.0.60.62 | [] | |
||||
+--------------------------------------+-------------+--------------------------------------+---------+------------+------------+-------+ |
||||
|
||||
|
||||
$ openstack --os-region-name RegionTwo appcontainer run --name container02 --net network=$container_net_id --image-driver glance $RegionTwo_container_cirros_id sudo nc -l -p 5000 |
||||
|
||||
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| Field | Value | |
||||
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
| tty | False | |
||||
| addresses | None | |
||||
| links | [{u'href': u'http://192.168.1.82/v1/containers/c359e48c-7637-4d9f-8219-95a4577683c3', u'rel': u'self'}, {u'href': u'http://192.168.1.82/containers/c359e48c-7637-4d9f-8219-95a4577683c3', u'rel': u'bookmark'}] | |
||||
| image | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | |
||||
| labels | {} | |
||||
| disk | 0 | |
||||
| security_groups | None | |
||||
| image_pull_policy | None | |
||||
| user_id | 57df611fd8c7415dad6d2530bf962ecd | |
||||
| uuid | c359e48c-7637-4d9f-8219-95a4577683c3 | |
||||
| hostname | None | |
||||
| auto_heal | False | |
||||
| environment | {} | |
||||
| memory | 0 | |
||||
| project_id | 2f314a39de10467bb62745bd96c5fe4d | |
||||
| privileged | False | |
||||
| status | Creating | |
||||
| workdir | None | |
||||
| healthcheck | None | |
||||
| auto_remove | False | |
||||
| status_detail | None | |
||||
| cpu_policy | shared | |
||||
| host | None | |
||||
| image_driver | glance | |
||||
| task_state | None | |
||||
| status_reason | None | |
||||
| name | container02 | |
||||
| restart_policy | None | |
||||
| ports | None | |
||||
| command | [u'sudo', u'nc', u'-l', u'-p', u'5000'] | |
||||
| runtime | None | |
||||
| registry_id | None | |
||||
| cpu | 0.0 | |
||||
| interactive | False | |
||||
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
||||
|
||||
$ openstack --os-region-name RegionTwo appcontainer list |
||||
|
||||
+--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+ |
||||
| uuid | name | image | status | task_state | addresses | ports | |
||||
+--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+ |
||||
| c359e48c-7637-4d9f-8219-95a4577683c3 | container02 | cd062c19-bb3a-4f60-b5ef-9688eb67b3da | Running | None | 10.0.60.134 | [] | |
||||
+--------------------------------------+-------------+--------------------------------------+---------+------------+-------------+-------+ |
||||
|
||||
- 4 Execute container in RegionOne and RegionTwo. |
||||
|
||||
.. code-block:: console |
||||
|
||||
$ openstack --os-region-name RegionOne appcontainer exec --interactive container01 /bin/sh |
||||
$ openstack --os-region-name RegionTwo appcontainer exec --interactive container02 /bin/sh |
||||
|
||||
- 5 By now, we successfully created multi-region container scenario. So we can do something |
||||
on cross-region container, e.g. 1) RegionOne container ping RegionTwo container 2) Cross-Region Container Load Balancing. |
||||
Loading…
Reference in new issue