385 Commits

Author SHA1 Message Date
Zuul
603951809a Merge "ovn-migration: Introduce migrate mode to DB sync tool" 2021-04-26 14:22:47 +00:00
Jakub Libosvar
456acdeb8f ovn-migration: Delete FIP agent gateway ports
In DVR environment, there are FG interfaces on nodes. We can delete
those after the migration.

Change-Id: I44967e55213d12af24acbf9561a96afb34548324
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2021-04-22 06:47:07 +00:00
Jakub Libosvar
ae78e812d1 ovn-migration: Introduce migrate mode to DB sync tool
The new "migrate" mode for the DB sync utility changes DB contents as
follows:

 - it changes vxlan networks to Geneve, including its allocation in
   order to avoid future collisions when creating new geneve networks
 - it removes settings from ports' vif_details that are no longer
   needed, such as hybrid plugging or bridge_name for the trunk bridges
 - it sets profile for subports - OVN doesn't use trunk_details but port
   profile to store data about trunk. Subports have tag and parent_name
   fileds.

Previously, the vxlan to Geneve change was done via ansible role. The
tasks in the role were replaced by the script therefore the role is
removed.

Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
Change-Id: I29a39108d9fddb30050ec63a1cdf6bba0400e136
2021-04-02 13:23:57 +00:00
Bence Romsics
7f35e4e857 Physical NIC RP should be child of agent RP
In the fix for #1853840 I made a mistake and since then we created
the physical NIC resource providers as a child of the hypervisor
resource provider instead of the agent resource provider. Here:

https://review.opendev.org/c/openstack/neutron/+/696600/3/neutron/agent/common/placement_report.py#159

This *did not* break the minimum bandwidth aware scheduling.
But still there are multiple problems:

1) If you created your physical NIC RPs before the fix for #1853840
   but upgraded to after the fix for #1853840, then resource syncs
   will throw an error in neutron-server at each physical NIC RP
   update. That pollutes the logs and wastes some resources since
   the prohibited update will be forever retried.

2) If you created your physical NIC RPs after the fix for #1853840
   then your physical NIC RPs have the wrong parent. Which again
   does not break minimum bandwidth aware scheduling. But it may pose
   problems for later features wanting to build on the originally
   planned RP tree structure.

3) Cleanup of decommissioned RPs is a bit different than expected.
   This cleanup was always left to the admin, so it only affects a
   manual process.

The proper RP structure was and should be the following:

The hypervisor RP(s) must be the root(s).
As a child of each hypervisor RP, there should be an agent RP.
The physical NIC RPs should be the children of the agent RPs.

Unfortunately at the moment the Placement API generically prohibits
update of the parent resource provider id in a PUT request:

https://docs.openstack.org/api-ref/placement/?expanded=update-resource-provider-detail#update-resource-provider

Therefore without a later Placement change we cannot fix the RPs
already created with the wrong parent. However we can fix the RPs
to be created later. We do that here. We also fix a bug in the unit
tests that allowed the wrong parent to pass unnoticed. Plus we
add an extra log message to direct the user seeing the pollution
in the logs to the proper bug report.

There may be a follow up patch later, because not all RP re-parenting
operations are problematic, therefore we are thinking of relaxing
this blanket prohibition in Placement. When Placement allows updates
to the parent id we can fix RPs already created with the wrong parent
too.

Change-Id: I7caa8827d22103600ca685a58294640fc831dbd9
Closes-Bug: #1921150
Co-Authored-By: "Balazs Gibizer" <balazs.gibizer@est.tech>
Related-Bug: #1853840
2021-03-26 15:52:18 +01:00
Jakub Libosvar
94bea77fd7 migration: Remove crudini when migrating tunnels
It turned out there doesn't need to be crudini tool present on the node
running the command. This patch fetches the Neutron conf file instead
and performs an ini lookup over it in order to get the DB connection
string.

Change-Id: Iaf79b8512a920e9f667bd6881d50e8852595fa71
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2021-03-24 18:03:20 +01:00
Zuul
ca86d4dbe3 Merge "ovn-migration: Use DHCP nodes to configure dnsmasq" 2021-03-24 14:29:31 +00:00
Jakub Libosvar
9483037f5d ovn-migration: Add doctext explaining VXLAN to Geneve migration
The Ansible command itself is hard to read, the patch adds some doctext
to improve understanding of the ansible task.

Change-Id: I4ab7a83da9bbf64ee6b19b2a0611fd64e09e2132
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2021-03-23 09:55:52 +01:00
Zuul
05d916c0e7 Merge "ovn-migration: Change network type from vxlan to Geneve" 2021-03-22 18:33:44 +00:00
Jakub Libosvar
569aa4452a ovn-migration: Use DHCP nodes to configure dnsmasq
Previously, the migration script used controller nodes to configure
dnsmasq when configuring new MTU. Controller nodes may not run DHCP
agents. The patch detects DHCP agents instead and uses those to
configure dnsmasq.

Change-Id: Ib468c04779af7aaf4dedf84ba885bce71078248b
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2021-03-22 14:49:54 +01:00
Jakub Libosvar
d3fce77ff5 ovn-migration: Change network type from vxlan to Geneve
The patch goes to one of ovn-dbs nodes and runs a SQL query to change
all vxlan networks to Geneve. It's done via Python and sqlalchemy in
neutron_api container because mysql client is not installed there. This
approach was chosen to avoid installing more dependencies.

Change-Id: Ic417a115fdc212527866122bc9d3c93ea9599bdf
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2021-03-19 16:55:41 +01:00
Jakub Libosvar
7bb5fb2de1 ovn-migration: UNDERCLOUD_NODE_USER variable
The undercloud node user is configurable in TripleO and isn't always set
to heat-admin. This patch introduces an environment variable for cases
where user is different.

Change-Id: If65925ded1b5df2bfdcfba50445ff7d821c725d8
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2021-03-19 12:13:01 +01:00
Flavio Fernandes
ee75371df4 [OVN] Updates to tools/migrate_names.txt
Updates to tools/migrate_names.txt to correctly represent
mapping between neutron and networking-ovn for metadata
agent.

Change-Id: I52f4d20c7190490b707a08e0f7dd06ccd6f43f69
2020-12-14 16:44:40 -05:00
Roman Safronov
526693ed3c Remove ML2OVS-specific templates before migration to OVN
When running migration from ML2OVS to ML2OVN using infrared plugin
some ML2OVS-specific templates remain in 'overcloud deploy' command.
This is not correct and can cause issues.

Change-Id: I41c81cd42ab85395cb5fc56f339d14f2e939ebbc
Closes-bug: #1896470
2020-10-12 12:15:58 +03:00
Zuul
21fc89401e Merge "Make abandon_old_reviews script to be compatible with python 3" 2020-09-28 08:52:29 +00:00
Slawek Kaplonski
e613ffab7d Make abandon_old_reviews script to be compatible with python 3
Change-Id: Ic845105050c2b32c4fe0943ae3bcc462a6f5ed51
2020-09-25 18:34:09 +02:00
Zuul
8575f60e86 Merge "[OVN] Updates to tools/migrate_names.txt" 2020-09-17 21:17:44 +00:00
Flavio Fernandes
3cae952b72 [OVN] Updates to tools/migrate_names.txt
Updates to tools/migrate_names.txt to correctly represent
mapping between neutron and networking-ovn.

networking_ovn/cmd/neutron_ovn_db_sync_util.py  neutron/cmd/ovn/neutron_ovn_db_sync_util.py
networking_ovn/tests/unit/cmd/test_neutron_ovn_db_sync_util.py  neutron/tests/unit/cmd/ovn/test_neutron_ovn_db_sync_util.py
networking_ovn/tests/unit/l3/test_l3_ovn.py  neutron/tests/unit/services/ovn_l3/test_plugin.py
networking_ovn/tests/unit/l3/test_l3_ovn_scheduler.py  neutron/tests/unit/scheduler/test_l3_ovn_scheduler.py

Change-Id: I66851f9bc4177a1d667303bc1ec1c6e7ba7a7e47
2020-09-14 18:36:06 +00:00
Jakub Libosvar
4b30dc2330 ovn migration: Fix neutron server container name
Previously docker command used grep to search for neutron server
container. It used image name instead of container name. With the switch
to podman, it stopped working. This patch fixes the container name.

Change-Id: Id650c2e7ac427dd53ebaab4e6d95d54efc657e3f
Closes-bug: #1895108
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2020-09-10 11:46:50 +02:00
Zuul
be4bd8aadb Merge "[OVN] Updates to tools/migrate_names.txt" 2020-09-08 22:27:30 +00:00
Zuul
b78a953d7f Merge "Force ncat install for functional/fullstack jobs" 2020-09-07 11:01:26 +00:00
Flavio Fernandes
1d13b20c94 [OVN] Updates to tools/migrate_names.txt
Updates to tools/migrate_names.txt to correctly represent
mapping between neutron and networking-ovn.

Change-Id: I4a85391b6b6422b66875f23bd9f93216be0bd087
2020-09-05 07:24:46 -04:00
Nate Johnston
b64a9c177f Force ncat install for functional/fullstack jobs
This is not available by default in Ubuntu Focal which we are migrating
to in this cycle as a community goal.

Change-Id: I2a7d03addc54a05aa310c7b72e6f91a98bf79b65
2020-08-28 16:00:41 +00:00
Lucas Alvares Gomes
68ec29abf0 [OVN] Use the OVN DevStack module
The DevStack module for OVN has now been moved to the DevStack repository,
this patch is deletes it from the Neutron repository.

Depends-On: https://review.opendev.org/#/c/748140/
Change-Id: I2c30b8130525380234d767a7cb46f9bcca2a0d6b
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2020-08-26 10:09:28 +01:00
Jakub Libosvar
4ad21fd23a migration: Restart OVS after setting protocols
This patch is a workaround for bug in OVS [1]. The OVS is restarted
after setting protocols for the integration bridge. This will cause
a data plane disruption. We can remove the workaround once
the bug [1] is fixed.

Closes-bug: #1890596

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1782834

Change-Id: Ia635f705fa03036a9e9c1c14eab59038ec39bda3
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2020-08-06 14:48:51 +02:00
Zuul
1075e79be5 Merge "ovn migration: Support stack name" 2020-07-24 09:44:21 +00:00
Zuul
d05fcfdf05 Merge "ovn-migration: Remove docker references" 2020-07-23 14:50:29 +00:00
Jakub Libosvar
afd63d0b36 ovn migration: Support stack name
The heat stack name is variable. This patch adds a new environment
variable to support migrating stacks that are different name than
overcloud.

Change-Id: I6fd72bf83def28ae633d720b8495888cea3ac0a3
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2020-07-22 16:33:32 +00:00
Zuul
5ed6fccc98 Merge "migration: Use ansible-inventory to parse tripleo inventory" 2020-07-14 14:40:33 +00:00
Rodolfo Alonso Hernandez
7ebd1d16eb Install python-openvswitch from PyPI using requirements
This package is available in PyPI as "ovs".

Trivial-Fix

Change-Id: I1e4de27fa5e20308d778f1f3d55bc5fd9481bee8
2020-07-09 12:29:50 +00:00
Maciej Jozefczyk
fb2806f808 [OVN] Unify OVN/OVS compilation
There were duplicated methods doing almost the same in terms
of OVS/OVN compilation.

This change:
 * move of OVS related compilation code to devstack/lib/ovs
 * delete of OVS related compilation code from devstack/lib/ovn_agent
 * source unified functions in devstack/lib/ovn_agent from
   devstack/lib/ovs
 * Unify NEUTRON_PATH variable to NEUTRON_DIR

Closes-Bug: #1877377

Change-Id: Ia012a8e116a276a6674f86366c803e0e2d8ff704
2020-07-07 22:05:59 +00:00
Oliver Walsh
aa6491a9d9 migration: Use ansible-inventory to parse tripleo inventory
Instead of adapting to changes to the tripleo inventory structure let
ansible parse it for us using ansible-inventory.

Change-Id: I34ad0fd5feed65dd1266993a77f6ebc69fecfdfb
Closes-bug: #1884764
2020-07-02 11:19:16 +01:00
Jakub Libosvar
0e597a36d3 ovn-migration: Remove docker references
Docker commands are replaced with podman. References to docker resources
is fixed. Some improvements to avoid storing container ids and
performing operations is also done in this patch.

Change-Id: I2b9661d06b30eeb7752bdb4fad73778d2ed940d7
2020-06-25 10:51:54 +02:00
Flavio Fernandes
e3e1d994d8 [OVN] Updates to tools/migrate_names.txt
Updates to tools/migrate_names.txt to correctly represent
mapping between neutron and networking-ovn.

Change-Id: I76562302119c6727a78bc72c4cf5346b3b8befe6
2020-06-20 19:41:54 +00:00
Jakub Libosvar
d086058895 ovn: Use newer OVS
OVS tag v2.13.0 is not compatible with current used kernel on upstream
Bionic Ubuntu kernel 4.15.0. This patch sticks to commit hash to unblock
the gate. We can either change to newer 2.13 tag once released or better
stop compiling OVS and use one packaged by Ubuntu.

Run functional/fullstack tests with selected OVN and OVS versions

Previously for functional tests OVS version was hardcoded - v2.12.0,
and OVN was installed from OVS repository.

After we merged OVN driver to Neutron tree we run both
Neutron/OVS and Neutron/OVN functional tests in one job.

This patch adds possibility to specify from which tag/branch OVS and OVN
should be checkout.

Change-Id: I83688031951b97bfe64f3aaa761ad7afc1d5ea55
Closes-Bug: #1883601
Closes-Bug: #1878160
2020-06-19 15:57:28 +02:00
Zuul
646d2db6d1 Merge "ovn-migration: Stop ml2/ovs agents before installing OVN resources" 2020-06-10 17:48:12 +00:00
Zuul
f6f82b18d9 Merge "Stop to use the __future__ module." 2020-06-05 14:25:29 +00:00
Zuul
84793094bc Merge "Fix ssh to nodes during ML2/OVS to ML2/OVN migration" 2020-06-04 13:41:37 +00:00
Hervé Beraud
236a0c4081 Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: Ia7c6b8ba6d3842a19e75fd64f9e56f2cf5f74409
2020-06-04 08:34:47 +00:00
Flavio Fernandes
132261aa70 [OVN] Updates to tools/migrate_names.txt
Updates to tools/migrate_names.txt to correctly represent
mapping between neutron and networking-ovn.

networking_ovn/common/ovn_client.py     neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py
networking_ovn/common/acl.py            neutron/common/ovn/acl.py
doc/source/contributor/design           doc/source/contributor/internals/ovn

Change-Id: Ib8973ee96431815377a36f85e2bef0ef87e96ca1
2020-05-29 16:00:15 -04:00
Roman Safronov
0402337c15 Fix ssh to nodes during ML2/OVS to ML2/OVN migration
Migration ansible playbooks are not able to connect to overcloud
nodes using ssh due to bad settings in the generated ansible
inventory file. Ctlplane subnet ip addresses should be used by
ansible in order to be able to establish connection successfully.

Closes-bug: #1881029
Change-Id: I1588b88279ba1e25706cc74f4a888901e5aadcb6
2020-05-29 19:27:29 +00:00
Jakub Libosvar
4d5b9a62b9 ovn-migration: Stop ml2/ovs agents before installing OVN resources
TripleO used to uninstall services if the resources were set to None. It
was not intentional and TripleO no longer does that. This patch manually
removes dhcp, l3, metadata and ovs agents from controller and compute
nodes, including its systemd service files.

Closes-Bug: #1878358
Change-Id: I140ae0c476bf9e19b4429ce388ddc7cc3f4cd421
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2020-05-13 09:45:07 +02:00
Zuul
73c26d6126 Merge "migration: Use ForceNeutronDriverUpdate when migrating" 2020-05-08 11:36:20 +00:00
Zuul
413b266bb0 Merge "migration: Replace paunch with podman and ip commands" 2020-05-06 11:02:14 +00:00
Lucas Alvares Gomes
e53dd36498 migration: Use ForceNeutronDriverUpdate when migrating
A change in TripleO [1] changed behavior and forbids update when backend
is changed. Unless ForceNeutronDriverUpdate is set. This patch sets the
parameter when migrating from ML2/OVS to OVN. More information at [2].

[1]
379f886119
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1769880

Change-Id: Ic85a214a96d3ffa054aa5e1c17ab430c71c4e4a1
Co-Authored-By: Jakub Libosvar <jlibosva@redhat.com>
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2020-04-29 15:19:32 +01:00
Brian Haley
4fb505891e Updates for python3.8
With the move to the Victoria job template in
https://review.opendev.org/#/c/722681/, the py37 jobs no
longer get run, so the check and gate job entries can
be removed.

Added a keepalived py38 KillFilter line to match the py36
and py37 ones.

Also updated TESTING.rst to use py38 in all examples.

Change-Id: Ief793b54d53c3239cfb24278e88e4f4189bbc2c2
2020-04-28 14:03:21 -04:00
Jakub Libosvar
bfeb52b975 migration: Replace paunch with podman and ip commands
The patch removes paunch and cleans network namespaces by iterating over
namespaces belonging to Neutron agents. It also clears side-car
containers spawned by Neutron using podman command.

Change-Id: I017e7b3c3674e065a523eff5ebbea352f619f45e
2020-04-28 17:15:56 +02:00
Lucas Alvares Gomes
b27940c205 Migrate the OVN migration scripts
This patch is migrating the OVN migration scripts. At the moment, only
migration from ML2/OVS to ML2/OVN in a TripleO environment is supported.

Co-Authored-By: Miguel Angel Ajo <majopela@redhat.com>
Co-Authored-By: Jakub Libosvar <libosvar@redhat.com>
Co-Authored-By: Daniel Alvarez <dalvarez@redhat.com>
Co-Authored-By: Maciej Józefczyk <mjozefcz@redhat.com>
Co-Authored-By: Numan Siddique <nusiddiq@redhat.com>
Co-Authored-By: Roman Safronov <rsafrono@redhat.com>
Co-Authored-By: Terry Wilson <twilson@redhat.com>

Related-Blueprint: neutron-ovn-merge
Change-Id: I925f4b650209b8807290d6a69440c31fd72a1762
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2020-03-31 15:35:35 +01:00
Flavio Fernandes
933049f529 [OVN] Updates to tools/migrate_names.txt
Updates to tools/migrate_names.txt to correctly represent
mapping between neutron and networking-ovn.

networking_ovn/ovsdb/commands.py        neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py
networking_ovn/ovsdb/impl_idl_ovn.py    neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py
networking_ovn/ovsdb/ovn_api.py         neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/api.py
networking_ovn/ovsdb/ovsdb_monitor.py   neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py
networking_ovn/ovsdb/worker.py          neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/worker.py

Change-Id: Icc4a9da85e69560c8980f75bcad44564e6f4906e
2020-03-10 22:15:44 +00:00
Zuul
6a8277d70e Merge "Merge networking-ovn documentation into neutron" 2020-02-14 12:55:43 +00:00
Rodolfo Alonso Hernandez
2dd6222198 Set an absolute path in rootwrap filters_path
If a relative path is set in rootwrap filters_path, the rootwrap
command must be executed from the neutron root directory. Some
commands, like those execute by RootHelperProcess, do not have
a defined working directory. To avoid the problems present in the
related bug, an absolute path is set in the rootwrap configuration.

Related-Bug: #1862927

Change-Id: I02ce4d6b84242c42ec26954eeb9b776cb779bff4
2020-02-13 17:10:53 +00:00