76 Commits

Author SHA1 Message Date
tpsilva
b76a592997 Explicitly load nf_conntrack_ipv4 module
When neutron-openvswitch-agent is using the openvswitch firewall,
it needs the nf_conntrack_ipv4 module to be loaded. Usually, this
module gets loaded by some other external tool, but in case this
does not happen, neither the charm nor neutron will load it, so
all traffic to the instances in this host will fail. This patch
fixes that by explicitly loading the module.

Change-Id: Ia788e870c124de7da17961c02259cfe80938e5d2
Closes-bug: #1834213
2019-08-29 18:40:08 -03:00
Zuul
3da7ab086f Merge "Replace SR-IOV systemd unit script" 2019-07-02 10:31:27 +00:00
Zuul
1551f9c297 Merge "Set MTU on dpdk devices when using a bond" 2019-07-01 21:54:11 +00:00
Liam Young
38452772a9 Set MTU on dpdk devices when using a bond
MTU is not being set on the dpdk devices when a dpdk bond is
being requested. Currently the mtu is being set in
neutron_ovs_utils.configure_ovs by iterating over the dictionary
returned by neutron_ovs_context.resolve_dpdk_bridges. But this
context is expecting the data-port config option to be a list of
bridge:mac mappings. In the case of a dpdk bond however,
data-port is set of bridge:bond-name mappings. The context then uses
the bond name as if it were a mac address to find the
underlying pci device which naturally fails and then returns
an empty context. This is fine as configure_ovs then moves on
to setup the dpdk bonds correctly. Unfortunately the code
to apply mtus to the devices in the case of a bond was missing
and this change adds it in.

Change-Id: I2fb8ccf48ffd1a3ab227b883ceacac89ff57ea02
2019-07-01 13:48:15 +00:00
Nicolas Pochet
d02fad482d
Replace SR-IOV systemd unit script
* The goal of this change is to enable the ability to configure only the
VFs that are configured through the charm and not fallback to the
blanket configuration.
* This python version of the script brings unit-tests that fully covers
it.
* Move the the template files to `files` and modify `neutron_ovs_utils`
accordingly.

Closes-Bug: 1832379
Depends-On: https://review.opendev.org/#/c/664837/

Change-Id: I7ad1ebc16883bda23cbad89a852e7e8f88f49c49
2019-06-28 17:37:51 +02:00
Zuul
bc3e150f88 Merge "Add support for FWaaS v2 logging" 2019-06-25 18:37:42 +00:00
James Page
9b0de9bbff Add support for FWaaS v2 logging
Enable support for configuration of FWaaS v2 firewall group
logging.

Configuration options mirror those for neutron-openvswitch
for security group logging.

This feature is currently only enabled for FWaaS v2 at Stein
for the charms (but is supported back to Queens in Neutron).

Change-Id: Ic60ee47078089c59ccb09b8659422e7ad7081149
Partial-Bug: 1831972
2019-06-25 16:53:24 +01:00
James Page
3c66c48bbd dpdk: misc fixes for EAL initialization
For OpenStack Queens and later instances are setup with vhostuser
ports in server mode, with the OVS side of the port connecting
as a client to the vhostuser socket on disk; as a result we no
longer need to pass permissions information via dpdk-extra (the
two passed options are not valid in later OVS versions).

In addition, we should also whitelist the devices we're going to
use; this ensures that EAL initialization does not take an extended
period of time reducing the amount of time taken to restart OVS.

Change-Id: I224e778de0ed6e279b2de7f4f46781df33121165
Closes-Bug: 1833734
Closes-Bug: 1793729
2019-06-24 10:11:04 +01:00
David Ames
a1639fe51f Enable isolated provider network metadata access
When an isolated provider network with no virtual routers metadata
access occurs in the qdhcp netns.

Without the force_metadata option in dhcp_agent.ini and the haproxy
package installed ns-metadata-proxy is not enabled. ns-metdata-proxy
sits in the ip netns and proxies requests from 169.254.169.254 to the
nova-api-metadata service outside the netns.

This change adds the force_metadata option and installs haproxy when
enable-local-dhcp-and-metadata is True.

Closes-Bug: #1831935

Change-Id: Iaad1501e8d7d58888ef0917b6700d22a7cf05ecf
2019-06-07 10:34:50 -07:00
Sahid Orentino Ferdjaoui
380adb7271 pci: use sriov-device-mappings when configure sriov devices
When 'sriov-numvfs' is configured in 'auto', only the devies set in
'sriov-device-mappings' are discovered and automatically configured.

Change-Id: I1be61a19639d366d787fb92815c3a8a5c302fbda
Closes-Bug: #1818975
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-05-23 09:58:02 +02:00
Zuul
0610e920a7 Merge "dpdk: set global mtu to the dpdk interfaces" 2019-05-20 08:15:41 +00:00
Zuul
611f604918 Merge "Sync charm-helpers to get fix for PR 320" 2019-05-17 09:09:06 +00:00
Edward Hope-Morley
f7cfbe8bd3 Sync charm-helpers to get fix for PR 320
Allows the charm to filter out interfaces from data-port
that don't exist on the local host.

Change-Id: I3a8ee204facf68753c564a297825666900c1b835
Closes-Bug: #1822558
2019-05-16 11:43:32 +01:00
Sahid Orentino Ferdjaoui
440d6f0f49 dpdk: set global mtu to the dpdk interfaces
Change-Id: I7d3c5fbd15b993e2ce8d48fac049ecab5e401fae
Closes-Bug: 1827256
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-05-14 17:13:16 +02:00
Sahid Orentino Ferdjaoui
9d0473c2a9 dpdk: add function to set mtu to dpdk interfaces
Change-Id: Id503f76740cc270b55e48480d95577a31933fea6
Related-Bug: 1827256
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-05-14 17:13:16 +02:00
Sahid Orentino Ferdjaoui
18e251b4b1 unittest: fix missing mock for list_nics
Change-Id: I0b31fb131bab4d87ad3a2695c584f13d2bee57f2
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-05-14 17:13:16 +02:00
Sahid Orentino Ferdjaoui
381a3b1a33 hooks: fix initial sriov configurations
To configure SRIOV devices it was expected that the 'sriov-numvfs'
config option to be changed but during an initial setup this not
happens.

In this commit we remove the condition but add a logic in
PCINetDevice to avoid reconfiguring PF devices if not necessary.

Change-Id: Ib8232b29f76ca7e25e1cd835d5e31a276000f1d4
Closes-Bug: #1817079
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@canonical.com>
2019-03-07 11:54:37 +00:00
Pete Vander Giessen
00ca87fec3 Added gc_threshold overrides to sysctl.conf
When clouds have a large number of hosts, the default size of the ARP
cache is too small. The cache can overflow, which means that the
system has no way to reach some ip addresses.

Setting the threshold limits higher addresses the situation, in a
reasonably safe way (the maximum impact is 5MB or so of additional RAM
used). Docs on ARP at http://man7.org/linux/man-pages/man7/arp.7.html,
and more discussion of the issue in the bug.

Change-Id: I329ec51eff85a2a99a929c67ff0c68b3b36d7273
Closes-Bug: 1780348
2019-02-28 11:55:05 +01:00
Dmitrii Shcherbakov
1486c83a1f Allow dvr_snat l3 agent mode to be used with DVR
Currently it is a requirement to have a network node with an l3 agent
running in the dvr_snat mode even for DVR deployments that do not use
SNAT or have a very limited usage of SNAT.

It is not possible to disable snat completely:
https://bugs.launchpad.net/neutron/+bug/1761591

Neutron creates a network:router_centralized_snat port and if it is not
possible to find a dvr_snat agent to schedule it on there are various
side-effects which are not seen at first. For example, Designate stops
creating records for floating IPs and Neutron/Designate integration is,
therefore, not functional.

The Neutron DVR documentation says that dvr_snat should be used on
network nodes. However, there is nothing restricting a DVR deployment
from using dvr_snat l3 agents on every compute node and not having
dedicated network nodes.

This change modifies neutron-openvswitch to optionally enable dvr_snat
l3 agent mode (this includes supporting L3HA routers if enabled). As a
result, it is possible to have deployments without neutron-gateway thus
saving on the amount of required nodes. Care should be taken when a
large amount of L3HA routers is used and using DVR routers without L3HA
is a recommended.

Change-Id: Iad3a64967f91c81312911f6db856ce2271b0e068
Closes-Bug: #1808045
2018-12-12 00:37:29 +03:00
David Ames
b404c18a50 Explicitly install dependencies for DVR
The DVR package neutron-l3-agent depends on python-neutron-fwaas or
python3-neutron-fwaas. On Rocky without being explicit it will
incorrectly install the python2 version which in turn installs many
python2 dependencies.

This change explicitly adds python3-neutron-fwaas as a dependency on
Rocky and updates python-neutron-fwaas as a purge package.

Change-Id: Idb537df84b044e8ea92527a5f56ab06a37b9ffad
Closes-Bug: #1803744
2018-11-16 11:59:13 -08:00
James Page
3015d95af8 py3: Switch to Python 3 for OpenStack Rocky
Switch to execution of Neutron agents under Python 3 for
OpenStack Rocky; this is triggered by the nova-compute charm
mutating the container scoped neutron-plugin relation post
OpenStack series upgrade.

Update default smoke test target to bionic-rocky.

Change-Id: Ic5e96336b6a2ca474fc28d358553c6a05e1a75ce
2018-10-04 14:58:19 +01:00
James Page
3b72caa031 Gate install of tmpfiles configuration
The vhost-user tmpfiles configuration is only applicable in deployments
using libvirt/kvm with nova-compute.

Ensure appropriate user and group exists before installing tmpfiles.d
configuration.

Change-Id: I471ff459e5f979cb6781193fb074f6f5f7ee967f
Closes-Bug: 1792414
2018-09-13 11:17:46 -06:00
James Page
96c1788e94 Misc updates for DPDK support
Fix use of OVS DPDK context by direct use of methods on context
for OVS table values.

For modern OVS versions that require the PCI address of the
DPDK device for type=dpdk ports, use a hash of the PCI address
for the port name rather than the index of the PCI device in
the current list of devices to use; this is idempotent in the
event that the configuration changes and new devices appear
in the list of devices to use for DPDK.

Only set OVS table values if the value has changed; OVS will
try to re-allocate hugepage memory, irrespective as to whether
the table value actually changed.

Switch to using /run/libvirt-vhost-user for libvirt created DPDK
sockets, allowing libvirt to directly create the socket as part
of instance creation; Use systemd-tmpfiles to ensure that the
vhost-user subdirectory is re-created on boot with the correct
permissions.

Scan data-port and dpdk-bond-mappings for PCI devices to use
for DPDK to avoid having to replicate all PCI devices in data-port
configuration when DPDK bonds are in use.

Change-Id: I2964046bc8681fa870d61c6cd23b6ad6fee47bf4
2018-09-04 12:43:03 +01:00
Edward Hope-Morley
e8af22633b Fix charm upgrade broken by commit 862c362
Currently, upgrading this charm on a host that is running
ovs >= 2.6 will break because the OVS_DEFAULT config file
is not expected to be written by the charm.

Change-Id: I33352deb3b60231347045d5f39f3508a29dda61e
2018-07-17 14:31:49 +01:00
Nikolay Nikolaev
587de9197e ovs-dpdk: add dpdk-bond-config config option
This allows more fine grained control over the bond mode
and LACP settings. Directly mapped to what OVS-DPDK configuration
exposes.

Change-Id: I1cca1043058f1ec99f194c1bdb611ebd603d646d
2018-06-21 12:38:02 +03:00
Nikolay Nikolaev
8225b4dca9 Add dpdk-bond-mappings configuration option
The current charm does not support creating and managing bonded network
interfaces. They are managed externaly. This is not possible when DPDK
is enabled. In this case OVS exposes the DPDK bond PMD which enslaves
the corresponding attached bond interfaces.

The new dpdk-bond-mappings configuration option allows such configuration
where mac:bond is specified. When the data-port configuration is processed
dpdk-bond-mappings are consulted to identify if the port belongs to a bond.
If this is true - then the bond is created with the mac designated interface
and the bond is added to the bridge. Subsequently more interfaces can be
added to the same bond.

Change-Id: I0224caaa1c2431c793c4f64caa7fc9e95b972fd7
2018-06-21 12:38:01 +03:00
Nikolay Nikolaev
862c362296 ovs-dpdk: support late initialization since ovs 2.6.0
OVS 2.6.0 introduces new mechanisms to configure the DPDK netdev
provider. It now relies on the database and allows dynamic runtime
configuration. Network interface binding is more fine grained by
specifying the NIC PCI address and not relying on special port naming
and indexing.

Here we introduce the support of post 2.6.0 OVS-DPDK and change the
relevant tests.

Change-Id: Ic0185097d65df04a2b566e16cb22bcbd088eed3e
2018-06-12 14:21:48 +03:00
James Page
f4ef607187 Fix misc unit test failures
A recent commit landed with failing unit tests, but due to
gate misconfiguration this was not picked up during pre-commit
testing.

Fixup offending code.

Change-Id: I20488efabe91b2423c85dd4e7474cbaf9a0a0261
2018-05-24 15:52:39 -07:00
Michael Skalka
07254a5c59 Enable IPFIX exporting for OVS bridges.
Adds a config option and calls to enable IPFIX exporting on all OVS
bridges created on a system by the OVS charm.

Closes-Bug: 1768016

Change-Id: Id2591ac5f39319d50ba235f6b9b5d493e7885d3a
2018-05-09 12:32:05 -04:00
James Page
fe9633856b Remove deploy from source support
Drop support for deployment from Git repositories, as deprecated
in the 17.02 charm release.  This feature is unmaintained and has
no known users.

Change-Id: Ib954ddd1fb63d409af77949d8e76a6d6da8f2cde
2018-01-10 12:01:23 +00:00
James Page
0e86a664be Update to run under Python 3
Refactoring of charm to support execution under Python 3.

Change-Id: I8d48614f38abf31159311d82b0cbb54e60f710c1
2017-11-20 08:54:31 -08:00
zhangyangyang
e4ccc0dbf1 change assert(Not)Equals to assert(Not)Equal
According to http://docs.python.org/2/library/unittest.html
assert(Not)Equals is a deprecated alias of assert(Not)Equal.

Change-Id: Ifa4bb3a204c7313ca8c24eb97956b150596ddf39
Closes-Bug: #1329757
2017-09-12 16:36:01 +08:00
Frode Nordahl
4ffbc2fe25 Fix handling of SR-IOV interface configuration
SR-IOV interfaces are currently only configured on charm
installation and not after seubsequent reboots.

The VFs need to be configured before the Neutron SR-IOV
agent is started. Charms should also really not be involved
in boot time system configuration. Due to these factors
this commit adds a init script and corrensponding systemd
unit file and upstart job to handle the boot-time configuration.

Keep configure_sriov function for runtime configuration. Add
warning about runtime configuration disrupting network service.

Add restart of Neutron SR-IOV agent after runtime configuration.

Cap value of sriov-numvfs at each interfaces sriov_totalvfs value.

Change-Id: I7bde7217bf027db09ded35a262c214ccb11d6d86
Closes-Bug: #1697572
2017-08-24 12:34:58 -07:00
Frode Nordahl
25450d27fa Handle neutron-sriov-agent on Kilo and Liberty
On Kilo and Liberty the agent is called 'neutron-plugin-sriov-agent'.

Add unit-test to verify package determination.
Add functional test to verify that configuration is written.

Change-Id: I8a40c12cbb7f6a692b19105d5c029fd7f2829504
Closes-Bug: #1696691
2017-06-13 16:59:37 +02:00
Jenkins
0fb6c60db4 Merge "Add support of a linuxbridge bridge in data-port config" 2017-05-01 00:57:28 +00:00
Alex Kavanagh
4c916a02ac Fix alphanumeric comparisons for openstack and ubuntu releases
- sync charmhelpers with fix-alpha helpers
- fix up code where the alpha comparisons are done
- fix tests which assumed mocks would just work on os_release()

Change-Id: Ifa495c37adeb24aa98e4e5e181b90cbbd5c0cddb
Related-Bug: #1659575
2017-04-03 17:59:54 +01:00
Matt Rae
f832f1073d Add support of a linuxbridge bridge in data-port config
When configuring data-port parameter with "ovs-bridge:linuxbridge"
a veth pair will be created to connect these two bridges. Name of
these virtual interfaces will be "veth-ovsbridge_name" and
"veth-linuxbridge_name".

Problem: When deploying neutron-openvswitch charm on a node contain
only one interface, we are not able to connect an ovs Bridge to
the physical interface because it is assigned to juju Bridge.

Change-Id: I5be72b9cc5948f5f791d522d1b46fd27e7303613
Closes-Bug:#1635067
2017-03-17 10:22:00 -07:00
James Page
790819c237 Update SR-IOV support for >= Mitaka
SR-IOV network for OpenStack release later than Mitaka requires the
use of the neutron-sriov-agent to support management of SR-IOV PF
and VF interface state by Neutron - said interfaces are still
consumed directly by nova-compute/libvirt via PCI device allocation
scheduling for instances.

Add new configuration options to the neutron-openvswitch charm to
support enablement of the SR-IOV agent; this could have been done
automatically from data presented from neutron-api, but its possible
that cloud deployments may only have subsets of compute nodes that
are SR-IOV enabled in terms of hardware.

Enabling this option ('enable-sriov') will install and configure
the neutron-sriov-agent; configuration of SR-IOV PF's are made
using the 'sriov-numvfs', which by default automatically configures
all SR-IOV devices on every machine to the maximum number of VF's
supported by the device.  This option can be used to configure
devices at an individual level as well.

Finally, neutron needs to understand what underlying provider
network each SR-IOV device maps to - this is configured using the
sriov-device-mappings configuration option.

Change-Id: Ie185fd347ddc1b11e9ed13cefaf44fb7c8546ab0
2017-02-07 14:55:32 +01:00
Liam Young
d57a6e9a4e Do not create MTU upstart scripts on xenial+
MTU scripts are no longer needed as MAAS 1.9 can set the mtu and
bring up the interfaces.

The charm has no systemd versions of the 'os-charm-phy-mic-mtu'
and 'ext-port' scripts either so for xenial to set mtu sizes
on physical nics use MAAS 1.9 and appropriate network config

Change-Id: I3aa4d2a80a08dd605d4ae08d53f35282017e1009
Partial-Bug: 1566786
2017-01-05 13:51:29 +00:00
Liam Young
4463c334ca Restart openvswitch-vswitch when requested
Restart requests can be sent by related charms. A request to restart
services did not previously restart openvswitch. This change adds the
ability to restart it.

Closes-Bug: 1628093
Change-Id: I0f57d84e2cdaa103c18a1cdacd996f9421fba46c
2016-09-27 15:20:04 +00:00
James Page
7e4bd68494 Add support for application version
Juju 2.0 provides support for display of the version of
an application deployed by a charm in juju status.

Insert the os_application_version_set function into the
existing assess_status function - this gets called after
all hook executions, and periodically after that, so any
changes in package versions due to normal system updates
will also be reflected in the status output.

This review also includes a resync of charm-helpers to
pickup hookenv and contrib.openstack support for this
feature.

Change-Id: Ia91a2de062fbc13fdb2b366217278bb96fc648fa
2016-09-20 12:32:57 +01:00
Corey Bryant
faaf51e7ba Add systemd init support for deploy from source
systemd is used instead of upstart by default since Ubuntu 15.10
(Wily).  This adds systemd init file support for nova services
that are deployed from source.

Change-Id: I7d031e86853a3fb8b91501dc6bbd7f5f1b67701d
2016-07-13 19:25:46 +00:00
James Page
ed83413331 Re-license charm as Apache-2.0
All contributions to this charm where made under Canonical
copyright; switch to Apache-2.0 license as agreed so we
can move forward with official project status.

Change-Id: I7bd44dc15ad951bf2536e5ee10de01ec592b8970
2016-07-01 17:36:39 +01:00
Corey Bryant
9ad5cf9637 Add defaults for openstack-origin-git config option
openstack-origin-git currently only supports YAML that specifies
the git repositories to deploy from.

This adds support for default openstack-origin-git values. The
default values supported are: icehouse, kilo, liberty, mitaka,
and master.  For example: openstack-origin-git=master.

Change-Id: I032cb58283d54a9ccfcc268a7fd70b460a03aa58
2016-06-20 09:42:16 -04:00
Alex Kavanagh
1dfe139242 Modify add_bridge_ports(...) to include port up and permisc code
This fixes bug#1570411 where the add_bridge_port(...) function was
modified to include a port type but missed off port up and promisc
features.

Change-Id: I2a304270be97ed1eae5a7ceeb5777514460d8b4f
Closes-Bug: #1570411
2016-04-15 12:18:13 +00:00
James Page
acd617f4ca Add support for DPDK userspace networking
Add full support for DPDK; this includes a number of configuration
options to allow the number of cores and memory allocated per
NUMA node to be changed.  By default, the first core and 1024MB of
RAM of each NUMA node will be configured for DPDK use.

When DPDK is enabled, OVS bridges are configured as datapath type
'netdev' rather than type 'system' to allow use of userspace
DPDK packet processing; Security groups are also disabled, as
iptables based rules cannot be applied against userspace sockets.

DPDK device binding is undertaken using /etc/dpdk/interfaces and
the dpdk init script provided as part of the DPDK package; device
resolution is determined using the data-port configuration option
using the <bridge:<mac address> format - MAC addresses are used
to resolve underlying PCI device names for binding with DPDK.

It's assumed that hugepage memory configuration is either done as
part of system boot as kernel command line options (set via MAAS)
or using the hugepages configuration option on the nova-compute
charm.

Change-Id: Ieb2ac522b07e495f1855e304d31eef59c316c0e4
2016-04-07 12:42:16 +01:00
Alex Kavanagh
a9b41dfbed Enhanced pause/resume for maintenance mode
Add in pause/resume feature for maintenance mode along with tests.
Sync charmhelpers with support for the maintenance mode feature.

Change-Id: I075459e56ce34e78f5206d116208165aa43aae21
2016-03-30 19:57:58 +00:00
James Page
fb8c6021b3 Deal with neutron changes forthcoming 2016-02-23 14:32:45 +00:00
Liam Young
bbecca866a Added workload status 2015-10-07 10:41:12 +00:00
James Page
8381190bc0 Tidy lint 2015-09-25 14:49:08 +01:00