Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
This change makes the metering agent retry creating the iptables
managers for each router and applying the metering rules.
This is needed in case the metering agent starts before some or all of
the namespaces are created.
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
Currently the metering iptables driver is using the old method
to load the interface driver. This patch allows it to load
interface driver via stevedore aliases.
The existing method is to meter outgoing traffic from the entire VM
to a specific "remote_ip_addr"(in case of outbound),
but the method I suggested can meter outgoing traffic
from a particular subnet to external network.
From a metering point of view,
it does not matter which address/CIDR is the destination for outbound.
But, it is important to know where(subnet or VM) the packet leaves from.
That way you can see how much VMs are using the external network.
For outbound metering, dst is 0.0.0.0/0 and src is network of VMs
and for inbound metering, src is 0.0.0.0/0 and dst is network of VMs.
When creating a meter-label-rule, it is correct
to specify src address for "remote_ip_prefix" option for outbound
and specify dst address for inbound.
If you are confused by the term "remote_ip_prefix",
you need to clarify this option.
Change network namespace add/delete/list code to use
pyroute2 library instead of calling /sbin/ip.
Also changed all in-tree callers to use the new calls.
Refactoring neutron agent linux and ovsdb config opts
to be in neutron/conf/agent so that all the config options
reside in a centralized location. This simplifies the
process of looking up the config opts and provides an easy
way to import.
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
If the metering-agent receives a label rule before it
has added the label, it will fail to update the iptables
rules as there are no existing chains.
When the action is "create", check if there is an existing
label, and create one and the corresponding iptables chains,
before trying to add the rule.
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
IPTables driver for metering was not handling the DVR router
namespaces properly for configuring the metering forward rules.
This patch addresses the issue by configuring the iptables
manager based on the availability of the namespace and selecting
the respective namespaces such as router namespace and snat
namespace with the right external device.
When syncing data from neutron server, metering-agent may receive
information about routers that are not hosted by the l3-agent on the
same host, because the server didn't filter them out. This could lead to
the following problems:
* metering-agent tries to setup iptables rules for a router that is not
on the host
* metering-agent tries to get get traffic counters for a router that
was once on the host but is already removed
* metering-agent not sets up iptables rules for a router that is
removed then added back to the host, because nothing about the
router is changed from metering-agent's perspective
This commit fixes the aforementioned problems by making metering-agent
only receive information about routers that are on the same host, and
update metering-agent's knowledge about which routers it should care.
However, there could still be problem if one removes then adds a router
back to the same l3-agent, or just sets the router's admin_state_up
property to False then True in a short time(shorter than the interval
between two syncs). Because the metering-agent sees nothing changed
while during the same time the router's namespace is removed and added
back on the host. Thus metering-agent will fail to get such router's
traffic counters. This commit also make iptables-driver to forget such
routers and leave the metering-agent to reconfigure them later.
Signed-off-by: Hunt Xu <email@example.com>
IPv6 utils is_enabled() doesn't actually determine if IPv6 is enabled on
the host. It checks if /proc/sys/net/ipv6/conf/default/disable_ipv6 is
present and is set to 0. This kernel configuration option controls if
the kernel will automatically assign IPv6 link-local addresses to newly
created network interfaces when their link state changes to up. The
existence of this /proc files does indicate that the Linux kernel has
the ipv6 module loaded or ipv6 was compiled in. Having this /proc file
set to zero does not indicate IPv6 is not available on the system, just
that newly created interfaces will inherit this configuration and will
not have IPv6 addresses bound to them unless the administrator changes
the interfaces specific /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
This check was added to Neutron so it could operate with distributions
which didn't load the ipv6 kernel module, preventing errors when
attempting to make IPv6 specific configurations in the iptables firewall
driver and the L3 agent. Removing it would break existing deployments.
Renaming this function to provide clarity for complex conditions tested
by this function. In fact it is a good security practice to set this
default disable_ipv6 option to 1, and explicitly enable IPv6 by setting
disable_ipv6=0 on individual interfaces which the administrator intends
to bind IPv6 addresses on. This establishes parity with IPv4 behavior
where interfaces are not active in an address family until the
administrator explicitly configures them to be active in that address
family. This practice does not currently work as expected with the
Neutron, since setting /proc/sys/net/ipv6/conf/default/disable_ipv6 to 1
unexpectedly disables creating IPv6 security group rules leaving
instances completely exposed via IPv6 regardless of security group
When update meter label or rule, iptables_manager will update iptables
rule in router's namespace. In order to, it will clean traffic counter
number collected in interval time, the other iptables always trashing
that will clean old iptalbes rule and generate new same significance
If router has no gateway port when metering-agent wants to add
a metering-label-rule method _process_metering_label_rules() fails
with error "cannot concatenate 'str' and 'NoneType' objects"
because there is no check that router has an external gateway port.
This patch adds this check and adds some unit test.
If state_less parameter is not specified then
neutron-postrouting-bottom rule goes up in POSTROUTING
chain, which causes premature NATing of traffic,
for ex. traffic between internal networks becomes NATed.
Co-Authored-By: Sergey Belous <firstname.lastname@example.org>
In the case of outbound traffic, set remote_ip to dst.
In the case of inbound traffic, set remote_ip to src.
DocImpact: Clarify remote_ip_prefix description of metering label rule in API site
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
The use_namespaces option has been defined as a workaround to kernels
not properly supporting namespaces. This limitation is behind us, it's
time to remove use_namespaces after its deprecation in Kilo in order to
simplify code and remove a poorly tested case (use_namespaces=False).
This change prepares for removal pullup_route method which was only
used when use_namespaces=False.
oslo.log now provides a logging helper that is similar to custom neutron
helper (actually, the helper in oslo.log started from neutron version).
Now switching to library implementation.
Now we send all labels and rules per rule create/delete
and rebuild whole iptables chains.
In this patch we send only affected rule and create/
delete only this rule from iptables.
The metering agent will fail if one of the iptables chains
is missing, which will cause errors extracting data from
all the other chains. Add a simple try/except to let the
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
This should remove the last bits of passing root_helper around
for execute(). Some tests store and pass the root_helper argument
outside of this, and have been left alone.
Partially-Implements: blueprint rootwrap-daemon-mode
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
This change enforces log hints use and removes debug level log
translation, modifications are validated through a hacking rule.
Validate that hacking rules apply to directories:
The existing code incorrectly used the stale value from a previous list
comprehension - and deleted the chains for the wrong router :(
(Found via pylint)
Also: change to using a set() rather than a list(), since it is used for
repeated membership tests.
Also: refactor test cases to remove test case duplication.
On systems where ipv6 module is not loaded in kernel we need to avoid
usage of ip6tables. This patch reads
/proc/sys/net/ipv6/conf/default/disable_ipv6 file and if ipv6 is
disabled then ip6tables are not used in IptablesManager
While adding iptables rule, cidr is added as destination for both ingress
and egress directions. Modified code to add -s for egress and -d for ingress.
Some config options(interface_driver, use_namespaces) are defined
multiple times in ad-hoc way. It causes DuplicateOptError exception
when using those module at the same time. Right now the exception is
avoided in ad-hoc way by each executable. Those duplicated
definitions should be consolidated and treated in uniformed way.
This is the blocker for blueprint: l3-agent-consolidation
conflicts with neutron.service one. Since there is no way to fix it
without changing existing behavior/default value, it is untouched for now.
Using tools/check_i18n.py to scan source directory, and fix most of
- Message internationalization
- First letter must be capital
- Using comma instead of percent in LOG.xxx
Note: all extension's description are not touched in this patch,
can be fixed after discussing.
Note: all nicira/check_nvp_config.py print messages are not fixed.