Ports with device_owner like:
* floating_ip,
* DHCP,
* some types of router ports, like: HA interface interface,
* distributed ports,
don't need to be configured in the dnsmasq file.
So there is no need to reload dnsmasq every time when such port is
added/updated to the network.
This patch adds skip in such case which should improve load on the
Neutron DHCP agent.
Closes-Bug: #1913269
Change-Id: I63221507713b941c261cdf88781133149da8ab8d
The main idea of the commit is to fix code
according with the latest oslo.i18n requirements
https://docs.openstack.org/oslo.i18n/latest/
1. removed log translation if log is not seen by users
in raised exception or api call response.
2. keep translated log if it's used in raised exception.
3. removed log message 'Error while reading %s'
which was "dead" (unused) code in the function
"_get_value_from_conf_file"
of module "agent/linux/dhcp.py".
Partial-Bug: 1600788
Change-Id: Ifb5455336b06c2c87a930b816c90b4a766856b1e
"@abc.abstractproperty" is deprecated since 3.3. Now it's possible
to use "@property" on top of "@abstractmethod".
Change-Id: I0cca37b626a94a05fb983a8528c22a660e89e673
Passing --conf-file= with no value has no effect on the dnsmasq
process. Intended effect here is for the default system dnsmasq.conf
file not to be read and included in configuring the process. For
that to happen some value has to be passed to --conf-file. Passing
/dev/null will invoke the desired outcome to skip the system
default conf file.
Closes-Bug: #1896945
Change-Id: I22570a44f84d14a792633747c04d7426ab231009
Send IPv6 metadata traffic (dst=fe80::a9fe:a9fe) to the metadata-agent.
When running on IPv6 enabled system bind haproxy (i.e. the
metadata-proxy) to 169.254.169.254 and to fe80::a9fe:a9fe also.
We do not introduce new config options. The usual config options
(enable_isolated_metadata, force_metadata, enable_metadata_proxy)
now control the metadata service over both IPv4 and IPv6.
This change series only affects the guests' access to the metadata
service (over tenant networks). They change nothing about how the
metadata-agent talks to Nova's metadata service.
Metadata access over IPv6 is supposed to work both on dual-stack and
v6-only networks.
In order to enable the metadata service on pre-existing isolated
networks during an upgrade, this change makes each dhcp-agent restart
trigger a quick restart of dhcp-agent-controlled metadata-proxies,
so they can pick up their new config making them also bind to
fe80::a9fe:a9fe.
Change-Id: If35f00d1fc9e4ab7e232660362410ce7320c45ba
Partial-Bug: #1460177
This pep8 error ocurrs sporadically, as reported in related bug.
The creation of a new DictModel class empty object during the
deepcopy process only needs the class type only.
Change-Id: Iba4f2ea700f01fd153104741614eec4855d0f387
Closes-Bug: #1893316
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
[0] https://review.opendev.org/#/c/738163/
[1] https://github.com/PyCQA/pylint/issues/2737
Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
Version 2.4.0 of neutron-lib has the DHCP port numbers
correct, so start using them.
Also updated other code in linux/dhcp.py to use the
constants as well, instead of re-defining them.
Closes-bug: #1882588
Change-Id: I5dc1d8e7bcc94efd1fab68d980d60e3130d5e5bc
There are places where we need to use a metadata address
in different forms:
169.254.169.254
- when binding to an address, used with a port
169.254.169.254/32
- when configuring an address on an interface
- when adding a route
169.254.0.0/16
- when checking if a metadata subnet is present
We were not always using them correctly in either the
DHCP or OVN code, try and correct the usage. This will
make it easier to update the code when adding support
for metadata over IPv6.
Change-Id: I1780aa99204cc24e668d9798f4a5111eae83ecdb
The method "_read_leases_file_leases" is not called with the
parameter "ip_version". This parameter can be removed from the
method signature and the related code.
Trivial-Fix
Change-Id: I3ba720243ae4c405c10895d423e8a014201f4067
This patch fixes:
- The IPv6 tag added in the "host" file if is supported in
dnsmasq. That shifts all other parameters in the register.
- IPv6 registers can have more than one IP address; in this
case, the method "_read_hosts_file_leases" should return a
tuple per IP address.
Change-Id: I4d0bc1eb9448366d8f1b2dacc9c5c2e4e6958253
Closes-Bug: #1884105
With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
A new pep8 style library must have been released which
is causing some new errors, E471 among them. Clean-up
on aisle 8.
Change-Id: I153abada74e8c522fe9866a239a36dbb8365a29e
In dnsmasq 2.81 there is a regression (see [1] for details).
Prior versions of dnsmasq would select a host record where:
a) no address is present in the host record.
b) an address matching address family of the client request
is present in the host record.
dnsmasq 2.81 will also use a host record where a only an address
not matching the address family of the client request is present.
The same issue is also backported to the dnsmasq-2.79-11.el8.x86_64
which is e.g. in RHEL 8.2 and Centos 8.
dnsmasq version 2.81 also adds support for using tag's on host
records. When a dhcpv6 request is received, dnsmasq automatically
sets the tag 'dhcpv6'.
This change adds a runtime check, testing for dnsmasq host entry
tag support. And adds 'tag:dhcpv6' to all IPv6 host records when
dnsmasq supports this.
Adding the tag makes dnsmasq prefer the tagged host for dhcpv6
requests, i.e it's a workaround fix for the regression issue.
[1] http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2020q2/014051.html
Closes-Bug: #1876094
Change-Id: Ie654c84137914226bdc3e31e16219345c2efaac9
Adds a new bool option dnsmasq_enable_addr6_list, when
enabled configuration for dnsmasq will be created with a
single dhcp-host entry specifying a list of ip addresses
allocated for a port.
Previously the dnsmasq dhcp-agent driver would write a
separate dhcp-host entry for each fixed-ip of a port in
the dnsmasq hosts file. The result of the previous
behaviour is that dnsmasq will only use one of the config
entries, i.e the first one matching the mac identifier.
The trade-off is that only a single dns_assignment will
be used for IPv6 addresses within the same subnet. (But
in practice, this was always the case since only the
first config entry would be used by dnsmasq.)
Why is this neccecary:
This is done to enable ironic provisioning over IPv6
using DHCPv6-stateful. For background info, please
read dnsmasq-discuss thread:
http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2020q1/thread.html#13671
Closes-Bug: #1861032
Change-Id: I833840e7daed2efa7efaece27cfd1ba28e0feb90
"DhcpLocalProcess._enable", instead of calling
"DhcpLocalProcess.restart", should only stop the server and continue
the method. This will avoid getting into a second active wait context.
Change-Id: Id321430c344723363786ab48584b1b48ba69f679
Closes-Bug: #1854940
The new hacking release found some pep8 errors, fix them.
Unfortunately, moving to 2.0.0 pulled a thread with other
dependencies causing issues, so for now pin it below
1.2.0 to get the gate functioning again.
Change-Id: Ic8ee670e03dde0669231477a4209b01eb8fba7fd
In the reported bug, a regression was introduced in [1] when limiting
the time to have a "dnsmasq" process enabled. It has been reported, as
documented in the bug, that in older versions (Queens), using Python 2
[2] and older versions of "ip_lib" (not implementing most of the
commands using Pyroute2), that the time needed to spawn a "dnsmasq"
process exceeds the default 60 seconds defined in
"common_utils.wait_until_true".
This patch increases this time by a reasonable 300 seconds.
[1] https://review.opendev.org/#/c/643732
[2] https://bugs.python.org/issue35757
Change-Id: I2d8693145da72825876b951f2d10afe9ca28ff6e
Closes-Bug: #1849676
After spawning the "dnsmasq" process in the method
"Dnsmasq._spawn_or_reload_process", we need to check that the "dnsmasq"
process is running and could be detected by the ProcessManager instance
controlling it.
ProcessManager determines if a process is "active":
- If the network ID is in the cmdline used to execute the process.
- If the process is detected by psutil.Process(pid), returning the
cmdline needed in the first condition.
- If the PID file exists; this is written by the dnsmasq process
once is started and is needed in the second condition.
To make this feature available for any other process using
ProcessManager, the implementation is done in this class.
Change-Id: I51dc9d342c613afcbcfdc50a1d2811502748f170
Closes-Bug: #1849502
In some cases (I don't know exactly how) it may happend that
when new subnet, e.g. IPv6 is added to the network, subnets
can change their order based on uuid.
As before this patch we were using in dnsmasq options tags like
"tagN" for subnets (where N was just number based on position of
the subnet in the sorted list) it could happend sometimes that
dnsmasq ended up with mismatch of tags configured in "dhcp-range"
cmd option and set in "opts" file. That caused problem with serving
proper DHCP options to the vms.
This patch fixes this issue by using tags with format:
"subnet-<uuid>" where uuid is id of the subnet. That was it's not
based on order of subnets in the list and will always match with tag
configured in opts file for specific subnet.
As we was currently using port id as tag for "per port" DHCP options,
this patch changes that to use tags like "port-<uuid>" to make it
consistent with options configured "per subnet" and to make it easier
to debug from where each option comes.
Change-Id: Idaea33d62fa31edd7149ec916ec314438375724a
Partial-Bug: #1848738
Seems that is_enabled_and_bind_by_default() from
neutron.common.ipv6_utils was copied directly into
oslo_utils.netutils, so start using it instead.
Trivialfix
Change-Id: I00fa441e7a20fcd1115485bb8ab75750e6a8cf07
When setting up the DHCP agent of a network, the DHCP namespace external
port is configured. If this port already exists and the fixed IP
addresses are correctly configured (in the DHCP subnets range), the port
is used as is.
Sometimes, because of 1627480 or 1841636, the port information is not
correctly retrieved. This patch does not solve it but mitigates the
process of resynchronizing the network DHCP. If the stored DHCP port
does not have the correct information, the agent calls the RPC plugin to
retrieve from the server the DHCP port updated information, including
the fixed IP address and the subnets.
Change-Id: Iff40e7bba645ee12c2001d7ce735a36e0ddc81e9
Related-Bug: #1627480
Related-Bug: #1841636
In order to support out of tree interface drivers it is required
to pass a callback to allow the drivers to query information about
the network.
- Allow passing **kwargs to interface drivers
- Pass get_networks() as `get_networks_cb` kw arg
`get_networks_cb` has the same API as
`neutron.neutron_plugin_base_v2.NeutronPluginBaseV2.get_networks()`
minus the the request context which will be embeded in the callback
itself.
The out of tree interface drivers in question are:
MultiInterfaceDriver - a per-physnet interface driver that delegates
operations on a per-physnet basis.
IPoIBInterfaceDriver - an interface driver for IPoIB (IP over Infiniband)
networks.
Those drivers are a part of networking-mlnx[1], Their implementation
is vendor agnostic so they can later be moved to a more common place
if desired.
[1] https://github.com/openstack/networking-mlnx
Change-Id: I74d9f449fb24f64548b0f6db4d5562f7447efb25
Closes-Bug: #1834176
This patch adds possibility to configure kill hooks used to kill
external processes, like dnsmasq or keepalived.
Change-Id: I29dfbedfb7167982323dcff1c4554ee780cc48db
Closes-Bug: #1825943
The dns_domain attribute of a network is intended for use
by neutron when creating DNS records in an external DNS
system such as Designate.
By using the networks dns_domain, the configured search
path on booted instances mismatches with the generated
dns assignments for instance ports in the hosts file
for dnsmasq which creates a mismatched forward/reverse
lookup behaviour.
This reverts commit 137a6d6105
and commit 7fdd6adc7a.
Closes-Bug: 1826419
Depends-On: I145144c042b100f7e12a02a8ac7e0fbbe41e984d
Change-Id: I5ff03b5ad8af432a9f7919ef953d7d8c434b93bd
https://review.opendev.org/#/c/236983/https://review.opendev.org/#/c/606383/
The above patchs that resolve the race condition of DHCP agent will
result in neutron-server raise DhcpPortInUse ERROR log. And, the
second patch may result in old dhcp agent create a redundant port.
Closes-Bug: #1829332
Change-Id: If7a7ac2f88ce5b0e799c1104c936735a6cc860aa
Dnsmasq emits a warning when started in most neutron deployments:
dnsmasq[27287]: LOUD WARNING: use --bind-dynamic rather than
--bind-interfaces to avoid DNS amplification attacks via
these interface(s)
Since option --bind-dynamic is available since dnsmasq 2.63
(https://github.com/liquidm/dnsmasq/blob/master/FAQ#L239) and
we require 2.67, change to use this option instead.
Change-Id: Id7971bd99b04aca38180ff109f542422b1a925d5
Closes-bug: #1828473
The dhcp-agent is initializing the iptables 'nat' table even
though it is never inserting any rules there besides the
ones being done at init time. Since this table is really
intended for the l3-agent, add an argument so we can control
the initialization.
Change-Id: Iebda49e7da99bd3bc8c985132516ae5edafdfe20
Rationale: if there are Subnet1 with Host1 and Subnet2 with Host2
and Host2 serves Network2 behind it, in order to reach Network2 it is
required to add Network2 to Subnet1's host_routes with Host2 as a
nexthop. In this case Neutron will offer in Subnet1 the following set
of RFC3442 routes:
Network2 -> Host2
Subnet2_cidr -> 0.0.0.0
which will lead to fail installing Network2 route on the Host1 since
at the moment of Network2's appearance there is no information about
Subnet2 route.
The proposed patch orders routes in such a way that 'connected' routes
(0.0.0.0) will be placed first and, thus, all subsequent routes will
be installed succesfully.
Change-Id: Ice70eb2090e1e99c72ae51601fefd7ec5cd8867a
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Sometimes, during restart of dnsmasq process it may happend that
after process is killed, start attempt is made too fast, before
old process really unbind from IP address on which it was listening.
That causes an issue with starting dnsmasq process again.
In patch [1] disable() method was changed that it can wait
until process is really not active (no pid for it) but that didn't
solve the problem with starting a new dnsmasq process completely and
sometimes it still happens, at least in functional tests.
So now, enable() method is changed so that it will try to enable
dnsmasq process for 1 minute, until it will really be spawned properly.
[1] https://review.openstack.org/#/c/634390/
Change-Id: I18d73b787fa3ab8803e12d5e5eb2bb7109205aba
Closes-Bug: #1811126
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Consume the rehomed constant. Remove the constant from neutron.
Change-Id: Ia5d6ec8b66344c0c0c2d1588f8c1215c6c2b1cbe
Depends-On: https://review.openstack.org/631795
Related-Bug: #1811639
If no override for dns_domain has been added by the user on the
tentant network, the default dns_domain should be used. Prior to
this fix, dns_domain got set to '' instead.
Closes-bug: #1774710
Change-Id: If206b943703eb638f7b22e59791ed8876f46f556
Instead of instantiating an IPDevice object just to get
the list of IPs, call get_devices_with_ip() instead since
that's what it's doing anyways.
Trivialfix
Change-Id: I5055d24a40d45f3f3b13b05249d353ea67acf4d5
Dnsmasq driver used by dhcp agent has restart() method which is
calling disable() and then enable() dnsmasq process again.
What can be observed in functional tests from time to time it may
happen that start dnsmasq process will be called before old process
is really down. That leads to error that IP address to which
dnsmasq wants to bind is already in use and it fails to start.
This patch adds possibility to call disable() method with block flag
set to True. In such case driver will ensure in disable() method that
process is really not active.
This blocking disable() is used in restart() method now.
Change-Id: I419a451633badbc3d32edcee1945fca3e3d9f6be
Closes-Bug: #1811126
The DHCP agent has a really strict enforcement of client ID, which
is part of the DHCP extra options. If a VM advertises a client ID,
DHCP agent will automatically release it's lease whenever *any* other
port is updated/deleted, even if no client ID is set on the port,
because it thinks the client ID has changed.
When reload_allocations() is called, the DHCP agent parses the leases
and hosts files, and gets the list of all the ports in the network from the
DB, computing 3 different sets. The set from the leases file (v4_leases)
could have a client ID, but the set from the port DB and hosts file will
have None.
As a result, the set subtraction does not filter out the entry,
and all ports that have an active lease with a client ID are released.
The Client ID should only be enforced and leases released
if it's actually set in the port DB's DHCP extra Opts.
In that case it means someone knows what they are doing,
and we want to check for a mismatch. If the client ID on a port is
empty, it should not be treated like an unused lease.
We can't expect end users that just create VMs with auto created ports
to know/care about DHCP client IDs, then manually update ports or
change app templates.
In some cases, like Windows VMs, the client ID is advertised as the MAC by default.
In fact, there is a Windows bug which prevents you from even turning this off:
https://support.microsoft.com/en-us/help/3004537/dhcp-client-always-includes-option-61-in-the-dhcp-request-in-windows-8
Linux VMs don't have this on by default, but it may be enabled
in some templates unknown to users.
Change-Id: I8021f740bd78e654915337bd3287b45b2c422e95
Closes-Bug: #1806770
Currently the metadata proxy binds to default 0.0.0.0, which does not
add any advantage (metadata requests are not sent to random IP
addresses), and may allow access to cloud information from
third parties.
This changes the generated configuration to bind to METADATA_DEFAULT_IP
address instead.
This is not enabled in other metadata proxy configuration (in the L3
agent), as this would require net.ipv4.ip_nonlocal_bind everywhere
(currently only enabled for DVR) or transparent mode in haproxy (which
requires net.ipv4.ip_nonlocal_bind anyway)
Changed set_ip_nonlocal_bind_for_namespace() to support setting the
value in both the given and root namespace correctly, since it was
only used from inside the neutron codebase according to codesearch.
Change-Id: I388391cf697dade1a163d15ab568b33134f7b2d9
Co-Authored-By: Andrey Arapov <andrey.arapov@nixaid.com>
Closes-Bug: #1745618
IPv6 address format in dnsmasq leases file is incorrect (correct format
is described in bug description). This bad formatting generates the
following error when initializing dnsmasq:
dnsmasq[20603]: failed to parse lease database, invalid line: \
1547121093 fa:16:3e:a0:3a:9a [fd5b:1fd5:8295:5339::43] * ...
This patch removes the IPv6 addresses from the leases file, as proposed
in the bug, because the DHCP agent does not have the IAID (identity
association identifier) of each IPv6 address assigned.
In case of agent restart, dnsmasq won't have any IPv6 address in the
leases file, but the hosts file and the additional hosts file will
contain all MAC/IPv6 previous assignations. When the IPv6 client sends
a DHCPDISCOVER, dnsmasq will offer the same IPv6 address to this client.
At the same time, the client will request to the server the same address:
DHCPDISCOVER(tap2c14823a-e6) fa:16:3e:54:c6:8e
DHCPOFFER(tap2c14823a-e6) fd5b:1fd5:8295:5339::43 fa:16:3e:54:c6:8e
DHCPREQUEST(tap2c14823a-e6) fd5b:1fd5:8295:5339::43 fa:16:3e:54:c6:8e
DHCPACK(tap2c14823a-e6) fd5b:1fd5:8295:5339::43 fa:16:3e:54:c6:8e \
host-fd5b-1fd5-8295-5339--43
Once dnsmasq updates the leases database, rewrites the leases file with the
new IPv6 address (including the IAID) and the server DUID (if not present).
Change-Id: Ib1b2f284ab81f1c4af7b08b5257b45a3f6e79c3e
Closes-Bug: #1722126
Bug #1244589 re-appeared for IPv6.
This change adds an ip6tables rule to fix the checksum of DHCPv6
response packets. Those checksums were left unfilled by virtio (as a
hypervisor internal optimization), but some picky dhcp clients (AFAIU
particularly ISC dhclient) try verifying the checksums, so they fail
to acquire an address if the checksums are left incorrect.
Change-Id: I4a045e0dcfcbd3c7959a78f1460d5bf7da0252ff
Closes-Bug: #1811639
Related-Bug: #1244589
DHCPv6 has different mode of operation and the request class 175
option no longer works for iPXE to allow configurations to be set
that delineate the address requestor as an iPXE client or a ROM
client seeking a bootloader over via DHCPv6.
In order to support iPXE, ironic needs to be able to match on to
the user class that is transmitted in the DHCPv6 requests.
From there, dnsmasq can choose the correct reply and a node can
boot up to be deployed by ironic.
Change-Id: I6328297deaabdc94b6356930dd978b02f432799c
Story: 2004502
Task: 28243
In attempting to fix option passing for Ironic, I discovered that
I would end up with option6:option6:59 if I passed "option6:59" or
I would end up with just "59".
According to the dnsmasq man page, "option6" labeling is mandatory.
That being said, I'll remove the transmission of "option6" from
ironic, but without fixing this issue IPv6 support for booting
baremetal nodes is broken.
Change-Id: I87c15908087111367358043f7a63dd02dd9d16ac
Story: 2004501
Task: 28219
Currently any dhcp agent instance will work as an open resolver. For
deployments using publicly routed addresses for tenant networks, this
allows the agent being abused in dDoS attacks, see [1].
By setting the `--local-service` option dnsmasq will filter DNS queries
and reply only to queries from directly attached networks.
[1] https://bugs.launchpad.net/neutron/+bug/1501206
Closes-Bug: 1501206
Change-Id: I76d810aad2ce0f15a88bd798963012fa0efca74e
Note: this is a squash of two changes since they are
dependent on each other, and are currently blocking
the gate queue.
Sometimes cleanup methods are failing in the check and
gate queues trying to delete non-existing namespaces.
Since they could have been deleted asynchronously, don't
raise if the failure is "No such file or directory" since
the system is in the intended state.
Cleaned-up the DHCP agent to longer check for existence
first, and the tests to longer mock-out the namespace
exists check.
Fix test_legacy_router_lifecycle failures
Multi-path routes returned via the pyroute2 library have
their outgoing interfaces in the 'multipath' dictionary
element, not in the route dictionary. In that case return
all the multipath routes correctly.
Change-Id: I5415cb3a88ff2640a19598a1fcb2278388815343
Closes-bug: #1795482
Closes-bug: #1795548
Slawek Kaplonski
Mitya_Eremeev
Rodolfo Alonso Hernandez
Dan Radez
Bence Romsics
Bernard Cafarelli
Brian Haley
Harald Jensås
Adrian Chiris
James Page
Yang JianFeng
Volodymyr Litovka
Boden R
Bence Romsics
Jakob Englisch
Arjun Baindur
Julia Kreger
Jens Harbott