If a user specifies a header in their request for metadata,
it could override what the proxy would have inserted on their
behalf. Make sure to remove any headers we don't want, and
override something that might be present in the request.
If the agent somehow gets a request with both headers it will
silently drop it.
Change-Id: Id6c103b7bcebe441c27c6049d349d84ba7fd15a6
Closes-bug: #1865036
(cherry picked from commit 5af046fd4e)
To fix bug 1722584 we inserted a checksum-fill rule for
metadata proxy replies. Recent kernels have disabled
this support for TCP because it was invalid, and
supposedly not doing anything, so let's get ahead of
things and remove the code.
Kernel mailing list discussion is at
https://lore.kernel.org/patchwork/patch/824819/
Partially reverts ed1c3b0217
Change-Id: Ib7cc8f82a91972f17987fb95130edc4069d9423f
Related-bug: #1722584
(cherry picked from commit b1b8a438fe)
Currently the metadata proxy binds to default 0.0.0.0, which does not
add any advantage (metadata requests are not sent to random IP
addresses), and may allow access to cloud information from
third parties.
This changes the generated configuration to bind to METADATA_DEFAULT_IP
address instead.
This is not enabled in other metadata proxy configuration (in the L3
agent), as this would require net.ipv4.ip_nonlocal_bind everywhere
(currently only enabled for DVR) or transparent mode in haproxy (which
requires net.ipv4.ip_nonlocal_bind anyway)
Changed set_ip_nonlocal_bind_for_namespace() to support setting the
value in both the given and root namespace correctly, since it was
only used from inside the neutron codebase according to codesearch.
Change-Id: I388391cf697dade1a163d15ab568b33134f7b2d9
Co-Authored-By: Andrey Arapov <andrey.arapov@nixaid.com>
Closes-Bug: #1745618
(cherry picked from commit 6124f60297)
This patch switches callbacks over to the payload object style events
[1] for ROUTER and ROUTER_GATEWAY BEFORE_DELETE based notifications. To
do so a DBEventPayload object is used with the publish() method to pass
along the related data.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: I3ce4475643f4f0afed01f2e9956b3bf84714e6f2
By adding a log-tag line to the haproxy config file that contains
the network or router id, we will be able to differentiate which
proxy is logging what. This should help with debugging.
Change-Id: I5bb57b7682c00645e20cce69847dbb3b72165aa8
Partial-bug: #1744359
Move the iptables metadata marking rule earlier in
router init, that way any stray metadata requests
that arrive before the filter metadata redirect rule is
installed will just be dropped. We do this irregardless
of whether we will be running the metadata proxy.
Partial-bug: #1735724
Change-Id: I8982523dbb94a7c5b8a4db88a196fabc4dd2873f
Sometimes a proxied metadata reply can be dropped by
the hypervisor because of an invalid checksum. Always
fill-in the checksum just like we do for DHCP replies.
Change-Id: I46987da3bf05577ff0a51a490f26cf2be3c3c266
Closes-bug: #1722584
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Without this commit, the run_as_root parameter is always True when
stopping a process, which leads to the usage of unnecessary sudo such as
in some functional tests, like the keepalived ones.
This commit fixes the aforemetioned problem by taking run_as_root into
account when stopping a process. However, run_as_root will still always
be True if the process is spawned in a netns.
Closes-Bug: #1491581
Change-Id: Ib40e1e3357b9a38e760f4e552bf615cdfd54ee5a
Signed-off-by: Hunt Xu <mhuntxu@gmail.com>
Instead, use is_debug_enabled public function from oslo.log. This
ensures we won't break if library owners for some reason decide to
rename or move the option.
The new function is available since 3.21.0. We depend on >= 3.22.0 so we
should be fine.
TrivialFix
Change-Id: Id00f9ba200173d2b33c1c926a971f0bb1c836706
For a HA router, when it's updated, the l3 agents which are standby
always call the after_router_added method, then duplicate metadata
rules are added to iptables table. Althrough these rules will not be
applied to system because of the _weed_out_duplicates method, they will
grow linearly with router update operations.
Because these metadata rules are added once router is added to the agent
and will not be cleaned until router is removed, calling the add_rule
method in after_router_updated is a waste.
This patch removes adding metadata rules in after_router_updated.
Change-Id: I6650f1071499ed6cabd936bb0fb36b32a4b60bca
Closes-Bug: #1658460
Due to the high memory footprint of current Python ns-metadata-proxy,
it has to be replaced with a lighter process to avoid OOM conditions in
large environments.
This patch spawns haproxy through a process monitor using a pidfile.
This allows tracking the process and respawn it if necessary as it was
done before. Also, it implements an upgrade path which consists of
detecting any running Python instance of ns-metadata-proxy and
replacing them by haproxy. Therefore, upgrades will take place by
simply restarting neutron-l3-agent and neutron-dhcp-agent.
According to /proc/<pid>/smaps, memory footprint goes down from ~50MB
to ~1.5MB.
Also, haproxy is added to bindep in order to ensure that it's installed.
UpgradeImpact
Depends-On: I36a5531cacc21c0d4bb7f20d4bec6da65d04c262
Depends-On: Ia37368a7ff38ea48c683a7bad76f87697e194b04
Closes-Bug: #1524916
Change-Id: I5a75cc582dca48defafb440207d10e2f7b4f218b
This comment should have been cleaned up as part of
Id0517758d06e036a36dc8b8772e41af55d986b4e but we missed it during
review. Now the comment only makes readers wonder how it's relevant to
METADATA_SERVICE_NAME constant that it seems to describe, though it
doesn't.
TrivialFix
Change-Id: Ibdd811f21c6a921e309a172e26c713c964bdffa5
This patch implements the callback handler for router update events;
This checks if the proxy process monitor is active, and if not, starts
the proxy.
This is particularly important if the metadata driver misses to receive
a create notification due to failures, which in turn generates an update
event because of a resync step.
Closes-bug: #1623732
Change-Id: I296a37daff1e5f018ae11eb8661c77ad346b8075
This fixes the iptables rules generated by the L3 agent
(SNAT, DNAT, set-mark and metadata), and the DHCP agent
(checksum-fill) to match the format that will be returned
by iptables-save to prevent excessive extra replacement
work done by the iptables manager.
It also fixes the iptables test that was not passing the
expected arguments (-p PROTO -m PROTO) for block rules.
A simple test was added to the L3 agent to ensure that the
rules have converged during the normal lifecycle tests.
Closes-Bug: #1566007
Change-Id: I5e8e27cdbf0d0448011881614671efe53bb1b6a1
Optimize the execution of router delete in the L3 agent
by not performing operations in the router namespace that
will be removed as the last step of the workflow.
Comparing the slopes of the least squares fit of delete
execution time shows that this optimization decreases that
slope by over half.
Change-Id: Ic425ab1f8c3afe882c9deaa72a814ac0d2053f3b
Signed-off-by: Ryan Moats <rmoats@us.ibm.com>
Closes-Bug: 1512450
This was a todo left over from refactoring. The one place where it
was used was in the metadata driver which can just as easily check
isinstance.
The is_ha property in the HA class only returned True if self.router
was True. The agent sets this on creation and so it will always be
True. Additionally, it didn't really make sense to me anyway given
the way this was being used in the metadata driver.
Change-Id: Idf91ee5f8b72ff9a1740dd60090327d07ee8e16e
Currently iptables rules set on L3 agent with metadata_proxy enabled
mark all packets coming from all interfaces including external interfaces.
This change updates PREROUTING rules from MANGLE table to mark packets
only from internal interfaces.
Change-Id: I01549df7b99be84cd46b6f97a5fd62aec1f43275
Closes-Bug: #1477553
Since a packet can only have one mark, and we will need to mark a
packet for multiple purposes, we need to use a coordinated bitmask for
the two cases of simple marking that we currently do in Neutron
leaving the other bits for address scopes.
DocImpact
Change-Id: Id0517758d06e036a36dc8b8772e41af55d986b4e
Partially-Implements: blueprint address-scopes
It's not always possible/convenient to get namespace name
when need to disable some process (like metadata process for stale
router, see related bug). Since namespace name is not required
for process manager to disable process we can remove this parameter
from disable_isolated_metadata_proxy()
Change-Id: I0e0da01d9640aa9920f41989804fc6f320c1c1eb
Related-Bug: #1455042
Currently metadata proxy cannot run with nobody user/group as metadata
proxy requires to connect to metadata_proxy_socket when queried.
This change allows to run metadata proxy with nobody user/group by
allowing to choose the metadata_proxy_socket mode with the new option
metadata_proxy_socket_mode (4 choices) in order to adapt socket
permissions to metadata proxy user/group.
This change refactors also where options are defined to enable
metadata_proxy_user/group options in the metadata agent.
In practice:
* if metadata_proxy_user is agent effective user or root, then:
* metadata proxy is allowed to use rootwrap (unsecure)
* set metadata_proxy_socket_mode = user (0o644)
* else if metadata_proxy_group is agent effective group, then:
* metadata proxy is not allowed to use rootwrap (secure)
* set metadata_proxy_socket_mode = group (0o664)
* set metadata_proxy_log_watch = false
* else:
* metadata proxy has lowest permissions (securest) but metadata proxy
socket can be opened by everyone
* set metadata_proxy_socket_mode = all (0o666)
* set metadata_proxy_log_watch = false
An alternative is to set metadata_proxy_socket_mode = deduce, in such
case metadata agent uses previous rules to choose the correct mode.
DocImpact
Closes-Bug: #1427228
Change-Id: I235a0cc4f0cbd55ae4ec1570daf2ebbb6a72441d
This commit completes the refactoring of the L3 agent callback mechanism.
The goal here is to also use the neutron/callbacks/ mechanism for L3 agent
notifications, instead of have two mechanisms.
[1] modified the L3 agent to send notifiactions for router create, udpate,
and delete events, using the neutron/callbacks/ mechanism.
[2] modified VPN to use this new mechanism, instead of the L3EventObservers
mechanism. Note:
[3] modified FW repo to no longer depended on the L3EventObserver and
related objects (it doesn't currently use the event notifications).
This commit removes the notifications for the L3EventObservers mechanism,
removed the related modules and tests, and adds in tests to verify that the
new notifications are called for the different events.
Once [1] and [2] are upstreamed, this commit can proceed.
Refs:
[1] https://review.openstack.org/#/c/164466/
[2] https://review.openstack.org/#/c/165226/
[3] https://review.openstack.org/#/c/167275/
Change-Id: I7c4b4ea5f9fb19abb812665cdae5fb70c84fe3ec
Depends-On: If5040a827a6903cc7cb5e59cdb7fb95f61b13d47
Closes-Bug: #1433552
The goal of this refactoring is to reduce duplication by
replacing the L3EventObservers mechanism (a specific
mechanism for L3 agent notifications), with the
CallbacksManager mechanism (a more general mechanism
currently in use), so that there is one method
used.
This is the first part of refactoring the L3 agent so that
it uses the new neutron.callbacks mechanism. To do this,
duplicate calls will be made for notifications related to
the router, only using the new callback mechanism.
This commit does two things. First, it puts in place the
notifiers for the new callback mechanism. Second, it updates
the metatdata proxy agent (which is in the same repo) to
use the new callback mechanism.
Later commits will update other repos from the old to new
callback mechanism, and to then remove the old callback
mechanism, once no longer used.
Change-Id: If134947957fd671aa99a0b2d2b37f7ec65e37766
Partial-Bug: #1433552
Currently metadata proxy cannot run with nobody user/group as
metadata proxy (as other services) uses WatchedFileHandler handler to
log to file which does not support permissions drop (the process must
be able to r/w after permissions drop to "watch" the file).
This change allows to enable/disable log watch in metadata proxies with
the new option metadata_proxy_log_watch. It should be disabled when
metadata_proxy_user/group is not allowed to read/write metadata proxy
log files. Option default value is deduced from metadata_proxy_user:
* True if metadata_proxy_user is agent effective user id/name,
* False otherwise.
When log watch is disabled and logrotate is enabled on metadata proxy
logging files, 'copytruncate' logrotate option must be used otherwise
metadata proxy logs will be lost after the first log rotation.
DocImpact
Change-Id: I40a7bd82a2c60d9198312fdb52e3010c60db3511
Partial-Bug: #1427228
Currently the metadata service can be requested on 169.254.169.254:80
from all interfaces including external interfaces. This change updates
PREROUTING rules to allow request on 169.254.169.254:80 only from
internal interfaces.
Change-Id: I44a9e03992f9e2a7bd4d798ae69d8aa7d75d3078
Closes-Bug: #1187102
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Refactor the ProcessMonitor API to reduce coupling to
ProcessManager, and stop replicating the whole API and
behavior of ProcessManager.
We introduced an abstract MonitoredProcess class to reduce
coupling, and allow other kinds of external processes
to be monitored too.
Partially Implements: blueprint agent-child-processes-status
Co-Authored-By: Mike Kolesnik <mkolesni@redhat.com>
Change-Id: I0da6071037f9728cc20403324e36c32116bcf00d
Iptables removes 0.0.0.0/0 as rule source (-s) because any packet
matches the filter. Based on the discussion in the patch [1],
references to 0.0.0.0/0 are removed in the current patch.
[1] https://review.openstack.org/#/c/160782/
Closes-Bug: #1428127
Change-Id: I8cd96438ef21edfd75483eec3ebfebcee24a8300
The modified unit test is currently the sole user of the
unmonitored _spawn_metadata_proxy. We must remove this usage so that
when the L3 agent manages the metadata proxy for HA routers,
the unmanaged methods may be removed.
Related-Bug: #1402010
Change-Id: Ib57e63dcadc378ac51702949ed24901d403002dd
Currently neutron-ns-metadata-proxy runs with root permissions when
namespaces are enabled on the dhcp agent because root permissions are
required to "enter" in the namespace. But neutron-ns-metadata-proxy
permissions should be reduced as much as possible because it is
reachable from vms.
This change allows to change neutron-ns-metadata-proxy permissions
after its startup through the 2 new options metadata_proxy_user and
metadata_proxy_group which allow to define user/group running metadata
proxy after its initialization. Their default values are
neutron-dhcp-agent effective user and group.
This change delegates metadata proxy management to metadata driver
methods in order to reuse the work already done on l3 agent side.
Permissions drop is done after metadata proxy daemon writes its
pid in its pidfile (it could be disallowed after permissions drop) and
after metadata proxy daemon binds its privileged server port (80).
Using nobody as metadata_proxy_user/group (more secure) is currently
not supported because:
* nobody has not the permission to connect the metadata socket,
* nobody has not the permission to log to file because neutron uses
WatchedFileHandler (which requires read/write permissions after
permissions drop).
This limitation will be addressed in a daughter change.
DocImpact
Closes-Bug: #1187107
Change-Id: I53e97254d560e608101010f67bd2dcdec81fb6a2
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
metadata service should be requested on 169.254.169.254:80 and router
namespace iptables rules redirect the request to the metadata-ns-proxy
on 127.0.0.1:$metadata_port. But currently the metadata-ns-proxy can be
requested directly on $router-ip:$metadata_port.
To avoid such behavior, this change marks packets redirection in mangle
table (PREROUTING), redirects (PREROUTING) them in nat table, accepts
them in filter table (INPUT) using the mark. Packets send to the
metadata proxy port without mark (so directly) are dropped. The
mark can be configured through the new option metadata_access_mark.
Remark: redirected packets are not local packets (in general), so
setting metadata proxy server host to 127.0.0.1 will disallow direct
queries but so redirected queries.
DocImpact
Partial-Bug: #1187102
Change-Id: I6a9bb12c8bf68c6fcf4e4060f8dfe44a309a41da
The ProcessMonitor class will watch over spawned external processes,
taking the administrator configured action in the case of any
of the external processes dying unexpectedly.
It covers both the neutron-ns-metadata-proxy for non-ha routers
and the IPv6 radvd external processes. Keepalived +
neutron-ns-metadata-proxy needs to be covered in a second follow up
patch when neutron-ns-metadata-proxy is handled by the l3-agent
(instead keepalived) in the ha-routers.
Implements: blueprint agent-child-processes-status
Change-Id: Id6cc4786d837b96c61429d51485bc86ae37872cb
This change moves metadata driver related config options to metadata
driver module to prepare the use of metadata driver method in the dhcp
agent (daughter change). The metadata_port option is not moved as the
dhcp agent uses a non-configurable port (80).
Change-Id: Ie45fdad86f33d35fca3096c4c52fae941a279e76
Partial-Bug: #1187107
2 iptables rules are defined to ensure the metadata proxy is reachable
from vms on 169.254.169.254:80:
* REDIRECT 169.254.169.254:80 packets to the router on port 9697
* ACCEPT traffic to 127.0.0.1 on port 9697
The REDIRECT rule replaces destination ip by:
* 127.0.0.1 if the packet is local,
* router ip (the one on the input interface, metadata proxy case).
So ACCEPT rule filter is not matched ... the metadata proxy is only
reachable because INPUT policy is ACCEPT.
This change removes the destination constraint in the ACCEPT rule.
Change-Id: Iea700bdd121bbc56a3489a63e2a5391867fad0d6
Closes-Bug: #1399462
This reverts commit 658dc9d30c.
The L3 agent metadata driver was added but then reverted
because it broke the functional job. The fix to the functional
job was merged, and this patch reverts the revert, thereby
adding the metadata driver back.
Change-Id: Ice7ae845d1d5063b116824f49961ff060a0b2baa
To-Do:
* vArmourL3NATAgent (Before this patch) overrides _spawn/destroy_metadata_proxy
with empty stubs. This behavior should be maintained so that the metadata proxy
is not created when using vArmourL3NATAgent. It also overrides _router_added
and _router_removed, causing L3 agent driver notifications to not send out.
Partially-implements: blueprint restructure-l3-agent
Change-Id: I36900b02bff34269f789956aa324379ff51eb81b
Brian Haley