Move the iptables metadata marking rule earlier in
router init, that way any stray metadata requests
that arrive before the filter metadata redirect rule is
installed will just be dropped. We do this irregardless
of whether we will be running the metadata proxy.
Partial-bug: #1735724
Change-Id: I8982523dbb94a7c5b8a4db88a196fabc4dd2873f
(cherry picked from commit 6941977827)
By adding a log-tag line to the haproxy config file that contains
the network or router id, we will be able to differentiate which
proxy is logging what. This should help with debugging.
Change-Id: I5bb57b7682c00645e20cce69847dbb3b72165aa8
Partial-bug: #1744359
(cherry picked from commit 703ff85b82)
Sometimes a proxied metadata reply can be dropped by
the hypervisor because of an invalid checksum. Always
fill-in the checksum just like we do for DHCP replies.
Change-Id: I46987da3bf05577ff0a51a490f26cf2be3c3c266
Closes-bug: #1722584
(cherry picked from commit ed1c3b0217)
Without this commit, the run_as_root parameter is always True when
stopping a process, which leads to the usage of unnecessary sudo such as
in some functional tests, like the keepalived ones.
This commit fixes the aforemetioned problem by taking run_as_root into
account when stopping a process. However, run_as_root will still always
be True if the process is spawned in a netns.
Closes-Bug: #1491581
Change-Id: Ib40e1e3357b9a38e760f4e552bf615cdfd54ee5a
Signed-off-by: Hunt Xu <mhuntxu@gmail.com>
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
Partial-Bug: #1563069
Change-Id: Iebac0cdd3bcfd0135349128921b7ad7a1a939ab8
Needed-By: Ib676003bbe909b5a9013a3178b12dbe291d936af
For a HA router, when it's updated, the l3 agents which are standby
always call the after_router_added method, then duplicate metadata
rules are added to iptables table. Althrough these rules will not be
applied to system because of the _weed_out_duplicates method, they will
grow linearly with router update operations.
Because these metadata rules are added once router is added to the agent
and will not be cleaned until router is removed, calling the add_rule
method in after_router_updated is a waste.
This patch removes adding metadata rules in after_router_updated.
Change-Id: I6650f1071499ed6cabd936bb0fb36b32a4b60bca
Closes-Bug: #1658460
Due to the high memory footprint of current Python ns-metadata-proxy,
it has to be replaced with a lighter process to avoid OOM conditions in
large environments.
This patch spawns haproxy through a process monitor using a pidfile.
This allows tracking the process and respawn it if necessary as it was
done before. Also, it implements an upgrade path which consists of
detecting any running Python instance of ns-metadata-proxy and
replacing them by haproxy. Therefore, upgrades will take place by
simply restarting neutron-l3-agent and neutron-dhcp-agent.
According to /proc/<pid>/smaps, memory footprint goes down from ~50MB
to ~1.5MB.
Also, haproxy is added to bindep in order to ensure that it's installed.
UpgradeImpact
Depends-On: I36a5531cacc21c0d4bb7f20d4bec6da65d04c262
Depends-On: Ia37368a7ff38ea48c683a7bad76f87697e194b04
Closes-Bug: #1524916
Change-Id: I5a75cc582dca48defafb440207d10e2f7b4f218b
nova_metadata_ip option name is a bit confusing as it accepts an ip but
also a dns name (which is required when https protocol is used).
This change deprecates nova_metadata_ip option in favor of
nova_metadata_host option and updates option help to highlight that we
can use an ip or a dns name.
DocImpact
Closes-Bug: #1585699
Change-Id: Ia6c2471c7b4f3e924941222133edbb90151757a5
agent object is a member of some sub classes of RouterInfo such as
HaRouter. This changeset makes it a member of the RouterInfo class
itself.
Prior to the change, the agent object has been passed in to some
methods of RouterInfo that requires it to access the agent object's
member information. The bugs in concern requires calling the PD object
that is a member of the agent object to get IPs that need to be
preserved in the gateway port. Without this change, signatures of the
methods external_gateway_added() and external_gateway_updated() have
to be modified to pass in the agent object. And any subclass of
RouterInfo that overwrites or uses the methods must be changed as
well. It doesn't seem to make sense considering the subclass such as
HaRouter has the agent object as one of its members already.
The changeset fixes the bugs by preserving the LLAs for prefix
delegation when the gateway port is being updated.
Closes-Bug: #1639042
Closes-Bug: #1640271
Change-Id: I61c6128ed1973deb8440c54234e77a66987d7e28
Refactoring neutron agent metadata config opts to be in
neutron/conf/agent/metadata so that all the configurations options
reside in a centralized location. This simplifies the process of looking
up the config opts and provides an easy way to import.
Change-Id: I8bae1facc58a4f9e21196f625478532403651545
Partial-Bug: #1563069
Refactoring l3 ha agent options to be in neutron/conf/agent/l3.
This would allow centralization of all configuration options and
provides an easy way to import.
Partial-Bug: #1563069
Change-Id: I2d6bd6beb0d1658baf88c49b954d2db3136e0c8d
This patch implements the callback handler for router update events;
This checks if the proxy process monitor is active, and if not, starts
the proxy.
This is particularly important if the metadata driver misses to receive
a create notification due to failures, which in turn generates an update
event because of a resync step.
Closes-bug: #1623732
Change-Id: I296a37daff1e5f018ae11eb8661c77ad346b8075
Refactoring neutron configuration options for l3 agent to be in
neutron/conf/agent/l3. This would allow centralization of all
configuration options in neutron/conf and provide an easy way to import.
Change-Id: Ie7533ea55eaa4d0f2c1919131a75f56e027c4d6e
Partial-Bug: #1563069
We start state reporting thread before forking child processes. This
leads to possibility of fork in the middle of reporting process. This
can cause child processes to start with corrupted global state. In [0]
it represented itself as metadata agent not being able to make RPC call
because of lock acquired in main process before fork and released after
fork. See bug description for details.
[0] https://review.openstack.org/312393
Partial-Bug: 1594439
Change-Id: Id5079a296bc402c6f5b6cdb2df72811eab5bc6ed
When the new (oslo.cache) way of configuring the cache is used, cache is
never hit, because self._cache.get() consistently raises exceptions:
TypeError: 'sha1() argument 1 must be string or buffer, not tuple'
It occurs because the key passed into the oslo.cache region does not
conform to oslo.cache requirements. The library enforces the key to be
compatible with sha1_mangle_key() function:
http://git.openstack.org/cgit/openstack/oslo.cache/tree/oslo_cache/core.py?id=8b8a718507b30a4a2fd36e6c14d1071bd6cca878#n140
With this patch, we transform the key to a string, to conform to the
requirements.
The bug sneaked into the tree unnoticed because of two reasons:
- there were no unit tests to validate the new way of cache
configuration.
- the 'legacy' code path was configuring the cache in a slightly
different way, omitting some oslo.cache code.
For the former, new unit tests were introduced that cover the cache on
par with the legacy mode.
For the latter, the legacy code path was modified to rely on the same
configuration path as for the new way.
Closes-Bug: #1593342
Change-Id: I2724aa21f66f0fb69147407bfcf3184585d7d5cd
Oslo incubator is about to stop for cache module. We could use
oslo.cache instead. The legacy memory backend is replaced by
oslo_cache.dict.
Closes-Bug: #1517883
Change-Id: I108242ca9f27c9ec47959ce7615bc7d84cae014b
The option was deprecated a long time ago, and will be removed in one of
the next library releases, which will render neutron broken if we keep
using the option.
More details:
http://lists.openstack.org/pipermail/openstack-dev/2016-May/095166.html
Closes-Bug: #1586066
Change-Id: I884b4cc3ed04e4b5489e265c146666e04eb1bc27
This fixes the iptables rules generated by the L3 agent
(SNAT, DNAT, set-mark and metadata), and the DHCP agent
(checksum-fill) to match the format that will be returned
by iptables-save to prevent excessive extra replacement
work done by the iptables manager.
It also fixes the iptables test that was not passing the
expected arguments (-p PROTO -m PROTO) for block rules.
A simple test was added to the L3 agent to ensure that the
rules have converged during the normal lifecycle tests.
Closes-Bug: #1566007
Change-Id: I5e8e27cdbf0d0448011881614671efe53bb1b6a1
There is no method called_once_with() in Mock object.
Use assert_called_once_with() instead.
Change-Id: I263c407ba2a82f53aace5f71cb27ad8974335e24
Closes-Bug: #1544522
The use_namespaces option has been defined as a workaround to kernels
not properly supporting namespaces. This limitation is behind us, it's
time to remove use_namespaces after its deprecation in Kilo in order to
simplify code and remove a poorly tested case (use_namespaces=False).
This change prepares for removal pullup_route method[1] which was only
used when use_namespaces=False.
[1] neutron.agent.linux.ip_lib
DocImpact
UpgradeImpact
Closes-Bug: #1508188
Related-Bug: #1435382
Depends-On: I303038eec560a6d99421140c2822aed8b518470b
Depends-On: I4feb2a15c7e1e4bfdbed2531b18b8e7d798ab3cc
Change-Id: I2fbf65df1250d9f9f1656b3964ee3b6de1ef1118
The new RPC interface has proved itself for two cycles, I don't
recollect any serious issues with it, so let's just clean up the
obsolete neutronclient based fallback mechanism.
The metadata agent configuration documentation should be updated
to not require API configuration values for the agent to talk to
neutron-server.
DocImpact
Change-Id: I254c575c66214f50fb93a94c46c4c9caebfc2937
Closes-Bug: #1502947
We attempt to log every successful metadata response with LOG.debug. But
as per oslo.log docs [1], we should make sure that what we pass into the
library is unicode.
Http.request returns a tuple of Response object and a string, which is
bytes in Python 2.x [2].
That's why we need to convert the response content to unicode before
passing it into oslo.log.
To achieve it, we utilize encodeutils.safe_decode with 'replace' errors
handling strategy, so that we don't get exceptions on input that does
not conform unicode.
For the unit test case, we pass a string that is not expected to convert
to unicode with errors='strict' strategy or similar, and check that we
still don't crash.
While at it, we remove a check for the number of log calls being
triggered, because it's something that we should avoid validating in
test cases, and it cannot trigger a real bug. The mock that was used to
count the number would also hide the bug that we try to reproduce.
Note that the bug does not require debug to be set because the crash
occurs before oslo.log machinery decides it should not log the message.
[1]: http://docs.openstack.org/developer/oslo.log/usage.html#no-more-implicit-conversion-to-unicode-str
[2]: http://bitworking.org/projects/httplib2/doc/html/libhttplib2.html#httplib2.Http.request
Closes-Bug: #1501772
Change-Id: I6a32c40ff117fae43913386134c8981539697ce8
Neutron[1] uses the option --metadata_proxy_watch_log=false to disable
log watch[2] in neutron-ns-metadata-proxy instances but should use the
option --nometadata_proxy_watch_log. It implies that
neutron-ns-metadata-proxy instances fail to start.
This changes updates neutron[1] to use the correct option.
The change also corrects associated functional tests[2], indeed
metadata_proxy_watch_log option has no effect if a log_file/dir is
defined for the agent running the neutron-ns-metadata-proxy.
[1] neutron.agent.common.config
[2] could be done by setting metadata_proxy_watch_log = false
[3] neutron.tests.functional.agent.test_l3_agent
Closes-Bug: #1490594
Change-Id: Iaec4a78847d802234c99514313440fd7c14bc554
Currently iptables rules set on L3 agent with metadata_proxy enabled
mark all packets coming from all interfaces including external interfaces.
This change updates PREROUTING rules from MANGLE table to mark packets
only from internal interfaces.
Change-Id: I01549df7b99be84cd46b6f97a5fd62aec1f43275
Closes-Bug: #1477553
There is nothing Linux or agent specific in the function. I need to use
it outside agent code in one of depending patches, hence moving it into
better location while leaving the previous symbol in place, with
deprecation warning, for backwards compatibility.
Change-Id: I252356a72f3c742e57c1b6127275030f0994a221
Since a packet can only have one mark, and we will need to mark a
packet for multiple purposes, we need to use a coordinated bitmask for
the two cases of simple marking that we currently do in Neutron
leaving the other bits for address scopes.
DocImpact
Change-Id: Id0517758d06e036a36dc8b8772e41af55d986b4e
Partially-Implements: blueprint address-scopes
Allow operators to set the endpoint url for neutron in the config
overriding the url that comes from the keystone catalog.
Change-Id: I93f81ef1be2de1038d9a899b0c4becdb5a8e8775
DocImpact
Closes-Bug: #1466258
oslo.service has graduated, so neutron should consume it.
Closes-Bug: #1466851
Depends-On: Ie0fd63f969f954029c3c3cf31337fbe38f59331a
Depends-On: I2093b37d411df9a26958fa50ff523c258bbe06ec
Depends-On: I4823d344878fc97e66ddd8fdae25c13a34dede40
Change-Id: I0155b3d8b72f6d031bf6f855488f80acebfc25d4
Removed use of contextlib.nested call from codebase, as it has been
deprecated since Python 2.7.
There are also known issues with contextlib.nested that were addressed
by the native support for multiple "with" variables. For instance, if
the first object is created but the second one throws an exception,
the first object's __exit__ is never called. For more information see
https://docs.python.org/2/library/contextlib.html#contextlib.nested
contextlib.nested is also not compatible with Python 3.
This is the first patch in a series for removing use of
contextlib.nested.
Added hacking check to catch if any new instances are added to
the codebase.
Line continuation markers (e.g. '\') had to be used or syntax
errors were thrown. While using parentheses is the preferred way
for multiple line statements, but in case of long with statements
backslashes are acceptable.
Partial-Bug: 1428424
Change-Id: I171fbdb89892a3d4548bf2ca52f4a7dd9ef8dccb
Avoid calling neutron.db.api.get_engine().pool.dispose() if
an engine facade has not yet been created since there won't
be any connections to get rid of. Calling it on services
that do not use the DB (e.g. agents) unnecessarily creates
a database connection engine that will never be used.
Change-Id: I3dbad1bef5da7b3765898e7d539b4d119b89e73a
Closes-Bug: #1433536
This change ensures that the structure of the unit test tree matches
that of the code tree to make it obvious where to find tests for a
given module. A check is added to the pep8 job to protect against
regressions.
The plugin test paths are relocated to neutron/tests/unit/plugins
but are otherwise ignored for now.
Change-Id: If307593259139171be21a71c58e3a34bf148cc7f
Partial-Bug: #1440834
Currently metadata proxy cannot run with nobody user/group as metadata
proxy requires to connect to metadata_proxy_socket when queried.
This change allows to run metadata proxy with nobody user/group by
allowing to choose the metadata_proxy_socket mode with the new option
metadata_proxy_socket_mode (4 choices) in order to adapt socket
permissions to metadata proxy user/group.
This change refactors also where options are defined to enable
metadata_proxy_user/group options in the metadata agent.
In practice:
* if metadata_proxy_user is agent effective user or root, then:
* metadata proxy is allowed to use rootwrap (unsecure)
* set metadata_proxy_socket_mode = user (0o644)
* else if metadata_proxy_group is agent effective group, then:
* metadata proxy is not allowed to use rootwrap (secure)
* set metadata_proxy_socket_mode = group (0o664)
* set metadata_proxy_log_watch = false
* else:
* metadata proxy has lowest permissions (securest) but metadata proxy
socket can be opened by everyone
* set metadata_proxy_socket_mode = all (0o666)
* set metadata_proxy_log_watch = false
An alternative is to set metadata_proxy_socket_mode = deduce, in such
case metadata agent uses previous rules to choose the correct mode.
DocImpact
Closes-Bug: #1427228
Change-Id: I235a0cc4f0cbd55ae4ec1570daf2ebbb6a72441d
Regarding https://review.openstack.org/#/c/145829/
The old code of DnsMasq will always get root_helper from
neutron.agent.dhcp.agent.
However, new code will only set run_as_root when namespace
is used. That will cause permission error when namespace
is disabled and dnsmasq need to be started.
Change-Id: Ib00d6e54dba44dbbbec158b9e0518e6e42baceec
Closes-Bug: #1428007
Currently metadata proxy cannot run with nobody user/group as
metadata proxy (as other services) uses WatchedFileHandler handler to
log to file which does not support permissions drop (the process must
be able to r/w after permissions drop to "watch" the file).
This change allows to enable/disable log watch in metadata proxies with
the new option metadata_proxy_log_watch. It should be disabled when
metadata_proxy_user/group is not allowed to read/write metadata proxy
log files. Option default value is deduced from metadata_proxy_user:
* True if metadata_proxy_user is agent effective user id/name,
* False otherwise.
When log watch is disabled and logrotate is enabled on metadata proxy
logging files, 'copytruncate' logrotate option must be used otherwise
metadata proxy logs will be lost after the first log rotation.
DocImpact
Change-Id: I40a7bd82a2c60d9198312fdb52e3010c60db3511
Partial-Bug: #1427228
The L3 agent gets keepalived state change notifications via
a unix domain socket. These events are now batched and
send out as a single RPC to the server. In case the same
router got updated multiple times during the batch period,
only the latest state is sent.
Partially-Implements: blueprint report-ha-router-master
Change-Id: I36834ad3d9e8a49a702f01acc29c7c38f2d48833
Currently the metadata service can be requested on 169.254.169.254:80
from all interfaces including external interfaces. This change updates
PREROUTING rules to allow request on 169.254.169.254:80 only from
internal interfaces.
Change-Id: I44a9e03992f9e2a7bd4d798ae69d8aa7d75d3078
Closes-Bug: #1187102
Previously L3 HA generated a bash script and copied it to a per-router
configuration directory that was visible to that router's keepalived
instance. This patch changes the in-line generated Bash script to a
Python script that can be maintained in the repository.
The bash script was used as a keepalived notifier script, that was invoked
by keepalived whenever a state transition occured. These notifier scripts
may be invoked by keepalived out of order in case it transitions quickly
twice. For example, if the master failed and two slaves fight for the new
master role. One will transition to master, and the other will often
transition to master and then immidiately back to standby. In this case,
the transition scripts were often fired out of order, resulting in the
wrong state being reported.
The proposed approach is to get rid of the keepalived notifier scripts
entirely. Instead, monitor IP changes on the HA device. If the omnipresent
IP address was configured on the HA device, it means that we're looking
at a master instance. If it was deleted, the router transition to standby
or fault.
In order to keep the L3 agent CPU usage down, it will spawn a process
per HA router. That process will start the ip address monitor.
Whenever it gets an IP address change event, it will notify the L3 agent
via a unix domain socket.
Partially-Implements: blueprint report-ha-router-master
Change-Id: I2022bced330d5f108fbedd40548a901225d7ea1c
Closes-Bug: #1402010
Closes-Bug: #1367705
Brian Haley