When add allowed-address-pair 0.0.0.0/0 to one port, it will
unexpectedly open all others' protocol under same security
group. IPv6 has the same problem.
The root cause is the openflow rules calculation of the
security group, it will unexpectedly allow all IP(4&6)
traffic to get through.
For openvswitch openflow firewall, this patch adds a source
mac address match for the allowed-address-pair which has
prefix lenght 0, that means all ethernet packets from this
mac will be accepted. It exactly will meet the request of
accepting any IP address from the configured VM.
Test result shows that the remote security group and
allowed address pair works:
1. Port has 0.0.0.0/0 allowed-address-pair clould send any
IP (src) packet out.
2. Port has x.x.x.x/y allowed-address-pair could be accepted
for those VMs under same security group.
3. Ports under same network can reach each other (remote
4. Protocol port number could be accessed only when there
has related rule.
(cherry picked from commit 00298fe6e8)
There is a lock synchronized created by Neutron IpsetManager
with common name 'neutron-ipset'. Some deployments
could use multiple IpsetManagers object in separated
namespaces. Because of this on heavy-loaded environments
with a lot of ports and related security groups
it could end with not-consumed rabbitmq queue on security
groups notification. To prevent that this lock should be
parametrized by namespace name.
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Co-Authored-By: Akihiro Motoki <firstname.lastname@example.org>
neutron-lib contains the synchronized lockutils decorator as well as
the SYNCHRONIZED_PREFIX global. This patch consumes them from
neutron-lib and removes them from neutron.
Through  ipset members are updated in update_security_group_members
instead of updating during firewall apply. In the same way, we will
delete conntrack entries immediately after deleting remote ipset
members(in update_security_group_members) instead of deleting them after
As explained in , this change partially fixes bug #1580377 i.e it
deletes conntrack entries on remote hosts for a removed port.
All the callers of set_exists() already know the set name,
so there's no reason to re-calculate it based on the id and
ethertype. Renamed to set_name_exists() to be clear on what
it is checking.
ipset was locking on every set_members call with an external
filesystem lock. This was expensive when lots of ports that
were a part of the same security group were on the same agent.
This patch adjusts it to check if it needs to make a change before
acquiring the semaphore.
When l2 agent execute ipset command, we should pass
parameter 'check_exit_code' to self.execute acording to
The intention is to safely fail if we try to destroy a
non existing set, or delete a non existing member.
Otherwise the agent gets stuck in a loop if such situation
happens. Such kind of event would be logged as errors.
Previously, the ipset_manager would pass in 0.0.0.0/0 or ::/0 if
these addresses were inputted as allowed address pairs. This causes
ipset to raise an error as it does not work with zero prefix sizes.
To solve this problem we use two ipset rules to represent this:
Ipv4: 0.0.0.0/1 and 22.214.171.124/1
IPv6: ::/1' and '8000::/1
All of this logic is handled via _sanitize_addresses() in the ipset_manager
which is called to convert the input.
The new NET prefix introduced by I8177699b157cd3eac46e2f481f47b5d966c49b07
increases collision chances by trimming the sg_id by 3 more chars.
This patch reduces the prefix to 1 single char and also reduces the
swap suffix to reduce the chances of collision.
The previous hash type was 'ip' and this caused a major
issue with the allowed address pairs extension since it
results in CIDRs being passed to ipset. When the hash type
is 'ip', a CIDR is completely enumerated into all of its
addresses so 10.100.0.0/16 results in ~65k entries. This
meant a single allowed_address_pairs entry could easily
exhaust an entire set.
This patch changes the hash type to 'net', which is designed
to handle a CIDRs as a single entry.
This patch also changes the names of the ipsets because
creating an ipset with different parameters will cause an
error and our ipset manager code isn't robust enough to handle
that at this time. There is another ongoing patch to fix
that but it won't be ready in time.
The related bug was closed by increasing the set limit, which
did alleviate the problem. However, this change would also
address the issue because the gate tests run an allowed address
pairs extension test with the CIDR mentioned above.
This reverts commit b5b919a7a3.
The current ipset manager code isn't robust enough to handle
ipsets that already exist with different parameters. This reverts
the ability to change the parameters so we don't break upgrades
Change ipset_manager _refresh_set() to make a copy of the list of
IPs when creating a set, instead of using a reference, else any
change to the set could update the caller's data.
Also made the IpsetManagerTestCase classes always pass maxelem and
hashsize to the parent class.
Recently, these messages have been noticed in both tempest
logs, as well as reported by downstream users syslog:
Set IPv4915d358d-2c5b-43b5-9862 is full, maxelem 65536 reached
So the default of 64K is not sufficient enough.
This change adds two config options to control both the number
of elements as well as the hashsize, since they should be
tuned together for best performance. Slightly different
formats were required for 'ipset create' and 'ipset restore'.
The default values for these are now set to 131072 (maxelem) and
2048 (hashsize), which is an increase over their typical default values
of 65536/1024 (respectively), in order to fix the errors seen in
the tempest tests.
Refactor the IpsetManager to move all the low level
knowledge about the ipset behaviour and performance
considerations from the firewall driver to the manager,
reduce redundant function names, and change missleading
variables talking about ipset chains, where they
should say ipset sets.
No logical changes to behaviour, just responsibilities
moved from one class to another.
Unit testing is move from iptables_firewall to ipset_manager
to test the new responsibilities of the class.
Implements: blueprint ipset-manager-refactor
Add functional testing to the ipset_manager module to verify
it works as expected in combination with iptables matching.
Implements: blueprint add-ipset-to-security
Iptables chain is linear storage and filtering, when iptables rules are
large, the load of l2 agent is heavy, this patch introduces ipset to
security group for improving the security group performance.
Implements: blueprint add-ipset-to-security