633 Commits

Author SHA1 Message Date
Anna Khmelnitsky
00ef2c98de [VMware] Adjust to enforcement point API change
Enforcement point was modified to hold single connection.
In addition, to avoid requirements conflict, modify devstack
scripts to work against stable/ocata branch of vmware-nsxlib.

Change-Id: I4b889851d1aa0e142e5b95a696ccaa60fa4a8448
2017-07-19 15:15:01 -07:00
Jenkins
4fb6e00261 Merge "[VMware] VMware NSX Policy driver" 2017-07-18 04:24:50 +00:00
Anna Khmelnitsky
e30de6e13d [VMware] VMware NSX Policy driver
This introduces driver for Vmware NSX Policy.
The driver assumes nsx_v3 core plugin.
It implements direct configuration of NSX Policy endpoint for security
and inherits connectivity functionality from resource mapping driver.

On startup, the driver will configure NSX Policy enforcement point to be
the NSX manager core plugin is running against.

The driver implements the following resource mapping:

Openstack project => NSX Policy domain
GBP group = > NSX Policy group + communication maps
GBP classifier => NSX Policy service
GBP rule set => NSX Policy communication profile

Change-Id: I0d5593b458f7e51c21fc2b34d1ab4d898abb6c51
2017-07-17 14:01:12 -07:00
Jenkins
6d05469467 Merge "Set AIM Tenant description field to apic_system_id" 2017-07-13 02:09:54 +00:00
Ivar Lazzaro
49d9b548aa
Set AIM Tenant description field to apic_system_id
Change-Id: I4bda528ee490dd61f59c257637d8ba2c2d2753aa
2017-07-11 13:43:34 -07:00
Thomas Bachman
e3e62e1197 Increase ip_pool to 256 characters
The length of the ip_pool parameter in L3 policies is limited,
and needs to be increased in order to support multiple prefixes,
as well as prefixes from all address families (v4 and v6).

Change-Id: I9dc03ac4275e0f269746011a075bf3b78e0d5233
2017-07-10 20:54:01 +00:00
Sumit Naiksatam
7eb3afc6f0 Ocata sync dsvm jobs udpate
This patch updates the setup for the various devstack gate jobs
to make them functional for Ocata and thus also validates
the parent patch Ocata sync patch.

Note that the NFP job still needs to be updated and has been
disabled in this patch. It can be re-enabled whenever its
updated in a followup patch.

Change-Id: I7cd5667fcc86577eb33c0233991cbb687c0ce8c9
2017-07-07 23:40:21 +00:00
Anna Khmelnitsky
8284bf9b3a Ocata sync
* use neutron_lib.directory for plugin retrieval

* switch to neutron_lib for neutron constants, exceptions,
  extensions

* add neutron.plugins.ml2.ovo_rpc to OUT_OF_PROCESS_NOTIFICATIONS:
  neutron added ovo rpc callback mechanism for ovo objects, and aim
  notification manager needs to recognize those as out of process.
  Since neutron moved away from get_session API to get_reader_session
  and get_writer_session, override for these was added.
  Few bugs were fixed in the delayed notification area as well.

* new engine facade: make use of reader and writer to grab db engine

* remove _update_fip_assoc override (didn't find a reason for the
  override)

* aim driver: a fix in update_subnetpool_precommit - not to assume
  address_scope_id field is returned from neutron update call if it
  was not updated.

* extend_XXX_dict call was switched to receive ovo instead of db
  object. As a result, foreign keys are not part of the object
  anymore, and need to be retrieved from db.

* remove_router_interface - receive port dictinary rather than port
  object

* fix patched neutron functions to receive correct parameter types
  (like patched_get_locked_port_and_binding)

* use add_agent_status_check_worker instead of add_agent_status_check

* advertise_mtu configuration parameter was removed from neutron. It
  is used in aim driver, hence added to aim driver config.

* use of project_id instead of tenant_id where required

* use segments_db module for network segments

* test_aim_mapping_driver: the test used to override uuid generation
  in order to get predictable uuid results. New neutron code makes
  use of python uuid module where overrides are complicated. It was
  easire to remove all uuid-based values from dictionaries under test

* add filters parameter to get_address_scopes calls, otherwise the
  call fails (probably should be fixed in neutron)

* in routing tests, remove the assumption that routes are returned in
  specific order

Change-Id: I1943fd4196ea6199d825ae53f0e9f5b54d54a260
2017-07-05 15:25:25 -07:00
Thomas Bachman
40b666359c Fix update floatingip description
The standard-attr-description extension isn't being honored during
floatingip update operations, due to the fact that the upstream neutron
code has been monkey-patched by GBP. This change set fixes the GBP
monkey-patch to properly support the standard-attr-description extension.

Change-Id: I874052879d3a51545a5b47cc362071d3f3e031d0
Closes-Bug: 1702073
2017-07-03 12:18:23 +00:00
Jenkins
e19cc9e2ca Merge "Fix network service policy deletion" 2017-06-27 17:25:03 +00:00
Anna Khmelnitsky
5408241ef2 Fix network service policy deletion
qos_policy_id key might or might not be set in precommit, but postcommit
assumed it exists

Change-Id: I8dda97be0e57aab715d6a7d5fa0a21f3d6295fab
2017-06-26 21:37:38 -07:00
Ivar Lazzaro
7dd4a1aec0
Use Neutron's Nova client for retrieving VMs' name
This will make sure we always use the most recently supported
authentication methods.

Change-Id: I66a64d11ae09c426ca0114020c8b985f9307515e
2017-06-26 12:37:17 -07:00
Mahesh Kurund
dc25f71543 NFP changes to support async model
Following are the NFP side changes to support async operations.
1. Added get_status api in nfp_node_driver to update operational
   state of servicechain node.
2. Using queue notifications to send rpc to orchestrator.
3. Extended ncp_node_instance_network_function_mappings table to
   have status and status_details of an network_function.

Change-Id: I5375066fb640d53c6bc5f0a7cf65902faa221519
Co-Authored-By: Ashutosh Mishra <mca.ashu4@gmail.com>
Closes-Bug: 1671077
2017-06-24 16:01:49 +05:30
Rajendra Machani
283a53b4e8 Support chain mapping driver in precommit mode
This adds support to the AIM mapping driver, by invoking
postcommit calls in precommit.

Change-Id: I8ce3a86ca6b46a029eb68daa85b3184692b91189
Closes-Bug: 1672674
2017-06-24 15:56:41 +05:30
Mahesh Kurund
8e9c4d0459 Added status support in servicechain resources
Modified ncp plugin to support status retrieval of servicechain resources.
Updating chain mapping driver to compute status of group based on its associated
servicechain.
Closes-Bug: 1668174

Change-Id: I1841c0c04012cb3d58d152a6f4bdd348d3d701dd
2017-06-24 13:54:39 +05:30
Thomas Bachman
02a94fb722 Remove APIC mapping policy driver
The APIC mapping policy driver was deprecated in mitaka and
replaced with the AIM mapping driver. This patch removes the
APIC mapping driver, starting with the ocata release.

Change-Id: I5f33cd2c0e06cf45e092e74e664809475904c047
2017-06-23 16:41:29 +00:00
Jenkins
074ea874bb Merge "Stop using neutron PLURALS dict" 2017-06-15 16:03:23 +00:00
Sumit Naiksatam
1d630b3a4a Remove monolithic service chain plugin and drivers
This plugin is subsumed by the Node Composition plugin
starting in the liberty cycle.

Switching to the NCP as default invalidated some UTs (since
NCP does not support more than one service_chain_spec
per service_chain_instance). These tests are being skipped.

Change-Id: I03383145eaa72681695e12649f731ba1a6b8bad8
2017-06-13 13:35:03 -07:00
Amit Bose
10813882d1 Adapt per-tenant-nat-epg table for tenant_id rename
Change-Id: I0b40cdc945f7e3c78ca99453b6225498fe69b2f1
Signed-off-by: Amit Bose <bose@noironetworks.com>
2017-06-11 06:54:26 +00:00
Sumit Naiksatam
4cb980661e Propagate retry exception in implicit subnet allocation
In concurrent execution cases a failure can occur in the Neutron
IPAM component due to failure to obtaining a lock. In such cases
the IPAM component raises a retry exception which should be not
be eaten, but relayed as is so that the operation can be retried.

This patch checks if the exception raised during a subnet allocation
from a subnetpool fails due to a retry exception, and if so, raises
it as is to facilitate a retry.

Change-Id: I381cdf533b27d710f68903f0cfb516043b4607d6
2017-06-09 01:39:58 -07:00
Jenkins
16dca6514e Merge "[aim] Fix retry issues and logging" 2017-06-08 18:47:32 +00:00
Thomas Bachman
5dc13b75a6 Fix auto-PTG policy for IPv6
The implicit policy created for the auto-PTG does not allow traffic
for IPv6. This prevents IPv6 traffic from flowing between PTs in the
auto-PTG and user-created PTGs, which includes things like ICMPv6,
DHCPv6, and IPv6 DNS and HTTP traffic between the DHCP server and PTs.

Change-Id: I28fe713e24744e36e2912d7f5d830b64a658f8bd
Closes-Bug: 1696438
2017-06-08 01:09:06 +00:00
Robert Kukura
953e5d6ae5 [aim] Fix retry issues and logging
Pass create_if_absent=False to AIM's get_status() to hopefully reduce
transaction retries due to DBDuplicateEntry exceptions. This required
unpinning the version of AIM used, as well as a couple of fixes in the
AIM repo.

Change the RPC handlers to use Neutron's retry_db_errors decorator
rather than its own, so that DBDuplicateEntry exceptions are retried.

Avoid logging at error level when processing retriable exceptions.

Change-Id: I53740eea3cb7cacafceae589deec3b573ef6a68a
2017-06-07 05:18:06 -04:00
Sumit Naiksatam
5b80125b21 [apci_aim] Remove use of with_lockmode in ml2plus
In certain cases of concurrent operations we are seeing an error
which suggests that a rolled back transaction is be reused. On debugging
it has been observed that the error manifests when the code path
executes the queries that are using with_lockmode in a couple of
places in the ml2 plugin component. Removing the with_lockmode usage
seems to prevent this issue and does not seem to be affecting the
correctness of behavior even in concurrent execution situations.

This patch removes the use of the with_lockmode in the identified place
when the ml2plus plugin configured.

Change-Id: If65c238cbf49a9cfd2546ca26d37ee721f6f986c
2017-06-01 00:31:36 -07:00
Robert Kukura
9b4b7276ad [apic-aim] Data migration for persisting mapping to APIC
A previous commit (https://review.openstack.org/#/c/450309/) added DB
tables mapping Neutron resource identities to APIC resource
identities, but did not include a data migration. This patch populates
the new tables during the DB migration with the APIC resource
identities for existing Neutron resources, using information from both
the Neutron and AIM DB tables.

Mechanism driver code that had been kept around in case it was needed
for the migration is also cleaned up.

Change-Id: Ia8a74b9c2289060234716ce89fb4b7b3d1c29596
2017-05-27 16:40:35 -04:00
Jenkins
ef12ed3d97 Merge "Dual-stack support for L3 Policy" 2017-05-26 20:10:17 +00:00
Thomas Bachman
d670a77961 Dual-stack support for L3 Policy
This adds dual-stack support for L3 Policy. It leverages
the existing parameters for subnetpools and address scopes,
and adds behaviors to support the implicit workflow.

Change-Id: Idedbb3d08b09e76abdba6d1aba0f62ba53a19a99
partially-implements: blueprint address-scope-mapping
2017-05-26 01:56:52 +00:00
Robert Kukura
b9f80c84c1 [ml2plus] Don't eat retriable exceptions in extend_*_dict functions
Change-Id: If6574e691151f90b648978cafa83345cc71556f1
2017-05-25 16:50:44 -04:00
Robert Kukura
6b7ea5ecfe [apic_aim] External connectivity for multi-scope routing
Manage external connectivity for all VRFs associated with a router.

Change-Id: I6016d85b433093bee960010b57a19ceb4b78b67d
2017-05-24 00:08:12 -04:00
Jenkins
690c0e8eca Merge "[aim_mapping] Create implicit AIM contracts for existing l3ps" 2017-05-19 19:55:52 +00:00
Jenkins
8f03ecda71 Merge "[AIM] suppress the DBDuplicateEntry error caused by concurrent transactions" 2017-05-19 05:49:47 +00:00
Jenkins
0016066f44 Merge "Reduce log level of subnet allocation exception" 2017-05-19 05:46:24 +00:00
Kent Wu
dde2fa0ea0 [AIM] suppress the DBDuplicateEntry error caused by concurrent transactions
Change-Id: I953d2a676bbafd7450059b2c8c951c537950c91d
2017-05-18 18:21:18 -07:00
Sumit Naiksatam
ee93a21cd6 [aim] Reduce log level of pymysql exception
The oslo_db code catches the pymysql SAVEPOINT exception, logs it error level,
and then raises it as an oslo_db exception, and due to which the operation
is retried. As such, the ERROR level logging of the pymysql exeption is a
bit misleading. This patch works around it by patching the oslo_db module
to log at debug level instead of error level.

Change-Id: If8afe36706f5d07d7ab9b840c877854b8ed11c42
2017-05-18 14:09:28 -07:00
Sumit Naiksatam
29e65bbcb2 Reduce log level of subnet allocation exception
We retry on this exception, hence the ERROR level of this exception
in the logs is misleading. Reducing it to info level.

Change-Id: I8e97b0aadfbc01bde6625ed86abe85c35bb3dec5
2017-05-18 14:06:42 -07:00
Anna Khmelnitsky
ef77ee3fdf Stop using neutron PLURALS dict
As part of ocata sync effort, stop using PLURALS defined in neutron,
as it was removed by Ifdf29b8fc4c824c0ee840c4f51593a5aa8a22127.
Use locally defined dictionary instead.

Change-Id: I763e6036177b57c0bc8ef7311b9c8bbbcefbe634
2017-05-17 19:19:58 -07:00
Robert Kukura
2ffa0ee9db [apic-aim] Persist mapping of Neutron resources to APIC
Add DB tables mapping Neutron resource identities to APIC resource
identities. This reduces the amount of DB querying, and helps unify
the handling of pre-existing APIC resources with those fully
orchestrated by the apic_aim drivers.

Currently, the mappings of address scopes and networks are
persisted. Persisting the relationship between routers and VRFs will
be considered later.

Note that since this patch will be back-ported to stable/newton and
the QoS feature will not, this patch's DB migration is sequenced
before the QoS DB migration.

Change-Id: Ie06281dde965d349d7fa1035f14124b35d60d85c
2017-05-16 20:08:51 -04:00
Robert Kukura
1626b7863d [apic_aim] Multi-scope routing
Allow subnets with different address scopes, as well as unscoped
subnets, to be attached as interfaces to the same router. Note that no
East/West routing is provided between differently scoped interfaces of
a router, but East/West routing is provided within each scope and
North/South routing is provided between each scope and the router's
gateway.

Routed IPv4 and IPv6 subnets on the same network currently either must
both be unscoped or each must be associated with isomorphic address
scopes (referencing the same VRF). Adding a subnet to a router results
in a NonIsomorphicNetworkRoutingUnsupported exception if this
constraint would be violated. Eventually, use of identity NAT to move
IPv6 traffic from the network's IPv4 VRF to its IPv6 VRF will allow
this constraint to be removed or relaxed.

A flag in interface_info is added for GBP to override network routing
topology validation when adding router interfaces. This should not be
used for any other purpose, and will eventually be removed without
warning.

External connectivity for routers associated with multiple VRFs will
require some follow-on work to correctly handle all cases.

Change-Id: Idbbd4400e570654937c2bee4577422a91224430e
2017-05-16 08:02:38 -04:00
Sumit Naiksatam
6572f1621a [aim_mapping] Create implicit AIM contracts for existing l3ps
This patch facilitates migrating a deployment which had l3_policies prior to the
aim_mapping policy driver moving to per-l3p implicit AIM contracts (see commit:
f50db6f1ce)

The following configuration:
[aim_mapping]
create_per_l3p_implicit_contracts=True

controls if the migration step is perform. This configuration is set to True by default
and hence the migration step is always performed at the time of the initialization of the
aim_mapping driver. For l3_policies which already have associated implicit contracts,
this step is a no-op.

The migration step can be turned off by setting the above configuration to False.
A Neutron server restart is required for the config change to take effect.

Since this mogration option is only for migrating newton or prior deployments, it
will be removed in the O release.

Change-Id: I7e5f793bdf3618655600898feba64aac7c099239
2017-05-13 08:10:45 -07:00
Sumit Naiksatam
f80ac75c36 [aim] ml2plus conditional registration of event handlers
In the following commit:
456e169f46

we updated ml2plus plugin to subscribe for subnet delete events. The
handlers for these event notifications had been added in stable/newton.
However, Red Hat OSP distro has not yet picked up these changes in
stable/newton, and hence the ml2plus plugin intialization fails (since
the event handlers are not found).

This patch allows ml2plus initialization to proceed if the event handlers
are not present. It is expected that nothing else will break by skipping
the subscription of these events.

Change-Id: Idd76eb4e5d2d66f8d05e1da8ba89ba40d16a5bb2
2017-05-08 15:47:37 -07:00
Jenkins
7596424c05 Merge "[AIM] Late binding of the VMM and physical domain" 2017-05-05 22:15:33 +00:00
Kent Wu
91d7ea20a4 [AIM] Late binding of the VMM and physical domain
1. only associate the domain with the EPG when a port is created on
a host that belongs to this domain.
2. also dis-associate the domain when the last port among all the hosts
under this domain is deleted.
3. User has to use:
'aimctl manager host-domain-mapping-create <host_name>
--vmm_domain_name=<vmm_dom> --physical_domain_name=<phys_dom>'
to create the host -> domain mapping.

Change-Id: Ie0882117b75ada3c2f32770adc7bc147a61dfd14
2017-05-04 18:51:12 -07:00
Jenkins
d3f65fa895 Merge "[aim_mapping] Per-l3p implicit contracts" 2017-05-02 21:18:09 +00:00
Anna Khmelnitsky
327bbbe34c Fix copy-paste errors in extension unit test
Change-Id: I68656de10c3287a73d230b8ec006d39973a9b002
2017-05-01 19:32:57 -07:00
Sumit Naiksatam
f50db6f1ce [aim_mapping] Per-l3p implicit contracts
The implicit AIM contracts were being earlier created
per tenant (lazily created when the first l2p is created)
and used by the default_epg and all other epgs created by
a consequence of the user actions. As we move towards a model
of supporting multiple l3ps in the same AIM VRF, we will
need per-l3p contracts to enforce isolation between l3ps. This
patch is the first step in that direction where the per-tenant
implicit contracts are now created per-l3p. Contracts are created
when l3p is created and deleted when l3p is deleted.

This patch also fixes the problem of implicit contracts being
not cleaned up when the last two l2ps in a tenant were being
deleted concurrently.

Existing AIM deployments might need to be migrated to this new model.
Migration strategies would differ per deployment but might at least
require running a script that creates the implicit contracts per
existing l3ps.

Change-Id: I7f18c672db5ffcec9ce445bc1a32d508a685c9c6
2017-05-01 12:30:40 -07:00
Jenkins
3b0e56e24c Merge "Deal with db-related neutron deprecations" 2017-04-28 21:46:18 +00:00
Anna Khmelnitsky
92511f3c12 Deal with db-related neutron deprecations
Use HasId, HasTenant and address_scope db from neutron_lib

Change-Id: I2b7f86d27251a7d952178f901d033dd841050b6c
2017-04-28 10:08:54 -07:00
Robert Kukura
d902e97ab3 [apic-aim] Isomorphic address scopes
Allow a single IPv4 address scope and a single IPv6 address scope to
reference the same VRF, which may be pre-existing or mapped from one
of the address scopes.

Change-Id: Ibe5288a3a6d5032e4c0ac509a0857ce5defafa9c
2017-04-28 08:55:34 -04:00
Sumit Naiksatam
4ba8f82380 Prioritizing in-process notifications
The aim_mapping policy driver currently leverages a scheme of delaying
dispatching notifications until the outermost transaction, which
initiated the notification, has committed. However, Neutron ML2 plugin
is moving to a model of more in-process notifications, and some of these
changes are being backported to stable/newton as well (see:
4a96b962b5).
These in-process notifications need not be delayed, and in fact they
should be dispatched and processed immediately, and within the scope
of the transaction.

This patch attempts to identify the notifications that need to be dispatched
immediately versus those that need to be delayed based on the module
names of the notfication handlers. The currently known list of agent and
registry notification handlers (used by the aim_mapping driver) is captured
in the OUT_OF_PROCESS_NOTIFICATIONS list in:
gbpservice/network/neutronv2/local_api.py

If a notification handler does not belong to the above list, it is dispatched
immediately.

As and when additional notification handlers are identified for which the
notification needs to be queued for delayed dispath, they should be added to
the above OUT_OF_PROCESS_NOTIFICATIONS list.

Change-Id: Ib5beb9c8056463c875c775ad19d2d3cbee3430dd
2017-04-27 02:53:34 -07:00
Rajendra Machani
8cdf47d158 Fix to read correct options of keystone_authtoken
Reading the correct option names from keystone_authtoken group for authentication.

Change-Id: If3d05592b67f9e75f34f14864e564084c991b9e7
Closes-Bug: 1678732
2017-04-26 09:48:17 +00:00