Adopt neutron-lib plugin directory

Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.

Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.

This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.

Usage pattern [1,2] can be translated to [3,4] respectively.

[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()

The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.

Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.

Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.

NeutronLibImpact

Partially-implements: blueprint neutron-lib

Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf

neutron/plugins/ml2/extensions/dns_integration.py

@@ -14,6 +14,7 @@
 #    under the License.
 
 from neutron_lib.api import validators
+from neutron_lib.plugins import directory
 from oslo_config import cfg
 from oslo_log import log as logging
 
@@ -24,7 +25,6 @@ from neutron.callbacks import resources
 from neutron.db import models_v2
 from neutron.db import segments_db
 from neutron.extensions import dns
-from neutron import manager
 from neutron.objects import network as net_obj
 from neutron.objects import ports as port_obj
 from neutron.plugins.common import utils as plugin_utils
@@ -264,7 +264,7 @@ class DNSExtensionDriver(api.ExtensionDriver):
                                       dns_data_db)
 
     def _get_network(self, context, network_id):
-        plugin = manager.NeutronManager.get_plugin()
+        plugin = directory.get_plugin()
         return plugin.get_network(context, network_id)