Unified SDK for OpenStack
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 

524 lines
20 KiB

  1. # Licensed under the Apache License, Version 2.0 (the "License"); you may
  2. # not use this file except in compliance with the License. You may obtain
  3. # a copy of the License at
  4. #
  5. # http://www.apache.org/licenses/LICENSE-2.0
  6. #
  7. # Unless required by applicable law or agreed to in writing, software
  8. # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  9. # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
  10. # License for the specific language governing permissions and limitations
  11. # under the License.
  12. """
  13. The :class:`~openstack.connection.Connection` class is the primary interface
  14. to the Python SDK. It maintains a context for a connection to a region of
  15. a cloud provider. The :class:`~openstack.connection.Connection` has an
  16. attribute to access each OpenStack service.
  17. At a minimum, the :class:`~openstack.connection.Connection` class needs to be
  18. created with a config or the parameters to build one.
  19. While the overall system is very flexible, there are four main use cases
  20. for different ways to create a :class:`~openstack.connection.Connection`.
  21. * Using config settings and keyword arguments as described in
  22. :ref:`openstack-config`
  23. * Using only keyword arguments passed to the constructor ignoring config files
  24. and environment variables.
  25. * Using an existing authenticated `keystoneauth1.session.Session`, such as
  26. might exist inside of an OpenStack service operational context.
  27. * Using an existing :class:`~openstack.config.cloud_region.CloudRegion`.
  28. Using config settings
  29. ---------------------
  30. For users who want to create a :class:`~openstack.connection.Connection` making
  31. use of named clouds in ``clouds.yaml`` files, ``OS_`` environment variables
  32. and python keyword arguments, the :func:`openstack.connect` factory function
  33. is the recommended way to go:
  34. .. code-block:: python
  35. import openstack
  36. conn = openstack.connect(cloud='example', region_name='earth1')
  37. If the application in question is a command line application that should also
  38. accept command line arguments, an `argparse.Namespace` can be passed to
  39. :func:`openstack.connect` that will have relevant arguments added to it and
  40. then subsequently consumed by the constructor:
  41. .. code-block:: python
  42. import argparse
  43. import openstack
  44. options = argparse.ArgumentParser(description='Awesome OpenStack App')
  45. conn = openstack.connect(options=options)
  46. Using Only Keyword Arguments
  47. ----------------------------
  48. If the application wants to avoid loading any settings from ``clouds.yaml`` or
  49. environment variables, use the :class:`~openstack.connection.Connection`
  50. constructor directly. As long as the ``cloud`` argument is omitted or ``None``,
  51. the :class:`~openstack.connection.Connection` constructor will not load
  52. settings from files or the environment.
  53. .. note::
  54. This is a different default behavior than the :func:`~openstack.connect`
  55. factory function. In :func:`~openstack.connect` if ``cloud`` is omitted
  56. or ``None``, a default cloud will be loaded, defaulting to the ``envvars``
  57. cloud if it exists.
  58. .. code-block:: python
  59. from openstack import connection
  60. conn = connection.Connection(
  61. region_name='example-region',
  62. auth=dict(
  63. auth_url='https://auth.example.com',
  64. username='amazing-user',
  65. password='super-secret-password',
  66. project_id='33aa1afc-03fe-43b8-8201-4e0d3b4b8ab5',
  67. user_domain_id='054abd68-9ad9-418b-96d3-3437bb376703'),
  68. compute_api_version='2',
  69. identity_interface='internal')
  70. Per-service settings as needed by `keystoneauth1.adapter.Adapter` such as
  71. ``api_version``, ``service_name``, and ``interface`` can be set, as seen
  72. above, by prefixing them with the official ``service-type`` name of the
  73. service. ``region_name`` is a setting for the entire
  74. :class:`~openstack.config.cloud_region.CloudRegion` and cannot be set per
  75. service.
  76. From existing authenticated Session
  77. -----------------------------------
  78. For applications that already have an authenticated Session, simply passing
  79. it to the :class:`~openstack.connection.Connection` constructor is all that
  80. is needed:
  81. .. code-block:: python
  82. from openstack import connection
  83. conn = connection.Connection(
  84. session=session,
  85. region_name='example-region',
  86. compute_api_version='2',
  87. identity_interface='internal')
  88. From oslo.conf CONF object
  89. --------------------------
  90. For applications that have an oslo.config ``CONF`` object that has been
  91. populated with ``keystoneauth1.loading.register_adapter_conf_options`` in
  92. groups named by the OpenStack service's project name, it is possible to
  93. construct a Connection with the ``CONF`` object and an authenticated Session.
  94. .. note::
  95. This is primarily intended for use by OpenStack services to talk amongst
  96. themselves.
  97. .. code-block:: python
  98. from openstack import connection
  99. conn = connection.Connection(
  100. session=session,
  101. oslo_conf=CONF)
  102. From existing CloudRegion
  103. -------------------------
  104. If you already have an :class:`~openstack.config.cloud_region.CloudRegion`
  105. you can pass it in instead:
  106. .. code-block:: python
  107. from openstack import connection
  108. import openstack.config
  109. config = openstack.config.get_cloud_region(
  110. cloud='example', region_name='earth')
  111. conn = connection.Connection(config=config)
  112. Using the Connection
  113. --------------------
  114. Services are accessed through an attribute named after the service's official
  115. service-type.
  116. List
  117. ~~~~
  118. An iterator containing a list of all the projects is retrieved in this manner:
  119. .. code-block:: python
  120. projects = conn.identity.projects()
  121. Find or create
  122. ~~~~~~~~~~~~~~
  123. If you wanted to make sure you had a network named 'zuul', you would first
  124. try to find it and if that fails, you would create it::
  125. network = conn.network.find_network("zuul")
  126. if network is None:
  127. network = conn.network.create_network(name="zuul")
  128. Additional information about the services can be found in the
  129. :ref:`service-proxies` documentation.
  130. """
  131. import warnings
  132. import weakref
  133. import concurrent.futures
  134. import keystoneauth1.exceptions
  135. import requestsexceptions
  136. import six
  137. from openstack import _log
  138. from openstack import _services_mixin
  139. from openstack.cloud import openstackcloud as _cloud
  140. from openstack.cloud import _accelerator
  141. from openstack.cloud import _baremetal
  142. from openstack.cloud import _block_storage
  143. from openstack.cloud import _compute
  144. from openstack.cloud import _clustering
  145. from openstack.cloud import _coe
  146. from openstack.cloud import _dns
  147. from openstack.cloud import _floating_ip
  148. from openstack.cloud import _identity
  149. from openstack.cloud import _image
  150. from openstack.cloud import _network
  151. from openstack.cloud import _network_common
  152. from openstack.cloud import _object_store
  153. from openstack.cloud import _orchestration
  154. from openstack.cloud import _security_group
  155. from openstack import config as _config
  156. from openstack.config import cloud_region
  157. from openstack import exceptions
  158. from openstack import service_description
  159. __all__ = [
  160. 'from_config',
  161. 'Connection',
  162. ]
  163. if requestsexceptions.SubjectAltNameWarning:
  164. warnings.filterwarnings(
  165. 'ignore', category=requestsexceptions.SubjectAltNameWarning)
  166. _logger = _log.setup_logging('openstack')
  167. def from_config(cloud=None, config=None, options=None, **kwargs):
  168. """Create a Connection using openstack.config
  169. :param str cloud:
  170. Use the `cloud` configuration details when creating the Connection.
  171. :param openstack.config.cloud_region.CloudRegion config:
  172. An existing CloudRegion configuration. If no `config` is provided,
  173. `openstack.config.OpenStackConfig` will be called, and the provided
  174. `name` will be used in determining which cloud's configuration
  175. details will be used in creation of the `Connection` instance.
  176. :param argparse.Namespace options:
  177. Allows direct passing in of options to be added to the cloud config.
  178. This does not have to be an actual instance of argparse.Namespace,
  179. despite the naming of the
  180. `openstack.config.loader.OpenStackConfig.get_one` argument to which
  181. it is passed.
  182. :rtype: :class:`~openstack.connection.Connection`
  183. """
  184. # TODO(mordred) Backwards compat while we transition
  185. cloud = kwargs.pop('cloud_name', cloud)
  186. config = kwargs.pop('cloud_config', config)
  187. if config is None:
  188. config = _config.OpenStackConfig().get_one(
  189. cloud=cloud, argparse=options, **kwargs)
  190. return Connection(config=config)
  191. class Connection(
  192. _services_mixin.ServicesMixin,
  193. _cloud._OpenStackCloudMixin,
  194. _accelerator.AcceleratorCloudMixin,
  195. _baremetal.BaremetalCloudMixin,
  196. _block_storage.BlockStorageCloudMixin,
  197. _compute.ComputeCloudMixin,
  198. _clustering.ClusteringCloudMixin,
  199. _coe.CoeCloudMixin,
  200. _dns.DnsCloudMixin,
  201. _floating_ip.FloatingIPCloudMixin,
  202. _identity.IdentityCloudMixin,
  203. _image.ImageCloudMixin,
  204. _network.NetworkCloudMixin,
  205. _network_common.NetworkCommonCloudMixin,
  206. _object_store.ObjectStoreCloudMixin,
  207. _orchestration.OrchestrationCloudMixin,
  208. _security_group.SecurityGroupCloudMixin
  209. ):
  210. def __init__(self, cloud=None, config=None, session=None,
  211. app_name=None, app_version=None,
  212. extra_services=None,
  213. strict=False,
  214. use_direct_get=False,
  215. task_manager=None,
  216. rate_limit=None,
  217. oslo_conf=None,
  218. service_types=None,
  219. global_request_id=None,
  220. strict_proxies=False,
  221. pool_executor=None,
  222. **kwargs):
  223. """Create a connection to a cloud.
  224. A connection needs information about how to connect, how to
  225. authenticate and how to select the appropriate services to use.
  226. The recommended way to provide this information is by referencing
  227. a named cloud config from an existing `clouds.yaml` file. The cloud
  228. name ``envvars`` may be used to consume a cloud configured via ``OS_``
  229. environment variables.
  230. A pre-existing :class:`~openstack.config.cloud_region.CloudRegion`
  231. object can be passed in lieu of a cloud name, for cases where the user
  232. already has a fully formed CloudRegion and just wants to use it.
  233. Similarly, if for some reason the user already has a
  234. :class:`~keystoneauth1.session.Session` and wants to use it, it may be
  235. passed in.
  236. :param str cloud: Name of the cloud from config to use.
  237. :param config: CloudRegion object representing the config for the
  238. region of the cloud in question.
  239. :type config: :class:`~openstack.config.cloud_region.CloudRegion`
  240. :param session: A session object compatible with
  241. :class:`~keystoneauth1.session.Session`.
  242. :type session: :class:`~keystoneauth1.session.Session`
  243. :param str app_name: Name of the application to be added to User Agent.
  244. :param str app_version: Version of the application to be added to
  245. User Agent.
  246. :param extra_services: List of
  247. :class:`~openstack.service_description.ServiceDescription`
  248. objects describing services that openstacksdk otherwise does not
  249. know about.
  250. :param bool use_direct_get:
  251. For get methods, make specific REST calls for server-side
  252. filtering instead of making list calls and filtering client-side.
  253. Default false.
  254. :param task_manager:
  255. Ignored. Exists for backwards compat during transition. Rate limit
  256. parameters should be passed directly to the `rate_limit` parameter.
  257. :param rate_limit:
  258. Client-side rate limit, expressed in calls per second. The
  259. parameter can either be a single float, or it can be a dict with
  260. keys as service-type and values as floats expressing the calls
  261. per second for that service. Defaults to None, which means no
  262. rate-limiting is performed.
  263. :param oslo_conf: An oslo.config CONF object.
  264. :type oslo_conf: :class:`~oslo_config.cfg.ConfigOpts`
  265. An oslo.config ``CONF`` object that has been populated with
  266. ``keystoneauth1.loading.register_adapter_conf_options`` in
  267. groups named by the OpenStack service's project name.
  268. :param service_types:
  269. A list/set of service types this Connection should support. All
  270. other service types will be disabled (will error if used).
  271. **Currently only supported in conjunction with the ``oslo_conf``
  272. kwarg.**
  273. :param global_request_id: A Request-id to send with all interactions.
  274. :param strict_proxies:
  275. If True, check proxies on creation and raise
  276. ServiceDiscoveryException if the service is unavailable.
  277. :type strict_proxies: bool
  278. Throw an ``openstack.exceptions.ServiceDiscoveryException`` if the
  279. endpoint for a given service doesn't work. This is useful for
  280. OpenStack services using sdk to talk to other OpenStack services
  281. where it can be expected that the deployer config is correct and
  282. errors should be reported immediately.
  283. Default false.
  284. :param pool_executor:
  285. :type pool_executor: :class:`~futurist.Executor`
  286. A futurist ``Executor`` object to be used for concurrent background
  287. activities. Defaults to None in which case a ThreadPoolExecutor
  288. will be created if needed.
  289. :param kwargs: If a config is not provided, the rest of the parameters
  290. provided are assumed to be arguments to be passed to the
  291. CloudRegion constructor.
  292. """
  293. self.config = config
  294. self._extra_services = {}
  295. self._strict_proxies = strict_proxies
  296. if extra_services:
  297. for service in extra_services:
  298. self._extra_services[service.service_type] = service
  299. if not self.config:
  300. if oslo_conf:
  301. self.config = cloud_region.from_conf(
  302. oslo_conf, session=session, app_name=app_name,
  303. app_version=app_version, service_types=service_types)
  304. elif session:
  305. self.config = cloud_region.from_session(
  306. session=session,
  307. app_name=app_name, app_version=app_version,
  308. load_yaml_config=False,
  309. load_envvars=False,
  310. rate_limit=rate_limit,
  311. **kwargs)
  312. else:
  313. self.config = _config.get_cloud_region(
  314. cloud=cloud,
  315. app_name=app_name, app_version=app_version,
  316. load_yaml_config=cloud is not None,
  317. load_envvars=cloud is not None,
  318. rate_limit=rate_limit,
  319. **kwargs)
  320. self._session = None
  321. self._proxies = {}
  322. self.__pool_executor = pool_executor
  323. self._global_request_id = global_request_id
  324. self.use_direct_get = use_direct_get
  325. self.strict_mode = strict
  326. # Call the _*CloudMixin constructors while we work on
  327. # integrating things better.
  328. _cloud._OpenStackCloudMixin.__init__(self)
  329. _accelerator.AcceleratorCloudMixin.__init__(self)
  330. _baremetal.BaremetalCloudMixin.__init__(self)
  331. _block_storage.BlockStorageCloudMixin.__init__(self)
  332. _clustering.ClusteringCloudMixin.__init__(self)
  333. _coe.CoeCloudMixin.__init__(self)
  334. _compute.ComputeCloudMixin.__init__(self)
  335. _dns.DnsCloudMixin.__init__(self)
  336. _floating_ip.FloatingIPCloudMixin.__init__(self)
  337. _identity.IdentityCloudMixin.__init__(self)
  338. _image.ImageCloudMixin.__init__(self)
  339. _network_common.NetworkCommonCloudMixin.__init__(self)
  340. _network.NetworkCloudMixin.__init__(self)
  341. _object_store.ObjectStoreCloudMixin.__init__(self)
  342. _orchestration.OrchestrationCloudMixin.__init__(self)
  343. _security_group.SecurityGroupCloudMixin.__init__(self)
  344. # Allow vendors to provide hooks. They will normally only receive a
  345. # connection object and a responsible to register additional services
  346. vendor_hook = kwargs.get('vendor_hook')
  347. if not vendor_hook and 'vendor_hook' in self.config.config:
  348. # Get the one from profile
  349. vendor_hook = self.config.config.get('vendor_hook')
  350. if vendor_hook:
  351. try:
  352. # NOTE(gtema): no class name in the hook, plain module:function
  353. # Split string hook into module and function
  354. try:
  355. (package_name, function) = vendor_hook.rsplit(':')
  356. if package_name and function:
  357. import pkg_resources
  358. ep = pkg_resources.EntryPoint(
  359. 'vendor_hook', package_name, attrs=(function,))
  360. hook = ep.resolve()
  361. hook(self)
  362. except ValueError:
  363. self.log.warning('Hook should be in the entrypoint '
  364. 'module:attribute format')
  365. except (ImportError, TypeError) as e:
  366. self.log.warning('Configured hook %s cannot be executed: %s',
  367. vendor_hook, e)
  368. @property
  369. def session(self):
  370. if not self._session:
  371. self._session = self.config.get_session()
  372. # Hide a reference to the connection on the session to help with
  373. # backwards compatibility for folks trying to just pass
  374. # conn.session to a Resource method's session argument.
  375. self.session._sdk_connection = weakref.proxy(self)
  376. return self._session
  377. def add_service(self, service):
  378. """Add a service to the Connection.
  379. Attaches an instance of the :class:`~openstack.proxy.Proxy`
  380. class contained in
  381. :class:`~openstack.service_description.ServiceDescription`.
  382. The :class:`~openstack.proxy.Proxy` will be attached to the
  383. `Connection` by its ``service_type`` and by any ``aliases`` that
  384. may be specified.
  385. :param openstack.service_description.ServiceDescription service:
  386. Object describing the service to be attached. As a convenience,
  387. if ``service`` is a string it will be treated as a ``service_type``
  388. and a basic
  389. :class:`~openstack.service_description.ServiceDescription`
  390. will be created.
  391. """
  392. # If we don't have a proxy, just instantiate Proxy so that
  393. # we get an adapter.
  394. if isinstance(service, six.string_types):
  395. service = service_description.ServiceDescription(service)
  396. # Directly invoke descriptor of the ServiceDescription
  397. def getter(self):
  398. return service.__get__(self, service)
  399. # Register the ServiceDescription class (as property)
  400. # with every known alias for a "runtime descriptor"
  401. for attr_name in service.all_types:
  402. setattr(
  403. self.__class__,
  404. attr_name.replace('-', '_'),
  405. property(fget=getter)
  406. )
  407. self.config.enable_service(service.service_type)
  408. def authorize(self):
  409. """Authorize this Connection
  410. .. note:: This method is optional. When an application makes a call
  411. to any OpenStack service, this method allows you to request
  412. a token manually before attempting to do anything else.
  413. :returns: A string token.
  414. :raises: :class:`~openstack.exceptions.HttpException` if the
  415. authorization fails due to reasons like the credentials
  416. provided are unable to be authorized or the `auth_type`
  417. argument is missing, etc.
  418. """
  419. try:
  420. return self.session.get_token()
  421. except keystoneauth1.exceptions.ClientException as e:
  422. raise exceptions.raise_from_response(e.response)
  423. @property
  424. def _pool_executor(self):
  425. if not self.__pool_executor:
  426. self.__pool_executor = concurrent.futures.ThreadPoolExecutor(
  427. max_workers=5)
  428. return self.__pool_executor
  429. def close(self):
  430. """Release any resources held open."""
  431. if self.__pool_executor:
  432. self.__pool_executor.shutdown()
  433. def set_global_request_id(self, global_request_id):
  434. self._global_request_id = global_request_id
  435. def __enter__(self):
  436. return self
  437. def __exit__(self, exc_type, exc_value, traceback):
  438. self.close()