Unified SDK for OpenStack
Go to file
Monty Taylor dffe0f0463
Add ability to register non-official services
openstacksdk is in the business of being an SDK for OpenStack. While
it's tempting to support other services that people might want to run
alongside their OpenStack cloud and register in their keystone catalog,
doing so is just not feasible.

At the same time, the 95% case is that the openstacksdk will be used for
OpenStack services, so using entrpoints-based plugin loading as part of
normal usage incurs a startup cost that can be rather high (it's based
on the number of python packages installed on the system, not the number
of plugins for openstacksdk)

Add a method and a constructor parameter to Connection that allows
people to programmatically enable support for additional non-OpenStack
services. This introduce a new Service description class that maps
service_type, Proxy class and optional service_type aliases. A subclass
of Service could be provided by whoever is writing the Proxy class and
associated Resoure objects, or the base class can be instantiated with
type and proxy_class as arguments.

While doing this, rework the loading of the official OpenStack services
to use the Service descriptor objects. Add an OpenStackService subclass
which does the importlib searching of the openstack module tree for
proxy classes. This gets all of the module searching and loading into
the openstack.services module and out of Connection.

This should allow us to delete the metric service from the tree but
provide people who want to use the metric service with openstacksdk a
mechanism to do so. It also should provide a vehicle for people
developing new not-yet-official services to develop their Resource and
Proxy classes out of tree, and then move them in once they are official

Change-Id: I6d1e0c45026a2e7b3c42983df9c0565b1c501bc3
2018-01-16 14:46:47 -06:00
devstack Merge tox, tests and other support files 2017-10-04 14:51:08 -05:00
doc Rationalize logging helpers and docs 2018-01-16 14:46:44 -06:00
examples Rationalize logging helpers and docs 2018-01-16 14:46:44 -06:00
extras Remove python-ironicclient 2017-12-10 07:34:21 -06:00
openstack Add ability to register non-official services 2018-01-16 14:46:47 -06:00
playbooks/devstack Remove use of tox-siblings role 2017-11-29 15:39:24 -06:00
releasenotes Add ability to register non-official services 2018-01-16 14:46:47 -06:00
tools Avoid tox_install.sh for constraints support 2017-12-01 08:58:30 +01:00
.coveragerc Fix coverage configuration and execution 2016-03-14 07:34:53 +00:00
.gitignore Merge tox, tests and other support files 2017-10-04 14:51:08 -05:00
.gitreview Update .gitreview for new namespace 2015-10-17 22:37:27 +00:00
.mailmap Merge tox, tests and other support files 2017-10-04 14:51:08 -05:00
.stestr.conf Merge shade and os-client-config into the tree 2017-11-15 09:03:23 -06:00
.zuul.yaml Re-enable octavia functional tests 2018-01-11 21:14:39 +00:00
babel.cfg setting up the initial layout; move the api proposals to api_strawman 2014-01-24 22:58:25 -06:00
bindep.txt Fix py35 and pypy tox env 2017-11-30 17:56:21 -06:00
CONTRIBUTING.rst Merge shade and os-client-config into the tree 2017-11-15 09:03:23 -06:00
create_yaml.sh Fix the network quota tests 2017-02-19 09:46:52 -07:00
docs-requirements.txt Add requirements.txt file for readthedocs 2015-05-21 08:16:44 -07:00
HACKING.rst Merge shade and os-client-config into the tree 2017-11-15 09:03:23 -06:00
LICENSE setting up the initial layout; move the api proposals to api_strawman 2014-01-24 22:58:25 -06:00
MANIFEST.in setting up the initial layout; move the api proposals to api_strawman 2014-01-24 22:58:25 -06:00
post_test_hook.sh Update load_balancer for v2 API 2017-07-18 18:05:29 -07:00
README.rst Rationalize logging helpers and docs 2018-01-16 14:46:44 -06:00
requirements.txt Updated from global requirements 2017-12-05 16:53:32 +00:00
setup.cfg Merge tox, tests and other support files 2017-10-04 14:51:08 -05:00
setup.py Updated from global requirements 2017-03-02 11:55:11 +00:00
SHADE-MERGE-TODO.rst Rationalize logging helpers and docs 2018-01-16 14:46:44 -06:00
test-requirements.txt Remove legacy client factory functions 2018-01-09 16:35:46 -06:00
tox.ini Update for new docs PTI 2018-01-05 12:44:02 -06:00

openstacksdk

openstacksdk is a client library for for building applications to work with OpenStack clouds. The project aims to provide a consistent and complete set of interactions with OpenStack's many services, along with complete documentation, examples, and tools.

It also contains an abstraction interface layer. Clouds can do many things, but there are probably only about 10 of them that most people care about with any regularity. If you want to do complicated things, the per-service oriented portions of the SDK are for you. However, if what you want to be able to write an application that talks to clouds no matter what crazy choices the deployer has made in an attempt to be more hipster than their self-entitled narcissist peers, then the Cloud Abstraction layer is for you.

A Brief History

openstacksdk started its life as three different libraries: shade, os-client-config and python-openstacksdk.

shade started its life as some code inside of OpenStack Infra's nodepool project, and as some code inside of the Ansible OpenStack Modules. Ansible had a bunch of different OpenStack related modules, and there was a ton of duplicated code. Eventually, between refactoring that duplication into an internal library, and adding the logic and features that the OpenStack Infra team had developed to run client applications at scale, it turned out that we'd written nine-tenths of what we'd need to have a standalone library.

Because of its background from nodepool, shade contained abstractions to work around deployment differences and is resource oriented rather than service oriented. This allows a user to think about Security Groups without having to know whether Security Groups are provided by Nova or Neutron on a given cloud. On the other hand, as an interface that provides an abstraction, it deviates from the published OpenStack REST API and adds its own opinions, which may not get in the way of more advanced users with specific needs.

os-client-config was a library for collecting client configuration for using an OpenStack cloud in a consistent and comprehensive manner, which introduced the clouds.yaml file for expressing named cloud configurations.

python-openstacksdk was a library that exposed the OpenStack APIs to developers in a consistent and predictable manner.

After a while it became clear that there was value in both the high-level layer that contains additional business logic and the lower-level SDK that exposes services and their resources faithfully and consistently as Python objects.

Even with both of those layers, it is still beneficial at times to be able to make direct REST calls and to do so with the same properly configured Session from python-requests.

This led to the merge of the three projects.

The original contents of the shade library have been moved into openstack.cloud and os-client-config has been moved in to openstack.config. The next release of shade will be a thin compatibility layer that subclasses the objects from openstack.cloud and provides different argument defaults where needed for compat. Similarly the next release of os-client-config will be a compat layer shim around openstack.config.

Note

The openstack.cloud.OpenStackCloud object and the openstack.connection.Connection object are going to be merged. It is recommended to not write any new code which consumes objects from the openstack.cloud namespace until that merge is complete.

openstack

List servers using objects configured with the clouds.yaml file:

import openstack

# Initialize and turn on debug logging
openstack.enable_logging(debug=True)

# Initialize cloud
conn = openstack.connect(cloud='mordred')

for server in conn.compute.servers():
    print(server.to_dict())

openstack.config

openstack.config will find cloud configuration for as few as 1 clouds and as many as you want to put in a config file. It will read environment variables and config files, and it also contains some vendor specific default values so that you don't have to know extra info to use OpenStack

  • If you have a config file, you will get the clouds listed in it
  • If you have environment variables, you will get a cloud named envvars
  • If you have neither, you will get a cloud named defaults with base defaults

Sometimes an example is nice.

Create a clouds.yaml file:

clouds:
 mordred:
   region_name: Dallas
   auth:
     username: 'mordred'
     password: XXXXXXX
     project_name: 'shade'
     auth_url: 'https://identity.example.com'

Please note: openstack.config will look for a file called clouds.yaml in the following locations:

  • Current Directory
  • ~/.config/openstack
  • /etc/openstack

More information at https://developer.openstack.org/sdks/python/openstacksdk/users/config

openstack.cloud

Create a server using objects configured with the clouds.yaml file:

import openstack

# Initialize and turn on debug logging
openstack.enable_logging(debug=True)

# Initialize cloud
# Cloud configs are read with openstack.config
cloud = openstack.openstack_cloud(cloud='mordred')

# Upload an image to the cloud
image = cloud.create_image(
    'ubuntu-trusty', filename='ubuntu-trusty.qcow2', wait=True)

# Find a flavor with at least 512M of RAM
flavor = cloud.get_flavor_by_ram(512)

# Boot a server, wait for it to boot, and then do whatever is needed
# to get a public ip for it.
cloud.create_server(
    'my-server', image=image, flavor=flavor, wait=True, auto_ip=True)

Links