b7ea6c7150
SwiftService uploads large objects using a thread pool. (The pool defaults to 5 and we're not currently configuring it larger or smaller) Instead of using that, spin up upload threads on our own so that we can get rid of the swiftclient depend. A few notes: - We're using the new async feature of the Adapter wrapper, which rate limits at the _start_ of a REST call. This is sane as far as we can tell, but also might not be what someone is expecting. - We'll skip the thread pool uploader for objects that are smaller than the default max segment size. - In splitting the file into segments, we'd like to avoid reading all of the segments into RAM when we don't need to - so there is a file-like wrapper class which can be passed to requests. This implements a read-view of a portion of the file. In a pathological case, this could be slower due to disk seeking on the read side. However, let's go back and deal with buffering when we have a problem - I imagine that the REST upload will be the bottleneck long before the overhead of interleaved disk seeks will be. Change-Id: Id9258980d2e0782e4e3c0ac26c7f11dc4db80354 |
||
|---|---|---|
| devstack | ||
| doc/source | ||
| extras | ||
| releasenotes/notes | ||
| shade | ||
| .coveragerc | ||
| .gitignore | ||
| .gitreview | ||
| .mailmap | ||
| .testr.conf | ||
| bindep.txt | ||
| CONTRIBUTING.rst | ||
| HACKING.rst | ||
| LICENSE | ||
| MANIFEST.in | ||
| README.rst | ||
| requirements.txt | ||
| setup.cfg | ||
| setup.py | ||
| test-requirements.txt | ||
| tox.ini | ||
Introduction
shade is a simple client library for interacting with OpenStack clouds. The key word here is simple. Clouds can do many many many things - but there are probably only about 10 of them that most people care about with any regularity. If you want to do complicated things, you should probably use the lower level client libraries - or even the REST API directly. However, if what you want is to be able to write an application that talks to clouds no matter what crazy choices the deployer has made in an attempt to be more hipster than their self-entitled narcissist peers, then shade is for you.
shade started its life as some code inside of ansible. ansible has a bunch of different OpenStack related modules, and there was a ton of duplicated code. Eventually, between refactoring that duplication into an internal library, and adding logic and features that the OpenStack Infra team had developed to run client applications at scale, it turned out that we'd written nine-tenths of what we'd need to have a standalone library.
Example
Sometimes an example is nice. :
import shade
# Initialize and turn on debug logging
shade.simple_logging(debug=True)
# Initialize cloud
# Cloud configs are read with os-client-config
cloud = shade.openstack_cloud(cloud='mordred')
# Upload an image to the cloud
image = cloud.create_image(
'ubuntu-trusty', filename='ubuntu-trusty.qcow2', wait=True)
# Find a flavor with at least 512M of RAM
flavor = cloud.get_flavor_by_ram(512)
# Boot a server, wait for it to boot, and then do whatever is needed
# to get a public ip for it.
cloud.create_server(
'my-server', image=image, flavor=flavor, wait=True, auto_ip=True)