42 KiB
Everything you need to know about using the OpenStack API
This document contains a presentation in presentty format. If you want to walk through it like a presentation, install presentty and run:
presentty doc/source/user/multi-cloud-demo.rst
The content is hopefully helpful even if it's not being narrated, so it's being included in the shade docs.
Everything you need to know about using the OpenStack API
Who am I?
Monty Taylor
- OpenStack Infra Core
- irc: mordred
- twitter: @e_monty
This Talk is Free Software
What are we going to talk about?
- Overview of some terminology
- Discussion of SDKs
- 'Simple' example
- Step by step through in much more detail
- Auth
- The Catalog
- Service Types
- Version Discovery
- Actually doing things
- Microversions
Expected Ranting
- Fixed vs. Floating IPs
- Build your own images
- Boot-time scripting with cloud-init
Relevant Docs
- https://specs.openstack.org/openstack/api-wg/
- https://service-types.openstack.org/
- https://developer.openstack.org/api-ref/identity/
- https://developer.openstack.org/api-ref/compute/
- https://docs.openstack.org/os-client-config/latest/
- https://docs.openstack.org/keystoneauth/latest/using-sessions.html
Cloud Terminology
Let's define a few terms, so that we can use them with ease:
- cloud - logically related collection of services
- region - completely independent subset of a given cloud
- patron - human who has an account
- user - account on a cloud
- project - logical collection of cloud resources
- domain - collection of users and projects
Cloud Terminology Relationships
- A cloud has one or more regions
- A patron has one or more users
- A patron has one or more projects
- A cloud has one or more domains
- In a cloud with one domain it is named "default"
- Each patron may have their own domain
- Each user is in one domain
- Each project is in one domain
- A user has one or more roles on one or more projects
HTTP Sessions
- HTTP interactions are authenticated via keystone
- Authenticating returns a token
- An authenticated HTTP Session is shared across a region
Cloud Regions
A cloud region is the basic unit of REST interaction.
- A cloud has a service catalog
- The service catalog is returned in the token
- The service catalog lists endpoint for each service in each region
- A region is completely autonomous
Availability Zones
- If you have exposure to AWS - OpenStack Availability Zones are not what you think they are.
- OpenStack Availability Zones are tagged groupings of hypervisor hosts, storage nodes and network nodes.
Users, Projects and Domains
In clouds with multiple domains, project and user names are only unique within a region.
- Names require domain information for uniqueness. IDs do not.
- Providing domain information when not needed is fine.
- project_name requires project_domain_name or project_domain_id
- project_id does not
- username requires user_domain_name or user_domain_id
- user_id does not
Auth vs. Identity
- Both provided by the
keystone
service identity
refers to CRUD operations on users, projects, etc.auth
refers to a user getting a token and catalog
Keystone v2 vs. Keystone v3
- Two major versions of identity API
- v2 is deprecated/legacy
project
was calledtenant
in v2domain
did not exist in v2
Confused Yet?
Awesome
SDKs
REST
- You can always do direct REST
- Auth and discovery are a bit of a pain
- Once you have those, it's all pretty easy
python
First:
- python-openstackclient ... CLI tool
openstack
Five main libraries to be aware of:
- keystoneauth
- os-client-config
- shade
- python-openstacksdk
- python-*client
python-*client
- Per-service client libraries
- Avoid them. They're not written for you. They make things harder.
keystoneauth
- wrapper around python-requests that handles auth, catalog endpoint and version discovery
- "REST" library
os-client-config
- Library to handle client configuration
- Can create python REST clients for you
- Used under the covers by shade, python-openstacksdk and python-openstackclient
- Handles clouds.yaml files, environment variables and command line arguments
shade
- a task and end-user oriented Python library
- abstracts deployment differences
- designed for multi-cloud
- simple to use
- massive scale
- optional advanced features to handle 20k servers a day
- Initial logic/design extracted from nodepool
- Librified to re-use in Ansible
python-openstacksdk
- Object oriented interface to OpenStack REST APIs
- Just got adopted by the shade team
- Interface is going to break soon
- shade, python-openstacksdk and os-client-config are going to merge
- shade is safe to use - there will always be backwards compat for shade
Multi-Cloud Shade Example
import shade
# Initialize and turn on debug logging
=True)
shade.simple_logging(debug
for cloud_name, region_name in [
'my-vexxhost', 'ca-ymq-1'),
('my-citycloud', 'Buf1'),
('my-internap', 'ams01')]:
(# Initialize cloud
= shade.openstack_cloud(cloud=cloud_name, region_name=region_name)
cloud
# Upload an image to the cloud
= cloud.get_image('Fedora 24 (Cloud Edition) [2016-09-23]')
image
# Find a flavor with at least 512M of RAM
= cloud.get_flavor_by_ram(512)
flavor
# Boot a server, wait for it to boot, and then do whatever is needed
# to get a public ip for it.
cloud.create_server('my-server', image=image, flavor=flavor, wait=True, auto_ip=True)
It's Not Possible To Cover Everything That Does
Simpler Example
It still may not be possible to cover it all.
import shade
# Initialize and turn on debug logging
=True)
shade.simple_logging(debug
# Initialize cloud
= shade.openstack_cloud(cloud='vexxhost')
cloud
# Upload an image to the cloud
= cloud.get_image('Fedora 24 (Cloud Edition) [2016-09-23]')
image
# Find a flavor with at least 512M of RAM
= cloud.get_flavor_by_ram(512)
flavor
# Boot a server, wait for it to boot, and then do whatever is needed
# to get a public ip for it.
cloud.create_server('my-server', image=image, flavor=flavor, wait=True, auto_ip=True)
Get Some Stuff Installed
virtualenv venv
source venv/bin/activate
pip install shade
Essential pieces of Auth Information
- auth_url
- password
- (username + (user_domain_name or user_domain_id)) or user_id)
- (project_name + (project_domain_name or project_domain_id)) or project_id
- Note: It's important to know if you have a name or id for your user and
-
project.
I Lied
Essential pieces of Auth Information
- auth_url
- auth API version
- auth_type
If nobody tells you an auth_type, assume password
If nobody tells you an auth_url, look at
https://docs.openstack.org/os-client-config/latest/user/vendor-support.html
Exercise - Find Auth URL for known cloud
The cloud is Vexxhost.
Answer
Version Discovery
- Each OpenStack service has a 'Version Discovery' document at its
unversioned root URL. We call this the 'Unversioned Discovery Document'
- Except for swift. (pending some work with discovery caching)
- Each OpenStack service ALSO has a 'Version Discovery' document at the root of each Versioned URL. We call this the 'Versioned Discovery Document'
- Version Discovery documents should not require Authentication
- If they do on your cloud, your cloud is broken, please file a bug
Exercise #2 - Find Auth API Version from Auth URL
curl -i https://auth.vexxhost.net
Auth Version Discovery Response
{
"versions": {
"values": [
{
"status": "stable",
"updated": "2017-02-22T00:00:00Z",
"media-types": [
{
"base": "application/json",
"type": "application/vnd.openstack.identity-v3+json"
}
],
"id": "v3.8",
"links": [
{
"href": "https://auth.vexxhost.net/v3/",
"rel": "self"
}
]
},
Auth Version Discovery Response Part 2
- ::
-
- {
-
"status": "deprecated", "updated": "2016-08-04T00:00:00Z", "media-types": [ { "base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json" } ], "id": "v2.0", "links": [ { "href": "https://auth.vexxhost.net/v2.0/", "rel": "self" }, { "href": "https://docs.openstack.org/", "type": "text/html", "rel": "describedby" } ]
]
}
Versioned Discovery Document
Sometimes a cloud gives you a versioned auth_url.
curl -i https://auth.vexxhost.net/v3
Auth Version Discovery Resonse
{
"version": {
"status": "stable",
"updated": "2017-02-22T00:00:00Z",
"media-types": [
{
"base": "application/json",
"type": "application/vnd.openstack.identity-v3+json"
}
],
"id": "v3.8",
"links": [
{
"href": "https://auth.vexxhost.net/v3/",
"rel": "self"
}
]
}
}
Finding a Better Version
Sometimes a cloud gives you a v2 versioned auth_url.
curl -i https://auth.vexxhost.net/v2.0
ALWAYS prefer v3 auth if it's available.
Old Auth Version Discovery Resonse
{
"version": {
"status": "deprecated",
"updated": "2016-08-04T00:00:00Z",
"media-types": [
{
"base": "application/json",
"type": "application/vnd.openstack.identity-v2.0+json"
}
],
"id": "v2.0",
"links": [
{
"href": "https://auth.vexxhost.net/v2.0/",
"rel": "self"
},
{
"href": "https://docs.openstack.org/",
"type": "text/html",
"rel": "describedby"
}
]
}
}
Status Deprecated
Strip the trailing v.*
portion from the URL and try
again.
{
"version": {
"status": "deprecated",
Strip the trailing v.*
portion from the URL and try
again.
If v3
is not available, complain to someone. Keystone v3
api is considered stable as of February 20, 2013.
Isn't this fun?
To Domain or Not To Domain
In v3
:
- Project Names and User Names are only unique within a domain
- If you don't have domain info, assume
domain_id
isdefault
In v2
:
- No such thing as domains
- Projects are called "tenant"
Essential pieces of Auth Information
- auth_url - https://auth.vexxhost.net
- auth version -
3
- versioned auth endpoint - https://auth.vexxhost.net/v3
- username - osdn-user-1
- user domain id -
default
- project_name - osdn-project-1
- project domain id =
default
- password - XXXXXXXX
Now we can authenticate!!!
curl -g -i -X POST https://auth.vexxhost.net/v3/auth/tokens -H "Content-Type: application/json" -H "Accept: application/json" -d '{"auth": {"identity": {"methods": ["password"], "password": {"user": {"domain": {"id": "default"}, "name": "osdn-user-1", "password": "XXXXXXXX"}}},"scope": {"project": {"domain": {"id": "default"}, "name": "osdn-project-1"}}}}'
What, you can't read that?
curl -g -i -X POST https://auth.vexxhost.net/v3/auth/tokens \
-H "Content-Type: application/json" \
-H "Accept: application/json" -d '{
"auth": {
"identity": {
"methods": ["password"],
"password": {
"user": {
"domain": {"id": "default"},
"name": "osdn-user-1",
"password": "XXXXXXXX"}}},
"scope": {
"project": {
"domain": {"id": "default"},
"name": "osdn-project-1"}}}}'
Authentication Response
Readable version of payload: http://paste.openstack.org/raw/623947/
HTTP/1.1 201 Created
Date: Tue, 17 Oct 2017 23:16:05 GMT
Server: Apache/2.4.6 (CentOS)
X-Subject-Token: gAAAAABZ5o81tyDsyJMXl6hVcVOqHgzKQ2WmpvepCSr2NWYM1D97VoaBy-i5OLT8-BbkWBZJq_aqrt03eutp2eEpORI-V4CYpYcWBDkR5jLNfq2gD-TDB9lzvRo7rHKu-PWSLIxEF7whsFJ8bVGDPMN4yc_ZwgfWcR13GxAKutU9_rzR9Gvewjg
Vary: X-Auth-Token
x-openstack-request-id: req-29d8c4e6-1924-42e9-8efa-4f154e813ee6
Content-Length: 11696
Content-Type: application/json
{"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "fd6d2130b7784b82aa6b49fd5901f70b", "name": "Member"}], "expires_at": "2017-10-18T23:16:05.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "bbb1e479a9d04f34941470f37c7a0ecb", "name": "osdn-project-1"}, "catalog": [{"endpoints": [{"region_id": "ca-ymq-1", "url": "https://key-manager-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "522041f846934310b928a6afab47241e"}, {"region_id": "ca-ymq-1", "url": "https://key-manager-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "660a742372554b5581c1f4870d4d42cf"}, {"region_id": "ca-ymq-1", "url": "https://key-manager-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "c1c32be0de8441eea4306b8c22442ba3"}], "type": "key-manager", "id": "023de54c09654931856403e708b60a22", "name": "barbican"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://container-infra-ca-ymq-1.vexxhost.net/v1", "region": "ca-ymq-1", "interface": "public", "id": "289f7dcf096743fb9a17924bd598e716"}, {"region_id": "ca-ymq-1", "url": "https://container-infra-ca-ymq-1.vexxhost.net/v1", "region": "ca-ymq-1", "interface": "admin", "id": "61b6d283ef4b419a8ad23fc53dea70df"}, {"region_id": "ca-ymq-1", "url": "https://container-infra-ca-ymq-1.vexxhost.net/v1", "region": "ca-ymq-1", "interface": "internal", "id": "e1e7c79ce7834ec0aa74308326e2f96c"}], "type": "container-infra", "id": "041eac2489bb495b8a900c40b93b2065", "name": "magnum"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://orchestration-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "203fd4e466b44569aa9ab8c78ef55bad"}, {"region_id": "ca-ymq-1", "url": "https://orchestration-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "da3d087e0c724338ba12c9a1168ef80c"}, {"region_id": "ca-ymq-1", "url": "https://orchestration-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "f0a311cb5dbf4ae788670107a3433ac2"}], "type": "orchestration", "id": "0e852aec60e6412b88b066e62c2cd940", "name": "heat"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://object-storage-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "4f1fafb25e58421e91a2139dc0382c53"}, {"region_id": "ca-ymq-1", "url": "https://object-storage-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "5808ab12037f4ab49c37cba0ac9cd58e"}, {"region_id": "ca-ymq-1", "url": "https://object-storage-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "a680b7674b3b430c9f5e008f72f3dfe4"}], "type": "s3", "id": "13f4cf29c7394a7a801d8aeacca24378", "name": "swift_s3"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "http://dns.vexxhost.net/", "region": "ca-ymq-1", "interface": "internal", "id": "0bba70ede2ef4bc0a998ed73110406cb"}, {"region_id": "ca-ymq-1", "url": "http://dns.vexxhost.net/", "region": "ca-ymq-1", "interface": "public", "id": "25ee42ace1ad4707b485494baa5d4693"}, {"region_id": "ca-ymq-1", "url": "http://dns.vexxhost.net/", "region": "ca-ymq-1", "interface": "admin", "id": "cf817bfde96d4aa6807cdb4be21eca3b"}], "type": "dns", "id": "1b6455f4855e44bd85be449258340a02", "name": "designate"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://identity-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "8df18f47fcdc4c348d521d4724a5b7ac"}, {"region_id": "ca-ymq-1", "url": "https://auth.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "c0dd9f5f8db248b093d6735b167e1af6"}, {"region_id": "ca-ymq-1", "url": "https://auth.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "c8505f07c349413aa7cd61d42337af99"}], "type": "identity", "id": "1ca1ba37600a436185c317bef6d88c36", "name": "keystone"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://placement-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "299df431ac7942c18e2137613d8e03d8"}, {"region_id": "ca-ymq-1", "url": "https://placement-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "bec896cde16646d6a3b56cff997d08cc"}, {"region_id": "ca-ymq-1", "url": "https://placement-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "ce70c7029b6a45b78e008bc833187559"}], "type": "placement", "id": "37077d282f2b4961ad65f9fb87f58789", "name": "placement"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "4babd2f2b8d842e4b2303468fbf5d3ac"}, {"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "6b3ddcf427e04182a7aa4e24d4649581"}, {"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "e8e1eb90f7394f5999aec5c8f8c75c88"}], "type": "volume", "id": "40686c5446374c2b92771e67249521b3", "name": "cinder"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://image-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "218d9460bd6e4eabb32b980fc580a26a"}, {"region_id": "ca-ymq-1", "url": "https://image-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "21a2898a0fcf4ec783974cf75d774916"}, {"region_id": "ca-ymq-1", "url": "https://image-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "8842c03d2c51449ebf9ff36778cf17c1"}], "type": "image", "id": "49e5dc4cfe294aa0971e797d3bb1db20", "name": "glance"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://cloudformation-ca-ymq-1.vexxhost.net/v1", "region": "ca-ymq-1", "interface": "admin", "id": "83fbacf0d0014a27955384fb8632de4c"}, {"region_id": "ca-ymq-1", "url": "https://cloudformation-ca-ymq-1.vexxhost.net/v1", "region": "ca-ymq-1", "interface": "internal", "id": "9308a40f3a6f4dff9d20afa2dfa6b186"}, {"region_id": "ca-ymq-1", "url": "https://cloudformation-ca-ymq-1.vexxhost.net/v1", "region": "ca-ymq-1", "interface": "public", "id": "e5cb7e4e6f3440a386d397ec7e00ca44"}], "type": "cloudformation", "id": "4e77820be3494d26a0c39301082da4d7", "name": "heat-cfn"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://data-processing-ca-ymq-1.vexxhost.net/v1.1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "5fdfd118a3d6460ab45c1995ca21b7f1"}, {"region_id": "ca-ymq-1", "url": "https://data-processing-ca-ymq-1.vexxhost.net/v1.1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "ab9e70b4fde943e49d92c41d2f0892d9"}, {"region_id": "ca-ymq-1", "url": "https://data-processing-ca-ymq-1.vexxhost.net/v1.1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "cfa16dc27b82420a8cf7db5ea79aa3dd"}], "type": "data-processing", "id": "523b8e9f4bd44e51aec65c524c953c32", "name": "sahara"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://network-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "37860b492dd947daa738f461b9084d2a"}, {"region_id": "ca-ymq-1", "url": "https://network-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "7a095734e4984cc7b8ac581aa6131f23"}, {"region_id": "ca-ymq-1", "url": "https://network-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "96357df3d6694477b0ad17fef6091210"}], "type": "network", "id": "98f33898880441ddb91522bac15410b5", "name": "neutron"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v2/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "01fdd8e07ca74c9daf80a8b66dcc8bf6"}, {"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v2/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "a25efaf48347441a8d36ce302f31d527"}, {"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v2/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "ceea6017e14e4b549882b6fe359d07e0"}], "type": "volumev2", "id": "9ec8ed2478e44b368f51dd71a6f2759c", "name": "cinderv2"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://compute-ca-ymq-1.vexxhost.net/v2.1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "2f582f99db974766af7548dda56c3b50"}, {"region_id": "ca-ymq-1", "url": "https://compute-ca-ymq-1.vexxhost.net/v2.1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "39e28fa99b69466195aad1fbfb610323"}, {"region_id": "ca-ymq-1", "url": "https://compute-ca-ymq-1.vexxhost.net/v2.1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "4d38fa91197e4712a2f2d3f89fcd7dad"}], "type": "compute", "id": "a30d9b8a170144f0a7bca3ce028c9ed3", "name": "nova"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v3/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "5d9aa667f2fb4c83bf14ee8ae52b541a"}, {"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v3/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "96bcaf12b54d4060bc9973ded5ce4093"}, {"region_id": "ca-ymq-1", "url": "https://block-storage-ca-ymq-1.vexxhost.net/v3/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "ac9138596d3942babe1f9241f93a5e04"}], "type": "volumev3", "id": "a5ae83af15b146559711905c6531c721", "name": "cinderv3"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://octavia-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "internal", "id": "39830aba2eea44a3a432ea0bd219c1ab"}, {"region_id": "ca-ymq-1", "url": "https://octavia-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "public", "id": "5883ae0fb85842c5aca05ed51327ac35"}, {"region_id": "ca-ymq-1", "url": "https://octavia-ca-ymq-1.vexxhost.net", "region": "ca-ymq-1", "interface": "admin", "id": "80bd3d21df274c43ac8457aa2230598f"}], "type": "octavia", "id": "b6a1520b90a146f4a96321960e7a243a", "name": "octavia"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "http://secure.vexxhost.com/console/api", "region": "ca-ymq-1", "interface": "public", "id": "156bf3730b834d2bacf7eff6c542f394"}], "type": "dashboard", "id": "b7bb9d5476c649f483dfbcb49359c17b", "name": "cloudconsole"}, {"endpoints": [{"region_id": "ca-ymq-1", "url": "https://object-storage-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "admin", "id": "20b24181722b49a3983d17d42147a22c"}, {"region_id": "ca-ymq-1", "url": "https://object-storage-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "public", "id": "79fa33ff42ec45118ae8b36789fcb8ae"}, {"region_id": "ca-ymq-1", "url": "https://object-storage-ca-ymq-1.vexxhost.net/v1/bbb1e479a9d04f34941470f37c7a0ecb", "region": "ca-ymq-1", "interface": "internal", "id": "b073b767f10d44f895d9d14fbc3e3d6b"}], "type": "object-store", "id": "c1efec5454ec49b1b016721938401182", "name": "swift"}], "user": {"password_expires_at": null, "domain": {"id": "default", "name": "Default"}, "id": "bc2260d5b7cb40ff8270043fd7af3a08", "name": "osdn-user-1"}, "audit_ids": ["6bu2KObmSwGfO2gsIcmPNw"], "issued_at": "2017-10-17T23:16:05.000000Z"}}
Authentication Response Readable Version
http://paste.openstack.org/raw/623947/
Listing Images
- Find 'image' Endpoint from the Catalog
- Find image api version
- Make API call
Consuming Service Catalog
https://specs.openstack.org/openstack/api-wg/guidelines/consuming-catalog.html
Docs on version discovery and cloud profile:
- https://review.openstack.org/#/c/459869 Add guideline describing a cloud profile document
- https://review.openstack.org/#/c/459710 Add guidelines on Version Discovery
- https://review.openstack.org/#/c/459405 Add document describing consuming version discovery
Find Endpoint
- Most important piece of info:
service_type
(type
in catalog) service_name
or name carries no meaning
Service Types Authority
https://service-types.openstack.org
- Registry of known service types
- Listing of historical aliases for those service types
- https://service-types.openstack.org/service-types.json
- Not release versoined - most current file is always accurate
- Python library: os-service-types
Find Image Endpoint
{'endpoints': [
{'id': '218d9460bd6e4eabb32b980fc580a26a',
'interface': 'internal',
'region': 'ca-ymq-1',
'region_id': 'ca-ymq-1',
'url': 'https://image-ca-ymq-1.vexxhost.net'},
{'id': '21a2898a0fcf4ec783974cf75d774916',
'interface': 'admin',
'region': 'ca-ymq-1',
'region_id': 'ca-ymq-1',
'url': 'https://image-ca-ymq-1.vexxhost.net'},
{'id': '8842c03d2c51449ebf9ff36778cf17c1',
'interface': 'public',
'region': 'ca-ymq-1',
'region_id': 'ca-ymq-1',
'url': 'https://image-ca-ymq-1.vexxhost.net'}],
'id': '49e5dc4cfe294aa0971e797d3bb1db20',
'name': 'glance',
'type': 'image'},
Endpoint Interface
- public - Public/External API access - default value
- internal - Internal network API access
- admin - Historical - was used for keystone - not used anymore
Endpoint Region Name
- If there is more than one region, region_name must be specified
- If there is only one, it can be omitted
Example with more than one region:
http://paste.openstack.org/raw/623900/
Endpoints Found
- cloud=vexxhost1,service_type=image
https://image-ca-ymq-1.vexxhost.net
- cloud=citycloud1,service_type=image,region_name=Sto2
https://sto2.citycloud.com:9292
Finding Image API Version
curl -i https://image-ca-ymq-1.vexxhost.net
http://paste.openstack.org/raw/623901/
- CURRENT - Best current version
https://image-ca-ymq-1.vexxhost.net/v2/
Listing Images
- Send value of X-Subject-Token from auth request in X-Auth-Token
curl -i -X GET https://image-ca-ymq-1.vexxhost.net/v2/images \
-H "Accept: application/json" \
-H "X-Auth-Token: gAAAAABZ5qpnytYif9wk-4fsqvgm3Ik81WQl_L4tdlfka7Vr4b-EG0w7QGF-JU_T4zkJsuql4XYSU_9epUnx-aE_UeKMMdaKAyecGG9om6TR3CA6yeeHbMJBTVFfdxvW-QFA76_JXoibCvXGIcKlpotK01BBl3DG5qYNxUFkSquk5cIQh-w63AY"
HTTP/1.1 200 OK
Content-Length: 24153
Content-Type: application/json
X-Openstack-Request-Id: req-8a602280-1bba-46b4-9406-97addcdd9b6c
Date: Wed, 18 Oct 2017 01:12:43 GMT
- Content: http://paste.openstack.org/raw/623902/
- Note pagination links
Getting Auth information
- Download a clouds.yaml file from horizon
- Download an legacy openrc file from horizon
- Investigate dashboard
- Carrier pigeon
Sometimes the files are wrong and you have to apply the earlier rules.
Incorrect openrc file (horizon bug)
This is downloaded from my account. Who can spot the problems?
export OS_AUTH_URL=https://auth.vexxhost.net/v2.0
export OS_PROJECT_ID=bbb1e479a9d04f34941470f37c7a0ecb
export OS_PROJECT_NAME="osdn-project-1"
export OS_USER_DOMAIN_NAME="None"
export OS_USERNAME="osdn-user-1"
export OS_REGION_NAME="ca-ymq-1"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
Put the info in a clouds.yaml file
clouds.yaml
Information about the clouds you want to connect to is stored in a file called clouds.yaml.
clouds.yaml can be in your homedir: ~/.config/openstack/clouds.yaml or system-wide: /etc/openstack/clouds.yaml.
Information in your homedir, if it exists, takes precedence.
Full docs on clouds.yaml are at https://docs.openstack.org/developer/os-client-config/
What about Mac and Windows?
USER_CONFIG_DIR is different on Linux, OSX and Windows.
- Linux: ~/.config/openstack
- OSX: ~/Library/Application Support/openstack
- Windows: C:UsersUSERNAMEAppDataLocalOpenStackopenstack
SITE_CONFIG_DIR is different on Linux, OSX and Windows.
- Linux: /etc/openstack
- OSX: /Library/Application Support/openstack
- Windows: C:ProgramDataOpenStackopenstack
Config Terminology
For multi-cloud, think of two types:
- profile - Facts about the cloud that are true for everyone
- cloud - Information specific to a given user
Apologies for the use of cloud twice.
Example clouds.yaml file
clouds:
vexxhost1:
identity_api_version: 3
auth:
auth_url: https://auth.vexxhost.net
user_domain_id: default
project_domain_id: default
project_name: osdn-project-1
username: osdn-user-1
password: XXXXXXXX
Vexxhost is known
clouds:
vexxhost1:
identity_api_version: 3
profile: vexxhost
auth:
user_domain_id: default
project_domain_id: default
project_name: osdn-project-1
username: osdn-user-1
password: XXXXXXXX
More than one cloud
Yes- I created the same user with the same password on CityCloud:
clouds:
citycloud1:
profile: citycloud
auth:
username: osdn-user-1
password: XXXXXXXX
project_name: osdn-project-1
user_domain_id: d0919bd5e8d74e49adf0e145807ffc38
project_domain_id: d0919bd5e8d74e49adf0e145807ffc38
vexxhost1:
profile: vexxhost
auth:
user_domain_id: default
project_domain_id: default
project_name: osdn-project-1
username: osdn-user-1
password: XXXXXXXX
Quick note about SSL
You can disable SSL verification:
unitedstack:
profile: unitedstack
verify: False
You can set specific SSL arguments:
openstackci-infracloud-vanilla:
region_name: RegionOne
cacert: /home/nodepool/.config/openstack/infracloud_vanilla_cacert.pem
auth:
auth_url: https://controller00.vanilla.ic.openstack.org:5000
username: openstackci
Auth per cloud, select per region
In general, the thing you need to know is:
- Configure authentication per cloud
- Select config to use by cloud and region
Environment Variables and Simple Usage
- Environment variables starting with OS_ go into a cloud called envvars
- If you only have one cloud, you don't have to specify it
- OS_CLOUD and OS_REGION_NAME are default values for cloud and region_name
Easier version of the above REST
- What you should be able to do
import os_client_config
= os_client_config.make_rest_client('image', cloud='vexxhost1')
client print(client.get('/images').json())
- There's a bug with make_rest_client and glance - I'll get it fixed this week. Workaround:
import os_client_config
from keystoneauth1 import adapter
= os_client_config.OpenStackConfig().get_one_cloud(cloud='vexxhost1')
config = adapter.Adapter(
client ='image', version='latest')
config.get_session(), service_typeprint(client.get('/images').json())
Sessions and Adapters
- A Session is an Authenticated HTTP connection to a cloud
= os_client_config.OpenStackConfig().get_one_cloud(cloud='vexxhost1')
config = config.get_session()
session 'https://image-ca-ymq-1.vexxhost.net/v2/images') session.get(
- An "Adapter" is a Session that is 'mounted' on a base endpoint
= os_client_config.OpenStackConfig().get_one_cloud(cloud='vexxhost1')
config = adapter.Adapter(
client ='image', version='latest')
config.get_session(), service_type'/images') session.get(
os_client_config.make_rest_client
returns a mounted Adapter- Latest keystoneauth library handles version discovery properly in Adapter
Creating keystoneauth session directly
from keystoneauth1 import adapter, loading, session
= loading.get_plugin_loader('password')
loader = loader.load_from_options(
auth ='https://auth.vexxhost.net',
auth_url='osdn-user-1',
username='default',
user_domain_id='XXXXXXXX',
password='osdn-project-1',
project_name='default')
project_domain_id# verify and ssl cert info go here too
= session.Session(auth)
session # interface, region_name also go here
= adapter.Adapter(session, service_type='image', version='latest')
client '/images') client.get(
Command Line
openstack --os-cloud=vexxhost1 image list
Back to shade
import shade
= shade.openstack_cloud(cloud='vexxhost')
cloud cloud.pprint(cloud.list_images())
Shade Logging
- shade uses standard python logging
- simple_logging does easy defaults
- Squelches some meaningless warnings
debug
- Logs shade loggers at debug level
http_debug Implies debug, turns on HTTP tracing
# Initialize and turn on debug logging
=True) shade.simple_logging(debug
Find out what REST calls shade is making
import shade
=True)
shade.simple_logging(http_debug= shade.openstack_cloud(cloud='vexxhost1')
cloud cloud.pprint(cloud.list_images())
http://paste.openstack.org/raw/623906/
Real tokens are not logged, only a sha of the token. Reusing the printed curl commands reuquires replacing the token with a real one.
"X-Auth-Token: {SHA1}23ffd369ff0c0e7673c17bb97e6a77783d789e5e"
Microversions
- Per-HTTP-request behavior differences.
- Services declare microversion support ranges in Version Discovery Document
curl -i https://compute-ca-ymq-1.vexxhost.net
Nova Compute Version Document Part One
HTTP/1.1 200 OK
Date: Wed, 18 Oct 2017 01:58:51 GMT
Server: Apache/2.4.6 (CentOS)
Content-Length: 399
Vary: Accept-Encoding
Content-Type: application/json
{
"versions": [
{
"status": "SUPPORTED",
"version": "",
"updated": "2011-01-21T11:33:21Z",
"id": "v2.0",
"links": [
{
"href": "https://compute-ca-ymq-1.vexxhost.net/v2/",
"rel": "self"
}
],
"min_version": ""
},
Nova Compute Version Document Part Two
Note 'version' and 'min_version'.
{
"status": "CURRENT",
"version": "2.53",
"updated": "2013-07-23T11:33:21Z",
"id": "v2.1",
"links": [
{
"href": "https://compute-ca-ymq-1.vexxhost.net/v2.1/",
"rel": "self"
}
],
"min_version": "2.1"
}
Microversion Field Names
The min and max available microversion for the given Major API version.
In some services they are:
- min_version
- version
In other services they are:
- min_version
- max_version
Request a Microversion
Add a OpenStack-API-Version header with service-type and version:
OpenStack-API-Version: compute 2.13
Response will contain the same header specifying which version was used:
OpenStack-API-Version: compute 2.13
Older versions of Nova (pre Newton - or versions that do not support a max version of 2.27 or higher), use a different form:
X-OpenStack-Nova-API-Version: 2.13
It doesn't hurt anything to send both - which is what keystoneauth does under the covers.
Documentation on Nova Microversion Changes
https://docs.openstack.org/nova/latest/reference/api-microversion-history.html
Simple Microversion Example with Default
Microversion 2.36 removed some calls. Before 2.36 nova had a
/images
API that would proxy to glance. It is removed in
2.36 and later.
Making a GET call defaults to the minimum (2.1 in this case)
import os_client_config
= os_client_config.make_rest_client('compute', cloud='vexxhost')
client '/images') client.get(
curl -g -i -X GET https://compute-ca-ymq-1.vexxhost.net/v2.1/db92b20496ae4fbda850a689ea9d563f/images -H "X-Auth-Token: {SHA1}23ffd369ff0c0e7673c17bb97e6a77783d789e5e" -H "User-Agent: os-client-config/1.28.1 keystoneauth1/3.2.1 python-requests/2.18.3 CPython/3.5.2"
[200] Date: Wed, 18 Oct 2017 02:10:49 GMT Server: Apache/2.4.6 (CentOS) OpenStack-API-Version: compute 2.1 X-OpenStack-Nova-API-Version: 2.1 Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding x-openstack-request-id: req-3e6bc318-dfd6-493b-9675-8bb1f9285cac x-compute-request-id: req-3e6bc318-dfd6-493b-9675-8bb1f9285cac Content-Encoding: gzip Content-Length: 2411 Content-Type: application/json
Simple Microversion Example with Microversion
Sending microversion 2.36 for the same call tells Nova to use the 2.36 behavior, which results in a 404.
import os_client_config
= os_client_config.make_rest_client('compute', cloud='vexxhost')
client '/images', microversion='2.36') client.get(
curl -g -i -X GET https://compute-ca-ymq-1.vexxhost.net/v2.1/db92b20496ae4fbda850a689ea9d563f/images -H "User-Agent: os-client-config/1.28.1 keystoneauth1/3.2.1 python-requests/2.18.3 CPython/3.5.2" -H "X-Auth-Token: {SHA1}23ffd369ff0c0e7673c17bb97e6a77783d789e5e" -H "OpenStack-API-Version: compute 2.36" -H "X-OpenStack-Nova-API-Version: 2.36"
[404] Date: Wed, 18 Oct 2017 02:11:00 GMT Server: Apache/2.4.6 (CentOS) OpenStack-API-Version: compute 2.36 X-OpenStack-Nova-API-Version: 2.36 Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version x-openstack-request-id: req-d733583e-d2c3-4ca4-bdfb-5e658e194028 x-compute-request-id: req-d733583e-d2c3-4ca4-bdfb-5e658e194028 Content-Length: 78 Content-Type: application/json; charset=UTF-8
{"itemNotFound": {"message": "The resource could not be found.", "code": 404}}
Create Server Group
The server group doesn't matter - this is just to have an object to show.
import os_client_config
= os_client_config.make_rest_client('compute', cloud='vexxhost')
client
client.post('/os-server-groups',
=dict(server_group=dict(name='test', policies=['anti-affinity']))) json
Example of No Microversion Content Behavior
No microversion specified.
'/os-server-groups') client.get(
curl -g -i -X GET https://compute-ca-ymq-1.vexxhost.net/v2.1/bbb1e479a9d04f34941470f37c7a0ecb/os-server-groups -H "X-Auth-Token: {SHA1}fe541768b55bbe23f79f97595b137fed3326166a" -H "User-Agent: os-client-config/1.28.1 keystoneauth1/3.2.1 python-requests/2.18.3 CPython/3.5.2"
[200] Date: Wed, 18 Oct 2017 02:18:30 GMT Server: Apache/2.4.6 (CentOS) OpenStack-API-Version: compute 2.1 X-OpenStack-Nova-API-Version: 2.1 Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding x-openstack-request-id: req-85e061c0-d474-497c-bd17-96b629086169 x-compute-request-id: req-85e061c0-d474-497c-bd17-96b629086169 Content-Encoding: gzip Content-Length: 137 Content-Type: application/json
{"server_groups": [{"members": [], "metadata": {}, "id": "f88c7aef-b34a-4b0e-b0c7-9076eab88de2", "policies": ["anti-affinity"], "name": "test"}]}
Example of Microversion Content Behavior
Request microversion 2.13, which adds user_id and project_id.
'/os-server-groups', microversion='2.13').json() client.get(
curl -g -i -X GET https://compute-ca-ymq-1.vexxhost.net/v2.1/bbb1e479a9d04f34941470f37c7a0ecb/os-server-groups -H "User-Agent: os-client-config/1.28.1 keystoneauth1/3.2.1 python-requests/2.18.3 CPython/3.5.2" -H "X-Auth-Token: {SHA1}fe541768b55bbe23f79f97595b137fed3326166a" -H "OpenStack-API-Version: compute 2.13" -H "X-OpenStack-Nova-API-Version: 2.13"
[200] Date: Wed, 18 Oct 2017 02:20:13 GMT Server: Apache/2.4.6 (CentOS) OpenStack-API-Version: compute 2.13 X-OpenStack-Nova-API-Version: 2.13 Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding x-openstack-request-id: req-6c71b008-185a-46c1-8762-5c2718be1741 x-compute-request-id: req-6c71b008-185a-46c1-8762-5c2718be1741 Content-Encoding: gzip Content-Length: 195 Content-Type: application/json
{"server_groups": [{"user_id": "bc2260d5b7cb40ff8270043fd7af3a08", "policies": ["anti-affinity"], "name": "test", "members": [], "project_id": "bbb1e479a9d04f34941470f37c7a0ecb", "id": "f88c7aef-b34a-4b0e-b0c7-9076eab88de2", "metadata": {}}]}