Adding autogenerated saharaclient API docs

Saharaclient API docs now can be autogenerated, that
prevents them from manual update. Also docstrings
were added to describe some unclear parameters.

Change-Id: Ibf9fdf332c91cdfb18d6f8ec5429a39dc2862494
This commit is contained in:
Andrey Pavlov
2015-11-12 19:20:02 +03:00
parent 1aaea83b02
commit 8a5a16f1d6
13 changed files with 190 additions and 55 deletions

View File

@@ -4,33 +4,26 @@ Sahara Client
Overview
--------
Sahara Client provides a list of Python interfaces to communicate with the Sahara REST API.
Sahara Client enables users to perform most of the existing operations like retrieving template lists,
creating Clusters, submitting EDP Jobs, etc.
Sahara Client provides a list of Python interfaces to communicate with the
Sahara REST API. Sahara Client enables users to perform most of the existing
operations like retrieving template lists, creating Clusters, submitting EDP
Jobs, etc.
Instantiating a Client
----------------------
To start using the Sahara Client users have to create an instance of the `Client` class.
The client constructor has a list of parameters to authenticate and locate Sahara endpoint.
To start using the Sahara Client users have to create an instance of the
`Client` class. The client constructor has a list of parameters to authenticate
and locate Sahara endpoint.
* auth_url - Keystone URL that will be used for authentication.
* sahara_url - Sahara REST API URL to communicate with.
* service_type - Sahara service name in Keystone catalog. (Default: data-processing)
* endpoint_type - Desired Sahara endpoint type. (Default: publicURL)
* username - Username for Keystone authentication.
* api_key - Password for Keystone authentication.
* project_name - Keystone Tenant name.
* project_id - Keystone Tenant id.
* input_auth_token - Keystone authorization token.
* insecure - Allow insecure.
* auth - Keystone Authentication Plugin object.
* session - Keystone Session object.
.. autoclass:: saharaclient.api.client.Client
:members:
**Important!**
It is not a mandatory rule to provide all of the parameters above. The minimum number should be enough
to determine Sahara endpoint, check user authentication and tenant to operate in.
It is not a mandatory rule to provide all of the parameters above. The minimum
number should be enough to determine Sahara endpoint, check user
authentication and tenant to operate in.
Authentication check
~~~~~~~~~~~~~~~~~~~~
@@ -61,9 +54,10 @@ For more information about Keystone Sessions, see `Using Sessions`_.
Sahara endpoint discovery
~~~~~~~~~~~~~~~~~~~~~~~~~
If user has a direct URL pointing to Sahara REST API, it may be specified as `sahara_url`.
If this parameter is missing, Sahara client will use Keystone Service Catalog to find the endpoint.
There are two parameters: `service_type` and `endpoint_type` to configure endpoint search. Both parameters have
If user has a direct URL pointing to Sahara REST API, it may be specified as
`sahara_url`. If this parameter is missing, Sahara client will use Keystone
Service Catalog to find the endpoint. There are two parameters: `service_type`
and `endpoint_type` to configure endpoint search. Both parameters have
default values.
.. sourcecode:: python
@@ -99,78 +93,75 @@ Sahara Client has a list of fields to operate with:
* job_executions
* job_types
Each of this fields is a reference to a Manager for a corresponding group of REST calls.
Each of this fields is a reference to a Manager for a corresponding group of
REST calls.
Supported operations
--------------------
The following methods are allowed for all Managers:
Plugin ops
~~~~~~~~~~
* list() - Get a list of all objects of specified type.
* get(id) - Get an object by id. (In case of plugins, the Plugin name should be provided)
* delete(id) - Delete an object by id. (Available only for templates and EDP objects)
Plugin Manager ops
~~~~~~~~~~~~~~~~~~
* get_version_details(plugin_name, hadoop_version) - Get the list of Services and Service Parameters for a specified Plugin and Hadoop Version
* convert_to_cluster_template(plugin_name, hadoop_version, template_name, filecontent) - This call is used to create Cluster Templates directly, avoiding Cluster Template mechanism.
.. autoclass:: saharaclient.api.plugins.PluginManager
:members:
Image Registry ops
~~~~~~~~~~~~~~~~~~
* update_image(image_id, user_name, descr) - Create or update an Image in Image Registry.
* unregister_image(image_id) - Remove an Image from Sahara Image Registry.
* update_tags(image_id, new_tags) - Updates Image tags. `new_tags` list will replace currently assigned tags.
.. autoclass:: saharaclient.api.images.ImageManager
:members:
Node Group Template ops
~~~~~~~~~~~~~~~~~~~~~~~
* create(name, plugin_name, hadoop_version, flavor_id, description, volumes_per_node, volumes_size, node_processes, node_configs, floating_ip_pool, security_groups, auto_security_group, availability_zone, volumes_availability_zone, volume_type, image_id, is_proxy_gateway, volume_local_to_instance, use_autoconfig, shares, is_public, is_protected) - Create a Node Group Template with specified parameters.
* update(ng_template_id, name, plugin_name, hadoop_version, flavor_id, description, volumes_per_node, volumes_size, node_processes, node_configs, floating_ip_pool, security_groups, auto_security_group, availability_zone, volumes_availability_zone, volume_type, image_id, is_proxy_gateway, volume_local_to_instance, use_autoconfig, shares, is_public, is_protected) - Update a Node Group Template with specified parameters.
.. autoclass:: saharaclient.api.node_group_templates.NodeGroupTemplateManager
:members:
Cluster Template ops
~~~~~~~~~~~~~~~~~~~~
* create(name, plugin_name, hadoop_version, description, cluster_configs, node_groups, anti_affinity, net_id, default_image_id, use_autoconfig, shares, is_public, is_protected) - Create a Cluster Template with specified parameters.
* update(cluster_template_id, name, plugin_name, hadoop_version, description, cluster_configs, node_groups, anti_affinity, net_id, default_image_id, use_autoconfig, shares, is_public, is_protected) - Update a Cluster Template with specified parameters.
.. autoclass:: saharaclient.api.cluster_templates.ClusterTemplateManager
:members:
Cluster ops
~~~~~~~~~~~
* create(name, plugin_name, hadoop_version, cluster_template_id, default_image_id, is_transient, description, cluster_configs, node_groups, user_keypair_id, anti_affinity, net_id, count, use_autoconfig, shares, is_public, is_protected) - Launch a Cluster with specified parameters.
* scale(cluster_id, scale_object) - Scale an existing Cluster. `scale_object` format is described in REST API doc.
* update(cluster_id, name, description, is_public, is_protected) - Update a Cluster with specified parameters.
.. autoclass:: saharaclient.api.clusters.ClusterManager
:members:
Data Source ops
~~~~~~~~~~~~~~~
* create(name, description, data_source_type, url, credential_user, credential_pass, is_public, is_protected) - Create a Data Source with specified parameters.
* update(data_source_id, update_data) - Update a Data Source with provided `data`.
.. autoclass:: saharaclient.api.data_sources.DataSourceManager
:members:
Job Binary Internal ops
~~~~~~~~~~~~~~~~~~~~~~~
* create(name, data) - Create a Job Binary Internal from provided `data`.
* update(job_binary_id, name, is_public, is_protected) - Update a Job Binary Internal with specified parameters
.. autoclass:: saharaclient.api.job_binary_internals.JobBinaryInternalsManager
:members: create, update
Job Binary ops
~~~~~~~~~~~~~~
* create(name, url, description, extra, is_public, is_protected) - Create a Job Binary with specified parameters.
* get_file(job_binary_id) - Download a Job Binary.
* update(job_binary_id, data) - Update Job Binary with provided `data`.
.. autoclass:: saharaclient.api.job_binaries.JobBinariesManager
:members:
Job ops
~~~~~~~
* create(name, type, mains, libs, description, interface, is_public, is_protected) - Create a Job with specified parameters.
* get_configs(job_type) - Get config hints for a specified Job type.
* update(job_id, name, description, is_public, is_protected) - Update a Job with specified parameters.
.. autoclass:: saharaclient.api.jobs.JobsManager
:members:
Job Execution ops
~~~~~~~~~~~~~~~~~
* create(job_id, cluster_id, input_id, output_id, configs, interface, is_public, is_protected) - Launch a Job with specified parameters.
* update(obj_id, is_public, is_protected) - Update a Job Execution with specified parameters.
.. autoclass:: saharaclient.api.job_executions.JobExecutionsManager
:members:
Job Types ops
~~~~~~~~~~~~~
.. autoclass:: saharaclient.api.job_types.JobTypesManager
:members:

View File

@@ -47,6 +47,26 @@ class HTTPClient(adapter.Adapter):
class Client(object):
"""Client for the OpenStack Data Processing v1 API.
:param str username: Username for Keystone authentication.
:param str api_key: Password for Keystone authentication.
:param str project_id: Keystone Tenant id.
:param str project_name: Keystone Tenant name.
:param str auth_url: Keystone URL that will be used for authentication.
:param str sahara_url: Sahara REST API URL to communicate with.
:param str endpoint_type: Desired Sahara endpoint type.
:param str service_type: Sahara service name in Keystone catalog.
:param str input_auth_token: Keystone authorization token.
:param session: Keystone Session object.
:param auth: Keystone Authentication Plugin object.
:param boolean insecure: Allow insecure.
:param string cacert: Path to the Privacy Enhanced Mail (PEM) file
which contains certificates needed to establish
SSL connection with the identity service.
:param string region_name: Name of a region to select when choosing an
endpoint from the service catalog.
"""
def __init__(self, username=None, api_key=None, project_id=None,
project_name=None, auth_url=None, sahara_url=None,
endpoint_type='publicURL', service_type='data-processing',

View File

@@ -27,6 +27,7 @@ class ClusterTemplateManager(base.ResourceManager):
cluster_configs=None, node_groups=None, anti_affinity=None,
net_id=None, default_image_id=None, use_autoconfig=None,
shares=None, is_public=None, is_protected=None):
"""Create a Cluster Template."""
data = {
'name': name,
@@ -53,6 +54,7 @@ class ClusterTemplateManager(base.ResourceManager):
node_groups=None, anti_affinity=None, net_id=None,
default_image_id=None, use_autoconfig=None, shares=None,
is_public=None, is_protected=None):
"""Update a Cluster Template."""
data = {}
self._copy_if_defined(data, name=name,
@@ -73,12 +75,15 @@ class ClusterTemplateManager(base.ResourceManager):
data, 'cluster_template')
def list(self, search_opts=None):
"""Get list of Cluster Templates."""
query = base.get_query_string(search_opts)
return self._list('/cluster-templates%s' % query, 'cluster_templates')
def get(self, cluster_template_id):
"""Get information about a Cluster Template."""
return self._get('/cluster-templates/%s' % cluster_template_id,
'cluster_template')
def delete(self, cluster_template_id):
"""Delete a Cluster Template."""
self._delete('/cluster-templates/%s' % cluster_template_id)

View File

@@ -32,6 +32,7 @@ class ClusterManager(base.ResourceManager):
anti_affinity=None, net_id=None, count=None,
use_autoconfig=None, shares=None,
is_public=None, is_protected=None):
"""Launch a Cluster."""
data = {
'name': name,
@@ -67,13 +68,44 @@ class ClusterManager(base.ResourceManager):
return self._create('/clusters', data, 'cluster')
def scale(self, cluster_id, scale_object):
"""Scale an existing Cluster.
:param scale_object: dict that describes scaling operation
:Example:
The following `scale_object` can be used to change the number of
instances in the node group and add instances of new node group to
existing cluster:
.. sourcecode:: json
{
"add_node_groups": [
{
"count": 3,
"name": "new_ng",
"node_group_template_id": "ngt_id"
}
],
"resize_node_groups": [
{
"count": 2,
"name": "old_ng"
}
]
}
"""
return self._update('/clusters/%s' % cluster_id, scale_object)
def list(self, search_opts=None):
"""Get a list of Clusters."""
query = base.get_query_string(search_opts)
return self._list('/clusters%s' % query, 'clusters')
def get(self, cluster_id, show_progress=False):
"""Get information about a Cluster."""
url = ('/clusters/%(cluster_id)s?%(params)s' %
{"cluster_id": cluster_id,
"params": parse.urlencode({"show_progress": show_progress})})
@@ -81,10 +113,12 @@ class ClusterManager(base.ResourceManager):
return self._get(url, 'cluster')
def delete(self, cluster_id):
"""Delete a Cluster."""
self._delete('/clusters/%s' % cluster_id)
def update(self, cluster_id, name=None, description=None, is_public=None,
is_protected=None, shares=None):
"""Update a Cluster."""
data = {}
self._copy_if_defined(data, name=name, description=description,

View File

@@ -26,6 +26,8 @@ class DataSourceManager(base.ResourceManager):
def create(self, name, description, data_source_type,
url, credential_user=None, credential_pass=None,
is_public=None, is_protected=None):
"""Create a Data Source."""
data = {
'name': name,
'description': description,
@@ -43,15 +45,33 @@ class DataSourceManager(base.ResourceManager):
return self._create('/data-sources', data, 'data_source')
def list(self, search_opts=None):
"""Get a list of Data Sources."""
query = base.get_query_string(search_opts)
return self._list('/data-sources%s' % query, 'data_sources')
def get(self, data_source_id):
"""Get information about a Data Source."""
return self._get('/data-sources/%s' % data_source_id, 'data_source')
def delete(self, data_source_id):
"""Delete a Data Source."""
self._delete('/data-sources/%s' % data_source_id)
def update(self, data_source_id, update_data):
"""Update a Data Source.
:param dict update_data: dict that contains fields that should be
updated with new values.
Fields that can be updated:
* name
* description
* type
* url
* is_public
* is_protected
* credentials - dict with `user` and `password` keyword arguments
"""
return self._update('/data-sources/%s' % data_source_id,
update_data)

View File

@@ -25,16 +25,20 @@ class ImageManager(base.ResourceManager):
resource_class = Image
def list(self, search_opts=None):
"""Get a list of registered images."""
query = base.get_query_string(search_opts)
return self._list('/images%s' % query, 'images')
def get(self, id):
"""Get information about an image"""
return self._get('/images/%s' % id, 'image')
def unregister_image(self, image_id):
"""Remove an Image from Sahara Image Registry."""
self._delete('/images/%s' % image_id)
def update_image(self, image_id, user_name, desc=None):
"""Create or update an Image in Image Registry."""
desc = desc if desc else ''
data = {"username": user_name,
"description": desc}
@@ -42,6 +46,11 @@ class ImageManager(base.ResourceManager):
return self._post('/images/%s' % image_id, data)
def update_tags(self, image_id, new_tags):
"""Update an Image tags.
:param list new_tags: list of tags that will replace currently
assigned tags
"""
old_image = self.get(image_id)
old_tags = frozenset(old_image.tags)

View File

@@ -25,6 +25,7 @@ class JobBinariesManager(base.ResourceManager):
def create(self, name, url, description=None, extra=None, is_public=None,
is_protected=None):
"""Create a Job Binary."""
data = {
"name": name,
"url": url
@@ -36,16 +37,20 @@ class JobBinariesManager(base.ResourceManager):
return self._create('/job-binaries', data, 'job_binary')
def list(self, search_opts=None):
"""Get a list of Job Binaries."""
query = base.get_query_string(search_opts)
return self._list('/job-binaries%s' % query, 'binaries')
def get(self, job_binary_id):
"""Get information about a Job Binary."""
return self._get('/job-binaries/%s' % job_binary_id, 'job_binary')
def delete(self, job_binary_id):
"""Delete a Job Binary."""
self._delete('/job-binaries/%s' % job_binary_id)
def get_file(self, job_binary_id):
"""Download a Job Binary."""
resp = self.api.get('/job-binaries/%s/data' % job_binary_id)
if resp.status_code != 200:
@@ -53,5 +58,19 @@ class JobBinariesManager(base.ResourceManager):
return resp.content
def update(self, job_binary_id, data):
"""Update Job Binary.
:param dict data: dict that contains fields that should be updated
with new values.
Fields that can be updated:
* name
* description
* url
* is_public
* is_protected
* extra - dict with `user` and `password` keyword arguments
"""
return self._update(
'/job-binaries/%s' % job_binary_id, data, 'job_binary')

View File

@@ -26,23 +26,31 @@ class JobBinaryInternalsManager(base.ResourceManager):
resource_class = JobBinaryInternal
def create(self, name, data):
"""Create a Job Binary Internal.
:param str data: raw data ot script text
"""
return self._update('/job-binary-internals/%s' %
urlparse.quote(name.encode('utf-8')), data,
'job_binary_internal', dump_json=False)
def list(self, search_opts=None):
"""Get a list of Job Binary Internals."""
query = base.get_query_string(search_opts)
return self._list('/job-binary-internals%s' % query, 'binaries')
def get(self, job_binary_id):
"""Get information about a Job Binary Internal."""
return self._get('/job-binary-internals/%s' % job_binary_id,
'job_binary_internal')
def delete(self, job_binary_id):
"""Delete a Job Binary Internal."""
self._delete('/job-binary-internals/%s' % job_binary_id)
def update(self, job_binary_id, name=None, is_public=None,
is_protected=None):
"""Update a Job Binary Internal."""
data = {}
self._copy_if_defined(data, name=name, is_public=is_public,

View File

@@ -24,18 +24,22 @@ class JobExecutionsManager(base.ResourceManager):
resource_class = JobExecution
def list(self, search_opts=None):
"""Get a list of Job Executions."""
query = base.get_query_string(search_opts)
return self._list('/job-executions%s' % query, 'job_executions')
def get(self, obj_id):
"""Get information about a Job Execution."""
return self._get('/job-executions/%s' % obj_id, 'job_execution')
def delete(self, obj_id):
"""Delete a Job Execution."""
self._delete('/job-executions/%s' % obj_id)
def create(self, job_id, cluster_id, input_id=None,
output_id=None, configs=None, interface=None, is_public=None,
is_protected=None):
"""Launch a Job."""
url = "/jobs/%s/execute" % job_id
data = {
@@ -49,6 +53,7 @@ class JobExecutionsManager(base.ResourceManager):
return self._create(url, data, 'job_execution')
def update(self, obj_id, is_public=None, is_protected=None):
"""Update a Job Execution."""
data = {}
self._copy_if_defined(data, is_public=is_public,

View File

@@ -24,5 +24,6 @@ class JobTypesManager(base.ResourceManager):
resource_class = JobType
def list(self, search_opts=None):
"""Get a list of job types supported by plugins."""
query = base.get_query_string(search_opts)
return self._list('/job-types%s' % query, 'job_types')

View File

@@ -25,6 +25,7 @@ class JobsManager(base.ResourceManager):
def create(self, name, type, mains=None, libs=None, description=None,
interface=None, is_public=None, is_protected=None):
"""Create a Job."""
data = {
'name': name,
'type': type
@@ -37,20 +38,25 @@ class JobsManager(base.ResourceManager):
return self._create('/jobs', data, 'job')
def list(self, search_opts=None):
"""Get a list of Jobs."""
query = base.get_query_string(search_opts)
return self._list('/jobs%s' % query, 'jobs')
def get(self, job_id):
"""Get information about a Job"""
return self._get('/jobs/%s' % job_id, 'job')
def get_configs(self, job_type):
"""Get config hints for a specified Job type."""
return self._get('/jobs/config-hints/%s' % job_type)
def delete(self, job_id):
"""Delete a Job"""
self._delete('/jobs/%s' % job_id)
def update(self, job_id, name=None, description=None, is_public=None,
is_protected=None):
"""Update a Job."""
data = {}
self._copy_if_defined(data, name=name, description=description,

View File

@@ -32,6 +32,7 @@ class NodeGroupTemplateManager(base.ResourceManager):
volume_local_to_instance=None, use_autoconfig=None,
shares=None, is_public=None, is_protected=None,
volume_mount_prefix=None):
"""Create a Node Group Template."""
data = {
'name': name,
@@ -83,6 +84,7 @@ class NodeGroupTemplateManager(base.ResourceManager):
volume_local_to_instance=None, use_autoconfig=None,
shares=None, is_public=None, is_protected=None,
volume_mount_prefix=None):
"""Update a Node Group Template."""
data = {}
self._copy_if_defined(
@@ -107,13 +109,16 @@ class NodeGroupTemplateManager(base.ResourceManager):
'node_group_template')
def list(self, search_opts=None):
"""Get a list of Node Group Templates."""
query = base.get_query_string(search_opts)
return self._list('/node-group-templates%s' % query,
'node_group_templates')
def get(self, ng_template_id):
"""Get information about a Node Group Template."""
return self._get('/node-group-templates/%s' % ng_template_id,
'node_group_template')
def delete(self, ng_template_id):
"""Delete a Node Group Template."""
self._delete('/node-group-templates/%s' % ng_template_id)

View File

@@ -32,18 +32,30 @@ class PluginManager(base.ResourceManager):
resource_class = Plugin
def list(self, search_opts=None):
"""Get a list of Plugins."""
query = base.get_query_string(search_opts)
return self._list('/plugins%s' % query, 'plugins')
def get(self, plugin_name):
"""Get information about a Plugin."""
return self._get('/plugins/%s' % plugin_name, 'plugin')
def get_version_details(self, plugin_name, hadoop_version):
"""Get version details
Get the list of Services and Service Parameters for a specified
Plugin and Plugin Version.
"""
return self._get('/plugins/%s/%s' % (plugin_name, hadoop_version),
'plugin')
def convert_to_cluster_template(self, plugin_name, hadoop_version,
template_name, filecontent):
"""Convert to cluster template
Create Cluster Template directly, avoiding Cluster Template
mechanism.
"""
resp = self.api.post('/plugins/%s/%s/convert-config/%s' %
(plugin_name,
hadoop_version,