Updated the API documentation for shipyard:

Incorporated feedback from reviews

Added steps to the action commands document

Added renderedconfigdocs resource for rendered representations of
documents

Updated POST configdocs to be collection based

Added append and force parameter behaviors for POST configdocs

Updated verbiage for the document staging API to be collection
based

Update GET behaviors for configdocs to clarify when collections
are removed vs. not represented.

Remove the secrets API, as it will be handled by the configdocs

change to have document versions referenced by verion=rollback|staged
instead of lastDeployed=true|false.
Updated to make clearer when documents move from staged to rollback

reformatted for 80 cols

removed extraneous trailing whitespace

Change-Id: I59ceb8643d7da5ef335b746ea98053a8338207be
This commit is contained in:
Bryan Strassner 2017-08-18 08:59:12 -05:00
parent 8526e18058
commit 59fabf8cac
2 changed files with 176 additions and 136 deletions

View File

@ -1,128 +1,110 @@
# Shipyard API
Logically, the API has three parts to handle the three areas of functionality in Shipyard.
Logically, the API has three parts to handle the three areas of functionality
in Shipyard.
1. Document Staging
2. Action Handling
3. Airflow Monitoring
2. Action Handling
3. Airflow Monitoring
## Naming used in the API
* Resource paths nodes follow an all lower case naming scheme, and pluralize the resource names. Nodes that refer to keys, ids or names that are externally controlled, the external naming will be honored.
* The version of the API resource path will be prefixed before the first node of the path for that resource using v#.# format.
* By default, the API will be namespaced by /api before the version. For the purposes of documentation, this will not be specified in each of the resource paths below. In more complex APIs, it makes sense to allow the /api node to be more specific to point to a particular service.
```
/api/v1.0/sampleresources/ExTeRnAlNAME-1234
^ ^ ^ ^
| | | defer to external naming
| | plural
| lower case
version here
```
## Standard Error Responses
There are several standard error responses that should be handled appropriately by a client of this API. Those errors listed here are not exhaustive, but shouldconsidered possible for all API invocations.
* 401 Authentication Error
If not authenticated
* 403 Forbidden
If not authorized
* 404 Not found
If a variable in the path doesn't reference a valid resource
* 503 Service Unavailable
Error if an upstream system cannot be reached (e.g. Deckhand for documents, Airflow for actions)
## Headers
### Required
* X-Auth-Token
The auth token to identify the invoking user.
### Optional
* X-Context-Marker
A context id that will be carried on all logs for this cleint-provided marker. This marker may only be a 36-character canonical representation of an UUID (8-4-4-4-12)
## Standards used by the API
See [UCP API conventions](https://github.com/att-comdev/ucp-integration/blob/master/docs/api-conventions.md)
---
## <a name="DocumentStagingAPI"></a> Document Staging API
>Shipyard will serve as the entrypoint for documents and secrets into a site. At any point in time, there will be two represented versions of documents in a site that are accessible via this API.
>Shipyard will serve as the entrypoint for documents (designs, secrets,
configurations, etc...) into a site. Documents are posted to Shipyard in
collections, rather than individually. At any point in time, there will be two
represented versions of documents in a site that are accessible via this API:
>
>* The lastDeployed version, which represents the last version of documents that successfully completed deployment. The lastDeployed tag will be applied by Shipyard at the successful completion of certain workflows, to indicate a baseline design for the site.
>* The staged version, which represents the document set that has been ingested by this API since the lastDeployed version.
>* The "Committed Documents" version, which represents the last version of
documents that were successfully commited with a commit_configdocs action.
>* The "Shipyard Buffer" version, which represents the collection of documents
that have been ingested by this API since the last commited version. Note that
only one set of documents maybe posted to the buffer at a time by default.
(This behavior can be overridden by query parameters issued by the user of
Shipyard)
>
> All versions of documents rely upon Deckhand for storage. Shipyard will rely upon the tagging features of Deckhand of to find the appropriate lastDeployed and staged version.
> All versions of documents rely upon Deckhand for storage. Shipyard uses the
tagging features of Deckhand of to find the appropriate Committed Documents
and Shipyard Buffer version.
---
### /v1.0/configdocs
Represents the site configuration documents
### /v1.0/configdocs/{collection_id}
Represents the site configuration documents
#### Payload Structure
The documents (commonly yaml), in a format understood by Deckhand
The documents as noted above (commonly yaml), in a format understood by
Deckhand
#### POST
Ingests site configuration documents. Synchronous.
##### Responses
* 201 Created
If the documents are successfully ingested, even with validation failures. Response message includes:
* a list of validation results
#### POST
Ingests a collection of documents. Synchronous. POSTing an empty body
indicates that the specified collection should be deleted if the Shipyard
Buffer is committed.
The response headers will include a Location indicating the GET endpoint to retrieve the configDocs
#### GET
Returns the source documents for the most recently staged version
##### Query Parameters
* lastDeployed=true|**false**
Return the documents for the currently deployed version instead of the most recently staged
* compiled=true|**false**
Return the documents in their compiled state instead of source documents
##### Responses
* 200 OK
If documents can be retrieved.
#### DELETE
Updates the configDocs to be the lastDeployed versions in deckhand, effectively discarding any staged documents. If there is no prior deployed version this effectively removes all configDocs.
##### Responses
* 204 No Content
---
### /v1.0/secrets
Represents the secrets documents that will be used in combination with the config docs.
#### Payload Structure
The secrets documents in a format understood by Deckhand
#### POST
Ingest new secrets documents.
* bufferMode=append|replace|**rejectOnContents**
Indicates how the existing Shipyard Buffer should be handled. By default,
Shipyard will reject the POST if contents already exist in the Shipyard
Buffer.
* append: Add the collection to the Shipyard Buffer, only if that
collection doesn't already exist in the Shipyard Buffer. If the collection
is already present, the request will be rejected and a 409 Conflict will be
returned.
* replace: Clear the Shipyard Buffer before adding the specified collection.
##### Responses
* 201 Created
If the documents are successfully ingested, even with validation failures. Response message includes:
If the documents are successfully ingested, even with validation failures.
Response message includes:
* a list of validation results
* The response headers will include a Location indicating the GET endpoint
to retrieve the configDocs
The response headers will include a Location indicating the GET endpoint to retrieve the configDocs
#### GET
Return the secrets documents
* 409 Conflict
* If a commit_configdocs action is in progress.
* If any collections exist in the Shipyard Buffer unless bufferMode=replace
or bufferMode=append.
* If bufferMode=append, and the collection being posted is already in the
Shipyard Buffer
#### GET
Returns the source documents for a collection of documents
##### Query Parameters
* lastDeployed=true|**false**
Return the documents for the currently deployed version instead of the most recently staged
* compiled=true|**false**
Return the documents in their compiled state instead of source documents
* version=committed|**buffer**
Return the documents for the version specified - buffer by default.
##### Responses
* 200 OK
If documents can be retrieved.
* If the response is 200 with an empty response body, this indicates that
the buffer version is attempting to 'delete' the collection when it is
committed. An empty response body will only be possible for version=buffer.
* 404 Not Found
If the collection is not represented
* When version=buffer, this indicates that no representations of this
collection have been POSTed since the last committed version.
* When version=committed, this indicates that either the collection has
never existed or has been deleted by a prior commit
#### DELETE
Updates the secrets to be the lastDeployed versions in deckhand, effectively discarding any staged documents. If there is no prior deployed version this effectively removes all secrets.
---
### /v1.0/renderedconfigdocs
Represents the site configuration documents, as a whole set - does not
consider collections in any way.
#### GET
Returns the full set of configdocs in their rendered form.
##### Query Parameters
* version=committed|**buffer**
Return the documents for the version specified - buffer by default.
##### Responses
* 204 No Content
* 200 OK
If documents can be retrieved.
---
### /v1.0/validations
Represents the validation messages for the documents in deckhand
#### Payload Structure
The list of validations
```
@ -131,12 +113,12 @@ The list of validations
]
```
#### GET
#### GET
Returns the validations
##### Query Parameters
* lastDeployed=true|**false**
Return the documents for the currently deployed version instead of the most recently staged
* version=committed|**buffer**
returns the validations if any associated with the version specified.
##### Responses
* 200 OK
@ -144,7 +126,9 @@ If the validations can be retrieved.
---
## <a name="ActionAPI"></a> Action API
>The Shipyard Action API is a resource that allows for creation, control and investigation of triggered workflows. These actions encapsulate a command interface for the Unercloud Platform.
>The Shipyard Action API is a resource that allows for creation, control and
investigation of triggered workflows. These actions encapsulate a command
interface for the Undercloud Platform.
See [Action Commands](API_action_commands.md) for supported actions
---
@ -160,7 +144,8 @@ A list of actions that have been executed through shipyard's action API.
```
#### GET
Returns the list of actions in the system that have been posted, and are accessible to the current user.
Returns the list of actions in the system that have been posted, and are
accessible to the current user.
##### Responses
* 200 OK
@ -168,47 +153,61 @@ If the actions can be retrieved.
#### POST
Creates an action in the system. This will cause some action to start.
The input body to this post will represent an action object that has at least these fields:
The input body to this post will represent an action object that has at least
these fields:
* name
The name of the action to invoke. This is likely to map to the actual DAG Names, but may be mapped for "nicer" values.
The name of the action to invoke, as noted in
[Action Commands](API_action_commands.md)
* parameters
A dictionary of parameters to use for the trigger invocation. The supported parameters will vary for the action invoked.
A dictionary of parameters to use for the trigger invocation. The supported
parameters will vary for the action invoked.
```
{
{
"name" : "action name",
"parameters" : { varies by action }
}
```
The POST will synchrounously create the action (a shell object that represents a DAG invocation), perform any checks to validate the preconditions to run the DAG, and trigger the invocation of the DAG.
The DAG will run asynchronously in airflow.
The POST will synchrounously create the action (a shell object that represents
a DAG invocation), perform any checks to validate the preconditions to run the
DAG, and trigger the invocation of the DAG. The DAG will run asynchronously in
airflow.
##### Responses
* 201 Created
If the action is created successfully, and all preconditions to run the DAG are successful. The response body is the action entity created.
If the action is created successfully, and all preconditions to run the DAG
are successful. The response body is the action entity created.
* 400 Bad Request
If the action name doesn't exist, or the payload is otherwise malformed.
* 409 Conflict
For any failed validations. The response body is the action entity created, with the failed validations. The DAG will not begin execution in this case.
For any failed pre-run validations. The response body is the action entity
created, with the failed validations. The DAG will not begin execution in this
case.
---
### /v1.0/actions/{action_id}
Each action will be assigned an unique id that can be used to get details for the action, including the execution status.
### /v1.0/actions/{action_id}
Each action will be assigned an unique id that can be used to get details for
the action, including the execution status.
#### Payload Structure
All actions will include fields that indicate the following data:
* id (assigned during POST)
* name - the name of the action - likely to be the DAG Names, but may be mapped for "nicer" values.
* name - the name of the action - likely to be the DAG Names, but may be mapped
for "nicer" values.
* parameters - a dictionary of the parameters configuring the action.
* tracking info - (user, time, etc)
* action_lifecycle value:
* Pending
* Validation Failed
* Processing
* Processing
* Complete
* Failed
* DAG_status - representing the status that airflow provides for an executing DAG
* validations - a list of validations that have been done, including any status information for those validations. During the lifecycle of the DAG, this list of validations may continue to grow.
* steps - the list of steps for this action, including the status for that step.
* DAG_status - representing the status that airflow provides for an executing
DAG
* validations - a list of validations that have been done, including any status
information for those validations. During the lifecycle of the DAG, this list
of validations may continue to grow.
* steps - the list of steps for this action, including the status for that
step.
```
{ TBD }
@ -217,10 +216,10 @@ All actions will include fields that indicate the following data:
#### GET
Returns the action entity for the specified id.
##### Responses
* 200 OK
* 200 OK
---
### /v1.0/actions/{action_id}/validationdetails/{validation_id}
### /v1.0/actions/{action_id}/validationdetails/{validation_id}
Allows for drilldown to validation detailed info.
#### Payload Structure
@ -232,14 +231,17 @@ The detailed information for a validation
#### GET
Returns the validation detail by Id for the supplied action Id.
##### Responses
* 200 OK
* 200 OK
---
### /v1.0/actions/{action_id}/steps/{step_id}
Allow for drilldown to step information. The step information includes details of the steps excution, successful or not, and enough to facilitate troubleshooting in as easy a fashion as possible.
Allow for drilldown to step information. The step information includes details
of the steps excution, successful or not, and enough to facilitate
troubleshooting in as easy a fashion as possible.
#### Payload Structure
The detailed information representing a single step of execution as part of an action
The detailed information representing a single step of execution as part of an
action.
```
{ TBD }
```
@ -247,7 +249,7 @@ The detailed information representing a single step of execution as part of an a
#### GET
Returns the details for a step by id for the given action by Id.
##### Responses
* 200 OK
* 200 OK
---
### /v1.0/actions/{action_id}/{control_verb}
@ -256,15 +258,19 @@ Allows for issuing DAG controls against an action.
#### Payload Structure
None, there is no associated body for this resource
#### POST
#### POST
Trigger a control action against an activity.- this includes: pause, unpause
##### Responses
* 202 Accepted
---
## <a name="AirflowMonitoringAPI"></a> Airflow Monitoring API
>Airflow has a primary function of scheduling DAGs, as opposed to Shipyard's primary case of triggering DAGs.
Shipyard provides functionality to allow for an operator to monitor and review these scheduled workflows (DAGs) in addition to the ones triggered by Shipyard. This API will allow for accessing Airflow DAGs of any type -- providing a peek into the totality of what is happening in Airflow.
>Airflow has a primary function of scheduling DAGs, as opposed to Shipyard's
primary case of triggering DAGs.
Shipyard provides functionality to allow for an operator to monitor and review
these scheduled workflows (DAGs) in addition to the ones triggered by Shipyard.
This API will allow for accessing Airflow DAGs of any type -- providing a peek
into the totality of what is happening in Airflow.
---
### /v1.0/workflows
@ -273,17 +279,19 @@ The resource that represents DAGs (workflows) in airflow
#### Payload Structure
A list of objects representing the DAGs that have run in airflow.
```
[
[
{TBD},
...
]
```
#### GET
Queries airflow for DAGs that are running or have run (successfully or unsuccessfully) and provides a summary of those things.
#### GET
Queries airflow for DAGs that are running or have run (successfully or
unsuccessfully) and provides a summary of those things.
##### Query parameters
* elapsedDays={n}
optional, the number of days of history to retrieve. Default is 30 days.
* since={iso8601 date (past) or duration}
optional, a boundary in the past within which to retrieve results. Default is
30 days in the past.
##### Responses
* 200 OK
@ -291,7 +299,8 @@ optional, the number of days of history to retrieve. Default is 30 days.
### /v1.0/workflows/{id}
#### Payload Structure
An object representing the information available from airflow regarding a DAG's execution
An object representing the information available from airflow regarding a DAG's
execution
```
{ TBD }

View File

@ -9,23 +9,54 @@ None at this time.
## Actions under development
These actions are under active development
* deploy site
Triggers the initial deployment of a site, using the staged design and secrets
* deploy_site
Triggers the initial deployment of a site, using the latest committed
configuration documents. Steps:
1) Concurrency check - to prevent concurrent site modifications by
conflicting actions/workflows.
2) Preflight checks - ensures all UCP components are in a responsive state.
3) Get design version - uses Deckhand to discover the latest committed
version of design for the site.
4) Validate design - asks each involved UCP component to validate the design.
This ensures that the design, which was previously committed is valid at the
present time.
5) Drydock build - orchestrates the Drydock component to configure hardware
and the Kubernetes environment (Drydock -> Promenade)
6) Check deployed node status - checks that the deployment of nodes is
successful.
7) Armada build - orchestrates Armada to configure software on the nodes as
designed.
* update site
Triggers the an update to an existing site, using the staged design and secrets for the site
* update_site
Triggers the initial deployment of a site, using the latest committed
configuration documents. Steps: (same as deploy_site)
* redeploy server
Using parameters to indicate which server(s), triggers a redeployment of server to the last known good design and secrets
* redeploy_server
Using parameters to indicate which server(s), triggers a redeployment of server
to the last known good design and secrets
* commit_configdocs
This signals to shipyard that the documents in the shipyard buffer should be
committed to the site as design. Steps:
1) Blocks Shipyard from accepting any documents for the duration of the
commit_configdocs action to prevent unexpected behavior of the buffer in the
case of validation failures.
2) Triggers validations by the various UCP components against the documents
in the Shipyard Buffer.
* If all validations pass, tags the revision in Deckhand as a committed
version. This will effecively "empty" the Shipyard Buffer.
* If validations fail, the Shipyard Buffer will remain in the state it
was before the commit_configdocs started.
---
## Future actions
These actions are anticipated for development
* test region
Invoke site validation testing - perhaps baseline is a invocation of all components regular "component" tests. This test would be used as a preflight-style test to ensure all components are in a working state.
Invoke site validation testing - perhaps baseline is a invocation of all
components regular "component" tests. This test would be used as a
preflight-style test to ensure all components are in a working state.
* test component
Invoke a particular platform component to test it. This test would be used to interrogate a particular platform component to ensure it is in a working state, and that its own downstream dependencies are also operational
* validate design
Use parameters to test a design. Parameters may support both quick tests to ensure basic format and a complete design, or a more in-depth test that interfaces with particular platform components to ensure a good design
Invoke a particular platform component to test it. This test would be used to
interrogate a particular platform component to ensure it is in a working state,
and that its own downstream dependencies are also operational