Update various API design details based on review feedback
This commit is contained in:
Bryan Strassner
parent
fc3923cdf9
commit
ea22d97202
173
docs/API.md
173
docs/API.md
|
|
@ -16,6 +16,16 @@ If a variable in the path doesn't reference a valid resource
|
|||
* 503 Service Unavailable
|
||||
Error if an upstream system cannot be reached (e.g. Deckhand for documents, Airflow for actions)
|
||||
|
||||
---
|
||||
## Headers
|
||||
### Required
|
||||
|
||||
* X-Auth-Token - The auth token to identify the invoking user.
|
||||
|
||||
### Optional
|
||||
|
||||
* X-Context-Marker - A context id that will be carried on all logs for this cleint-provided marker. This marker may only be a 36-character canonical representation of an UUID (8-4-4-4-12)
|
||||
|
||||
---
|
||||
## <a name="DocumentStagingAPI"></a> Document Staging API
|
||||
Shipyard will serve as the entrypoint for documents and secrets into a site. At any point in time, there will be two represented versions of documents in a site that are accessible via this API.
|
||||
|
|
@ -26,49 +36,94 @@ Shipyard will serve as the entrypoint for documents and secrets into a site. At
|
|||
### /v1.0/configDocs
|
||||
Represents the site configuration documents.
|
||||
|
||||
#### Payload Structure
|
||||
The yaml documents separated by --- and concluded by ...
|
||||
|
||||
|
||||
#### POST
|
||||
Ingests site configuration documents. Synchronous.
|
||||
##### Responses
|
||||
* 200 OK
|
||||
If the documents are successfully ingested, even with validation failures. Response message includes:
|
||||
* a link to the GET /configDocs
|
||||
* a list of validation results
|
||||
* 201 Created
|
||||
If the documents are successfully ingested, even with validation failures. Response message includes:
|
||||
* a list of validation results
|
||||
|
||||
The response headers will include a Location indicating the GET endpoint to retrieve the configDocs
|
||||
|
||||
#### GET
|
||||
Returns the source documents for the most recently staged version
|
||||
Query Parameter "lastDeployed=true" will insetead return the documents for the currently deployed version.
|
||||
##### Query Parameters
|
||||
* lastDeployed=true|**false**
|
||||
Return the documents for the currently deployed version instead of the most recently staged
|
||||
* compiled=true|**false**
|
||||
Return the documents in their compiled state instead of source documents
|
||||
|
||||
##### Responses
|
||||
* 200 OK
|
||||
If documents can be retrieved.
|
||||
|
||||
#### DELETE
|
||||
Updates the configDocs to be the lastDeployed versions in deckhand, effectively discarding any staged documents.
|
||||
###### Responses
|
||||
Updates the configDocs to be the lastDeployed versions in deckhand, effectively discarding any staged documents. If there is no prior deployed version this effectively removes all secrets.
|
||||
|
||||
##### Responses
|
||||
* 204 No Content
|
||||
|
||||
|
||||
### /v1.0/secrets
|
||||
Represents the secrets documents that will be used in combination with the config docs.
|
||||
|
||||
##### POST
|
||||
#### Payload Structure
|
||||
The yaml documents separated by --- and concluded by ...
|
||||
|
||||
#### POST
|
||||
Ingest new secrets documents.
|
||||
###### Responses
|
||||
201 Created
|
||||
##### Responses
|
||||
* 201 Created
|
||||
If the documents are successfully ingested, even with validation failures. Response message includes:
|
||||
* a link to the GET /configDocs
|
||||
* a list of validation results
|
||||
|
||||
The response headers will include a Location indicating the GET endpoint to retrieve the configDocs
|
||||
|
||||
#### GET
|
||||
Return the secrets documents
|
||||
|
||||
##### Query Parameters
|
||||
* lastDeployed=true|**false**
|
||||
Return the documents for the currently deployed version instead of the most recently staged
|
||||
* compiled=true|**false**
|
||||
Return the documents in their compiled state instead of source documents
|
||||
|
||||
##### Responses
|
||||
* 200 OK
|
||||
If documents can be retrieved.
|
||||
|
||||
#### DELETE
|
||||
Updates the secrets to be the lastDeployed versions in deckhand, effectively discarding any staged documents.
|
||||
Updates the secrets to be the lastDeployed versions in deckhand, effectively discarding any staged documents. If there is no prior deployed version this effectively removes all secrets.
|
||||
##### Responses
|
||||
* 204 No Content
|
||||
|
||||
|
||||
### /v1.0/validations
|
||||
Represents the validation messages for the documents in deckhand
|
||||
|
||||
#### Payload Structure
|
||||
The list of validations
|
||||
```
|
||||
[
|
||||
{ TBD }
|
||||
]
|
||||
```
|
||||
|
||||
#### GET
|
||||
Returns the validations
|
||||
|
||||
##### Query Parameters
|
||||
* lastDeployed=true|**false**
|
||||
Return the documents for the currently deployed version instead of the most recently staged
|
||||
|
||||
##### Responses
|
||||
* 200 OK
|
||||
If the validations can be retrieved.
|
||||
|
||||
---
|
||||
## <a name="ActionAPI"></a> Action API
|
||||
Signal to Shipyard to take an action (e.g. Invoke a named DAG)
|
||||
|
|
@ -76,20 +131,15 @@ See [Action Commands](API_action_commands.md) for supported actions
|
|||
|
||||
|
||||
### /v1.0/actions
|
||||
All actions will include fields that indicate the following data:
|
||||
* id (assigned during POST)
|
||||
* name - the name of the action - likely to be the DAG Names, but may be mapped for "nicer" values.
|
||||
* parameters - a dictionary of the parameters configuring the action.
|
||||
* tracking info - (user, time, etc)
|
||||
* action_lifecycle value:
|
||||
* Pending
|
||||
* Validation Failed
|
||||
* In DAG
|
||||
* Complete
|
||||
* Failed
|
||||
* DAG_status - representing the status that airflow provides for an executing DAG
|
||||
* validations - a list of validaitons that have been done, including any status information for those validations. During the lifecycle of the DAG, this list of validations may continue to grow.
|
||||
* steps - the list of steps for this action, including the status for that step.
|
||||
|
||||
#### Payload Structure
|
||||
A list of actions that have been executed through shipyard's action API.
|
||||
```
|
||||
[
|
||||
{ Action objects summarized, TBD },
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
#### GET
|
||||
Returns the list of actions in the system that have been posted, and are accessible to the current user.
|
||||
|
|
@ -105,9 +155,15 @@ The input body to this post will represent an action object that has at least th
|
|||
The name of the action to invoke. This is likely to map to the actual DAG Names, but may be mapped for "nicer" values.
|
||||
* parameters
|
||||
A dictionary of parameters to use for the trigger invocation. The supported parameters will vary for the action invoked.
|
||||
```
|
||||
{
|
||||
"name" : "action name",
|
||||
"parameters" : { varies by action }
|
||||
}
|
||||
```
|
||||
|
||||
The POST will synchrounously create the action (a shell object that represents a DAG invocation), perform any checks to validate the preconditions to run the DAG, and trigger the invocation of the DAG.
|
||||
The DAG will run Asynchrounously in airflow.
|
||||
The DAG will run asynchronously in airflow.
|
||||
##### Responses
|
||||
* 201 Created
|
||||
If the action is created successfully, and all preconditions to run the DAG are successful. The response body is the action entity created.
|
||||
|
|
@ -120,6 +176,26 @@ For any failed validations. The response body is the action entity created, with
|
|||
### /v1.0/actions/{action_id}
|
||||
Each action will be assigned an unique id that can be used to get details for the action, including the execution status.
|
||||
|
||||
#### Payload Structure
|
||||
All actions will include fields that indicate the following data:
|
||||
* id (assigned during POST)
|
||||
* name - the name of the action - likely to be the DAG Names, but may be mapped for "nicer" values.
|
||||
* parameters - a dictionary of the parameters configuring the action.
|
||||
* tracking info - (user, time, etc)
|
||||
* action_lifecycle value:
|
||||
* Pending
|
||||
* Validation Failed
|
||||
* Processing
|
||||
* Complete
|
||||
* Failed
|
||||
* DAG_status - representing the status that airflow provides for an executing DAG
|
||||
* validations - a list of validations that have been done, including any status information for those validations. During the lifecycle of the DAG, this list of validations may continue to grow.
|
||||
* steps - the list of steps for this action, including the status for that step.
|
||||
|
||||
```
|
||||
{ TBD }
|
||||
```
|
||||
|
||||
#### GET
|
||||
Returns the action entity for the specified id.
|
||||
##### Responses
|
||||
|
|
@ -129,6 +205,12 @@ Returns the action entity for the specified id.
|
|||
### /v1.0/actions/{action_id}/validationDetails/{validation_id}
|
||||
Allows for drilldown to validation detailed info.
|
||||
|
||||
#### Payload Structure
|
||||
The detailed information for a validation
|
||||
```
|
||||
{ TBD }
|
||||
```
|
||||
|
||||
#### GET
|
||||
Returns the validation detail by Id for the supplied action Id.
|
||||
##### Responses
|
||||
|
|
@ -138,6 +220,12 @@ Returns the validation detail by Id for the supplied action Id.
|
|||
### /v1.0/actions/{action_id}/steps/{step_id}
|
||||
Allow for drilldown to step information. The step information includes details of the steps excution, successful or not, and enough to facilitate troubleshooting in as easy a fashion as possible.
|
||||
|
||||
#### Payload Structure
|
||||
The detailed information representing a single step of execution as part of an action
|
||||
```
|
||||
{ TBD }
|
||||
```
|
||||
|
||||
#### GET
|
||||
Returns the details for a step by id for the given action by Id.
|
||||
##### Responses
|
||||
|
|
@ -147,8 +235,11 @@ Returns the details for a step by id for the given action by Id.
|
|||
### /v1.0/actions/{action_id}/{control_verb}
|
||||
Allows for issuing DAG controls against an action.
|
||||
|
||||
#### Payload Structure
|
||||
None, there is no associated body for this resource
|
||||
|
||||
#### POST
|
||||
Trigger a control action against an activity.- this includes: pause, unpause, abandon
|
||||
Trigger a control action against an activity.- this includes: pause, unpause
|
||||
##### Responses
|
||||
* 202 Accepted
|
||||
|
||||
|
|
@ -156,9 +247,18 @@ Trigger a control action against an activity.- this includes: pause, unpause, ab
|
|||
## <a name="AirflowMonitoringAPI"></a> Airflow Monitoring API
|
||||
Airflow has a primary function of scheduling DAGs, as opposed to Shipyard's primary case of triggering DAGs. Shipyard will need to provide functionality to allow for an operator to monitor and review the scheduled tasks. This API will allow for accessing Airflow DAGs of either type.
|
||||
|
||||
### /v1.0/scheduledDAGs
|
||||
### /v1.0/completedDAGs
|
||||
The resource that represents DAGs in airflow
|
||||
|
||||
#### Payload Structure
|
||||
A list of objects representing the DAGs that have run in airflow.
|
||||
```
|
||||
[
|
||||
{TBD},
|
||||
...
|
||||
]
|
||||
```
|
||||
|
||||
#### GET
|
||||
Queries airflow for recent completed DAGs (a summary).
|
||||
##### Query parameters
|
||||
|
|
@ -168,9 +268,18 @@ optional, the number of days of history to retrieve. Default is 30 days.
|
|||
* 200 OK
|
||||
|
||||
|
||||
#### /v1.0/scheduledDAGs/{id}
|
||||
#### /v1.0/completedDAGs/{id}
|
||||
|
||||
##### GET
|
||||
|
||||
#### Payload Structure
|
||||
An object representing the information available from airflow regarding a DAG's execution
|
||||
|
||||
```
|
||||
{ TBD }
|
||||
```
|
||||
|
||||
|
||||
#### GET
|
||||
Further details of a particular scheduled DAG's output
|
||||
###### Responses
|
||||
##### Responses
|
||||
* 200 OK
|
||||
|
|
|
|||
Loading…
Reference in New Issue