airship/shipyard
Code Issues Proposed changes
38e58cfd30
shipyard/docs
History
Bryan Strassner 38e58cfd30 Add Action API 
This change introduces a large section of the API for the next major
version of Shipyard - the action api.  By interfacing with Airflow,
Shipyard will invoke workflows and allow for controlling and querying
status of those workflows. Foundationally, this patchset introduces
a lot of framework code for other apis, including error handling
to a common output format, database interaction for persistence of
action information, and use of oslo_config for configuration support.

Add GET all actions primary code - db connection not yet impl
Update base classes to have more structure
Add POST actions framework
Add GET action by id
Add GET of validations and steps
Add control api
Add unit tests of action api methods
Re-Removed duplicate deps from test reqs
Add routes for API
Removed a lot of code better handled by falcon directly
Cleaned up error flows- handlers and defaults
Refactored existing airflow tests to match standard output format
Updated json validation to be more specific
Added basic start for alembic
Added alembic upgrade at startup
Added table creation definitions
Added base revision for alembic upgrade
Bug fixes - DB queries, airflow comm, logic issues, logging issues
Bug fixes - date formats and alignment of keys between systems
Exclusions to bandit / tox.ini
Resolved merge conflicts with integration of auth
Update to use oslo config and PBR
Update the context middleware to check uuid in a less contentious way
Removed routes and resources for regions endpoint - not used
Add auth policies for action api
Restructure execptions to be consistent class hierarchy and common handler
Add generation of config and policy examples
Update tests to init configs
Update database configs to not use env. vars
Removed examples directory, it was no longer accurate
Addressed/removed several TODOs - left some behind as well
Aligned input to DAGs with action: header
Retrieved all sub-steps for dags
Expanded step information
Refactored auth handling for better logging
rename create_actions policy to create_action
removed some templated file comments in env.py generated by alembic
updated inconsistent exception parameters
updated to use ulid instead of uuid for action ids
added action control audit code per review suggestion
Fixed correlation date betwen dags/actions by more string parsing

Change-Id: I2f9ea5250923f45456aa86826e344fc055bba762
2017-09-22 21:47:13 -05:00
..
API_action_commands.md Update to make the commitconfigdocs API be sync. 2017-08-29 09:55:42 -05:00
API.md Add Action API 2017-09-22 21:47:13 -05:00
deployment_guide.md Update Deployment Guide 2017-08-16 19:12:00 +00:00
README.md Update README with new location of issues for github 2017-08-30 10:33:20 -05:00
REST_API.md Add documentation for the shipyard API. 2017-07-28 15:07:30 -05:00

README.md

Shipyard

Shipyard adopts the Falcon web framework and uses Apache Airflow as the backend engine to programmatically author, schedule and monitor workflows.

The current workflow is as follows:

  1. Inital region/site data will be passed to Shipyard from either a human operator or Jenkins
  2. The data (in YAML format) will be sent to DeckHand for processing
  3. Shipyard will make use of the post-processed data from DeckHand to interact with DryDock
  4. DryDock will interact with Promenade to provision and deploy bare metal nodes using Ubuntu MAAS and a resilient Kubernetes cluster will be created at the end of the process
  5. Once the Kubernetes clusters are up and validated to be working properly, Shipyard will interact with Armada to deploy OpenStack using OpenStack Helm
  6. Once the OpenStack cluster is deployed, Shipyard will trigger a workflow to perform basic sanity health checks on the cluster

Note: This project, along with the tools used within are community-based and open sourced.

Mission

The goal for Shipyard is to provide a customizable framework for operators and developers alike. This framework will enable end-users to orchestrate and deploy a fully functional container-based Cloud

Roadmap

The detailed Roadmap can be viewed on the LCOO JIRA

  • Create Helm Charts for Airflow
  • Create Helm Charts for Shipyard
  • Create Falcon API Framework
  • Integrate with DeckHand, DryDock/Promenade, Armada

Getting Started

This project is under development at the moment. We encourage anyone who is interested in Shipyard to review our Installation documentation

Bugs

Bugs are traced in LCOO JIRA. If you find a bug, please feel free to create a GitHub issue and it will be synced to JIRA.