Felipe Monteiro 2a8d2638b3 pki: Port Promenade's PKI catalog into Pegleg

This patch set implements the PKICatalog [0] requirements
as well as PeglegManagedDocument [1] generation requirements
outlined in the spec [2].

Included in this patch set:

* New CLI entry point called "pegleg site secrets generate-pki"
* PeglegManagedDocument generation logic in
  engine.cache.managed_document
* Refactored PKICatalog logic in engine.cache.pki_catalog derived
  from the Promenade PKI implementation [3], responsible for
  generating certificates, CAs, and keypairs
* Refactored PKIGenerator logic in engine.cache.pki_generator
  derived from Promenade Generator implementation [4],
  responsible for reading in pegleg/PKICatalog/v1 documents (as
  well as promenade/PKICatalog/v1 documents for backwards
  compatibility) and generating required secrets and storing
  them into the paths specified under [0]
* Unit tests for all of the above [5]
* Example pki-catalog.yaml document under pegleg/site_yamls
* Validation schema for pki-catalog.yaml (TODO: implement
  validation logic here: [6])
* Updates to CLI documentation and inclusion of PKICatalog
  and PeglegManagedDocument documentation
* Documentation updates with PKI information [7]

TODO (in follow-up patch sets):

* Expand on overview documentation to include new Pegleg
  responsibilities
* Allow the original repository (not the copied one) to
  be the destination where the secrets are written to
* Finish up cert expiry/revocation logic

[0] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
[1] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
[2] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html
[3] https://github.com/openstack/airship-promenade/blob/master/promenade/pki.py
[4] https://github.com/openstack/airship-promenade/blob/master/promenade/generator.py
[5] https://review.openstack.org/#/c/611739/
[6] https://review.openstack.org/#/c/608159/
[7] https://review.openstack.org/#/c/611738/

Change-Id: I3010d04cac6d22c656d144f0dafeaa5e19a13068

2019-01-15 13:29:21 -06:00

4.3 KiB

Raw Blame History

Developer Overview of Pegleg

Pegleg's core mission is to, alongside Deckhand, facilitate document authoring strategies within Airship, by:

aggregating documents across multiple revisioned repositories, each of which contains multiple documents defining sites' software and hardware stacks
providing early linting of documents prior to their collection and eventual deployment
including utility functions enabling operators and developers alike to list available sites, render individual manifests via Deckhand, bootstrap repositories with Pegleg-compliant directory layouts, to name a few

Architecture

Pegleg, as a CLI, has a rather simplistic architecture. It is meaningful to visualize Pegleg alongside Deckhand:

Components

cli

The Pegleg cli module implements the user-facing CLI. For more information about this module, reference the CLI documentation <pegleg-cli>.

engine

The engine module implements the following functionality:

document linting
document rendering via Deckhand
document aggregation
additional document utility functions

Developer Workflow

Because Airship is a container-centric platform, the developer workflow heavily utilizes containers for testing and publishing. It also requires Pegleg to produce multiple artifacts that are related, but separate: the Python package, the Docker image and the Helm chart. The code is published via the Docker image artifact.

Pegleg strives to conform to the Airship coding conventions.

Python

The Pegleg code base lives under pegleg. Pegleg supports py36 interpreter.

Docker

The Pegleg Dockerfile is located in /images/pegleg along with any artifacts built specifically to enable the container image. Make targets are used for generating and testing the artifacts.

make images - Build the Pegleg Docker image.

Pegleg, as a containerized CLI, uses Docker via tools/pegleg.sh to execute CLI commands. Commands can also be executed using the Makefile target: run_pegleg.

Virtual Environment

Rather than, after each local code change, rebuilding the Pegleg image and overriding the IMAGE environment variable so tools/pegleg.sh uses the latest code changes, it is possible to use a virtual environment for much faster development.

This can achieved by issuing the following commands (from the root Pegleg directory):

# Quick way of building a virtualenv and installing all required
# dependencies into it.
tox -e py36 --notest
source .tox/py36/bin/activate
pip install -e .

# Now is it possible to run the Pegleg CLI to test local changes:
pegleg <command> <options>

# Or run unit tests:
pytest -k <regex>

Note that after setting up the virtual environment, one only needs to source it in order to re-run unit tests or Pegleg CLI commands, to test local changes.

Testing

All Pegleg tests are nested under tests.

Pegleg comes equipped with a number of tox targets for running unit tests, as well as pep8 and Bandit checks.

Unit Tests

To run all unit tests, execute:

$ tox -e py36

To run unit tests using a regex, execute:

$ tox -e py36 -- <regex>

4.3 KiB Raw Blame History