pki: Add documentation for PKI implementation

This adds documentation for the PKI implementation
in Pegleg that was recently ported from Promenade.

Change-Id: I248ec62a0c3523fc5f3950a6a678cf6d998b093a

doc/source/cli/cli.rst

@@ -59,7 +59,7 @@ For example:
 .. code-block:: console
 
   cd /opt/airship-pegleg
-  export WORKSPACE=/opt/airship-treasuremap
+  export WORKSPACE=/opt/airship/treasuremap
   ./tools/pegleg.sh site -r /workspace --help
 
 .. note::
@@ -71,7 +71,7 @@ For example:
   .. code-block:: console
 
     cd /opt/airship-pegleg
-    export WORKSPACE=/opt/airship-treasuremap
+    export WORKSPACE=/opt/airship/treasuremap
     sudo -E ./tools/pegleg.sh site -r /workspace --help
 
 CLI Options
@@ -95,23 +95,23 @@ Options
 
 Path to the root of the site repository (containing site_definition.yaml) repo.
 
-For example: /opt/airship-treasuremap
+For example: /opt/airship/treasuremap
 
 The revision can also be specified via (for example):
 
 ::
 
-  -r /opt/airship-treasuremap@revision
+  -r /opt/airship/treasuremap@revision
 
 **-p / --clone-path** (Optional, Default=/tmp/).
 
-The path where the repo will be cloned. If this option is included and the
+The path where the repo will be cloned. By default the repo will be cloned to
-repo already exists, then the repo will not be cloned again and the user must
+the /tmp path. If this option is included and the repo already exists, then the
-specify a new clone path or pass in the local copy of the repository as the
+repo will not be cloned again and the user must specify a new clone path or
-site repository. Suppose the repo name is airship-treasuremap and the clone
+pass in the local copy of the repository as the site repository. Suppose the
-path is /tmp/mypath then the following directory is created
+repo name is ``treasuremap`` and the clone path is ``/tmp/mypath`` then the
-/tmp/mypath/airship-treasuremap which will contain the contents of the repo.
+following directory is created ``/tmp/mypath/treasuremap`` which will contain
-Example of using clone path:
+the contents of the repo. Example of using clone path:
 
 ::
 
@@ -145,13 +145,13 @@ Options
 
 Path to the root of the site repository (containing site_definition.yaml) repo.
 
-For example: /opt/airship-treasuremap
+For example: /opt/airship/treasuremap
 
 The revision can also be specified via (for example):
 
 ::
 
-  -r /opt/airship-treasuremap@revision
+  -r /opt/airship/treasuremap@revision
 
 **-e / --extra-repository** (Optional).
 
@@ -166,13 +166,13 @@ These should be named per the site-definition file, e.g.:
 
 **-p / --clone-path** (Optional, Default=/tmp/).
 
-The path where the repo will be cloned. If this option is included and the
+The path where the repo will be cloned. By default the repo will be cloned to
-repo already exists, then the repo will not be cloned again and the user must
+the /tmp path. If this option is included and the repo already exists, then the
-specify a new clone path or pass in the local copy of the repository as the
+repo will not be cloned again and the user must specify a new clone path or
-site repository. Suppose the repo name is airship-treasuremap and the clone
+pass in the local copy of the repository as the site repository. Suppose the
-path is /tmp/mypath then the following directory is created
+repo name is ``treasuremap`` and the clone path is ``/tmp/mypath`` then the
-/tmp/mypath/airship-treasuremap which will contain the contents of the repo.
+following directory is created ``/tmp/mypath/treasuremap`` which will
-Example of using clone path:
+contain the contents of the repo. Example of using clone path:
 
 ::
 
@@ -443,7 +443,9 @@ Usage:
 Site Secrets Group
 ------------------
 
-Subgroup of :ref:`site-group`.
+Subgroup of :ref:`site-group`. The commands below create
+:ref:`PeglegManagedDocument manifests <pegleg-managed-document>` in the local
+repository.
 
 A sub-group of site command group, which allows you to perform secrets
 level operations for secrets documents of a site.
@@ -473,7 +475,7 @@ Generate PKI
 ^^^^^^^^^^^^
 
 Generate certificates and keys according to all PKICatalog documents in the
-site using the PKI module. The default behavior is to generate all
+site using the :ref:`pki` module. The default behavior is to generate all
 certificates that are not yet present. For example, the first time generate PKI
 is run or when new entries are added to the PKICatalogue, only those new
 entries will be generated on subsequent runs.

doc/source/index.rst

@@ -34,6 +34,7 @@ Design
 
    artifacts
    authoring_strategy
+   pki/index
 
 Developer's Guide
 -----------------

doc/source/pki/document-types.rst

@@ -0,0 +1,161 @@
+..
+  Copyright 2018 AT&T Intellectual Property.
+  All Rights Reserved.
+
+  Licensed under the Apache License, Version 2.0 (the "License"); you may
+  not use this file except in compliance with the License. You may obtain
+  a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+  License for the specific language governing permissions and limitations
+  under the License.
+
+.. _pegleg-document-types:
+
+Pegleg Document Types
+=====================
+
+Overview
+--------
+
+Pegleg is not only the custodian of deployment manifests that handles
+responsibilities such as aggregation and linting, but is also the author of
+certain `Deckhand-formatted`_ manifests. These manifests are generated via
+``Catalog`` classes.
+
+.. todo::
+
+  Provide documentation for what Catalog classes are.
+
+Documents
+---------
+
+Pegleg generates or ingests each of the documents below, each identified by
+its schema.
+
+.. _pegleg-managed-document:
+
+``pegleg/PeglegManagedDocument/v1``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Pegleg both generates and ingests this type of document. A
+``PeglegManagedDocument`` can have one or both of the following data elements:
+
+* ``generated``
+* ``encrypted``
+
+A ``PeglegManagedDocument`` serves as a wrapper around other documents, and the
+wrapping serves to capture additional metadata that is necessary, but
+separate from the managed document proper.
+
+The managed document data lives in the ``data.managedDocument`` portion
+of a ``PeglegManagedDocument``.
+
+Generated
+~~~~~~~~~
+
+If a ``PeglegManagedDocument`` is ``generated``, then its contents have been
+created by Pegleg, and it thus includes provenance information per this
+example::
+
+  schema: pegleg/PeglegManagedDocument/v1
+  metadata:
+    name: matches-document-name
+    schema: deckhand/Document/v1
+    labels:
+      matching: wrapped-doc
+    layeringDefinition:
+      abstract: true
+      # Pegleg will initially support generation at site level only
+      layer: site
+    storagePolicy: encrypted
+  data:
+    generated:
+      at: <timestamp>
+      by: <author>
+      specifiedBy:
+        repo: <...>
+        reference: <git ref-head or similar>
+        path: <PKICatalog/PassphraseCatalog details>
+    managedDocument:
+      schema: <as appropriate for wrapped document>
+      metadata:
+        storagePolicy: encrypted
+        schema: <as appropriate for wrapped document>
+        <metadata from parent PeglegManagedDocument>
+        <any other metadata as appropriate>
+      data: <generated data>
+
+Encrypted
+~~~~~~~~~
+
+If a ``PeglegManagedDocument`` is ``encrypted``, then its contents have been
+encrypted by Pegleg, and it thus includes provenance information per this
+example::
+
+  schema: pegleg/PeglegManagedDocument/v1
+  metadata:
+    name: matches-document-name
+    schema: deckhand/Document/v1
+    labels:
+      matching: wrapped-doc
+    layeringDefinition:
+      abstract: false
+      layer: matching-wrapped-doc
+    storagePolicy: encrypted
+  data:
+    encrypted:
+      at: <timestamp>
+      by: <author>
+    managedDocument:
+      schema: <as appropriate for wrapped document>
+      metadata:
+        storagePolicy: encrypted
+        schema: <as appropriate for wrapped document>
+        <metadata from parent PeglegManagedDocument>
+        <any other metadata as appropriate>
+      data: <encrypted string blob>
+
+Note that this ``encrypted`` has a different purpose than the Deckhand
+``storagePolicy: encrypted`` `metadata`_, which indicates an *intent* for
+Deckhand to store a document encrypted at rest in the cluster. The two can be
+used together to ensure security. If a document is marked as
+``storagePolicy: encrypted``, then automation may validate that it is only
+persisted (e.g. to a Git repository) if it is in fact encrypted within
+a ``PeglegManagedDocument``.
+
+Generated & Encrypted
+~~~~~~~~~~~~~~~~~~~~~
+
+A ``PeglegManagedDocument`` that is both generated via a ``Catalog``, and
+encrypted (as specified by the ``Catalog``) will contain both ``generated`` and
+``encrypted`` stanzas.
+
+Supported Managed Documents
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Supported managed document schemas include one of the following
+`Deckhand schemas`_:
+
+* Certificates:
+
+  * ``deckhand/Certificate/v1``
+  * ``deckhand/CertificateKey/v1``
+
+* Certificate Authorities:
+
+  * ``deckhand/CertificateAuthority/v1``
+  * ``deckhand/CertificateAuthorityKey/v1``
+
+* Keypairs:
+
+  * ``deckhand/PrivateKey/v1``
+  * ``deckhand/PublicKey/v1``
+
+.. _Deckhand-formatted: https://airship-deckhand.readthedocs.io/en/latest/users/documents.html
+.. _metadata: https://airship-deckhand.readthedocs.io/en/latest/users/encryption.html
+.. _Deckhand schemas: https://airship-deckhand.readthedocs.io/en/latest/users/document-types.html#provided-utility-document-kinds

doc/source/pki/index.rst

@@ -0,0 +1,35 @@
+..
+      Copyright 2018 AT&T Intellectual Property.
+      All Rights Reserved.
+
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+===============
+Secrets Catalog
+===============
+
+Artifacts
+---------
+
+.. toctree::
+   :maxdepth: 2
+
+   document-types
+
+Public Key Infrastructure (PKI)
+-------------------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   pki

doc/source/pki/pki.rst

@@ -0,0 +1,52 @@
+..
+  Copyright 2018 AT&T Intellectual Property.
+  All Rights Reserved.
+
+  Licensed under the Apache License, Version 2.0 (the "License"); you may
+  not use this file except in compliance with the License. You may obtain
+  a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+  License for the specific language governing permissions and limitations
+  under the License.
+
+.. _pki:
+
+Public Key Infrastructure (PKI) Catalog
+=======================================
+
+Configuration for certificate and keypair generation in the cluster.  The
+``pegleg generate-certs`` command will read all ``PKICatalog`` documents and
+either find pre-existing certificates/keys, or generate new ones based on the
+given definition.
+
+Dependencies
+------------
+
+Pegleg's PKI Catalog depends on `CloudFlare's PKI/TLS toolkit`_, which is
+installed as a part of Pegleg's `Dockerfile`_.
+
+.. _sample-pki-catalog-yaml:
+
+Sample Document
+---------------
+
+Here is a sample document:
+
+.. literalinclude:: ../../../site_yamls/site/pki-catalog.yaml
+
+Certificate Authorities
+-----------------------
+
+The data in the ``certificate-authorities`` key is used to generate
+certificates for each authority and node.
+
+Each certificate authority requires essential host-specific information for
+each node.
+
+.. _CloudFlare's PKI/TLS toolkit: https://github.com/cloudflare/cfssl
+.. _Dockerfile: https://github.com/openstack/airship-pegleg/blob/master/images/pegleg/Dockerfile