Adding DISTRO parameter for makefile to invoke distribution specific
Dockerfile and build image accordingly.
Updated logic for existing jobs to have distro specific logic
for building and publishing images.
Added multiple distro specific document in operator section.
Change-Id: I415ab28b06ea17b21e76d28ccb3e284041c8072d
Remove manual generation of UML diagrams; they will be built by Sphinx
itself.
Require to install PlantUML and Graphviz via bindep in
openstack-tox-docs job; ReadTheDocs service already has both PlantUML
and Graphviz installed for documentation builds.
Change-Id: I21ab124a80e7768bc9edd891b975b4e0f8a6e50d
Story: 2004747
This patch set adds use cases for each of the mutation
operations used by Deckhand:
* substitution
* layering
* replacement
To help document authors decide which operations they should use.
Change-Id: I8bc470c7ddc2f3c0aa9d4372a2323fa84632e369
This patch set adds additional documentation and unit tests
to validate further replacement scenarios.
In particular this commit adds an additional document check that
looks for documents exisitng in different layers that contain the
same name and same schema without any of them having `replacement: true`
Change-Id: I7c033d32a6755f36e609789a748cbc6d4af06bc2
This PS adds configuration documentation that includes
a literalinclude of the config file as well as some
information on each of the cache config options as
these are important on performance.
Change-Id: I3b06012b8843b7bfbd46307f81397172a41d3675
This package is used for generation autodoc documentation
automatically which can be linked to by Deckhand
documentation from other places. This is to make autodoc
generation work in RTD.
More info: https://pypi.org/project/sphinxcontrib-apidoc/
Change-Id: I43aac82728e5935a5a2626f2fd29d7a7188d19f9
This patch set adds `./entrypoint.sh alembic upgrade head` command
to manual install section in Getting Started guide, without which
Deckhand server cannot start correctly.
Change-Id: Id486cc98b8fd93271a8571364adfb9fb30bf4bcc
This patch set adds documentation related to data redaction in
order to capture the purpose behind it and how it adds an
added layer of security to Deckhand.
Change-Id: Icb28970684a9026fda200273a14d9ba421f627d7
- If a document has a storage policy of encrypted
- Redacts (sha256) the data section.
- Redacts (sha256) the substition paths.
- Uses the same /documents endpoint, adds a new query parameter
?cleartext-secrets=true to show the non-redacted values.
Change-Id: I42808901b97c667a1148c00fbb7717a0847c9981
This patch set pins down package requirements for the following
reasons:
* Inconsistencies between ranges and direct pins
* Deckhand isn't gated against requirements repo so changes to
some packages isn't actively tested
* Other Airship components use pins, so Deckhand might break them
when package requirements go out of sync between components
* Deckhand as of yet has no stable release so Airship relies on
master for stability so it is better to ensure stability over
anything else
Change-Id: I20ef4be3e01cd267771098d33447ccb61aee20b4
This patchset elaborates on document layering in the documentation
to provide much greater clarity into what layering is and its
associated concepts, including: layer, layer order, layering policy,
layering definition, document abstraction, parent selection,
layering actions, etc.
Change-Id: I584e67b7984fa4035cef481a116ae3b8a3eb2906
This patch set reorganizes Deckhand's documentation structure
for better organization into 3 distinct categories:
* developer's guide
* operator's guide
* user's guide
This means that the RTD navigation menu on the left-hand side
will have fewer links (see list above) making navigation much
easier. This is similar to how Armada organizes its documentation
too.
This patch set also updates README section with a better
overview and trims some fat from it (remove testing
documentation as it doesn't really belong there -- there
is a dedicated page for that already).
Finally, this patch set changes the exceptions page to
render as a basic list of autoexception classes because
the current tabularized view is not rendering correctly
on RTD [0].
[0] https://airship-deckhand.readthedocs.io/en/latest/exceptions.html
Change-Id: I162383bf8e3bbd5004603c979ac7b0d760a210c4
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes
Change-Id: I12b2fa8fbec37a483a0ad50382e08f51ed97533a
GET /revisions/{{revision_id}}/deepdiff/{{comparison_revision_id}}
- Added deepdiff api for generating diff between
two rendered documents.
- Deep diffing for data and metadata
- Refactor diff functions
- Client update
- Added unit testcases
- Added funtional testcases
- Doc update
Change-Id: Ib60fa60a3b33e9125a1595a999272ca595721b38
Patterns may now be replaced recursively. This can be
achieved by using specifying a ``pattern`` value and
``recurse`` (with a required ``depth`` argument).
Example:
substitutions:
- dest:
path: .
pattern: REGEX
recurse:
depth: -1
src:
schema: deckhand/Passphrase/v1
name: example-password
path: .
NOTE:
Recursive selection of patterns will only consider
matching patterns. Non-matching patterns will be ignored.
Thus, even if recursion can "pass over" non-matching patterns,
they will be silently ignored.
This is useful for reducing the number of substitution
stanzas that are required for the purposes of performing
all the string pattern replacements that are required.
Best practice is to limit the scope of the recursion
as much as possible: e.g. avoid passing in "$" as the
``jsonpath``, but rather a JSON path that lives closer
to the nested strings in question.
Included in this patch set:
* recursive implementation for pattern replacement
* unit tests for most scenarios and edge cases
* docstring updates
* documentation updates
* schema updates for validation
Change-Id: I85048349097ed696667fae80f1180808d264bbcf
This patch set updates document types documentation
which includes expounding on control documents, layering policy
documents, and dataschema documents.
Change-Id: Id31fcb6b68ca30fdf681dec8348c2fc4237cd48d
This renames some instances of ucp to airship in the documentation
and makes some trivial documentation fixes.
Change-Id: I9a4a81d15bfc13b4fe089b7d65f0df43eeade9fb
This patchset moves retrieval of encrypted documents to the
Deckhand controller so that components like Pegleg and
Promenade can consume the Deckhand engine offline without
running into Barbican errors.
Components can pass in `encryption_sources` to Deckhand's
rendering module which Deckhand will now use instead to resolve
secret references.
`encryption_sources` is a dictionary that maps the reference
contained in the destination document's data section to the
actual unecrypted data. If encrypting data with Barbican, the
reference will be a Barbican secret reference.
Change-Id: I1a457d3bd37101d73a28882845c2ce74ac09fdf4
- Treat internal Deckhand schemas equivalent to other
service schemas
- Remove validating sections other than `data` outside of
base schema
- Create schemas for metadata sections metadata/Control/v1 and
metadata/Document/v1
- Use a single validator and let that validator check for document
structure (validate against the base schema and metadata)
and for post-validation also validate against service schemas
Change-Id: I5f9b9a3cfa1692a69b5982a6424edd65bdfed0ef
This patchset adds developer overview documentation for providing
a high-level introduction to Deckhand, including its architecture,
modules, test utilities, Helm utilities, and other errata. This
work is based off this Drydock patchset:
https://review.openstack.org/#/c/571298/
Change-Id: Ic3382d4e04edf02a65184651d272fe9cd1db56a4
This patchset expands on document uniquness in Deckhand which is
a bit involved.
At the **database** level, documents are uniquely identified by
the combination of:
#. ``metadata.name``
#. ``schema``
#. ``metadata.layeringDefinition.layer``
This means that raw revision documents -- which are persisted
in Deckhand's database -- require that the combination of all
3 parameters be unique.
However, **post-rendered documents** are only uniquely
identified by the combination of:
#. ``metadata.name``
#. ``schema``
Because collisions with respect to the the third parameter --
``metadata.layeringDefinition.layer`` -- can only occur with
replacement. But after document rendering, the replacement-parent
documents are never returned.
Change-Id: I841caa83bef46bf5e155136fbcd7444b672d06f8
This PS adds an integration test scenario for validating that
encrypting a generic document type and using it as a substitution
source during document rendering works.
Deckhand will now submit all generic documents to be encrypted
to Barbican with a 'secret_type' of 'passphrase'. No encoding
is provided Deckhand-side (i.e. base64) because encoding is
deprecated in Barbican since it lead to strange behavior;
Barbican will figure out what to encode the payload as
automatically. For more information, see [0] and [1].
In addition, this PS handles 2 edge cases around secret
payloads that are rejected by Barbican if not handled
correctly by Deckhand: empty payloads and non-string
type payloads [2]. For the first case Deckhand forcibly
changes the document to cleartext because there is no
point in encrypting a document with an empty payload.
For the second case Deckhand sets overrides any
previously set secret_type to 'opaque' and encodes
the payload to base64 -- when it goes to render
the secret it decodes the payload also using base64.
Integration tests have been added to handle both edge
cases described above.
[0] https://bugs.launchpad.net/python-barbicanclient/+bug/1419166
[1] 49505b9aec/barbicanclient/v1/secrets.py (L252)
[2] 49505b9aec/barbicanclient/v1/secrets.py (L297)
Change-Id: I1964aa84ad07b6f310b39974f078b84a1dc84983
This patchset adds documentation on document encryption. It also
adds appropriate references to other sections of the documentation
where linking to document encryption is apropos. Finally, this
patchset adds further information and structure around the document
rendering process in general, in order to underscore the pieces
of documentation that undergird the general concept of document
rendering: substitution, layering and replacement.
Change-Id: I566cac4a3374556a0e62e0e1962e153029f7ec05
This patchset updates docs to doc to align with OpenStack
standard. Follow-up patchset will be needed to publish
documentation to OpenStack [0].
[0] https://docs.openstack.org/doc-contrib-guide/project-guides.html
Change-Id: Ia191ac1cc4536af1232aedd4bb491f3829651730
This PS integrates ValidationPolicy logic with the
Deckhand Validations API.
Support for multiple ValidationPolicy documents is
included.
If a ValidationPolicy is found, then the validations
contained therein are used to determine whether
a revision is successful or not. For example,
if a VP contains 'promenade-schema-validation' then
DH will return success if the externally registered
validation result for that validation is success.
However, if the result was never registered in DH
then the returned result is 'failure'.
In addition, if "extra" validations are registered
(that is validations not present in any VP) then
they are effectively ignored. An error message is
added with enough details to indicate why the validation
is ignored.
This PS adds unit tests to verify the correct behavior
for the above scenarios.
Functional tests and documentation changes will be added
in a follow up once design is ironed out.
Change-Id: I44c657974589ea3563e0a23ad667894329048b46
This PS removes microversions from document versions because
no services use microversions and microversioning for documents
isn't supported in Deckhand.
Change-Id: I3635d15513a2c7b8154ec6be4d0b8577e7d4ce3d
Use YAML formatting for built-in Deckhand schemas
used for validations to align with other UCP services.
The second most important intention behind this PS
is to allow pre_validate flag to cascade correctly
between the layering and document_validation modules.
If pre_validate is true, then:
* the base_schema validates ALL documents
* ALL built-in schemas validate the appropriate
document given a schema match
* NO externally registered DataSchema documents
are used for validation
Else (if pre_validate is false):
* the base_schema validates ALL documents
* ALL built-in schemas validate the appropriate
document given a schema match
* ALL externally registered DataSchema documents
are used for validation given a schema match
A more minor change is setting pre_validate flags in
all modules to True for consistency. The idea is to
facilitate the way other projects that import Deckhand
in directly interface with Deckhand.
Change-Id: I859f61989ec15bede1c104b86625d116064f056d
This is to fix up a slight rendering issue with getting
started documentation related to double code block syntax.
Change-Id: I4b6777f77d148113e5a798aeb3daa520d05c1f0e
Update the README by moving a lot of the manual installation into
Getting Started docs and simplify some of the documentation because
Deckhand can be re-run using sqlite (for dev purposes, for example).
Change-Id: I5742dc75b95e2af67a18b419d04e769c10a1e43e
