This patch set adds additional documentation and unit tests
to validate further replacement scenarios.
In particular this commit adds an additional document check that
looks for documents exisitng in different layers that contain the
same name and same schema without any of them having `replacement: true`
Change-Id: I7c033d32a6755f36e609789a748cbc6d4af06bc2
The document.py `is_control` method incorrectly checks if a document
is a Control document. Per the documentation [0], Control documents
have `metadata.schema` of `metadata/Control/v1`. This commit updates
the `is_control` method to correctly check for Control documents.
[0] 1d4cc81dfa/doc/source/users/document-types.rst (control-documents)
Change-Id: I60ca8f31a61987b4e756784fce0f5a751639ae9e
This package is used for generation autodoc documentation
automatically which can be linked to by Deckhand
documentation from other places. This is to make autodoc
generation work in RTD.
More info: https://pypi.org/project/sphinxcontrib-apidoc/
Change-Id: I43aac82728e5935a5a2626f2fd29d7a7188d19f9
I872b90e03a8f9e3de3fd717ed340b53f095f01a5 tried to fix a autodoc
issue with RTD but in turn introduced an issue causing RTD doc
publish job to fail; this fixes that.
Change-Id: I53fdcb609f8129f61abf8266e05ff49bfbef18b5
This patch set ensures that documents that substitute data from
encrypted document sources are themselves redacted, assuming that
cleartext-secrets=true. Note that this redaction fix only applies
to the substitution dest/src paths. The data section is already
being correctly redacted for secondhand sources.
Change-Id: I6ce16a109628259b2cc8132cd9db63261b5dbace
Arbitrary labels could be added as `make` parameter `LABEL=`, which is used in
att-comdev/cicd Jenkins pipelines.
Sample parameter:
'LABEL=org.label-schema.vcs-url=${GERRIT_CHANGE_URL} \
--label org.label-schema.base-image=${base_sha256}'
Sample usage:
See Jenkinsfile files under images/ directory in att-comdev/cicd repo.
In addition to that, if `COMMIT` variable is undefined when invoking `make`, we
use result of `git rev-parse HEAD` command, which should output latest git
commit ID.
[0] https://github.com/att-comdev/cicd
Change-Id: I795d1443f60f362c5ecb244cb90ee8052603854f
This patch set adds `./entrypoint.sh alembic upgrade head` command
to manual install section in Getting Started guide, without which
Deckhand server cannot start correctly.
Change-Id: Id486cc98b8fd93271a8571364adfb9fb30bf4bcc
This patch set refactors replacement validation checks
in Deckhand's layering module into a separate module for
better code organization.
Change-Id: If973148ac8220b96f61128b8a7266e6fd57e76b9
This patch set adds documentation related to data redaction in
order to capture the purpose behind it and how it adds an
added layer of security to Deckhand.
Change-Id: Icb28970684a9026fda200273a14d9ba421f627d7
- If a document has a storage policy of encrypted
- Redacts (sha256) the data section.
- Redacts (sha256) the substition paths.
- Uses the same /documents endpoint, adds a new query parameter
?cleartext-secrets=true to show the non-redacted values.
Change-Id: I42808901b97c667a1148c00fbb7717a0847c9981
Adds a unit test to validate following scenario:
1) create revision 1 with document
2) create revision 2 with no documents
3) rollback to revision 1 (creating revision 3)
Validate that diffing works for rolled-back revision.
All cases above use same bucket.
Also refactors some test logic for neatness.
Change-Id: I71bf7d34e8aae3ad5abb3c53b05cb96a7038ddc2
1. There is no exception called `InvalidRollback` in Deckhand (it
was removed a while back). Instead, the only exception that
db_api.revision_rollback raises is RevisionNotFound from
the revision_get call internally.
So catch that instead from the controller.
2. The default value of parameters is `str` so when revision_id
of '0' is passed to the db module for processing, it skips over
the check for `if revision_id == 0` as revision_id is a str,
not int. So this leverages builtin int converter logic in
falcon [0] but requires uplifting the version of falcon to
at least 1.3.0 to make use of it [1].
[0] https://falcon.readthedocs.io/en/stable/api/routing.html#field-converters
[1] https://falcon.readthedocs.io/en/1.3.0/api/routing.html#field-converters
Change-Id: I068cd9e9b6818a5d51501f2718ee2d40d556c094
This patch set pins down package requirements for the following
reasons:
* Inconsistencies between ranges and direct pins
* Deckhand isn't gated against requirements repo so changes to
some packages isn't actively tested
* Other Airship components use pins, so Deckhand might break them
when package requirements go out of sync between components
* Deckhand as of yet has no stable release so Airship relies on
master for stability so it is better to ensure stability over
anything else
Change-Id: I20ef4be3e01cd267771098d33447ccb61aee20b4
This patch set adds validation logic to document_validation.py (in
Deckhand's engine module) so that components that rely on
Deckhand's engine for document rendering (such as Promenade
or Pegleg) can fail fast when they provide Deckhand with a
duplicate document. Must pass pre_validate=True to layering
module which currently is the case for Promenade, et. al.
Before this change, Deckand only supported this logic at
the DB level (requiring service instantion); this is now no longer
the case.
Change-Id: I6d1c8214775aa0f3b5efb1049972cf847f74585b
Added container image tagging on quay.io with commit id on every merge,
when certain files but not the code are changed (docs, charts, etc.).
Previously we were building updated container images, adding tags and
uploading images onto quay.io only when the code has been changed. The
main reason for not re-building images for every single merged commit
was a long waiting time in Zuul `post` queue for a node (VM)
assignment for the job to run on.
With this change we will have image tag for every merged commit,
including documentation commits, and those tags would be added almost
immediately (within ~1 min), as job runs directly on Zuul and does not
wait in Zuul queue for a node (VM) assignment.
Change-Id: I9692947b74a16de0eb3e8100d30cd4310069abfe
This patchset elaborates on document layering in the documentation
to provide much greater clarity into what layering is and its
associated concepts, including: layer, layer order, layering policy,
layering definition, document abstraction, parent selection,
layering actions, etc.
Change-Id: I584e67b7984fa4035cef481a116ae3b8a3eb2906
This patch set corrects logic for an edge case in layering where
the action `path` is set to `.data`. In this case this means
that the root of the data section should be used, i.e. '.'
or '$.'. The previous adjustment was incorrect: .data was being
changed to empty string ''. This fixes that logic to change to
'.'.
Change-Id: Id6cf0d4d65020220c540eb162a33055035336cde
Pegleg has linting rules dedicated to checking for explicit starts
and so on, so it makes sense that Deckhand just adds this in for
every response as it is a nice feature that better delineates
starting and endpoints points for individual YAML documents.
Change-Id: I6324cfa268ddf250a9c78cb663e7015a171bbc19
Related-Change: https://review.openstack.org/#/c/604123
This patch set pins oslo.messaging to the last stable version
to fix the gate [0]. 9.0.0 is incompatible with current
requirements.
[0] https://pypi.org/project/oslo.messaging/#history
Change-Id: I6cf3e9f33ac3efcbf48675273899af38fef86e91
This patch set adds logic to optimize performance around rendered
documents caching further by implementing logic to skip over
post-rendering validation when cache hits occur. This works because:
* if the result set isn't cached, then post-validation is performed;
if the post-validation fails the just-cached result set is
invalidated
* if the result set is cached, then post-validation for that
result set has already been performed successfully, so it
can be safely skipped over
It was discovered via profiler anaylsis that document validation
accounts for an appreciable amount of run time, so optimizing
around this makes a noticeable difference.
Change-Id: I18344ea750cf1028a377028f80ecdd931d9672c0
This patch set reorganizes Deckhand's documentation structure
for better organization into 3 distinct categories:
* developer's guide
* operator's guide
* user's guide
This means that the RTD navigation menu on the left-hand side
will have fewer links (see list above) making navigation much
easier. This is similar to how Armada organizes its documentation
too.
This patch set also updates README section with a better
overview and trims some fat from it (remove testing
documentation as it doesn't really belong there -- there
is a dedicated page for that already).
Finally, this patch set changes the exceptions page to
render as a basic list of autoexception classes because
the current tabularized view is not rendering correctly
on RTD [0].
[0] https://airship-deckhand.readthedocs.io/en/latest/exceptions.html
Change-Id: I162383bf8e3bbd5004603c979ac7b0d760a210c4
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes
Change-Id: I12b2fa8fbec37a483a0ad50382e08f51ed97533a
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
a few other properties (annotations) according to the latest Specs
Change-Id: Iebb37edd003204d3adc41aa9af76612ab419993a