Commit Graph

209 Commits

Author SHA1 Message Date
Felipe Monteiro
9f216e03a8 pki: Add pki_utility unit tests
Add unit tests for most functionality in the pki_utility
module.

Change-Id: Icc2430067f1eb84bc9a83cb95aa207b7a7b423c8
2019-03-03 01:20:00 -05:00
Zuul
fe2484cb18 Merge "Fix secrets linting error" 2019-03-01 16:39:11 +00:00
Zuul
566c792658 Merge "Fix exception handling and add tests" 2019-02-28 21:53:33 +00:00
Lev Morgan
2596e7c840 Unwrap managed documents before linting
Unwrap managed documents during the linting process before passing
them to deckhand, to avoid namespace collisions.

Change-Id: I0467560154c737dc664a126241cd56257091125d
2019-02-26 20:51:45 +00:00
Lev Morgan
484772eb64 Fix secrets linting error
Fix an error where secrets in global directories are erroneously
flagged for being outside a secrets directory. Now, any file
that is a child of a directory called secrets should be handled
correctly.

Change-Id: I827aa75110d761601dc65df64e1accf1b1a54544
2019-02-25 20:23:11 -06:00
Drew Walters
32a9124c0d requirements: Update Deckhand version
This commit uplifts the version of Deckhand utilized in Pegleg to the
latest.

Change-Id: I5642f24eb10eab1b87605df13241f26327582b06
2019-02-25 20:00:13 +00:00
Tin Lam
aa241081c9 Fix exception handling and add tests
Per [0], fernet decrypt can never throw an InvalidSignature exception as that
is re-raised as InvalidToken. This patch set corrects the handling of the
exception, and added additional unit tests for coverage.

[0] https://cryptography.io/en/latest/fernet/#cryptography.fernet.Fernet.decrypt

Co-Authored-By: Drew Walters <drewwalters96@gmail.com>
Change-Id: Ic5ee7ef451a5657519c5397fc4b903b5adcb1d18
Signed-off-by: Tin Lam <tin@irrational.io>
2019-02-22 16:14:25 -06:00
Alexander Hughes
1aa46d77af Update CLI documentation for generate salt and generate passphrase
Documentation previously did not clearly indicate behavior of invalid lengths
Lengths less than 24 default to 24.

Change-Id: I490ac840f4ed215e6bbbc6cc823eb958076c411c
2019-02-22 10:50:05 -06:00
Zuul
56b3e10cae Merge "Add CLI generation of salt" 2019-02-21 19:27:22 +00:00
Alexander Hughes
671b77f6a7 Add CLI generation of salt
Salts and Passphrases are both strings used in cryptography.  This patch:
1. Adds CLI generation of salt
2. Adds unit test for CLI generation of salt
3. Updates passphrase.py code to be more generic as it is used to generate
both a passphrase and a salt
4. Update name of passphrase.py to be more generic
5. Update all references to, and tests of passphrase.py
6. Add documentation for CLI generation of salt

Co-Authored-By: chittibabu <cg329x@att.com>

Change-Id: I71858d63a2846290d22be96686ccfea3ba8aa6c0
2019-02-20 16:51:01 +00:00
Lev Morgan
af8016511a Fixed metadata schema for managed documents
Metadata schema for managed documents should now be metadata/Document/v1,
not deckhand/Document/v1.

Change-Id: Id500fc7d5b68a4c296848c07a9644f375e949289
2019-02-13 20:00:32 -06:00
Alexander Hughes
4b00a4340c Add CLI passphrase generation
1. Add support to pegleg to generate a passphrase from CLI
2. Update unit test to ensure encryption/decryption supports passphrase rotation
3. Update order of import statements to satisfy pep8
4. Add unit test for CLI passphrase generation
5. Resolve merge conflicts via rebase

Change-Id: I5cb9e41b2f0fac2451bd2b74f33c48cda417c22d
2019-02-04 12:32:39 -06:00
Tin Lam
e6af6ae87e Fix a tox mismatch
This is patch set adds in a trivial fix for a mismatch between a comment
and the actual hacking rule that is being enabled.

Change-Id: I7b1b6146f0cc0222d050e9aabd42a965af63730f
Signed-off-by: Tin Lam <tin@irrational.io>
2019-02-02 20:30:48 +00:00
pallav
b79d5b7a98 CLI capability to generate and encrypt passphrases
1. Adds the passphrases generation capability in Pegleg CLI,
so that pegleg can generation random passwords based on a
specification declared in pegleg/PassphrasesCatalog documents
2. Pegleg also wraps the generated passphrase documents in
pegleg managed documents, and encrypts the data.
3. Adds unit test cases for passphrase generation.
4. Updates pegleg CLI document.

Change-Id: I21d7668788cc24a8e0cc9cb0fb11df97600d0090
2019-01-29 16:24:31 -06:00
Zuul
1de8d5b68f Merge "pki: Port Promenade's PKI catalog into Pegleg" 2019-01-26 00:29:39 +00:00
Roman Gorshunov
ef63344332 Typo fix: getting started URL
Change-Id: Ia1082d2625c37570907cea07d4437ab24e5376a2
2019-01-23 09:35:27 +01:00
Felipe Monteiro
2a8d2638b3 pki: Port Promenade's PKI catalog into Pegleg
This patch set implements the PKICatalog [0] requirements
as well as PeglegManagedDocument [1] generation requirements
outlined in the spec [2].

Included in this patch set:

* New CLI entry point called "pegleg site secrets generate-pki"
* PeglegManagedDocument generation logic in
  engine.cache.managed_document
* Refactored PKICatalog logic in engine.cache.pki_catalog derived
  from the Promenade PKI implementation [3], responsible for
  generating certificates, CAs, and keypairs
* Refactored PKIGenerator logic in engine.cache.pki_generator
  derived from Promenade Generator implementation [4],
  responsible for reading in pegleg/PKICatalog/v1 documents (as
  well as promenade/PKICatalog/v1 documents for backwards
  compatibility) and generating required secrets and storing
  them into the paths specified under [0]
* Unit tests for all of the above [5]
* Example pki-catalog.yaml document under pegleg/site_yamls
* Validation schema for pki-catalog.yaml (TODO: implement
  validation logic here: [6])
* Updates to CLI documentation and inclusion of PKICatalog
  and PeglegManagedDocument documentation
* Documentation updates with PKI information [7]

TODO (in follow-up patch sets):

* Expand on overview documentation to include new Pegleg
  responsibilities
* Allow the original repository (not the copied one) to
  be the destination where the secrets are written to
* Finish up cert expiry/revocation logic

[0] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
[1] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
[2] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html
[3] https://github.com/openstack/airship-promenade/blob/master/promenade/pki.py
[4] https://github.com/openstack/airship-promenade/blob/master/promenade/generator.py
[5] https://review.openstack.org/#/c/611739/
[6] https://review.openstack.org/#/c/608159/
[7] https://review.openstack.org/#/c/611738/

Change-Id: I3010d04cac6d22c656d144f0dafeaa5e19a13068
2019-01-15 13:29:21 -06:00
Felipe Monteiro
40da373023 tests: Increase test coverage for lint checks
This patch set expands on the unit test coverage for lint checks
in test_selectable_linting which only covers a small subset of
the lint checks handled by Pegleg. This logic should be properly
tested as linting is fundamental to Pegleg functionality.

Change-Id: I6a59295982abd22bba8036827cefd4186b68e2fb
2019-01-01 19:17:11 +00:00
Zuul
a019d131d1 Merge "docs: Document all Pegleg lint codes" 2019-01-01 18:13:08 +00:00
Zuul
b9747a8beb Merge "Update docs to replace aic-site-clcp-manifests with airship-treasuremap" 2019-01-01 18:13:08 +00:00
Tin Lam
1a325a400b Add hacking extension
This patch set adds hacking rule to pegleg and fixes outstanding non-
docstring related violations.

Change-Id: I5bb5e78c211f24cf95669124bfcf9603bea8bf15
Signed-off-by: Tin Lam <tin@irrational.io>
2019-01-01 00:18:03 -06:00
Evgeny L
d3010a213f Update docs to replace aic-site-clcp-manifests with airship-treasuremap
Change-Id: Ic53d5d768fcecaac07b4703ce818fe73a52d1c56
2018-12-26 21:01:15 +00:00
Zuul
213ca596e3 Merge "trivial: fix whitespace-linter catching false positives" 2018-12-23 18:55:01 +00:00
Zuul
99fe65952e Merge "Add releasenotes" 2018-12-04 20:43:56 +00:00
Tin Lam
dc1d36ba1b Add releasenotes
This patch set adds releasenotes (reno) to the airship-pegleg repository.

Change-Id: I1ed98460f40eb851ca53f8a5b68ab17f8272a2e2
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-28 02:48:40 -06:00
Nishant Kumar
9113d249ff CLI: Add support for uploading documents to Shipyard
This PS enables Pegleg to upload documents directly to Shipyard
thus ensuring that unencrypted data never gets stored in disk.

The flow for this new CLI command is as follows:

- Collect documents as per the provided site repository
- Decrypt the collected documets(TODO)
- Upload document to Shipyard:
  - one collection per repository will be uploaded to Shipyard

    Eg-
        pegleg site -r /opt/aic-clcp-site-manifests \
        -e global=/opt/aic-clcp-manifests upload <site-name>

        Two collections will be created in shipyard since there are two
        repositories provided. The name of the collections will be the
        name of repositories provided.
  - Commit the documents in shipyard buffer.

Change-Id: I6275252b044ebb82d8bb2009c0bea6ebf7033bce
2018-11-27 19:29:42 +00:00
zhangdebo
e3159d223a Fix link addresses
Change-Id: I7086b4cac5e3bcd2ac2a1958b678615e05ccbd1d
2018-11-27 20:09:48 +08:00
Felipe Monteiro
e405dc4fd7 docs: Document all Pegleg lint codes
This adds Pegleg's lint codes to the operator documentation
to obviously assist operators with figuring out failing Pegleg
lint checks.

Change-Id: I6742af48c563f6dd4b4f1d363b678cdcd000347e
2018-11-25 17:36:16 +00:00
Felipe Monteiro
e3f6efbb1f trivial: fix whitespace-linter catching false positives
Ignores .pyc files and files contained in htmlcov as a coverage
job was recently added.

Change-Id: I33b1b88e90060f3b26f41b6acbbc0c7091caeb57
2018-11-25 12:30:56 -05:00
Tin Lam
a3da86e311 Flake8 fix
This removes all PEP8 ignores and places in default settings for flake8.

Change-Id: I3c4df02dea959dfe58f44e7c0e0ac58078a81abc
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-13 14:50:42 -06:00
Zuul
b7556bd89e Merge "Fix URLs" 2018-11-13 17:44:58 +00:00
Zuul
93cd0f5d1e Merge "Adding missing format character" 2018-11-13 16:50:50 +00:00
Zuul
f6cfb1cf2a Merge "Refactor Pegleg exceptions" 2018-11-13 16:50:49 +00:00
Zuul
ec2ed74f05 Merge "Consolidate linter jobs" 2018-11-13 15:52:49 +00:00
Tin Lam
ae81ade94d Refactor Pegleg exceptions
This patch set cleans up the current implementation of pegleg.  As
all the git exceptions inconsistently inits or override the message.
This also cleans up a handful of incorrect kwarg keywords, and improve
handling of the exception messages.

Change-Id: I438eb032728c71cbf972c2120a76d06106cb1580
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-13 08:58:54 -06:00
Zuul
1f59d7e1e0 Merge "Remove the duplicated "and"." 2018-11-12 10:35:07 +00:00
zhangzs
a0be40846c Remove the duplicated "and".
Change-Id: I7cba441990d5bf67548982f2a2738e43f0a28b27
2018-11-12 17:15:51 +08:00
Tin Lam
152b42bc3b Consolidate linter jobs
Consolidating all the linting to a single job rather than having a one-off
just to lint for trailing whitespaces. As most of this projects are python,
this should already be covered by PEP8, but will be scanning for yamls with
trailing whitespaces.

Change-Id: Iee33a69ff234d21c08217faa33a19e11dfef0ad9
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-12 02:30:20 -06:00
Zuul
03634de6c8 Merge "Remove py35 support" 2018-11-11 22:22:13 +00:00
Tin Lam
2a0e1ac1d4 Fix URLs
This patch set sanitizes and changes the default proxy URLs.

Change-Id: Ibaeb501b59eeea8427fa812078236cfc4fe39623
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-09 10:25:39 -06:00
Felipe Monteiro
f8d79e119c Only collect/parse Deckhand-formatted documents for processing
This patch set changes Pegleg in two similar ways:

1) Ignore certain types of files altogether:
   - those located in hidden folders
   - those prefixed with "." (files like .zuul.yaml)
2) Only read Deckhand-formatted documents for lint/collect/etc.
   commands as Pegleg need not consider other types of documents
   (it separately reads the site-definition.yaml for internal
    processing still).

The tools/ subfolder is also ignored as it can contain
.yaml files which are not Deckhand-formatted documents,
so need not be processed by pegleg.engine.

Change-Id: I8996b5d430cf893122af648ef8e5805b36c1bfd9
2018-11-08 20:07:03 -05:00
Roman Gorshunov
d7740b0f40 Minor: meaningful default label
Making default label meaningful and conformant to "key=value" requirement.

Change-Id: Ibb915e3333d74f61742fbb9c4f3c7851d2fef040
2018-11-08 19:32:19 +01:00
Zuul
348428ca32 Merge "Update decrypt secrets to return a list of docs" 2018-11-07 20:49:49 +00:00
Tin Lam
253b15cc33 Remove py35 support
As the default Docker image support for Pegleg is using py36 as a base
image, this patch set simply removes the support for py35. This allows
us to use py36 features such as secrets and f-strings without needing to
cope with backward compatibility.

Change-Id: Ic156ca67d2a0fe6fdc74c2c9da253a1cb8a3f456
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-07 13:04:34 -06:00
Tin Lam
178c058474 Add coverage job
This patch set adds in a python coverage job for pegleg using pytest.

Change-Id: Ice0332cc454f488e0372d5ab03e8b0a37db9efd1
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-06 20:59:01 -06:00
Ahmad Mahmoudi
fb8e6f73ac Update decrypt secrets to return a list of docs
1. Added the method to decrypt a secret file and return its contents
as a list of documents (instead of printing out the file content).
2. Added clarifications for a encrypt and decrypt commands.

Change-Id: I77bce21be214c880c8413f5e6a2d0c2d1993fc8e
2018-11-06 00:22:10 -06:00
Lev Morgan
a912c8beb9 Adding missing format character
It looks like there's a missing % in pegleg/engine/util/definition.py, where
pluck creates its ClickException. This adds it in.

Change-Id: I26c12423af11f7d02944a077f762bcdd1e03be41
2018-11-02 15:55:01 -05:00
Zuul
c7270554da Merge "FIX: Passing correct fields to process site" 2018-11-02 15:03:04 +00:00
Tin Lam
025fa74939 Update .gitignore
Update the .gitignore to be more comprehensive and that it is more
inline with airship-deckhand's .gitignore.

Change-Id: I0f2860034d060461a441be65feea274ef1fef30a
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-31 15:08:42 -05:00
Nishant Kumar
cfa57e09a7 FIX: Passing correct fields to process site
This PS passes correct fields which is required for processing
site_definiton.yaml appropriately.

Change-Id: I546caa1a034fe1fd05c3568137982cb7b4ef7bec
2018-10-31 16:58:10 +00:00