Browse Source

Update decrypt secrets to return a list of docs

1. Added the method to decrypt a secret file and return its contents
as a list of documents (instead of printing out the file content).
2. Added clarifications for a encrypt and decrypt commands.

Change-Id: I77bce21be214c880c8413f5e6a2d0c2d1993fc8e
Ahmad Mahmoudi 5 months ago
parent
commit
fb8e6f73ac

+ 56
- 13
doc/source/cli/cli.rst View File

@@ -397,6 +397,20 @@ Secrets
397 397
 A sub-group of site command group, which allows you to perform secrets
398 398
 level operations for secrets documents of a site.
399 399
 
400
+.. note::
401
+
402
+  For the CLI commands ``encrypt`` and ``decrypt`` in the ``secrets`` command
403
+  group, which encrypt or decrypt site secrets, two  environment variables,
404
+  ``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``, are  used to capture the
405
+  master passphrase, and the salt needed for encryption and decryption of the
406
+  site secrets. The contents of ``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``
407
+  are not generated by Pegleg, but are created externally, and set by a
408
+  deployment engineers or tooling.
409
+
410
+  A minimum length of 24 for master passphrases will be checked by all CLI
411
+  commands, which use the ``PEGLEG_PASSPHRASE``. All other criteria around
412
+  master passphrase strength are assumed to be enforced elsewhere.
413
+
400 414
 ::
401 415
 
402 416
   ./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>
@@ -406,26 +420,52 @@ Encrypt
406 420
 ^^^^^^^
407 421
 
408 422
 Encrypt one site's secrets documents, which have the
409
-metadata.storagePolicy set to encrypted, and wrap them in `pegleg managed
410
-documents <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_.
423
+``metadata.storagePolicy`` set to encrypted, and wrap them in
424
+`Pegleg Managed Documents`_
425
+
426
+.. note::
411 427
 
412
-**Note**: The encrypt command is idempotent. If the command is executed more
413
-than once for a given site, it will skip the files, which are already
414
-encrypted and wrapped in a pegleg managed document, and will only encrypt the
415
-documents not encrypted before.
428
+  The encrypt command is idempotent. If the command is executed more
429
+  than once for a given site, it will skip the files, which are already
430
+  encrypted and wrapped in a pegleg managed document, and will only encrypt the
431
+  documents not encrypted before.
416 432
 
417 433
 **site_name** (Required).
418 434
 
419
-Name of the site.
435
+Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
436
+repository folder structure. The ``encrypt`` command looks up the
437
+``site-name`` in the site repository, and searches recursively the
438
+``site_name`` folder structure for secrets files (i.e. files with documents,
439
+whose ``encryptionPolicy`` is set to ``encrypted``), and encrypts the
440
+documents in those files.
420 441
 
421 442
 **-a / --author** (Required)
422 443
 
423
-Identifier for the program or person who is encrypting the secrets documents.
444
+Author is the identifier for the program or the person, who is encrypting
445
+the secrets documents.
446
+Author is intended to document the entity or the individual, who
447
+encrypts the site secrets documents, mostly for tracking purposes, and is
448
+expected to be leveraged in an operator-specific manner.
449
+For instance the ``author`` can be the "userid" of the person running the
450
+command, or the "application-id" of the application executing the command.
424 451
 
425 452
 **-s / --save-location** (Optional).
426 453
 
427
-Where to output encrypted and wrapped documents. If omitted, the results
428
-will overwrite the original documents.
454
+Where to output the encrypted and wrapped documents.
455
+
456
+.. warning::
457
+
458
+  If the ``save-location`` parameter is not provided, the encrypted result
459
+  documents will overwrite the original ``cleartext`` documents for the site.
460
+  The reason for this default behavior, is to ensure that site secrets are
461
+  only stored on disk or in any version control system as encrypted.
462
+
463
+  If the user for any reason wants to avoid overwriting the original
464
+  cleartext files, the ``save-location`` parameter will provide the option to
465
+  override this default behavior, and forces the encrypt command to write
466
+  the encrypted documents in a different location than the original
467
+  unencrypted files.
468
+
429 469
 
430 470
 Usage:
431 471
 
@@ -457,14 +497,16 @@ Example without optional save location:
457 497
 Decrypt
458 498
 ^^^^^^^
459 499
 
460
-Unwrap an encrypted secrets document from a `pegleg managed
461
-document <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_,
500
+Unwrap an encrypted secrets document from a `Pegleg Managed Documents`_,
462 501
 decrypt the encrypted secrets, and dump the cleartext secrets file to
463 502
 ``stdout``.
464 503
 
465 504
 **site_name** (Required).
466 505
 
467
-Name of the site.
506
+Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
507
+repository folder structure. The ``decrypt`` command also validates that the
508
+``site-name`` exists in the file path, before unwrapping and decrypting the
509
+documents in the ``filename``.
468 510
 
469 511
 **-f / filename** (Required).
470 512
 
@@ -598,3 +640,4 @@ P003 - All repos contain expected directories.
598 640
 
599 641
 .. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html
600 642
 .. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html
643
+.. _Pegleg Managed Documents: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument

+ 1
- 1
pegleg/engine/secrets.py View File

@@ -74,7 +74,7 @@ def decrypt(file_path, site_name):
74 74
     :param file_path: Path to the file to be unwrapped and decrypted.
75 75
     :type file_path: string
76 76
     :param site_name: The name of the site to search for the file.
77
-    :type site_name: string providing the site name
77
+    :type site_name: string
78 78
     """
79 79
 
80 80
     LOG.info('Started decrypting...')

+ 35
- 10
pegleg/engine/util/pegleg_secret_management.py View File

@@ -34,17 +34,29 @@ ENV_SALT = 'PEGLEG_SALT'
34 34
 class PeglegSecretManagement():
35 35
     """An object to handle operations on of a pegleg managed file."""
36 36
 
37
-    def __init__(self, file_path):
37
+    def __init__(self, file_path=None, docs=None):
38 38
         """
39 39
         Read the source file and the environment data needed to wrap and
40 40
         process the file documents as pegleg managed document.
41
+        Either of the ``file_path`` or ``docs`` must be
42
+        provided.
41 43
         """
42 44
 
45
+        if all([file_path, docs]) or \
46
+            not any([file_path, docs]):
47
+            raise ValueError(
48
+                'Either `file_path` or `docs` must be specified.')
49
+
43 50
         self.__check_environment()
44 51
         self.file_path = file_path
45 52
         self.documents = list()
46
-        for doc in files.read(file_path):
47
-            self.documents.append(PeglegManagedSecret(doc))
53
+        if docs:
54
+            for doc in docs:
55
+                self.documents.append(PeglegManagedSecret(doc))
56
+        else:
57
+            self.file_path = file_path
58
+            for doc in files.read(file_path):
59
+                self.documents.append(PeglegManagedSecret(doc))
48 60
 
49 61
         self.passphrase = os.environ.get(ENV_PASSPHRASE).encode()
50 62
         self.salt = os.environ.get(ENV_SALT).encode()
@@ -119,9 +131,27 @@ class PeglegSecretManagement():
119 131
         included in a site secrets file, and print the result to the standard
120 132
         out."""
121 133
 
134
+        yaml.safe_dump_all(
135
+            self.get_decrypted_secrets(),
136
+            sys.stdout,
137
+            explicit_start=True,
138
+            explicit_end=True,
139
+            default_flow_style=False)
140
+
141
+    def get_decrypted_secrets(self):
142
+        """
143
+        Unwrap and decrypt all the pegleg managed documents in a secrets
144
+        file, and return the result as a list of documents.
145
+
146
+        The method is idempotent. If the method is called on not
147
+        encrypted files, or documents inside the file, it will return
148
+        the original unwrapped and unencrypted documents.
149
+
150
+        """
151
+
122 152
         doc_list = []
123 153
         for doc in self.documents:
124
-            # only decrypt an encrypted document
154
+            # do not decrypt already decrypted data
125 155
             if doc.is_encrypted():
126 156
                 doc.set_secret(
127 157
                     decrypt(doc.get_secret(),
@@ -129,9 +159,4 @@ class PeglegSecretManagement():
129 159
                             self.salt).decode())
130 160
                 doc.set_decrypted()
131 161
             doc_list.append(doc.embedded_document)
132
-        yaml.safe_dump_all(
133
-            doc_list,
134
-            sys.stdout,
135
-            explicit_start=True,
136
-            explicit_end=True,
137
-            default_flow_style=False)
162
+        return doc_list

+ 62
- 6
tests/unit/engine/test_encryption.py View File

@@ -22,12 +22,14 @@ import yaml
22 22
 
23 23
 from pegleg.engine.util import encryption as crypt
24 24
 from tests.unit import test_utils
25
-from pegleg.engine import secrets
26 25
 from pegleg.engine.util.pegleg_managed_document import \
27 26
     PeglegManagedSecretsDocument
28 27
 from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
29 28
 from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
30 29
 from pegleg.engine.util.pegleg_secret_management import ENV_SALT
30
+from tests.unit.fixtures import temp_path
31
+from pegleg.engine.util import files
32
+
31 33
 
32 34
 TEST_DATA = """
33 35
 ---
@@ -57,8 +59,9 @@ def test_encrypt_and_decrypt():
57 59
     assert data == dec2
58 60
 
59 61
 
60
-@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'aShortPassphrase',
61
-                              ENV_SALT: 'MySecretSalt'})
62
+@mock.patch.dict(os.environ, {
63
+    ENV_PASSPHRASE:'aShortPassphrase',
64
+    ENV_SALT: 'MySecretSalt'})
62 65
 def test_short_passphrase():
63 66
     with pytest.raises(click.ClickException,
64 67
                        match=r'.*is not at least 24-character long.*'):
@@ -72,9 +75,21 @@ def test_PeglegManagedDocument():
72 75
     assert doc.is_encrypted() is False
73 76
 
74 77
 
75
-@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
76
-                              ENV_SALT: 'MySecretSalt'})
77
-def test_encrypt_document():
78
+def test_PeglegSecretManagement():
79
+    with pytest.raises(ValueError) as err_info:
80
+        PeglegSecretManagement(file_path=None, docs=None)
81
+    assert 'Either `file_path` or `docs` must be specified.' in str(
82
+        err_info.value)
83
+    with pytest.raises(ValueError) as err_info:
84
+        PeglegSecretManagement(file_path='file_path', docs=['doc1'])
85
+    assert 'Either `file_path` or `docs` must be specified.' in str(
86
+        err_info.value)
87
+
88
+
89
+@mock.patch.dict(os.environ, {
90
+    ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
91
+    ENV_SALT: 'MySecretSalt'})
92
+def test_encrypt_file():
78 93
     # write the test data to temp file
79 94
     test_data = yaml.load(TEST_DATA)
80 95
     dir = tempfile.mkdtemp()
@@ -92,3 +107,44 @@ def test_encrypt_document():
92 107
     doc = doc_mgr.documents[0]
93 108
     assert doc.is_encrypted()
94 109
     assert doc.data['encrypted']['by'] == 'test_author'
110
+
111
+
112
+@mock.patch.dict(os.environ, {
113
+    ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
114
+    ENV_SALT: 'MySecretSalt'})
115
+def test_encrypt_decrypt_file(temp_path):
116
+    # write the test data to temp file
117
+    test_data = list(yaml.safe_load_all(TEST_DATA))
118
+    file_path = os.path.join(temp_path, 'secrets_file.yaml')
119
+    files.write(file_path, test_data)
120
+    save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
121
+    doc_mgr = PeglegSecretManagement(file_path=file_path)
122
+    doc_mgr.encrypt_secrets(save_path, 'test_author')
123
+    # read back the encrypted file
124
+    doc_mgr = PeglegSecretManagement(save_path)
125
+    decrypted_data = doc_mgr.get_decrypted_secrets()
126
+    assert test_data[0]['data'] == decrypted_data[0]['data']
127
+    assert test_data[0]['schema'] == decrypted_data[0]['schema']
128
+
129
+
130
+@mock.patch.dict(os.environ, {
131
+    ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
132
+    ENV_SALT: 'MySecretSalt'})
133
+def test_decrypt_document(temp_path):
134
+    # write the test data to temp file
135
+    test_data = list(yaml.safe_load_all(TEST_DATA))
136
+    save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
137
+    doc_mgr = PeglegSecretManagement(docs=test_data)
138
+    doc_mgr.encrypt_secrets(save_path, 'test_author')
139
+    # read back the encrypted file
140
+    with open(save_path) as stream:
141
+        encrypted_data = list(yaml.safe_load_all(stream))
142
+    # this time pass a list of dicts to peglegSecretManager
143
+    doc_mgr = PeglegSecretManagement(docs=encrypted_data)
144
+    decrypted_data = doc_mgr.get_decrypted_secrets()
145
+    assert test_data[0]['data'] == decrypted_data[0]['data']
146
+    assert test_data[0]['schema'] == decrypted_data[0]['schema']
147
+    assert test_data[0]['metadata']['name'] == decrypted_data[0][
148
+        'metadata']['name']
149
+    assert test_data[0]['metadata']['storagePolicy'] == decrypted_data[0][
150
+        'metadata']['storagePolicy']

Loading…
Cancel
Save