Update decrypt secrets to return a list of docs
1. Added the method to decrypt a secret file and return its contents as a list of documents (instead of printing out the file content). 2. Added clarifications for a encrypt and decrypt commands. Change-Id: I77bce21be214c880c8413f5e6a2d0c2d1993fc8e
This commit is contained in:
parent
4a352510d2
commit
fb8e6f73ac
@ -397,6 +397,20 @@ Secrets
|
|||||||
A sub-group of site command group, which allows you to perform secrets
|
A sub-group of site command group, which allows you to perform secrets
|
||||||
level operations for secrets documents of a site.
|
level operations for secrets documents of a site.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
For the CLI commands ``encrypt`` and ``decrypt`` in the ``secrets`` command
|
||||||
|
group, which encrypt or decrypt site secrets, two environment variables,
|
||||||
|
``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``, are used to capture the
|
||||||
|
master passphrase, and the salt needed for encryption and decryption of the
|
||||||
|
site secrets. The contents of ``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``
|
||||||
|
are not generated by Pegleg, but are created externally, and set by a
|
||||||
|
deployment engineers or tooling.
|
||||||
|
|
||||||
|
A minimum length of 24 for master passphrases will be checked by all CLI
|
||||||
|
commands, which use the ``PEGLEG_PASSPHRASE``. All other criteria around
|
||||||
|
master passphrase strength are assumed to be enforced elsewhere.
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>
|
./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>
|
||||||
@ -406,26 +420,52 @@ Encrypt
|
|||||||
^^^^^^^
|
^^^^^^^
|
||||||
|
|
||||||
Encrypt one site's secrets documents, which have the
|
Encrypt one site's secrets documents, which have the
|
||||||
metadata.storagePolicy set to encrypted, and wrap them in `pegleg managed
|
``metadata.storagePolicy`` set to encrypted, and wrap them in
|
||||||
documents <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_.
|
`Pegleg Managed Documents`_
|
||||||
|
|
||||||
**Note**: The encrypt command is idempotent. If the command is executed more
|
.. note::
|
||||||
than once for a given site, it will skip the files, which are already
|
|
||||||
encrypted and wrapped in a pegleg managed document, and will only encrypt the
|
The encrypt command is idempotent. If the command is executed more
|
||||||
documents not encrypted before.
|
than once for a given site, it will skip the files, which are already
|
||||||
|
encrypted and wrapped in a pegleg managed document, and will only encrypt the
|
||||||
|
documents not encrypted before.
|
||||||
|
|
||||||
**site_name** (Required).
|
**site_name** (Required).
|
||||||
|
|
||||||
Name of the site.
|
Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
|
||||||
|
repository folder structure. The ``encrypt`` command looks up the
|
||||||
|
``site-name`` in the site repository, and searches recursively the
|
||||||
|
``site_name`` folder structure for secrets files (i.e. files with documents,
|
||||||
|
whose ``encryptionPolicy`` is set to ``encrypted``), and encrypts the
|
||||||
|
documents in those files.
|
||||||
|
|
||||||
**-a / --author** (Required)
|
**-a / --author** (Required)
|
||||||
|
|
||||||
Identifier for the program or person who is encrypting the secrets documents.
|
Author is the identifier for the program or the person, who is encrypting
|
||||||
|
the secrets documents.
|
||||||
|
Author is intended to document the entity or the individual, who
|
||||||
|
encrypts the site secrets documents, mostly for tracking purposes, and is
|
||||||
|
expected to be leveraged in an operator-specific manner.
|
||||||
|
For instance the ``author`` can be the "userid" of the person running the
|
||||||
|
command, or the "application-id" of the application executing the command.
|
||||||
|
|
||||||
**-s / --save-location** (Optional).
|
**-s / --save-location** (Optional).
|
||||||
|
|
||||||
Where to output encrypted and wrapped documents. If omitted, the results
|
Where to output the encrypted and wrapped documents.
|
||||||
will overwrite the original documents.
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
If the ``save-location`` parameter is not provided, the encrypted result
|
||||||
|
documents will overwrite the original ``cleartext`` documents for the site.
|
||||||
|
The reason for this default behavior, is to ensure that site secrets are
|
||||||
|
only stored on disk or in any version control system as encrypted.
|
||||||
|
|
||||||
|
If the user for any reason wants to avoid overwriting the original
|
||||||
|
cleartext files, the ``save-location`` parameter will provide the option to
|
||||||
|
override this default behavior, and forces the encrypt command to write
|
||||||
|
the encrypted documents in a different location than the original
|
||||||
|
unencrypted files.
|
||||||
|
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
|
|
||||||
@ -457,14 +497,16 @@ Example without optional save location:
|
|||||||
Decrypt
|
Decrypt
|
||||||
^^^^^^^
|
^^^^^^^
|
||||||
|
|
||||||
Unwrap an encrypted secrets document from a `pegleg managed
|
Unwrap an encrypted secrets document from a `Pegleg Managed Documents`_,
|
||||||
document <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_,
|
|
||||||
decrypt the encrypted secrets, and dump the cleartext secrets file to
|
decrypt the encrypted secrets, and dump the cleartext secrets file to
|
||||||
``stdout``.
|
``stdout``.
|
||||||
|
|
||||||
**site_name** (Required).
|
**site_name** (Required).
|
||||||
|
|
||||||
Name of the site.
|
Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
|
||||||
|
repository folder structure. The ``decrypt`` command also validates that the
|
||||||
|
``site-name`` exists in the file path, before unwrapping and decrypting the
|
||||||
|
documents in the ``filename``.
|
||||||
|
|
||||||
**-f / filename** (Required).
|
**-f / filename** (Required).
|
||||||
|
|
||||||
@ -598,3 +640,4 @@ P003 - All repos contain expected directories.
|
|||||||
|
|
||||||
.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html
|
.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html
|
||||||
.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html
|
.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html
|
||||||
|
.. _Pegleg Managed Documents: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
|
@ -74,7 +74,7 @@ def decrypt(file_path, site_name):
|
|||||||
:param file_path: Path to the file to be unwrapped and decrypted.
|
:param file_path: Path to the file to be unwrapped and decrypted.
|
||||||
:type file_path: string
|
:type file_path: string
|
||||||
:param site_name: The name of the site to search for the file.
|
:param site_name: The name of the site to search for the file.
|
||||||
:type site_name: string providing the site name
|
:type site_name: string
|
||||||
"""
|
"""
|
||||||
|
|
||||||
LOG.info('Started decrypting...')
|
LOG.info('Started decrypting...')
|
||||||
|
@ -34,17 +34,29 @@ ENV_SALT = 'PEGLEG_SALT'
|
|||||||
class PeglegSecretManagement():
|
class PeglegSecretManagement():
|
||||||
"""An object to handle operations on of a pegleg managed file."""
|
"""An object to handle operations on of a pegleg managed file."""
|
||||||
|
|
||||||
def __init__(self, file_path):
|
def __init__(self, file_path=None, docs=None):
|
||||||
"""
|
"""
|
||||||
Read the source file and the environment data needed to wrap and
|
Read the source file and the environment data needed to wrap and
|
||||||
process the file documents as pegleg managed document.
|
process the file documents as pegleg managed document.
|
||||||
|
Either of the ``file_path`` or ``docs`` must be
|
||||||
|
provided.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
if all([file_path, docs]) or \
|
||||||
|
not any([file_path, docs]):
|
||||||
|
raise ValueError(
|
||||||
|
'Either `file_path` or `docs` must be specified.')
|
||||||
|
|
||||||
self.__check_environment()
|
self.__check_environment()
|
||||||
self.file_path = file_path
|
self.file_path = file_path
|
||||||
self.documents = list()
|
self.documents = list()
|
||||||
for doc in files.read(file_path):
|
if docs:
|
||||||
self.documents.append(PeglegManagedSecret(doc))
|
for doc in docs:
|
||||||
|
self.documents.append(PeglegManagedSecret(doc))
|
||||||
|
else:
|
||||||
|
self.file_path = file_path
|
||||||
|
for doc in files.read(file_path):
|
||||||
|
self.documents.append(PeglegManagedSecret(doc))
|
||||||
|
|
||||||
self.passphrase = os.environ.get(ENV_PASSPHRASE).encode()
|
self.passphrase = os.environ.get(ENV_PASSPHRASE).encode()
|
||||||
self.salt = os.environ.get(ENV_SALT).encode()
|
self.salt = os.environ.get(ENV_SALT).encode()
|
||||||
@ -119,9 +131,27 @@ class PeglegSecretManagement():
|
|||||||
included in a site secrets file, and print the result to the standard
|
included in a site secrets file, and print the result to the standard
|
||||||
out."""
|
out."""
|
||||||
|
|
||||||
|
yaml.safe_dump_all(
|
||||||
|
self.get_decrypted_secrets(),
|
||||||
|
sys.stdout,
|
||||||
|
explicit_start=True,
|
||||||
|
explicit_end=True,
|
||||||
|
default_flow_style=False)
|
||||||
|
|
||||||
|
def get_decrypted_secrets(self):
|
||||||
|
"""
|
||||||
|
Unwrap and decrypt all the pegleg managed documents in a secrets
|
||||||
|
file, and return the result as a list of documents.
|
||||||
|
|
||||||
|
The method is idempotent. If the method is called on not
|
||||||
|
encrypted files, or documents inside the file, it will return
|
||||||
|
the original unwrapped and unencrypted documents.
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
doc_list = []
|
doc_list = []
|
||||||
for doc in self.documents:
|
for doc in self.documents:
|
||||||
# only decrypt an encrypted document
|
# do not decrypt already decrypted data
|
||||||
if doc.is_encrypted():
|
if doc.is_encrypted():
|
||||||
doc.set_secret(
|
doc.set_secret(
|
||||||
decrypt(doc.get_secret(),
|
decrypt(doc.get_secret(),
|
||||||
@ -129,9 +159,4 @@ class PeglegSecretManagement():
|
|||||||
self.salt).decode())
|
self.salt).decode())
|
||||||
doc.set_decrypted()
|
doc.set_decrypted()
|
||||||
doc_list.append(doc.embedded_document)
|
doc_list.append(doc.embedded_document)
|
||||||
yaml.safe_dump_all(
|
return doc_list
|
||||||
doc_list,
|
|
||||||
sys.stdout,
|
|
||||||
explicit_start=True,
|
|
||||||
explicit_end=True,
|
|
||||||
default_flow_style=False)
|
|
||||||
|
@ -22,12 +22,14 @@ import yaml
|
|||||||
|
|
||||||
from pegleg.engine.util import encryption as crypt
|
from pegleg.engine.util import encryption as crypt
|
||||||
from tests.unit import test_utils
|
from tests.unit import test_utils
|
||||||
from pegleg.engine import secrets
|
|
||||||
from pegleg.engine.util.pegleg_managed_document import \
|
from pegleg.engine.util.pegleg_managed_document import \
|
||||||
PeglegManagedSecretsDocument
|
PeglegManagedSecretsDocument
|
||||||
from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
|
from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
|
||||||
from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
|
from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
|
||||||
from pegleg.engine.util.pegleg_secret_management import ENV_SALT
|
from pegleg.engine.util.pegleg_secret_management import ENV_SALT
|
||||||
|
from tests.unit.fixtures import temp_path
|
||||||
|
from pegleg.engine.util import files
|
||||||
|
|
||||||
|
|
||||||
TEST_DATA = """
|
TEST_DATA = """
|
||||||
---
|
---
|
||||||
@ -57,8 +59,9 @@ def test_encrypt_and_decrypt():
|
|||||||
assert data == dec2
|
assert data == dec2
|
||||||
|
|
||||||
|
|
||||||
@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'aShortPassphrase',
|
@mock.patch.dict(os.environ, {
|
||||||
ENV_SALT: 'MySecretSalt'})
|
ENV_PASSPHRASE:'aShortPassphrase',
|
||||||
|
ENV_SALT: 'MySecretSalt'})
|
||||||
def test_short_passphrase():
|
def test_short_passphrase():
|
||||||
with pytest.raises(click.ClickException,
|
with pytest.raises(click.ClickException,
|
||||||
match=r'.*is not at least 24-character long.*'):
|
match=r'.*is not at least 24-character long.*'):
|
||||||
@ -72,9 +75,21 @@ def test_PeglegManagedDocument():
|
|||||||
assert doc.is_encrypted() is False
|
assert doc.is_encrypted() is False
|
||||||
|
|
||||||
|
|
||||||
@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
|
def test_PeglegSecretManagement():
|
||||||
ENV_SALT: 'MySecretSalt'})
|
with pytest.raises(ValueError) as err_info:
|
||||||
def test_encrypt_document():
|
PeglegSecretManagement(file_path=None, docs=None)
|
||||||
|
assert 'Either `file_path` or `docs` must be specified.' in str(
|
||||||
|
err_info.value)
|
||||||
|
with pytest.raises(ValueError) as err_info:
|
||||||
|
PeglegSecretManagement(file_path='file_path', docs=['doc1'])
|
||||||
|
assert 'Either `file_path` or `docs` must be specified.' in str(
|
||||||
|
err_info.value)
|
||||||
|
|
||||||
|
|
||||||
|
@mock.patch.dict(os.environ, {
|
||||||
|
ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
|
||||||
|
ENV_SALT: 'MySecretSalt'})
|
||||||
|
def test_encrypt_file():
|
||||||
# write the test data to temp file
|
# write the test data to temp file
|
||||||
test_data = yaml.load(TEST_DATA)
|
test_data = yaml.load(TEST_DATA)
|
||||||
dir = tempfile.mkdtemp()
|
dir = tempfile.mkdtemp()
|
||||||
@ -92,3 +107,44 @@ def test_encrypt_document():
|
|||||||
doc = doc_mgr.documents[0]
|
doc = doc_mgr.documents[0]
|
||||||
assert doc.is_encrypted()
|
assert doc.is_encrypted()
|
||||||
assert doc.data['encrypted']['by'] == 'test_author'
|
assert doc.data['encrypted']['by'] == 'test_author'
|
||||||
|
|
||||||
|
|
||||||
|
@mock.patch.dict(os.environ, {
|
||||||
|
ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
|
||||||
|
ENV_SALT: 'MySecretSalt'})
|
||||||
|
def test_encrypt_decrypt_file(temp_path):
|
||||||
|
# write the test data to temp file
|
||||||
|
test_data = list(yaml.safe_load_all(TEST_DATA))
|
||||||
|
file_path = os.path.join(temp_path, 'secrets_file.yaml')
|
||||||
|
files.write(file_path, test_data)
|
||||||
|
save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
|
||||||
|
doc_mgr = PeglegSecretManagement(file_path=file_path)
|
||||||
|
doc_mgr.encrypt_secrets(save_path, 'test_author')
|
||||||
|
# read back the encrypted file
|
||||||
|
doc_mgr = PeglegSecretManagement(save_path)
|
||||||
|
decrypted_data = doc_mgr.get_decrypted_secrets()
|
||||||
|
assert test_data[0]['data'] == decrypted_data[0]['data']
|
||||||
|
assert test_data[0]['schema'] == decrypted_data[0]['schema']
|
||||||
|
|
||||||
|
|
||||||
|
@mock.patch.dict(os.environ, {
|
||||||
|
ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
|
||||||
|
ENV_SALT: 'MySecretSalt'})
|
||||||
|
def test_decrypt_document(temp_path):
|
||||||
|
# write the test data to temp file
|
||||||
|
test_data = list(yaml.safe_load_all(TEST_DATA))
|
||||||
|
save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
|
||||||
|
doc_mgr = PeglegSecretManagement(docs=test_data)
|
||||||
|
doc_mgr.encrypt_secrets(save_path, 'test_author')
|
||||||
|
# read back the encrypted file
|
||||||
|
with open(save_path) as stream:
|
||||||
|
encrypted_data = list(yaml.safe_load_all(stream))
|
||||||
|
# this time pass a list of dicts to peglegSecretManager
|
||||||
|
doc_mgr = PeglegSecretManagement(docs=encrypted_data)
|
||||||
|
decrypted_data = doc_mgr.get_decrypted_secrets()
|
||||||
|
assert test_data[0]['data'] == decrypted_data[0]['data']
|
||||||
|
assert test_data[0]['schema'] == decrypted_data[0]['schema']
|
||||||
|
assert test_data[0]['metadata']['name'] == decrypted_data[0][
|
||||||
|
'metadata']['name']
|
||||||
|
assert test_data[0]['metadata']['storagePolicy'] == decrypted_data[0][
|
||||||
|
'metadata']['storagePolicy']
|
||||||
|
Loading…
Reference in New Issue
Block a user