26 KiB

Raw Blame History

Pegleg CLI

The Pegleg CLI is used in conjunction with the script located in pegleg/tools called pegleg.sh.

Note

The default workspace for the pegleg.sh script is /workspace. The examples below require that this workspace be used.

Environment Variables

$WORKSPACE = Location of the folder that holds the repositories containing
the site definition libraries. Pegleg makes no assumptions about the root
directory. $WORKSPACE is /workspace in the container context.

    Example: $WORKSPACE=/home/ubuntu/all_repos

$IMAGE = Location of pegleg docker image.

    Example: $IMAGE=quay.io/airshipit/pegleg:latest

Usage

To run:

export WORKSPACE=<repo_location>
export IMAGE=<docker_image>
./pegleg.sh <command> <options>

For example:

cd /opt/airship-pegleg
export WORKSPACE=/opt/airship-treasuremap
./tools/pegleg.sh site -r /workspace --help

Note

If sudo permissions are required to execute pegleg.sh, then it is necessary to use the -E flag with sudo in order for the current environment to be used. For example:

cd /opt/airship-pegleg
export WORKSPACE=/opt/airship-treasuremap
sudo -E ./tools/pegleg.sh site -r /workspace --help

CLI Options

-v / --verbose (Optional). False by default.

Enable debug logging.

Repo Group

Allows you to perform repository-level operations.

Options

-r / --site-repository (Required).

Path to the root of the site repository (containing site_definition.yaml) repo.

For example: /opt/airship-treasuremap

The revision can also be specified via (for example):

-r /opt/airship-treasuremap@revision

-p / --clone-path (Optional).

The path where the repo will be cloned. By default the repo will be cloned to the /tmp path. If this option is included and the repo already exists, then the repo will not be cloned again and the user must specify a new clone path or pass in the local copy of the repository as the site repository. Suppose the repo name is airship-treasuremap and the clone path is /tmp/mypath then the following directory is created /tmp/mypath/airship-treasuremap which will contain the contents of the repo. Example of using clone path:

-p /tmp/mypath

Lint

Sanity checks for repository content (all sites in the repository). To lint a specific site, see site-level linting <cli-site-lint>.

See linting for more information.

Site Group

Allows you to perform site-level operations.

./pegleg.sh site -r <site_repo> -e <extra_repo> <command> <options>

Options

-r / --site-repository (Required).

Path to the root of the site repository (containing site_definition.yaml) repo.

For example: /opt/airship-treasuremap

The revision can also be specified via (for example):

-r /opt/airship-treasuremap@revision

-e / --extra-repository (Optional).

Path to the root of extra repositories used for overriding those specified under the repositories field in a given site-definition.yaml.

These should be named per the site-definition file, e.g.:

-e global=/opt/global -e secrets=/opt/secrets

-p / --clone-path (Optional).

-p /tmp/mypath

Repository Overrides

By default, the revision specified in the site-definition.yaml for the site will be leveraged but can be overridden <command-line-repository-overrides> using:

-e global=/opt/global@revision

-k / --repo-key (Optional, SSH only).

The SSH public key to use when cloning remote authenticated repositories.

Required for cloning repositories via SSH protocol.

-u / --repo-username (Optional, SSH only).

The SSH username to use when cloning remote authenticated repositories specified in the site-definition file. Any occurrences of REPO_USERNAME in an entry under the repositories field in a given site-definition.yaml will be replaced with this value.

Required for cloning repositories via SSH protocol.

Examples

Example usage:

./pegleg.sh site -r /opt/site-manifests/ \
  -u <AUTH_USER> \
  -k /opt/.ssh/git.pub \
  -e global=ssh://REPO_USERNAME@<GIT URL>:29418/global-manifests.git@master \
  <command> <options>

Collect

Output complete config for one site.

site_name (Required).

Name of the site.

-s / --save-location (Optional).

Where to output collected documents. If omitted, the results will be dumped to stdout.

-x <code> (Optional, validation only).

Will exclude the specified lint option. -w takes priority over -x.

-w <code> (Optional, validation only).

Will warn of lint failures from the specified lint options.

--validate (Optional, validation only). False by default.

Perform validation of documents prior to collection. See cli-site-lint for additional information on document linting. It is recommended that document linting be executed prior to document collection. However, --validate is False by default for backwards compatibility concerns.

Usage:

./pegleg.sh <command> <options> collect <site_name> -s <save_location> \
  -x P001 -w P002 --validate

Examples

Example without validation:

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  collect <site_name> -s /workspace

Example with validation:

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  collect <site_name> -s /workspace -x P004 --validate

List

List known sites.

-o/--output (Optional).

Where to output.

./pegleg <command> <options> list

Results are dumped to stdout by default.

Examples

Example:

./pegleg site -r /opt/site-manifests list -o /workspace

Show

Show details for one site.

site_name (Required).

Name of site.

-o/--output (Optional).

Where to output.

./pegleg <command> <options> show site_name

Results are dumped to stdout by default.

Examples

Example:

./pegleg site -r /opt/site-manifests show site_name -o /workspace

Render

Render documents via Deckhand for one site.

site_name (Required).

Name of site.

-o/--output (Optional).

Where to output.

./pegleg <command> <options> render site_name

Results are dumped to stdout by default.

Examples

Example:

./pegleg site -r /opt/site-manifests render site_name -o output

Lint

Sanity checks for repository content (for a specific site in the repository). Validations for linting are done utilizing Deckhand Validations.

To lint all sites in the repository, see repository-level linting <cli-repo-lint>.

See linting for more information.

Examples

Generic example:

./pegleg.sh site -r <site_repo> -e <extra_repo> \
  lint <site_name> \
  -f -x <lint_code> -w <lint_code>

The most basic way to lint a document set is as follows:

./pegleg.sh site -r <site_repo> -e <extra_repo> lint <site_name>

A more complex example involves excluding certain linting checks:

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  lint <site_name> \
  -x P001 -x P002 -w P003

Upload

Uploads documents to Shipyard.

site_name (Required).

Name of the site. The site_name must match a site name in the site repository folder structure

--os-<various>=<value> (Required).

Shipyard needs these options for authenticating with OpenStack Keystone. This option can be set as environment variables or it can be passed via the command line.

Please reference Shipyard's CLI documentation for information related to these options.

--context-marker=<uuid> (Optional).

Specifies a UUID (8-4-4-4-12 format) that will be used to correlate logs, transactions, etc. in downstream activities triggered by this interaction.

Usage:

./pegleg.sh site <options> upload <site_name> --context-marker=<uuid>

Site Secrets Group

Subgroup of site-group.

A sub-group of site command group, which allows you to perform secrets level operations for secrets documents of a site.

Note

For the CLI commands encrypt, decrypt, generate-pki, and wrap in the secrets command group, which encrypt or decrypt site secrets, two environment variables, PEGLEG_PASSPHRASE, and PEGLEG_SALT, are used to capture the master passphrase, and the salt needed for encryption and decryption of the site secrets. The contents of PEGLEG_PASSPHRASE, and PEGLEG_SALT are not generated by Pegleg, but are created externally, and set by deployment engineers or tooling.

A minimum length of 24 for master passphrases will be checked by all CLI commands, which use the PEGLEG_PASSPHRASE and PEGLEG_SALT. All other criteria around master passphrase strength are assumed to be enforced elsewhere.

./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>

Generate PKI

Generate certificates and keys according to all PKICatalog documents in the site using the PKI module. Regenerating certificates can be accomplished by re-running this command.

Pegleg places generated document files in <site>/secrets/passphrases, <site>/secrets/certificates, or <site>/secrets/keypairs as appropriate:

The generated filenames for passphrases will follow the pattern <passphrase-doc-name>.yaml.
The generated filenames for certificate authorities will follow the pattern <ca-name>_ca.yaml.
The generated filenames for certificates will follow the pattern <ca-name>_<certificate-doc-name>_certificate.yaml.
The generated filenames for certificate keys will follow the pattern <ca-name>_<certificate-doc-name>_key.yaml.
The generated filenames for keypairs will follow the pattern <keypair-doc-name>.yaml.

Dashes in the document names will be converted to underscores for consistency.

site_name (Required).

Name of site.

days (Optional).

Duration (in days) certificates should be valid. Default=365, minimum=0, no maximum.

NOTE: A generated certificate where days = 0 should only be used for testing. A certificate generated in such a way will be valid for 0 seconds.

Examples

./pegleg.sh site -r <site_repo> -e <extra_repo> \
  upload <site_name> <options>

./pegleg.sh site -r <site_repo> -e <extra_repo> \
  secrets generate-pki \
  <site_name> \
  -o <output> \
  -f <filename> \
  -d <days>

Check PKI Certs

Determine if any PKI certificates from a site are expired, or will be expired within N days (default N=60, no maximum, minimum 0). Print those cert names and expiration dates to stdout.

-d / --days (Optional).

Number of days past today's date to check certificate expirations. Default days=60. Minimum days=0, days less than 0 will raise an exception. No maximum days.

site_name (Required).

Name of the site. The site_name must match a site name in the site repository folder structure.

Usage:

./pegleg.sh site -r <site_repo> \
  secrets check-pki-certs <site_name> <options>

Examples

Example without days specified:

./pegleg.sh site -r <site_repo> secrets check-pki-certs <site_name>

Example with days specified:

./pegleg.sh site -r <site_repo> secrets check-pki-certs <site_name> -d <days>

Secrets

A sub-group of site command group, which allows you to perform secrets level operations for secrets documents of a site.

Note

For the CLI commands encrypt and decrypt in the secrets command group, which encrypt or decrypt site secrets, two environment variables, PEGLEG_PASSPHRASE, and PEGLEG_SALT, are used to capture the master passphrase, and the salt needed for encryption and decryption of the site secrets. The contents of PEGLEG_PASSPHRASE, and PEGLEG_SALT are not generated by Pegleg, but are created externally, and set by a deployment engineers or tooling.

A minimum length of 24 for master passphrases will be checked by all CLI commands, which use the PEGLEG_PASSPHRASE. All other criteria around master passphrase strength are assumed to be enforced elsewhere.

./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>

Encrypt

Encrypt one site's secrets documents, which have the metadata.storagePolicy set to encrypted, and wrap them in Pegleg Managed Documents

Note

The encrypt command is idempotent. If the command is executed more than once for a given site, it will skip the files, which are already encrypted and wrapped in a pegleg managed document, and will only encrypt the documents not encrypted before.

site_name (Required).

Name of the site. The site_name must match a site name in the site repository folder structure. The encrypt command looks up the site-name in the site repository, and searches recursively the site_name folder structure for secrets files (i.e. files with documents, whose encryptionPolicy is set to encrypted), and encrypts the documents in those files.

-a / --author (Required)

Author is the identifier for the program or the person, who is encrypting the secrets documents. Author is intended to document the entity or the individual, who encrypts the site secrets documents, mostly for tracking purposes, and is expected to be leveraged in an operator-specific manner. For instance the author can be the "userid" of the person running the command, or the "application-id" of the application executing the command.

-s / --save-location (Optional).

Where to output the encrypted and wrapped documents.

Warning

If the save-location parameter is not provided, the encrypted result documents will overwrite the original cleartext documents for the site. The reason for this default behavior, is to ensure that site secrets are only stored on disk or in any version control system as encrypted.

If the user for any reason wants to avoid overwriting the original cleartext files, the save-location parameter will provide the option to override this default behavior, and forces the encrypt command to write the encrypted documents in a different location than the original unencrypted files.

Usage:

./pegleg.sh site <options> secrets encrypt <site_name> -a <author_id> -s <save_location>

Examples

Example with optional save location:

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  -e secrets=/opt/security-manifests \
  secrets encrypt <site_name> -a <author_id> -s /workspace

Example without optional save location:

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  -e secrets=/opt/security-manifests \
  secrets encrypt <site_name> -a <author_id>

Decrypt

Unwrap an encrypted secrets document from a Pegleg Managed Documents, decrypt the encrypted secrets, and dump the cleartext secrets file to stdout.

site_name (Required).

Name of the site. The site_name must match a site name in the site repository folder structure. The decrypt command also validates that the site-name exists in the file path, before unwrapping and decrypting the documents in the filename.

-f / filename (Required).

The absolute path to the pegleg managed encrypted secrets file.

Usage:

./pegleg.sh site <options> secrets decrypt <site_name> -f <file_path>

Examples

Example:

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  -e secrets=/opt/security-manifests \
  secrets decrypt site1 -f \
  /opt/security-manifests/site/site1/passwords/password1.yaml

Wrap

Wrap bare files (e.g. pem or crt) in a PeglegManagedDocument and optionally encrypt them.

site_name (Required).

Name of site.

-a / --author

Identifying name of the author generating new certificates. Used for tracking provenance information in the PeglegManagedDocuments. An attempt is made to automatically determine this value, but should be provided.

-f / --filename

The relative path to the file to be wrapped.

-o / --output-path

The output path for the wrapped file. (default: input path with the extension replaced with .yaml)

-s / --schema

The schema for the document to be wrapped, e.g. deckhand/Certificate/v1

-n / --name

The name for the document to be wrapped, e.g. new-cert.

-l / --layer

The layer for the document to be wrapped, e.g. site.

--encrypt / --no-encrypt

A flag specifying whether to encrypt the output file. (default: True)

Examples

./pegleg.sh site -r /home/myuser/myrepo \
  secrets wrap -a myuser -f secrets/certificates/new_cert.crt \
  -o secrets/certificates/new_cert.yaml -s "deckhand/Certificate/v1" \
  -n "new-cert" -l site mysite

genesis_bundle

Constructs genesis bundle based on a site configuration.

Note

This command requires the environment variable PEGLEG_PASSPHRASE to be set and at least 24 characters long, to be used for encrypting genesis bundle data. PEGLEG_SALT must be set as well. There are no constraints on its length, but at least 24 characters is recommended.

-b / --build-dir (Required).

Destination directory for the genesis bundle.

--include-validators (Optional). False by default.

A flag to request build genesis validation scripts as well.

Usage:

::: ./pegleg.sh site <options> genesis_bundle <site_name> -b <build_locaton> -k <encryption_passphrase/key> --validators

Examples

./pegleg.sh site -r  ./site-manifests \
  genesis_bundle site1 \
  -b ../../site1_build \
  -k yourEncryptionPassphrase \
  --validators

generate

A sub-group of secrets command group, which allows you to auto-generate secrets documents of a site.

Note

The types of documents that pegleg cli generates are passphrases, certificate authorities, certificates and keys. Passphrases are declared in a new pegleg/PassphraseCatalog/v1 document, while CAs, certificates, and keys are declared in the pegleg/PKICatalog/v1.

The pegleg/PKICatalog/v1 schema is identical with the existing promenade/PKICatalog/v1, promenade currently uses to generate the site CAs, certificates, and keys.

The pegleg/PassphraseCatalog/v1 schema is specified in Pegleg Passphrase Catalog

./pegleg.sh site -r <site_repo> -e <extra_repo> secrets generate <command> <options>

passphrases

Generates, wraps and encrypts passphrase documents specified in the pegleg/PassphraseCatalog/v1 document for a site. The site name, and the directory to store the generated documents are provided by the site_name, and the save_location command line parameters respectively. The generated passphrases are stored in:

<save_location>/site/<site_name>/passphrases/<passphrase_name.yaml>

The schema for the generated passphrases is defined in Pegleg Managed Documents

site_name (Required).

Name of the site. The site_name must match a site name in the site repository folder structure. The generate command looks up the site-name, and searches recursively the site_name folder structure in the site repository for pegleg/PassphraseCatalog/v1 documents. Then it parses the passphrase catalog documents it found, and generates one passphrase document for each passphrase document_name declared in the site passphrase catalog.

-a / --author (Required)

Author is intended to document the application or the individual, who generates the site passphrase documents, mostly for tracking purposes. It is expected to be leveraged in an operator-specific manner. For instance the author can be the "userid" of the person running the command, or the "application-id" of the application executing the command.

-s / --save-location (Required).

Where to output generated passphrase documents. The passphrase documents are placed in the following folder structure under save_location:

<save_location>/site/<site_name>/secrets/passphrases/<passphrase_name.yaml>

Usage:

./pegleg.sh site <options> secrets generate passphrases <site_name> -a
<author_id> -s <save_location>

Example

./pegleg.sh site -r /opt/site-manifests \
  -e global=/opt/manifests \
  -e secrets=/opt/security-manifests \
  secrets generate passphrases <site_name> -a <author_id> -s /workspace

CLI Repository Overrides

Repository overrides should only be used for entries included underneath the repositories field for a given site-definition.yaml.

Overrides are specified via the -e flag for all site-group commands. They have the following format:

-e <REPO_NAME>=<REPO_PATH_OR_URL>@<REVISION>

Where:

REPO_NAME is one of: global, secrets or site.

REPO_PATH_OR_URL is one of:

path (relative or absolute) - /opt/global or ../global though absolute is recommended

url (fully qualified) - must have following formats:

ssh - <PROTOCOL>://<REPO_USERNAME>@<GIT URL>:<PORT>/<REPO_NAME>.git

http|https - <PROTOCOL>://<GIT URL>/<REPO_NAME>.git

Where:

<PROTOCOL> must be a valid authentication protocol: ssh, https, or http

<REPO_USERNAME> must be a user with access rights to the repository. This value will replace the literal string REPO_USERNAME in the corresponding entry under the repositories field in the relevant site-definition.yaml using -u CLI flag

<GIT_URL> must be a valid Git URL

<PORT> must be a valid authentication port for SSH

<REVISION> must be a valid git-reference

<REPO_NAME> must be a valid Git repository name, e.g. site-manifests

Self-Contained Repository

For self-contained repositories, specification of extra repositories is unnecessary. The following command can be used to deploy the manifests in the example repository /opt/airship-in-a-bottle for the currently checked out revision:

pegleg site -r /opt/airship-in-a-bottle/deployment_files <command> <options>

To specify a specific revision at run time, execute:

pegleg site -r /opt/airship-in-a-bottle/deployment_files@<REVISION> \
  <command> <options>

Where <REVISION> must be a valid git-reference.

Git Reference

Valid Git references for checking out repositories include:

47676764d3935e4934624bf9593e9115984fe668 (commit ID)

refs/changes/79/47079/12 (ref)

master (branch name)

Linting

-f / --fail-on-missing-sub-src (Optional).

Raise Deckhand exception on missing substitution sources. Defaults to True.

-x <code> (Optional).

Will exclude the specified lint option. -w takes priority over -x.

-w <code> (Optional).

Will warn of lint failures from the specified lint options.

If you expect certain lint failures, then those lint options can be excluded or you can choose to be warned about those failures using the codes below.

P001 - Document has storagePolicy cleartext (expected is encrypted) yet its schema is a mandatory encrypted type.

Where mandatory encrypted schema type is one of:

deckhand/CertificateAuthorityKey/v1
deckhand/CertificateKey/v1
deckhand/Passphrase/v1
deckhand/PrivateKey/v1

P002 - Deckhand rendering is expected to complete without errors. P003 - All repos contain expected directories.

Generate

Allows you to perform generate operations.

Passphrase

Generate a passphrase and print to stdout.

-l / --length (Optional).

Length of passphrase to generate. By default length is 24. Minimum length is 24. Lengths less than minimum will default to 24. No maximum length.

Usage:

./pegleg.sh generate passphrase -l <length>

Examples

Example without length specified:

./pegleg.sh generate passphrase

Example with length specified:

./pegleg.sh generate passphrase -l <length>

Salt

Generate a salt and print to stdout.

-l / --length (Optional).

Length of salt to generate. By default length is 24. Minimum length is 24. Lengths less than minimum will default to 24. No maximum length.

Usage:

./pegleg.sh generate salt -l <length>

Examples

Example without length specified:

./pegleg.sh generate salt

Example with length specified:

./pegleg.sh generate salt -l <length>

26 KiB Raw Blame History

Pegleg CLI

Environment Variables

Usage

CLI Options

Repo Group

Options

Lint

Site Group

Options

Repository Overrides

Examples

Collect

Examples

List

Examples

Show

Examples

Render

Examples

Lint

Examples

Upload

Site Secrets Group

Generate PKI

Examples

Check PKI Certs

Examples

Secrets

Encrypt

Examples

Decrypt

Examples

Wrap

Examples

genesis_bundle

Examples

generate

passphrases

Example

CLI Repository Overrides

Self-Contained Repository

Git Reference

Linting

Generate

Passphrase

Examples

Salt

Examples

26 KiB

Raw Blame History