26 Commits

Author SHA1 Message Date
Alexander Hughes
8946663381 Update Pegleg docs jobs
This patch brings Pegleg in line with other Airship projects in their
method of updating documentation.  This is achieved by:
1. Adding docs-on-readthedocs job
2. Adding readthedocs webhook and project name vars used for Pegleg
3. Removing airship-pegleg-doc-build
4. Removing doc-build playbook

Change-Id: Iaa4139ccb1cd9d7ca20a6b502ef2a152908147d0
2019-05-20 08:16:16 -05:00
Rajeshwari Dharwadkar
6ee2aaf845 Support pegleg to run on opensuse leap15 image
Add DISTRO parameter to support multiple distros
Add Dockerfile for opensuse to build leap 15 image.

Change-Id: I7a529476937494e042a4801117489325aa6621c7
2019-05-14 09:41:21 -07:00
Zuul
05ae434952 Merge "tests: Improve unit tests runtime performance" 2019-05-08 19:20:48 +00:00
Zuul
177486d8e6 Merge "zuul: Remove redundant airship-pegleg-tox-py36 job." 2019-05-08 18:58:35 +00:00
Ian H. Pittwood
33286a1173 Simplify whitespace-linter script
Pegleg currently uses `find` to search files in the whitespace-linter
script. A more simplified approach could be taken by using `git grep`
instead. This method levarages .gitignore so a separate list of files
in the script no longer needs to be maintained. This is the method used
by Airship Armada.

Change-Id: I26a2a95f533b9ff62de784d004f25ade552a5b31
2019-04-10 15:37:08 -05:00
Felipe Monteiro
b28788325f tests: Improve unit tests runtime performance
This patch set does 2 things:

1) Improves unit test runtime peformance via pytest-xdist [0]
2) Reduces finnicky nature of `is_connected` helpers which
   sometimes skip even when there is access to the internet;
   logic has been added to make these checks more accurate
   to avoid skipping tests

Note that while there are newer alternatives to pytest-xdist they
are only compatible with much newer versions of Python.

[0] https://pypi.org/project/pytest-xdist/

Change-Id: Ib04b48ebabca0551058e5e1065056f4e559fbfe6
2019-04-04 14:21:43 +00:00
Scott Hussey
a640ebf85a (zuul) Fix image publish job
- Fix issue in post pipeline image publish job introduced
  by Ansible update

Change-Id: I39fa51ea11804db065203761c475b48d269796f4
2019-04-03 14:51:51 -05:00
Felipe Monteiro
6c1b6e65b8 zuul: Remove redundant airship-pegleg-tox-py36 job.
The openstack-python36-jobs template already exists; thus there
is no need for airship-pegleg-tox-py36. Since airship-pegleg-tox-py36
installs cfssl as a prerequisite for unit tests, move the install-cfssl.sh
command to tox.ini prior to unit test execution, allowing for the
airship-pegleg-tox-py36 Zuul job and its associated playbook to be
removed.

Change-Id: I66de957a1a57ef246476c1a81954cd0f822cb8be
2019-03-26 20:38:14 +00:00
Aaron Sheffield
39119d5895 Updating Docker Gate use of zuul.newrev
- Zuul updated ansible to 2.7, no longer uses missing variables.
- Using an if to try and address.

Change-Id: I547097efc9ed2b2ca308cf394fe14dab16abacc4
2019-03-22 11:47:12 -05:00
Felipe Monteiro
2a8d2638b3 pki: Port Promenade's PKI catalog into Pegleg
This patch set implements the PKICatalog [0] requirements
as well as PeglegManagedDocument [1] generation requirements
outlined in the spec [2].

Included in this patch set:

* New CLI entry point called "pegleg site secrets generate-pki"
* PeglegManagedDocument generation logic in
  engine.cache.managed_document
* Refactored PKICatalog logic in engine.cache.pki_catalog derived
  from the Promenade PKI implementation [3], responsible for
  generating certificates, CAs, and keypairs
* Refactored PKIGenerator logic in engine.cache.pki_generator
  derived from Promenade Generator implementation [4],
  responsible for reading in pegleg/PKICatalog/v1 documents (as
  well as promenade/PKICatalog/v1 documents for backwards
  compatibility) and generating required secrets and storing
  them into the paths specified under [0]
* Unit tests for all of the above [5]
* Example pki-catalog.yaml document under pegleg/site_yamls
* Validation schema for pki-catalog.yaml (TODO: implement
  validation logic here: [6])
* Updates to CLI documentation and inclusion of PKICatalog
  and PeglegManagedDocument documentation
* Documentation updates with PKI information [7]

TODO (in follow-up patch sets):

* Expand on overview documentation to include new Pegleg
  responsibilities
* Allow the original repository (not the copied one) to
  be the destination where the secrets are written to
* Finish up cert expiry/revocation logic

[0] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
[1] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
[2] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html
[3] https://github.com/openstack/airship-promenade/blob/master/promenade/pki.py
[4] https://github.com/openstack/airship-promenade/blob/master/promenade/generator.py
[5] https://review.openstack.org/#/c/611739/
[6] https://review.openstack.org/#/c/608159/
[7] https://review.openstack.org/#/c/611738/

Change-Id: I3010d04cac6d22c656d144f0dafeaa5e19a13068
2019-01-15 13:29:21 -06:00
Felipe Monteiro
e3f6efbb1f trivial: fix whitespace-linter catching false positives
Ignores .pyc files and files contained in htmlcov as a coverage
job was recently added.

Change-Id: I33b1b88e90060f3b26f41b6acbbc0c7091caeb57
2018-11-25 12:30:56 -05:00
Tin Lam
152b42bc3b Consolidate linter jobs
Consolidating all the linting to a single job rather than having a one-off
just to lint for trailing whitespaces. As most of this projects are python,
this should already be covered by PEP8, but will be scanning for yamls with
trailing whitespaces.

Change-Id: Iee33a69ff234d21c08217faa33a19e11dfef0ad9
Signed-off-by: Tin Lam <tin@irrational.io>
2018-11-12 02:30:20 -06:00
Felipe Monteiro
ec34ed056f Allow tox to support regexes for unit tests
This patch set adds a shell wrapper script to allow
tox to be used to run a subset of unit tests via regexes:

To run all unit tests, execute::

  $ tox -epy35

To run unit tests using a regex, execute::

  $ tox -epy35 -- <regex>

Change-Id: I2ba1e18226d686cb549a075e020ba02e24204829
2018-10-22 10:18:05 -04:00
Zuul
2ea774a744 Merge "Switch to openstack-python35-jobs template for py35 CI gate" 2018-10-15 19:25:30 +00:00
Felipe Monteiro
17b4d6e563 Switch to openstack-python35-jobs template for py35 CI gate
This patch set switches to openstack-python35-jobs for the py35
CI gate.

Change-Id: I12239086984a387dde33e5fdc451a9c51deec953
2018-10-02 22:32:47 +00:00
Felipe Monteiro
57a6c6a84e Implement global lint and lint by site logic
With the implementation of revisioned repository to the CLI in
https://review.openstack.org/#/c/577886 there was a change to
the lint command [0], which changed it from being a global lint
to a site-level (targetted lint)... kind of: Only the CLI logic
was modified to support targetted single-site linting. Thus,
the first issue this patch set addresses is implementing the
back-end logic to realize targetted, single-site linting.

The second issue this patch set addresses is re-supporting global
linting (linting all sites within a repository) which means that
this partially reverts [0] which had (kind of) replaced global
linting with per-site linting.

So, this patch set:

1) Implements targetted, single-site linting back-end logic
2) Re-implements global linting for all sites in a repo
3) Adds unit tests for both
4) Adds some helper functions to util.engine.definition to
   help with 1) and 2)

[0] https://review.openstack.org/#/c/577886/4/src/bin/pegleg/pegleg/cli.py@191

Change-Id: I5147282556763d93dfaf06912d2c4c876e1bd69f
2018-10-02 21:41:56 +01:00
Zuul
6c6bea992d Merge "Fix: git commit id labels on images" 2018-10-01 12:07:52 +00:00
Roman Gorshunov
de6486c380 Fix: git commit id labels on images
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
   or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
   a few other properties (annotations) according to the latest Specs

Change-Id: I57318d4662d90b439d4b7766f7c67571e0f69f15
2018-09-21 03:31:12 +02:00
Felipe Monteiro
adb23bc2a6 docs: Add developer overview guide
This patch set introduces a developer overview documentation
page to Pegleg to help developers onboard.

Change-Id: Ia453d76f024db39c6bdd97a44bfe1db1c25193f9
2018-09-13 18:28:58 -06:00
Gage Hugo
ad7e855cf8 Consolidate pep8/bandit zuul gating
This change adds the global zuul pep8 tox job, which runs both
bandit and pep8 using tox. This also removes the two other airship
specific lint-pep8 and bandit zuul jobs since they are both covered
by the default openstack global one.

Also cleaned up the tox.ini by moving the requirements into the
test-requirements.txt file.

Change-Id: Iab37a8090515936732e390b1f7c6d281e014e31c
2018-08-21 20:55:39 +00:00
Jerome Brette
4727df6b80 Update Dockerfile to allow override of FROM variable
l is to let user customize the base image of the component
by passing FROM=myimage during the build process. This would let any
project leveraging Airship ensure that the base image is matching the
security requirements for that project and still use the same Dockerfile.
This will also ease the control of the /etc/apt/source.list
and thereby the result of apt-get update/upgrade procedure.
2. The above goal is achievable by using docker-ce feature such as:
ARG FROM="defaultbaseimage:xx"
FROM ${FROM}
For this reason, the installation of docker.io in the Zuul gating is beeing
replaced by docker-ce.
3. Third Goal is to bring consistency with the other compoenents leveraging
Helm such as the openstack-helm and potentially use bindep the same way
the LOCI images are to ensure
4. The new syntax in the Dockerfile is still commented out until the associated
image builder have been updated to use docker-ce as they have been for the LOCI
images.

Change-Id: I6703589f32487f5668d709f485dae5782b13c002
2018-07-17 14:37:08 -05:00
Zuul
c7dc0e71dc Merge "Rename docs to doc to align with OpenStack standard" 2018-06-10 01:24:15 +00:00
Scott Hussey
f7b7946e4b (zuul) Docker image jobs
- Add check/gate jobs for image building
- Add post job for image building and publish to quay.io

Change-Id: Ia7c1f67ec4a5ae11e1fad489259f8418bac0a644
2018-06-08 08:54:34 -05:00
Felipe Monteiro
f5fc46c7af Rename docs to doc to align with OpenStack standard
This patchset updates docs to doc to align with OpenStack
standard. Follow-up patchset will be needed to publish
documentation to OpenStack [0].

[0] https://docs.openstack.org/doc-contrib-guide/project-guides.html

Change-Id: I90e5f9129207901402e26ed9488ec6e065568fe1
2018-06-06 09:25:14 -04:00
Scott Hussey
4ad86e0728 Basic zuul gates
- Pep8, bandit and unit tests
- Document build (no publish)

Change-Id: I2ca67e69f80aff63576bebd14da412e2f138f54a
2018-05-31 13:10:41 -05:00
Felipe Monteiro
bfac6ada6c Zuul: Initial Airship-Pegleg checks
This PS adds the skeleton for a set of zuul checks and gates for
Airship, using the framework from OpenStack-Helm.

Change-Id: I4cc071d0076e265e5c34f15616ccf40c0b3a83a8
2018-05-17 22:09:35 +00:00