Commit Graph

40 Commits

Author SHA1 Message Date
Felipe Monteiro
a552bf2a0f Zuul: Integration tests via uwsgi.
This adds a uwsgi integration test gate to .zuul.yaml so that
deploying Deckhand via uwsgi (in a more standalone fashion,
sans containerization) works as intended.

Change-Id: I933f4781cd72e2df309efcb0515441db7ab96895
2018-05-31 17:31:50 +00:00
Felipe Monteiro
3d53d53712 Add uwsgi functional test check to .zuul.yaml
This adds a uwsgi functional test check to .zuul.yaml so that
deploying Deckhand via uwsgi (in a more standalone fashion,
sans containerization) works as intended.

Change-Id: I931ab4d11719daca7665d3a25b00e353c707237e
2018-05-29 20:25:15 +00:00
Felipe Monteiro
119080b597 Use Ansible playbooks for functional testing gating
This patchset converts much of the previous logic in
functional-tests.sh into Ansible playbooks to be executed
by Zuul. This mainly includes all the Docker-related
deployment logic.

The functional-tests.sh script has been slimmed down to
just work with uwsgi so that a standalone functional
test deployment can be performed relatively easily,
mainly by developers.

Finally, py27 support for the gate has been dropped
as the Dockerfile in this project currently assumes
python3 for installing requirements and so forth,
leading to requirements issues blocking the gate.

Change-Id: I903a2845390061641d292fb0c016ba6a53723fc9
2018-05-29 15:17:28 +00:00
Felipe Monteiro
5cd77c8a22 Drop gather prom metrics from airship-deckhand-ubuntu job
Prometheus scrapers aren't running in the airship-deckhand
deployment job, so no need to include this here. It is
mainly being removed because of a POST_FAILURE issue
resulting from including the role in question in this job.

Change-Id: Ia080ff333c82f5b4b13d2e5db8c88741c0dc4613
2018-05-23 22:55:52 +00:00
Felipe Monteiro
1cbe993b6b Add functional tests to .zuul.yaml
This patchset adds functional tests to .zuul.yaml. Additionally
it adds a functional-py35 job as well which will also be kicked
off via Zuul.

Change-Id: Ic2d1db4d3cd65c4d93c3a6f04e6efeeba9755f07
2018-05-19 05:22:24 +00:00
Pete Birley
7a32dc92e0 Zuul: Initial Airship-Deckhand checks
This PS adds the skeleton for a set of zuul checks and gates for
Airship, using the framework from OpenStack-Helm.

Change-Id: If5b6550c6f0d9d1f96c4822e72d7d519dfca3c79
Signed-off-by: Pete Birley <pete@port.direct>
2018-05-17 12:36:19 -05:00
Anthony Lin
a004c7a19e Update Deckhand API Pod Labels
As part of ongoing effort to update the "application" and
"component" labels for the UCP components, there is a need
to align with the convention. We will update the label for
the deckhand API pod in this case.

Also updated helm_tk.sh to point to openstack-helm-infra for
reference to helm-toolkit as helm-toolkit has been removed
from the openstack-helm repo [0]

[0] https://review.openstack.org/#/c/558065/

Change-Id: I753c4ce653790250b79986c670224d0962f7676f
2018-05-14 04:54:55 +00:00
Luna Das
8538ff5671 Add no oauth middleware to bypass keystone authentication
This PS adds noauth middleware to bypass keystone authentication
which will occur when Deckhand's server is executed in development
mode. Development mode is enabled by setting development_mode as True
in etc/deckhand/deckhand.conf.sample.

The logic is similar to Drydock's here: [0].

[0] 1c78477e95/drydock_provisioner/util.py (L43)

Co-Authored-By: Luna Das <luna.das@imaginea.com>
Co-Authored-By: Felipe Monteiro <felipe.monteiro@att.com>
Change-Id: I677d3d92768e0aa1a550772700403e0f028b0c59
2018-05-08 03:46:52 +01:00
Felipe Monteiro
444e4d9dcc [fix gate] Unblock failing integration job
This PS fixes the integration job failing on the following:

  Unable to match $.[0].data.`split(:, 0, 1)` + "://" +
  $.[0].data.`split(/, 2, 3)` + "/v1" as
  http://barbican.openstack.svc.cluster.local:80/,
  got http://barbican.openstack.svc.cluster.local:80/v1

This change corrects this issue.

Change-Id: Iccc172efa37fe06df3af59a569bbc6f135c773fe
2018-05-07 18:05:13 +00:00
Felipe Monteiro
c094b16ff6 Clean up integration test script
This PS:

* adds a trap to clean up OSH which is deployed in the
  course of integration tests. It appears as though node cleanup
  in Jenkins is hanging so this is to try to ameliorate that
* creates a deckhand.conf.test to be used by functional and
  integration tests instead of writing it out dynamically [0]
* updates logging.conf.sample to dump logs to stdout/stderr
  by default as this is amenable to containers
* makes test_gabbi.py common between functional and integration
  tests to avoid unnecessary code duplication

[0] review comments in https://review.gerrithub.io/#/c/att-comdev/deckhand/+/407638/

Change-Id: I762fb0bde5f75effcde56316d92bd57b30026995
2018-05-01 21:45:03 +00:00
Felipe Monteiro
2165e8b396 [test] Cover all secret Deckhand types in integration tests
This patch set accomplishes 2 things:
1) Fixes an issue in Deckhand caused by improperly using the wrong
   Barbican secret type for the PrivateKey Deckhand document type.
2) Tests all Deckhand secret types against Barbican via integration
   testing.

The current error being raised is this: "Provided object does not match
schema 'Secret': u'privatekey' is not one of ['symmetric', 'passphrase',
'private', 'public', 'certificate', 'opaque']."

Change-Id: I8231c87782902850fe0632d0346c567c7481e95f
2018-04-27 14:25:29 -04:00
Felipe Monteiro
1566b9541a Clean up functional test directory and entrypoint script
This PS simply reorganizes Deckhand's functional test directory
to make it more maintainable and readable as right now it is
hard to figure out what is covered by a functional test and
what isn't.

Additionally, the entrypoint for these tests in tools/functional-tests.sh
has also been refactored slightly.

Change-Id: I262c7e1f7cbce248c12ee013a9bab4e32b89adee
2018-04-20 22:07:04 +01:00
Felipe Monteiro
f30484a14c Add integration tests
This patch set adds integration tests to Deckhand
where "integration" means the interaction between
Deckhand, Barbican and Keystone. OSH is used to
deploy Keystone and Barbican and Docker to deploy
PostgreSQL and Deckhand.

Unlike functional testing in Deckhand, all
integration tests use the default in-code policy
defaults and an admin token supplied by keystone
to validate authN and authZ.

The test scenarios consist of Deckhand secret
lifecycle management as well as document rendering
with secrets retrieved from Barbican.

Change-Id: Ib5ae1b345b2a4bd579671ec4ae9a232c2e3887dc
2018-04-18 09:05:04 -04:00
Felipe Monteiro
e23f46c152 Fix running functional tests via uwsgi
Recently https://review.gerrithub.io/#/c/406626/ broke functional
tests via uwsgi because it changed how entrypoint.sh is called
which is performed during functional-tests.sh when uwsgi is used
to drive the server for testing. This changes how entrypoint.sh
is called so that the tests now pass with uwsgi.

Change-Id: I8252350676e61d5214da11e9ed282cc3399288d9
2018-04-10 16:48:03 -04:00
Bryan Strassner
5f1fbbee3c [396582] Add alembic support to Deckhand
Updates Deckhand to use alembic to manage database upgrades.
Moves from creating tables at startup of Deckhand to the
db-sync job.

Change-Id: I6f4cb237fadc46fbee81d1c33096f48a720f589f
2018-04-06 23:30:16 -04:00
Felipe Monteiro
5c9efa9d74 Enable multiple threads, disabled muliple workers
This sets multiple threads in Deckhand's chart config (4)
and set workers to just 1.

Deckhand's database is not configured to work with multiprocessing.
Currently there is a data race on acquiring shared SQLAlchemy
engine pooled connection strings when workers > 1. As a
workaround, we use multiple threads but only 1 worker. For more
information, see:

https://github.com/att-comdev/deckhand/issues/20

Change-Id: I60adeffff5461fdda957124232bc5a606baae413
2018-04-02 12:38:20 -04:00
Felipe Monteiro
9cae8f3b2b Switch to stestr
stestr is actively maintained. OpenStack is migrating to stestr,
so we should follow suit.

Change-Id: I1b958681cbf53281c9da91c747c9a2cfe2beccf3
2018-03-28 13:06:46 -04:00
Bryan Strassner
e2d2b89005 Merge "Improve document validation module." 2018-01-19 13:04:06 -05:00
Felipe Monteiro
ba1fe1c9ec Fix pifpaf not returning error code upon test failure
For whatever reason, the following command:

   pifpaf run postgresql -- <test command>

is not returning a non-zero error code on test failure.

(An example print out is included below.)

This PS updates pretty_tox.sh to forcibly raise a non-zero error
code in the event of test failure. It also renames the script
to run_pifpaf.sh to be more intuitive.

Example:

======
Totals
======
Ran: 7 tests in 5.7673 sec.
 - Passed: 6
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 1
Sum of execute time for each test: 2.6962 sec.

==============
Worker Balance
==============
 - Worker 0 (7 tests) => 0:00:02.698323
+ exit 0

  py35: commands succeeded
  congratulations :)

Change-Id: I7b1fa9d42295d06752997f251a0ec14082b44d03
2018-01-15 17:01:34 -05:00
Felipe Monteiro
453927facf Improve document validation module.
This PS rewrites the document_validation module in
Deckhand to achieve the following goals:

  * better validation resiliency
  * add support for different document schema versions
  * better support for DataSchema validation
  * separation of concerns by splitting up validations
    into separate classes
  * support for validating documents that rely on
    a DataSchema passed in via the same payload
  * support for generating multiple validation errors
    rather than returning after the first one found
  * increase testing validations for unit/functional
    tests

Better validation resiliency is achieved through more
robust exception handling. For example, it is possible
for a ``DataSchema`` to be 100% valid from the POV of
built-in schema validation, but if the "data" section
itself is utterly invalid, then an exception will be
raised -- such an exception is treated as a critical
failure.

Better generation of error messages is achieved by
creation more validation error message results.

DataSchema validation was previously wonky. A DataSchema
had to first be created in 1 revision before it could be
referenced by a batch of documents in sequential revisions.
Now, a DataSchema can be created in the same (or previous)
revision as documents that rely on it and used to validate
said documents.

Finally, the module was heavily rewritten so that more
nuanced validations can be built by inheriting from
``BaseValidator`` so as to allow for easier code
readability and maintainability.

Change-Id: Ie75742b984b7ad392cb41decc203d42842050c80
2018-01-15 16:51:52 -05:00
Felipe Monteiro
75d84312de Sorting/filtering for rendered-documents.
This PS implements sorting and filtering for rendered-documents
endpoint, adds additional validations for sorting, filtering
and other layering scenarios, and updates rendered-documents
and buckets documentation.

Layering scenarios added:
  - Updating the LayeringPolicy with 2 layers in the layerOrder
    (down from 3) such that the site document should have its
    parent document recomputed as the global document.
  - A deletion action layering scenario (DH currently only has
    merge, replace scenarios in its funcitonal test suite.)

Documentation updated:
  - clarify the access levels for buckets, which has been a
    source of confusion.
  - update api-ref documentation for rendered-documents

Change-Id: Idb9b42351dfbdf75a19282c8478065e7564cfc26
2018-01-15 15:25:08 -04:00
Felipe Monteiro
7e460e0f8a Update Deckhand image: logging configuration values.
This is to update the logging values that get provided to logging.conf
to be in line with logging in containers: outputting logging messages
to stdout and stderr.

Change-Id: Ib780a35c51cb6ba0cbb66ee8b2ea1836b83b9a61
2018-01-12 19:51:10 -05:00
Felipe Monteiro
2f0d5796e3 Revert fix pifpaf run postgresql failing
This reverts https://review.gerrithub.io/#/c/393980/ which was
a temporary workaround to unblock the Deckhand gate. pifpaf should
be used to run unit tests as having to install Docker just to kick
off unit tests is excessive.

However, the unit-tests.sh script is maintained in tools/ directory
as a fallback.

Change-Id: I24a10d4b3ea00006004f27d0086719fb0bf86dd9
2018-01-12 11:57:44 -04:00
Felipe Monteiro
0fc02a0ce2 fix: Testing with multiple workers
This PS allows Deckhand to be able to run functional tests with
multiple workers. To achieve that, a document validation bug
was fixed: undeleted data schemas from all previous revisions are
considered. (The test schema-validation-success_add_invalid_document
was failing sporadically because of data race conditions with
only considering data schemas from the last revision with multiple
workers.)

The number of workers for running functional tests via uwsgi
has been increased to the number of CPU cores available on
the server to consistently validate concurrency.

Change-Id: I12589c2ed10495a1eb30757b6bacc5370503d0f4
2018-01-09 20:03:14 -04:00
Felipe Monteiro
3d9dbc88cd functional tests: Dump logs to stdout/stderr
This PS creates a logging.conf file to be used inside the Deckhand
container for functional tests. The logging.conf file overrides
logging options to dump all messages to stdout and stderr, which
is the common practice for container logging. This file is
referenced in turn by the Deckhand configuration file.

Now it should be possible to actually see debug and error messages
in the CICD gate following functional test execution failure.

Currently, failures are resulting in completely useless output
like this:

  === Deckhand Server Log ===
  + cat deckhand.log
  cat: deckhand.log: No such file or directory
  + cleanup
  ...

Which in other words makes debugging using the Jenkins logs
virtually impossible.

Change-Id: Id64e6e1113fde08bfd476b0cfa3fc716f7b9801e
2018-01-09 14:43:05 +00:00
Felipe Monteiro
2bfb16e44d [Gate fix] Fix pifpaf run postgresql failing.
This PS unblocks the gate by replacing pifpaf to run postgresql
for unit tests with docker, as a workaround. This is because
"pifpaf run postgresql" is failing with pifpaf not being able
to find the command "pifpaf run". Steps to reproduce:

python3 -m virtualenv -p python3 /tmp/venv
source /tmp/venv/bin/activate
pip install -U pip wheel devpi-client setuptools
pip install pifpaf

$pifpaf run postgresql
>> pifpaf: 'run' is not a pifpaf command. See 'pifpaf --help'.
>> Did you mean one of these?
     help

The unit test script for spinning up the docker postgresql container
and then running unit tests is very similar to the pre-existing
script for running functional tests located in tools/ directory.

Change-Id: Ib0f414ff58007037ac12161876dcd7a10e91f48c
2018-01-08 17:28:01 -04:00
Mark Burnett
b710eb64ec Test fix: remove conflicting docker run option
Change-Id: I898589664bcbf4faf2cb61c8eba5d4cbb89b0ca3
2017-12-14 09:16:22 -06:00
Felipe Monteiro
564b0e08f1 Functional tests via Deckhand container and Docker
Change-Id: Ibe863cd5f647fab060de9e5e6937cbcd8b68d318
2017-12-12 18:52:54 -04:00
Felipe Monteiro
6ff443de2d Deckhand Makefile for CICD
Add Makefile to Deckhand for building charts and running lint checks
against the code base (including charts).

Modeled after Shipyard's Makefile.

Change-Id: I23a9b746369f5765cf4d75e9cb0854b9f5f65972
2017-10-30 16:21:49 -04:00
Felipe Monteiro
c55fa41238 Create results directory for functional test results if doesn't exist
Change-Id: Ib9a806a8115bda7e190662fc70744788852f3a36
2017-10-26 21:33:44 +01:00
Felipe Monteiro
bead00e23e HTML test report for Deckhand functional tests
This PS leverages pytest-html to generate an HTML report for
Deckhand functional tests. The test_gabbi.py file was updated to
use pytest as the test runner as it is compatible with pytest-html.
The report is saved in results/index.html. Test docs were updated.

Change-Id: I0b611bf124bf87d801ab93dd2a5d16f136e4801d
2017-10-25 18:18:15 -04:00
Felipe Monteiro
cdec6356a5 Add health resource for ucp-integration API convention
It is a UCP API convention to include a health resource in order
for other components to access and validate Deckhand's health status [0].

As such, this PS accomplishes that goal. Also add uwsgi.ini file to
instantiate the webserver using a more complex configuration that
can be overriden more easily.

[0] https://github.com/att-comdev/ucp-integration/blob/master/docs/api-conventions.md#health-check-api

Change-Id: Ice24cec6d0b98c16af62d9436925083d4092a032
2017-10-23 17:34:03 +01:00
Felipe Monteiro
90226c2ae1 Integrate Deckhand with keystone auth
This PS integrates Deckhand with keystone auth so
that Deckhand can check whether a keystone token is
authenticated (by way of keystonemiddleware)
before proceeding with any requests.

The architecture for this PS is borrowed from [0]
which successfully integrates keystone authentication
with the falcon web application framework. However,
additional Deckhand-specific changes were made for
tests to pass.

The following changes have been made:

  - add paste deploy configuration file which adds
    keystonemiddleware integration to Deckhand; this
    makes it trivial for keystonemiddleware to determine
    whether a token in the X-Auth-Token header is authenticated
  - use paste.deploy to create a web app
  - update unit tests for testing controllers
  - update functional test script to ignore keystone authentication
    because functional tests don't currently support keystone
    integration

[0] https://github.com/stannum-l/nautilus

Change-Id: I6eeeb4a4d9ab1f1cc8fb338e5cc21136ab4d5684
2017-10-16 19:54:46 +01:00
Felipe Monteiro
582dee6fb9 DECKHAND-61: oslo.policy integration
This PS implements oslo.policy integration in Deckhand.
The policy.py file implements 2 types of functions for
performing policy enforcement in Deckhand: authorize,
which is a decorator that is used directly around
falcon on_HTTP_VERB methods that raises a 403 immediately
if policy enforcement fails; and conditional_authorize,
to be used inside controller code conditionally.

For example, since Deckhand has two types of documents
with respect to security -- encrypted and cleartext
documents -- policy enforcement is conditioned on the
type of the documents' metadata.storagePolicy.

Included in this PS:
  - policy framework implementation
  - policy in code and policy documentation for all
    Deckhand policies
  - modification of functional test script to override
    default admin-only policies with custom policy file
    dynamically created using lax permissions
  - bug fix for filtering out deleted documents (and
    its predecessors in previous revisions) for
    PUT /revisions/{revision_id}/documents
  - policy documentation
  - basic unit tests for policy enforcement framework
  - allow functional tests to be filtered via regex

Due to the size of this PS, functional tests related to
policy enforcement will be done in a follow up.

Change-Id: If418129f9b401091e098c0bd6c7336b8a5cd2359
2017-10-07 18:43:28 +01:00
Felipe Monteiro
5e6b0bfe5c Merge "[tests] Downgrade postgresql to 9.5 for functional tests" 2017-10-04 10:00:53 -04:00
Felipe Monteiro
8bf4f7407d Revamp document hashing
This PS revamps document hashing. Instead of relying on Python's
built-in hash function to hash the contents of a document (i.e.
metadata and data values), sha256 from hashlib is used instead,
mostly for security purposes.

Further, new parameters have been added to the document DB model:
data_hash and metadata_hash, and the old value hash has been
dropped. The data type for storing the hashes has been changed
to String from BigInt.

Finally, testing documentation was added.

Change-Id: I428ddcbce1007ea990ca0df1aa630072a050c722
2017-10-02 18:09:13 +01:00
Felipe Monteiro
a05137ee5e [tests] Downgrade postgresql to 9.5 for functional tests
Given that the Deckhand chart uses 9.5 for production, functional
tests should align with that reality [0].

[0] 7a0d8758c4/deckhand/values.yaml (L24)

Change-Id: I3342ea3c7e1c89099c059aeebc280c9b312cdf18
2017-10-02 17:25:35 +01:00
Felipe Monteiro
af0bfd813d Deckhand postgresql compatibility.
Currently, Deckhand is not fully compatible with postgresql as
it uses sqlite for all of its testing, including functional testing.
Since postgresql will be used in prod, Deckhand obviously must
support it, in addition to sqlite, needed for unit testing.

This commit alters the functional testing script to use postgresql
as well as makes necessary back-end changes to support postgresql.

Included in this commit:

  - alter tools/functional-tests.sh so that it uses postgresql
    as the db connection
  - modifies primary key for Bucket DB model to be an Integer rather
    than a String
  - updates foreign key to point to new primary key
  - updates necessary integration logic so that the bucket name
    is still known by the Document DB model and returned in
    appropriate response bodies

Change-Id: I7bc806fb18f7b47c13978dcd806d422a573a06b3
2017-09-22 19:28:47 +01:00
Felipe Monteiro
905ca1732b Fix Deckhand logging
The following deployment logic should be included to get logging
to work correctly:

1) tox -egenconfig
   - Store the output in /etc/deckhand/deckhand.conf for example
2) Copy logging.conf.sample in etc folder to /etc/deckhand/logging.conf
3) Set the following options in under [DEFAULT] in
   /etc/deckhand/deckhand.conf:

   - log_config_append = /etc/deckhand/logging.conf
   - log_file = deckhand.log
   - log_dir = <path/to/deckhand/dir>
   - debug = true (optionally)

Change-Id: I8e8ebd041e801a5eef0f10b1bbc76ce95aecbf55
2017-09-19 15:45:40 -04:00
Mark Burnett
ee3a96d518 Replace existing functional tests with Gabbi
This gives a starting point for data-driven functional testing.

Change-Id: I22c2fcd593b92b2e27c809cbe28cc6f44d2774cb
2017-08-18 12:24:08 -05:00