29 Commits

Author SHA1 Message Date
Felipe Monteiro
29894ee854 Move to stestr for functional/integration tests
This moves to stestr for functional/integration tests to
align with preferred OpenStack test runner and also because
py.test is causing issues here: [0].

[0] http://logs.openstack.org/45/573045/8/check/airship-deckhand-integration-uwsgi-py35/60bd7ae/job-output.txt.gz#_2018-06-22_03_00_51_267408

Change-Id: I701c71aef2a122b8f1fc64285cb799f71cfe520f
2018-07-21 15:32:26 -04:00
Felipe Monteiro
a552bf2a0f Zuul: Integration tests via uwsgi.
This adds a uwsgi integration test gate to .zuul.yaml so that
deploying Deckhand via uwsgi (in a more standalone fashion,
sans containerization) works as intended.

Change-Id: I933f4781cd72e2df309efcb0515441db7ab96895
2018-05-31 17:31:50 +00:00
Felipe Monteiro
3d53d53712 Add uwsgi functional test check to .zuul.yaml
This adds a uwsgi functional test check to .zuul.yaml so that
deploying Deckhand via uwsgi (in a more standalone fashion,
sans containerization) works as intended.

Change-Id: I931ab4d11719daca7665d3a25b00e353c707237e
2018-05-29 20:25:15 +00:00
Felipe Monteiro
119080b597 Use Ansible playbooks for functional testing gating
This patchset converts much of the previous logic in
functional-tests.sh into Ansible playbooks to be executed
by Zuul. This mainly includes all the Docker-related
deployment logic.

The functional-tests.sh script has been slimmed down to
just work with uwsgi so that a standalone functional
test deployment can be performed relatively easily,
mainly by developers.

Finally, py27 support for the gate has been dropped
as the Dockerfile in this project currently assumes
python3 for installing requirements and so forth,
leading to requirements issues blocking the gate.

Change-Id: I903a2845390061641d292fb0c016ba6a53723fc9
2018-05-29 15:17:28 +00:00
Felipe Monteiro
1cbe993b6b Add functional tests to .zuul.yaml
This patchset adds functional tests to .zuul.yaml. Additionally
it adds a functional-py35 job as well which will also be kicked
off via Zuul.

Change-Id: Ic2d1db4d3cd65c4d93c3a6f04e6efeeba9755f07
2018-05-19 05:22:24 +00:00
Luna Das
8538ff5671 Add no oauth middleware to bypass keystone authentication
This PS adds noauth middleware to bypass keystone authentication
which will occur when Deckhand's server is executed in development
mode. Development mode is enabled by setting development_mode as True
in etc/deckhand/deckhand.conf.sample.

The logic is similar to Drydock's here: [0].

[0] 1c78477e95/drydock_provisioner/util.py (L43)

Co-Authored-By: Luna Das <luna.das@imaginea.com>
Co-Authored-By: Felipe Monteiro <felipe.monteiro@att.com>
Change-Id: I677d3d92768e0aa1a550772700403e0f028b0c59
2018-05-08 03:46:52 +01:00
Felipe Monteiro
c094b16ff6 Clean up integration test script
This PS:

* adds a trap to clean up OSH which is deployed in the
  course of integration tests. It appears as though node cleanup
  in Jenkins is hanging so this is to try to ameliorate that
* creates a deckhand.conf.test to be used by functional and
  integration tests instead of writing it out dynamically [0]
* updates logging.conf.sample to dump logs to stdout/stderr
  by default as this is amenable to containers
* makes test_gabbi.py common between functional and integration
  tests to avoid unnecessary code duplication

[0] review comments in https://review.gerrithub.io/#/c/att-comdev/deckhand/+/407638/

Change-Id: I762fb0bde5f75effcde56316d92bd57b30026995
2018-05-01 21:45:03 +00:00
Felipe Monteiro
1566b9541a Clean up functional test directory and entrypoint script
This PS simply reorganizes Deckhand's functional test directory
to make it more maintainable and readable as right now it is
hard to figure out what is covered by a functional test and
what isn't.

Additionally, the entrypoint for these tests in tools/functional-tests.sh
has also been refactored slightly.

Change-Id: I262c7e1f7cbce248c12ee013a9bab4e32b89adee
2018-04-20 22:07:04 +01:00
Felipe Monteiro
f30484a14c Add integration tests
This patch set adds integration tests to Deckhand
where "integration" means the interaction between
Deckhand, Barbican and Keystone. OSH is used to
deploy Keystone and Barbican and Docker to deploy
PostgreSQL and Deckhand.

Unlike functional testing in Deckhand, all
integration tests use the default in-code policy
defaults and an admin token supplied by keystone
to validate authN and authZ.

The test scenarios consist of Deckhand secret
lifecycle management as well as document rendering
with secrets retrieved from Barbican.

Change-Id: Ib5ae1b345b2a4bd579671ec4ae9a232c2e3887dc
2018-04-18 09:05:04 -04:00
Felipe Monteiro
e23f46c152 Fix running functional tests via uwsgi
Recently https://review.gerrithub.io/#/c/406626/ broke functional
tests via uwsgi because it changed how entrypoint.sh is called
which is performed during functional-tests.sh when uwsgi is used
to drive the server for testing. This changes how entrypoint.sh
is called so that the tests now pass with uwsgi.

Change-Id: I8252350676e61d5214da11e9ed282cc3399288d9
2018-04-10 16:48:03 -04:00
Bryan Strassner
5f1fbbee3c [396582] Add alembic support to Deckhand
Updates Deckhand to use alembic to manage database upgrades.
Moves from creating tables at startup of Deckhand to the
db-sync job.

Change-Id: I6f4cb237fadc46fbee81d1c33096f48a720f589f
2018-04-06 23:30:16 -04:00
Felipe Monteiro
5c9efa9d74 Enable multiple threads, disabled muliple workers
This sets multiple threads in Deckhand's chart config (4)
and set workers to just 1.

Deckhand's database is not configured to work with multiprocessing.
Currently there is a data race on acquiring shared SQLAlchemy
engine pooled connection strings when workers > 1. As a
workaround, we use multiple threads but only 1 worker. For more
information, see:

https://github.com/att-comdev/deckhand/issues/20

Change-Id: I60adeffff5461fdda957124232bc5a606baae413
2018-04-02 12:38:20 -04:00
Felipe Monteiro
453927facf Improve document validation module.
This PS rewrites the document_validation module in
Deckhand to achieve the following goals:

  * better validation resiliency
  * add support for different document schema versions
  * better support for DataSchema validation
  * separation of concerns by splitting up validations
    into separate classes
  * support for validating documents that rely on
    a DataSchema passed in via the same payload
  * support for generating multiple validation errors
    rather than returning after the first one found
  * increase testing validations for unit/functional
    tests

Better validation resiliency is achieved through more
robust exception handling. For example, it is possible
for a ``DataSchema`` to be 100% valid from the POV of
built-in schema validation, but if the "data" section
itself is utterly invalid, then an exception will be
raised -- such an exception is treated as a critical
failure.

Better generation of error messages is achieved by
creation more validation error message results.

DataSchema validation was previously wonky. A DataSchema
had to first be created in 1 revision before it could be
referenced by a batch of documents in sequential revisions.
Now, a DataSchema can be created in the same (or previous)
revision as documents that rely on it and used to validate
said documents.

Finally, the module was heavily rewritten so that more
nuanced validations can be built by inheriting from
``BaseValidator`` so as to allow for easier code
readability and maintainability.

Change-Id: Ie75742b984b7ad392cb41decc203d42842050c80
2018-01-15 16:51:52 -05:00
Felipe Monteiro
75d84312de Sorting/filtering for rendered-documents.
This PS implements sorting and filtering for rendered-documents
endpoint, adds additional validations for sorting, filtering
and other layering scenarios, and updates rendered-documents
and buckets documentation.

Layering scenarios added:
  - Updating the LayeringPolicy with 2 layers in the layerOrder
    (down from 3) such that the site document should have its
    parent document recomputed as the global document.
  - A deletion action layering scenario (DH currently only has
    merge, replace scenarios in its funcitonal test suite.)

Documentation updated:
  - clarify the access levels for buckets, which has been a
    source of confusion.
  - update api-ref documentation for rendered-documents

Change-Id: Idb9b42351dfbdf75a19282c8478065e7564cfc26
2018-01-15 15:25:08 -04:00
Felipe Monteiro
7e460e0f8a Update Deckhand image: logging configuration values.
This is to update the logging values that get provided to logging.conf
to be in line with logging in containers: outputting logging messages
to stdout and stderr.

Change-Id: Ib780a35c51cb6ba0cbb66ee8b2ea1836b83b9a61
2018-01-12 19:51:10 -05:00
Felipe Monteiro
2f0d5796e3 Revert fix pifpaf run postgresql failing
This reverts https://review.gerrithub.io/#/c/393980/ which was
a temporary workaround to unblock the Deckhand gate. pifpaf should
be used to run unit tests as having to install Docker just to kick
off unit tests is excessive.

However, the unit-tests.sh script is maintained in tools/ directory
as a fallback.

Change-Id: I24a10d4b3ea00006004f27d0086719fb0bf86dd9
2018-01-12 11:57:44 -04:00
Felipe Monteiro
0fc02a0ce2 fix: Testing with multiple workers
This PS allows Deckhand to be able to run functional tests with
multiple workers. To achieve that, a document validation bug
was fixed: undeleted data schemas from all previous revisions are
considered. (The test schema-validation-success_add_invalid_document
was failing sporadically because of data race conditions with
only considering data schemas from the last revision with multiple
workers.)

The number of workers for running functional tests via uwsgi
has been increased to the number of CPU cores available on
the server to consistently validate concurrency.

Change-Id: I12589c2ed10495a1eb30757b6bacc5370503d0f4
2018-01-09 20:03:14 -04:00
Felipe Monteiro
3d9dbc88cd functional tests: Dump logs to stdout/stderr
This PS creates a logging.conf file to be used inside the Deckhand
container for functional tests. The logging.conf file overrides
logging options to dump all messages to stdout and stderr, which
is the common practice for container logging. This file is
referenced in turn by the Deckhand configuration file.

Now it should be possible to actually see debug and error messages
in the CICD gate following functional test execution failure.

Currently, failures are resulting in completely useless output
like this:

  === Deckhand Server Log ===
  + cat deckhand.log
  cat: deckhand.log: No such file or directory
  + cleanup
  ...

Which in other words makes debugging using the Jenkins logs
virtually impossible.

Change-Id: Id64e6e1113fde08bfd476b0cfa3fc716f7b9801e
2018-01-09 14:43:05 +00:00
Mark Burnett
b710eb64ec Test fix: remove conflicting docker run option
Change-Id: I898589664bcbf4faf2cb61c8eba5d4cbb89b0ca3
2017-12-14 09:16:22 -06:00
Felipe Monteiro
564b0e08f1 Functional tests via Deckhand container and Docker
Change-Id: Ibe863cd5f647fab060de9e5e6937cbcd8b68d318
2017-12-12 18:52:54 -04:00
Felipe Monteiro
c55fa41238 Create results directory for functional test results if doesn't exist
Change-Id: Ib9a806a8115bda7e190662fc70744788852f3a36
2017-10-26 21:33:44 +01:00
Felipe Monteiro
bead00e23e HTML test report for Deckhand functional tests
This PS leverages pytest-html to generate an HTML report for
Deckhand functional tests. The test_gabbi.py file was updated to
use pytest as the test runner as it is compatible with pytest-html.
The report is saved in results/index.html. Test docs were updated.

Change-Id: I0b611bf124bf87d801ab93dd2a5d16f136e4801d
2017-10-25 18:18:15 -04:00
Felipe Monteiro
cdec6356a5 Add health resource for ucp-integration API convention
It is a UCP API convention to include a health resource in order
for other components to access and validate Deckhand's health status [0].

As such, this PS accomplishes that goal. Also add uwsgi.ini file to
instantiate the webserver using a more complex configuration that
can be overriden more easily.

[0] https://github.com/att-comdev/ucp-integration/blob/master/docs/api-conventions.md#health-check-api

Change-Id: Ice24cec6d0b98c16af62d9436925083d4092a032
2017-10-23 17:34:03 +01:00
Felipe Monteiro
90226c2ae1 Integrate Deckhand with keystone auth
This PS integrates Deckhand with keystone auth so
that Deckhand can check whether a keystone token is
authenticated (by way of keystonemiddleware)
before proceeding with any requests.

The architecture for this PS is borrowed from [0]
which successfully integrates keystone authentication
with the falcon web application framework. However,
additional Deckhand-specific changes were made for
tests to pass.

The following changes have been made:

  - add paste deploy configuration file which adds
    keystonemiddleware integration to Deckhand; this
    makes it trivial for keystonemiddleware to determine
    whether a token in the X-Auth-Token header is authenticated
  - use paste.deploy to create a web app
  - update unit tests for testing controllers
  - update functional test script to ignore keystone authentication
    because functional tests don't currently support keystone
    integration

[0] https://github.com/stannum-l/nautilus

Change-Id: I6eeeb4a4d9ab1f1cc8fb338e5cc21136ab4d5684
2017-10-16 19:54:46 +01:00
Felipe Monteiro
582dee6fb9 DECKHAND-61: oslo.policy integration
This PS implements oslo.policy integration in Deckhand.
The policy.py file implements 2 types of functions for
performing policy enforcement in Deckhand: authorize,
which is a decorator that is used directly around
falcon on_HTTP_VERB methods that raises a 403 immediately
if policy enforcement fails; and conditional_authorize,
to be used inside controller code conditionally.

For example, since Deckhand has two types of documents
with respect to security -- encrypted and cleartext
documents -- policy enforcement is conditioned on the
type of the documents' metadata.storagePolicy.

Included in this PS:
  - policy framework implementation
  - policy in code and policy documentation for all
    Deckhand policies
  - modification of functional test script to override
    default admin-only policies with custom policy file
    dynamically created using lax permissions
  - bug fix for filtering out deleted documents (and
    its predecessors in previous revisions) for
    PUT /revisions/{revision_id}/documents
  - policy documentation
  - basic unit tests for policy enforcement framework
  - allow functional tests to be filtered via regex

Due to the size of this PS, functional tests related to
policy enforcement will be done in a follow up.

Change-Id: If418129f9b401091e098c0bd6c7336b8a5cd2359
2017-10-07 18:43:28 +01:00
Felipe Monteiro
a05137ee5e [tests] Downgrade postgresql to 9.5 for functional tests
Given that the Deckhand chart uses 9.5 for production, functional
tests should align with that reality [0].

[0] 7a0d8758c4/deckhand/values.yaml (L24)

Change-Id: I3342ea3c7e1c89099c059aeebc280c9b312cdf18
2017-10-02 17:25:35 +01:00
Felipe Monteiro
af0bfd813d Deckhand postgresql compatibility.
Currently, Deckhand is not fully compatible with postgresql as
it uses sqlite for all of its testing, including functional testing.
Since postgresql will be used in prod, Deckhand obviously must
support it, in addition to sqlite, needed for unit testing.

This commit alters the functional testing script to use postgresql
as well as makes necessary back-end changes to support postgresql.

Included in this commit:

  - alter tools/functional-tests.sh so that it uses postgresql
    as the db connection
  - modifies primary key for Bucket DB model to be an Integer rather
    than a String
  - updates foreign key to point to new primary key
  - updates necessary integration logic so that the bucket name
    is still known by the Document DB model and returned in
    appropriate response bodies

Change-Id: I7bc806fb18f7b47c13978dcd806d422a573a06b3
2017-09-22 19:28:47 +01:00
Felipe Monteiro
905ca1732b Fix Deckhand logging
The following deployment logic should be included to get logging
to work correctly:

1) tox -egenconfig
   - Store the output in /etc/deckhand/deckhand.conf for example
2) Copy logging.conf.sample in etc folder to /etc/deckhand/logging.conf
3) Set the following options in under [DEFAULT] in
   /etc/deckhand/deckhand.conf:

   - log_config_append = /etc/deckhand/logging.conf
   - log_file = deckhand.log
   - log_dir = <path/to/deckhand/dir>
   - debug = true (optionally)

Change-Id: I8e8ebd041e801a5eef0f10b1bbc76ce95aecbf55
2017-09-19 15:45:40 -04:00
Mark Burnett
ee3a96d518 Replace existing functional tests with Gabbi
This gives a starting point for data-driven functional testing.

Change-Id: I22c2fcd593b92b2e27c809cbe28cc6f44d2774cb
2017-08-18 12:24:08 -05:00