281a660e82
Fixes #124 Thought pytest was collecting and running tests, the results were not actually being handled. If a test failed, it still appeared to pass. The fundamental reason for this is that only the GabbiSuite which contains all the tests from each YAML file was being checked. These look like tests to pytest and pass. The eventual fix is fairly complex and could maybe be made less so by learning how to use modern parameterized pytest[1] rather than the old yield style being used here. The fix includes: * Creating a PyTestResult class which translates various unitest result types into pytest skips or xfails or reraises errors as required. * Works around the GabbiSuite.run() based fixture handling that unitest-based runners automatically use but won't work properly in the pytest yield setting by adding start() and stop() methods the suite and yielding artifical tests which call start and stop.[2] Besides getting failing tests to actually fail this also gets some other features working: * xfail and skip work, including skipping an entire yaml file with he SkipFixture * If a single test from a file is selected, all the prior tests in the file will run too, but only the one requested will report. [1] http://pytest.org/latest/parametrize.html#pytest-generate-tests [2] This causes the number of tests to increase but it seems to be the only way to get things working without larger changes.
42 lines
1.3 KiB
Python
42 lines
1.3 KiB
Python
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
# not use this file except in compliance with the License. You may obtain
|
|
# a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
# License for the specific language governing permissions and limitations
|
|
# under the License.
|
|
"""Test pytest driving of tests.
|
|
|
|
Unittest loaders don't see this file and pytest doesn't see load_tests,
|
|
so we manage to get coverage across both types of drivers, from tox,
|
|
without duplication.
|
|
"""
|
|
|
|
import os
|
|
|
|
from gabbi import driver
|
|
from gabbi.tests import simple_wsgi
|
|
from gabbi.tests import test_intercept
|
|
|
|
TESTS_DIR = 'gabbits_intercept'
|
|
|
|
|
|
def test_from_build():
|
|
|
|
os.environ['GABBI_TEST_URL'] = 'takingnames'
|
|
test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
|
|
test_generator = driver.py_test_generator(
|
|
test_dir, intercept=simple_wsgi.SimpleWsgi,
|
|
fixture_module=test_intercept,
|
|
response_handlers=[test_intercept.TestResponseHandler])
|
|
|
|
# TODO(cdent): Where is our Python3!
|
|
# yield from test_generator
|
|
for test in test_generator:
|
|
yield test
|