This change implements template substitution ($ENVIRON, $RESPONSE,
etc) in the left hand side of a response_json_paths test. This enables
testing in situations where responses may include a server side
generated identifier as the key in an object structure.
As things stand, this is the bare minimum implementation that assumes
that the test creator will be responsible for dealing with the
vargrancies of quoting that will be required in order to generate
legitimate JSON path expressions. For example if there is a uuid (with
hyphens) at $RESPONSE['$.id'] then this expression is likely to fail:
$.nested.structure.$RESPONSE['$.id'].name: foobar
as it will evaluate to something like
$.nested.structure.ADC8AAFC-D564-40D1-9724-7680D3C010C2.name: foobar
which _may_ be treated as an arithemtic expression by the json parser.
The test creator would need to do:
$.nested.structure["$RESPONSE['$.id']"].name: foobar
or variants thereof to get a useful expression.
Docs to follow if this is considered an okay way to implement the
feature.
Fixes #170
Change the handling in response_replacer and jsonhandler.replace so
that numerals are preserved. The primary change here is that replace
does not always return a str: it returns the found data. The
remaining changes cascade from this fix.
* If the 'message' being replaced is fully the regex, don't do
regex-based replacement (which requires a string). Instead return
the new value.
* Do template replacing on test data _before_ dumping it to a string.
This allows the small 'message' replacements described in the previous
point.
* Assure we can recurse through lists, not just dicts, when
replacing in data structures.
In the pull request at #206 there is a different mode of coercing
and preserving data types operates by adding a coerce method to
content handlers. While inspecting this it seemed like it might be
too complex, so this alternate option has been explored. This might
work, or perhaps a hybrid of the two.
The coerce.yaml file from #206 was copied here and adapted to fit
the different assumptions that are made in this change. Some of the
assumptions in 206 don't map well to YAML handling of literal data
formats.
Add optional `test` param for custom content handlers
This resolves #211.
`multipart/form-data` transmission requires modifying the `Content-Type`
header with a `boundary` section identifier, and as such a custom
content handler would not have previously had access to the running test
case it was working on. This should hopefully be useful for other custom
content handlers in the future.
This overcomes a long standing bug in the testr tool: it doesn't
specify a dbm type to use, using a default which changes from
one python version to another (depending on build settings). It
is also unable to identify what was already used (which would be a
normal thing to do) and use that. Instead it just fails awkwardly.
The workaround is to remove the times.db file before each run
because it is not relevant.
Each of the response and content handlers has an expected type that
the test data should be in (in the YAML file). For example
response_strings should be a list. These changes add some
enforcement of those types, without which it is possible to (rarely)
get false positives when using a str value where a list value is
expected and the assertions using that value use 'in'.
Fixes #209
Since we know that count must always be an int and delay can be
either and int or float, the yaml data is passed through
replace_template and then cast to int or float explicitly.
This is different previous solution which tries to rely on the
casting happening in replace_template. With the new coerce style of
numbers-in-data handling being developed in #206 that strategy won't
work. In any case, this way is more contained and explicit.
Fixes #150
The comment explains how this is a pragmatic fix that could be done
in a more robust way by changing data structures to distinguish
between server and gabbi test generated urls.
If a reponse includes an absolute link, re-using that link as the
url in a subsequent test when a prefix is being used in the tests
can result in a 404 because the prefix will be duplicated.
This change ensures that the prefix will not be duplicated if the
URL already begins with the prefix.
Fixes #165
While trying to figure out some tests for the fix in #203 it became
clear that the handling of encoded-on-disk-but-textual content
when using <@ to load content was not sufficiently robust in both
python 2 and 3.
We want to load content to become unicode so that it is used
internally correctly, for example in the replace_template handling
used for substitutions, but when an HTTP request is made the content
must become whatever the versioned equivalent of bytes is.
Therefore the code has been changed to load non_binary content and
decode it to six.text_type, do replace_template on it, and then turn
the body, if it is not already bytes, into bytes.
This is verified by adjusting data.json to have some utf8 in it and
adding and loading a utf8.txt example file.
Change the reporter used by gabbi-run so that it provides a more
complete test name in its output, using the id() of the test. This
will include the name of the yaml file.
Also track failures per input file and provide a summary of which
files contained failures (if any) at the end.
Both of these changes were done in the simplest way possible so
there is probably considerably room for improvement now that the
concept has been proven.
Fixes #181
Some test runners will swallow exceptions and tracebacks that happen
during the start_fixture method of a Fixture. This can lead to a
great deal of confusion when creating new fixtures. This change
ensures that the traceback is made available by forcing an error to
be reported on the first test in the test suite using the fixture.
Additional detail can be found in the comment associated with the
change.
The added test_suite file tests this change by creating a suite with
two tests that use a fixture that will raise an exception in
start_fixture. The test confirms that the right test is errored,
that both are skipped and that the traceback includes expected
information.
Fixes #199
Running just one dynamically generated test can be a bit confusing
so some advice is added to the faq. At some point this should
probably be intergrated into the mainline docs.
If a request is content-type application/json, but the body is
empty, a request to verbosely print the body of the test will
cause a JSONdecoder error.
The addition of the gnocchi tox job means there is a git repo in there
after that tox job has run, which has files that will not delete without
a force.
