From 0477320662e448b21bd1855be0480e3327065bc5 Mon Sep 17 00:00:00 2001 From: Tim Buckley Date: Wed, 15 Jun 2016 10:26:33 -0600 Subject: [PATCH] Add normalization documentation to README This adds details about test name normalization to the README, along with additional documentation on potential output differences and other caveats involved with this plugin's output. Change-Id: Id2718ead4a572bd33796706c8d5c04bd671c19c1 --- README.md | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index c0af9ed..fb009a3 100644 --- a/README.md +++ b/README.md @@ -41,10 +41,64 @@ module.exports = function(config) { subunitReporter: { outputFile: 'karma.subunit', - tags: [], // tag strings to append to all tests - slug: false, // convert whitespace to '_' - separator: '.' // separator for suite components + test name + tags: [], // tag strings to append to all tests + slug: false, // if true, convert whitespace to '_' + separator: '.', // separator for suite components + test name + normalize: null // an optional normalization function, see below } }); }; ``` + +### Normalizing Test Names + +Given that JavaScript's test name formatting conventions are dissimilar to those +in other subunit-supported languages (particularly, Python) it may be desirable +to convert tokens to something that more closely resembles the other languages +in your environment. Additionally, tooling may require that test names be +reasonably well-formed for parsing and categorization purposes. + +In all cases, test name tokens are gathered from karma's `suite` field along +with the test's description. For Jasmine tests, `suite` is an ordered list of +enclosing `describe()` strings for each test. The list of tokens (i.e. suites +plus description) are then passed to some mapping function. + +If not otherwise configured, this mapping function simply returns the list +of tokens unaltered; however, it can also be configured to perform simple +conversions from Jasmine-style test descriptions to simple, dot-separated +Python-ish test names. If configured with `slug: true`, spaces will be collapsed +and converted to `_`, which yields decent results in some cases. + +If more advanced conversion is needed, a custom mapping function can be provided +using the `normalize` parameter, which accepts a list of tokens and returns a +modified list of tokens. + +As a final step, mapped tokens are then joined using the `separator` parameter +(`.` by default). + +## Tags + +The plugin automatically appends certain tags to each test: the browser ID +(`browser-*`) and the spec ID (`spec-*`). If multiple browsers are tested +concurrently these tags can be used to match results across browsers. + +Constant tags to apply to all tests can be also be specified using the `tags` +config option. Each string in the list is unconditionally appended to all tests, +and can be used as a potential compatibility measure (for example, to support +tools that assume a `worker-*` tag should exist). + +## Output Differences + +While in theory this plugin's output should work correctly with all +subunit-compatible tooling, in practice it may violate some assumptions that +can be made safely in other languages. + +For one, tests only run in a single worker thread (and unless configured, no +`worker-#` tag is present). Additionally, within the single worker, tests are +(at least potentially) run asynchronously and may have overlapping execution +timelines. + +Some other minor differences also exist. Currently file attachments aren't +supported, so log output, error messages, and tracebacks won't be included in +the subunit output. There's also no 'setUpClass' or other events that might be +logged in Python tests.