Python Data-Processing Module (stackviz-export)
The main purpose of
stackviz-export is to parse subunit and dstat logs in order to generate configuration files for the AngularJS front-end.
Once Stackviz has been cloned into a suitable directory, setting up the module is as simple as:
sudo pip install .
stackviz-export [options] <DEST>
Where DEST is the output directory of the module. If DEST does not exist, a new directory will be created. One of the following input options must be chosen:
- -f, --stream-file FILE
Specifies a subunit stream file to be used with the exporter. This argument can be used multiple times to specify additional subunit files.
- -i, --stdin
Instructs stackviz-export to read a subunit stream from stdin.
- -r, --repository REPOSITORY
.testrepositoryto read subunit streams from. This argument can be used multiple times to specify additional repositories.
Stackviz also visualizes machine utilization statistics using dstat. To attach a dstat.csv log to the subunit output, specify the following option:
- --dstat FILE
Specifies a csv-formatted dstat log file that corresponds with the provided subunit stream file.
- -h --help
Print help message.
- -z --gzip
Enables gzip compression for data files.
stackviz-export outputs the following files to the destination directory. Note that <source> in the details, raw, and tree logs refer to what stream source the
Contains all the basic information about a dataset that the front-end needs. There will be one tempest entry for every dataset that was generated in
stackviz-export. Each tempest entry has general information about each run, as well as the locations of the details, raw, and tree JSON files.
This file will only be present if a dstat log was used in the corresponding
stackviz-exportrun. Has a wide variety of system statistics including CPU, memory, and disk utilization. This information is displayed on the timeline graph.
The details log contains timestamp and status information for tests in addition to all of the logs associated with the test (e.g. tracebacks). These artifacts are displayed in the test details page.
- Contains nearly all information available about tests:
status: pass, fail, or skipped
name: full name of test
tags: which worker the test was run on
details: empty, this info is available in the details JSON
duration: how long the test took, in seconds
timestamps: timestamps at test begin and test end
This file is used in the timeline and test details page.
Stores test names in a hierarchy for display on the deprecated sunburst diagram. Not currently used by any page in Stackviz.