Add to stackviz-export and stackviz-front docs
Adds additional information to the stackviz-export and stackviz-front technical documents. Info about `stackviz-export` output files is now included in the corresponding document. Directive and service descriptions were added to stackviz-front, alongside inline comments in the appropriate .js files. Due to the unique structure of GitHub's RST renderer, includes will no longer be used in the main README. Change-Id: Iaebf1f1c3b5e4cbb4ea5e262f85672bc082bbe2f
This commit is contained in:
Austin Clark
parent
d096ef67b7
commit
e3af3b2bd3
@ -1 +1,19 @@
|
||||
.. include:: ./doc/source/contributing.rst
|
||||
Contributing
|
||||
============
|
||||
If you would like to contribute to the development of OpenStack, you must
|
||||
follow the steps in this page:
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html
|
||||
|
||||
If you already have a good understanding of how the system works and your
|
||||
OpenStack accounts are set up, you can skip to the development workflow
|
||||
section of this documentation to learn how changes to OpenStack should be
|
||||
submitted for review via the Gerrit tool:
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html#development-workflow
|
||||
|
||||
Pull requests submitted through GitHub will be ignored.
|
||||
|
||||
Bugs should be filed on Launchpad, not GitHub:
|
||||
|
||||
https://bugs.launchpad.net/stackviz
|
||||
|
||||
91
README.rst
91
README.rst
@ -1 +1,90 @@
|
||||
.. include:: ./doc/source/readme.rst
|
||||
========
|
||||
StackViz
|
||||
========
|
||||
A visualization utility to help analyze the performance of DevStack setup and
|
||||
Tempest executions. This repository can be cloned and built to use Stackviz
|
||||
with local run data. Stackviz is currently in the process of being implemented
|
||||
upstream (see Roadmap and Planning). To use Stackviz with upstream gate runs,
|
||||
please see the server deployment project at:
|
||||
|
||||
https://github.com/timothyb89/stackviz-deployer
|
||||
|
||||
Installation
|
||||
============
|
||||
Installation - Frontend
|
||||
-----------------------
|
||||
Installation of the frontend requires Node.js and Gulp. On Ubuntu::
|
||||
sudo apt-get install nodejs npm nodejs-legacy
|
||||
sudo npm install -g gulp
|
||||
|
||||
Then, install the Node modules by running, from the project directory::
|
||||
|
||||
npm install
|
||||
|
||||
Installation - Processing
|
||||
-------------------------
|
||||
The data processor is a small Python module located in the same source tree. To
|
||||
install, run::
|
||||
|
||||
sudo pip install .
|
||||
|
||||
Usage
|
||||
========
|
||||
Usage - Development
|
||||
-------------------
|
||||
A development server can be run as follows::
|
||||
|
||||
gulp dev
|
||||
|
||||
This will open a web browser and reload code automatically as it changes on the
|
||||
filesystem.
|
||||
|
||||
If you have subunit and dstat logs, you can create a config.json to display
|
||||
your runs::
|
||||
|
||||
stackviz-export -f <path/to/subunit> --dstat <path/to/dstat> app/data/
|
||||
|
||||
During :code:`gulp dev`, files written to :code:`app/data/` will be
|
||||
automatically synchronized with the browser. Note that these files will *not* be
|
||||
copied to :code:`build/` during :code:`gulp prod`, but you can copy them
|
||||
manually using :code:`gulp data`.
|
||||
|
||||
Usage - Production
|
||||
------------------
|
||||
The production application can be build using::
|
||||
|
||||
gulp prod
|
||||
|
||||
The result will be written to :code:`./build` and should be appropriate for
|
||||
distribution. Note that all files are not required:
|
||||
|
||||
- Directory structure (:code:`js/`, :code:`css/`, :code:`fonts/`,
|
||||
:code:`images/`): required.
|
||||
- Static resources (:code:`fonts/`, :code:`images/`): required.
|
||||
- Core files (:code:`index.html`, :code:`js/main.js`, :code:`css/main.css`):
|
||||
required unless gzipped versions are used.
|
||||
- Gzipped versions of core files (:code:`*.gz`): not required, but preferred.
|
||||
Use instead of plain core files to save on disk usage and bandwidth.
|
||||
- Source maps (:code:`js/main.js.map`, :code:`js/main.js.map.gz`): only required
|
||||
for debugging purposes.
|
||||
|
||||
Data should be written to :code:`build/data/` using :code:`stackviz-export` like
|
||||
above. Note that the static production code generated above is portable, and can
|
||||
be generated anywhere and copied to another host to be combined with exported
|
||||
data.
|
||||
|
||||
Testing
|
||||
=======
|
||||
* Python tests: :code:`tox -epy27`
|
||||
* JavaScript unit tests: :code:`gulp unit`
|
||||
* JavaScript E2E tests: :code:`gulp e2e`
|
||||
|
||||
Manuals & Developer Docs
|
||||
========================
|
||||
For more detailed information on how Stackviz works, please see the manuals
|
||||
located at doc/source/man/
|
||||
|
||||
Roadmap and Planning
|
||||
====================
|
||||
- Planning: https://etherpad.openstack.org/p/stackviz
|
||||
- Gate integration planning: https://etherpad.openstack.org/p/BKgWlKIjgQ
|
||||
|
||||
@ -6,6 +6,14 @@ var controllersModule = require('./_index');
|
||||
* @ngInject
|
||||
*/
|
||||
var TestDetailsCtrl =
|
||||
/**
|
||||
* Responsible for making three calls to the dataset service. First, the
|
||||
* dataset corresponding to the given int id is loaded, then the raw and details
|
||||
* JSON files are loaded and placed into state variables. Also note that a copy
|
||||
* of the details JSON is kept in `originalDetails` so that information is not
|
||||
* lost when parsing. Progress of the dataset service calls is recorded and
|
||||
* displayed in a progress bar on `test-details.html`.
|
||||
*/
|
||||
function($scope, $location, $stateParams, $log, datasetService, progressService) {
|
||||
var vm = this;
|
||||
vm.datasetId = $stateParams.datasetId;
|
||||
@ -43,7 +51,18 @@ function($scope, $location, $stateParams, $log, datasetService, progressService)
|
||||
progressService.done();
|
||||
});
|
||||
|
||||
vm.parsePythonLogging = function(showINFO, showDEBUG, showWARNING, showERROR) {
|
||||
vm.parsePythonLogging =
|
||||
/**
|
||||
* This function changes the `itemDetails.pythonlogging` variable to only
|
||||
* show lines with the log levels specified by the four boolean parameters.
|
||||
* EX: If the `showINFO` parameter is set to true, `itemDetails.pythonlogging`
|
||||
* will display lines that contain the text `INFO`.
|
||||
* @param {boolean} showINFO
|
||||
* @param {boolean} showDEBUG
|
||||
* @param {boolean} showWARNING
|
||||
* @param {boolean} showERROR
|
||||
*/
|
||||
function(showINFO, showDEBUG, showWARNING, showERROR) {
|
||||
if (vm.originalDetails && vm.originalDetails.pythonlogging) {
|
||||
var log = vm.originalDetails.pythonlogging;
|
||||
var ret = [];
|
||||
|
||||
@ -10,7 +10,13 @@ function tempestSummary() {
|
||||
/**
|
||||
* @ngInject
|
||||
*/
|
||||
var controller = function($scope, $attrs, datasetService) {
|
||||
var controller =
|
||||
/**
|
||||
* Responsible for getting the basic run summary stats via the dataset service.
|
||||
* Also calculates the duration of the run - `timeDiff` - by subtracting the
|
||||
* run's start and end timestamps.
|
||||
*/
|
||||
function($scope, $attrs, datasetService) {
|
||||
$scope.$watch('dataset', function(dataset) {
|
||||
var stats = dataset.stats;
|
||||
$scope.stats = stats;
|
||||
|
||||
@ -10,7 +10,14 @@ function testDetailsSearch() {
|
||||
/**
|
||||
* @ngInject
|
||||
*/
|
||||
var controller = function($scope, $element) {
|
||||
var controller =
|
||||
/**
|
||||
* Responsible for calling the `parsePythonLogging` filter function in
|
||||
* `TestDetailsController` when the log level buttons change state. The
|
||||
* `filter` function is passed from `test-details` to `test-details-search`
|
||||
* when the directive is initially instantiated.
|
||||
*/
|
||||
function($scope, $element) {
|
||||
var self = this;
|
||||
this.open = false;
|
||||
this.showINFO = true;
|
||||
@ -18,10 +25,12 @@ function testDetailsSearch() {
|
||||
this.showWARNING = true;
|
||||
this.showERROR = true;
|
||||
|
||||
// Wrapper for parent controller's filter function.
|
||||
var update = function() {
|
||||
$scope.filter(self.showINFO, self.showDEBUG, self.showWARNING, self.showERROR);
|
||||
};
|
||||
|
||||
// Watchers to signal update function upon button state change.
|
||||
$scope.$watch(function() { return self.query; }, update);
|
||||
$scope.$watch(function() { return self.showINFO; }, update);
|
||||
$scope.$watch(function() { return self.showDEBUG; }, update);
|
||||
|
||||
@ -1,19 +1 @@
|
||||
Contributing
|
||||
============
|
||||
If you would like to contribute to the development of OpenStack, you must
|
||||
follow the steps in this page:
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html
|
||||
|
||||
If you already have a good understanding of how the system works and your
|
||||
OpenStack accounts are set up, you can skip to the development workflow
|
||||
section of this documentation to learn how changes to OpenStack should be
|
||||
submitted for review via the Gerrit tool:
|
||||
|
||||
http://docs.openstack.org/infra/manual/developers.html#development-workflow
|
||||
|
||||
Pull requests submitted through GitHub will be ignored.
|
||||
|
||||
Bugs should be filed on Launchpad, not GitHub:
|
||||
|
||||
https://bugs.launchpad.net/stackviz
|
||||
.. include:: ../../CONTRIBUTING.rst
|
||||
|
||||
@ -14,7 +14,6 @@ is as simple as::
|
||||
|
||||
Usage
|
||||
=====
|
||||
|
||||
:code:`stackviz-export [options] <DEST>`
|
||||
|
||||
Where DEST is the output directory of the module. If DEST does not exist, a new
|
||||
@ -45,3 +44,40 @@ Additional options:
|
||||
|
||||
**-z --gzip**
|
||||
Enables gzip compression for data files.
|
||||
|
||||
Output
|
||||
======
|
||||
:code:`stackviz-export` outputs the following files to the destination directory.
|
||||
Note that <source> in the details, raw, and tree logs refer to what stream
|
||||
source the
|
||||
|
||||
**config.json**
|
||||
Contains all the basic information about a dataset that the front-end needs.
|
||||
There will be one `tempest` entry for every dataset that was generated in
|
||||
:code:`stackviz-export`. Each `tempest` entry has general information about
|
||||
each run, as well as the locations of the details, raw, and tree JSON files.
|
||||
|
||||
**dstat_log.csv**
|
||||
This file will only be present if a dstat log was used in the corresponding
|
||||
:code:`stackviz-export` run. Has a wide variety of system statistics
|
||||
including CPU, memory, and disk utilization. This information is displayed
|
||||
on the timeline graph.
|
||||
|
||||
**tempest_<source>_<id>_details.json**
|
||||
The details log contains timestamp and status information for tests in
|
||||
addition to all of the logs associated with the test (e.g. tracebacks).
|
||||
These artifacts are displayed in the test details page.
|
||||
|
||||
**tempest_<source>_<id>_raw.json**
|
||||
Contains nearly all information available about tests:
|
||||
- :code:`status`: pass, fail, or skipped
|
||||
- :code:`name`: full name of test
|
||||
- :code:`tags`: which worker the test was run on
|
||||
- :code:`details`: empty, this info is available in the details JSON
|
||||
- :code:`duration`: how long the test took, in seconds
|
||||
- :code:`timestamps`: timestamps at test begin and test end
|
||||
This file is used in the timeline and test details page.
|
||||
|
||||
**tempest_<source>_<id>_tree.json**
|
||||
Stores test names in a hierarchy for display on the deprecated
|
||||
sunburst diagram. Not currently used by any page in Stackviz.
|
||||
|
||||
@ -49,7 +49,7 @@ Test Details
|
||||
------------
|
||||
:Path: <host>/#/<run>/test-details/<test>/
|
||||
:Directive: :code:`./app/views/test-details.html`
|
||||
:Controller: :code:`./app/js/controllers/home.js`
|
||||
:Controller: :code:`./app/js/controllers/test-details.js`
|
||||
|
||||
The test details page consists of one panel that displays various log info
|
||||
from one test. The first tab contains summary information similar to the info
|
||||
@ -80,17 +80,62 @@ additional information to aid debugging. The most common tabs include:
|
||||
When enough information has been gleaned from more detailed logs, the button
|
||||
in the panel filter can be used to quickly navigate back to the timeline page.
|
||||
|
||||
Controllers
|
||||
===========
|
||||
|
||||
Directives
|
||||
==========
|
||||
|
||||
Filters
|
||||
=======
|
||||
**tempestSummary**
|
||||
The tempest summary directive consists of one panel that shows stats for
|
||||
one run: Duration of the run, number of tests run, number of tests skipped,
|
||||
and number of tests failed. :code:`timeDiff` (the duration of the run) is
|
||||
calculated from the start and end timestamps contained in summary data.
|
||||
All other fields are populated directly from the summary data, via a call
|
||||
to the dataset service.
|
||||
|
||||
**testDetailsSearch**
|
||||
:code:`testDetailsSearch` uses two HTML pages to search the test details
|
||||
page: :code:`test-details-search-popover.html` and :code:`test-details-search.html`.
|
||||
The popover contains the filter levels for the :code:`pythonlogging` tab:
|
||||
INFO, DEBUG, WARNING, ERROR. This directive is used as the template for
|
||||
:code:`test-details-search`, per AngularJS popover convention. The function
|
||||
used to parse the logs, :code:`parsePythonLogging` actually lives in the
|
||||
controller for testDetailsSearch, and is passed through both the prior
|
||||
directives' scopes. This function reads in the :code:`pythonLogging` tab
|
||||
as one text object, then splits it by :code:`\n` to create an array of
|
||||
lines. Each line is then added back to the :code:`pythonLogging` tab if it
|
||||
contains the specific log level somewhere in the line.
|
||||
|
||||
**timeline**
|
||||
The timeline directive is a container for the actual timeline components,
|
||||
detailed below.
|
||||
|
||||
**timelineDetails**
|
||||
|
||||
**timelineDstat**
|
||||
|
||||
**timelineOverview**
|
||||
|
||||
**timelineSearch**
|
||||
|
||||
**timelineViewport**
|
||||
|
||||
|
||||
Services
|
||||
========
|
||||
|
||||
Util
|
||||
====
|
||||
**dataset**
|
||||
The dataset service is an API that provides the front-end with all of the
|
||||
data generated by :code:`stackviz-export`. All data processed by
|
||||
:code:`stackviz-export` ends up in the `./app/data/` directory to be called
|
||||
by dataset service with :code:`$http` and :code:`$q` directives. Below is
|
||||
the list of calls:
|
||||
- :code:`list` returns `config.json` using GET.
|
||||
- :code:`get(id)` calls :code:`list`, then iterates through all the
|
||||
available datasets for the requested id number. Rejects if not found.
|
||||
- :code:`raw(dataset)` returns `<dataset>_raw.json` file using GET.
|
||||
- :code:`details(dataset)` returns `<dataset>_details.json` file using GET.
|
||||
- :code:`tree(dataset)` returns `<dataset>_tree.json` file using GET.
|
||||
- :code:`dstat(dataset)` returns `dstat_log.csv` file using GET, if available.
|
||||
|
||||
**progress**
|
||||
A wrapper for :code:`nprogress`, a progress bar library. Used in the timeline
|
||||
and test details pages to show progress in loading datasets.
|
||||
|
||||
@ -1,30 +1 @@
|
||||
========
|
||||
StackViz
|
||||
========
|
||||
A visualization utility to help analyze the performance of DevStack setup and
|
||||
Tempest executions. This repository can be cloned and built to use Stackviz
|
||||
with local run data. Stackviz is currently in the process of being implemented
|
||||
upstream (see Roadmap and Planning). To use Stackviz with upstream gate runs,
|
||||
please see the server deployment project at:
|
||||
|
||||
https://github.com/timothyb89/stackviz-deployer
|
||||
|
||||
.. include:: ./installation.rst
|
||||
|
||||
.. include:: ./usage.rst
|
||||
|
||||
Testing
|
||||
=======
|
||||
* Python tests: :code:`tox -epy27`
|
||||
* JavaScript unit tests: :code:`gulp unit`
|
||||
* JavaScript E2E tests: :code:`gulp e2e`
|
||||
|
||||
Manuals & Developer Docs
|
||||
========================
|
||||
For more detailed information on how Stackviz works, please see the manuals
|
||||
located at doc/source/man/
|
||||
|
||||
Roadmap and Planning
|
||||
====================
|
||||
- Planning: https://etherpad.openstack.org/p/stackviz
|
||||
- Gate integration planning: https://etherpad.openstack.org/p/BKgWlKIjgQ
|
||||
.. include:: ../../README.rst
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user