Performance and debugging visualization for DevStack and Tempest
Go to file
Ian Wienand 02dc3ab7f5
Have stackviz output a usable python sdist
It seems that on a CI node, we just want to have approximately "pip
install stackviz" and have it "just work".

This makes "npm run prod" produce a .tar.gz appropriate for this.

Firstly, I have moved the gulp output to "gulp-build" rather than
"build" because "build" is generic and can get distutils confused.

Second, after building with gulp, we tack on a "setup.py sdist" call
to generate the final package.

Thirdly, the MANIFEST.in is updated to prune out unneeded files from
the final distribution.  pbr is putting everything in git into the
manifest, but we can trim all all the nodejs source stuff.  The
generated files in gulp-build are added to the manifest (since they're
generated, and not in git).

Finally, these are added via data-files as "share/stackviz-html".  This
is essentially standalone and can be moved, or symlinked to as required.

The end result is that the .tar.gz created is installable on nodes an
contains everything required to run stackviz.

Documentation is updated.

Change-Id: I64cbc7e90bd2610733c9d5c6275cce99f2571883
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-31 09:14:10 -04:00
app Fix timeline worker sorting 2016-07-18 11:23:03 -06:00
doc/source Correct stream-file argument 2016-09-02 10:53:11 +10:00
gulp Have stackviz output a usable python sdist 2017-05-31 09:14:10 -04:00
stackviz Adapt to python 3 2017-04-11 09:37:46 -06:00
test Add new configuration file format 2016-04-21 16:21:52 -06:00
.coveragerc Update .coveragerc after the removal of openstack directory 2016-11-03 19:33:33 +05:30
.eslintignore Switch to eslint. 2015-11-18 15:36:36 -07:00
.eslintrc.json Fix phantomjs package error and update eslint-config-openstack 2016-03-10 13:34:03 -07:00
.gitignore Have stackviz output a usable python sdist 2017-05-31 09:14:10 -04:00
.gitreview Add .gitreview file 2015-09-14 15:59:30 -06:00
.testr.conf Add more test cookiecutter template items 2015-08-03 11:26:08 -06:00
babel.cfg Add stub unit tests from cookiecutter 2015-08-03 10:37:55 -06:00
bindep.txt Initial commit of bindep.txt 2017-04-13 10:58:54 -04:00
CONTRIBUTING.rst Add to stackviz-export and stackviz-front docs 2016-04-14 10:18:28 -06:00
gulpfile.js Add initial Angular boilerplate files. 2015-09-25 16:26:44 -06:00
HACKING.rst Add .gitignore entries, HACKING.rst, and __init__.py contents from cookiecutter 2015-08-03 10:01:01 -06:00
LICENSE Add some missing base files from cookiecutter (LICENSE, manifest, setuptools config) 2015-08-03 09:11:51 -06:00
MANIFEST.in Have stackviz output a usable python sdist 2017-05-31 09:14:10 -04:00
package.json Have stackviz output a usable python sdist 2017-05-31 09:14:10 -04:00
README.rst Have stackviz output a usable python sdist 2017-05-31 09:14:10 -04:00
requirements.txt Drop mysql-python 2017-04-06 21:59:58 +00:00
setup.cfg Have stackviz output a usable python sdist 2017-05-31 09:14:10 -04:00
setup.py Add some missing base files from cookiecutter (LICENSE, manifest, setuptools config) 2015-08-03 09:11:51 -06:00
test-requirements.txt Fix docs build 2017-02-18 14:05:35 -07:00
tox.ini Drop mysql-python 2017-04-06 21:59:58 +00:00

Team and repository tags

image

StackViz

A visualization utility to help analyze the performance of DevStack setup and Tempest executions. This repository can be cloned and built to use Stackviz with local run data. Stackviz is currently in the process of being implemented upstream (see Roadmap and Planning). To use Stackviz with upstream gate runs, please see the server deployment project at:

https://github.com/timothyb89/stackviz-deployer

Installation

Installation - Frontend

Installation of the frontend requires Node.js and Gulp. On Ubuntu::

sudo apt-get install nodejs npm nodejs-legacy sudo npm install -g gulp

Then, install the Node modules by running, from the project directory:

npm install

Installation - Processing

The data processor is a small Python module located in the same source tree. To install, run:

sudo pip install .

Usage

Usage - Development

A development server can be run as follows:

gulp dev

This will open a web browser and reload code automatically as it changes on the filesystem.

If you have subunit and dstat logs, you can create a config.json to display your runs:

stackviz-export -f <path/to/subunit> --dstat <path/to/dstat> app/data/

During gulp dev, files written to app/data/ will be automatically synchronized with the browser. Note that these files will not be copied to build/ during gulp prod, but you can copy them manually using gulp data.

Usage - Production

The production application can be build using:

gulp prod

This will automatically build portable html/javascript and python utilities into dist/stackviz-VERSION.tar.gz.

You should probably install this into a virtualenv on the target system:

virtualenv stackviz
./virtualenv/bin/pip install /path/to/stackviz-VERSION.tar.gz
# to run stackviz export
./virtualenv/bin/stackviz-export

Note the required html will be placed in virtualenv/share/stackviz-html as a data-file (or elsewhere, if installed as a system package; this may vary on distributions). This can be moved as required. Note that all files in there are not required:

  • Directory structure (js/, css/, fonts/, images/): required.
  • Static resources (fonts/, images/): required.
  • Core files (index.html, js/main.js, css/main.css): required unless gzipped versions are used.
  • Gzipped versions of core files (*.gz): not required, but preferred. Use instead of plain core files to save on disk usage and bandwidth.
  • Source maps (js/main.js.map, js/main.js.map.gz): only required for debugging purposes.

Data should be written to stackviz-html/data/ using stackviz-export like above.

Testing

  • Python tests: tox -epy27
  • JavaScript unit tests: gulp unit
  • JavaScript E2E tests: gulp e2e

Manuals & Developer Docs

For more detailed information on how Stackviz works, please see the manuals located at doc/source/man/

Roadmap and Planning