Refactor tox & update docs
- Updated tox envlist, so just running `tox` from the CLI will now run all voting gate tests - Reduce duplicated definitions and commands - Remove any reliance on run_tests within tox - Removes all doc references to run_tests.sh, and replaces them with their tox equivalent. Where necessary, language around the tox commands has been altered or extended so that it makes sense and is consistent with other parts of the docs. Also adds a new "Test Environment" list to the docs, so that newcomers do not have to piece together CLI commands and their cryptic extensions from tox.ini - Move the inline shell scripting to its own file. Also fixes a bug when passing args, since the logic assumed you were attempting a subset test run (try `tox -e py27 -- --pdb` on master to compare) - Moved translation tooling from run_tests to manage.py, w/ help text and arg restrictions. This is much more flexible so that plugins can use it without having to copy commands, but still defaults to exactly the same parameters/behaviour from run_tests. Docs updated appropriately. - Removed npm/karma strange reliance on either .venv or tox/py27. Now it only uses tox/npm. Change-Id: I883f885bd424955d39ddcfde5ba396a88cfc041e Implements: blueprint enhance-tox Closes-Bug: 1638672
This commit is contained in:
parent
2394397bfd
commit
36d1d1ac68
1
.gitignore
vendored
1
.gitignore
vendored
@ -1,5 +1,6 @@
|
||||
*.egg*
|
||||
*.mo
|
||||
*.pot
|
||||
*.pyc
|
||||
*.sw?
|
||||
*.sqlite3
|
||||
|
@ -89,9 +89,8 @@ After You Write Your Patch
|
||||
|
||||
Once you've made your changes, there are a few things to do:
|
||||
|
||||
* Make sure the unit tests pass: ``./run_tests.sh`` for Python, and ``npm run test`` for JS.
|
||||
* Make sure the linting tasks pass: ``./run_tests.sh --pep8`` for Python, and ``npm run lint`` for JS.
|
||||
* Make sure your code is ready for translation: ``./run_tests.sh --pseudo de`` See :ref:`pseudo_translation` for more information.
|
||||
* Make sure the unit tests and linting tasks pass by running ``tox``
|
||||
* Make sure your code is ready for translation: See :ref:`pseudo_translation`.
|
||||
* Make sure your code is up-to-date with the latest master: ``git pull --rebase``
|
||||
* Finally, run ``git review`` to upload your changes to Gerrit for review.
|
||||
|
||||
@ -132,7 +131,7 @@ Python
|
||||
------
|
||||
|
||||
We follow PEP8_ for all our Python code, and use ``pep8.py`` (available
|
||||
via the shortcut ``./run_tests.sh --pep8``) to validate that our code
|
||||
via the shortcut ``tox -e pep8``) to validate that our code
|
||||
meets proper Python style guidelines.
|
||||
|
||||
.. _PEP8: http://www.python.org/dev/peps/pep-0008/
|
||||
|
@ -22,20 +22,10 @@ On RPM-based distributions (e.g., Fedora/RHEL/CentOS/Scientific Linux)::
|
||||
Setup
|
||||
=====
|
||||
|
||||
To setup a Horizon development environment simply clone the Horizon git
|
||||
repository from http://github.com/openstack/horizon and execute the
|
||||
``run_tests.sh`` script from the root folder (see :doc:`ref/run_tests`)::
|
||||
To begin setting up a Horizon development environment simply clone the Horizon
|
||||
git repository from https://git.openstack.org/cgit/openstack/horizon.::
|
||||
|
||||
> git clone https://github.com/openstack/horizon.git
|
||||
> cd horizon
|
||||
> ./run_tests.sh
|
||||
|
||||
.. note::
|
||||
|
||||
Running ``run_tests.sh`` will build a virtualenv, ``.venv``, where all the
|
||||
python dependencies for Horizon are installed and referenced. After the
|
||||
dependencies are installed, the unit test suites in the Horizon repo will be
|
||||
executed. There should be no errors from the tests.
|
||||
> git clone https://git.openstack.org/openstack/horizon
|
||||
|
||||
Next you will need to setup your Django application config by copying ``openstack_dashboard/local/local_settings.py.example`` to ``openstack_dashboard/local/local_settings.py``. To do this quickly you can use the following command::
|
||||
|
||||
@ -92,21 +82,21 @@ order to prevent Conflicts for future migrations::
|
||||
> mv openstack_dashboard/local/local_settings.diff openstack_dashboard/local/local_settings.diff.old
|
||||
> python manage.py migrate_settings --gendiff
|
||||
|
||||
To start the Horizon development server use ``run_tests.sh``::
|
||||
To start the Horizon development server use ``tox``::
|
||||
|
||||
> ./run_tests.sh --runserver localhost:9000
|
||||
> tox -e runserver
|
||||
|
||||
.. note::
|
||||
|
||||
The default port for runserver is 8000 which is already consumed by
|
||||
heat-api-cfn in DevStack. If not running in DevStack
|
||||
`./run_tests.sh --runserver` will start the test server at
|
||||
`http://localhost:8000`.
|
||||
heat-api-cfn in DevStack. If running in DevStack
|
||||
`tox -e runserver -- localhost:9000` will start the test server at
|
||||
`http://localhost:9000`.
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
The ``run_tests.sh`` script provides wrappers around ``manage.py``.
|
||||
The ``tox`` environments provide wrappers around ``manage.py``.
|
||||
For more information on manage.py which is a django, see
|
||||
`https://docs.djangoproject.com/en/dev/ref/django-admin/`
|
||||
|
||||
|
@ -2,6 +2,12 @@
|
||||
The ``run_tests.sh`` Script
|
||||
===========================
|
||||
|
||||
.. warning::
|
||||
|
||||
This script is deprecated as of Newton (11.0), and will be removed in
|
||||
Queens (13.0), in favor of tox. The tox docs can be found at
|
||||
https://tox.readthedocs.io/en/latest/
|
||||
|
||||
.. contents:: Contents:
|
||||
:local:
|
||||
|
||||
|
@ -10,34 +10,33 @@ Because Horizon is composed of both the ``horizon`` app and the
|
||||
tests. While they can be run individually without problem, there is an easier
|
||||
way:
|
||||
|
||||
Included at the root of the repository is the ``run_tests.sh`` script
|
||||
Included at the root of the repository is the ``tox.ini`` config
|
||||
which invokes both sets of tests, and optionally generates analyses on both
|
||||
components in the process. This script is what Jenkins uses to verify the
|
||||
components in the process. ``tox`` is what Jenkins uses to verify the
|
||||
stability of the project, so you should make sure you run it and it passes
|
||||
before you submit any pull requests/patches.
|
||||
|
||||
To run the tests::
|
||||
To run all tests::
|
||||
|
||||
$ ./run_tests.sh
|
||||
|
||||
It's also possible to :doc:`run a subset of unit tests<ref/run_tests>`.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:doc:`ref/run_tests`
|
||||
Full reference for the ``run_tests.sh`` script.
|
||||
$ tox
|
||||
|
||||
It's also possible to run a subset of the tests. Open ``tox.ini`` in the
|
||||
Horizon root directory to see a list of test environments. You can read more
|
||||
about tox in general at https://tox.readthedocs.io/en/latest/.
|
||||
|
||||
By default running the Selenium tests will open your Firefox browser (you have
|
||||
to install it first, else an error is raised), and you will be able to see the
|
||||
tests actions.
|
||||
tests actions::
|
||||
|
||||
$ tox -e selenium-headless
|
||||
|
||||
If you want to run the suite headless, without being able to see them (as they
|
||||
are ran on Jenkins), you can run the tests::
|
||||
|
||||
$ ./run_tests.sh --with-selenium --selenium-headless
|
||||
$ tox -e selenium-headless
|
||||
|
||||
Selenium will use a virtual display in this case, instead of your own. In order
|
||||
to run the tests this way you have to install the dependency `xvfb`, like
|
||||
to run the tests this way you have to install the dependency `xvfb`, like
|
||||
this::
|
||||
|
||||
$ sudo apt-get install xvfb
|
||||
@ -49,12 +48,90 @@ for a Debian OS flavour, or for Fedora/Red Hat flavours::
|
||||
If you can't run a virtual display, or would prefer not to, you can use the
|
||||
PhantomJS web driver instead::
|
||||
|
||||
$ ./run_tests.sh --with-selenium --selenium-phantomjs
|
||||
$ tox -e selenium-phantomjs
|
||||
|
||||
If you need to install PhantomJS, you may do so with `npm` like this::
|
||||
|
||||
$ npm -g install phantomjs
|
||||
|
||||
Alternatively, many distributions have system packages for phantomjs, or
|
||||
it can be downloaded from http://phantomjs.org/download.html.
|
||||
|
||||
tox Test Environments
|
||||
=====================
|
||||
|
||||
This is a list of test environments available to be executed by
|
||||
``tox -e <name>``.
|
||||
|
||||
pep8
|
||||
----
|
||||
|
||||
Runs pep8, which is a tool that checks Python code style. You can read more
|
||||
about pep8 at https://www.python.org/dev/peps/pep-0008/
|
||||
|
||||
py27dj18, py27dj19, py27dj110
|
||||
-----------------------------
|
||||
|
||||
Runs the Python unit tests against Django 1.8, Django 1.9 and Django 1.10
|
||||
respectively
|
||||
|
||||
All other dependencies are as defined by the upper-constraints file at
|
||||
https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt
|
||||
|
||||
You can run a subset of the tests by passing the test path as an argument to
|
||||
tox::
|
||||
|
||||
$ tox -e py27dj18 -- openstack_dashboard.dashboards.identity.users.tests
|
||||
|
||||
You can also pass other arguments. For example, to drop into a live debugger
|
||||
when a test fails you can use::
|
||||
|
||||
$ tox -e py27dj18 -- --pdb
|
||||
|
||||
py34
|
||||
----
|
||||
|
||||
Runs the Python unit tests with a Python 3.4 environment.
|
||||
|
||||
py35
|
||||
----
|
||||
|
||||
Runs the Python unit tests with a Python 3.5 environment.
|
||||
|
||||
releasenotes
|
||||
------------
|
||||
|
||||
Outputs Horizons release notes as HTML to ``releasenotes/build/html``.
|
||||
|
||||
Also takes an alternative builder as an optional argument, such as
|
||||
``tox -e docs -- <builder>``, which will output to
|
||||
``releasenotes/build/<builder>``. Available builders are listed at
|
||||
http://www.sphinx-doc.org/en/latest/builders.html
|
||||
|
||||
npm
|
||||
---
|
||||
|
||||
Installs the npm dependencies listed in ``package.json`` and runs the
|
||||
JavaScript tests. Can also take optional arguments, which will be executed
|
||||
as an npm script following the dependency install, instead of ``test``.
|
||||
|
||||
Example::
|
||||
|
||||
$ tox -e npm -- lintq
|
||||
|
||||
docs
|
||||
----
|
||||
|
||||
Outputs Horizons documentation as HTML to ``doc/build/html``.
|
||||
|
||||
Also takes an alternative builder as an optional argument, such as
|
||||
``tox -e docs -- <builder>``, which will output to ``doc/build/<builder>``.
|
||||
Available builders are listed at
|
||||
http://www.sphinx-doc.org/en/latest/builders.html
|
||||
|
||||
Example::
|
||||
|
||||
$ tox -e docs -- latexpdf
|
||||
|
||||
Writing tests
|
||||
=============
|
||||
|
@ -38,7 +38,7 @@ ESLint
|
||||
|
||||
ESLint is a tool for identifying and reporting on patterns in your JS code, and
|
||||
is part of the automated tests run by Jenkins. You can run ESLint from the
|
||||
horizon root directory with ``npm run lint``, or alternatively on a specific
|
||||
horizon root directory with ``tox -e npm -- lint``, or alternatively on a specific
|
||||
directory or file with ``eslint file.js``.
|
||||
|
||||
Horizon includes a `.eslintrc` in its root directory, that is used by the
|
||||
@ -217,10 +217,13 @@ Testing
|
||||
=======
|
||||
|
||||
1. Open <dev_server_ip:port>/jasmine in a browser. The development server can be run
|
||||
with``./run_tests.sh --runserver`` from the horizon root directory.
|
||||
2. ``npm run test`` from the horizon root directory.
|
||||
with ``tox -e runserver`` from the horizon root directory; by default, this will
|
||||
run the development server at ``http://localhost:8000``.
|
||||
2. ``tox -e npm`` from the horizon root directory.
|
||||
|
||||
The code linting job can be run with ``npm run lint``.
|
||||
The code linting job can be run with ``tox -e npm -- lint``. If there are many
|
||||
warnings, you can also use ``tox -e npm -- lintq`` to see only errors and
|
||||
ignore warnings.
|
||||
|
||||
For more detailed information, see :doc:`javascript_testing`.
|
||||
|
||||
|
@ -68,7 +68,7 @@ theme's ``_variables.scss``::
|
||||
@import "/themes/default/variables";
|
||||
|
||||
Once you have made your changes you must re-generate the static files with
|
||||
``./run_tests.sh -m collectstatic``.
|
||||
``tox -e manage -- collectstatic``.
|
||||
|
||||
By default, all of the themes configured by ``AVAILABLE_THEMES`` setting are
|
||||
collected by horizon during the `collectstatic` process. By default, the themes
|
||||
|
@ -39,7 +39,7 @@ Installation
|
||||
message catalogs::
|
||||
|
||||
$ sudo apt-get install gettext
|
||||
$ ./run_tests.sh --compilemessages
|
||||
$ tox -e manage -- compilemessages
|
||||
|
||||
This command compiles translation message catalogs within Python
|
||||
virtualenv named ``.venv``. After this step, you can remove
|
||||
|
@ -31,12 +31,13 @@ Running Tests
|
||||
Tests can be run in two ways:
|
||||
|
||||
1. Open <dev_server_ip:port>/jasmine in a browser. The development server can be
|
||||
run with ``./run_tests.sh --runserver`` from the horizon root directory.
|
||||
2. ``npm run test`` from the horizon root directory. This runs Karma,
|
||||
run with ``tox -e runserver`` from the horizon root directory.
|
||||
2. ``tox -e npm`` from the horizon root directory. This runs Karma,
|
||||
so it will run all the tests against PhantomJS and generate coverage
|
||||
reports.
|
||||
|
||||
The code linting job can be run with ``npm run lint``.
|
||||
The code linting job can be run with ``tox -e npm -- lint``, or
|
||||
``tox -e npm -- lintq`` to show errors, but not warnings.
|
||||
|
||||
Coverage Reports
|
||||
----------------
|
||||
@ -45,7 +46,7 @@ Our Karma setup includes a plugin to generate test coverage reports. When
|
||||
developing, be sure to check the coverage reports on the master branch and
|
||||
compare your development branch; this will help identify missing tests.
|
||||
|
||||
To generate coverage reports, run ``npm run test``. The coverage reports can be
|
||||
To generate coverage reports, run ``tox -e npm``. The coverage reports can be
|
||||
found at ``horizon/coverage-karma/`` (framework tests) and
|
||||
``openstack_dashboard/coverage-karma/`` (dashboard tests). Load
|
||||
``<browser>/index.html`` in a browser to view the reports.
|
||||
|
@ -46,13 +46,19 @@ translated. Lets break this up into steps we can follow:
|
||||
to locate them. Refer to the guide below on how to use translation and what
|
||||
these markers look like.
|
||||
|
||||
2. Once marked, we can then run ``./run_tests.sh --makemessages``, which
|
||||
2. Once marked, we can then run ``tox -e manage -- extract_messages``, which
|
||||
searches the codebase for these markers and extracts them into a Portable
|
||||
Object Template (POT) file. In horizon, we extract from both the ``horizon``
|
||||
folder and the ``openstack_dashboard`` folder. We use the AngularJS extractor
|
||||
for JavaScript and HTML files and the Django extractor for Python and Django
|
||||
templates; both extractors are Babel plugins.
|
||||
|
||||
3. To update the .po files, you can run ``tox -e manage -- update_catalog`` to
|
||||
update the .po file for every language, or you can specify a specific
|
||||
language to update like this: ``tox -e manage -- update_catalog de``. This
|
||||
is useful if you want to add a few extra translatabale strings for a
|
||||
downstream customisation.
|
||||
|
||||
.. Note ::
|
||||
|
||||
When pushing code upstream, the only requirement is to mark the strings
|
||||
@ -242,12 +248,12 @@ translations to validate that your code is ready for translation.
|
||||
Running the pseudo translation tool
|
||||
-----------------------------------
|
||||
|
||||
#. Make sure your English po file is up to date:
|
||||
``./run_tests.sh --makemessages``
|
||||
#. Make sure your .pot files are up to date:
|
||||
``tox -e manage -- extract_messages``
|
||||
#. Run the pseudo tool to create pseudo translations. For example, to replace
|
||||
the German translation with a pseudo translation:
|
||||
``./run_tests.sh --pseudo de``
|
||||
#. Compile the catalog: ``./run_tests.sh --compilemessages``
|
||||
``tox -e manage -- update_catalog de --pseudo``
|
||||
#. Compile the catalog: ``tox -e manage -- compilemessages``
|
||||
#. Run your development server.
|
||||
#. Log in and change to the language you pseudo translated.
|
||||
|
||||
|
@ -30,20 +30,19 @@ The quick version
|
||||
-----------------
|
||||
|
||||
Horizon provides a custom management command to create a typical base
|
||||
dashboard structure for you. Run the following commands at the same location
|
||||
where the ``run_tests.sh`` file resides. It generates most of the boilerplate
|
||||
code you need::
|
||||
dashboard structure for you. Run the following commands in your Horizon root
|
||||
directory. It generates most of the boilerplate code you need::
|
||||
|
||||
mkdir openstack_dashboard/dashboards/mydashboard
|
||||
$ mkdir openstack_dashboard/dashboards/mydashboard
|
||||
|
||||
./run_tests.sh -m startdash mydashboard \
|
||||
--target openstack_dashboard/dashboards/mydashboard
|
||||
$ tox -e manage -- startdash mydashboard \
|
||||
--target openstack_dashboard/dashboards/mydashboard
|
||||
|
||||
mkdir openstack_dashboard/dashboards/mydashboard/mypanel
|
||||
$ mkdir openstack_dashboard/dashboards/mydashboard/mypanel
|
||||
|
||||
./run_tests.sh -m startpanel mypanel \
|
||||
--dashboard=openstack_dashboard.dashboards.mydashboard \
|
||||
--target=openstack_dashboard/dashboards/mydashboard/mypanel
|
||||
$ tox -e manage -- startpanel mypanel \
|
||||
--dashboard=openstack_dashboard.dashboards.mydashboard \
|
||||
--target=openstack_dashboard/dashboards/mydashboard/mypanel
|
||||
|
||||
|
||||
You will notice that the directory ``mydashboard`` gets automatically
|
||||
@ -562,10 +561,9 @@ Run and check the dashboard
|
||||
|
||||
Everything is in place, now run ``Horizon`` on the different port::
|
||||
|
||||
./run_tests.sh --runserver 0.0.0.0:8877
|
||||
$ tox -e runserver -- 0:9000
|
||||
|
||||
|
||||
Go to ``http://<your server>:8877`` using a browser. After login as an admin
|
||||
Go to ``http://<your server>:9000`` using a browser. After login as an admin
|
||||
you should be able see ``My Dashboard`` shows up at the left side on horizon.
|
||||
Click it, ``My Group`` will expand with ``My Panel``. Click on ``My Panel``,
|
||||
the right side panel will display an ``Instances Tab`` which has an
|
||||
|
@ -274,10 +274,10 @@ Run and check the dashboard
|
||||
|
||||
We must once again run horizon to verify our dashboard is working::
|
||||
|
||||
./run_tests.sh --runserver 0.0.0.0:8877
|
||||
$ tox -e runserver -- 0:9000
|
||||
|
||||
|
||||
Go to ``http://<your server>:8877`` using a browser. After login as an admin,
|
||||
Go to ``http://<your server>:9000`` using a browser. After login as an admin,
|
||||
display ``My Panel`` to see the ``Instances`` table. For every ``ACTIVE``
|
||||
instance in the table, there will be a ``Create Snapshot`` action on the row.
|
||||
Click on ``Create Snapshot``, enter a snapshot name in the form that is shown,
|
||||
|
@ -20,23 +20,14 @@ var fs = require('fs');
|
||||
var path = require('path');
|
||||
|
||||
module.exports = function (config) {
|
||||
var xstaticPath;
|
||||
var basePaths = [
|
||||
'./.venv',
|
||||
'./.tox/py27'
|
||||
];
|
||||
var xstaticPath = path.resolve('./.tox/npm');
|
||||
|
||||
for (var i = 0; i < basePaths.length; i++) {
|
||||
var basePath = path.resolve(basePaths[i]);
|
||||
|
||||
if (fs.existsSync(basePath)) {
|
||||
xstaticPath = basePath + '/lib/python2.7/site-packages/xstatic/pkg/';
|
||||
break;
|
||||
}
|
||||
if (fs.existsSync(xstaticPath)) {
|
||||
xstaticPath += '/lib/python2.7/site-packages/xstatic/pkg/';
|
||||
}
|
||||
|
||||
if (!xstaticPath) {
|
||||
console.error('xStatic libraries not found, please set up venv');
|
||||
console.error('xStatic libraries not found, please run `tox -e npm`');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
|
@ -20,23 +20,14 @@ var fs = require('fs');
|
||||
var path = require('path');
|
||||
|
||||
module.exports = function (config) {
|
||||
var xstaticPath;
|
||||
var basePaths = [
|
||||
'./.venv',
|
||||
'./.tox/py27'
|
||||
];
|
||||
var xstaticPath = path.resolve('./.tox/npm');
|
||||
|
||||
for (var i = 0; i < basePaths.length; i++) {
|
||||
var basePath = path.resolve(basePaths[i]);
|
||||
|
||||
if (fs.existsSync(basePath)) {
|
||||
xstaticPath = basePath + '/lib/python2.7/site-packages/xstatic/pkg/';
|
||||
break;
|
||||
}
|
||||
if (fs.existsSync(xstaticPath)) {
|
||||
xstaticPath += '/lib/python2.7/site-packages/xstatic/pkg/';
|
||||
}
|
||||
|
||||
if (!xstaticPath) {
|
||||
console.error('xStatic libraries not found, please set up venv');
|
||||
console.error('xStatic libraries not found, please run `tox -e npm`');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
|
57
openstack_dashboard/management/commands/extract_messages.py
Normal file
57
openstack_dashboard/management/commands/extract_messages.py
Normal file
@ -0,0 +1,57 @@
|
||||
# Copyright 2016 Cisco Systems, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from distutils.dist import Distribution
|
||||
import os
|
||||
from subprocess import call
|
||||
|
||||
from django.core.management.base import BaseCommand
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = ('Extract strings that have been marked for translation into .POT '
|
||||
'files.')
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument('-m', '--module', type=str, nargs='+',
|
||||
default=['openstack_dashboard', 'horizon'],
|
||||
help=("The target python module(s) to extract "
|
||||
"strings from"))
|
||||
parser.add_argument('-d', '--domain', choices=['django', 'djangojs'],
|
||||
nargs='+', default=['django', 'djangojs'],
|
||||
help="Domain(s) of the .pot file")
|
||||
parser.add_argument('--check-only', action='store_true',
|
||||
help=("Checks that extraction works correctly, "
|
||||
"then deletes the .pot file to avoid "
|
||||
"polluting the source code"))
|
||||
|
||||
def handle(self, *args, **options):
|
||||
cmd = ('python setup.py extract_messages -F babel-{domain}.cfg '
|
||||
'-o {module}/locale/{domain}.pot')
|
||||
distribution = Distribution()
|
||||
distribution.parse_config_files(distribution.find_config_files())
|
||||
|
||||
if options['check_only']:
|
||||
cmd += " ; rm {module}/locale/{domain}.pot"
|
||||
|
||||
for module in options['module']:
|
||||
for domain in options['domain']:
|
||||
potfile = '{module}/locale/{domain}.pot'.format(module=module,
|
||||
domain=domain)
|
||||
if not os.path.exists(potfile):
|
||||
with open(potfile, 'wb') as f:
|
||||
f.write(b'')
|
||||
|
||||
call(cmd.format(module=module, domain=domain, potfile=potfile),
|
||||
shell=True)
|
122
openstack_dashboard/management/commands/update_catalog.py
Normal file
122
openstack_dashboard/management/commands/update_catalog.py
Normal file
@ -0,0 +1,122 @@
|
||||
# coding: utf-8
|
||||
|
||||
# Copyright 2016 Cisco Systems, Inc.
|
||||
# Copyright 2015 IBM Corp.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import babel.messages.catalog as catalog
|
||||
import os
|
||||
from subprocess import call
|
||||
|
||||
from django.conf import settings
|
||||
from django.core.management.base import BaseCommand
|
||||
from django.utils import translation
|
||||
|
||||
LANGUAGE_CODES = [language[0] for language in settings.LANGUAGES
|
||||
if language[0] != 'en']
|
||||
POTFILE = "{module}/locale/{domain}.pot"
|
||||
POFILE = "{module}/locale/{locale}/LC_MESSAGES/{domain}.po"
|
||||
|
||||
|
||||
def translate(segment):
|
||||
prefix = u""
|
||||
# When the id starts with a newline the mo compiler enforces that
|
||||
# the translated message must also start with a newline. Make
|
||||
# sure that doesn't get broken when prepending the bracket.
|
||||
if segment.startswith('\n'):
|
||||
prefix = u"\n"
|
||||
orig_size = len(segment)
|
||||
# Add extra expansion space based on recommendation from
|
||||
# http://www-01.ibm.com/software/globalization/guidelines/a3.html
|
||||
if orig_size < 20:
|
||||
multiplier = 1
|
||||
elif orig_size < 30:
|
||||
multiplier = 0.8
|
||||
elif orig_size < 50:
|
||||
multiplier = 0.6
|
||||
elif orig_size < 70:
|
||||
multiplier = 0.4
|
||||
else:
|
||||
multiplier = 0.3
|
||||
extra_length = int(max(0, (orig_size * multiplier) - 10))
|
||||
extra_chars = "~" * extra_length
|
||||
return u"{0}[~{1}~您好яшçあ{2}]".format(prefix, segment, extra_chars)
|
||||
|
||||
|
||||
class Command(BaseCommand):
|
||||
help = 'Update a translation catalog for a specified language'
|
||||
|
||||
def add_arguments(self, parser):
|
||||
parser.add_argument('-l', '--language', choices=LANGUAGE_CODES,
|
||||
default=LANGUAGE_CODES, nargs='+',
|
||||
help=("The language code(s) to pseudo translate"))
|
||||
parser.add_argument('-m', '--module', type=str, nargs='+',
|
||||
default=['openstack_dashboard', 'horizon'],
|
||||
help=("The target python module(s) to extract "
|
||||
"strings from"))
|
||||
parser.add_argument('-d', '--domain', choices=['django', 'djangojs'],
|
||||
nargs='+', default=['django', 'djangojs'],
|
||||
help="Domain(s) of the .POT file")
|
||||
parser.add_argument('--pseudo', action='store_true',
|
||||
help=("Creates a pseudo translation for the "
|
||||
"specified locale, to check for "
|
||||
"translatable string coverage"))
|
||||
|
||||
def handle(self, *args, **options):
|
||||
for module in options['module']:
|
||||
for domain in options['domain']:
|
||||
potfile = POTFILE.format(module=module, domain=domain)
|
||||
|
||||
for language in options['language']:
|
||||
# Get the locale code for the language code given and
|
||||
# work around broken django conversion function
|
||||
locales = {'ko': 'ko_KR', 'pl': 'pl_PL', 'tr': 'tr_TR'}
|
||||
locale = locales.get(language,
|
||||
translation.to_locale(language))
|
||||
pofile = POFILE.format(module=module, locale=locale,
|
||||
domain=domain)
|
||||
|
||||
# If this isn't a pseudo translation, execute pybabel
|
||||
if not options['pseudo']:
|
||||
if not os.path.exists(pofile):
|
||||
with open(pofile, 'wb') as fobj:
|
||||
fobj.write(b'')
|
||||
|
||||
cmd = ('pybabel update -l {locale} -i {potfile} '
|
||||
'-o {pofile}').format(locale=locale,
|
||||
potfile=potfile,
|
||||
pofile=pofile)
|
||||
call(cmd, shell=True)
|
||||
continue
|
||||
|
||||
# Pseudo translation logic
|
||||
with open(potfile, 'r') as f:
|
||||
pot_cat = pofile.read_po(f, ignore_obsolete=True)
|
||||
|
||||
new_cat = catalog.Catalog(locale=locale,
|
||||
last_translator="pseudo.py",
|
||||
charset="utf-8")
|
||||
num_plurals = new_cat.num_plurals
|
||||
|
||||
for msg in pot_cat:
|
||||
if msg.pluralizable:
|
||||
msg.string = [
|
||||
translate(u"{}:{}".format(i, msg.id[0]))
|
||||
for i in range(num_plurals)]
|
||||
else:
|
||||
msg.string = translate(msg.id)
|
||||
new_cat[msg.id] = msg
|
||||
|
||||
with open(pofile, 'w') as f:
|
||||
pofile.write_po(f, new_cat, ignore_obsolete=True)
|
@ -19,7 +19,7 @@
|
||||
"karma-threshold-reporter": "0.1.15"
|
||||
},
|
||||
"scripts": {
|
||||
"postinstall": "if [ ! -d .venv ]; then tox -epy27 --notest; fi",
|
||||
"postinstall": "if [ ! -d .tox/npm ]; then tox -e npm --notest; fi",
|
||||
"test": "karma start horizon/karma.conf.js --single-run && karma start openstack_dashboard/karma.conf.js --single-run",
|
||||
"lint": "eslint --no-color openstack_dashboard/static horizon/static openstack_dashboard/dashboards/*/static",
|
||||
"lintq": "eslint --quiet openstack_dashboard/static horizon/static openstack_dashboard/dashboards/*/static"
|
||||
|
11
releasenotes/notes/bp/enhance-tox-26f73a048b88df2f.yaml
Normal file
11
releasenotes/notes/bp/enhance-tox-26f73a048b88df2f.yaml
Normal file
@ -0,0 +1,11 @@
|
||||
---
|
||||
features:
|
||||
- The hard-coded run_tests commands for extracting translatable strings and
|
||||
updating message catalogs have been ported to django management commands
|
||||
as extract_messages and update_catalog. These accept several parameters
|
||||
to make them easier to use with downstream customisations and string
|
||||
modifications, but the default behaviour is the same as before.
|
||||
deprecations:
|
||||
- The run_tests.sh script is now deprecated and all functionality has
|
||||
been provided by either tox or manage.py. run_tests will be removed
|
||||
in Queens (13.0).
|
@ -19,6 +19,10 @@ import argparse
|
||||
import babel.messages.catalog as catalog
|
||||
import babel.messages.pofile as pofile
|
||||
|
||||
# NOTE: This implementation has been superseded by the pseudo_translate
|
||||
# management command, and will be removed in Queens (13.0) when run_tests.sh
|
||||
# is also removed.
|
||||
|
||||
|
||||
def translate(segment):
|
||||
prefix = u""
|
||||
|
30
tools/unit_tests.sh
Executable file
30
tools/unit_tests.sh
Executable file
@ -0,0 +1,30 @@
|
||||
# Uses envpython and toxinidir from tox run to construct a test command
|
||||
testcommand="${1} ${2}/manage.py test"
|
||||
posargs="${@:3}"
|
||||
|
||||
# Attempt to identify if any of the arguments passed from tox is a test subset
|
||||
if [ -n "$posargs" ]; then
|
||||
for arg in "$posargs"
|
||||
do
|
||||
if [ ${arg:0:1} != "-" ]; then
|
||||
subset=$arg
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# If we are running a test subset, supply the correct settings file.
|
||||
# If not, simply run the entire test suite.
|
||||
if [ -n "$subset" ]; then
|
||||
project="${subset%%.*}"
|
||||
|
||||
if [ $project == "horizon" ]; then
|
||||
$testcommand --settings=horizon.test.settings $posargs
|
||||
elif [ $project == "openstack_dashboard" ]; then
|
||||
$testcommand --settings=openstack_dashboard.test.settings \
|
||||
--exclude-dir=openstack_dashboard/test/integration_tests $posargs
|
||||
fi
|
||||
else
|
||||
$testcommand horizon --settings=horizon.test.settings $posargs
|
||||
$testcommand openstack_dashboard --settings=openstack_dashboard.test.settings \
|
||||
--exclude-dir=openstack_dashboard/test/integration_tests $posargs
|
||||
fi
|
147
tox.ini
147
tox.ini
@ -1,105 +1,64 @@
|
||||
[tox]
|
||||
envlist = pep8,py27dj18,py27,py34,py35,releasenotes
|
||||
minversion = 1.6
|
||||
envlist = pep8,py27dj{18,19,110},py34,py35,releasenotes,npm
|
||||
minversion = 2.3.2
|
||||
skipsdist = True
|
||||
|
||||
[testenv]
|
||||
basepython=python2.7
|
||||
install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt} {opts} {packages}
|
||||
setenv =
|
||||
VIRTUAL_ENV={envdir}
|
||||
INTEGRATION_TESTS=0
|
||||
SELENIUM_HEADLESS=0
|
||||
SELENIUM_PHANTOMJS=0
|
||||
NOSE_WITH_OPENSTACK=1
|
||||
NOSE_OPENSTACK_SHOW_ELAPSED=1
|
||||
whitelist_externals =
|
||||
bash
|
||||
deps =
|
||||
-r{toxinidir}/requirements.txt
|
||||
-r{toxinidir}/test-requirements.txt
|
||||
whitelist_externals =
|
||||
bash
|
||||
commands =
|
||||
# Try to detect whether a limited test suite is being specified and if so
|
||||
# direct the testing to that suite's project; otherwise run the full suite
|
||||
# in both horizon and openstack_dashboard "projects".
|
||||
bash -c 'project=`echo {posargs} | cut -d. -f1`; \
|
||||
if [ -z "$project" ]; then \
|
||||
EXIT_STATUS=0; \
|
||||
{envpython} {toxinidir}/manage.py test horizon --settings=horizon.test.settings {posargs} || EXIT_STATUS=$?; \
|
||||
{envpython} {toxinidir}/manage.py test openstack_dashboard --settings=openstack_dashboard.test.settings --exclude-dir=openstack_dashboard/test/integration_tests {posargs} || EXIT_STATUS=$?; \
|
||||
exit $EXIT_STATUS; \
|
||||
else \
|
||||
{envpython} {toxinidir}/manage.py test {posargs} --settings=$project.test.settings --exclude-dir=openstack_dashboard/test/integration_tests; \
|
||||
fi'
|
||||
|
||||
# Django-1.8 is LTS
|
||||
[testenv:py27dj18]
|
||||
commands =
|
||||
pip install django>=1.8,<1.9
|
||||
{envpython} {toxinidir}/manage.py test horizon --settings=horizon.test.settings {posargs}
|
||||
{envpython} {toxinidir}/manage.py test openstack_dashboard --settings=openstack_dashboard.test.settings --exclude-dir=openstack_dashboard/test/integration_tests {posargs}
|
||||
docs: sphinx-build -W -b html doc/source doc/build/html
|
||||
horizon: {envpython} {toxinidir}/manage.py test --settings=horizon.test.settings {posargs}
|
||||
manage: {envpython} {toxinidir}/manage.py {posargs}
|
||||
py27: {[unit_tests]commands}
|
||||
py27dj18: {[unit_tests]commands}
|
||||
py34: {[unit_tests]commands}
|
||||
py35: {[unit_tests]commands}
|
||||
openstack_dashboard: {envpython} {toxinidir}/manage.py test --settings=openstack_dashboard.test.settings {posargs}
|
||||
releasenotes: sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html
|
||||
runserver: {envpython} {toxinidir}/manage.py runserver {posargs}
|
||||
venv: {posargs}
|
||||
|
||||
[testenv:py34]
|
||||
basepython = python3.4
|
||||
setenv =
|
||||
PYTHONUNBUFFERED = 1
|
||||
{[testenv]setenv}
|
||||
PYTHONUNBUFFERED=1
|
||||
commands = {[unit_tests]commands}
|
||||
|
||||
[testenv:py35]
|
||||
basepython = python3.5
|
||||
setenv =
|
||||
PYTHONUNBUFFERED = 1
|
||||
{[testenv]setenv}
|
||||
PYTHONUNBUFFERED=1
|
||||
commands = {[unit_tests]commands}
|
||||
|
||||
[testenv:py27dj19]
|
||||
commands =
|
||||
pip install -U django>=1.9,<1.10
|
||||
{[unit_tests]commands}
|
||||
|
||||
[testenv:py27dj110]
|
||||
commands =
|
||||
pip install -U django>=1.10,<1.11
|
||||
{[unit_tests]commands}
|
||||
|
||||
[unit_tests]
|
||||
commands = bash {toxinidir}/tools/unit_tests.sh {envpython} {toxinidir} {posargs}
|
||||
|
||||
[testenv:pep8]
|
||||
usedevelop = True
|
||||
whitelist_externals =
|
||||
git
|
||||
rm
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
DJANGO_SETTINGS_MODULE=openstack_dashboard.test.settings
|
||||
commands =
|
||||
{[testenv:extractmessages_check]commands}
|
||||
{[testenv:docs]commands}
|
||||
{envpython} {toxinidir}/manage.py extract_messages --check-only
|
||||
flake8
|
||||
|
||||
[testenv:extractmessages]
|
||||
usedevelop = True
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
commands =
|
||||
{envpython} {toxinidir}/setup.py extract_messages -F babel-django.cfg -o horizon/locale/django.pot --input-dirs horizon/
|
||||
{envpython} {toxinidir}/setup.py extract_messages -F babel-djangojs.cfg -o horizon/locale/djangojs.pot --input-dirs horizon/
|
||||
{envpython} {toxinidir}/setup.py extract_messages -F babel-django.cfg -o openstack_dashboard/locale/django.pot --input-dirs openstack_dashboard/
|
||||
{envpython} {toxinidir}/setup.py extract_messages -F babel-djangojs.cfg -o openstack_dashboard/locale/djangojs.pot --input-dirs openstack_dashboard/
|
||||
|
||||
[testenv:extractmessages_check]
|
||||
# Only checks to see if translation files can be extracted and cleans afterwards
|
||||
usedevelop = True
|
||||
whitelist_externals =
|
||||
rm
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
commands =
|
||||
{[testenv:extractmessages]commands}
|
||||
rm horizon/locale/django.pot
|
||||
rm horizon/locale/djangojs.pot
|
||||
rm openstack_dashboard/locale/django.pot
|
||||
rm openstack_dashboard/locale/djangojs.pot
|
||||
|
||||
[testenv:compilemessages]
|
||||
usedevelop = False
|
||||
commands =
|
||||
/bin/bash {toxinidir}/run_tests.sh --compilemessages -N
|
||||
|
||||
[testenv:venv]
|
||||
commands = {posargs}
|
||||
|
||||
[testenv:manage]
|
||||
# Env to launch manage.py commands
|
||||
commands = {envpython} {toxinidir}/manage.py {posargs}
|
||||
|
||||
[testenv:cover]
|
||||
commands =
|
||||
coverage erase
|
||||
@ -108,13 +67,28 @@ commands =
|
||||
coverage xml
|
||||
coverage html
|
||||
|
||||
[testenv:py27dj19]
|
||||
commands = pip install django>=1.9,<1.10
|
||||
/bin/bash run_tests.sh -N --no-pep8 {posargs}
|
||||
[testenv:selenium]
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
WITH_SELENIUM=1
|
||||
SKIP_UNITTESTS=1
|
||||
commands = {[unit_tests]commands}
|
||||
|
||||
[testenv:py27dj110]
|
||||
commands = pip install django>=1.10,<1.11
|
||||
/bin/bash run_tests.sh -N --no-pep8 {posargs}
|
||||
[testenv:selenium-headless]
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
SELENIUM_HEADLESS=1
|
||||
WITH_SELENIUM=1
|
||||
SKIP_UNITTESTS=1
|
||||
commands = {[unit_tests]commands}
|
||||
|
||||
[testenv:selenium-phantomjs]
|
||||
setenv =
|
||||
{[testenv]setenv}
|
||||
SELENIUM_PHANTOMJS=1
|
||||
WITH_SELENIUM=1
|
||||
SKIP_UNITTESTS=1
|
||||
commands = {[unit_tests]commands}
|
||||
|
||||
[testenv:py27integration]
|
||||
# Run integration tests only
|
||||
@ -134,16 +108,6 @@ commands =
|
||||
npm install
|
||||
npm run {posargs:test}
|
||||
|
||||
[testenv:docs]
|
||||
setenv = DJANGO_SETTINGS_MODULE=openstack_dashboard.test.settings
|
||||
commands = sphinx-build -W -b html doc/source doc/build/html
|
||||
|
||||
[testenv:releasenotes]
|
||||
commands = sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html
|
||||
|
||||
[testenv:runserver]
|
||||
commands = {envpython} {toxinidir}/manage.py runserver {posargs}
|
||||
|
||||
[testenv:tests_system_packages]
|
||||
# Provide an environment for system packagers that dont want anything from pip
|
||||
# Any extra deps needed for this env can be passed by setting TOX_EXTRA_DEPS
|
||||
@ -153,8 +117,7 @@ passenv = TOX_EXTRA_DEPS
|
||||
deps =
|
||||
commands =
|
||||
pip install -U {env:TOX_EXTRA_DEPS:}
|
||||
{envpython} {toxinidir}/manage.py test horizon --settings=horizon.test.settings {posargs}
|
||||
{envpython} {toxinidir}/manage.py test openstack_dashboard --settings=openstack_dashboard.test.settings {posargs}
|
||||
{[unit_tests]commands}
|
||||
|
||||
[flake8]
|
||||
exclude = .venv,.git,.tox,dist,*lib/python*,*egg,build,panel_template,dash_template,local_settings.py,*/local/*,*/test/test_plugins/*,.ropeproject,node_modules
|
||||
|
Loading…
Reference in New Issue
Block a user