Release notes for 0.4.0
Change-Id: I29f214772d538f242cc31f98c93428d6a242dce9
This commit is contained in:
Andrey Kurilin
parent
e598a569ee
commit
36c54b86ca
278
doc/release_notes/archive/v0.4.0.rst
Normal file
278
doc/release_notes/archive/v0.4.0.rst
Normal file
@ -0,0 +1,278 @@
|
||||
============
|
||||
Rally v0.4.0
|
||||
============
|
||||
|
||||
Information
|
||||
-----------
|
||||
|
||||
+------------------+-----------------------+
|
||||
| Commits | **76** |
|
||||
+------------------+-----------------------+
|
||||
| Bug fixes | **12** |
|
||||
+------------------+-----------------------+
|
||||
| Dev cycle | **28 days** |
|
||||
+------------------+-----------------------+
|
||||
| Release date | **4/18/2016** |
|
||||
+------------------+-----------------------+
|
||||
|
||||
|
||||
Details
|
||||
-------
|
||||
|
||||
.. warning:: Rally DB schema was changed since previous release.
|
||||
See `HOWTO <http://rally.readthedocs.org/en/latest/db_migrations.html>`_
|
||||
about updating your database.
|
||||
|
||||
|
||||
CLI changes
|
||||
~~~~~~~~~~~
|
||||
|
||||
* Add status messages of db migration process
|
||||
|
||||
* Display task errors in human-friendly form
|
||||
|
||||
* Support OS_PROJECT_NAME as well as OS_TENANT_NAME
|
||||
|
||||
Messages
|
||||
~~~~~~~~
|
||||
|
||||
* Removed deprecation warning in case of transmitted "name" attribute while
|
||||
creation neutron resources.
|
||||
|
||||
.. warning:: Deprecated code was deleted.
|
||||
|
||||
* Suppress warning insecure URL messages
|
||||
|
||||
Do not spam end users by insecure URL messages because it is
|
||||
quite valid case in testing process
|
||||
|
||||
Database
|
||||
~~~~~~~~
|
||||
|
||||
While preparing for deployment refactoring:
|
||||
|
||||
* db schema was changed;
|
||||
* migration with new column `credentials` to deployment model was added;
|
||||
* columns `users` and `admin` were dropped.
|
||||
|
||||
Rally Task
|
||||
~~~~~~~~~~
|
||||
|
||||
* Remove deprecated scenario output mechanism via returing value
|
||||
|
||||
.. warning:: Deprecated code was deleted.
|
||||
|
||||
* Friendlier error message with empty task file
|
||||
|
||||
This is particularly useful when a Jinja2 template results in an empty
|
||||
task. The current error message isn't very helpful:
|
||||
|
||||
Task config is invalid: `'NoneType' object has no attribute 'get'`
|
||||
|
||||
* Add Heat template validator
|
||||
|
||||
Plugins
|
||||
~~~~~~~
|
||||
|
||||
**Scenarios**:
|
||||
|
||||
* Extend VM bind actions with "pause_unpause", "suspend_resume", "lock_unlock",
|
||||
"shelve_unshelve".
|
||||
|
||||
* Add exact error message into `VMTasks.runcommand_heat scenario`__
|
||||
|
||||
__ http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#vmtasks-runcommand-heat-scenario
|
||||
|
||||
* Add heat scenarios: output-show, output-list
|
||||
|
||||
Current patch contains 4 scenarios from heat repo:
|
||||
- `output-show for old algorithm
|
||||
<http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#heatstacks-create-stack-and-list-output-via-api-scenario>`_
|
||||
- `output-show for new algorithm
|
||||
<http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#heatstacks-create-stack-and-show-output-scenario>`_
|
||||
- `output-list for old algorithm
|
||||
<http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#heatstacks-create-stack-and-list-output-via-api-scenario>`_
|
||||
- `output-list for new algorithm
|
||||
<http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#heatstacks-create-stack-and-list-output-scenario>`_
|
||||
|
||||
|
||||
**Contexts**:
|
||||
|
||||
* Reduce default speed of users creation in users context from 30 to 20
|
||||
by default.
|
||||
|
||||
**SLAs**:
|
||||
|
||||
* *NEW!!* MaxAverageDurationPerAtomic : Maximum average duration of one
|
||||
iterations atomic actions in seconds.
|
||||
|
||||
`Plugin Reference <http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#max-avg-duration-per-atomic-sla>`_
|
||||
|
||||
**Reports**:
|
||||
|
||||
* Improve results calculation in charts.Table
|
||||
|
||||
* Use int instead of float for Y axis. It's number of parallel iterations and
|
||||
it can't be float.
|
||||
|
||||
* Remove accuracy that makes no sense, and creates a lot of noise on this graph
|
||||
|
||||
* Include failed iterations as well, otherwise we will calculate load
|
||||
incorrectly
|
||||
|
||||
* Graph should start from 0 (begging of experiment)
|
||||
|
||||
* Add 2 points at the end of graph to get at the end of graph 0 iterations
|
||||
in parallel
|
||||
|
||||
**Task Exporter**:
|
||||
|
||||
In previous release we introduced new mechanism to export results in various
|
||||
external systems and various formats.
|
||||
|
||||
In this release, we added first plugin for this stuff - `file_exporter`
|
||||
|
||||
**Services**:
|
||||
|
||||
Remove hardcoded timeout from heat service
|
||||
|
||||
**Utils**:
|
||||
|
||||
Make glance web uploads streamable
|
||||
|
||||
Without this change entire file get's downloaded into memory and can cause
|
||||
issues.
|
||||
|
||||
Rally Verify
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* Set time precision to 3 digits (instead of 5) after dot.
|
||||
|
||||
* Don't use "--parallel" flag when concurrency == 1
|
||||
|
||||
If concurrency equals to 1, it means that we use only one thread to run
|
||||
Tempest tests and the "--parallel" flag is not needed.
|
||||
|
||||
Plugin for DevStack
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* Support to be enabled with different plugin name
|
||||
|
||||
Allow rally to be installed by devstack through a different plugin
|
||||
name, e.g:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
enable_plugin test-rally http://github.com/rally/rally.git master
|
||||
|
||||
* Removed uncalled code
|
||||
|
||||
Devstack won't "source plugin.sh source" any more.
|
||||
|
||||
Bug fixes
|
||||
~~~~~~~~~
|
||||
|
||||
**12 bugs were fixed**:
|
||||
|
||||
* X-Fail mechanism did not work for TestCase which failed on setUp step
|
||||
|
||||
If Tempest fails in a test's setUpClass(), there is only one subunit event
|
||||
for each TestCase. In this case, Rally did not check partial test with x-fail
|
||||
list and marked test as "fail" insted of "x-fail".
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1568133>`_
|
||||
|
||||
* Weak isolation of scenario arguments between iterations
|
||||
|
||||
Input arguments for sub-task were shared between all iterations. Rally team
|
||||
found one scenario which modified mutable input variable.
|
||||
|
||||
Affected scenario: NeutronNetworks.create_and_update_ports
|
||||
|
||||
* Incompatible filters between V1 and V2 for Glance images listing
|
||||
|
||||
Glance V1 and V2 have different filters. For example, "owner" is a separate
|
||||
kwarg in V1, not a generic filter. Also, visibility has different labels in
|
||||
different APIs. We modified our Glance wrapper to support Glance V2 format
|
||||
of filters for both V1 and V2
|
||||
|
||||
* Wrong way to store validation errors
|
||||
|
||||
Results of failed task validations saved in incorrect format. It broke and
|
||||
made un-userfriendly `rally task detailed` command.
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1562713>`_
|
||||
|
||||
* Hardcoded task's status in `rally task results`
|
||||
|
||||
If there are no results for task, `rally task results` printed message that
|
||||
task has failed status, but it can be not true(tasks in running state do not
|
||||
have results).
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1539096>`_
|
||||
|
||||
* Tempest context failed to create network resources
|
||||
|
||||
While we merged improvement for keystoneclient, we used wrong way to obtain
|
||||
tenant id in TempestContext.
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1550848>`_
|
||||
|
||||
* Tasks based on Tempest failed to parse execution time.
|
||||
|
||||
There is an ability in Rally to launch tasks based on Tempest. Since launch
|
||||
of Tempest is just subprocess, it is needed to parse subunit to set correct
|
||||
atomic actions.
|
||||
|
||||
There was an issue while converting task execution time.
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1566712>`_
|
||||
|
||||
* JSONSchema huge impact on task performance
|
||||
|
||||
Before runner sent data to engine we were checking jsonschema. This operation
|
||||
is very expensive and in some cases it can take a lot of time.
|
||||
|
||||
Here are test results, with Dummy.dummy_output scenario, sleep 0.5s
|
||||
(added manually), 8000 iterations, 400 in parallel:
|
||||
|
||||
* on master branch before the fix:
|
||||
Load duration: 117.659588099
|
||||
Full duration: 227.451056004
|
||||
|
||||
* on master before the fix but remove jsonschema validation in scenario:
|
||||
Load duration: 12.5437350273
|
||||
Full duration: 128.942219973
|
||||
|
||||
* on this patch before the fix (pure python validation):
|
||||
Load duration: 11.5991640091
|
||||
Full duration: 22.7199981213
|
||||
|
||||
* Wrong Calculation of running iterations in parallel
|
||||
|
||||
Load profile chart was calculated wrongly.
|
||||
It showed more running iterations in parallel than actually are running.
|
||||
|
||||
* Rally did not show "missing argument" error raised by argparse while parsing
|
||||
cli args
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1562916>`_
|
||||
|
||||
* Issue while checking required arguments in CLI
|
||||
|
||||
There was a possible issue in case of several required arguments
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1555764>`_
|
||||
|
||||
* Prepare step of verification did not check visibility of obtained image
|
||||
|
||||
When we request a list of images to choose one of them for tests, we should
|
||||
make sure all images are active and they are PUBLIC. If images are not
|
||||
public, we will have failures of Tempest tests as described in the bug.
|
||||
|
||||
`Launchpad bug-report <https://bugs.launchpad.net/rally/+bug/1564431>`_
|
||||
|
||||
Thanks
|
||||
~~~~~~
|
||||
|
||||
2 Everybody!
|
||||
@ -1 +1 @@
|
||||
archive/v0.3.3.rst
|
||||
./archive/v0.4.0.rst
|
||||
Loading…
Reference in New Issue
Block a user