diff --git a/CHANGELOG.rst b/CHANGELOG.rst new file mode 100644 index 00000000..76e0cc56 --- /dev/null +++ b/CHANGELOG.rst @@ -0,0 +1,67 @@ +========= +Changelog +========= + +.. Changelogs are for humans, not machines. The end users of Rally project are + human beings who care about what's is changing, why and how it affects them. + Please leave these notes as much as possible human oriented. + +.. Each release can use the next sections: + - **Added** for new features. + - **Changed** for changes in existing functionality. + - **Deprecated** for soon-to-be removed features/plugins. + - **Removed** for now removed features/plugins. + - **Fixed** for any bug fixes. + +.. Release notes for existing releases are MUTABLE! If there is something that + was missed or can be improved, feel free to change it! + +[Unreleased] +============ + +Start a fork of `rally/plugins/openstack module of original OpenStack Rally +project +`_ + +Added +----- + +* GnocchiArchivePolicy.list_archive_policy +* GnocchiArchivePolicy.create_archive_policy +* GnocchiArchivePolicy.create_delete_archive_policy +* GnocchiResourceType.list_resource_type +* GnocchiResourceType.create_resource_type +* GnocchiResourceType.create_delete_resource_type +* [ci] New Zuul V3 native jobs + +Changed +------- + +* Methods for association and dissociation floating ips were deprecated in + novaclient a year ago and latest major release (python-novaclient 10) + `doesn't include them + `_. + These actions should be performed via neutronclient now. It is not as simple + as it was via Nova-API and you can find more neutron-related atomic actions + in results of scenarios. + +Removed +------- + +* *os-hosts* CLIs and python API bindings had been deprecated in + python-novaclient 9.0.0 and became removed in `10.0.0 release + `_. + This decision affected 2 scenarios `NovaHosts.list_hosts + `_ + and `NovaHosts.list_and_get_hosts + `_ + which become redundant and we cannot leave them (python-novaclient doesn't + have proper interfaces any more). + +Fixed +----- + +* The support of `kubernetes python client + `_ (which is used by Magnum plugins) + is not limited by 3.0.0 max version. You can use more modern releases of that + library. diff --git a/doc/release_notes/archive.rst b/doc/release_notes/archive.rst deleted file mode 100644 index d028cdd8..00000000 --- a/doc/release_notes/archive.rst +++ /dev/null @@ -1,11 +0,0 @@ -.. _release_notes/archive: - -================= -All release notes -================= - -.. toctree:: - :glob: - :maxdepth: 1 - - archive/* diff --git a/doc/release_notes/archive/v0.0.1.rst b/doc/release_notes/archive/v0.0.1.rst deleted file mode 100644 index 6ea46f23..00000000 --- a/doc/release_notes/archive/v0.0.1.rst +++ /dev/null @@ -1,28 +0,0 @@ -============ -Rally v0.0.1 -============ - - -Information ------------ - - -+------------------+-----------------+ -| Commits | **1039** | -+------------------+-----------------+ -| Bug fixes | **0** | -+------------------+-----------------+ -| Dev cycle | **547 days** | -+------------------+-----------------+ -| Release date | **26/Jan/2015** | -+------------------+-----------------+ - - - -Details -------- - -Rally is awesome tool for generic testing of OpenStack clouds. - -A lot of people started using Rally in their CI/CD so Rally team should provide -more stable product with clear strategy of deprecation and upgrades. diff --git a/doc/release_notes/archive/v0.0.2.rst b/doc/release_notes/archive/v0.0.2.rst deleted file mode 100644 index bcd72311..00000000 --- a/doc/release_notes/archive/v0.0.2.rst +++ /dev/null @@ -1,223 +0,0 @@ -============ -Rally v0.0.2 -============ - -Information ------------ - -+------------------+-----------------+ -| Commits | **100** | -+------------------+-----------------+ -| Bug fixes | **18** | -+------------------+-----------------+ -| Dev cycle | **45 days** | -+------------------+-----------------+ -| Release date | **12/Mar/2015** | -+------------------+-----------------+ - - -Details -------- - -This release contains new features, new task plugins, bug fixes, -various code and API improvements. - - -New Features -~~~~~~~~~~~~ - -* rally task start **--abort-on-sla-failure** - - Stopping load before things go wrong. - Load generation will be interrupted if SLA criteria stop passing. - -* Rally verify command supports multiple Tempest sources now. - -* python34 support - -* postgres DB backend support - - -API changes -~~~~~~~~~~~ - - -* [new] **rally [deployment | verify | task] use** subcommand - - It should be used instead of root command **rally use** - -* [new] Rally as a Lib API - - To avoid code duplication between Rally as CLI tool and Rally as a Service - we decide to make Rally as a Lib as a common part between these 2 modes. - - Rally as a Service will be a daemon that just maps HTTP request to Rally - as a Lib API. - -* [deprecated] **rally use** CLI command - -* [deprecated] Old Rally as a Lib API - - Old Rally API was quite mixed up so we decide to deprecate it - - -Plugins -~~~~~~~ - -* **Task Runners**: - - [improved] Improved algorithm of generation load in **constant runner** - - Before we used processes to generate load, now it creates pool of - processes (amount of processes is equal to CPU count) after that in - each process use threads to generate load. So now you can easily - generate load of 1k concurrent scenarios. - - [improved] Unify code of **constant** and **rps** runners - - [interface] Added **abort()** to runner's plugin interface - - New method **abort()** is used to immediately interrupt execution. - - -* **Task Scenarios**: - - [new] DesignateBasic.create_and_delete_server - - [new] DesignateBasic.create_and_list_servers - - [new] DesignateBasic.list_servers - - [new] MistralWorkbooks.list_workbooks - - [new] MistralWorkbooks.create_workbook - - [new] Quotas.neutron_update - - [new] HeatStacks.create_update_delete_stack - - [new] HeatStacks.list_stacks_and_resources - - [new] HeatStacks.create_suspend_resume_delete_stac - - [new] HeatStacks.create_check_delete_stack - - [new] NeutronNetworks.create_and_delete_routers - - [new] NovaKeypair.create_and_delete_keypair - - [new] NovaKeypair.create_and_list_keypairs - - [new] NovaKeypair.boot_and_delete_server_with_keypair - - [new] NovaServers.boot_server_from_volume_and_live_migrate - - [new] NovaServers.boot_server_attach_created_volume_and_live_migrate - - [new] CinderVolumes.create_and_upload_volume_to_image - - [fix] CinderVolumes.create_and_attach_volume - - Pass optional \*\*kwargs only to create server command - - [fix] GlanceImages.create_image_and_boot_instances - - Pass optional \*\*kwargs only to create server command - - [fix] TempestScenario.\* removed stress cleanup. - - Major issue is that tempest stress cleanup cleans whole OpenStack. - This is very dangerous, so it's better to remove it and leave some - extra resources. - - [improved] NovaSecGroup.boot_and_delete_server_with_secgroups - - Add optional \*\*kwargs that are passed to boot server comment - - -* **Task Context**: - - [new] **stacks** - - Generates passed amount of heat stacks for all tenants. - - [new] **custom_image** - - Prepares images for internal VMs testing. - - To Support generating workloads in VMs by existing tools like: IPerf, - Blogbench, HPCC and others we have to have prepared images, with - already installed and configured tools. - - Rally team decide to generate such images on fly from passed to avoid - requirements of having big repository with a lot of images. - - This context is abstract context that allows to automate next steps: - - 1) runs VM with passed image (with floating ip and other stuff) - 2) execute abstract method that has access to VM - 3) snapshot this image - - In future we are going to use this as a base for making context that - prepares images. - - [improved] **allow_ssh** - - Automatically disable it if security group are disabled in neutron. - - [improved] **keypair** - - Key pairs are stored in "users" space it means that accessing keypair - from scenario is simpler now: - - self.context["user"]["keypair"]["private"] - - [fix] **users** - - Pass proper EndpointType for newly created users - - [fix] **sahara_edp** - - The Job Binaries data should be treated as a binary content - - -* **Task SLA**: - - [interface] SLA calculations is done in additive way now - - Resolves scale issues, because now we don't need to have whole - array of iterations in memory to process SLA. - - This is required to implement **--abort-on-sla-failure** feature - - [all] SLA plugins were rewritten to implement new interface - - -Bug fixes -~~~~~~~~~ - -**18 bugs were fixed, the most critical are**: - - -* Fix **rally task detailed --iterations-data** - - It didn't work in case of missing atomic actions. Such situation can occur - if scenario method raises exceptions - -* Add user-friendly message if the task cannot be deleted - - In case of trying to delete task that is not in "finished" status users get - traces instead of user-friendly message try to run it with --force key. - - -* Network context cleanups networks properly now - - -Documentation -~~~~~~~~~~~~~ - -* Image sizes are fixed - -* New tutorial in "Step by Step" relate to **--abort-on-sla-failure** - -* Various fixes diff --git a/doc/release_notes/archive/v0.0.3.rst b/doc/release_notes/archive/v0.0.3.rst deleted file mode 100644 index a4e3cd57..00000000 --- a/doc/release_notes/archive/v0.0.3.rst +++ /dev/null @@ -1,153 +0,0 @@ -============ -Rally v0.0.3 -============ - -Information ------------ - -+------------------+-----------------+ -| Commits | **53** | -+------------------+-----------------+ -| Bug fixes | **14** | -+------------------+-----------------+ -| Dev cycle | **33 days** | -+------------------+-----------------+ -| Release date | **14/Apr/2015** | -+------------------+-----------------+ - - -Details -------- - -This release contains new features, new task plugins, bug fixes, -various code and API improvements. - - -New Features & API changes -~~~~~~~~~~~~~~~~~~~~~~~~~~ - - - * Add the ability to specify versions for clients in scenarios - - You can call self.clients("glance", "2") and get any client for - specific version. - - * Add API for tempest uninstall - - $ rally-manage tempest uninstall - # removes fully tempest for active deployment - - - * Add a --uuids-only option to rally task list - - $ rally task list --uuids-only # returns list with only task uuids - - * Adds endpoint to --fromenv deployment creation - - $ rally deployment create --fromenv - # recognizes standard OS_ENDPOINT environment variable - - * Configure SSL per deployment - - Now SSL information is deployment specific not Rally specific and - rally.conf option is deprecated - - Like in this sample - https://github.com/openstack/rally/blob/14d0b5ba0c75ececfdb6a6c121d9cf2810571f77/samples/deployments/existing.json#L11-L12 - - -Specs -~~~~~ - - * [spec] Proposal for new task input file format - - This spec describes new task input format that will allow us to generate - multi scenario load which is crucial for HA and more real life testing: - - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/new_rally_input_task_format.rst - - -Plugins -~~~~~~~ - -* **Task Runners**: - - * Add a maximum concurrency option to rps runner - - To avoid running to heavy load you can set 'concurrency' to configuration - and in case if cloud is not able to process all requests it won't start - more parallel requests then 'concurrency' value. - - -* **Task Scenarios**: - - [new] CeilometerAlarms.create_alarm_and_get_history - - [new] KeystoneBasic.get_entities - - [new] EC2Servers.boot_server - - [new] KeystoneBasic.create_and_delete_service - - [new] MuranoEnvironments.list_environments - - [new] MuranoEnvironments.create_and_delete_environment - - [new] NovaServers.suspend_and_resume_server - - [new] NovaServers.pause_and_unpause_server - - [new] NovaServers.boot_and_rebuild_server - - [new] KeystoneBasic.create_and_list_services - - [new] HeatStacks.list_stacks_and_events - - [improved] VMTask.boot_runcommand_delete - - restore ability to use fixed IP and floating IP to connect to VM - via ssh - - [fix] NovaServers.boot_server_attach_created_volume_and_live_migrate - - Kwargs in nova scenario were wrongly passed - - - -* **Task SLA**: - - * [new] aborted_on_sla - - This is internal SLA criteria, that is added if task was aborted - - - * [new] something_went_wrong - - This is internal SLA criteria, that is added if something went wrong, - context failed to create or runner raised some exceptions - - -Bug fixes -~~~~~~~~~ - -**14 bugs were fixed, the most critical are**: - - * Set default task uuid to running task. Before it was set only after - task was fully finished. - - * The "rally task results" command showed a disorienting "task not found" - message for a task that is currently running. - - * Rally didn't know how to reconnect to OpenStack in case if token - expired. - - -Documentation -~~~~~~~~~~~~~ - -* New tutorial **task templates** - -https://rally.readthedocs.org/en/latest/tutorial/step_5_task_templates.html - -* Various fixes - diff --git a/doc/release_notes/archive/v0.0.4.rst b/doc/release_notes/archive/v0.0.4.rst deleted file mode 100644 index 8363ebc4..00000000 --- a/doc/release_notes/archive/v0.0.4.rst +++ /dev/null @@ -1,222 +0,0 @@ -============ -Rally v0.0.4 -============ - -Information ------------ - -+------------------+-----------------+ -| Commits | **87** | -+------------------+-----------------+ -| Bug fixes | **21** | -+------------------+-----------------+ -| Dev cycle | **30 days** | -+------------------+-----------------+ -| Release date | **14/May/2015** | -+------------------+-----------------+ - - -Details -------- - -This release contains new features, new task plugins, bug fixes, various code and API improvements. - - -New Features & API changes -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Rally now can generate load with users that already exist - - Now one can use Rally for testing OpenStack clouds that are using LDAP, AD or - any other read-only keystone backend where it is not possible to create any - users. To do this, one should set up the "users" section of the deployment - configuration of the ExistingCloud type. This feature also makes it safer to - run Rally against production clouds: when run from an isolated group of - users, Rally won't affect rest of the cloud users if something goes wrong. - -* New decorator *@osclients.Clients.register* can add new OpenStack clients - at runtime - - It is now possible to add a new OpenStack client dynamically at runtime. - The added client will be available from osclients.Clients at the - module level and cached. Example: - -.. code-block:: none - - >>> from rally import osclients - >>> @osclients.Clients.register("supernova") - ... def another_nova_client(self): - ... from novaclient import client as nova - ... return nova.Client("2", auth_token=self.keystone().auth_token, - ... **self._get_auth_info(password_key="key")) - ... - >>> clients = osclients.Clients.create_from_env() - >>> clients.supernova().services.list()[:2] - [, ] - -* Assert methods now available for scenarios and contexts - - There is now a new *FunctionalMixin* class that implements basic unittest - assert methods. The *base.Context* and *base.Scenario* classes inherit from - this mixin, so now it is possible to use *base.assertX()* - methods in scenarios and contexts. - -* Improved installation script - - The installation script has been almost completely rewritten. After this - change, it can be run from an unprivileged user, supports different database - types, allows to specify a custom python binary, always asks confirmation - before doing potentially dangerous actions, automatically install needed - software if run as root, and also automatically cleans up the - virtualenv and/or the downloaded repository if interrupted. - - -Specs & Feature requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* [Spec] Reorder plugins - - The spec describes how to split Rally framework and plugins codebase to make - it simpler for newbies to understand how Rally code is organized and - how it works. - -* [Feature request] Specify what subtasks to execute in task - - This feature request proposes to add the ability to specify subtask(s) - to be executed when the user runs the *rally task start* command. A possible - solution would be to add a special flag to the *rally task start* command. - - -Plugins -~~~~~~~ - -* **Task Runners**: - - * Add limits for maximum Core usage to constant and rps runners - - The new 'max_cpu_usage' parameter can be used to avoid possible 100% - usage of all available CPU cores by reducing the number of CPU cores - available for processes started by the corresponding runner. - - -* **Task Scenarios**: - - * [new] KeystoneBasic.create_update_and_delete_tenant - - * [new] KeystoneBasic.create_user_update_password - - * [new] NovaServers.shelve_and_unshelve_server - - * [new] NovaServers.boot_and_associate_floating_ip - - * [new] NovaServers.boot_lock_unlock_and_delete - - * [new] NovaHypervisors.list_hypervisors - - * [new] CeilometerSamples.list_samples - - * [new] CeilometerResource.get_resources_on_tenant - - * [new] SwiftObjects.create_container_and_object_then_delete_all - - * [new] SwiftObjects.create_container_and_object_then_download_object - - * [new] SwiftObjects.create_container_and_object_then_list_objects - - * [new] MuranoEnvironments.create_and_deploy_environment - - * [new] HttpRequests.check_random_request - - * [new] HttpRequests.check_request - - * [improved] NovaServers live migrate scenarios - - add 'min_sleep' and 'max_sleep' parameters to simulate a pause between - VM booting and running live migration - - * [improved] NovaServers.boot_and_live_migrate_server - - add a usage sample to samples/tasks - - * [improved] CinderVolumes scenarios - - support size range to be passed to the 'size' argument as a dictionary - *{"min": , "max": }* - - -* **Task Contexts**: - - * [new] MuranoPackage - - This new context can upload a package to Murano from some specified path. - - * [new] CeilometerSampleGenerator - - Context that can be used for creating samples and collecting resources - for testing of list operations. - - -* **Task SLA**: - - * [new] outliers - - This new SLA checks that the number of outliers (calculated from the mean - and standard deviation of the iteration durations) does not exceed some - maximum value. The SLA is highly configurable: the parameters used for - outliers threshold calculation can be set by the user. - - -Bug fixes -~~~~~~~~~ - -**21 bugs were fixed, the most critical are**: - -* Make it possible to use relative imports for plugins that are outside of - rally package. - -* Fix heat stacks cleanup by deleting them only 1 time per tenant - (get rid of "stack not found" errors in logs). - -* Fix the wrong behavior of 'rally task detailed --iterations-data' - (it lacked the iteration info before). - -* Fix security groups cleanup: a security group called "default", created - automatically by Neutron, did not get deleted for each tenant. - - -Other changes -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* Streaming algorithms that scale - - This release introduces the common/streaming_algorithms.py module. - This module is going to contain implementations of task data processing - algorithms that scale: these algorithms do not store exhaustive information - about every single subtask iteration duration processed. For now, the module - contains implementations of algorithms for - computation of mean & standard deviation. - -* Coverage job to check that new patches come with unit tests - - Rally now has a coverage job that checks that every patch submitted for - review does not decrease the number of lines covered by unit tests - (at least too much). This job allows to mark most patches with no - unit tests with '-1'. - -* Splitting the plugins code (Runners & SLA) into common/openstack plugins - - According to the spec "Reorder plugins" (see above), the plugins code for - runners and SLA has been moved to the *plugins/common/* directory. - Only base classes now remain in the *benchmark/* directory. - - -Documentation -~~~~~~~~~~~~~ - -* Various fixes - - * Remove obsolete *.rst* files - (*deploy_engines.rst* / *server_providers.rst* / ...) - * Restructure the docs files to make them easier to navigate through - * Move the chapter on task templates to the 4th step in the tutorial - * Update the info about meetings (new release meeting & time changes) diff --git a/doc/release_notes/archive/v0.1.0.rst b/doc/release_notes/archive/v0.1.0.rst deleted file mode 100644 index 4eeede43..00000000 --- a/doc/release_notes/archive/v0.1.0.rst +++ /dev/null @@ -1,514 +0,0 @@ -============ -Rally v0.1.0 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **355** | -+------------------+-----------------------+ -| Bug fixes | **90** | -+------------------+-----------------------+ -| Dev cycle | **132 days** | -+------------------+-----------------------+ -| Release date | **25/September/2015** | -+------------------+-----------------------+ - - -Details -------- - -This release contains new features, new 42 plugins, 90 bug fixes, -various code and API improvements. - - -New Features & API changes -~~~~~~~~~~~~~~~~~~~~~~~~~~ -* **Improved installation script** - - * Add parameters: - - * ``--develop`` parameter to install rally in editable (develop) mode - - * ``--no-color`` to switch off output colorizing - useful for automated output parsing and terminals that don't - support colors. - - * Puts rally.conf under virtualenv etc/rally/ so you can have several - rally installations in virtualenv - - * Many fixes related to access of different file, like: rally.conf, - rally db file in case of sqlite - - * Update pip before Rally installation - - * Fix reinstallation - - -* **Separated Rally plugins & framework** - - Now plugins are here: - https://github.com/openstack/rally/tree/master/rally/plugins - - Plugins are as well separated common/* for common plugins - that can be use no matter what is tested and OpenStack related - plugins - - -* **New Rally Task framework** - - * All plugins has the same Plugin base: - rally.common.plugin.pluing.Plugin They have the same mechanisms for: - discovering, providing information based on docstrings, and in future - they will use the same deprecation/rename mechanism. - - * Some of files are moved: - - * rally/benchmark -> rally/task - - *This was done to unify naming of rally task command and - actually code that implements it.* - - * rally/benchmark/sla/base.py -> rally/task/sla.py - - * rally/benchmark/context/base.py -> rally/task/context.py - - * rally/benchmark/scenarios/base.py -> rally/task/scenario.py - - * rally/benchmark/runners/base.py -> rally/task/runner.py - - * rally/benchmark/scenarios/utils.py -> rally/task/utils.py - - This was done to: - - * avoid doing rally.benchmark.scenarios import base as scenario_base - - * remove one level of nesting - - * simplify framework structure - - - * Some of classes and methods were renamed - - * Plugin configuration: - - * context.context() -> context.configure() - - * scenario.scenario() -> scenario.configure() - - * Introduced runner.configure() - - * Introduced sla.configure() - - This resolves 3 problems: - - * Unifies configuration of different types of plugins - - * Simplifies plugin interface - - * Looks nice with new modules path: - >>> from rally.task import scenario - >>> @scenario.configure() - - - * Atomic Actions were changed: - - * New rally.task.atomic module - - This allow us in future to reuse atomic actions in Context plugins - - * Renames: - - rally.benchmark.scenarios.base.AtomicAction - -> rally.task.atomic.ActionTimer - - rally.benchmark.scenarios.base.atomic_action() - -> rally.task.atomic.action_timer() - - * **Context plugins decide how to map their data for scenario** - - Now Context.map_for_scenario method can be override to decide - how to pass context object to each iteration of scenario. - - * Samples of NEW vs OLD context, sla, scenario and runner plugins: - - * Context - - .. code-block:: python - - # Old - from rally.benchmark.context import base - - @base.context(name="users", order=100) - class YourContext(base.Context): - - def setup(self): - # ... - - def cleanup(self): - # ... - - # New - from rally.task import context - - @context.configure(name="users", order=100) - class YourContext(context.Context): - - def setup(self): - # ... - - def cleanup(self): - # ... - - def map_for_scenario(self): - # Maps context object to the scenario context object - # like context["users"] -> context["user"] and so on. - .. - - - * Scenario - - .. code-block:: python - - # Old Scenario - - from rally.benchmark.scenarios import base - from rally.benchmark import validation - - class ScenarioPlugin(base.Scenario): - - @base.scenario() - def some(self): - self._do_some_action() - - - @base.atomic_action_timer("some_timer") - def _do_some_action(self): - # ... - - # New Scenario - - from rally.task import atomic - from rally.task import scenario - from rally.task import validation - - # OpenStack scenario has different base now: - # rally.plugins.openstack.scenario.OpenStackScenario - class ScenarioPlugin(scenario.Scenario): - - @scenario.configure() - def some(self): - self._do_some_action() - - @atomic.action_timer("some_action") - def _do_some_action(self): - # ... - .. - - * Runner - - .. code-block:: python - - ## Old - - from rally.benchmark.runners import base - - class SomeRunner(base.ScenarioRunner): - - __execution_type__ = "some_runner" - - def _run_scenario(self, cls, method_name, context, args) - # Load generation - - def abort(self): - # Method that aborts load generation - - ## New - - from rally.task import runner - - @runner.configure(name="some_runner") - class SomeRunner(runner.ScenarioRunner): - - def _run_scenario(self, cls, method_name, context, args) - # Load generation - - def abort(self): - # Method that aborts load generation - - .. - - * SLA - - .. code-block:: python - - # Old - - from rally.benchmark import sla - - class FailureRate(sla.SLA): - # ... - - # New - - from rally.task import sla - - @sla.configure(name="failure_rate") - class FailureRate(sla.SLA): - # ... - .. - - -* **Rally Task aborted command** - - Finally you can gracefully shutdown running task by calling: - - .. code:: bash - - rally task abort - .. - -* **Rally CLI changes** - - * [add] ``rally --plugin-paths`` specify the list of directories with plugins - - * [add] ``rally task report --junit`` - generate a JUnit report - This allows users to feed reports to tools such as Jenkins. - - * [add] ``rally task abort`` - aborts running Rally task - when run with the ``--soft`` key, the ``rally task abort`` command is - waiting until the currently running subtask is finished, otherwise the - command interrupts subtask immediately after current scenario iterations - are finished. - - * [add] ``rally plugin show`` prints detailed information about plugin - - * [add] ``rally plugin list`` prints table with rally plugin names and titles - - * [add] ``rally verify genconfig`` generates tempest.conf without running it. - - * [add] ``rally verify install`` install tempest for specified deployment - - * [add] ``rally verify reinstall`` removes tempest for specified deployment - - * [add] ``rally verify uninstall`` uninstall tempest of specified deployment - - * [fix] ``rally verify start --no-use`` --no-use was always turned on - - * [remove] ``rally use`` now each command has subcommand ``use`` - - * [remove] ``rally info`` - - * [remove] ``rally-manage tempest`` now it is covered by ``rally verify`` - - -* **New Rally task reports** - - * New code is based on OOP style which is base step to make pluggable Reports - - * Reports are now generated for only one iteration over the resulting data - which resolves scalability issues when we are working with large - amount of iterations. - - * New Load profiler plot that shows amount of iterations that are working - in parallel - - * Failed iterations are shown as a red areas on stacked are graphic. - -Non backward compatible changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -* [remove] ``rally use`` cli command - -* [remove] ``rally info`` cli command - -* [remove] ``--uuid`` parameter from ``rally deployment `` - -* [remove ``--deploy-id`` parameter from: - ``rally task ``, ``rally verify ``, ``rally show `` - -Specs & Feature requests -~~~~~~~~~~~~~~~~~~~~~~~~ - - [feature request] Explicitly specify existing users for scenarios - - [feature request] Improve install script and add --uninstall and --version - - [feature request] Allows specific repos & packages in install-rally.sh - - [feature request] Add ability to capture logs from tested services - - [feature request] Check RPC queue perfdata - - [spec] Refactoring Rally cleanup - - [spec] Consistent resource names - -Plugins -~~~~~~~ - -* **Scenarios**: - - [new] CinderVolumes.create_volume_backup - - [new] CinderVolumes.create_and_restore_volume_backup - - [new] KeystoneBasic.add_and_remove_user_role - - [new] KeystoneBasic.create_and_delete_role - - [new] KeystoneBasic.create_add_and_list_user_roles - - [new] FuelEnvironments.list_environments - - [new] CinderVolumes.modify_volume_metadata - - [new] NovaServers.boot_and_delete_multiple_servers - - [new] NeutronLoadbalancerV1.create_and_list_pool - - [new] ManilaShares.list_shares - - [new] CeilometerEvents.create_user_and_get_event - - [new] CeilometerEvents.create_user_and_list_event_types - - [new] CeilometerEvents.create_user_and_list_events - - [new] CeilometerTraits.create_user_and_list_trait_descriptions - - [new] CeilometerTraits.create_user_and_list_traits - - [new] NeutronLoadbalancerV1.create_and_delete_pools - - [new] NeutronLoadbalancerV1.create_and_update_pools - - [new] ManilaShares.create_and_delete_share - - [new] ManilaShares.create_share_network_and_delete - - [new] ManilaShares.create_share_network_and_list - - [new] HeatStacks.create_and_delete_stack - - [new] ManilaShares.list_share_servers - - [new] HeatStacks.create_snapshot_restore_delete_stack - - [new] KeystoneBasic.create_and_delete_ec2credential - - [new] KeystoneBasic.create_and_list_ec2credentials - - [new] HeatStacks.create_stack_and_scale - - [new] ManilaShares.create_security_service_and_delete - - [new] KeystoneBasic.create_user_set_enabled_and_delete - - [new] ManilaShares.attach_security_service_to_share_network - - [new] IronicNodes.create_and_delete_node - - [new] IronicNodes.create_and_list_node - - [new] CinderVolumes.create_and_list_volume_backups - - [new] NovaNetworks.create_and_list_networks - - [new] NovaNetworks.create_and_delete_network - - [new] EC2Servers.list_servers - - [new] VMTasks.boot_runcommand_delete_custom_imagea - - [new] CinderVolumes.create_and_update_volume - - -* **Contexts**: - - [new] ManilaQuotas - - Add context for setting up Manila quotas: - shares, gigabytes, snapshots, snapshot_gigabytes, share_networks - - [new] ManilaShareNetworks - - Context for share networks that will be used in case of usage - deployment with existing users. Provided share networks via context - option "share_networks" will be balanced between all share creations - of scenarios. - - [new] Lbaas - - Context to create LBaaS-v1 resources - - [new] ImageCommandCustomizerContext - - Allows image customization using side effects of a command execution. - E.g. one can install an application to the image and use these image - for 'boot_runcommand_delete' scenario afterwards. - - [new] EC2ServerGenerator - - Context that creates servers using EC2 api - - [new] ExistingNetwork - - This context lets you use existing networks that have already been - created instead of creating new networks with Rally. This is useful - when, for instance, you are using Neutron with a dumb router that is - not capable of creating new networks on the fly. - - -* **SLA**: - - [remove] max_failure_rate - use failure_rate instead - - -Bug fixes -~~~~~~~~~ - -**90 bugs were fixed, the most critical are**: - -* Many fixes related that fixes access of rally.conf and DB files - -* Incorrect apt-get "-yes" parameter in install_rally.sh script - -* Rally bash completion doesn't exist in a virtualenv - -* Rally show networks CLI command worked only with nova networks - -* RPS runner was not properly generating load - -* Check is dhcp_agent_scheduler support or not in network cleanup - -* NetworkContext doesn't work with Nova V2.1 - -* Rally task input file was not able to use jinja2 include directive - -* Rally in docker image was not able to - -* Rally docker image didn't contain samples - -* Do not update the average duration when iteration failed - - -Documentation -~~~~~~~~~~~~~ - -* **Add plugin reference page** - - :ref:`Rally Plugins Reference page ` page contains a - full list with - -* **Add maintainers section on project info page** - - :ref:`Rally Maintainers section ` contains information - about core contributors of OpenStack Rally their responsibilities and - contacts. This will help us to make our community more transparent and open - for newbies. - -* **Added who is using section in docs** - -* **Many small fixes** diff --git a/doc/release_notes/archive/v0.1.1.rst b/doc/release_notes/archive/v0.1.1.rst deleted file mode 100644 index 5d17ba1e..00000000 --- a/doc/release_notes/archive/v0.1.1.rst +++ /dev/null @@ -1,131 +0,0 @@ -============ -Rally v0.1.1 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **32** | -+------------------+-----------------------+ -| Bug fixes | **9** | -+------------------+-----------------------+ -| Dev cycle | **11 days** | -+------------------+-----------------------+ -| Release date | **6/October/2015** | -+------------------+-----------------------+ - - -Details -------- - -This release contains new features, new 6 plugins, 9 bug fixes, -various code and API improvements. - - -New Features -~~~~~~~~~~~~ - -* **Rally verify generates proper tempest.conf file now** - - Improved script that generates tempest.conf, now it works out of box for - most of the clouds and most of Tempest tests will pass without hacking it. - -* **Import Tempest results to Rally DB** - - ``rally verify import`` command allows you to import already existing Tempest - results and work with them as regular "rally verify start" results: - generate HTML/CSV reports & compare different runs. - - -API Changes -~~~~~~~~~~~~ - -**Rally CLI changes** - - * [add] ``rally verify import`` imports raw Tempest results to Rally - - -Specs & Feature requests -~~~~~~~~~~~~~~~~~~~~~~~~ - - There is no new specs and feature requests. - -Plugins -~~~~~~~ - -* **Scenarios**: - - [new] NeutronNetworks.create_and_list_floating_ips - - [new] NeutronNetworks.create_and_delete_floating_ips - - [new] MuranoPackages.import_and_list_packages - - [new] MuranoPackages.import_and_delete_package - - [new] MuranoPackages.import_and_filter_applications - - [new] MuranoPackages.package_lifecycle - - [improved] NovaKeypair.boot_and_delete_server_with_keypair - - New argument ``server_kwargs``, these kwargs are used to boot server. - - [fix] NeutronLoadbalancerV1.create_and_delete_vips - - Now it works in case of concurrency > 1 - - -* **Contexts**: - - [improved] network - - Network context accepts two new arguments: - ``subnets_per_network`` and ``network_create_args``. - - [fix] network - - Fix cleanup if nova-network is used. Networks should be dissociate from - project before deletion - - [fix] custom_image - - Nova server that is used to create custom image was not deleted if - script that prepares server failed. - - -Bug fixes -~~~~~~~~~ - -**9 bugs were fixed, the most critical are**: - -* Fix install_rally.sh script - - Set 777 access to /var/lib/rally/database file if system-wide method of - installation is used. - -* Rally HTML reports Overview table had few mistakes - - * Success rate was always 100% - - * Percentiles were wrongly calculated - -* Missing Ironic, Murano and Workload(vm) options in default config file - -* ``rally verify start`` failed while getting network_id - -* ``rally verify genconfig`` hangs forever if Horizon is not available - - -Documentation -~~~~~~~~~~~~~ - -* **Fix project maintainers page** - - Update the information about Rally maintainers - -* **Document rally --plugin-paths CLI argument** - -* **Code blocks in documentation looks prettier now** - diff --git a/doc/release_notes/archive/v0.1.2.rst b/doc/release_notes/archive/v0.1.2.rst deleted file mode 100644 index 4bd70971..00000000 --- a/doc/release_notes/archive/v0.1.2.rst +++ /dev/null @@ -1,206 +0,0 @@ -============ -Rally v0.1.2 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **208** | -+------------------+-----------------------+ -| Bug fixes | **37** | -+------------------+-----------------------+ -| Dev cycle | **77 days** | -+------------------+-----------------------+ -| Release date | **23/December/2015** | -+------------------+-----------------------+ - - -Details -------- - -This release, as well as all previous ones, includes a lot of internal and -external changes. Most important of them are listed below. - -.. warning:: Release 0.1.2 is the last release with Python 2.6 support. - - -Deprecations -~~~~~~~~~~~~ - -* Class `rally.common.objects.Endpoint` was renamed to `Credentials`. Old - class is kept for backward compatibility. Please, stop using the old class - in your plugins. - - .. warning:: dict key was changed too in user context from "endpoint" to "credential" - -* rally.task.utils: wait_is_ready(), wait_for(), wait_for_delete() deprecated - you should use wait_for_status() instead. - - -Rally Verify -~~~~~~~~~~~~ - -* Added possibility to run Tempest tests listed in a file(--tests-file argument in ``verify start``) - -* Added possibility to upload Tempest subunit stream logs into data base - -* Improvements in generating Tempest config file - -* Reworked subunit stream parser - -* Don't install Tempest when `rally verify [gen/show]config` - -* Rally team tries to simplify usage of each our component. - Now Rally verification has some kind of a context like in Tasks. - Before launching each verification, Rally checks existence of required - resources(networks, images, flavours, etc) in Tempest configuration file and - pre-creates them. Do not worry, all these resources will not be forgotten - and left, Rally will clean them after verification. - - -Rally Task -~~~~~~~~~~ - -* Add --html-static argument to ``rally task report`` which allows to - generate HTML reports that doesn't require Internet. - -* Rally supports different API versions now via api_versions context: - -.. code-block:: none - - CinderVolumes.create_and_delete_volume: - - - args: - size: 1 - runner: - type: "constant" - times: 2 - concurrency: 2 - context: - users: - tenants: 2 - users_per_tenant: 2 - api_versions: - cinder: - version: 2 - service_name: cinderv2 - -* Move rally.osclients.Clients to plugin base - - Rally OSclients is pluggable now and it is very easy to extend OSClient for - your cloud out of Rally tree. - -* Add 'merge' functionality to SLA - - All SLA plugins should implement merge() method now. - In future this will be used for distributed load generation. - Where SLA results from different runners will be merged together. - -* New optional_action_timer decorator - - Allows to make the methods that can be both atomic_action or regular - method. Method changes behavior based on value in extra key "atomic_action" - - -Rally Certification -~~~~~~~~~~~~~~~~~~~ - -* Fix Glance certification arguments - -* Add Neutron Quotas only if Neutron service is available - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* Spec consistent-resource-names: - - Resource name is based on Task id now. It is a huge step to persistence - and disaster cleanups. - -* Add a spec for distributed load generation: - - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/distributed_runner.rst - -* Improvements for scenario output format - - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/improve_scenario_output_format.rst - -* Task and Verify results export command - - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/task_and_verification_export.rst - - -Plugins -~~~~~~~ - -* **Scenarios**: - - * [new] NovaServers.boot_and_get_console_output - * [new] NovaServers.boot_and_show_server - * [new] NovaServers.boot_server_attach_created_volume_and_resize - * [new] NovaServers.boot_server_from_volume_and_resize - * [new] NeutronSecurityGroup.create_and_delete_security_groups - * [new] NeutronSecurityGroup.create_and_list_security_groups - * [new] NeutronSecurityGroup.create_and_update_security_groups - * [new] NeutronLoadbalancerV1.create_and_delete_healthmonitors - * [new] NeutronLoadbalancerV1.create_and_list_healthmonitors - * [new] NeutronLoadbalancerV1.create_and_update_healthmonitors - * [new] SwiftObjects.list_and_download_objects_in_containers - * [new] SwiftObjects.list_objects_in_containers - * [new] FuelNodes.add_and_remove_node - * [new] CeilometerMeters.list_matched_meters - * [new] CeilometerResource.list_matched_resources - * [new] CeilometerSamples.list_matched_samples - * [new] CeilometerStats.get_stats - * [new] Authenticate.validate_monasca - * [new] DesignateBasic.create_and_delete_zone - * [new] DesignateBasic.create_and_list_zones - * [new] DesignateBasic.list_recordsets - * [new] DesignateBasic.list_zones - * [fix] CinderVolumes.create_nested_snapshots_and_attach_volume - Remove random nested level which produce different amount of atomic - actions and bad reports. - * Support for Designate V2 api - * A lot of improvements in Sahara scenarios - -* **Context**: - - * [new] api_versions - - Context allows us to setup client to communicate to specific service. - - * [new] swift_objects - - Context pre creates swift objects for future usage in scenarios - - * [update] sahara_cluster - - It supports proxy server which allows to use single floating IP for - whole cluster. - - * [fix] cleanup - - Fix cleanup of networks remove vip before port. - -Bug fixes -~~~~~~~~~ - -**37 bugs were fixed, the most critical are**: - -* Follow symlinks in plugin discovery -* Use sed without -i option for portability (install_rally.sh) -* Fixed race in rally.common.broker -* Fixed incorrect iteration number on "Failures" Tab -* Fixing issue with create_isolated_networks = False -* Fix docker build command - -Documentation -~~~~~~~~~~~~~ - -Fixed some minor typos and inaccuracies. - -Thanks -~~~~~~ - -We would like to thank Andreas Jaeger for ability to provide Python 2.6 support in this release. diff --git a/doc/release_notes/archive/v0.10.0.rst b/doc/release_notes/archive/v0.10.0.rst deleted file mode 100644 index b8d83364..00000000 --- a/doc/release_notes/archive/v0.10.0.rst +++ /dev/null @@ -1,520 +0,0 @@ -============= -Rally v0.10.0 -============= - -Overview --------- - -+------------------+-----------------------+ -| Release date | **10/20/2017** | -+------------------+-----------------------+ - -* Ability to use OpenStack deployments without admin credentials -* Better validation process of tasks -* New task format (with an ability to set description for workloads) -* New JSON report -* ElasticSearch exporter -* `OSProfiler support - `_ -* Restructure of configuration options. - -Details -------- - -Command Line Interface -~~~~~~~~~~~~~~~~~~~~~~ - -* Introduce `rally task import - `_ - command for importing task results into database. - -* Extend tags support for tasks. Now you can specify several tags for a single - task using `--tag argument - `_. - Also filtering tasks by tags is now available. - -* Move DB category from ``rally-manage db`` to `rally db - `_ and - introduce `rally db show - `_ - command for printing the used connection string. - -Deployments -~~~~~~~~~~~ - -This release we started a huge work related to simplification of deployment -component of Rally. There is a good progress which includes several nice -features: - -* The format. - "ExistingCloud" deployment type covers 99.99% cases and is used as a base for - all new things. Also, it will be extended to support different platforms - soon. The new format looks like (for OpenStack case): - - .. code-block:: json - - { - "openstack": { - "admin": { - "username": "admin", - "password": "changeme", - "tenant_name": "foo", - }, - "auth_url": "https://example.com", - } - } - - -* admin user is optional in case of setting existing users. - From the beginning, setting admin credentials was a required section of Rally - deployment configuration. Even with introducing existing users feature, this - behaviour left. - Finally, we finished a big code refactoring and admin credential become - optional section. If a set of plugins for particular workload doesn't require - admin user, you can launch this task at deployment without setting it. - - The information about the requirements of plugins you can find at - `Plugins Reference page - `_ (see - ``Requires platform(s):`` section at the bottom of each plugin). - -* Originally, Rally project was designed to check performance of OpenStack and - we succeeded in building awesome tool. We do not plan to stop and just want - to inform about our future plans to expand a number of supported platforms. - Subscribe to our `GitHub organization - `_ to not miss new plugins. - -Task component -~~~~~~~~~~~~~~ - -* The new task format is introduced. It includes a bunch of improvements, - unification, etc. All the docs and samples will be updated soon. - - As for now, you can check `a spec - `_ - for this big change. - -* SLA failure_rate max=0 become a default if nothing else is specified. - -* Totally reworked atomic actions. The atomic actions now supports nested - actions which allows to measure durations inside the scenario even more - precise. You can find them in HTML report or in our new json report - (see ``rally task report --json``). - -* Generation of names for new resources takes care about particular workload - id, so it helps to provide a better cleanup and prepare for new feature - - disaster cleanup. - -Plugins -~~~~~~~ - -We started supporting discovering plugins by entry-points, so you can easily -deliver your custom plugins as a simple python package. - -To make you package after-discoverable, you need to specify the proper -entry-point at your setup.cfg file: - - .. code-block:: - - rally_plugins = - path=package_name - -**Deployment Engines**: - -Remove serverproviders & rarely used deployers - -Unfortunately, seems like nobody is using deployers for deploying -their clouds and mostly people would like just to execute their code. - -1) Remove server provides -2) Remove engines that uses server providers - -**OpenStack clients**: - -* Deprecate EC2 client. It wasn't used in any of plugins and doesn't support - keystone v3 - -* Move ``rally.osclients`` module to ``rally.plugins.openstack.oscliens`` - -**Scenarios**: - -The old way to describe scenario plugin via method is finally removed. -Initially Rally scenario plugins were methods of special class, like below: - - .. code-block:: python - - from rally.task import scenario - - class SomeBasicClass(scenario.Scenario): - - @scenario.configure() - def some_scenario(self, arg1): - """An implementation of SomeBasicClass.foo scenario.""" - - @scenario.configure() - def another_scenario(self): - """Implementation of another scenario, SomeBasicClass.bar.""" - -However to unify scenarios with other plugins we moved to model where -plugin is class. It was done long time ago. - - .. code-block:: python - - from rally.task import scenario - - @scenario.configure(name="CustomName") - class Some(scenario.Scenario): - - def run(self, arg1): - """An implementation of the scenario.""" - -We had a bunch of code that was used for transition and backward compatibility -that we finally removed. - - -* *NEW!!* - - - `CinderQos.create_and_get_qos - `_ - - - `CinderQos.create_and_list_qos - `_ - - - `CinderQos.create_and_set_qos - `_ - - - `CinderQos.create_qos_associate_and_disassociate_type - `_ - - - `CinderVolumeTypes.create_and_get_volume_type - `_ - - - `CinderVolumeTypes.create_and_list_volume_types - `_ - - - `CinderVolumeTypes.create_and_update_encryption_type - `_ - - - `CinderVolumeTypes.create_and_update_volume_type - `_ - - - `CinderVolumeTypes.create_get_and_delete_encryption_type - `_ - - - `CinderVolumeTypes.create_volume_type_add_and_list_type_access - `_ - - - `Dummy.openstack - `_ - - - `GlanceImages.create_and_deactivate_image - `_ - - - `GlanceImages.create_and_download_image - `_ - - - `GlanceImages.create_and_get_image - `_ - - - `GlanceImages.create_and_update_image - `_ - - - `K8sPods.create_pods - `_ - - - `K8sPods.create_rcs - `_ - - - `K8sPods.list_pods - `_ - - - `ManilaShares.create_and_extend_share - `_ - - - `ManilaShares.create_and_shrink_share - `_ - - - `ManilaShares.create_share_then_allow_and_deny_access - `_ - - - `NeutronBGPVPN.create_and_delete_bgpvpns - `_ - - - `NeutronBGPVPN.create_and_list_bgpvpns - `_ - - - `NeutronBGPVPN.create_and_list_networks_associations - `_ - - - `NeutronBGPVPN.create_and_list_routers_associations - `_ - - - `NeutronBGPVPN.create_and_update_bgpvpns - `_ - - - `NeutronBGPVPN.create_bgpvpn_assoc_disassoc_networks - `_ - - - `NeutronBGPVPN.create_bgpvpn_assoc_disassoc_routers - `_ - - - `NeutronNetworks.create_and_show_ports - `_ - - - `NeutronNetworks.create_and_show_routers - `_ - - - `NeutronNetworks.create_and_show_subnets - `_ - - - `NeutronNetworks.set_and_clear_router_gateway - `_ - - - `NeutronSecurityGroup.create_and_delete_security_group_rule - `_ - - - `NeutronSecurityGroup.create_and_list_security_group_rules - `_ - - - `NeutronSecurityGroup.create_and_show_security_group - `_ - - - `NeutronSecurityGroup.create_and_show_security_group_rule - `_ - - - `NovaServerGroups.create_and_delete_server_group - `_ - - - `NovaServerGroups.create_and_get_server_group - `_ - - - `NovaServers.boot_and_get_console_url - `_ - - - `NovaServers.boot_server_and_attach_interface - `_ - - - `NovaServers.boot_server_and_list_interfaces - `_ - - - `NovaServers.boot_server_attach_volume_and_list_attachments - `_ - -* *UPDATED!!* - - - The new argument ``properties`` is added to scenario - `IronicNodes.create_and_list_node - `_ - - -* *DELETED* - -Fuel and Nova-Network are not alive any more. So we removed those scenarios. -If any of those scenarios a critical for you, please contact us. - - - `FuelEnvironments.create_and_delete_environment - `_ - - - `FuelEnvironments.create_and_list_environments - `_ - - - `FuelNodes.add_and_remove_node - `_ - - - `NovaFloatingIpsBulk.create_and_delete_floating_ips_bulk - `_ - - - `NovaFloatingIpsBulk.create_and_list_floating_ips_bulk - `_ - - - `NovaNetworks.create_and_delete_network - `_ - - - `NovaNetworks.create_and_list_networks - `_ - - - `NovaSecGroup.boot_and_delete_server_with_secgroups - `_ - - - `NovaSecGroup.boot_server_and_add_secgroups - `_ - - - `NovaSecGroup.create_and_delete_secgroups - `_ - - - `NovaSecGroup.create_and_list_secgroups - `_ - - - `NovaSecGroup.create_and_update_secgroups - `_ - -**Validators**: - -The validators refactoring was a long-term task which blocked us to abandon -alignment to only OpenStack platform and requirements of setting admin -credentials. In this release, we made a great progress and fixed a lot of -issues and blockers which made possible to refactor validators. -Now validation step is equal for all types of plugins (Scenario, SLA, Context, -Hooks, Runners, etc). - -The old way to add validator for scenario is deprecated. The new unified way -looks like: - - .. code-block:: python - - import time - - from rally.common import validation - from rally.task import scenario - - @validation.add("number", param_name="timeout", minval=0) - @scenario.configure(name="Foo.bar") - class FooScenario(scenario.Scenario): - def run(self, timeout): - time.sleep() - -The old validators from ``rally.task.validators`` module is deprecated too, see -equivalents which can be used with ``add`` decorator: - - - required_openstack --> `required_platform - `_ - with setting platform argument to "openstack" - - - external_network_exists ->`external_network_exists - `_ - - - file_exists ->`file_exists - `_ - - - flavor_exists ->`flavor_exists - `_ - - - image_exists ->`image_exists - `_ - - - image_valid_on_flavor ->`image_valid_on_flavor - `_ - - - number ->`number - `_ - - - required_api_versions ->`required_api_versions - `_ - - - required_cinder_services ->`required_cinder_services - `_ - - - required_clients ->`required_clients - `_ - - - required_contexts ->`required_contexts - `_ - - - required_neutron_extensions ->`required_neutron_extensions - `_ - - - required_param_or_context ->`required_param_or_context - `_ - - - required_services ->`required_services - `_ - - - restricted_parameters ->`restricted_parameters - `_ - - - validate_heat_template ->`validate_heat_template - `_ - - - volume_type_exists ->`volume_type_exists - `_ - - - workbook_contains_workflow -> `workbook_contains_workflow - `_ - - - network_exists is removed, since we do not find any customers for it. - Please contact us if it was useful for you. - - - validate_share_proto is removed in favor of enum validator - -Fixed bugs -~~~~~~~~~~ - -* [plugins] JSON schema of `servers - `_ - context allows to transmit a list of nics in two formats. First one is a - format that novaclient expects to see (each network should be represented - like ``{"nic-id": "the id of the network"}``). The second one is more - user-friendly - just list of strings (each network should be represented - just by id of the network). Unfortunately, the second case was not covered - by our code base. - - Also, the first described format works in limited cases due to bad - serialization. - - `Launchpad bug-report #1695245 - `_ - -* [deployment] ~/rally/.openrc not working for keystone v3 - - `Launchpad bug-report #1683820 - `_ - -* [plugins] Failed to list volumes in case of missed name in the object. - -* [backported into 0.9.1][deployment] Credentials is not updated as soon as - deployment is recreated. Need to call recreate request twice. - - `Launchpad bug-report #1675271 - `_ - -* [backported into 0.9.1][plugins] Scenario `IronicNodes.create_and_list_node - `_ - had a wrong check that list of all nodes contains newly created one. - -* [backported into 0.9.1][task][cleanup] Do not remove quotas in case of - existing users - -* [backported into 0.9.1][task][cleanup] Various traces of neutron resources - -* [backported into 0.9.1][core] Keystone v3, authentication error for Rally - users if the value of project_domain_name of admin user isn't equal "default" - - `Launchpad bug-report #1680837 - `_ - -* [backported into 0.9.1][task] Scenario `NovaHosts.list_and_get_hosts - `_ - obtains hostname for all hosts. But it fails in some environments if host is - not compute. - - `Launchpad bug-report #1675254 - `_ - -* [backported into 0.9.1][verification] Rally fails to run on systems on which - python-virtualenv is not installed - - `Launchpad bug-report #1678047 - `_ - -* [backported into 0.9.1][verification] CLI `rally verify rerun - `_ - fails with TypeError due to wrong integration with Rally API. - -* [plugins] Rally fails while creating neutron router on the clouds where - ext-gw-mode extension is not installed. - -* [plugins] Scenario `CinderVolumes.create_nested_snapshots_and_attach_volume - `_ - fails on a big load due to the fact that one server is used for several - iterations. In such case we are facing 2 issues: the maximum number of - volumes per VM is 26 (which is a number of available names for volumes); - detaching volume of one iteration can block attaching of other iterations. - - `Launchpad bug-report #1708160 - `_ - - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.10.1.rst b/doc/release_notes/archive/v0.10.1.rst deleted file mode 100644 index e1051c93..00000000 --- a/doc/release_notes/archive/v0.10.1.rst +++ /dev/null @@ -1,47 +0,0 @@ -============= -Rally v0.10.1 -============= - -Overview --------- - -+------------------+-----------------------+ -| Release date | **12/05/2017** | -+------------------+-----------------------+ - -Details -------- - -This release is fully backward compatible with Rally 0.10.0 and contains just -bug fixes. - -Fixed bugs -~~~~~~~~~~ - -* [deployment] Suppress deprecation warning about an old format in case of - using `--fromenv option of rally deployment create - `_ - -* [deployment] Failure `rally deployment show - `_ - while displaying the information about deployment with a config in an old - format. - -* [task] New json report processed the hook results in a wrong way - - `Launchpad bug-report #1734336 - `_ - -* [task] Failure while generating trends reports in case of failures in setup - method of any context - - `Launchpad bug-report #1732193 - `_ - -* [task] Failure to export results in ElasticSearch 5.x cluster in case of - extra ``/`` in the end of destination url. - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.11.0.rst b/doc/release_notes/archive/v0.11.0.rst deleted file mode 100644 index 5778cd7a..00000000 --- a/doc/release_notes/archive/v0.11.0.rst +++ /dev/null @@ -1,263 +0,0 @@ -============= -Rally v0.11.0 -============= - -Overview --------- - -+------------------+-----------------------+ -| Release date | **02/16/2018** | -+------------------+-----------------------+ - -* Stabilize Rally releases (see requirements section) -* Publishing docker images is returned! -* Environment introduction (see Deployment section) - -Details -------- - -Requirements -~~~~~~~~~~~~ - -As for long time we tried to make our releases stable and workable even after -a year from release date. For this purpose, upper limits for all our -requirements were used. Such solution helped to make releases more stable, but -it did not work well and created more pain than a profit. -The main issue was related to new releases of not-direct rally -dependencies which can be incompatible which each other. - -From Rally 0.11.0 we are stopping adding upper version limits for our -`requirements -`_ (this does -not apply to cases when we sure that some new releases of dependency broke us -for sure). - -As alternative solution, the `upper-constraints file -`_ is -selected. It is a file with a list of packages and their versions which we used -in our CI while testing. Versions from this list are suggested to use in -production. -Please note, that it also contains not direct Rally dependencies (dependencies -of rally dependencies and dependencies of dependencies of rally dependencies -as well) and packages which possibly doesn't relate to Rally at all. - -The example ou usage: - - .. console:: bash - - $ git clone https://github.com/openstack/rally && cd rally - $ pip install . --constraint upper-constraints.txt - -Logging -~~~~~~~ - -The default value of ``use_stderr`` option of Rally config is changed to -**True**. It is done to ensure that json output of rally commands will not be -messed with logging and integration with third-party tools and scripts should -become simpler. - -Docker -~~~~~~ - -Unfortunately, we lost access to `rallyforge organization at DockerHub -`_, so our docker images were not -published anywhere officially for some time. Docker Support did not help us a -lot. At least, the original repo is removed now and we can start new -organization at DockerHub from the scratch. - -The new docker images for Rally+OpenStack plugins with an updated HowTo you -can find at `xrally/xrally-openstack repo -`_. It contains all Rally -releases + latest tag which maps to master branch. - - -Command Line Interface -~~~~~~~~~~~~~~~~~~~~~~ - -* Introduce `rally db ensure - `_ - command. It is going to create Rally DB if it doesn't exist, otherwise it - does nothing. - -* Various improvements for `rally db - `_. - Such as printing results of operations as well as connection string to DB, - hiding credentials by default, etc. - -* Various changes in returning error codes. Error codes become different for - different exceptions. Also, `rally task start - `_ - began to return 1 in case of any runtime issues and 2 if any workload doesn't - pass it's SLA. - -* The new CLI for new component (see ``Environments & Deployments`` section - for more details) - `rally env - `_ - -Environments & Deployments -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Deployment Component is fundamental part of Rally which is used by Task and -Verification components. Whereas Task and Verification components experienced -redesigns (some parts were redesigned even several times), Deployment component -was only extended over the time. -Currently, we are at the point when further extending requires much work and -user experience become worth. It is a hard decision, as like we had done with -Verification component in Rally 0.8, the Deployment component is re-written -from the scratch. To be clear, the new version of the component has not only -the new code, but the new name as well - it is the Environment. - -We are at the stage when Rally is suitable not only for OpenStack but for -various platforms and systems. The Environment is some entity against which -Rally should perform load. The specific Environment can include credentials for -OpenStack or for Kubernetes or for both. The Environment is a way to describe -the complex system/cloud which all of us have. As well it can be used for -simple systems as easy as for complex. - -If you are regular Rally user which tests OpenStack clusters, nothing is -changing for you at this moment. `rally deployment -`_ -CLI still works and it is not even deprecated. It will be kept for a while. -But you can start looking at our new CLI `rally env -`_ . - -As for writing plugins for external platforms - we are working on updating our -documentary. - -Task component -~~~~~~~~~~~~~~ - -* The json results are extended with context execution stats (i.e when and - which context was executed, how much time setup and cleanup methods took, - etc). Also, `volumes@openstack - `_ - and - `volume_types@openstack - `_ - are ported to include detailed stats of executed actions. In further - releases, all existing contexts will be adopted to the similar behavior. - -* Better OSProfiler integration - - Rally environment&deployment config began accept - ``profiler_conn_str`` property which is used to generate OSProfiler native - html-report and to embed it to Rally's html-report. - -* HTML report is extended to include a timestamp of failures. - -Plugins -~~~~~~~ - -As it mentioned above, the Deployment Component will be replaced soon. Such -decision led to abandoning all deployment-related plugins (engines, providers, -etc). - - -**Scenarios**: - -* *NEW!!* - - `NovaServers.boot_server_attach_volume_and_list_attachments - `_ - - -* *UPDATED!!* - - - Extend several Nova&Neutron related scenarios with - ``create_floating_ip_args`` parameter - - `NovaServers.boot_and_associate_floating_ip - `_ - `NovaServers.boot_server_associate_and_dissociate_floating_ip - `_ - - - Modify Bgpvpn scenarios to test true bgpvpn - - All Bgpvpn scenarios began to boot a server to add active port in the - network associated to the bgpvpn which allows to test not only the record in - the database, but true bgpvpn - - -**Contexts**: - -*UPDATED!!* - -`network@openstack -`_ -context is extended with ability to specify external router information. - -Fixed bugs -~~~~~~~~~~ - -* [backported into 0.10.1][deployment] Suppress deprecation warning about an - old format in case of using `--fromenv option of rally deployment create - `_ - -* [backported into 0.10.1][deployment] Failure `rally deployment show - `_ - while displaying the information about deployment with a config in an old - format. - -* [backported into 0.10.1][task] New json report processed the hook results in - a wrong way - - `Launchpad bug-report #1734336 - `_ - -* [backported into 0.10.1][task] Failure while generating trends reports in - case of failures in setup method of any context - - `Launchpad bug-report #1732193 - `_ - -* [backported into 0.10.1][task] Failure to export results in ElasticSearch 5.x - cluster in case of extra ``/`` in the end of destination url. - -* [deployment] OpenStack deployment creation with `--fromenv - `_ - option used old deprecated format. - -* [verify] Rally did not support creating verifiers from Gerrit/SSH source. - - `Launchpad bug-report #1737529 - `_ - -* [task][openstack] Removing default security group in users@openstack context - did not take into account that neutron can return multiple resources in some - configuration instead of one security group which relates to requested - tenant. - -* [task][openstack] Existing openstack users get their roles un-assigned if - they are equal to what roles@openstack context is configured to assign. - - `Launchpad bug-report #1720270 - `_ - -* [task][openstack] Validation step ignores roles@openstack context and - marks as "invalid" valid cases - - Some actions in openstack can be performed only if the user has specific - role. Since these roles can be different in different OpenStack installations - Rally has `roles@openstack context - `_ - context which can assign roles to the users. - Validation step did not check for specified roles in workload config and made - wrong assumption about accessibility of resources - - `Launchpad bug-report #1539878 - `_ - -* [task][openstack] Wrong identifiers were used for filtering Mistral resources - while cleanup step. - -* [task][openstack] `NovaServers.boot_and_live_migrate_server - `_ - does wrong target host selection - - `Launchpad bug-report #1734914 - `_ - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.2.0.rst b/doc/release_notes/archive/v0.2.0.rst deleted file mode 100644 index a09f9b7d..00000000 --- a/doc/release_notes/archive/v0.2.0.rst +++ /dev/null @@ -1,165 +0,0 @@ -============ -Rally v0.2.0 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **48** | -+------------------+-----------------------+ -| Bug fixes | *6** | -+------------------+-----------------------+ -| Dev cycle | **19 days** | -+------------------+-----------------------+ -| Release date | **1/11/2015** | -+------------------+-----------------------+ - - -Details -------- - -This release, as well as all previous ones, includes a lot of internal and -external changes. Most important of them are listed below. - -.. warning:: Release 0.2.0 doesn't support python 26 - - -Deprecations -~~~~~~~~~~~~ - -* Option --system-wide-install for ``rally verify start`` was deprecated in - favor of --system-wide - -* `rally show` commands were deprecated because of 3 reasons: - - It blocks us to make Rally generic testing tool - - It complicates work on Rally as a Service - - You can always use standard OpenStack clients to do the same - -Rally Verify -~~~~~~~~~~~~ - -* Add "xfail" mechanism for Tempest tests. - - This mechanism allows us to list some tests, that are expected to fail, - in a YAML file and these tests will have "xfail" status instead of "fail". - - Use new argument "--xfails-file" of rally verify start command. - - -Rally Task -~~~~~~~~~~ - -* --out argument of `rally task report` is optional now - - If you don't specify --out it will just print the resulting report - -* Better scenario output support - - As far as you know each scenario plugin are able to return data as a dict. - This dict contained set of key-values {: } where each name - was line on graph and each number was one of point. Each scenario run adds - a single point for each line on that graph. - - This allows to add extra data to the Rally and see how some values were - changed over time. However, in case when Rally was used to execute some other - tool and collect it's data this was useless. - - To address this **Scenario.add_output(additive, complete)** was introduced: - - Now it is possible to generate as many as you need graphs by calling this - method multiple times. - There are two types of graph additive and complete. **Additive** is the same - as legacy concept of output data which is generated from results of all - iterations, **complete** are used when you would like to return whole chart - from each iteration. - - HTML report has proper sub-tabs *Aggregated* and *Per iteration* - inside *Scenario Data* tab. - - Here is a simple example how output can be added in any - scenario plugin: - - .. code-block:: python - - # This represents a single X point in result StackedArea. - # Values from other X points are taken from other iterations. - self.add_output(additive={"title": "How do A and B changes", - "description": ("Trend for A and B " - "during the scenario run"), - "chart_plugin": "StackedArea", - "data": [["foo", 42], ["bar", 24]]}) - # This is a complete Pie chart that belongs to this concrete iteration - self.add_output( - complete={"title": "", - "description": ("Complete results for Foo and Bar " - "from this iteration"), - "chart_plugin": "Pie", - "data": [["foo", 42], ["bar", 24]]}) - -Rally Certification -~~~~~~~~~~~~~~~~~~~ - - None. - - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - - [Spec][Implemented] improve_scenario_output_format - - https://github.com/openstack/rally/blob/master/doc/specs/implemented/improve_scenario_output_format.rst - - -Plugins -~~~~~~~ - -* **Scenarios**: - - * [new] DesignateBasic.create_and_update_domain - - * [improved] CinderVolumes.create_and_attach_volume - - .. warning:: Use "create_vm_params" dict argument instead of ``**kwargs`` - for instance parameters. - - -* **Context**: - - * [improved] images - - .. warning:: The min_ram and min_disk arguments in favor of image_args, - which lets the user specify any image creation keyword arguments they want. - - -Bug fixes -~~~~~~~~~ - -**6 bugs were fixed**: - - * #1522935: CinderVolumes.create_and_attach_volume does not accept additional - args for create_volume - - * #1530770: "rally verify" fails with error 'TempestResourcesContext' object - has no attribute 'generate_random_name' - - * #1530075: cirros_img_url in rally.conf doesn't take effective in - verification tempest - - * #1517839: Make CONF.set_override with parameter enforce_type=True by default - - * #1489059: "db type could not be determined" running py34 - - * #1262123: Horizon is unreachable outside VM when we are using DevStack + - OpenStack - - -Documentation -~~~~~~~~~~~~~ - - None. - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.3.0.rst b/doc/release_notes/archive/v0.3.0.rst deleted file mode 100644 index bdb7e452..00000000 --- a/doc/release_notes/archive/v0.3.0.rst +++ /dev/null @@ -1,188 +0,0 @@ -============ -Rally v0.3.0 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **69** | -+------------------+-----------------------+ -| Bug fixes | **7** | -+------------------+-----------------------+ -| Dev cycle | **29 days** | -+------------------+-----------------------+ -| Release date | **2/16/2016** | -+------------------+-----------------------+ - - -Details -------- - -This release, as well as all previous ones, includes a lot of internal and -external changes. Most important of them are listed below. - -.. warning:: In this release Rally DB schema migration is introduced. While - upgrading Rally from previous versions it is required now to run - ``rally-manade db upgrade``. Please see 'Documentation' section for details. - - -CLI changes -~~~~~~~~~~~ - -* .. warning:: [Removed] ``rally info`` in favor of ``rally plugin *``. - It was deprecated for a long time. - -* [Modified] ``rally deployment check`` now prints services, which don't have - names, since such services can be used via api_versions context. - -* .. warning:: [Modified] ``rally verify [re]install`` - option --no-tempest-venv was deprecated in favor of --system-wide - -* [Added] ``rally-manage db revision`` displays current revision of - Rally database schema - -* [Added] ``rally-manage db upgrade`` upgrades pre-existing Rally - database schema to the latest revision - -* [Added] ``rally-manage db downgrade`` to downgrades existing Rally - database schema to previous revision - -* [Added] ``rally task export`` exports task results to external - services (only CLI command introduced, no real service support - implemented yet, however one could write own plugins) - -* [Added] ``rally verify export`` exports verification results to - external services (only CLI command introduced, no real service support - implemented yet, however one could write own plugins) - -Rally Deployment -~~~~~~~~~~~~~~~~ - -* .. warning:: ``fuel`` deployment engine is removed since it was outdated and - lacked both usage and support - -Rally Task -~~~~~~~~~~ - -Add custom labels for "Scenario Output" charts - -* X-axis label can be specified to add_output() by - "axis_label" key of chart options dict. - The key is named "axis_label" but not "x_label" - because chart can be displayed as table, so we explicitly - mention "axis" in option name to make this parameter - useless for tables - -* Y-axis label can be specified to add_output() by - "label" key of chart options dict - In some cases this parameter can be used for rendering - tables - it becomes column name in case if chart with - single iteration is transformed into table - -* As mentioned above, if we have output chart - with single iteration, then it is transformed to table, - because chart with single value is useless - -* OutputLinesChart is added, it is displayed by - NVD3 lineChart() - -* Chart "description" is optional now. Description is - not shown if it is not specified explicitly - -* Scenario Dummy.add_output is improved to display labels - and OutputLinesChart - -* Fix: If Y-values are too long and overlaps chart box, - then JavaScript updates chart width in runtime to fit - width of chart graphs + Y values to their DOM container - -Rally Certification -~~~~~~~~~~~~~~~~~~~ - - None. - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* [Spec][Introduced] Export task and verification results to external services - - https://github.com/openstack/rally/blob/master/doc/specs/in-progress/task_and_verification_export.rst - -* [Spec][Implemented] Consistent resource names - - https://github.com/openstack/rally/blob/master/doc/specs/implemented/consistent_resource_names.rst - -* [Feature request][Implemented] Tempest concurrency - - https://github.com/openstack/rally/blob/master/doc/feature_request/implemented/add_possibility_to_specify_concurrency_for_tempest.rst - -Plugins -~~~~~~~ - -* **Scenarios**: - - - [added] VMTasks.workload_heat - - - [added] NovaFlavors.list_flavors - - - [updated] Flavors for Master and Worker node groups are now - configured separately for SaharaCluster.* scenarios - -* **Context**: - - - .. warning:: [deprecated] rally.plugins.openstack.context.cleanup - in favor of rally.plugins.openstack.cleanup - - - [improved] sahara_cluster - - Flavors for Master and Worker node groups are now - configured separately in ``sahara_cluster`` context - -Miscellaneous -~~~~~~~~~~~~~ - -* Cinder version 2 is used by default - -* Keystone API v3 compatibility improved - - - Auth URL in both formats http://foo.rally:5000/v3 - and http://foo.rally:5000 is supported for Keystone API v3 - - - Tempest configuration file is created properly according - to Keystone API version used - -* ``install_rally.sh --branch`` now accepts all git tree-ish, - not just branches or tags - -* VM console logs are now printed when Rally fails to connect to VM - -* Add support for Rally database schema migration (see 'Documentation' section) - -Bug fixes -~~~~~~~~~ - -**7 bugs were fixed**: - -* #1540563: Rally is incompatible with liberty Neutron client - - The root cause is that in Neutron Liberty client, - the _fx function doesn't take any explicit keyword parameter - but Rally is passing one (tenant_id). - -* #1543414: The `rally verify start` command fails when running - a verification against Kilo OpenStack - -* #1538341: Error in logic to retrieve image details in image_valid_on_flavor - -Documentation -~~~~~~~~~~~~~ - -* Add documentation for DB migration - - https://github.com/openstack/rally/blob/master/rally/common/db/sqlalchemy/migrations/README.rst - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.3.1.rst b/doc/release_notes/archive/v0.3.1.rst deleted file mode 100644 index 9d41b77a..00000000 --- a/doc/release_notes/archive/v0.3.1.rst +++ /dev/null @@ -1,47 +0,0 @@ -============ -Rally v0.3.1 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **9** | -+------------------+-----------------------+ -| Bug fixes | **6** | -+------------------+-----------------------+ -| Dev cycle | **2 days** | -+------------------+-----------------------+ -| Release date | **2/18/2016** | -+------------------+-----------------------+ - - -Details -------- - -This release is more about bug-fixes than features. - -.. warning:: Please, update 0.3.0 to latest one. - - -Features -~~~~~~~~ - -* Pass api_versions info to glance images context - -* [Verify] Don't create new flavor when flavor already exists - -Bug fixes -~~~~~~~~~ - -**6 bugs were fixed, the most critical are**: - -* #1545889: Existing deployment with given endpoint doesn't work anymore - -* #1547092: Insecure doesn't work with Rally 0.3.0 - -* #1547083: Rally Cleanup failed with api_versions context in 0.3.0 release - -* #1544839: Job gate-rally-dsvm-zaqar-zaqar fails since the recent Rally patch - -* #1544522: Non-existing "called_once_with" method of Mock library is used diff --git a/doc/release_notes/archive/v0.3.2.rst b/doc/release_notes/archive/v0.3.2.rst deleted file mode 100644 index d1901720..00000000 --- a/doc/release_notes/archive/v0.3.2.rst +++ /dev/null @@ -1,216 +0,0 @@ -============ -Rally v0.3.2 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **55** | -+------------------+-----------------------+ -| Dev cycle | **25 days** | -+------------------+-----------------------+ -| Release date | **3/14/2016** | -+------------------+-----------------------+ - - -Details -------- - -This release, as well as all previous ones, includes a lot of internal and -external changes. Most important of them are listed below. - - -CLI changes -~~~~~~~~~~~ - -* .. warning:: [Modified] Option '--tempest-config' for 'rally verify - reinstall' command was deprecated for removal. - -* .. warning:: [Removed] Option `--system-wide-install` was removed from - `rally verify` commands in favor of `--system-wide` option. - -* .. warning:: [Modified] Step of installation of Tempest during execution of - the `rally verify start` command was deprecated and will be removed in the - future. Please use `rally verify install` instead. - -* Rework commands.task.TaskCommands.detailed. Now output of the command - contains the same results as in HTML report. - -Rally Verify -~~~~~~~~~~~~ - -* Re-run failed Tempest tests - - Add the ability to re-run the tempest tests that failed in the last test - execution. Sometimes Tempest tests fail due to a special temporary condition - in the environment, in such cases it is very useful to be able to re-execute - those tests. - - Running the following command will re-run all the test that failed during - the last test execution regardless of what test suite was run. - - .. code-block:: bash - - rally verify start --failing - - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* `[Spec][Introduced] Refactoring scenario utils`__ - -__ https://github.com/openstack/rally/blob/master/doc/specs/in-progress/refactor_scenario_utils.rst - -* `[Spec] Deployment unification`__ - -__ https://github.com/openstack/rally/blob/master/doc/specs/in-progress/deployment_type.rst - -Plugins -~~~~~~~ - -* **Scenarios**: - - * [updated] Fix flavor for cloudera manager - - Cloudera manager need master-node flavor - - * [added] Add more Nova API scenarios - - Add support for listing nova hosts, agents, availability-zones - and aggregates. - - * [updated] Make sure VolumeGenerator uses the api version info while cleanup - - * Designate V2 - Add recordset scenarios - - Add create_and_(list|delete)_recordset scenarios - Remove the test also that checks the allowed methods, this is in order for - us to be able to have a private method _walk_pages that will do fetching of - pages for us vs attempting to fetch 1 giant list at once. - - * unify \*_kwargs name in scenarios - - When running a scenario, `kwargs` is used as default key-word arguments. - But in some scenarios, there are more and one services being called, and - we use xxx_kwargs for this case. - - However, some xxx_kwargs are not unified for same usage[0]. Unifying - these could avoid misleading for end users. Another improvement is to - add xxx_kwargs with empty settings for scenario config files. - - [0] http://paste.openstack.org/show/489505/ - - * .. warning:: Deprecated arguments 'script' and 'interpreter' were removed - in favor of 'command' argument. - - VM task scenarios executes a script with a interpreter provided through a - formatted argument called 'command' which expects the remote_path or - local_path of the script and optionally an interpreter with which the - script has to be executed. - -Miscellaneous -~~~~~~~~~~~~~ - -* Avoid using `len(x)` to check if x is empty - - This cases are using `len()` to check if collection has items. As - collections have a boolean representation too, directly check for true / - false. And fix the wrong mock in its unit test. - -* Fix install_rally.sh to get it to work on MacOSX - - On MacOSX, `mktemp` requires being passed a template. This change modifies - the calls to `mktemp` to explicitly pass a template so that the code works - on both MacOSX and linux. - -* Use new-style Python classes - - There are some classes in the code that didn't inherited from - nothing and this is an old-style classes. A "New Class" is the - recommended way to create a class in modern Python.A "New Class" - should always inherit from `object` or another new-style class. - - Hacking rule added as well. - -* Make Rally cope with unversioned keystone URL - - With the change, the client version that's returned is now determined by - the keystoneclient library itself based on whether you supply a URL with a - version in it or not. - -* Fix rally-mos job to work with mos-8.0 - - Also remove hardcoded values for some other jobs. - -* Add name() to ResourceManager - - This will allow us to perform cleanup based on the name. - -* Add task_id argument to name_matches_object - - This will be used to ensure that we are only deleting resources for a - particular Rally task. - -* Extend api.Task.get_detailed - - Extend api.Task.get_detailed with ability to return task data as dict with - extended results. - -Bug fixes -~~~~~~~~~ - -**The most critical fixed bugs are**: - -* #1547624: Wrong configuration for baremetal(ironic) tempest tests - -* #1536800: openrc values are not quoted - - The openrc file created after rally deployment --fromenv did not quote the - values for environment variables that will be exported. - -* #1509027: Heat delete_stack never exits if status is DELETE_FAILED - -* #1540545: Refactored atomic action in authenticate scenario - -* #1469897: Incompatible with Keystone v3 argument in service create scenario - -* #1550262: Different results in ``rally task detailed``, ``rally task report`` - and ``rally task status`` commands. - -* #1553024: Backward incompatible change in neutronclient(release 4.1.0) broke - Tempest config generation to support latest neutronclient. - -Documentation -~~~~~~~~~~~~~ - -* Add documentation for DB migration - -* Make documentation for output plugins - - * Add descriptive docstrings for plugins based on OutputChart - * Register these plugins in `Rally Plugins Reference`__ - -__ https://docs.openstack.org/rally/latest/plugins/plugin_reference.html - -* Documentation tox fix - - Added information about debugging unit test with tox. Replace 3 references - to py26 with py34 to reflect current rally tox configuration. - -* Change structure of rally plugin and plugin references page - -* Update the scenario development, runner and context sections - -* The design of `Rally Plugins Reference`__ page was improved - -__ https://docs.openstack.org/rally/latest/plugins/plugin_reference.html - -* New page was added - `CLI references`__ - -__ https://docs.openstack.org/rally/latest/cli_reference.html - -Thanks -~~~~~~ - -To Everybody! diff --git a/doc/release_notes/archive/v0.3.3.rst b/doc/release_notes/archive/v0.3.3.rst deleted file mode 100644 index 7fe551e8..00000000 --- a/doc/release_notes/archive/v0.3.3.rst +++ /dev/null @@ -1,112 +0,0 @@ -============ -Rally v0.3.3 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **20** | -+------------------+-----------------------+ -| Dev cycle | **10 days** | -+------------------+-----------------------+ -| Release date | **3/24/2016** | -+------------------+-----------------------+ - - -Details -------- - -A half of patches relate to Cleanup. We have once again proved that ideal -stuff can be improved. :) - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* `[Spec][Introduced] Improve atomic actions format`__ - -__ https://github.com/openstack/rally/blob/master/doc/specs/in-progress/improve_atomic_actions_format.rst - -Plugins -~~~~~~~ - -* **Cleanups**: - - * Use proper attribute to get heat stack name - - * Always assign a name to created images. - - This is necessary for name-based cleanup. If a name is not specified, one - will be generated automatically. - - * Improve filtering glance images in case of V2 API - - * Delete only images created by images context - - Since the images context allows creating images with arbitrary names, - name-based cleanup won't work for it, so we have to delete the exact list - of images that it created instead. - - * New config option to set cleanup threads - - Allow the user to change the number of cleanup threads via the rally - config. When scaling out to thousands of instances, the cleanup can take - forever with the static 20 threads. - - * Add inexact matching to name_matches_object - - This will support places where we create resources with names that start - with a given name pattern, but include some additional identifier - afterwards. For instance, when bulk creating instances, Nova appends a UUID - to each instance name. - -* **Scenarios**: - - * Add sample of template for testing for testing heat caching. - - * Introduced new scenario `Dummy.dummy_random_action`__. It is suitable for - demonstration of upcoming trends report. - -__ http://rally.readthedocs.org/en/latest/plugin/plugin_reference.html#dummy-dummy-random-action-scenario - -* **Contexts**: - - `api_versions`__ context was extended to support switch between Keystone V2 - and V3 API versions. Now it is possible to use one Rally deployment to check - both Keystone APIs. - -__ http://rally.readthedocs.org/en/latest/plugin/plugin_reference.html#api-versions-context - -* **Newcomer in the family**: - - All ResourceType classes are pluggable now and it is much easier to use and - extend them. - - .. warning:: Decorator ``rally.task.types.set`` is deprecated now in favor of - ``rally.task.types.convert``. - - -Bug fixes -~~~~~~~~~ - -* #1536172: rally deployment destroy failed with traceback for failed - deployments. At current moment it is impossible to delete deployment if for - some reason deployment engine plugin cannot be found, because exception will - be thrown. - -Documentation -~~~~~~~~~~~~~ - -* Remove extra link in `All release notes` - - Previously, two links for latest release were presented. - -* Update release notes for 0.3.2 - - * Fixed indents for warning messages - * Fixed all references - -Thanks -~~~~~~ - - To Everybody! diff --git a/doc/release_notes/archive/v0.4.0.rst b/doc/release_notes/archive/v0.4.0.rst deleted file mode 100644 index ea1618be..00000000 --- a/doc/release_notes/archive/v0.4.0.rst +++ /dev/null @@ -1,286 +0,0 @@ -============ -Rally v0.4.0 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **76** | -+------------------+-----------------------+ -| Bug fixes | **12** | -+------------------+-----------------------+ -| Dev cycle | **28 days** | -+------------------+-----------------------+ -| Release date | **4/18/2016** | -+------------------+-----------------------+ - - -Details -------- - -.. warning:: Rally DB schema was changed since previous release. - See `HOWTO `_ - about updating your database. - - -CLI changes -~~~~~~~~~~~ - -* Add status messages of db migration process - -* Display task errors in human-friendly form - -* Support OS_PROJECT_NAME as well as OS_TENANT_NAME - -Messages -~~~~~~~~ - -* Removed deprecation warning in case of transmitted "name" attribute while - creation neutron resources. - - .. warning:: Deprecated code was deleted. - -* Suppress warning insecure URL messages - - Do not spam end users by insecure URL messages because it is - quite valid case in testing process - -Database -~~~~~~~~ - -While preparing for deployment refactoring: - -* db schema was changed; -* migration with new column `credentials` to deployment model was added; -* columns `users` and `admin` were dropped. - -Rally Task -~~~~~~~~~~ - -* Remove deprecated scenario output mechanism via returing value - - .. warning:: Deprecated code was deleted. - -* Friendlier error message with empty task file - - This is particularly useful when a Jinja2 template results in an empty - task. The current error message isn't very helpful: - - Task config is invalid: `'NoneType' object has no attribute 'get'` - -* Add Heat template validator - -Plugins -~~~~~~~ - -**Scenarios**: - -* Extend VM bind actions with "pause_unpause", "suspend_resume", "lock_unlock", - "shelve_unshelve". - -* Add exact error message into `VMTasks.runcommand_heat scenario`__ - -__ http://rally.readthedocs.org/en/0.4.0/plugin/plugin_reference.html#vmtasks-runcommand-heat-scenario - -* Add heat scenarios: output-show, output-list - - Current patch contains 4 scenarios from heat repo: - - `output-show for old algorithm - `_ - - `output-show for new algorithm - `_ - - `output-list for old algorithm - `_ - - `output-list for new algorithm - `_ - - -**Contexts**: - -* Reduce default speed of users creation in users context from 30 to 20 - by default. - -**SLAs**: - -* *NEW!!* MaxAverageDurationPerAtomic : Maximum average duration of one - iterations atomic actions in seconds. - - `Plugin Reference `_ - -**Reports**: - -* Improve results calculation in charts.Table - -* Use int instead of float for Y axis. It's number of parallel iterations and - it can't be float. - -* Remove accuracy that makes no sense, and creates a lot of noise on this graph - -* Include failed iterations as well, otherwise we will calculate load - incorrectly - -* Graph should start from 0 (begging of experiment) - -* Add 2 points at the end of graph to get at the end of graph 0 iterations - in parallel - -**Task Exporter**: - -In previous release we introduced new mechanism to export results in various -external systems and various formats. - -In this release, we added first plugin for this stuff - `file_exporter` - -**Services**: - -Remove hardcoded timeout from heat service - -**Utils**: - -Make glance web uploads streamable - -Without this change entire file get's downloaded into memory and can cause -issues. - -Rally Verify -~~~~~~~~~~~~ - -* Set time precision to 3 digits (instead of 5) after dot. - -* Don't use "--parallel" flag when concurrency == 1 - - If concurrency equals to 1, it means that we use only one thread to run - Tempest tests and the "--parallel" flag is not needed. - -Plugin for DevStack -~~~~~~~~~~~~~~~~~~~ - -* Support to be enabled with different plugin name - - Allow rally to be installed by devstack through a different plugin - name, e.g: - - .. code-block:: bash - - enable_plugin test-rally http://github.com/rally/rally.git master - -* Removed uncalled code - - Devstack won't "source plugin.sh source" any more. - -Bug fixes -~~~~~~~~~ - -**12 bugs were fixed**: - -* X-Fail mechanism did not work for TestCase which failed on setUp step - - If Tempest fails in a test's setUpClass(), there is only one subunit event - for each TestCase. In this case, Rally did not check partial test with x-fail - list and marked test as "fail" insted of "x-fail". - - `Launchpad bug-report #1568133 - `_ - -* Weak isolation of scenario arguments between iterations - - Input arguments for sub-task were shared between all iterations. Rally team - found one scenario which modified mutable input variable. - - Affected scenario: NeutronNetworks.create_and_update_ports - -* Incompatible filters between V1 and V2 for Glance images listing - - Glance V1 and V2 have different filters. For example, "owner" is a separate - kwarg in V1, not a generic filter. Also, visibility has different labels in - different APIs. We modified our Glance wrapper to support Glance V2 format - of filters for both V1 and V2 - -* Wrong way to store validation errors - - Results of failed task validations saved in incorrect format. It broke and - made un-userfriendly `rally task detailed` command. - - `Launchpad bug-report #1562713 - `_ - -* Hardcoded task's status in `rally task results` - - If there are no results for task, `rally task results` printed message that - task has failed status, but it can be not true(tasks in running state do not - have results). - - `Launchpad bug-report #1539096 - `_ - -* Tempest context failed to create network resources - - While we merged improvement for keystoneclient, we used wrong way to obtain - tenant id in TempestContext. - - `Launchpad bug-report #1550848 - `_ - -* Tasks based on Tempest failed to parse execution time. - - There is an ability in Rally to launch tasks based on Tempest. Since launch - of Tempest is just subprocess, it is needed to parse subunit to set correct - atomic actions. - - There was an issue while converting task execution time. - - `Launchpad bug-report #1566712 - `_ - -* JSONSchema huge impact on task performance - - Before runner sent data to engine we were checking jsonschema. This operation - is very expensive and in some cases it can take a lot of time. - - Here are test results, with Dummy.dummy_output scenario, sleep 0.5s - (added manually), 8000 iterations, 400 in parallel: - - * on master branch before the fix: - Load duration: 117.659588099 - Full duration: 227.451056004 - - * on master before the fix but remove jsonschema validation in scenario: - Load duration: 12.5437350273 - Full duration: 128.942219973 - - * on this patch before the fix (pure python validation): - Load duration: 11.5991640091 - Full duration: 22.7199981213 - -* Wrong Calculation of running iterations in parallel - - Load profile chart was calculated wrongly. - It showed more running iterations in parallel than actually are running. - -* Rally did not show "missing argument" error raised by argparse while parsing - cli args - - `Launchpad bug-report #1562916 - `_ - -* Issue while checking required arguments in CLI - - There was a possible issue in case of several required arguments - - `Launchpad bug-report #1555764 - `_ - -* Prepare step of verification did not check visibility of obtained image - - When we request a list of images to choose one of them for tests, we should - make sure all images are active and they are PUBLIC. If images are not - public, we will have failures of Tempest tests as described in the bug. - - `Launchpad bug-report #1564431 - `_ - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.5.0.rst b/doc/release_notes/archive/v0.5.0.rst deleted file mode 100644 index 14bdd3cf..00000000 --- a/doc/release_notes/archive/v0.5.0.rst +++ /dev/null @@ -1,410 +0,0 @@ -============ -Rally v0.5.0 -============ - -Information ------------ - -+------------------+-----------------------+ -| Commits | **175** | -+------------------+-----------------------+ -| Bug fixes | **19** | -+------------------+-----------------------+ -| Dev cycle | **93 days** | -+------------------+-----------------------+ -| Release date | **7/20/2016** | -+------------------+-----------------------+ - - -Details -------- - -This release took much more time than we expected, but we have a lot of -reasons for such delay and if you look at our change-log, you will understand -them.:) - -Here is a quick introduction: - -* To make our releases as much as possible stable, we added upper limits for - each of our requirements; -* A lot of deprecated lines of code were removed, so be careful; -* Statistics trends for given tasks were introduced; -* Support for tempest plugins was added; -* Several new pages at docs. - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* `[Introduced && implemented] Introduce class-based scenario implementation `_ - -* `[Introduced] Rally Task Validation refactoring `_ - -* `[Introduced] Scaling & Refactoring Rally DB `_ - -* `[Introduced] SLA Performance degradation plugin `_ - -Logging -~~~~~~~ - -* disable urllib3 warnings only if the library provide them - -Database -~~~~~~~~ - -[doesn't require migration] -Transform DB layer to return dicts, not SQLAlchemy models - -Rally Deployment -~~~~~~~~~~~~~~~~ - -* Support single-AZ deployment - - This supports the case where OpenStack is deployed with a single AZ for both - controller(s) and compute(s), and not all hosts in the AZ that contains an - instance are guaranteed to have the nova-compute service. - -* Extend creation from environment with several new vars - - - OS_ENDPOINT_TYPE/OS_INTERFACE - - OS_USER_DOMAIN_NAME - - OS_PROJECT_DOMAIN_NAME - -* Improve devstack plugin for Keystone V3 - -Rally Task -~~~~~~~~~~ - -*NEW!!* Statistics trends for given tasks. - - -Rally Verify -~~~~~~~~~~~~ - -* Remove '--tempest-config' arg from 'reinstall' command - - .. warning:: Using `--tempest-config` is became an error from this release. - Use `rally verify genconfig` cmd for all config related stuff. - -* Don't install Tempest when `rally verify start` - - .. warning:: Use should use `rally verify install` cmd to install tempest now - -* Add ability to setup version of Tempest to install - - `CLI argument to setup version `_ - -* Configure 'aodh' service in 'service_available' section - -* Check existence of Tempest-tree in `rally verify discover` cmd - -* Make Tempest work with auth url which doesn't include keystone version - - Tempest needs /v2.0 and /v3 at the end of URLs. Actually, we can't fix - Tempest, so we extend our configuration module with workaround which allow - to specify auth_url without version in rally deployment config. - -* Use default list of plugins for sahara - -* Move tempest related options of rally configuration to separate section. - -* *NEW!!* Support for tempest plugins. - - `CLI argument to install them `_ - - -Plugins -~~~~~~~ - -In this release we are happy to introduce new entity - plugins Base classes - -We have a lot of base plugin entities: Context, Scenario, SLA and etc. -Sometimes plugins of different bases can have equal names(i.e ceilometer -OSClient and ceilometer Context). It is normal and we should allow such -conflicts. To support such cases we introduced new entity - plugin base. -Statements of plugin bases: - - - Each plugin base is unique entity; - - Names of plugin bases can't conflict with each other; - - Names of two or more plugins in one plugin base can't conflict with each - other(in case of same platform). - - Names of two or more plugins in different plugin base can conflict - -Current list of plugin bases: - - - rally.task.context.Context - - rally.task.scenario.Scenario - - rally.task.types.ResourceType - - rally.task.exporter.TaskExporter - - rally.task.processing.charts.Chart - - rally.task.runner.ScenarioRunner - - rally.task.sla.SLA - - rally.deployment.serverprovider.provider.ProviderFactory - - rally.deployment.engine.Engine - - rally.osclients.OSClient - -**OSClients** - -* *NEW!!* Support for Senlin client - -* *NEW!!* Support for Gnocchi client - -* *NEW!!* Support for Magnum client - -* *NEW!!* Support for Watcher client - -* Transmit endpoint_type to saharaclient - -**Scenarios**: - -* *NEW!!*: - - - `Authenticate.validate_ceilometer `_ - - `CinderVolumes.create_volume_from_snapshot `_ - - `CinderVolumes.create_volume_and_clone `_ - - `NovaFlavors.create_and_list_flavor_access `_ - - `NovaFlavors.create_flavor `_ - - `NovaServers.boot_and_update_server `_ - - `NovaServers.boot_server_from_volume_snapshot `_ - -* [Sahara] Add configs to MapR plugin - -* Extend CinderVolumes.create_and_upload_volume_to_image with "image" argument - - `Plugin Reference `_ - -* Deprecate Dummy.dummy_with_scenario_output scenario in favor of Dummy.dummy_output - - .. warning:: Dummy.dummy_with_scenario_output scenario will be removed after - several releases - - `Deprecated Plugin Reference `_ - `New Plugin Reference `_ - -* Extend CinderVolumes.create_volume_and_clone with nested_level - - Add nested_level argument for nested cloning volume to new volume - -* Extend `CinderVolumes.create_nested_snapshots_and_attach_volume - `_ - - Two new arguments were added: create_volume_kwargs and create_snapshot_kwargs - - .. warning:: All arguments related to snapshot creation should be transmitted - only via create_snapshot_kwargs. - -* Introduce new style of scenarios - class based. - - `Spec Reference `_ - -* Improve report for VMTasks.boot_runcommand_delete - -* [Sahara] Added 5.5.0 version for cdh-plugin and 1.6.0 version for spark - -* Extend boot_server_from_volume_and_delete, boot_server_from_volume, - boot_server_from_volume_and_live_migrate, boot_server_from_volume_snapshot - scenarios of NovaServers class with "volume_type" parameter. - -**Contexts**: - -* *NEW!!*: - - - `Cinder volume_types `_ - - `Murano environments `_ - - `Heat dataplane `_ - -* Use Broker Pattern in Keystone roles context - -* Use immutable types for locking context configuration - - Since context configuration passed to Context.__init__() was a mutable type - (dict or list), sometimes we had unexpected changes done by unpredictable - code (for example, in wrappers). - -* Add possibility to balance usage of users - - For the moment all users for tasks were taken randomly and there was no way - to balance them between tasks. It may be very useful when we have difference - between first usage of tenant/user and all consecutive. In this case we get - different load results. - - Therefore, "users" context was extended with new config option - 'user_choice_method' that defines approach for picking up users. - - Two values are available: - - random - - round_robin - - Default one is compatible with old approach - "random". - -* Make sahara_image and custom_image contexts glance v2 compatible - -* Extend servers context with "nics" parameter - -* Extend network context with "dns_nameservers" parameter - -* Extend volume context with "volume_type" parameter - -**Cleanup**: - -* Mark several cleanup resources as tenant_resource - - Nova servers and security groups are tenant related resources, but resource - decorator missed that fact which makes cleanup tries to delete one resources - several times. - -* Turn off redundant nova servers cleanup for NovaFlavors.list_flavors scenario - -* Add neutron cleanup for NeutronSecurityGroup.create_and_delete_security_groups - -**Exporter**: - -Rename task-exporter "file-exporter" to "file". - -.. warning:: "file-exporter" is deprecated and will be removed in further - releases. - -**Types**: - -Remove deprecated types. - -.. warning:: you should use rally.task.types.convert instead of - rally.task.types.set decorator - -**Validators** - -* Add a required_api_version validator -* Add validators for scenario arguments - -**Utils**: - -Use glance wrapper where appropriate to support compatibility between V1 and V2 - -Bug fixes -~~~~~~~~~ - -**19 bugs were fixed**: - -* Wrong arguments order of Keystone wrapper in case of V2 and V3 - -* AttributeError while disabling urllib3 warnings on old installations - - `Launchpad bug-report #1573650 - `_ - -* install_rally.sh script is failed while obtaining setuptools - -* "-inf" load duration in case of wrong runner plugin and failed start of - contexts - -* Strange input task in the report - - `Launchpad bug-report #1570328 - `_ - -* Wrong behaviour of boot_server_from_volume scenarios in case of booting - server from image. - - The arg of image must be None, when booting server from volume. Otherwise - still boot server from image. - - Affected scenarios: - NovaServers.boot_server_from_volume - NovaServers.boot_server_from_volume_and_delete - NovaServers.boot_server_from_volume_and_resize - NovaServers.boot_server_from_volume_and_live_migrate - - `Launchpad bug-report #1578556 - `_ - -* Weak validation of json schema of RPS runner - - JSON Schema of RPS runner doesn't have "required" field. It means that - users are able to pass wrong configs and we will have runtime error while - running task. - -* Rally doesn't take cacert setting while creating keystone session - - `Launchpad bug-report #1577360 - `_ - -* Heat scenarios fail when API uses TLS - - `Launchpad bug-report #1585456 - `_ - -* Example in comment of context manila_share_networks wrong - - `Launchpad bug-report #1587164 - `_ - -* There is no way to get UUID of a verification after it is created by - "rally verify start" or "rally verify import_results" when --no-use is set - - `Launchpad bug-report #1587034 - `_ - -* Exposed ssh timeout and interval in vm scenario - - `Launchpad bug-report #1587728 - `_ - -* Ceilometer scenario doesn't require "ceilometer" ctx - - `Launchpad bug-report #1557642 - `_ - -* "servers" context requires setting network id for multiple possible networks - found. - - `Launchpad bug-report #1592292 - `_ - -* nested_level data type incorrect in create_nested_snapshots_and_attach_volume - - `Launchpad bug-report #1594656 - `_ - -* Rally cleanup servers raises exception - - `Launchpad bug-report #1584104 - `_ - -* Stopping server is redundant before cold-migrating server - - `Launchpad bug-report #1594730 - `_ - -* existing_users context doesn't work in case of Keystone v3 - -* Whether validates flavor's disk or not depends on booting type of the instance - - `Launchpad bug-report #1596756 - `_ - -Documentation -~~~~~~~~~~~~~ - -* Re-use openstack theme for building docs outside rtd. - - `Rally Docs at docs.openstack.org - `_ - -* Add page for Verification component - - `RTD page for Verification component - `_ - -* Add glossary page - - `RTD page for Glossary - `_ - -* Adjust docs reference to "KeystoneBasic.authenticate" scenario - - `Step 6. Aborting load generation on success criteria failure - `_ - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.6.0.rst b/doc/release_notes/archive/v0.6.0.rst deleted file mode 100644 index 2649ac69..00000000 --- a/doc/release_notes/archive/v0.6.0.rst +++ /dev/null @@ -1,198 +0,0 @@ -============ -Rally v0.6.0 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **9/05/2016** | -+------------------+-----------------------+ - -Details -------- - -Common -~~~~~~ - -* Added Python 3.5 support -* Sync requirements with OpenStack global-requirements -* Start using latest way of authentication - keystoneauth library -* Start porting all scenario plugins to class-based view. - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* `[Implemented] SLA Performance degradation plugin `_ -* `[Proposed] New Tasks Configuration section - hook `_ - -Database -~~~~~~~~ - -* disable db downgrade api -* [require migration] upgrade deployment config - -Docker image -~~~~~~~~~~~~ - -* Add sudo rights to rally user - Rally is a pluggable framework. External plugins can require installation of - additional python or system packages, so we decided to add sudo rights. - -* Move from ubuntu:14.04 base image to ubuntu:16.04 . - Ubuntu 16.04 is current/latest LTS release. Let's use it. - -* pre-install vim - Since there are a lot of users who like to experiment and modify samples - inside container, rally team decided to pre-install vim - -* configure/pre-install bash-completion - Rally provides bash-completion script, but it doesn't work without installed - `bash-completion` package and now it is included in our image. - - -Rally Deployment -~~~~~~~~~~~~~~~~ - -* Add strict jsonschema validation for ExistingCloud deployments. All incorrect - and unexpected properties will not be ignored anymore. If you need to store - some extra parameters, you can use new "extra" property. -* Fix an issue with endpoint_type. - Previously, endpoint type was not transmitted to keystone client. In this - case, keystoneclient used default endpoint type (for different API calls it - can differ). Behaviour after the fix: - - - None endpoint type -> Rally will initialize all clients without setting - endpoint type. It means that clients will choose what default values for - endpoint type use by itself. Most of clients have "public" as default - values. Keystone use "admin" or "internal" by default. - - Not none endpoint type -> Rally will initialize all clients with this - endpoint. Be careful, by default most of keystone v2 api calls do not work - with public endpoint type. - - -Rally Task -~~~~~~~~~~ - -* [core] Iterations numbers in logging and reports must be synchronized. Now - they start from 1 . - -* [config] users_context.keystone_default_role is a new config option - (Defaults to "member") for setting default user role for new users in case - of Keystone V3. - -* [Reports] Embed Rally version into HTML reports - This adds Rally version via meta tag into HTML reports: - - - -* [Reports] Expand menu if there is only one menu group - -* [logging] Remove deprecated rally.common.log module - -* [Trends][Reports] Add success rate chart to trends report - -* [Reports] Hide menu list if there is no data at all - -Rally Verify -~~~~~~~~~~~~ - -* Updating Tempest config file - - - Some tests (for boto, horizon, etc.) were removed from Tempest and now there - is no need to keep the corresponding options in Tempest config file. - - - Some options in Tempest were moved from one section to another and we should - to do the corresponding changes in Rally to be up to date with the latest - Tempest version. - -* Adding '--skip-list' arg to `rally verify start` cmd - - `CLI argument for --skip-list `_ - -* *NEW!!*: - - - `Command for plugin listing `_ - - `Command to uninstall plugins `_ - -* Rename and deprecated several arguments for `rally verify start` cmd: - - - tests-file -> load-list - - xfails-file -> xfail-list - -Plugins -~~~~~~~ - -**Scenarios**: - -* Extend Sahara scenarios with autoconfig param - - Affected plugins: - - - `SaharaClusters.create_and_delete_cluster `_ - - `SaharaClusters.create_scale_delete_cluster `_ - - `SaharaNodeGroupTemplates.create_and_list_node_group_templates `_ - - `SaharaNodeGroupTemplates.create_delete_node_group_templates `_ - -* *NEW!!*: - - - `MonascaMetrics.list_metrics `_ - - `SenlinClusters.create_and_delete_cluster `_ - - `Watcher.create_audit_template_and_delete `_ - - `Watcher.create_audit_and_delete `_ - - `Watcher.list_audit_templates `_ - -* Rename **murano.create_service** to **murano.create_services** atomic action - -**SLA**: - -*NEW!!*: `performance degradation plugin `_ - -**Contexts**: - -* *NEW!!*: - - - `Monasca monasca_metrics `_ - - `Senlin profiles `_ - - `Watcher audit_templates `_ - -* Extend `manila_share_networks `_ - context with share-network autocreation support. - -* Extend `volumes `_ - context to allow volume_type to be None to allow using default value - -Bug fixes -~~~~~~~~~ - -* [existing users] Quota context does not restore original settings on exit - - `Launchpad bug-report #1595578 `_ - -* [keystone v3] Rally task's test user role setting failed - - `Launchpad bug-report #1595081 `_ - -* [existing users] context cannot fetch 'tenant' and 'user' details from cloud - deployment - - `Launchpad bug-report #1602157 `_ - -* UnboundLocalError: local variable 'cmd' referenced before assignment - - `Launchpad bug-report #1587941 `_ - -* [Reports] Fix trends report generation if there are n/a results - - -Documentation -~~~~~~~~~~~~~ - -* Add page about task reports - - `RTD page for reports `_ - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.7.0.rst b/doc/release_notes/archive/v0.7.0.rst deleted file mode 100644 index e49ac0ba..00000000 --- a/doc/release_notes/archive/v0.7.0.rst +++ /dev/null @@ -1,135 +0,0 @@ -============ -Rally v0.7.0 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **10/11/2016** | -+------------------+-----------------------+ - -Details -------- - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* [Used] Ported all rally scenarios to class base - - `Spec reference `_ - -* `[Implemented] New Plugins Type - Hook `_ - -Database -~~~~~~~~ - -.. warning:: Database schema is changed, you must run - `rally-manage db upgrade `_ - to be able to use old Rally installation with latest release. - -* [require migration] fix for wrong format of "verification_log" of tasks -* [require migration] remove admin_domain_name from OpenStack deployments - -Rally Deployment -~~~~~~~~~~~~~~~~ - -* Remove admin_domain_name from openstack deployment - Reason: admin_domain_name parameter is absent in Keystone Credentials. - - -Rally Task -~~~~~~~~~~ - -* [Trends][Reports] Use timestamps on X axis in trends report - -* [Reports] Add new OutputTextArea chart plugin - - New chart plugin can show arbitrary textual data on - "Scenario Stata -> Per iteration" tab. - - This finally allows to show non-numeric data like IP addresses, notes and - even long comments. - - Plugin `Dummy.dummy_output `_ - is also updated to provide demonstration. - -* [cli] Add version info to *rally task start* output - -* [api] Allow to delete stopped tasks without force=True - - It is reasonable to protect deletion of running tasks (statuses INIT, - VERIFYING, RUNNING, ABORTING and so on...) but it is strange to protect - deletion for stopped tasks (statuses FAILED and ABORTED). Also this is - annoying in CLI usage. - -* Added hooks and triggers. - - Hook is a new entity which can be launched on specific events. Trigger is - another new entity which processes events and launches hooks. - For example, hook can launch specific destructive action - just execute cli - command(we have sys_call hook for this task) and it can be launched by - simple trigger on specific iteration(s) or time (there is event trigger). - -Rally Verify -~~~~~~~~~~~~ - -Scenario tests in Tempest require an image file. Logic of obtaining this image -is changed: - -* If CONF.tempest.img_name_regex is set, Rally tries to find an image matching - to the regex in Glance and download it for the tests. -* If CONF.tempest.img_name_regex is not set (or Rally didn't find the image - matching to CONF.tempest.img_name_regex), Rally downloads the image by the - link specified in CONF.tempest.img_url. - -Plugins -~~~~~~~ - -**Scenarios**: - -* *Removed*: `Dummy.dummy_with_scenario_output `_ - - It was deprecated in 0.5.0 - - .. warning:: This plugin is not available anymore in 0.7.0 - -* *NEW!!*: - - - `MagnumClusterTemplates.list_cluster_templates `_ - - `MagnumClusters.list_clusters `_ - - `MagnumClusters.create_and_list_clusters `_ - - `NovaAggregates.create_aggregate_add_and_remove_host `_ - - `NovaAggregates.create_and_list_aggregates `_ - - `NovaAggregates.create_and_delete_aggregate `_ - - `NovaAggregates.create_and_update_aggregate `_ - - `NovaFlavors.create_and_get_flavor `_ - - `NovaFlavors.create_flavor_and_set_keys `_ - - `NovaHypervisors.list_and_get_hypervisors `_ - - `NovaServers.boot_server_associate_and_dissociate_floating_ip `_ - - `KeystoneBasic.authenticate_user_and_validate_token `_ - -**Contexts**: - -* *NEW!!*: - - - `Manila manila_security_services `_ - - `Magnum cluster_templates `_ - - `Magnum clusters `_ - -**OSClients**: - -Port all openstack clients to use keystone session. - -Bug fixes -~~~~~~~~~ - -* [tasks] rally task detailed incorrect / inconsistent output - - `Launchpad bug-report #1562713 `_ - - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.8.0.rst b/doc/release_notes/archive/v0.8.0.rst deleted file mode 100644 index 0630e4f5..00000000 --- a/doc/release_notes/archive/v0.8.0.rst +++ /dev/null @@ -1,232 +0,0 @@ -============ -Rally v0.8.0 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **1/25/2017** | -+------------------+-----------------------+ - -Details -------- - -Specs & Feature Requests -~~~~~~~~~~~~~~~~~~~~~~~~ - -* `[Implemented] Refactor Verification Component - `_ - -* `[Implemented] Scaling & Refactoring Rally DB - `_ - -Installation -~~~~~~~~~~~~ - -We switched to use bindep library for checking required system packages. -All our dependencies moved to separate file (like requirements.txt for python -packages) `bindep.txt -`_. - -Database -~~~~~~~~ - -.. warning:: Database schema is changed, you must run - `rally-manage db upgrade `_ - to be able to use old Rally installation with latest release. - -* change structure of database to be more flexible -* save raw task results in chunks (see raw_result_chunk_size option of - [DEFAULT] rally configuration section) -* add db revision check in rally API, so it is impossible to use rally with - wrong db now. - -Rally API -~~~~~~~~~ - -Single entry point for Rally API is added - rally.api.API . Old API classes -(``rally.api.Task``, ``rally.api.Verification``, ``rally.api.Deployment``) are -deprecated now. - -Rally CLI -~~~~~~~~~ - -* ``rally task sla_check`` is deprecated now in favor of - ``rally task sla-check`` - -* Deprecated category ``rally show`` was removed. - -* `rally plugin list` is extended with plugin base column - -Task Component -~~~~~~~~~~~~~~ - -- [Random names] scenario for checking performance of generate_random_name - method is added to our CI with proper SLA. Be sure, whatever number of random - names you need, it will not affect performance of Rally at all, we checked. - -- [atomic actions] scenario for checking performance of calculating atomic - actions is added to our CI with proper SLA. Be sure, whatever number atomics - you have in scenarios, it will not affect performance of Rally at all, we - checked. - -- [services] new entity is introduced for helping to provide compatibility - layer between different API versions of one service. - -Verification component -~~~~~~~~~~~~~~~~~~~~~~ - -We completely redesign the whole Verification component. For more details see -`our new docs for that component -`_ - -Unfortunately, such big change could not be done in backward compatible way, -so old code is not compatible with new one. See `HowTo migrate from -Verification component 0.7.0 to 0.8.0 -`_ - -Plugins -~~~~~~~ - -**Services**: - -* Glance: - - Switched from V1 to V2 API by default. - -* Keystone: - - - Transmit endpoint_type to keystoneclient - - Full keystone V3 support - -**Scenarios**: - -* *Updated*: - - - The meaning of the volume_type argument is changes in - `CinderVolumes.create_snapshot_and_attach_volume - `_ - scenario. It should contain actual volume type instead of boolean value to - choose random volume type. - - Extend `GlanceImages.create_image_and_boot_instances - `_ - with create_image_kwargs and boot_server_kwargs arguments. - -* *NEW!!*: - - - `CeilometerAlarms.create_and_get_alarm - `_ - - `CinderVolumeBackups.create_incremental_volume_backup - `_ - - `CinderVolumeTypes.create_and_delete_volume_type - `_ - - `CinderVolumeTypes.create_volume_type_and_encryption_type - `_ - - `CinderVolumes.create_and_accept_transfer - `_ - - `CinderVolumes.create_and_get_volume - `_ - - `CinderVolumes.create_volume_and_update_readonly_flag - `_ - - `CinderVolumes.list_transfers - `_ - - `CinderVolumes.list_types - `_ - - `KeystoneBasic.create_and_get_role - `_ - - `ManilaShares.create_and_list_share - `_ - - `ManilaShares.set_and_delete_metadata - `_ - - `MistralExecutions.create_execution_from_workbook - `_ - - `MistralExecutions.list_executions - `_ - - `NeutronLoadbalancerV2.create_and_list_loadbalancers - `_ - - `NeutronNetworks.create_and_show_network - `_ - - `NeutronNetworks.list_agents - `_ - - `NovaAggregates.create_aggregate_add_host_and_boot_server - `_ - - `NovaAggregates.create_and_get_aggregate_details - `_ - - `NovaFlavors.create_and_delete_flavor - `_ - - `NovaFlavors.create_flavor_and_add_tenant_access - `_ - - `NovaHosts.list_and_get_hosts - `_ - - `NovaHypervisors.list_and_get_uptime_hypervisors - `_ - - `NovaHypervisors.list_and_search_hypervisors - `_ - - `NovaHypervisors.statistics_hypervisors - `_ - - `NovaSecGroup.boot_server_and_add_secgroups - `_ - - `NovaServerGroups.create_and_list_server_groups - `_ - - `Quotas.nova_get - `_ - -**Hooks**: - -* *NEW!!*: - - - `fault_injection - `_ - -**Runners** - -* *Updated*: - - - `RPS runner - `_ - is extended with ability to increase 'rps' value by arithmetic progression - across certain duration. Now it can be also a dict specifying progression - parameters: - - .. code-block:: json - - rps": { - "start": 1, - "end": 10, - "step": 1, - "duration": 2 - } - - This will generate rps value: ``start, start + step, start + 2 * step, .., - end`` across certain 'duration' seconds each step. If iteration count not - ended at the last step of progression, then rps will continue to generate - with "end" value. Note that the last rps could be generated smaller. - -Fixed bugs -~~~~~~~~~~ - -* [hooks] incorrect encoding of stdout/stderr streams opened by sys_call hook - for py3 - -* [hooks] sorting Hook column at HTML report doesn't work - -* [tasks][scenarios][neutron] L3 HA: Unable to complete operation on subnet - - `Launchpad bug-report #1562878 `_ - -* [tasks] JSON report doesn't save order of atomics - -* [tasks][cleanup][nova] Failed to remove aggregate which has hosts in it - -* [tasks] `--abort-on-sla-failure - `_ - mechanism works only for current workload, but does not stop the next ones. - -* [hooks] hooks section isn't displayed in HTML report - - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.8.1.rst b/doc/release_notes/archive/v0.8.1.rst deleted file mode 100644 index 08bc4179..00000000 --- a/doc/release_notes/archive/v0.8.1.rst +++ /dev/null @@ -1,40 +0,0 @@ -============ -Rally v0.8.1 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **1/27/2017** | -+------------------+-----------------------+ - -Details -------- - -Fix for python requirements list. - -Plugins -~~~~~~~ - -**Scenarios**: - -* *Updated*: - - - Use new network for each subnet at - `NeutronNetworks.create_and_list_subnets - `_ - scenario. - -* *NEW!!*: - - - `CinderVolumeTypes.create_and_list_encryption_type - `_ - - - `Quotas.cinder_get - `_ - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.9.0.rst b/doc/release_notes/archive/v0.9.0.rst deleted file mode 100644 index e88846e4..00000000 --- a/doc/release_notes/archive/v0.9.0.rst +++ /dev/null @@ -1,163 +0,0 @@ -============ -Rally v0.9.0 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **3/20/2017** | -+------------------+-----------------------+ - -Details -------- - -Command Line Interface -~~~~~~~~~~~~~~~~~~~~~~ - -* `rally plugin list` now does not contain hidden plugins. - -Task component -~~~~~~~~~~~~~~ - -* Added check for duplicated keys in task files. - -* The order of subtasks (scenarios/workloads) is not ignored any more. You can - generate whatever you want load or use that feature for up the cloud (put - small scenario to the start of task to wake up the cloud before the real - load). - -* Information about workload creation is added to HTML-reports. - -* Task statuses is changed to be more clear and cover more cases: - - - ``verifying`` is renamed to ``validating``. - - ``failed`` is divided for 2 statuses - ``validation_failed``, which means - that task did not pass validation step, and ``crashed``, which means that - something went wrong in rally engine. - -* Our awesome cleanup become more awesome! The filter mechanism is improved to - discover resources in projects created only by Rally (it works for most of - resources, except several network-related ). It makes possible to run Rally - with existing users in real tenants without fear to remove something - important. - - -Verification component -~~~~~~~~~~~~~~~~~~~~~~ - -* Fixed an issue with missed tests while listing all supported tests of - specified verifier. - -* Fixed an issue with displaying the wrong version of verifier in case of - cloning from the local directory. - -* Extend `rally verify rerun - `_ - with ``--detailed``, ``--no-use``, ``--tag`` and ``--concurrency`` arguments. - -* Add output examples for `JSON - `_ and - `JUnit-XML - `_ - reporters. - -Plugins -~~~~~~~ - -**Contexts** - -* Extend cinder quotas to support ``backups`` and ``backup_gigabytes``. - - -**Deployment Engines**: - -*Updated* Extend `DevstackEngine -`_ -with ``enable_plugin`` option. - -**OpenStack clients**: - -* Extend support for auth urls like ``https://example.com:35357/foo/bar/v3`` - -* Pass endpoint type to heatclient - - -**Scenarios**: - -* *NEW!!* - - - `CinderVolumeTypes.create_and_delete_encryption_type - `_ - - - `CinderVolumeTypes.create_and_set_volume_type_keys - `_ - - - `KeystoneBasic.create_and_list_roles - `_ - - - `KeystoneBasic.create_and_update_user - `_ - - - `NovaKeypair.create_and_get_keypair - `_ - - - `NovaServers.resize_shutoff_server - `_ - - - `VMTasks.dd_load_test - `_ - -* *UPDATED!!* - - - Extend `VMTasks.boot_runcommand_delete - `_ - to display just raw text output of executed command. - -* *DELETED* - - Scenario `VMTasks.boot_runcommand_delete_custom_image - `_ - is removed since `VMTasks.boot_runcommand_delete - `_ - covers the case of that particular scenario without adding any complexity. - -**Validators**: - -* Extend ``required_contexts`` validator to support ``at least one of the`` - logic. - -* Fix a bunch of JSON schemas which are used for validation of all plugins. - -Documentation -~~~~~~~~~~~~~ - -We totally reworked `Plugins Reference -`_ page. -Now it looks more like `Command Line Interface -`_, which means that -you can get links for particular parameter of particular plugin. - -Also, you can find expected parameters and their types of all contexts, hooks, -SLAs and so on! Most of them still miss descriptions, but we are working on -adding them. - -Fixed bugs -~~~~~~~~~~ - -* [osclients] Custom auth mechanism was used for zaqarclient instead of unified - keystonesession, which led to auth errors at some envs. - -* [plugins] During running - `CinderVolumes.create_and_restore_volume_backup - `_ - scenario we had a race problem with backup deleting due to wrong check of - backup status. - -* [plugins][verifications] Jenkins expexts "classname" JUnitXML attribute - instead of "class_name". - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.9.1.rst b/doc/release_notes/archive/v0.9.1.rst deleted file mode 100644 index 85537baf..00000000 --- a/doc/release_notes/archive/v0.9.1.rst +++ /dev/null @@ -1,63 +0,0 @@ -============ -Rally v0.9.1 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **4/12/2017** | -+------------------+-----------------------+ - -Details -------- - -Unfortunately, Rally 0.9.0 contains various bugs. We work hard to fix them, -improve our CI to avoid such issues in future and ready to present a new Rally -release which includes only bug-fixes. - -Fixed bugs -~~~~~~~~~~ - -* [deployment] Credentials is not updated as soon as deployment is recreated. - Need to call recreate request twice. - - `Launchpad bug-report #1675271 - `_ - -* [task] Scenario `IronicNodes.create_and_list_node - `_ - had a wrong check that list of all nodes contains newly created one. - -* [task][cleanup] Do not remove quotas in case of existing users - -* [task][cleanup] Various traces of neutron resources - -* [core] Keystone v3, authentication error for Rally users if the value of - project_domain_name of admin user isn't equal "default" - - `Launchpad bug-report #1680837 - `_ - -* [task] Scenario `NovaHosts.list_and_get_hosts - `_ - obtains hostname for all hosts. But it fails in some environments if host is - not compute. - - `Launchpad bug-report #1675254 - `_ - -* [verification] Rally fails to run on systems on which python-virtualenv is - not installed - - `Launchpad bug-report #1678047 - `_ - -* [verification] CLI `rally verify rerun - `_ - fails with TypeError due to wring integration with Rally API. - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/archive/v0.9.2.rst b/doc/release_notes/archive/v0.9.2.rst deleted file mode 100644 index e0ef7c21..00000000 --- a/doc/release_notes/archive/v0.9.2.rst +++ /dev/null @@ -1,35 +0,0 @@ -============ -Rally v0.9.2 -============ - -Overview --------- - -+------------------+-----------------------+ -| Release date | **10/20/2017** | -+------------------+-----------------------+ - -Details -------- - -The latest OpenStack merged a bunch of incompatible changes. This release -is an attempt to fix compatibitily issuses in Rally 0.9. -Now it works well for old and new OpenStack releases. - -.. note:: OpenStack Nova abandoned networking and image API. It is impossible - to do anything with it, so we suggest you to use Neutron and Glance instead. - -Fixed bugs -~~~~~~~~~~ - -* [broken dependency] One of hook plugins required ansible which released - incompatible version. To fix this, the proper version of os-faults lib should - be used - -* [keystone] The format of keystone URL's completely change which resulted in a - wrong processing it from our side. - -Thanks -~~~~~~ - - 2 Everybody! diff --git a/doc/release_notes/latest.rst b/doc/release_notes/latest.rst deleted file mode 120000 index 2a846ea9..00000000 --- a/doc/release_notes/latest.rst +++ /dev/null @@ -1 +0,0 @@ -archive/v0.11.0.rst \ No newline at end of file