diff --git a/doc/feature_request/check_queue_perfdata.rst b/doc/feature_request/check_queue_perfdata.rst index a95ed179..9d4b83d2 100644 --- a/doc/feature_request/check_queue_perfdata.rst +++ b/doc/feature_request/check_queue_perfdata.rst @@ -4,6 +4,7 @@ Check queue perfdata Use case -------- + Sometimes OpenStack services use common messaging system very prodigally. For example Neutron metering agent sending all database table data on new object creation i.e https://review.openstack.org/#/c/143672/. It cause to Neutron @@ -19,4 +20,4 @@ Possible solution ----------------- * Before running tests/benchmarks start process which will connect to queue -topics and measure messages count, size and other data which we need. + topics and measure messages count, size and other data which we need. diff --git a/doc/feature_request/installing_isolated.rst b/doc/feature_request/installing_isolated.rst index 2d502595..154a6c34 100644 --- a/doc/feature_request/installing_isolated.rst +++ b/doc/feature_request/installing_isolated.rst @@ -1,6 +1,6 @@ -================================================================================= +================================================================================== Installation script: ``--pypi-mirror``, ``--package-mirror`` and ``--venv-mirror`` -================================================================================= +================================================================================== Use case diff --git a/doc/feature_request/launch_specific_benchmark.rst b/doc/feature_request/launch_specific_benchmark.rst index 0b910c4e..d9588f8f 100644 --- a/doc/feature_request/launch_specific_benchmark.rst +++ b/doc/feature_request/launch_specific_benchmark.rst @@ -24,4 +24,4 @@ Possible solution ----------------- * Add optional flag to rally task start command to specify one or more -benchmarks to execute as part of that test run. + benchmarks to execute as part of that test run. diff --git a/doc/release_notes/archive/v0.1.0.rst b/doc/release_notes/archive/v0.1.0.rst index 84297780..4eeede43 100644 --- a/doc/release_notes/archive/v0.1.0.rst +++ b/doc/release_notes/archive/v0.1.0.rst @@ -134,6 +134,7 @@ New Features & API changes * Samples of NEW vs OLD context, sla, scenario and runner plugins: * Context + .. code-block:: python # Old @@ -167,6 +168,7 @@ New Features & API changes * Scenario + .. code-block:: python # Old Scenario @@ -205,6 +207,7 @@ New Features & API changes .. * Runner + .. code-block:: python ## Old @@ -237,6 +240,7 @@ New Features & API changes .. * SLA + .. code-block:: python # Old @@ -495,7 +499,7 @@ Documentation * **Add plugin reference page** - :ref:`Rally Plugins Reference page ` page contains a + :ref:`Rally Plugins Reference page ` page contains a full list with * **Add maintainers section on project info page** diff --git a/doc/release_notes/archive/v0.1.2.rst b/doc/release_notes/archive/v0.1.2.rst index df7d6357..4bd70971 100644 --- a/doc/release_notes/archive/v0.1.2.rst +++ b/doc/release_notes/archive/v0.1.2.rst @@ -67,7 +67,7 @@ Rally Task * Rally supports different API versions now via api_versions context: -.. code:: none +.. code-block:: none CinderVolumes.create_and_delete_volume: - diff --git a/doc/release_notes/archive/v0.2.0.rst b/doc/release_notes/archive/v0.2.0.rst index 79a9fb79..a09f9b7d 100644 --- a/doc/release_notes/archive/v0.2.0.rst +++ b/doc/release_notes/archive/v0.2.0.rst @@ -120,7 +120,8 @@ Plugins * [improved] CinderVolumes.create_and_attach_volume - .. warning:: Use "create_vm_params" dict argument instead of **kwargs for instance parameters. + .. warning:: Use "create_vm_params" dict argument instead of ``**kwargs`` + for instance parameters. * **Context**: diff --git a/doc/release_notes/archive/v0.3.0.rst b/doc/release_notes/archive/v0.3.0.rst index dd4a1a51..bdb7e452 100644 --- a/doc/release_notes/archive/v0.3.0.rst +++ b/doc/release_notes/archive/v0.3.0.rst @@ -31,7 +31,7 @@ CLI changes ~~~~~~~~~~~ * .. warning:: [Removed] ``rally info`` in favor of ``rally plugin *``. - It was deprecated for a long time. + It was deprecated for a long time. * [Modified] ``rally deployment check`` now prints services, which don't have names, since such services can be used via api_versions context. diff --git a/doc/release_notes/archive/v0.4.0.rst b/doc/release_notes/archive/v0.4.0.rst index ff97eae0..ea1618be 100644 --- a/doc/release_notes/archive/v0.4.0.rst +++ b/doc/release_notes/archive/v0.4.0.rst @@ -180,7 +180,8 @@ Bug fixes for each TestCase. In this case, Rally did not check partial test with x-fail list and marked test as "fail" insted of "x-fail". - `Launchpad bug-report `_ + `Launchpad bug-report #1568133 + `_ * Weak isolation of scenario arguments between iterations @@ -201,7 +202,8 @@ Bug fixes Results of failed task validations saved in incorrect format. It broke and made un-userfriendly `rally task detailed` command. - `Launchpad bug-report `_ + `Launchpad bug-report #1562713 + `_ * Hardcoded task's status in `rally task results` @@ -209,14 +211,16 @@ Bug fixes task has failed status, but it can be not true(tasks in running state do not have results). - `Launchpad bug-report `_ + `Launchpad bug-report #1539096 + `_ * Tempest context failed to create network resources While we merged improvement for keystoneclient, we used wrong way to obtain tenant id in TempestContext. - `Launchpad bug-report `_ + `Launchpad bug-report #1550848 + `_ * Tasks based on Tempest failed to parse execution time. @@ -226,7 +230,8 @@ Bug fixes There was an issue while converting task execution time. - `Launchpad bug-report `_ + `Launchpad bug-report #1566712 + `_ * JSONSchema huge impact on task performance @@ -256,13 +261,15 @@ Bug fixes * Rally did not show "missing argument" error raised by argparse while parsing cli args - `Launchpad bug-report `_ + `Launchpad bug-report #1562916 + `_ * Issue while checking required arguments in CLI There was a possible issue in case of several required arguments - `Launchpad bug-report `_ + `Launchpad bug-report #1555764 + `_ * Prepare step of verification did not check visibility of obtained image @@ -270,7 +277,8 @@ Bug fixes make sure all images are active and they are PUBLIC. If images are not public, we will have failures of Tempest tests as described in the bug. - `Launchpad bug-report `_ + `Launchpad bug-report #1564431 + `_ Thanks ~~~~~~ diff --git a/doc/release_notes/archive/v0.5.0.rst b/doc/release_notes/archive/v0.5.0.rst index e953f31d..383ad064 100644 --- a/doc/release_notes/archive/v0.5.0.rst +++ b/doc/release_notes/archive/v0.5.0.rst @@ -122,6 +122,7 @@ Sometimes plugins of different bases can have equal names(i.e ceilometer OSClient and ceilometer Context). It is normal and we should allow such conflicts. To support such cases we introduced new entity - plugin base. Statements of plugin bases: + - Each plugin base is unique entity; - Names of plugin bases can't conflict with each other; - Names of two or more plugins in one plugin base can't conflict with each @@ -129,6 +130,7 @@ Statements of plugin bases: - Names of two or more plugins in different plugin base can conflict Current list of plugin bases: + - rally.task.context.Context - rally.task.scenario.Scenario - rally.task.types.ResourceType @@ -182,17 +184,14 @@ Current list of plugin bases: Add nested_level argument for nested cloning volume to new volume - `Plugin Reference `_ - -* Extend CinderVolumes.create_nested_snapshots_and_attach_volume +* Extend `CinderVolumes.create_nested_snapshots_and_attach_volume + `_ Two new arguments were added: create_volume_kwargs and create_snapshot_kwargs .. warning:: All arguments related to snapshot creation should be transmitted only via create_snapshot_kwargs. - `Plugin Reference `_ - * Introduce new style of scenarios - class based. `Spec Reference `_ @@ -289,7 +288,8 @@ Bug fixes * AttributeError while disabling urllib3 warnings on old installations - `Launchpad bug-report `_ + `Launchpad bug-report #1573650 + `_ * install_rally.sh script is failed while obtaining setuptools @@ -298,7 +298,8 @@ Bug fixes * Strange input task in the report - `Launchpad bug-report `_ + `Launchpad bug-report #1570328 + `_ * Wrong behaviour of boot_server_from_volume scenarios in case of booting server from image. @@ -312,7 +313,8 @@ Bug fixes NovaServers.boot_server_from_volume_and_resize NovaServers.boot_server_from_volume_and_live_migrate - `Launchpad bug-report `_ + `Launchpad bug-report #1578556 + `_ * Weak validation of json schema of RPS runner @@ -322,70 +324,85 @@ Bug fixes * Rally doesn't take cacert setting while creating keystone session - `Launchpad bug-report `_ + `Launchpad bug-report #1577360 + `_ * Heat scenarios fail when API uses TLS - `Launchpad bug-report `_ + `Launchpad bug-report #1585456 + `_ * Example in comment of context manila_share_networks wrong - `Launchpad bug-report `_ + `Launchpad bug-report #1587164 + `_ * There is no way to get UUID of a verification after it is created by "rally verify start" or "rally verify import_results" when --no-use is set - `Launchpad bug-report `_ + `Launchpad bug-report #1587034 + `_ * Exposed ssh timeout and interval in vm scenario - `Launchpad bug-report `_ + `Launchpad bug-report #1587728 + `_ * Ceilometer scenario doesn't require "ceilometer" ctx - `Launchpad bug-report `_ + `Launchpad bug-report #1557642 + `_ * "servers" context requires setting network id for multiple possible networks found. - `Launchpad bug-report `_ + `Launchpad bug-report #1592292 + `_ * nested_level data type incorrect in create_nested_snapshots_and_attach_volume - `Launchpad bug-report `_ + `Launchpad bug-report #1594656 + `_ * Rally cleanup servers raises exception - `Launchpad bug-report `_ + `Launchpad bug-report #1584104 + `_ * Stopping server is redundant before cold-migrating server - `Launchpad bug-report `_ + `Launchpad bug-report #1594730 + `_ * existing_users context doesn't work in case of Keystone v3 * Whether validates flavor's disk or not depends on booting type of the instance - `Launchpad bug-report `_ + `Launchpad bug-report #1596756 + `_ Documentation ~~~~~~~~~~~~~ * Re-use openstack theme for building docs outside rtd. - `Rally Docs at docs.openstack.org `_ + `Rally Docs at docs.openstack.org + `_ * Add page for Verification component - `RTD page `_ + `RTD page for Verification component + `_ * Add glossary page - `RTD page `_ + `RTD page for Glossary + `_ * Adjust docs reference to "KeystoneBasic.authenticate" scenario - `RTD page `_ + `Step 6. Aborting load generation on success criteria failure + `_ Thanks ~~~~~~ diff --git a/doc/release_notes/archive/v0.6.0.rst b/doc/release_notes/archive/v0.6.0.rst index 3b2607b0..2649ac69 100644 --- a/doc/release_notes/archive/v0.6.0.rst +++ b/doc/release_notes/archive/v0.6.0.rst @@ -98,6 +98,7 @@ Rally Verify ~~~~~~~~~~~~ * Updating Tempest config file + - Some tests (for boto, horizon, etc.) were removed from Tempest and now there is no need to keep the corresponding options in Tempest config file. @@ -134,6 +135,7 @@ Plugins - `SaharaNodeGroupTemplates.create_delete_node_group_templates `_ * *NEW!!*: + - `MonascaMetrics.list_metrics `_ - `SenlinClusters.create_and_delete_cluster `_ - `Watcher.create_audit_template_and_delete `_ @@ -149,6 +151,7 @@ Plugins **Contexts**: * *NEW!!*: + - `Monasca monasca_metrics `_ - `Senlin profiles `_ - `Watcher audit_templates `_ diff --git a/doc/release_notes/archive/v0.7.0.rst b/doc/release_notes/archive/v0.7.0.rst index 073a4a9f..e49ac0ba 100644 --- a/doc/release_notes/archive/v0.7.0.rst +++ b/doc/release_notes/archive/v0.7.0.rst @@ -95,6 +95,7 @@ Plugins .. warning:: This plugin is not available anymore in 0.7.0 * *NEW!!*: + - `MagnumClusterTemplates.list_cluster_templates `_ - `MagnumClusters.list_clusters `_ - `MagnumClusters.create_and_list_clusters `_ @@ -111,6 +112,7 @@ Plugins **Contexts**: * *NEW!!*: + - `Manila manila_security_services `_ - `Magnum cluster_templates `_ - `Magnum clusters `_ diff --git a/doc/release_notes/archive/v0.8.0.rst b/doc/release_notes/archive/v0.8.0.rst index ac102103..0630e4f5 100644 --- a/doc/release_notes/archive/v0.8.0.rst +++ b/doc/release_notes/archive/v0.8.0.rst @@ -35,6 +35,7 @@ Database .. warning:: Database schema is changed, you must run `rally-manage db upgrade `_ to be able to use old Rally installation with latest release. + * change structure of database to be more flexible * save raw task results in chunks (see raw_result_chunk_size option of [DEFAULT] rally configuration section) @@ -102,9 +103,10 @@ Plugins **Scenarios**: * *Updated*: + - The meaning of the volume_type argument is changes in `CinderVolumes.create_snapshot_and_attach_volume - `_ + `_ scenario. It should contain actual volume type instead of boolean value to choose random volume type. - Extend `GlanceImages.create_image_and_boot_instances @@ -112,6 +114,7 @@ Plugins with create_image_kwargs and boot_server_kwargs arguments. * *NEW!!*: + - `CeilometerAlarms.create_and_get_alarm `_ - `CinderVolumeBackups.create_incremental_volume_backup @@ -172,12 +175,14 @@ Plugins **Hooks**: * *NEW!!*: + - `fault_injection `_ **Runners** * *Updated*: + - `RPS runner `_ is extended with ability to increase 'rps' value by arithmetic progression diff --git a/doc/release_notes/archive/v0.8.1.rst b/doc/release_notes/archive/v0.8.1.rst index ede1a734..08bc4179 100644 --- a/doc/release_notes/archive/v0.8.1.rst +++ b/doc/release_notes/archive/v0.8.1.rst @@ -20,16 +20,20 @@ Plugins **Scenarios**: * *Updated*: + - Use new network for each subnet at `NeutronNetworks.create_and_list_subnets - `_ + `_ scenario. * *NEW!!*: + - `CinderVolumeTypes.create_and_list_encryption_type `_ + - `Quotas.cinder_get `_ + Thanks ~~~~~~ diff --git a/doc/source/images/Report-Verify-xfail.png b/doc/source/images/Report-Verify-xfail.png new file mode 100644 index 00000000..79028427 Binary files /dev/null and b/doc/source/images/Report-Verify-xfail.png differ diff --git a/doc/source/project_info/index.rst b/doc/source/project_info/index.rst index 2a69c976..c37d5cf8 100644 --- a/doc/source/project_info/index.rst +++ b/doc/source/project_info/index.rst @@ -144,6 +144,8 @@ Where can I discuss and propose changes? `subscription and usage instructions`_); - `Rally team on Launchpad`_: Answers/Bugs/Blueprints. +.. _release_notes: + .. include:: release_notes.rst .. references: diff --git a/doc/source/project_info/release_notes.rst b/doc/source/project_info/release_notes.rst index 811c5e42..b26b4a6b 100644 --- a/doc/source/project_info/release_notes.rst +++ b/doc/source/project_info/release_notes.rst @@ -13,8 +13,6 @@ License for the specific language governing permissions and limitations under the License. -.. _release_notes: - Release Notes ------------- diff --git a/doc/source/task/index.rst b/doc/source/task/index.rst index fdf0993f..60c6f45d 100644 --- a/doc/source/task/index.rst +++ b/doc/source/task/index.rst @@ -124,23 +124,23 @@ input file. For example, having this in task input file: -.. code-block:: json + .. code-block:: json - "sla": { - "performance_degradation": { - "max_degradation": 50 - }, - "max_seconds_per_iteration": 1.0, - "failure_rate": { - "max": 0 - }, - "outliers": { - "max": 1, - "min_iterations": 10, - "sigmas": 10 - }, - "max_avg_duration": 0.5 - } + "sla": { + "performance_degradation": { + "max_degradation": 50 + }, + "max_seconds_per_iteration": 1.0, + "failure_rate": { + "max": 0 + }, + "outliers": { + "max": 1, + "min_iterations": 10, + "sigmas": 10 + }, + "max_avg_duration": 0.5 + } will result SLA section similar to the following: @@ -168,7 +168,7 @@ Table with statistics data **Action** Name of the workload metric that has some duration saved. This is either an atomic action name or *Total* which points to workload -`load duration <#load-duration>`_. + `load duration <#load-duration>`_. **Min (sec)** `Minimal`_ duration value diff --git a/doc/source/verification/howto/add_new_reporter.rst b/doc/source/verification/howto/add_new_reporter.rst index d16a4a50..7611c2a5 100644 --- a/doc/source/verification/howto/add_new_reporter.rst +++ b/doc/source/verification/howto/add_new_reporter.rst @@ -44,8 +44,8 @@ Basically, you need to implement only two methods "validate" and "generate". Method "validate" should check that destination of the report is right. Method "generate" should build a report or export results somewhere; actually, -it is up to you what it should do but return format is strict, see :ref:`spec` -section for what it can return. +it is up to you what it should do but return format is strict, see +`Spec <#spec>`_ section for what it can return. .. code-block:: python diff --git a/doc/source/verification/howto/migrate_from_old_design.rst b/doc/source/verification/howto/migrate_from_old_design.rst index 406bb76b..a74eeb7b 100644 --- a/doc/source/verification/howto/migrate_from_old_design.rst +++ b/doc/source/verification/howto/migrate_from_old_design.rst @@ -286,14 +286,6 @@ Commands for Rally 0.7.0: $ rally verify genconfig --deployment --tempest-config \ --add-options --override -* The command for showing configs `rally verify showconfig - `_ - - .. code-block:: console - - $ rally verify showconfig --deployment - - Command since Rally 0.8.0: .. code-block:: console @@ -315,13 +307,6 @@ Changes: 3) The argument ``--reconfigure`` replaces old ``--override``. It means that existing file will be ignored and new one will be used/created. -4) If the argument ``--show`` is specified, a configuration of verifier will be - displayed at the end of command execution. - - .. note:: We do not have a separate command for showing configurations - anymore. ``rally verify configure-verifier --show`` shows an existing - configuration if it exists and ``--reconfigure`` argument is not specified - Show config """"""""""" @@ -399,7 +384,7 @@ Commands for Rally 0.7.0: $ rally verify show --uuid --sort-by --detailed * Separate command which calls ``rally verify show`` with hardcoded - ``--detailed`` flag `rally verify showconfig + ``--detailed`` flag `rally verify detailed `_ .. code-block:: console diff --git a/doc/user_stories/keystone/authenticate.rst b/doc/user_stories/keystone/authenticate.rst index b91141b0..ef9d0997 100644 --- a/doc/user_stories/keystone/authenticate.rst +++ b/doc/user_stories/keystone/authenticate.rst @@ -1,6 +1,6 @@ -=================================================================================== +==================================================================================== 4x performance increase in Keystone inside Apache using the token creation benchmark -=================================================================================== +==================================================================================== *(Contributed by Neependra Khare, Red Hat)*