Add plan<->result references everywhere

Change-Id: I68acbcd3f656faa57540d2d7d4b8e1f2ffb8cfae
Closes-Bug: #1609927
This commit is contained in:
Dina Belova 2016-08-11 10:09:36 -07:00
parent d391be06bc
commit 20b0943204
34 changed files with 162 additions and 21 deletions

View File

@ -202,3 +202,8 @@ Priority Value Measurement Units Description
spawn
======== =============== ================= =================================
Reports
=======
Test plan execution reports:
* :ref:`1000_nodes_report`

View File

@ -20,14 +20,12 @@ Measuring performance of container cluster systems
Kubernetes, and Docker Swarm,
which provide various ways to manage groups of containers.
:Conventions:
Test Plan
=========
Test Environment
----------------
Preparation
^^^^^^^^^^^
Typical container frameworks consist of several components:
@ -46,6 +44,7 @@ framework.
Test Case #1: A new application creation time
---------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the create
@ -138,6 +137,7 @@ Measuring values
Test Case #2: A application CPU update time
-------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update CPU
@ -234,6 +234,7 @@ Measuring values
Test Case #3: A application memory update time
----------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update memory
@ -330,6 +331,7 @@ Measuring values
Test Case #4: A application disk update time
--------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update disk
@ -426,6 +428,7 @@ Measuring values
Test Case #5: A application instances update time
-------------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update number of
@ -521,6 +524,7 @@ Measuring values
Test Case #6: A application restart time
----------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the restart
@ -617,6 +621,7 @@ Measuring values
Test Case #7: A application delete time
---------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the delete
@ -713,6 +718,7 @@ Measuring values
Test Case #8: Health check Performance
--------------------------------------
Description
^^^^^^^^^^^
This test is aimed at validating the response time of a health check when many
@ -783,6 +789,7 @@ Measuring REAL_INTERVAL(TASK_COUNT) values
Applications
============
list of container platforms
---------------------------
@ -795,3 +802,9 @@ list of container platforms
+-----------------------------+---------+
| Docker | |
+-----------------------------+---------+
Reports
=======
Test plan execution reports:
* :ref:`Results_of_measuring_performance_of_Mesos_Marathon`

View File

@ -330,3 +330,11 @@ Proposed docker file
.. _Docker Registry2: https://docs.docker.com/registry
.. _JFrog Artifactory: https://www.jfrog.com/artifactory
.. _Docker: https://www.docker.com
Reports
=======
Test plan execution reports:
* :ref:`Measuring_performance_of_Sonatype_Nexus`
* :ref:`Measuring_performance_of_JFrog_Artifactory_Pro`
* :ref:`Measuring_performance_of_docker_registry`

View File

@ -364,3 +364,9 @@ Example of Rally scenario configuration
.. literalinclude:: test_plans/keystone_authenticate.json
:language: bash
Reports
=======
Test plan execution reports:
* :ref:`openstack_control_plane_performance_report`

View File

@ -185,3 +185,9 @@ may look like:
+===============+================+====================+===================+=============+=============+
| create_volume | <mean_numbers> | <90%ile_numbers> | <50%ile_numbers> | <max_value> | <min_value> |
+---------------+----------------+--------------------+-------------------+-------------+-------------+
Reports
=======
Test plan execution reports:
* :ref:`Results_of_OpenStack_services_density_testing`

View File

@ -323,3 +323,10 @@ Elapsed time or latency for high-level OpenStack operations like virtual
machine creation or network configuration may involve many database queries.
The maximum query throughput the database layer can process with
reasonable latency determines the maximum cluster size that can be supported.
Reports
=======
Test plan execution reports:
* :ref:`db_performance_mysql`
* :ref:`db_performance_mariadb`

View File

@ -517,3 +517,9 @@ Priority Value Measurement Units Description
======== =============== ================= =====================================================
.. _Intel XL710 controller: http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-10-40-gbe-controller-brief.html
Reports
=======
Test plan execution reports:
* :ref:`hw_features_offloads`

View File

@ -418,3 +418,9 @@ Priority Value Measurement Units Description
======== ============== ================= =========================================
1 Operation time milliseconds Time spent on every HTTP/RPC/DB operation
======== ============== ================= =========================================
Reports
=======
Test plan execution reports:
* :ref:`keystone_performance_all_in_one`

View File

@ -434,3 +434,10 @@ This section contains tools that can be used to perform the test plan.
.. _RPC call: http://docs.openstack.org/developer/oslo.messaging/rpcclient.html#oslo_messaging.RPCClient.call
.. _RPC cast: http://docs.openstack.org/developer/oslo.messaging/rpcclient.html#oslo_messaging.RPCClient.cast
.. _Notification: http://docs.openstack.org/developer/oslo.messaging/notifier.html#notifier
Reports
=======
Test plan execution reports:
* :ref:`mq_rabbit_report`
* :ref:`zeromq_report`

View File

@ -242,7 +242,8 @@ Reports
=======
Test plan execution reports:
* :ref:`mq_ha_rabbit_report`
* :ref:`mq_ha_rabbit_report_disabled`
* :ref:`mq_ha_rabbit_report_enabled`
.. references:

View File

@ -319,3 +319,10 @@ Priority Value Measurement Units Description
======== =============== ================= ====================================
1 UDP bandwidth % Loss of UDP packets of 64 bytes size
======== =============== ================= ====================================
Reports
=======
Test plan execution reports:
* :ref:`neutron_l3_ha_test_report_liberty`
* :ref:`neutron_l3_ha_test_report_mitaka`

View File

@ -186,3 +186,9 @@ In this scenario Rally boots Nova servers at constant rate:
#. boot server
.. literalinclude:: rally_scenarios/nova_boot_servers.json
Reports
=======
Test plan execution reports:
* :ref:`openstack_load_report`

View File

@ -354,3 +354,9 @@ Full script for collecting performance metrics
.. _dstat: http://dag.wiee.rs/home-made/dstat/
.. _Cobbler: http://cobbler.github.io/
.. _Razor: https://github.com/puppetlabs/razor-server
Reports
=======
Test plan execution reports:
* :ref:`Measuring_performance_of_Cobbler`

View File

@ -375,3 +375,9 @@ Detailed results description
In this section, specify detailed description of test results,
including factor impact.
Reports
=======
Test plan execution reports:
* :ref:`reliability_testing_results`

View File

@ -423,3 +423,9 @@ Tools
This section contains tools that can be used to perform the test plan.
.. include:: shaker.rst
Reports
=======
Test plan execution reports:
* :ref:`tenant_networking_report_vxlan_dvr_200_nodes`

View File

@ -1,6 +1,14 @@
.. raw:: pdf
PageBreak oneColumn
.. _1000_nodes_report:
1000 Compute nodes resource scalability testing
===============================================
This report is generated for :ref:`1000_nodes` test plan.
Environment setup
-----------------

View File

@ -7,13 +7,12 @@ Results of measuring performance of Mesos Marathon
:Abstract:
This document includes performance test results for `Mesos`_ as a container
platform for `Docker`_ with `Marathon`_. All test have been performed
regarding
:ref:`Measuring_performance_of_mesos_marathon`
platform for `Docker`_ with `Marathon`_. All tests have been performed
regarding :ref:`Measuring_performance_of_container_cluster_systems`
Environment description
=======================
Cluster description
-------------------
* 3 master nodes with Mesos, Marathon and Zookeeper
@ -30,8 +29,6 @@ Running options
---------------
* Marathon extra options are "-Xmx2048m --max_tasks_per_offer 20"
Hardware configuration of each server
-------------------------------------

View File

@ -14,6 +14,7 @@ Results of measuring performance of JFrog Artifactory Pro
Environment description
=======================
Hardware configuration of each server
-------------------------------------
@ -79,6 +80,7 @@ p1p1 interface of a server:
Software configuration of the Artifactory service
-------------------------------------------------
Installation of Artifactory:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
@ -125,6 +127,7 @@ You can find outputs of some commands and /etc folder in the following archive:
Software configuration of the node with test tool
-------------------------------------------------
Test tool:
^^^^^^^^^^
Firstly we need to install docker-engine:
@ -212,8 +215,10 @@ As a result of this part we got the following CSV files:
Results
=======
Push action results
-------------------
PUSH_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for push action in depend on
@ -277,6 +282,7 @@ parameter.
Pull action results
-------------------
PULL_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for pull action in depend on

View File

@ -14,6 +14,7 @@ Results of measuring performance of Sonatype Nexus
Environment description
=======================
Hardware configuration of each server
-------------------------------------
@ -78,7 +79,8 @@ p1p1 interface of a server:
no snmp trap link-status
Software configuration of the Nexus service
----------------------------------------------------
-------------------------------------------
Installation of Nexus:
^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
@ -136,6 +138,7 @@ You can find outputs of some commands and /etc folder in the following archive:
Software configuration of the node with test tool
-------------------------------------------------
Test tool:
^^^^^^^^^^
Firstly we need to install docker-engine:
@ -222,8 +225,10 @@ As a result of this part we got the following CSV files:
Results
=======
Push action results
-------------------
PUSH_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for push action in depend on
@ -286,6 +291,7 @@ parameter.
Pull action results
-------------------
PULL_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for pull action in depend on

View File

@ -14,6 +14,7 @@ Results of measuring performance of Docker Registry
Environment description
=======================
Hardware configuration of each server
-------------------------------------
@ -79,6 +80,7 @@ p1p1 interface of a server:
Software configuration of the DockerRegistry service
----------------------------------------------------
Installation of Registry2:
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
@ -106,6 +108,7 @@ You can find outputs of some commands and /etc folder in the following archive:
Software configuration of the node with test tool
-------------------------------------------------
Test tool:
^^^^^^^^^^
Firstly we need to install docker-engine:
@ -191,8 +194,10 @@ As a result of this part we got the following CSV files:
Results
=======
Push action results
-------------------
PUSH_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for push action in depend on
@ -255,6 +260,7 @@ parameter.
Pull action results
-------------------
PULL_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for pull action in depend on

View File

@ -9,7 +9,7 @@ Results of OpenStack Services density testing
This document includes density test results of OpenStack
services. All tests have been performed
regarding :ref:`controlplane_density_`
regarding :ref:`controlplane_density`
Environment description
=======================

View File

@ -2,6 +2,8 @@
PageBreak oneColumn
.. _db_performance_mariadb:
===================================
MariaDB Database performance report
===================================

View File

@ -2,6 +2,8 @@
PageBreak oneColumn
.. _db_performance_mysql:
=================================
MySQL Database performance report
=================================

View File

@ -1,7 +1,12 @@
.. _hw_features_offloads:
================================
Hardware Offloads - Test results
================================
This report is generated for :ref:`hardware_offloads_performance_analysis` test
plan.
Baseline physical scenario
--------------------------

View File

@ -1,3 +1,5 @@
.. _keystone_performance_all_in_one:
Keystone DB / cache operations analysis
---------------------------------------

View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _mq_rabbit_report:
===========================
RabbitMQ Performance Report

View File

@ -1,3 +1,5 @@
.. _zeromq_report:
ZeroMQ performance
------------------

View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _mq_ha_rabbit_report_enabled:
===========================================
RabbitMQ HA Test Reports: HA queues enabled

View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _mq_ha_rabbit_report_disabled:
============================================
RabbitMQ HA Test Reports: HA queues disabled

View File

@ -1,6 +1,10 @@
.. _neutron_l3_ha_test_report_liberty:
Neutron L3 HA test results Liberty
----------------------------------
This report is generated for :ref:`neutron_l3_ha_test_plan` test plan.
Environment description
=======================

View File

@ -1,3 +1,5 @@
.. _neutron_l3_ha_test_report_mitaka:
Neutron L3 HA test results Mitaka
---------------------------------
@ -5,6 +7,8 @@ As we have already done initial testing of L3 HA functionality for Liberty,
in this testing load parameters are increased (for example, concurrency for Rally
tests, number of routers for manual tests).
This report is generated for :ref:`neutron_l3_ha_test_plan` test plan.
Environment description
=======================

View File

@ -8,7 +8,7 @@ Results of measuring performance of Cobbler
:Abstract:
This document includes performance test results of `Cobbler`_
service as a provisioning system. All test have been performed
service as a provisioning system. All tests have been performed
regarding :ref:`Measuring_performance_of_provisioning_systems`

View File

@ -10,7 +10,8 @@ OpenStack reliability testing
:Abstract:
This document describes an abstract methodology for OpenStack cluster
high-availability testing and analysis. OpenStack data plane testing
at this moment is out of scope but will be described in future.
at this moment is out of scope but will be described in future. All tests
have been performed regarding :ref:`reliability_testing`
:Conventions:

View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _tenant_networking_report_vxlan_dvr_200_nodes:
===================================================
Tenant Networking: Neutron DVR + VXLAN at 200 nodes