Add plan<->result references everywhere
Change-Id: I68acbcd3f656faa57540d2d7d4b8e1f2ffb8cfae Closes-Bug: #1609927
This commit is contained in:
Dina Belova
@@ -202,3 +202,8 @@ Priority Value Measurement Units Description
|
||||
spawn
|
||||
======== =============== ================= =================================
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`1000_nodes_report`
|
||||
|
||||
@@ -20,14 +20,12 @@ Measuring performance of container cluster systems
|
||||
Kubernetes, and Docker Swarm,
|
||||
which provide various ways to manage groups of containers.
|
||||
|
||||
:Conventions:
|
||||
|
||||
|
||||
|
||||
Test Plan
|
||||
=========
|
||||
|
||||
Test Environment
|
||||
----------------
|
||||
|
||||
Preparation
|
||||
^^^^^^^^^^^
|
||||
Typical container frameworks consist of several components:
|
||||
@@ -46,6 +44,7 @@ framework.
|
||||
|
||||
Test Case #1: A new application creation time
|
||||
---------------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the create
|
||||
@@ -138,6 +137,7 @@ Measuring values
|
||||
|
||||
Test Case #2: A application CPU update time
|
||||
-------------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the update CPU
|
||||
@@ -234,6 +234,7 @@ Measuring values
|
||||
|
||||
Test Case #3: A application memory update time
|
||||
----------------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the update memory
|
||||
@@ -330,6 +331,7 @@ Measuring values
|
||||
|
||||
Test Case #4: A application disk update time
|
||||
--------------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the update disk
|
||||
@@ -426,6 +428,7 @@ Measuring values
|
||||
|
||||
Test Case #5: A application instances update time
|
||||
-------------------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the update number of
|
||||
@@ -521,6 +524,7 @@ Measuring values
|
||||
|
||||
Test Case #6: A application restart time
|
||||
----------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the restart
|
||||
@@ -617,6 +621,7 @@ Measuring values
|
||||
|
||||
Test Case #7: A application delete time
|
||||
---------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at measuring the total elapsed time of the delete
|
||||
@@ -713,6 +718,7 @@ Measuring values
|
||||
|
||||
Test Case #8: Health check Performance
|
||||
--------------------------------------
|
||||
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
This test is aimed at validating the response time of a health check when many
|
||||
@@ -783,6 +789,7 @@ Measuring REAL_INTERVAL(TASK_COUNT) values
|
||||
|
||||
Applications
|
||||
============
|
||||
|
||||
list of container platforms
|
||||
---------------------------
|
||||
|
||||
@@ -795,3 +802,9 @@ list of container platforms
|
||||
+-----------------------------+---------+
|
||||
| Docker | |
|
||||
+-----------------------------+---------+
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`Results_of_measuring_performance_of_Mesos_Marathon`
|
||||
|
||||
@@ -330,3 +330,11 @@ Proposed docker file
|
||||
.. _Docker Registry2: https://docs.docker.com/registry
|
||||
.. _JFrog Artifactory: https://www.jfrog.com/artifactory
|
||||
.. _Docker: https://www.docker.com
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`Measuring_performance_of_Sonatype_Nexus`
|
||||
* :ref:`Measuring_performance_of_JFrog_Artifactory_Pro`
|
||||
* :ref:`Measuring_performance_of_docker_registry`
|
||||
|
||||
@@ -364,3 +364,9 @@ Example of Rally scenario configuration
|
||||
|
||||
.. literalinclude:: test_plans/keystone_authenticate.json
|
||||
:language: bash
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`openstack_control_plane_performance_report`
|
||||
|
||||
@@ -185,3 +185,9 @@ may look like:
|
||||
+===============+================+====================+===================+=============+=============+
|
||||
| create_volume | <mean_numbers> | <90%ile_numbers> | <50%ile_numbers> | <max_value> | <min_value> |
|
||||
+---------------+----------------+--------------------+-------------------+-------------+-------------+
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`Results_of_OpenStack_services_density_testing`
|
||||
|
||||
@@ -323,3 +323,10 @@ Elapsed time or latency for high-level OpenStack operations like virtual
|
||||
machine creation or network configuration may involve many database queries.
|
||||
The maximum query throughput the database layer can process with
|
||||
reasonable latency determines the maximum cluster size that can be supported.
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`db_performance_mysql`
|
||||
* :ref:`db_performance_mariadb`
|
||||
|
||||
@@ -517,3 +517,9 @@ Priority Value Measurement Units Description
|
||||
======== =============== ================= =====================================================
|
||||
|
||||
.. _Intel XL710 controller: http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-10-40-gbe-controller-brief.html
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`hw_features_offloads`
|
||||
|
||||
@@ -418,3 +418,9 @@ Priority Value Measurement Units Description
|
||||
======== ============== ================= =========================================
|
||||
1 Operation time milliseconds Time spent on every HTTP/RPC/DB operation
|
||||
======== ============== ================= =========================================
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`keystone_performance_all_in_one`
|
||||
|
||||
@@ -434,3 +434,10 @@ This section contains tools that can be used to perform the test plan.
|
||||
.. _RPC call: http://docs.openstack.org/developer/oslo.messaging/rpcclient.html#oslo_messaging.RPCClient.call
|
||||
.. _RPC cast: http://docs.openstack.org/developer/oslo.messaging/rpcclient.html#oslo_messaging.RPCClient.cast
|
||||
.. _Notification: http://docs.openstack.org/developer/oslo.messaging/notifier.html#notifier
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`mq_rabbit_report`
|
||||
* :ref:`zeromq_report`
|
||||
|
||||
@@ -242,7 +242,8 @@ Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`mq_ha_rabbit_report`
|
||||
* :ref:`mq_ha_rabbit_report_disabled`
|
||||
* :ref:`mq_ha_rabbit_report_enabled`
|
||||
|
||||
.. references:
|
||||
|
||||
|
||||
@@ -319,3 +319,10 @@ Priority Value Measurement Units Description
|
||||
======== =============== ================= ====================================
|
||||
1 UDP bandwidth % Loss of UDP packets of 64 bytes size
|
||||
======== =============== ================= ====================================
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`neutron_l3_ha_test_report_liberty`
|
||||
* :ref:`neutron_l3_ha_test_report_mitaka`
|
||||
|
||||
@@ -186,3 +186,9 @@ In this scenario Rally boots Nova servers at constant rate:
|
||||
#. boot server
|
||||
|
||||
.. literalinclude:: rally_scenarios/nova_boot_servers.json
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`openstack_load_report`
|
||||
|
||||
@@ -354,3 +354,9 @@ Full script for collecting performance metrics
|
||||
.. _dstat: http://dag.wiee.rs/home-made/dstat/
|
||||
.. _Cobbler: http://cobbler.github.io/
|
||||
.. _Razor: https://github.com/puppetlabs/razor-server
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`Measuring_performance_of_Cobbler`
|
||||
|
||||
@@ -375,3 +375,9 @@ Detailed results description
|
||||
|
||||
In this section, specify detailed description of test results,
|
||||
including factor impact.
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`reliability_testing_results`
|
||||
|
||||
@@ -423,3 +423,9 @@ Tools
|
||||
This section contains tools that can be used to perform the test plan.
|
||||
|
||||
.. include:: shaker.rst
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`tenant_networking_report_vxlan_dvr_200_nodes`
|
||||
|
||||
@@ -1,6 +1,14 @@
|
||||
.. raw:: pdf
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _1000_nodes_report:
|
||||
|
||||
1000 Compute nodes resource scalability testing
|
||||
===============================================
|
||||
|
||||
This report is generated for :ref:`1000_nodes` test plan.
|
||||
|
||||
Environment setup
|
||||
-----------------
|
||||
|
||||
|
||||
@@ -7,13 +7,12 @@ Results of measuring performance of Mesos Marathon
|
||||
:Abstract:
|
||||
|
||||
This document includes performance test results for `Mesos`_ as a container
|
||||
platform for `Docker`_ with `Marathon`_. All test have been performed
|
||||
regarding
|
||||
:ref:`Measuring_performance_of_mesos_marathon`
|
||||
|
||||
platform for `Docker`_ with `Marathon`_. All tests have been performed
|
||||
regarding :ref:`Measuring_performance_of_container_cluster_systems`
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
Cluster description
|
||||
-------------------
|
||||
* 3 master nodes with Mesos, Marathon and Zookeeper
|
||||
@@ -30,8 +29,6 @@ Running options
|
||||
---------------
|
||||
* Marathon extra options are "-Xmx2048m --max_tasks_per_offer 20"
|
||||
|
||||
|
||||
|
||||
Hardware configuration of each server
|
||||
-------------------------------------
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ Results of measuring performance of JFrog Artifactory Pro
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
Hardware configuration of each server
|
||||
-------------------------------------
|
||||
|
||||
@@ -79,6 +80,7 @@ p1p1 interface of a server:
|
||||
|
||||
Software configuration of the Artifactory service
|
||||
-------------------------------------------------
|
||||
|
||||
Installation of Artifactory:
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
.. code:: bash
|
||||
@@ -125,6 +127,7 @@ You can find outputs of some commands and /etc folder in the following archive:
|
||||
|
||||
Software configuration of the node with test tool
|
||||
-------------------------------------------------
|
||||
|
||||
Test tool:
|
||||
^^^^^^^^^^
|
||||
Firstly we need to install docker-engine:
|
||||
@@ -212,8 +215,10 @@ As a result of this part we got the following CSV files:
|
||||
|
||||
Results
|
||||
=======
|
||||
|
||||
Push action results
|
||||
-------------------
|
||||
|
||||
PUSH_TIME(ITERATION)
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
After simple processing results the following plots for push action in depend on
|
||||
@@ -277,6 +282,7 @@ parameter.
|
||||
|
||||
Pull action results
|
||||
-------------------
|
||||
|
||||
PULL_TIME(ITERATION)
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
After simple processing results the following plots for pull action in depend on
|
||||
|
||||
@@ -14,6 +14,7 @@ Results of measuring performance of Sonatype Nexus
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
Hardware configuration of each server
|
||||
-------------------------------------
|
||||
|
||||
@@ -78,7 +79,8 @@ p1p1 interface of a server:
|
||||
no snmp trap link-status
|
||||
|
||||
Software configuration of the Nexus service
|
||||
----------------------------------------------------
|
||||
-------------------------------------------
|
||||
|
||||
Installation of Nexus:
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
.. code:: bash
|
||||
@@ -136,6 +138,7 @@ You can find outputs of some commands and /etc folder in the following archive:
|
||||
|
||||
Software configuration of the node with test tool
|
||||
-------------------------------------------------
|
||||
|
||||
Test tool:
|
||||
^^^^^^^^^^
|
||||
Firstly we need to install docker-engine:
|
||||
@@ -222,8 +225,10 @@ As a result of this part we got the following CSV files:
|
||||
|
||||
Results
|
||||
=======
|
||||
|
||||
Push action results
|
||||
-------------------
|
||||
|
||||
PUSH_TIME(ITERATION)
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
After simple processing results the following plots for push action in depend on
|
||||
@@ -286,6 +291,7 @@ parameter.
|
||||
|
||||
Pull action results
|
||||
-------------------
|
||||
|
||||
PULL_TIME(ITERATION)
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
After simple processing results the following plots for pull action in depend on
|
||||
|
||||
@@ -14,6 +14,7 @@ Results of measuring performance of Docker Registry
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
Hardware configuration of each server
|
||||
-------------------------------------
|
||||
|
||||
@@ -79,6 +80,7 @@ p1p1 interface of a server:
|
||||
|
||||
Software configuration of the DockerRegistry service
|
||||
----------------------------------------------------
|
||||
|
||||
Installation of Registry2:
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
.. code:: bash
|
||||
@@ -106,6 +108,7 @@ You can find outputs of some commands and /etc folder in the following archive:
|
||||
|
||||
Software configuration of the node with test tool
|
||||
-------------------------------------------------
|
||||
|
||||
Test tool:
|
||||
^^^^^^^^^^
|
||||
Firstly we need to install docker-engine:
|
||||
@@ -191,8 +194,10 @@ As a result of this part we got the following CSV files:
|
||||
|
||||
Results
|
||||
=======
|
||||
|
||||
Push action results
|
||||
-------------------
|
||||
|
||||
PUSH_TIME(ITERATION)
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
After simple processing results the following plots for push action in depend on
|
||||
@@ -255,6 +260,7 @@ parameter.
|
||||
|
||||
Pull action results
|
||||
-------------------
|
||||
|
||||
PULL_TIME(ITERATION)
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
After simple processing results the following plots for pull action in depend on
|
||||
|
||||
@@ -9,7 +9,7 @@ Results of OpenStack Services density testing
|
||||
|
||||
This document includes density test results of OpenStack
|
||||
services. All tests have been performed
|
||||
regarding :ref:`controlplane_density_`
|
||||
regarding :ref:`controlplane_density`
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _db_performance_mariadb:
|
||||
|
||||
===================================
|
||||
MariaDB Database performance report
|
||||
===================================
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _db_performance_mysql:
|
||||
|
||||
=================================
|
||||
MySQL Database performance report
|
||||
=================================
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
.. _hw_features_offloads:
|
||||
|
||||
================================
|
||||
Hardware Offloads - Test results
|
||||
================================
|
||||
|
||||
This report is generated for :ref:`hardware_offloads_performance_analysis` test
|
||||
plan.
|
||||
|
||||
Baseline physical scenario
|
||||
--------------------------
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
.. _keystone_performance_all_in_one:
|
||||
|
||||
Keystone DB / cache operations analysis
|
||||
---------------------------------------
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _mq_ha_rabbit_report:
|
||||
.. _mq_rabbit_report:
|
||||
|
||||
===========================
|
||||
RabbitMQ Performance Report
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
.. _zeromq_report:
|
||||
|
||||
ZeroMQ performance
|
||||
------------------
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _mq_ha_rabbit_report:
|
||||
.. _mq_ha_rabbit_report_enabled:
|
||||
|
||||
===========================================
|
||||
RabbitMQ HA Test Reports: HA queues enabled
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _mq_ha_rabbit_report:
|
||||
.. _mq_ha_rabbit_report_disabled:
|
||||
|
||||
============================================
|
||||
RabbitMQ HA Test Reports: HA queues disabled
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
.. _neutron_l3_ha_test_report_liberty:
|
||||
|
||||
Neutron L3 HA test results Liberty
|
||||
----------------------------------
|
||||
|
||||
This report is generated for :ref:`neutron_l3_ha_test_plan` test plan.
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
.. _neutron_l3_ha_test_report_mitaka:
|
||||
|
||||
Neutron L3 HA test results Mitaka
|
||||
---------------------------------
|
||||
|
||||
@@ -5,6 +7,8 @@ As we have already done initial testing of L3 HA functionality for Liberty,
|
||||
in this testing load parameters are increased (for example, concurrency for Rally
|
||||
tests, number of routers for manual tests).
|
||||
|
||||
This report is generated for :ref:`neutron_l3_ha_test_plan` test plan.
|
||||
|
||||
Environment description
|
||||
=======================
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Results of measuring performance of Cobbler
|
||||
:Abstract:
|
||||
|
||||
This document includes performance test results of `Cobbler`_
|
||||
service as a provisioning system. All test have been performed
|
||||
service as a provisioning system. All tests have been performed
|
||||
regarding :ref:`Measuring_performance_of_provisioning_systems`
|
||||
|
||||
|
||||
|
||||
@@ -10,7 +10,8 @@ OpenStack reliability testing
|
||||
:Abstract:
|
||||
This document describes an abstract methodology for OpenStack cluster
|
||||
high-availability testing and analysis. OpenStack data plane testing
|
||||
at this moment is out of scope but will be described in future.
|
||||
at this moment is out of scope but will be described in future. All tests
|
||||
have been performed regarding :ref:`reliability_testing`
|
||||
|
||||
:Conventions:
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
PageBreak oneColumn
|
||||
|
||||
.. _mq_ha_rabbit_report:
|
||||
.. _tenant_networking_report_vxlan_dvr_200_nodes:
|
||||
|
||||
===================================================
|
||||
Tenant Networking: Neutron DVR + VXLAN at 200 nodes
|
||||
|
||||
Reference in New Issue
Block a user