From 1f39d723ef7d6587bb8e68363584859b866b8337 Mon Sep 17 00:00:00 2001 From: Ihtisham ul Haq Date: Mon, 9 Sep 2019 16:44:53 +0200 Subject: [PATCH] Add VMTP and incorporate developers feedback Change-Id: Ia57f44ff2923d22a0b2980022eebe5a09c01980e --- doc/source/testing/tools-list.rst | 74 ++++++++++++++++++------------- 1 file changed, 44 insertions(+), 30 deletions(-) diff --git a/doc/source/testing/tools-list.rst b/doc/source/testing/tools-list.rst index a5e1847..723e619 100644 --- a/doc/source/testing/tools-list.rst +++ b/doc/source/testing/tools-list.rst @@ -18,29 +18,30 @@ with a unified solution. Here is list of tools and the testing features that are provided by those (as of August 2019): -=================================== ======= ======= =========== =========== =========== ============ ========== ========== ============================================================================================================== - Tools/Features Control Data Performance Scalability Concurrency Disruptions/ OpenStack Report Active community [#]_ - plane plane testing testing testing Failure Deployment generation - testing testing injection -=================================== ======= ======= =========== =========== =========== ============ ========== ========== ============================================================================================================== -Rally_ (Uses Tempest) X X X X X X X `Medium `__ -Yardstick_ X X X X X X X `NA since May 2019 `__ -Tempest_ X X `High `__ -Shaker_ X X X X `Low `__ -OS-Faults_ X `NA since February 2019 `__ -KloudBuster_ X X X X `NA since June 2019 `__ -HA-Failure-Test_ (Uses Tempest) X X `NA since November 2016 `__ -Cloud99_ X X X X `NA since May 2017 `__ -Browbeat_ (Uses Rally and Shaker) X X X X X `High `__ -Harbinger_ X `Low `__ -Enos_ (Uses Rally and Shaker) X X X X X `NA since May 2019 `__ -Chaostoolkit-openstack_ X X `NA since April 2019 `__ -Jepsen_ (not specific to OpenStack) X X `High `__ -OPNFV-Doctor_ X `NA since April 2019 `__ -Stepler_ (Uses OS-faults) X `NA since February 2019 `__ -NFVbench_ X X X `Medium `__ -Eris_ (In development) X X X X X X X `NA since November 2017 `__ -=================================== ======= ======= =========== =========== =========== ============ ========== ========== ============================================================================================================== +=================================== ======= ======= ======= =========== =========== =========== ============ ========== ========== ============================================================================================================== + Tools/Features Control Data Storage Performance Scalability Concurrency Disruptions/ OpenStack Report Active community [#]_ + plane plane plane testing testing testing Failure Deployment generation + testing testing testing injection +=================================== ======= ======= ======= =========== =========== =========== ============ ========== ========== ============================================================================================================== +Rally_ (Uses Tempest) X X X X X X X `Medium `__ +Yardstick_ X X X X X X X X `NA since May 2019 `__ +Tempest_ X X `High `__ +Shaker_ X X X X `Low `__ +OS-Faults_ X `NA since February 2019 `__ +KloudBuster_ X X X X X `NA since June 2019 `__ +HA-Failure-Test_ (Uses Tempest) X X `NA since November 2016 `__ +Cloud99_ X X X X `NA since May 2017 `__ +Browbeat_ (Uses Rally and Shaker) X X X X X `High `__ +Harbinger_ X X `Low `__ +Enos_ (Uses Rally and Shaker) X X X X X `NA since May 2019 `__ +Chaostoolkit-openstack_ X X `NA since April 2019 `__ +Jepsen_ (not specific to OpenStack) X X `High `__ +OPNFV-Doctor_ X `NA since April 2019 `__ +Stepler_ (Uses OS-faults) X `NA since February 2019 `__ +NFVbench_ X X X X X `Medium `__ +VMTP_ X X X `Low `__ +Eris_ (In development) X X X X X X X `NA since November 2017 `__ +=================================== ======= ======= ======= =========== =========== =========== ============ ========== ========== ============================================================================================================== .. [#] Commits per month: Low<5, Medium<15, High=>15, NA=No Activity @@ -101,7 +102,7 @@ possible as well. - Report generation Network load generation, testing and benchmarking. Based on 'iperf(3)' -and 'netperf' -> network testing topologies +and 'netperf' -> network testing topologies. .. _OS-Faults: @@ -122,7 +123,7 @@ Used by Rally to introduce faults. - Benchmarking of data plane and storage services in OpenStack A fully integrated framework which is easy to use with basic knowledge -of OpenStack but isn't flexible or programmable to do everything. +of OpenStack. .. _HA-Failure-Test: @@ -144,7 +145,7 @@ of OpenStack but isn't flexible or programmable to do everything. - Runner: Perform actual tests with disruptions - All 3 runs as seperate threads and parallel -Makes use of Rally runner +Makes use of Rally runner. .. _Browbeat: @@ -168,8 +169,8 @@ Makes use of Rally runner .. _Enos: -`Enos `__ -================================================== +`Enos `__ +================================================= - Deployment of OpenStack on VMs or bare metal - Setup with different configurations @@ -200,7 +201,7 @@ A minimal chaostoolkit driver for OpenStack compute resources. - Checkers for analyzing history Not specific to OpenStack but general library for distributed systems, -written in Clojure +written in Clojure. .. _OPNFV-Doctor: @@ -222,7 +223,7 @@ written in Clojure - Test the availability of OpenStack cluster incase of node/service failure(s) -Make use of os-faults +Make use of os-faults. .. _NFVbench: @@ -231,7 +232,20 @@ Make use of os-faults - Measure network performance in OpenStack - Data plane performance testing +- Benchmarking NFVi platforms and packet paths that are critical for NFV workloads +Adjusted more for NFV workloads. + +.. _VMTP: + +`VMTP `__ +================================================= + +- Benchmarking VM to VM data path performance measurement +- Latency measurement +- TCP/UDP throughput measurement + +Similar to NFVbench but more adapted towards enterprise workloads. .. _Eris: