openstack/fuel-qa
Code Issues Proposed changes
934a9a30bf
fuel-qa/doc/base_tests.rst
Alexander Kurenyshev 7e9484782a Tests for unlock settings tab from different cluster states 
Add tests for Unlock Settings Tab Feature for
partially deployed cluster and for failed
deployment.

Check unlock settings tab after partially deploy
  Scenario:
    1. Revert snapshot ready_with_3_slaves
    2. Create a new env
    3. Add controller and 2 computes
    4. Provision nodes without deploy
    5. Select some nodes (not all) and deploy them
    6. Download current settings and modify some of them
    7. Upload changed settings
    8. Re-deploy cluster
    9. Run OSTF
    10. Make snapshot
  Duration 90m
  Snapshot partially_deployed_unlock

Check unlock settings tab after failed deploy
  Scenario:
    1. Revert snapshot ready_with_3_slaves
    2. Create a new env
    3. Add controller and 2 computes
    4. Change netconfig task to fail deploy
    5. Deploy the env
    6. Download current settings and modify some of them
    7. Upload changed settings
    8. Change netconfig task to normal state
    9. Re-deploy cluster
    10. Run OSTF
    11. Make snapshot

  Duration 60m
  Snapshot failed_deploy_unlock

Change-Id: I84759ae0a603c4749dce2ac9b36c3a3c822514ee
(cherry picked from commit 399114cc56)
2016-05-23 13:30:10 +03:00

22 KiB
Raw Blame History

Base tests

General OpenStack/Fuel Tests

General tests

Base Test Case

fuelweb_test.tests.base_test_case

Admin Node Tests

fuelweb_test.tests.test_admin_node

Test Admin Node Backup-Restore

fuelweb_test.tests.test_backup_restore

Test Bonding base

fuelweb_test.tests.test_bonding_base

Test Bonding

fuelweb_test.tests.test_bonding

Test Bond offloading types

fuelweb_test.tests.test_bond_offloading

Test Ceph

fuelweb_test.tests.test_ceph

Test Cli

fuelweb_test.tests.test_cli

Test Cli Base

fuelweb_test.tests.test_cli_base

Test Cli role component (creade/update/delete role)

fuelweb_test.tests.tests_cli.test_cli_role

Test Cli deploy (deploy neutron tun)

fuelweb_test.tests.tests_cli.test_cli_deploy

Test Cli deploy ceph neutron tun

fuelweb_test.tests.tests_cli.test_cli_deploy_ceph

Test Clone Environment

fuelweb_test.tests.test_clone_env

Test custom hostname

fuelweb_test.tests.test_custom_hostname

Test custom graph

fuelweb_test.tests.tests_custom_graph.test_custom_graph

Prepare target image file

fuelweb_test.config_templates.prepare_release_image

Test DPDK

fuelweb_test.tests.test_dpdk

Test Environment Action

fuelweb_test.tests.test_environment_action

Test ha NeutronTUN deployment group 1 (controller+baseos multirole and ceph for images/objects)

fuelweb_test.tests.tests_deployments.tests_neutron_tun.test_ha_tun_group_1

Test ha NeutronTUN deployment group 2 (ceph for all, baseos node and ceph for all, untag networks and changed OS credentials)

fuelweb_test.tests.tests_deployments.tests_neutron_tun.test_ha_tun_group_2

Test ha NeutronTUN deployment group 3 (5 controllers, ceph for images/ephemeral and no volumes, ceph for images/ephemeral)

fuelweb_test.tests.tests_deployments.tests_neutron_tun.test_ha_tun_group_3

Test ha neutron vlan deployment group 1 (cinder/ceph for images and ceph for volumes/swift)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_1

Test ha neutron vlan deployment group 2 (cinder/ceph for ephemeral and cinder/ceph for images/ephemeral)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_2

Test ha neutron vlan deployment group 3(no volumes storage/ceph volumes, ephemeral)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_3

Test ha neutron vlan deployment group 4(cinder volumes, ceph images and rados gw/ default storage)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_4

Test ha neutron vlan deployment group 5 (ceph for volumes/images/ephemeral/rados and cinder/ceph for images/ephemeral/rados)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_5

Test ha neutron vlan deployment group 6 (no volumes and ceph for images/ephemeral/rados and ceph for volumes/images/ephemeral)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_6

Test ha neutron vlan deployment group 7 (no volumes/ceph for images and cinder/swift/base os)

fuelweb_test.tests.tests_deployments.tests_neutron_vlan.test_ha_vlan_group_7

Test Sahara OS component with vlan and ceph

fuelweb_test.tests.tests_os_components.test_sahara_os_component

Test Murano OS component with vlan

fuelweb_test.tests.tests_os_components.test_murano_os_component

Test mixed OS components

fuelweb_test.tests.tests_os_components.test_mixed_os_components

Test failover group 1

fuelweb_test.tests.tests_strength.test_failover_group_1

Test failover group 2

fuelweb_test.tests.tests_strength.test_failover_group_2

Test failover group 3

fuelweb_test.tests.tests_strength.test_failover_group_3

Test failover mongo

fuelweb_test.tests.tests_strength.test_failover_mongo

Test Mongo Multirole

fuelweb_test.tests.tests_multirole.test_mongo_multirole

Test scale neutron vlan deployment add/delete compute/cinder+cinder+ceph

fuelweb_test.tests.tests_scale.test_scale_group_5

Test scale neutron tun deployment add/delete compute+cinder+ceph+ephemeral

fuelweb_test.tests.tests_scale.test_scale_group_6

Test High Availability on one controller

fuelweb_test.tests.test_ha_one_controller

Test High Availability on one controller base

fuelweb_test.tests.test_ha_one_controller_base

Test jumbo frames

fuelweb_test.tests.test_jumbo_frames

Test manual VIP allocation

fuelweb_test.tests.test_manual_vip_allocation

Test Multiple Networks

fuelweb_test.tests.test_multiple_networks

Test multirole group 1 (controller+ceph/compute+cinder and controller+ceph+cinder/compute+ceph+cinder)

fuelweb_test.tests.tests_multirole.test_multirole_group_1

Test network templates base

fuelweb_test.tests.test_net_templates_base

Test network templates

fuelweb_test.tests.test_net_templates

Test multiple networks templates

fuelweb_test.tests.test_net_templates_multiple_networks

Test Neutron

fuelweb_test.tests.test_neutron

Test Neutron Public

fuelweb_test.tests.test_neutron_public

Test Neutron VXLAN

fuelweb_test.tests.test_neutron_tun

Test Neutron VXLAN base

fuelweb_test.tests.test_neutron_tun_base

Test Neutron IPv6 base functionality

fuelweb_test.tests.test_neutron_ipv6

Test Node reinstallation

fuelweb_test.tests.test_node_reinstallation

Test Node reassignment

fuelweb_test.tests.test_node_reassignment

Test offloading types

fuelweb_test.tests.test_offloading_types

Test Pull Requests

fuelweb_test.tests.test_pullrequest

Test Reduced Footprint

fuelweb_test.tests.test_reduced_footprint

Test scale group 1 (add controllers with stop and add ceph nodes with stop)

fuelweb_test.tests.tests_scale.test_scale_group_1

Test scale group 2 (replace primary controller and remove 2 controllers)

fuelweb_test.tests.tests_scale.test_scale_group_2

Test scale group 3 (add/delete compute and add/delete cinder)

fuelweb_test.tests.tests_scale.test_scale_group_3

Test scale group 4 (add/delete ceph and add/delete cinder+ceph)

fuelweb_test.tests.tests_scale.test_scale_group_4

Test Services

fuelweb_test.tests.test_services

Test Ubuntu bootstrap

fuelweb_test.tests.test_ubuntu_bootstrap

Test Ubuntu Cloud Archive

fuelweb_test.tests.tests_uca.test_uca

Test Vcenter

fuelweb_test.tests.test_vcenter

Test Ironic

fuelweb_test.tests.test_ironic_base

Test Services reconfiguration

fuelweb_test.tests.test_services_reconfiguration

Test Support HugePages

fuelweb_test.tests.test_support_hugepages

Test CPU pinning

fuelweb_test.tests.test_cpu_pinning

Test extra computes

fuelweb_test.tests.tests_extra_computes.base_extra_computes

fuelweb_test.tests.tests_extra_computes.test_rh_basic_actions

fuelweb_test.tests.tests_extra_computes.test_rh_migration

Test Daemon Resource Allocation Control

fuelweb_test.tests.test_cgroups

Test LCM base

fuelweb_test.tests.tests_lcm.base_lcm_test

Test task idempotency

fuelweb_test.tests.tests_lcm.test_idempotency

Test task ensurability

fuelweb_test.tests.tests_lcm.test_ensurability

Test for unlock settings tab from different cluster states

fuelweb_test.tests.test_states_unlock_settings_tab

Gating tests

Test Fuel agent

gates_tests.tests.test_review_in_fuel_agent

Test Fuel cli

gates_tests.tests.test_review_in_fuel_client

Test Fuel astute

gates_tests.tests.test_review_in_astute

Test Fuel nailgun agent

gates_tests.tests.test_nailgun_agent

Test Fuel web

gates_tests.tests.test_review_fuel_web

Fuel mirror verification

Tests to check that mirror is created in various scenarios

Fuel create mirror is made to simplify process of mirror creation for our customers who do not have internet access on-site. It is rewritten from bash to python.

Fuel create mirror features:

  1. Minimize size of packages in a mirror;
  2. Download packages in parallel.

Such features can cause some problems:

  1. During packages resolving to minimize mirror size we found such issues:

1.1) Incorrect versions. When we have multiple mirrors, some version can be skipped due to name duplication. But it is still needed by bootstrap/deploy.

1.2) Mirror/version collisions. Sometimes package present in number of mirrors and not always correct version corresponds to correct site.

1.3) There are special mirror on Fuel iso, which differs from http://mirror.fuel-infra.org/ .

  1. With concurrent packages fetching complications are:

2.1) Some mirrors are unable to support download in multiple threads and fail or reject to support concurrency. In such cases we are abandoning concurrent downloads on such mirrors.

2.2) Common concurrency pitfalls: race conditions for resources like lists to process.

2.3) Problems with offset based downloads. Some packages were broken and it had been found out only during package installation.

fuelweb_test.tests.tests_mirrors.test_create_mirror

Tests to verify installation from packages mirrors

After mirror is created we should be able to deploy environment with it.

Fuel-mirror updates default repo urls for deployment and we do not have to set them up for new environments.But be careful. If you want to deploy environments with vanilla mirrors from iso, You should update settings in environment. Currently there is no option to update default mirrors from UI/cli.

Fuel-mirror updates repo list with internal structures: https://github.com/bgaifullin/packetary/blob/packetary3/contrib/fuel_mirror/fuel_mirror/commands/create.py#L224-L243

Repository should be able to do two things:

  1. Create bootstrap iso for provisioning;

2) Provide packages for deployment. Packages from dependencies in http://mirror.fuel-infra.org/ do not cover all the needed packages. So we need to mix in list of required packages: https://github.com/bgaifullin/packetary/blob/packetary3/contrib/fuel_mirror/etc/config.yaml#L46-L96

Problems:

1) We need to install not only 'depends', but also 'recommends' packages: https://wiki.ubuntu.com/LucidLynx/ReleaseNotes/#Recommended_packages_installed_by_default http://askubuntu.com/questions/18545/installing-suggested-recommended-packages

2) We have a problem with support of a custom packages list. It is only tracked via system test failure without exact team assigned for a job. Also debootstrap and other tools are not informative about package errors. It may fail with 'unable to mount', '/proc not mounted', 'file not found' even if a problem is a missing package.

fuelweb_test.tests.tests_mirrors.test_use_mirror

GD based tests

Test Neutron

fuelweb_test.tests.gd_based_tests.test_neutron

Test Neutron Vlan Ceph Mongo

fuelweb_test.tests.gd_based_tests.test_neutron_vlan_ceph_mongo

Plugins tests

Contrail tests

fuelweb_test.tests.plugins.plugin_contrail.test_fuel_plugin_contrail

Elasticsearch-Kibana tests

fuelweb_test.tests.plugins.plugin_elasticsearch.test_plugin_elasticsearch

Emc tests

fuelweb_test.tests.plugins.plugin_emc.test_plugin_emc

Example tests

fuelweb_test.tests.plugins.plugin_example.test_fuel_plugin_example

Example tests for plugin installation after cluster create

fuelweb_test.tests.plugins.plugin_example.test_fuel_plugin_example_postdeploy

Glusterfs tests

fuelweb_test.tests.plugins.plugin_glusterfs.test_plugin_glusterfs

InfluxDB-Grafana tests

fuelweb_test.tests.plugins.plugin_influxdb.test_plugin_influxdb

Lbaas tests

fuelweb_test.tests.plugins.plugin_lbaas.test_plugin_lbaas

LMA collector tests

fuelweb_test.tests.plugins.plugin_lma_collector.test_plugin_lma_collector

LMA infrastructure alerting tests

fuelweb_test.tests.plugins.plugin_lma_infra_alerting.test_plugin_lma_infra_alerting

Reboot tests

fuelweb_test.tests.plugins.plugin_reboot.test_plugin_reboot_task

Vip reservation tests

fuelweb_test.tests.plugins.plugin_vip_reservation.test_plugin_vip_reservation

Zabbix tests

fuelweb_test.tests.plugins.plugin_zabbix.test_plugin_zabbix

Patching tests

Patching tests

fuelweb_test.tests.tests_patching.test_patching

Security tests

Nessus scan tests

fuelweb_test.tests.tests_security.test_run_nessus

Strength tests

Cic maintenance mode tests

fuelweb_test.tests.tests_strength.test_cic_maintenance_mode

Failover tests

fuelweb_test.tests.tests_strength.test_failover

Base failover tests

fuelweb_test.tests.tests_strength.test_failover_base

Failover with CEPH tests

fuelweb_test.tests.tests_strength.test_failover_with_ceph

Huge environments tests

fuelweb_test.tests.tests_strength.test_huge_environments

Image based tests

fuelweb_test.tests.tests_strength.test_image_based

Base load tests

fuelweb_test.tests.tests_strength.test_load_base

Load tests

fuelweb_test.tests.tests_strength.test_load

Master node failover tests

fuelweb_test.tests.tests_strength.test_master_node_failover

Neutron tests

fuelweb_test.tests.tests_strength.test_neutron

Base Neutron tests

fuelweb_test.tests.tests_strength.test_neutron_base

OSTF repeatable tests

fuelweb_test.tests.tests_strength.test_ostf_repeatable_tests

Repetitive restart tests

fuelweb_test.tests.tests_strength.test_repetitive_restart

Restart tests

fuelweb_test.tests.tests_strength.test_restart

Upgrade tests

Test Data-Driven Upgrade

fuelweb_test.tests.tests_upgrade.test_data_driven_upgrade

OS upgrade tests

Test OpenStack Upgrades

fuelweb_test.tests.test_os_upgrade

Tests for separated services

Test for separate keystone service

fuelweb_test.tests.tests_separate_services.test_separate_keystone

Test for separate keystone service and ceph

fuelweb_test.tests.tests_separate_services.test_separate_keystone_ceph

Test for separate horizon service

fuelweb_test.tests.tests_separate_services.test_separate_horizon

Test for separate mysql service

fuelweb_test.tests.tests_separate_services.test_separate_db

Test for separate mysql service and ceph

fuelweb_test.tests.tests_separate_services.test_separate_db_ceph

Test for separate multiroles

fuelweb_test.tests.tests_separate_services.test_separate_multiroles

Test for separate rabbitmq service

fuelweb_test.tests.tests_separate_services.test_separate_rabbitmq

Test for separate rabbitmq service and ceph

fuelweb_test.tests.tests_separate_services.test_separate_rabbitmq_ceph

Deployment with platform components

fuelweb_test.tests.tests_separate_services.test_deploy_platform_components

Test for ssl components

fuelweb_test.tests.test_ssl

Test for network outage

fuelweb_test.tests.tests_strength.test_network_outage

Test for Image Based Provisioning

fuelweb_test.tests.tests_ibp.test_ibp

Tests for configDB api

fuelweb_test.tests.tests_configdb.test_configdb_api

Test for tracking /etc dir by etckeeper plugin

fuelweb_test.tests.plugins.plugin_etckeeper.test_plugin_etckeeper

Test for separate master node deployment

system_test.tests.test_centos_master_deploy_ceph

Tests for configDB api

fuelweb_test.tests.tests_configdb.test_configdb_api

Test for multipath devices

fuelweb_test.tests.test_multipath_devices