StarlingX Test tools
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Lemus Contreras, Cristopher J c0c2a40db2 Fix test cases related to lock/unlock of compute and storage nodes 4 weeks ago
Config Change cirros image 5 days ago
Libraries Add test for DISPLAY environment variable 2 months ago
Qemu Update multinode_dedicated_storage.yaml 5 days ago
Resources Update unlock on compute nodes 5 days ago
Tests Fix test cases related to lock/unlock of compute and storage nodes 5 days ago
Utils Remove ifparser module dependency. 5 months ago
Variables [robot] Add libraries and resources used by the suite 7 months ago
baremetal [robot] Add configs and environment setup 7 months ago
docs [robot] Add documentation of the robot suite 7 months ago
testcases Initial directory structure 1 year ago
LICENSE [robot] Add documentation of the robot suite 7 months ago
LICENSE.apache [robot] Add documentation of the robot suite 7 months ago
README.rst Change cirros image 5 days ago
requirements.txt Remove ifparser module dependency. 5 months ago [robot] Initial submission for robot test suite 7 months ago
setup.cfg [robot] Add documentation of the robot suite 7 months ago
test-requirements.txt Enable python3 fort the test suite 6 months ago
tox.ini [robot] Add documentation of the robot suite 7 months ago


StarlingX Test Suite


Table of contents:


This Test Suite provides an automated way to Setup, Provision and do a Sanity Test of the 4 basic StarlingX Deployments options described at StarlingX Installation Guide. Currently the suite has fully support to deploy on virtual environments using Libvirt/Qemu to simulate the nodes, installation on BareMetal is only supported for a very specific infrastructure that is described on BareMetal, complete documentation for this process will be ready soon.

Suite is based on Robot Framework and Python, please follow below instructions to properly use the suite.

Suite should be executed with Python3 (tested with 3.5.2)

Quick Start

This guide is focused on a clean OS installation, any kind of issue not document here the user must solve it.

The recommend OS system is Ubuntu 16.04 LTS, you can download it from the following link:

Download Ubuntu 16.04 LTS.

* Note* Was also tested on Debian 9 and Fedora 27

Updating the system

In order to get the system up-to-date you must run the following commands:

automated-robot-suite repository --------------------------

Installing Git

Before to be able for clone the repository, a tool is needed and you must install it typing the following command:

Cloning the repository

The next step is to make a copy of this repository in your local machine:

Git configuration

Make sure that you have git correctly configured:

If you have any issues please visit Troubleshooting section

Host package requirements

Please execute below steps to enable Qemu-Libvirt on your host

  1. Add your linux user to /etc/sudoers file at the end of the file:

  2. Install the following packages

    Package Description
    virt-manager Display the virtual machine desktop management tool
    libvirt-bin Programs for the libvirt library
    qemu-system QEMU full system emulation binaries
  3. Start the libvirt service daemon with the following command:

  4. Be sure that the daemon is loaded and running

5. Reboot the system in order that the current user be reflected in libvirtd group, needed to run the services related.

Project requirements

Every python project has requirement files, in this case the repository automated-robot-suite has the following files:

  • requirements.txt: which contains all the requirements that the project needs.
  • test-requirements.txt: which contains all the test requirements that the project needs.

Python virtual environments

Python “Virtual Environments” allow Python packages to be installed in an isolated location for a particular application, rather than being installed globally.

Installation on Virtual Environment

Make sure you have python virtualenv package installed in your host machine.

You can manage your virtual environments for the two options explained below:

Managing virtual environments with virtualenvwrapper

While virtual environments certainly solve some big problems with package management, they’re not perfect. After creating a few environments, you will start to see that they create some problems of their own, most of which revolve around managing the environments themselves. To help with this, the virtualenvwrapper tool was created, which is just some wrapper scripts around the main virtualenv tool A few of the more useful features of virtualenvwrapper are that it:

Organizes all of your virtual environments in one location Provides methods to help you easily create, delete, and copy environments Provides a single command to switch between environments

To get started, you can download the wrapper with pip

Once installed, you will need to activate its shell functions, which can be done by running source on the installed script

Using that path, add the following lines to your shell’s startup file which is your ~/.bashrc

Finally, reload your bashrc file

For help and examples on Virtualenvwrapper please visit Help section

Managing virtual environments raw

If you want a more direct way to work with virtual environment on python you can follow below steps:

Install the project requirements on virtual environment.

Now that virtualenv is activated you need to install the needed packages.


Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by os.pathsep (e.g: colons on Unix or semicolons on Windows). Non-existent directories are silently ignored. In addition to normal directories, individual PYTHONPATH entries may refer to zip files containing pure Python modules (in either source or compiled form). Extension modules cannot be imported from zip files. The default search path is installation dependent, but generally begins with /prefix/lib/pythonversion. It is always appended to PYTHONPATH.


PYTHONPATH environment variable is a pre requisite for this project. Please setup PYTHONPATH in your local bashrc like below:1

Using the suite

This section will describe how to configure, interact and run test on the suite based on robot framework, this suite supports two diferent environments Virtual and BareMetal


Virtual deployment is based on qemu/libvirt to create virtual machines that will host the StarlingX deployment

NOTE There are minimum HW requirements to deploy on virtual environments please refer to installation_libvirt_qemu for more details

Download Artifacts

Suite needs an ISO to be installed and the associated Helm Chart to deploy OpenStack services sot they should be downloaded and put inside of automated-robot-suite/ path.

There is daily build under CENGEN infrastructure so there above items can be downloaded form there from:

StarlingX Mirror

ISO = /<release>/outputs/iso/bootimage.iso

HELM_CHART = /<release>/outputs/helm-charts/stx-openstack-<VERSION>-centos-stable-versioned.tgz

Suite Configuration

  • config.ini: This file contains information that will use directly by the suite to Setup a deployment, parameters that should be updated are:

    1. STX_ISO_FILE: The name of the ISO, for automation purposes recommended to let bootimage.iso and create a symlink to the required ISO.

    2. CHART_MANIFEST: With the name of the Helm chart associated to the ISO, as well is recommended to have a symlink
    3. STX_DEPLOY_USER_NAME: The user name to be setup on the deployment.
    4. STX_DEPLOY_USER_PSWD: The password to be setup on the deployment.
  • stx-<configuration>.yml: Is the configuration file used to configure the StarlingX deployment. There is file for Simplex, Duplex and Multinode configurations. The structure of this file is out of the scope of this document please refer to the official StarlingX documentation for more information
  • VM's Resources Yaml: Definition of the resources that will be used by libvirt to create the VM's. those files are stored at Qemu/configs and are set with the minimum resources needed hence values only can be increased according to the host resources.

Suite Execution

The suite is divides in 3 main stages that will be explained below:


In this stage all the virtual machines are created for the specific configuration selected and with the attributes previously defined, the ISO will be installed on the master controller and be configured to be a SatrlingX deployment.


In this stage all other nodes are installed and system is provisioned following the steps defined at StarlingX Installation Guides

Test Execution

In this stage the system is already provisioning and Test can be executed, below are the steps to execute a Sanity-Test suite

  1. Download required images

External: - Cirros - Ubuntu - Centos - Windows

  1. Update config.ini with the name of the downloaded images.

  2. Run Tests


Infrastructure Diagram


PXE client - This is the main StarlingX controller (controller-0).

PXE Server - StarlingX test suite must be executed on this host. Also, these services are running:

  • TFTP - Used to serve uefi/shim.efi file. Indicating where the pxe client is going to connect to download installation packages.
  • HTTP - Serving the full content of an ISO to the pxe client.
  • DHCP - This service assigns a temporal IP address to the pxe client, it also tells the clients where to grab the boot shim file.

These services should be running through OAM network. You need to ensure that TFTP and DHCP are configured properly to serve the shim file. Also, the test suite needs to identify the temporal IP address that the pxe client is going to use.

The following is an example of a DHCP configuration file to assing temporal IP to a pxe client:

host standard_example {
hardware ethernet aa:bb:cc:dd:ee:ff;

Also, you need to have this option on the same dhcp configuration file:

filename "uefi/shim.efi";

Test suite will do the following steps to start an install:

  1. Mount bootimage.iso and expose it with HTTP
  2. Take info from the mounted files to create a custom shim file. This file will automatically setup the required boot options for the pxe client.
  3. It will use BMC network to send a signal to the pxe client, telling it to boot on the first network adapter (pxe boot).
  4. Open a SOL connection to the host to monitor the progress of the install, once completed, it will change sysadmin password to the one defined on the .yml file
  5. Copy required rpms to install secondary nodes. This is done using scp from the pxe server to the pxe client using the temporal IP address

Results and Logs

Every execution on the suite generate a separate directory with logs, this is placed under Results/ and also a a link to the mos recent execution can be acceded by latest-results/ symlink, the list of the available logs is:

  • debug.log: Showing the output form Robot Framework activity.
  • iso_setup_console.txt : Showing the serial output of the ISO installation and Configuration on virtual environments.
  • iso_setup.error.log: Filtering only the errors on the serial console.
  • qemu_setup.error.log: Showing the information related to Qemu and Libvirt
  • log.html: Showing the debug.log in HTML format
  • output.xml: Showing the debug.log in XML format
  • report.html: Showing the results on a visual and customizable format.




  • AttributeError: 'module' object has no attribute 'SSL_ST_INIT'

    This error is because the python module that comes with the distribution is incompatible with pip version. Please do the following steps to fix it:


    This is a common issue and this mean that your system date is out-to-date. To fix this please setup the correct date in your system.


  • Nodes not being installed

    In some cases was seen that during virtual deployment of Duplex or Multi-node the extra nodes (controller-1, computes and storage) are not being installed and keeps waiting for PXE image until timeout expires, we found that for those cases the guilty of causing controllers not booting for pxe is docker, for some reason (not yet discovered why) docker is sending packages to the interfaces used by the VMs to be installed by PXE and this causes unknown traffic on the interface making PXE installation fail. The workaround for now is to kill docker daemon to avoid this issues.


This section will show different topics that could help on he suite usage.

Increase resources on virtual environment

Suite has set the minimum requirements on the virtual machines to support a StarlingX deployment, but is also possible to increase those values if the host machine has enough resources, follow below steps to increase resources

  1. Go to Qemu/configs/ and open yaml file of your configuration
  2. Edit file with the values for:
  • partition_a (in GB)
  • partition_b (in GB)
  • partition_d (in GB)
  • memory_size (in MB)
  • system_cores
  1. Vales can be increased on Controllers, Computes and Storage nodes

Using proxies to download docker images

With the support of containers on StarlingX deployment there is a need of downloading docker images, if you are using a proxy please follow below steps to successfully configure your deployment.

  1. Open your configuration file at Config/stx-<config>.ini and add below section

  2. Save the file and run Setup to have a StarlingX deployment configured with docker proxies.

Using local registry to download docker images

With the support of containers on StarlingX deployment there is a need of downloading docker images, if you don't have access to public repositories you can point docker to sue local registry (how to setup a local registry is out of the scope of this document), follow below steps:

  1. Open your configuration file at Config/stx-<config>.ini and delete [DNS] and [DOCKER_PROXY] if exists
  2. add below section

Virtualenvwrapper useful commands

cmd Description
workon List or change working virtual environments
deactivate Programs for the libvirt library
rmvirtualenv Remove an environment
mkvirtualenv QEMU full system emulation binaries
lsvirtualenv List all of the environments
lssitepackages Shows contents of site-packages directory

Virtualenvwrapper Exampes

  1. where ../ indicates the absolute path to the project.