f1d20ec569
The UI test section otdated and incomplete. This commit updates docs with up to date instructions Change-Id: I42563f76fbee8af3b7a45fb369de57dfd50d1363
208 lines
8.1 KiB
ReStructuredText
208 lines
8.1 KiB
ReStructuredText
..
|
||
Copyright 2014 2014 Mirantis, Inc.
|
||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||
not use this file except in compliance with the License. You may obtain
|
||
a copy of the License at
|
||
|
||
http//www.apache.org/licenses/LICENSE-2.0
|
||
|
||
Unless required by applicable law or agreed to in writing, software
|
||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||
License for the specific language governing permissions and limitations
|
||
under the License.
|
||
|
||
==================================
|
||
Murano automated tests description
|
||
==================================
|
||
This page describes automated tests for a Murano project:
|
||
|
||
* where tests are located
|
||
* how they are run
|
||
* how execute tests on a local machine
|
||
* how to find the root of problems with FAILed tests
|
||
|
||
Murano continuous integration service
|
||
=====================================
|
||
Murano project has separate CI server, which runs tests for all commits and verifies that new code does not break anything.
|
||
|
||
Murano CI uses OpenStack QA cloud for testing infrastructure.
|
||
|
||
Murano CI url: https://murano-ci.mirantis.com/jenkins/ Anyone can login to that server, using launchpad credentials.
|
||
|
||
There you can find each job for each repository: one for the **murano** and another one for **murano-dashboard**.
|
||
|
||
* "gate-murano-dashboard-ubuntu\*" verifies each commit to murano-dashboard
|
||
repository
|
||
* "gate-murano-ubuntu\*" verifies each commit to murano repository
|
||
|
||
Other jobs allow to build and test Murano documentation and perform another useful work to support Murano CI infrastructure.
|
||
All jobs are run on fresh installation of operating system and all components are installed on each run.
|
||
|
||
Murano automated tests: UI tests
|
||
================================
|
||
|
||
The murano project has a web user interface and all possible user scenarios should be tested.
|
||
All UI tests are located at the https://git.openstack.org/cgit/openstack/murano-dashboard/tree/muranodashboard/tests/functional
|
||
|
||
Automated tests for Murano Web UI are written in Python using special Selenium library. This library is used to automate web browser interaction from Python.
|
||
For more information please visit https://selenium-python.readthedocs.org/
|
||
|
||
|
||
Prerequisites:
|
||
++++++++++++++
|
||
|
||
* Install the Python module called nose by performing **easy_install nose**
|
||
or **pip install nose**. This will install the nose libraries, as well as
|
||
the nosetests script, which you can use to automatically discover and run tests.
|
||
* Install external Python libraries, which are required for Murano Web UI
|
||
tests: **testtools** and **selenium**.
|
||
* Verify that you have one of the following web browsers installed:
|
||
|
||
* Mozilla Firefox 46.0
|
||
|
||
.. note::
|
||
|
||
If you do not have firefox package out of the box,
|
||
install and remove it. Otherwise, you will need to install
|
||
dependant libraries manually. To downgrade Firefox:
|
||
|
||
.. code-block:: console
|
||
|
||
apt-get remove firefox
|
||
wget https://ftp.mozilla.org/pub/firefox/releases/46.0/linux-x86_64/en-US/firefox-46.0.tar.bz2
|
||
tar -xjf firefox-46.0.tar.bz2
|
||
rm -rf /opt/firefox
|
||
mv firefox /opt/firefox46
|
||
ln -s /opt/firefox46/firefox /usr/bin/firefox
|
||
|
||
* Google Chrome
|
||
|
||
* To run the tests on a remote server, configure the remote X server. Use VNC Software to see the test results in real-time.
|
||
|
||
#. Specify the display environment variable:
|
||
|
||
.. code-block:: console
|
||
|
||
$DISPLAY=: <value>
|
||
|
||
#. Configure remote X server and VNC Software by typing:
|
||
|
||
.. code-block:: console
|
||
|
||
apt-get install xvfb xfonts-100dpi xfonts-75dpi xfonts-cyrillic xorg dbus-x11
|
||
"Xvfb -fp "/usr/share/fonts/X11/misc/" :$DISPLAY -screen 0 "1280x1024x16" &"
|
||
apt-get install --yes x11vnc
|
||
x11vnc -bg -forever -nopw -display :$DISPLAY -ncache 10
|
||
sudo iptables -I INPUT 1 -p tcp --dport 5900 -j ACCEPT
|
||
|
||
Download and run tests:
|
||
+++++++++++++++++++++++
|
||
|
||
First of all make sure that all additional components are installed.
|
||
|
||
* Clone murano-dashboard git repository:
|
||
|
||
* git clone git://git.openstack.org/openstack/murano-dashboard*
|
||
|
||
* To change the default settings:
|
||
|
||
#. Specify the Murano Repository URL variable for Horizon local settings in ``murano_dashboard/muranodashboard/local/local_settings.d/_50_murano.py``:
|
||
.. code-block:: console
|
||
|
||
MURANO_REPO_URL = 'http://localhost:8099'
|
||
|
||
#. Copy ``muranodashboard/tests/functional/config/config.conf.sample`` to``config.conf``.
|
||
|
||
#. Set appropriate urls and credentials for your OpenStack lab. Only admin users are appropriate.
|
||
|
||
|
||
::
|
||
|
||
[murano]
|
||
|
||
horizon_url = http://localhost/dashboard
|
||
murano_url = http://localhost:8082
|
||
user = ***
|
||
password = ***
|
||
tenant = ***
|
||
keystone_url = http://localhost:5000/v3
|
||
|
||
|
||
|
||
All tests are kept in *sanity_check.py* and divided into 10 test suites:
|
||
|
||
* TestSuiteSmoke - verification of Murano panels; check, that could be open without errors.
|
||
* TestSuiteEnvironment - verification of all operations with environment are finished successfully.
|
||
* TestSuiteImage - verification of operations with images.
|
||
* TestSuiteFields - verification of custom fields validators.
|
||
* TestSuitePackages - verification of operations with Murano packages.
|
||
* TestSuiteApplications - verification of Application Catalog page and of application creation process.
|
||
|
||
* TestSuiteAppsPagination - verification of apps pagination in case of many applications installed.
|
||
* TestSuiteRepository - verification of importing packages and bundles.
|
||
* TestSuitePackageCategory - verification of main operations with categories.
|
||
* TestSuiteCategoriesPagination - verification of categories pagination in case of many categories created.
|
||
* TestSuiteMultipleEnvironments - verification of ability to apply action to multiple environments.
|
||
|
||
To specify which tests/suite to run, pass test/suite names on the command line:
|
||
|
||
* to run all tests: ``nosetests sanity_check.py``
|
||
* to run a single suite: ``nosetests sanity_check.py:<test suite name>``
|
||
* to run a single test: ``nosetests sanity_check.py:<test suite name>.<test name>``
|
||
|
||
|
||
In case of SUCCESS execution, you should see something like this:
|
||
|
||
::
|
||
|
||
.........................
|
||
|
||
Ran 34 tests in 1.440s
|
||
|
||
OK
|
||
|
||
In case of FAILURE, folder with screenshots of the last operation of tests that finished with errors would be created.
|
||
It's located in *muranodashboard/tests/functional* folder.
|
||
|
||
There are also a number of command line options that can be used to control the test execution and generated outputs. For more details about *nosetests*, try:
|
||
::
|
||
|
||
nosetests -h
|
||
|
||
|
||
Murano Automated Tests: Tempest Tests
|
||
=====================================
|
||
|
||
All Murano services have tempest-based automated tests, which allow to verify API interfaces and deployment scenarios.
|
||
|
||
Tempest tests for Murano are located at the: https://git.openstack.org/cgit/openstack/murano/tree/murano/tests/functional
|
||
|
||
The following Python files contains basic tests suites for different Murano components.
|
||
|
||
API Tests
|
||
+++++++++
|
||
|
||
Murano API tests are run on devstack gate and located at https://git.openstack.org/cgit/openstack/murano/tree/murano/tests/functional/api
|
||
|
||
* *test_murano_envs.py* contains test suite with actions on murano's environments(create, delete, get and etc.)
|
||
* *test_murano_sessions.py* contains test suite with actions on murano's sessions(create, delete, get and etc.)
|
||
* *test_murano_services.py* contains test suite with actions on murano's services(create, delete, get and etc.)
|
||
* *test_murano_repository.py* contains test suite with actions on murano's package repository
|
||
|
||
Engine Tests
|
||
+++++++++++++++++++
|
||
|
||
Murano Engine Tests are run on murano-ci : https://git.openstack.org/cgit/openstack/murano/tree/murano/tests/functional/engine
|
||
|
||
* *base.py* contains base test class and tests with actions on deploy Murano services such as 'Telnet' and 'Apache'.
|
||
|
||
Command Line Tests
|
||
+++++++++++++++++++++++++
|
||
|
||
Murano CLI tests case are currently in the middle of creation. The current scope is read only operations on a cloud that are hard to test via unit tests.
|
||
|
||
|
||
All tests have description and execution steps in there docstrings.
|