RETIRED, CI for the TripleO project
Go to file
Gabriele Cerami 7f9e1017b0 Collect logs: Handle errors and timeout
currently if the collect logs script has error or is timing out, none of
the copy tasks are run, and we may end up with no logs at all even on
the log collection itself.
This patch encloses the log collection within a block to handle errors,
and runs the script with a specified timeout, to always leave some time
for copying what logs we have from the node to the executor

Change-Id: I428bd0aa8e35a2a94f2cf3039dd9e3ae683334a6
2019-01-15 12:39:35 +00:00
docs [Documentation] Move the link to the documentation. 2018-12-04 10:11:47 +01:00
heat-templates Implement major upgrade for Newton to Ocata 2017-01-13 11:46:52 +00:00
playbooks Collect logs: Handle errors and timeout 2019-01-15 12:39:35 +00:00
roles Removes EXTRA_TAGS from toci_gate_test&quickstart.sh.j2 2019-01-04 14:52:28 +00:00
scripts Remove dead code 2019-01-09 09:48:29 -05:00
test-environments Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
toci-quickstart/config Merge "Add option to run OVB jobs without te-broker" 2018-12-17 15:26:21 +00:00
undercloud-configs Remove undercloud-configs/net-config-multinode.yaml 2018-03-04 16:51:39 +00:00
zuul.d Make scenario004-standalone voting too 2019-01-10 14:50:49 +02:00
.ansible-lint Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
.gitignore Add python script to dynamically compose releases 2018-05-16 11:47:27 +02:00
.gitreview Add .gitreview file 2013-07-22 15:03:30 +01:00
.pre-commit-config.yaml Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
.yamllint Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
bindep.txt Add bindep.txt file 2018-04-23 13:56:24 -04:00
README.rst [Documentation] Move the link to the documentation. 2018-12-04 10:11:47 +01:00
requirements.txt Add a script to compare reviews CI artifacts 2018-09-19 15:29:22 +01:00
setup.cfg [Core] Change openstack-dev to openstack-discuss. 2018-12-04 11:40:39 +01:00
setup.py Add setup/tox/requirements files 2015-09-28 10:18:38 -04:00
test-requirements.txt Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
testenv-client Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
toci_gate_test.sh update toci_gate_test for no tebroker 2019-01-10 16:46:30 +00:00
toci_quickstart.sh Adopt use of pre-commit linting 2018-12-14 15:50:06 +00:00
tox.ini fix tox python3 overrides 2019-01-11 06:02:10 +00:00

toci

Description

TripleO CI test framework.

Tools to help run CI jobs for TripleO. Includes things like:

  • Shell scripts to help execute jobs on CI slave nodes (Jenkins slaves)
  • A test environment broker framework which uses a client-server model to execute jobs on a remote bare metal machine in an isolated test environment (using VMs).
  • Image elements to help build images for the test environment broker nodes.
  • Heat templates to help deploy and maintain test environment nodes using an undercloud.
  • Helper script(s) to generate CI status reports. (tox -ecireport -- -f)
  • Helper getthelogs script to download important job logs locally. Then you may want to inspect the logs for known errors and contribute discovered search patterns as the elastic-recheck queries.

OpenStack Infrastructure is deploying multiple jobs with different scenarios. OpenStack services are balanced between different scenarios because OpenStack Infastructure Jenkins slaves can not afford the load of running everything on the same node.

Usage Details

On March 2017, the ansible quickstart framework was added to TOCI to gradually replace the bash scripts that drove the jobs. Part of the original framework has been changed to allow the new framework to handle jobs, but maintaining backwards compatibility with the original framework while jobs are being transitioned

TOCI entry point

Upon starting a job, based on the configuration of its layout, zuul will call devstack-gate, which is needed for the basic nodepool node setup, but will then pass the control to toci_gate_test.sh. During the transition, this will be a symbolic link to toci_gate_test-oooq.sh.

Quickstart Transition scripts

The initial part of toci_gate_test-oooq.sh script decides whether to exec the original workflow or continue with the quickstart workflow, based on the job type passed down by zuul layout parameters. To move a job to the new quickstart framework, it is enough to propose a change to zuul layout to add a "featureset" keyword on its type. When using the quickstart workflow, the rest of the script will assemble a set of arguments to pass to quickstart scripts, based on the components of the job type separated by dashes e.g. a job type value of "periodic-ovb-featureset001" will make the script assemble arguments to deal with "ovb" provisioning, and set featurset001 to be the test matrix for the job. This script will also invoke the test environment broker to create the proper ovb environment. At the end the toci_gate_test-oooq.sh will pass control to toci_quickstart.sh script that will actually call quickstart with its parameters.

Quickstart Framework

toci_quickstart.sh consists of three parts, setup, invocationo and logs collection.

For more information about feature sets and test matrix please see .. _Featureset Documentation: https://docs.openstack.org/developer/tripleo-quickstart/feature-configuration.html from quickstart documentation The new workflow uses the directory toci-quickstart/ to store TripleO ci specific configurations, roles or playbooks for the quickstart workflow The parts of quickstart under scripts/ are instead handled by the original framework only

Original Framework

Job parameters are configured in toci_gate_test-orig.sh. Control passes to one of the toci_instack_*.sh scripts (depending on the type of job being run) which do environment-specific setup. These scripts then call scripts/deploy.sh to run the actual deployment steps. For most things, deploy.sh simply calls scripts/tripleo.sh with the appropriate parameters.

In ascii art, the flow of the scripts would look like:

toci_gate_test -> toci_instack* -> deploy.sh -> tripleo.sh

There's some additional complexity that this description glosses over, but for the most part only tripleo-ci admins need to worry about it.

temprevert, cherry-pick, pin

There are three functions available in tripleo-ci which can be used to alter the git repos of non-tripleo projects that are used in tripleo-ci. They only work on projects that are part of OpenStack. Use of these should be avoided whenever possible as any changes made during a ci run will not apply to regular users of TripleO. However, they can be useful for determining which commit broke something, and in rare cases we may want to use them until a project can sort out a problem itself.

To apply one of these functions during a ci run, add it to the appropriate location in the toci_instack_*.sh script. There should be a comment that says "Tempreverts/cherry-picks/pins go here."

Note

Do not include the bug number in the commit message where you propose adding one of these functions. Any change whose commit message includes a reference to the bug will not apply the function. This is to allow testing of patches intended to fix the bug.

Warning

As of this writing, these functions all apply against the latest master branch of the project in question. They do not respect the current-tripleo repo versions.

  • temprevert

    Revert a commit from a project. Takes 3 parameters: project, commit id, and bug number. Example:

    temprevert neutron 2ad9c679ed8718633732da1e97307f9fd9647dcc 1654032
  • pin

    Pin to a commit from a project. This usually is not necessary now that our repos are gated by the promotion jobs. Takes 3 parameters: project, commit id, and bug number. Example:

    pin neutron 2ad9c679ed8718633732da1e97307f9fd9647dcc 1654032
  • cherrypick

    Cherry-pick an active review from a project. Takes 3 parameters: project, Gerrit refspec, and bug number. The Gerrit refspec can be found under the download button of the change in question. Example:

    cherrypick neutron refs/changes/49/317949/28 1654032

Service testing matrix

The CI testing matrix for all scenarios is defined in tripleo-heat-templates. This matrix describes the services that will run in each environment.

Feature testing matrix

Feature nonha ha updates

undercloud ssl overcloud ssl ceph ipv4 net-iso ipv6 net-iso pacemaker predictable placement introspection

X X X

X

X

X X

X

X X

Documentation

Please refer to the official TripleO documentation for details regarding TripleO CI.

Please refer to the official TripleO Quickstart documentation for details regarding the tools used in TripleO CI.