52 Commits

Author SHA1 Message Date
Emilien Macchi
70ac070db1 Invert the logic of Puppetfile
I98fc088cd87c1412544f9590ce7a925b413297e2 broke unit 5.5 for
puppet-tripleo and probably puppet-pacemaker, since it can't find the
Puppetfile by default anymore.

Change-Id: I43718a849d09653e6c352b581aef6bbbf372fc41
2019-12-20 17:08:09 -05:00
b44d0bdd5a Add CentOS8 support and jobs
- scenario001 and 004 will be running without ceph
  until ceph repos get available for CentOS8
- scenario003 will run not run with linuxbridge until next CentOS
  minor version as it has issues:-
    - https://bugzilla.redhat.com/show_bug.cgi?id=1720637
- Add Puppetfile_centos7 to have different puppet modules
  as compared to other distros. Currently only puppet-mysql
  needs to be kept pinned for centos7 as newere mariadb version
  is required to remove the pin, CentOS8 and ubuntu have required
  mariadb version so pin is removed. So automatic updates of puppet
  modules will not be applied for CentOS7 as those jobs are going to
  be removed in coming months.
- check for rdo_dlrn url existence only for RedHat distros.

Change-Id: I98fc088cd87c1412544f9590ce7a925b413297e2
2019-12-20 10:18:18 +00:00
Tobias Urdin
fff72afe75 Remove zuul-cloner usage
Depends-On: https://review.opendev.org/#/c/667439/
Change-Id: I6e11c12ad73c89466aefbb926c111ed1d00784b8
2019-06-26 23:16:01 +02:00
ZhongShengping
a8bf4ac2e8 Move to opendev
Change-Id: I467e86b132314d6bf5331ff44a254d38a84dfab0
2019-04-22 09:11:52 +08:00
Tobias Urdin
0ccadc01ed Add Ceph Nautilus support
Adds support to set the jobs to use Ceph
Nautilus when testing.

Change-Id: Ibf2b82435a0f9c29d363c67a973399228bfb90a3
2019-04-04 14:35:41 +02:00
Ian Wienand
6208fc4f3c Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I34c2be045d56d99792fcc1533b67c2c8de1aedac
2019-03-24 20:35:42 +00:00
Tobias Urdin
a5b22a67fe Use hiera v5 format
Use hiera version 5 instead of version 3 which is
deprecated and outputs a warning.

Change-Id: I26ee3329aa75109625476d6c355e82fed2c050e6
2019-01-15 14:49:57 +01:00
0c938590d2 Add support to all Scenarios to work with Fedora and RedHat > 7
Disable SSL as currently there are known issues in services
to work with python3 + SSL on Fedora.

Also fixed following:-
- watcher to work without SSL
- Handle source /etc/bashrc for users don't have .bashrc created
- Install python3-rbd in Fedora and RedHat > 7

Change-Id: I5b67f40ec3c687f5282c65e10b4a1ee1fe5528f3
2019-01-09 20:11:43 +05:30
Tobias Urdin
047ea125dd Add puppet core providers for puppet6
In puppet 6 they have moved out a lot of builtin
resources to their owns modules since they didn't
belong in the Puppet core however in deb/rpm packaging
they are still bundled in but not when installing from
gem.

So we must add them in here so that they are available
in our unit tests.

Change-Id: I7060191395eff9c1e3146cb10594a38bef113117
2018-11-20 11:37:55 +01:00
Tobias Urdin
eace0d6b15 Add puppet6 testing
Add puppet6 testing and run all those on Ubuntu Bionic.
Change from Xenial to Bionic for all other default jobs.

Run libraries unit for 6.0 on Ubuntu Bionic.

Change-Id: I4d9f1d1b62acece9bc0b4ce7bd49d2093af8e5ee
2018-11-15 10:19:25 +01:00
dc9ccd6a20 Ignore sealert notices for Fedora also
Don't fail on selalert notices with scenario-py3,
Fedora jobs are failing on that, so remove the
Fedora check.

Change-Id: I46dc06d3e8e005ff0e18a10d17a75ef9e1c9e23d
2018-09-07 20:05:57 +05:30
Tobias Urdin
2daefe49a8 Change Puppet repo for Bionic
Puppet 5 for Bionic is now released and we don't need
to run the nightly puppet5 builds anymore.

Change-Id: If91bd851f3f67813cc44064be01290b9bd617cbb
2018-07-27 12:27:54 +02:00
Tobias Urdin
a696192bf2 Add Ubuntu 18.04 Bionic to CI
We now have support to Mimic and can select that version
and this patch fixes Puppet 5 for Bionic we can add it to CI.

When this is fixed the only thing left is to add beaker acceptance
testing for Bionic.

Bionic only has Puppet 5 and can currently only use Ceph Mimic.

Change-Id: I6f94e214a70f92278f1f3806471ef2f622a938c3
2018-07-18 16:25:12 +02:00
Cloud User
edca617537 Add new scenario to test python3 services
Also modify provision manifest to skip provisioning
some services.
The new scenario will be used for testing py3 services
starting with keystone and glance.

Also update puppet-apache reference to the required commit,
this can be updated once a new tag is released with the commit.

Change-Id: I55558f35068457175e2ef508f9e5989151db4aa2
2018-07-05 10:55:39 +05:30
Tobias Urdin
1cafb2d4fc Fix select ceph version failing beaker tests
Change-Id: Ie4bd3c46882a1cd4ec2be8c6bb4850e17fe79905
2018-07-03 11:05:30 +02:00
Tobias Urdin
eaacd3f841 Add support to select ceph version
We are closing in on adding Ceph Mimic support and
want to ensure we can select either Luminous or Mimic
for running our tests.

For example Ubuntu 18.04 must run Ceph Mimic because
Luminous is not available (yet), Debian doesn't have
Ceph Mimic packages so we must run Luminous there.

Depends-On: https://review.openstack.org/#/c/576247/
Change-Id: If84ecbf86f3ebde8e69a1a26759ad967d061725a
2018-06-28 19:54:10 +02:00
e931cd247a Allow running poi scenarios on Fedora
Change-Id: I409c11f31d4b82ef9a3d6ba20e2dd045156d66a9
2018-06-13 19:07:13 +05:30
Tobias Urdin
b5ca31399a Use latest puppet5 repos
This change makes sure the latest puppet5 version
is used when running integration jobs for puppet5.

The puppetlabs yum mirror is ready to be used and
has the proper packages and is mirrored, example:

* http://mirror.mtl01.inap.openstack.org/yum-puppetlabs/puppet5/

For the puppetlabs apt mirror the puppet5 pool is
needed before this will work. A patch has been fixed
for that here [1]. Therefore this change cannot be
merged until [1] is merged.

[1] https://review.openstack.org/#/c/565867/

Change-Id: I1a2716fdd4b960b12ff24303038bec5ea8fb80a3
2018-05-08 23:28:18 +02:00
Mohammed Naser
a45fae3c03 Fix MANAGE_REPOS for Debian-based OS
Change-Id: Iab59f0e818f8c5be1beba200f6c475669ce6ba0b
2018-04-16 19:40:14 +02:00
Thomas Goirand
ca8fd0d8b8 Add GPG-KEY-ceph
At first, I thought this would go in puppet-ceph, but in fact,
every compute node needs a librados dependency which is in the
Ceph repo. So at then end, all nodes need GPG-KEY-ceph, so it
is a good idea to install the repository key there.

Change-Id: Ia33718a8350a60c69ce9632eddb6156f9b43c745
2018-04-11 22:24:00 +02:00
Mohammed Naser
f416da3a9a Drop unnecessary clean-ups for Puppet
Nodepool images no longer have Puppet in them anymore so there
is no point in removing files and packages which are not there
anymore in the first place.

Change-Id: Idb89512251ca9125fcb3f44768873d2cb76eb879
2018-04-05 09:41:58 +02:00
Thomas Goirand
7a25b04990 Fixed puppet repo & key in Debian
It appears that the puppetlabs Debian repository is set to always
install a repository for Xenial (instead of using lsb_release to
find the OS), and that the Puppetlabs repo GPG key is outdated. This
patch fixes that.

Change-Id: I35231cdf7503d129bb408bc5d29a2cfe95c4e08e
2018-04-03 14:48:56 +02:00
ZhongShengping
f8d4883f52 Grep command error
Error[1] when executing grep command. add sudo for the grep commands
when it is currently enabled.

[1]:http://logs.openstack.org/63/554163/1/check/puppet-openstack-integration-4-scenario002-tempest-ubuntu-xenial/dd454e9/job-output.txt.gz#_2018-03-19_08_44_16_398090

Change-Id: I5e5e89a4d691fc3890fbc863caa1fc87e42e7d87
Closes-Bug: #1756832
2018-03-19 17:47:42 +08:00
Emilien Macchi
3c8f4ccd85 Revert "Remove r10k env vars with proper args"
It broke all unit tests jobs:
http://logs.openstack.org/98/535698/16/gate/puppet-openstack-unit-4.8-centos-7/bd7dedf/job-output.txt.gz#_2018-02-05_23_19_05_369251


This reverts commit 9f14f75ec650a4563c95cbbe4ccf1aa65f2ba0c7.

Change-Id: Ibcb2bb60a668d232371e4d87febad6d67b3302d8
2018-02-06 03:34:38 +00:00
Zuul
b3bdcfa99d Merge "Remove r10k env vars with proper args" 2018-02-05 21:40:03 +00:00
Emilien Macchi
83e2127db8 Use mirrors to deploy Puppet from Puppetlabs
Change-Id: Ia832c1f8dd27bd1ec919f9156df38a67b4d51cc1
2018-01-30 04:33:41 +00:00
Tobias Urdin
9f14f75ec6 Remove r10k env vars with proper args
The PUPPETFILE and PUPPETFILE_DIR environment
variables are deprecated since several releases
and are superseeded by the --puppetfile and
--moduledir arguments. [0] [1]

This patch replaces all those occurences to future
proof it when the env vars are removed.

[0] http://logs.openstack.org/57/529657/2/check/puppet-openstack-integration-4-scenario004-tempest-ubuntu-xenial/ce6f987/job-output.txt.gz#_2017-12-21_22_52_53_499924
[1] https://github.com/puppetlabs/r10k/pull/479

Change-Id: I216fe01a13e46c90bbb1455df72b25daf899c9f0
2017-12-22 15:29:24 +01:00
Mohammed Naser
bde052c8e4
Add Zuul V3 support for Puppet Jobs
This patch moves in the Puppet OpenStack integration
jobs as well as the module tests in-repo.

Depends-On: Ib96ee354cbf9c88f9265f5743a97b2d013addc70
Change-Id: Idd846c569a6e067125e130f189be9ae8ffba1646
2017-09-28 22:37:09 -04:00
Bogdan Dobrelya
885abd1ae0 Rework r10k verbose option
The command "r10k puppetfile install -v" fails with:
Unrecognized options: verbose.

Rework the command to be compatible with the r10k 1.1.4.

Closes-bug: #1707629

Change-Id: I05d888125284d2808fb248e52afc95ab2b948353
Signed-off-by: Bogdan Dobrelya <bogdando@mail.ru>
2017-07-31 13:35:45 +02:00
Emilien Macchi
9b1bc1bfef Remove workarounds for Hammer / Mitaka zuul clones
We can remove workarounds, we don't support Mitaka & Hammer anymore
(EOL).

Change-Id: I9e4bee87bac23a2d28937249620cb9c8208f4368
2017-07-07 14:11:36 +00:00
Emilien Macchi
01c4f962ae Add support to test Puppet 5
Allow to deploy Puppet 5 (experimental) when setting
PUPPET_MAJ_VERSION=5 (currently done by experimental jobs).

It will pull packages from nightly builds, so we can use it and see if
things are working.

Change-Id: I3d0f2f4309c425435370d7f7d4d367c62396c8fc
2017-06-05 14:36:13 +02:00
Alfredo Moralejo
2887c1652f Don't install puppetlabs repo when MANAGE_REPOS=false
Currently, p-o-i allways tries to install puppet package
from puppetlabs repo. In some cases, as RDO-CI we use p-o-i
to test puppet modules from RDO distro and it's recommended
to test it using puppet version provided in RDO repos which
uses different paths.

This patch provides a way to override this behavior by using
env variables as:

MANAGE_REPOS=false
PUPPET_PKG=puppet
PUPPETFILE_DIR=/etc/puppet/modules

Change-Id: I680504ab65d780deba53387e9b6604d3102de47b
2017-05-30 16:25:33 +02:00
Alex Schultz
7f9907baa6 Drop puppet3 support
Puppet 3 was EOL as of December 31, 2016. Since we don't use it in the
gate for Pike, let's drop the configuration bits in p-o-i for it. This
change leaves the PUPPET_MAJ_VERSION support on the off chance we need
it in the future but basically 4 is currently the only supported
version.

Change-Id: I0f04a3cb194ec2225010fb4e062703195b79baf1
2017-05-26 13:43:35 -06:00
Alfredo Moralejo
845c3288f3 Clean messages in console in puppet errors
I think warning messages doesn't add much value in console
and sometimes we miss messages with (err).

Change-Id: Ic06ad59f71393dcc2f0f861e45d367fb835a14fc
2017-03-08 17:10:32 +01:00
Alex Schultz
5a078e69d3 Fix puppet log move for non-root
If you run something like all-in-one.sh as a non-root user, it will fail
when it goes to move the puppet log file. We need to run it as sudo if
the user is not root.

Change-Id: I8b9d7e587f31a82fb4b59de770882139fd168667
2016-12-02 21:20:31 +00:00
Alex Schultz
e4c6f48efa Split out puppet logs
To improve troubleshooting and readability, let's not have the puppet
logs print to the console.  This change updates the puppet apply to
output the logs to puppet.log which will still be captured and provided
in the CI results.  When the puppet apply fails, we will grep for
warnings and errors from the puppet log and only print those out which
should simplify troubleshooting failures.

Change-Id: I1084223f805f1960aebb9c5868fdef84ff3667e4
2016-11-01 14:41:01 -06:00
Matthew J. Black
305cfa5e61 Update cp line to remove redundant param
The use of -r for cp command is redundant with -a being used as
documentation on Linux has -R and -r as the same parameter. The
issue on Mac OS X is that it will error on about using -R and -r
with cp. Change removes redundant -r from cp command.

Change-Id: If7122efc8b84f8aa8a07fe4f8ee7568a048c9461
2016-09-23 16:03:57 -04:00
David Moreau-Simard
c2551e6145 Allow p-o-i to install itself from current source when used without Zuul
puppet-openstack-integration currently takes care of installing itself.
This is done through the Puppetfile which references the p-o-i repository.

This works with zuul-cloner because Zuul with ensure the current reference
is cloned.
However, when running outside of the gate, this means that p-o-i will
completely disregard any local code and instead install from the git source
specified in the Puppetfile.

Change-Id: I665347dbede2c53bd5537d4a29d07bfe679c50e5
2016-09-21 14:52:56 -04:00
Emilien Macchi
918f49b085 Unpin Puppetlabs release package
Puppetlabs release package was fixed so we can unpin it in our CI.

Change-Id: Ie18ff211b919b8ae4bc5a6d28ab8afe948991ed4
2016-09-09 16:12:28 +00:00
Emilien Macchi
834ab8a5fb Align the stars to bring our CI back
1) Add 2GB swap by default
Add some swap when running CI jobs, so we can reduce the number of
failures in CI related to missing memory.
By default 2GB, can be disabled or increased/decreased with parameters.

2) Promote RDO repository to have Oslo DB from master
python-oslo-db 4.13.x releases are known to break Gnocchi.
https://review.openstack.org/367221 will fix it, but it is not yet
in any release, so RDO unpinned oslo.db until that happens.

3) Disable Ironic testing on Ubuntu. Packages are broken in recent
   Newton upgrade. They are working on it.

4) Enable br_netfilter kernel module on Ubuntu.
See https://bugs.launchpad.net/cloud-archive/+bug/1621651
Even if it's not critical, it's a nice-to-have because it removed the
ERROR that we had in neutron logs. We'll see how the bug report evolve
and maybe remove this workaround.

5) Export lsmod in logs, easier for us to debug later.

6) Disable linuxbridge on scenario003 for Ubuntu
Also see https://bugs.launchpad.net/cloud-archive/+bug/1621651

7) Pin Puppetlabs release package.
Context: https://tickets.puppetlabs.com/browse/CPR-378

Change-Id: I732ef375a681f03af751ebb6ebd5b12df55a0ecc
2016-09-08 20:33:20 -04:00
Emilien Macchi
b1343de9bc functions: clone poi mitaka when cloning ceph/hammer
When submitting a patch in puppet-ceph stable/hammer, we want to clone
puppet-openstack-integration from stable/mitaka.

This patch makes sure of that.

Change-Id: I7a7190f0c9e7257f74672d8a8d63678706c63243
2016-08-15 13:10:55 +00:00
Emilien Macchi
038af6ec7c Workaround for puppet-ceph zuul-cloning
Currently, puppet-ceph stable/hammer CI tries to clone Puppet OpenStack
modules using stable/hammer branch but they don't have it, only
puppet-ceph does.
We need to override ZUUL_BRANCH with a fake 'stable/mitaka' for Puppet
OpenStack modules and then checkout 'stable/hammer' for puppet-ceph.

This workaround will allow us to:
* test puppet-ceph/hammer with mitaka modules
* test puppet-ceph/master with newton and beyond modules

Note: for our r10k users, no problem, they use git to checkout the right
branch, so they don't have the bug.

Change-Id: I004ee95e6bd97992aeebd08447396e3b8a29e09f
2016-08-04 19:02:26 +00:00
Alex Schultz
581e13e6b8 Add hiera support
This change adds hiera support into the scenarios so we can override
items via the common.yaml heira file if need be. This change adds
support for two environment variables with run_tests.sh.

HIERA_CONFIG - full path to the hiera yaml configuration file
MANAGE_HIERA - used to enable/disable writing out a hiera.yaml config
file

Change-Id: Iad0050e09e0cc610e0eadc00e328151e88eee10e
2016-07-15 20:49:10 +00:00
David Moreau-Simard
22c29c79ac Ensure $GEM_BIN_DIR is in $PATH
This makes gem binary usage consistent (i.e, r10k) and prevents
encoutering a command not found error.

Change-Id: I3ef01274417cd88403d71af0e6216850d77cc81c
Closes-Bug: #1592113
2016-06-13 14:11:46 -04:00
Emilien Macchi
5210671dbb refacto: move run_puppet & catch_selinux_alerts into functions
Change-Id: I78324c8787b129e91f1ac32694085bde5b6043fe
2016-06-09 22:09:00 +00:00
Emilien Macchi
c309447bc1 refacto: create install_puppet function
Change-Id: Iefed86a09eaf4e571d38c801f79e9fc989bc57d6
2016-06-09 13:11:05 +00:00
Emilien Macchi
48ab117d2e functions: add defaults for ZUUL_BRANCH & ZUUL_REF
When running periodic jobs, ZUUL_BRANCH & ZUUL_REF are empty because we
don't have any reference or branch set for the jobs.
To not make zuul failing when cloning Puppet modules, let's set defaults
values:

ZUUL_BRANCH default to master (we are running tests against master now)
If we need to test stable branches, we'll revisit later but it would be
a bigger change (job names, etc).

ZUUL_REF default to None.

Change-Id: I34b298230bc174a3fdd431095043543c4c223296
2016-04-20 09:42:31 -04:00
Alex Schultz
20730feff0 Add a function for generating action headers
In order to help be able to determine what part of the script a failure
occurs in, this change adds a print header function and updates the
shell scripts to print some header information for various actions.

Change-Id: I8e622c4d1c9c03aa3dbcb27712a346ce3284003f
2016-04-14 14:57:22 -06:00
Sofer Athlan-Guyot
0de28fbab3 Fix rspec testing outside of the Openstack CI.
When testing rspec locally the r10k binary is never found because the
GEM_BIN_DIR is wrongly set to GEM_HOME/bin.  GEM_HOME is set by bundler
and GEM_HOME/bin do not point where bundler put the binary.  The
binaries are handled by bundler by changing the PATH.  rbenv, use
similar tricks.

install_all is only used outside of the CI, so this modification
wouldn't change anything as far as the official CI is concerned.

Change-Id: I07668d64f9adf6713f5c267cf45650ba92ee0b9c
Closes-Bug: 1550331
2016-02-26 14:03:16 +01:00
Emilien Macchi
e1c20c9b1f Create manifests directory for common bits
Change-Id: Ibf7d3aa962e2e0727f0593f70186521602c48ca6
2015-11-18 15:09:18 +00:00