Commit Graph

90 Commits (master)

Author SHA1 Message Date
Zuul 802fd35cd5 Merge "Better checking for tags when cloning puppet modules" 2020-01-16 23:01:33 +00:00
Zuul 0eaee6c131 Merge "Support puppet5 for bionic" 2018-08-22 21:26:36 +00:00
Ian Wienand 40879dae3b Support puppet5 for bionic
In our wheel build jobs, we try to install puppet with this script on
bionic nodes.  However, there is no <5 puppet distributions for

This detects bionic and installs from puppetlabs (note the name seems
different as it's not pc1 any more)

Change-Id: Ica34b7525bab53f8a6d161401f7fb9a2dbe37bc3
2018-08-15 07:34:18 +10:00
Monty Taylor 67376f4c4c
Update install_modules to not need puppet
The only thing we actually use puppet for in install_modules, amazingly
enough, is doing a module list. That can be accomplished by doing a find
on /etc/puppet/modules.

Change-Id: I9c7969f2acd5a296a48d4fc0bbff47c0b4994ef1
2018-08-03 09:05:13 -05:00
Colleen Murphy d4d0ae0e40 Add playbook to upgrade puppet
Add a playbook to rerun with PUPPET_VERSION=4. Also
make the script smarter about figuring out the puppet
version so that the update_puppet.yaml playbook, which updates the
puppet config and puppet modules but not the puppet package, does not
need to be changed.

When we're ready to start upgrading nodes, we'll add them to the puppet4
group in `modules/openstack_project/files/puppetmaster/groups.txt`.

Change-Id: Ic41d277b2d70e7c25669e0c07e668fb9479b8abf
2018-06-05 00:25:21 +02:00
Colleen Murphy f10f116f60 Fix puppet config for puppet 4
There are some assumptions made about the puppet config paths that
aren't applicable for puppet 4. Configuration, including modules,
belongs under /etc/puppetlabs. It's also no longer necessary to fix up
the templatedir or server configs in puppet.conf.

Change-Id: I3b544b6ce4a96a7a2478522a78402f77ff86a5a5
2018-04-20 14:24:17 +02:00
Ian Wienand 661e91a2fe Better checking for tags when cloning puppet modules
One of the puppet modules changed tags from "2.0" to "v2.1" which I
missed.  The testing was rather unhelpful ... when "2.1" didn't exist
it went ahead and left it checked out on master and then ran the test
suite against that.  So it was quite confusing as to what the root
cause was.

Do a more thorough check of the wanted tag; if it's not there at
checkout time, abort.

Change-Id: Ic358857c56c0b5a95f1052c21bfb89e17a9816cc
2017-12-11 10:22:00 +00:00
Monty Taylor 002e9b5b31 Retry git clones in
github is making it impossible to land changes in system-config. That is
dangerous. So add in a short retry loop to the git clones.

Change-Id: Ie49ae930f93aa9740e40e42935782823ed4dde35
2015-11-28 09:22:53 -05:00
Spencer Krum 5756c6fe8d Remove 'removed' modules, they have been removed
Change-Id: I8d946dccf4337b02833caa4c0bbf12c4023d1792
2015-11-06 15:05:09 -08:00
Spencer Krum 0546ac343d Only install modules from source
Change-Id: I0d42fbcbc93cb033a22fe7621881ea89f4c2754f
2015-11-06 15:04:38 -08:00
Spencer Krum 8cdaf92609 Treat openstack_health normally
This is a partial revert of 05c16e2e that removes special casing around
the openstack-health/openstack_health repo.

Change-Id: I832f1b764a34c128d737a467fa568cc064226c5a
2015-11-06 10:00:14 -08:00
Matthew Treinish 05c16e2e35
Add openstack-health api server
This commit adds the puppet config for running an openstack-health
api instance. It'll use the subunit2sql trove db node as a data
source. The puppet-openstack-health module is incorrectly named which
is causing issues, a workaround to rename the repo is added to This is a temporary measure until the gerrit
rename is completed. There is also a workaround in the apply test.

Change-Id: I7e6d9664d087e7bdc21d92624991d0d5f86c0c99
2015-10-19 15:27:00 -04:00
Yolanda Robla 7cd91b36ed Remove deletion of vcsrepo module on each run
This is creating a race, because it deletes vcsrepo
module every time, and recreate it again, causing it
to be unavailable for long periods during clones.

Change-Id: I8576959ecd2dbba52e08a37bba2dd6938e259002
2015-09-28 16:05:39 +02:00
Paul Belanger ae9d89ac9c Remove puppet-apache now that we have migrated
We now fully support puppet-httpd, so lets remove puppet-apache

Change-Id: I48924b277ccbadf805947ae400b25807f3887cdc
Depends-On: I67aea79c04d725f9b9889a38aff018f0895321a9
Depends-On: I0bcc68e049396560e9f51a2cb4ba941f8a8d2862
Signed-off-by: Paul Belanger <>
2015-08-05 13:48:13 -04:00
Paul Belanger aa35aaabdb Remove color output from puppet list command
This helps when reviewing the log file within Jenkins.

Change-Id: I9855fcc628cf8764cbe53b01247613359e731cd4
Signed-off-by: Paul Belanger <>
2015-07-15 09:40:01 -04:00
Spencer Krum 4cc08edcd3 Set modulepath to the first element of puppet's modulepath.
This duplicates the behavior of puppet module install for the git
clone operations.

Puppet module install will use the first element of the modulepath.
This makes the git clone operations go into the same directory as
the modules installed via the puppet module tool.

This helps us downstream because we are moving the directory that
holds these modules out of /etc/ and into /opt/.

Change-Id: I0c7833a6d9a79fa613efc9daebee916f3192306a
2015-01-22 02:41:20 -08:00
Jenkins c49fb5a0e3 Merge "Turn off color in module install" 2014-12-12 16:59:02 +00:00
Spencer Krum ed46e9303f Source modules.env from
This causes to get the list of puppet modules to do
integration test with from modules.env. Modules.env has been
refactored to have three lists of modules: Package modules(MODULES),

folded into SOURCE_MODULES.

In, INTEGRATION_MODULES is sourced from modules.env
and some string/array manipulation is performed to pass each entry
to zuul-cloner.

Change-Id: I47302c5c7e8c41b985f16a05c4e9b8078ea867a3
2014-11-18 21:23:43 +00:00
Spencer Krum 3d4838ac33 Turn off color in module install
This ends up putting a bunch of ansi escapes in the apply test
output. We could conditionally inject it form the apply test
if people want color when this is run normally.

Change-Id: I004bcd29671344430d271aad7f54186bc725215c
2014-11-17 07:54:17 +00:00
Edward Raigosa (wenlock) 7299bf0326 give install_modules options for loading requirements externally
Use case:  Our team would like to use
  in other places beyond openstack-infra/config, we would
  also like to provide some control over what
  installs beyond what is given as defaults.

  * Added a copyright notice so we can use it in other projects
  * if a modules.env file is found in the same directory
    as then source the modules.env
  * an export called MODULE_ENV_FILE can be included to
    augment the name of the file from modules.env
    to an alternative file name like modules.test.env.
    This gives users of the script an option to provide
    different modules on different test environments.
  * as export to change the path from the default
    directory to an alternate directory can be
    specified with the export MODULE_ENV_PATH var.
    The default directory is the same location as
    the script.

Change-Id: I0c3f9bcb3e91236ceca492426ee6105242a2c8eb
2014-10-20 22:00:48 +00:00
K Jonathan Harker 948aff1f02 Clean up bashate failures
Fix all of the bashate failures.  Also, bashate will install its negative
tests in its virtualenv, so we need to generate a list of files to pass to
bashate that excludes .tox/

Change-Id: I55559bf6137f705aec9a7f277928ad8d4cadc2ca
2014-09-30 12:40:59 -07:00
Jenkins 764934ef74 Merge "Update for new home for puppetboard module" 2014-09-16 23:19:13 +00:00
Ian Wienand 1bdbee9943 Upgrade to puppetlabs-postgresql 4.0.0
This version has a range of fixes required to make the module work on
Fedora and CentOS 7

Change-Id: Ie867cfcc13a81a48e0c83cf70325032ed44bddef
2014-09-16 08:03:35 +10:00
Spencer Krum 179750c5f5 Update for new home for puppetboard module
Change-Id: If5047aae6bfe2901faed1a2177a8b11de325907d
2014-09-11 15:27:39 -07:00
James E. Blair 9d23185ac5 Add integration apply test
Make module installation conditional in so that
we can specify that we _do not_ want it to install the modules
that will be participating in the integration test.

Add a variant of the apply test script (currently that
uses zuul-cloner to check out the appropriate versions of modules
that are participating in the integration test.

Add jobs that run this to the experimental pipeline.

Change-Id: I16055cdd45f6e05aa8ebceb500c5709f265268df
2014-09-03 15:03:09 -07:00
Jenkins c7e5b0d19c Merge "Update git puppet modules properly." 2014-09-02 19:19:01 +00:00
Jenkins 15910f883f Merge "Bump the apt module version from 1.1 to 1.4" 2014-08-29 19:49:14 +00:00
Colleen Murphy fbe1dbe611 Update version of puppetlabs-stdlib in
Without this patch, will attempt to install
puppetlabs-stdlib 3.2.0 and fail because puppetlabs-concat installs
puppetlabs-stdlib > 4. This is a problem because it causes inconsistency
between the version listed in and the actual installed
version, and it causes errors during subsequent runs of the shell script. This
patch resolves the problem by updating the version listed in to the latest version.

Closes-Bug: 1324645
Change-Id: I7822597785946330346c476753ff60ae1b345014
2014-08-28 18:40:44 -05:00
Clark Boylan e98f519fb9 Update git puppet modules properly.
When using git sourced puppet modules we need to specify the desired ref
as a tag, sha1, or remote branch because does not
reset local branches to match remote branches.

Do a git remote update instaed of a git fetch to make sure that all
remotes are updated.

Change-Id: I1785af6fd02e6cda8f5f62b54bdb83efb47b7197
2014-08-28 15:39:36 -07:00
Jenkins 26cf0f7d28 Merge "Use puppet-storyboard@master" 2014-08-20 20:32:18 +00:00
Jenkins 4a44f9931b Merge "Bump postgres module version" 2014-08-19 19:42:47 +00:00
Jenkins e6e2f774ab Merge "Git rev-list --max-count needs an = for assignment" 2014-08-15 20:51:02 +00:00
Michael Krotscheck 2ab45223e3 Use puppet-storyboard@master
This change makes sure that the version of puppet-storyboard in use
is always the most recent one. It also removes example42::puppi, as
as of it is no longer
a requirement.

Depends on

Change-Id: I7024476a8eea9c1bb61f082782fc1149eda37a6e
2014-08-15 09:42:05 -07:00
Michael Krotscheck 9223b79612 Update storyboard version to include rabbitmq
This updates the version of storyboard that is used by infra to include

Change-Id: I8cbfb17715a75cf63dd60ad4bae24bc40b3a6944
2014-08-13 14:48:30 -07:00
Spencer Krum 74bd3d233e Use split-out storyboard module
This uses the source install modules functionality to grab the storyboard
module. It pins to the commit hash right now so we will need to update that

This removes the storyboard module directory entirely since we no longer
need it.

Change-Id: Iceab5501fa0c177f55db1a6b723b6b215fca9ed0
2014-08-13 00:34:50 +00:00
Spencer Krum 9dc552a8a6 Bump postgres module version
The key benefit here is that trusty can autodetect postgres version
without manually setting things.

Change-Id: I066c39e1437dfadf06047a714c716adea30bdb5d
2014-08-12 08:36:56 -07:00
Michael Krotscheck 682019f1fa Modularized StoryBoard Module
In order to get the puppet module for storyboard up to a level where
we can publish it to puppetforge, I did some work on it to create
separate modules which can be used by anyone to install storyboard.

- API and Webclient are now installed via storyboard::application,
  which assumes that you can provide the DB connection criteria.
- storyboard::cert is now a separate class, which accepts either
  files or strings, which generates the SSL certificate and chain
  files for storyboard.
- storyboard::params is our dependency checker.
- storyboard::init will install a standalone, entirely
  self-contained instance of storyboard.
- Added various puppet module files necessary for eventual
  deployment to puppetforge.
- Added documentation for later puppetforge addition.

This patch also includes a new module: example42-puppi, which is a
series of convenience utilities useful for deployment. For example,
puppi::netinstall (used here) will fetch tarballs and zip files and
extract them into a provided directory. It also contains changes to
the storyboard configuration for the new refresh token support patch
in #94363

Change-Id: I6ab8c24b308df38774fc0694d218dcb5022cd899
2014-08-08 10:58:17 -07:00
Jeremy Stanley 3f1491d1e4 Git rev-list --max-count needs an = for assignment
* When invoking git rev-list with --max-count it
only supports = and not space before the value. The previous
implementation was flawed and only ever resulted in git errors about
an unrecognized ref for which it was misinterpreting the --max-count
value as representing.

Change-Id: I11f8f5b6e13d4640d6558f423292ad7e7b89961f
2014-08-04 16:18:53 -07:00
Spencer Krum 671b282f2a Bump the apt module version from 1.1 to 1.4
At this version the apt module also includes a
apt::unattended_upgrades class that we might be able to use
to deprecate our home-spun unattended_upgrades module.

Change-Id: Idd6f3a935cf4d543ee10d909900c8e01947e8ced
2014-07-10 13:29:10 -07:00
K Jonathan Harker a0b5912853 Update git refs before comparing refs
The current check will work since an known tag is already at the right
ref and an unknown tag will give an unequal value from git rev-list. But
this fails to work if the version is, for example, 'origin/master'
rather than a git tag.

Change-Id: I746624649acaa7fab217b8ff5570f5822003b2f3
2014-07-09 14:30:16 -07:00
K Jonathan Harker 7af40db7f2 Revert "Revert "Downgrade puppetlabs-apache to version 0.0.4.""
This reverts commit 3afc75132a.

The new apache module has started managing /etc/httpd/conf/httpd.conf
with a template that has some significant differences than our template
in the cgit module.

Change-Id: I99795d35596f35dfc34e89891155dd2b83e465fe
2014-07-01 17:38:58 -07:00
K Jonathan Harker 3afc75132a Revert "Downgrade puppetlabs-apache to version 0.0.4."
This reverts commit 7b9ea298cf.

Upgrading puppetlabs-apache to version 0.4.0 will give us important
variables in apache::params (specifically $conf_dir and $vdir) which
will allow us to modify the cgit module to run on both Debuntu and RHEL
systems -- which is useful because should be testing all modules
on both systems.

Of the two issues that 7b9ea298 addresses, I believe that our Oneiric
hosts have all gone the way of the dinosaur and we can work around the
docroot check by replacing 'MEANINGLESS ARGUMENT' with


Change-Id: I7c08f85db6810ab28fa044f1923833359271e8ec
2014-06-27 15:29:03 -07:00
Dan ba3eb9f791 Allow modules to be installed from source
This patch adds logic that can be used to clone
modules from remote git repositories and set the
resulting local repos to a know revision.

This logic has been added to make it easier to
decouple modules that are currently included as
a part of config into their own repos. Allowing
source installation will allow for edits of these
modules to be consumed more easily (without requiring
an additional release to the forge).

Additionally, this patch removes the puppet forge
installation logic that was installing without deps.
If modules are needed without their dependencies, they
can be installed from source to limit the amounts of ways
that modules can be installed.

Change-Id: I6e25cd65569f82bd5b0ef7d00b32e93ecdda3d71
2014-06-10 17:07:04 -07:00
Clark Boylan 30c2f7d462 Remove puppet dashboard.
Puppet board has replaced puppet dashboard. Remove reference to puppet
dashboard as it shouldn't be used.

Change-Id: I5eeee2984729ef5d1b883b4762347d19786e28ed
2014-04-17 14:48:19 -07:00
Spencer Krum 101e58991c Further changes to bring puppetboard online
Use inherits keyword to bring puppetboard::params into scope
Use 2>&1 in install_modules to silence warnings
Set puppetdb class to open port 80 when puppetboard is used
Remove inherits from params check from puppet lint

Change-Id: I75fc35542e9f2641bcab613d7e1aef83d9300217
2014-03-22 12:54:38 -07:00
Spencer Krum 5740ed3118 Add PuppetBoard to openstack infra
This places the puppet web dashboard puppetboard from onto the puppetdb server.

This enables anyone to see the output of puppet runs, which nodes
are failing and which nodes have not checked in over time. We can
also quickly and interactively query the facts database.

This also modifies to support a list of modules
that will be installed without dependency resolution.

Change-Id: I09d09410dd8fa71759ce6b2f9c7f34f78eefe204
2014-03-20 00:50:54 -07:00
Spencer Krum 04b114ba82 Add puppetdb to a new puppetdb host
This brings in the puppetdb module via and
creates a new class, openstack_project::puppetdb. It was modeled
on openstack_project::puppetmaster. Note that this will not enable
puppetdb to participate in the Puppet ecosystem yet. An additional
class will have to be added to the master:

  class { 'puppetdb::master::config':
    puppetdb_server => 'puppetdb',

I will leave this out of this change so we can bring up puppetdb in

This paves the way for puppetboard to be brought online.

Change-Id: I8194372bd31e08f12a815fd04dcdf338565ed911
2014-02-15 16:14:33 -08:00
Dan Prince 0f445410ce Drop the saz-gearman module (we don't use it)
This module confused me in that I though we were actually
using gearmand somewhere. Infra has its own version of
gearman fork in python... so no need to keep this around anymore.

This change adds a new remove_module function to
and updates the script to use that function to remove gearman
and the associated dependency modules.

Change-Id: Ifc3808407eba9853b6991ef9744adb96453d46a8
2013-11-21 09:21:53 -05:00
Clark Boylan f2bfd4b0e0 Upgrade puppetlabs-postgresql to fix concat module
* Upgrade puppetlabs-postgresql to version 3.0.0 to
fix conflicts in the concat module. Old postgresql uses ripienaar's
module other modules want puppetlabs' concat module. This upgrade resolves
that conflict.

Change-Id: I3813abea48d933421ebb4f81e84e783fa51618db
2013-10-18 15:07:52 -07:00
Clark Boylan 19d6de12ad Upgrade puppetlabs-haproxy to fix concat module. Upgrade puppetlabs-haproxy to version 0.4.1 to fix
conflicts in the concat module. Old haproxy uses ripienaar's module
other modules want puppetlabs' concat module. This upgrade resolves
that conflict.

Change-Id: Ie036091088a6c325264f8a0b4400dcf601827d50
2013-10-18 14:10:24 -07:00