76 Commits

Author SHA1 Message Date
Paul Belanger
aa35aaabdb Remove color output from puppet list command
This helps when reviewing the log file within Jenkins.

Change-Id: I9855fcc628cf8764cbe53b01247613359e731cd4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-07-15 09:40:01 -04:00
Spencer Krum
4cc08edcd3 Set modulepath to the first element of puppet's modulepath.
This duplicates the behavior of puppet module install for the git
clone operations.

Puppet module install will use the first element of the modulepath.
This makes the git clone operations go into the same directory as
the modules installed via the puppet module tool.

This helps us downstream because we are moving the directory that
holds these modules out of /etc/ and into /opt/.

Change-Id: I0c7833a6d9a79fa613efc9daebee916f3192306a
2015-01-22 02:41:20 -08:00
Jenkins
c49fb5a0e3 Merge "Turn off color in module install" 2014-12-12 16:59:02 +00:00
Spencer Krum
ed46e9303f Source modules.env from apply-test.sh
This causes apply-test.sh to get the list of puppet modules to do
integration test with from modules.env. Modules.env has been
refactored to have three lists of modules: Package modules(MODULES),
SOURCE_MODULES, and INTEGRATION_MODULES.

When PUPPET_INTEGRATION_TEST is not set, INTEGRATION_MODULES is
folded into SOURCE_MODULES.

In apply-test.sh, INTEGRATION_MODULES is sourced from modules.env
and some string/array manipulation is performed to pass each entry
to zuul-cloner.

Change-Id: I47302c5c7e8c41b985f16a05c4e9b8078ea867a3
2014-11-18 21:23:43 +00:00
Spencer Krum
3d4838ac33 Turn off color in module install
This ends up putting a bunch of ansi escapes in the apply test
output. We could conditionally inject it form the apply test
if people want color when this is run normally.

Change-Id: I004bcd29671344430d271aad7f54186bc725215c
2014-11-17 07:54:17 +00:00
Edward Raigosa (wenlock)
7299bf0326 give install_modules options for loading requirements externally
Use case:  Our team would like to use install_modules.sh
  in other places beyond openstack-infra/config, we would
  also like to provide some control over what install_modules.sh
  installs beyond what is given as defaults.

  * Added a copyright notice so we can use it in other projects
  * if a modules.env file is found in the same directory
    as install_modules.sh then source the modules.env
  * an export called MODULE_ENV_FILE can be included to
    augment the name of the file from modules.env
    to an alternative file name like modules.test.env.
    This gives users of the script an option to provide
    different modules on different test environments.
  * as export to change the path from the default
    directory to an alternate directory can be
    specified with the export MODULE_ENV_PATH var.
    The default directory is the same location as
    the install_modules.sh script.

Change-Id: I0c3f9bcb3e91236ceca492426ee6105242a2c8eb
2014-10-20 22:00:48 +00:00
K Jonathan Harker
948aff1f02 Clean up bashate failures
Fix all of the bashate failures.  Also, bashate will install its negative
tests in its virtualenv, so we need to generate a list of files to pass to
bashate that excludes .tox/

Change-Id: I55559bf6137f705aec9a7f277928ad8d4cadc2ca
2014-09-30 12:40:59 -07:00
Jenkins
764934ef74 Merge "Update for new home for puppetboard module" 2014-09-16 23:19:13 +00:00
Ian Wienand
1bdbee9943 Upgrade to puppetlabs-postgresql 4.0.0
This version has a range of fixes required to make the module work on
Fedora and CentOS 7

Change-Id: Ie867cfcc13a81a48e0c83cf70325032ed44bddef
2014-09-16 08:03:35 +10:00
Spencer Krum
179750c5f5 Update for new home for puppetboard module
Change-Id: If5047aae6bfe2901faed1a2177a8b11de325907d
2014-09-11 15:27:39 -07:00
James E. Blair
9d23185ac5 Add integration apply test
Make module installation conditional in install_modules.sh so that
we can specify that we _do not_ want it to install the modules
that will be participating in the integration test.

Add a variant of the apply test script (currently test.sh) that
uses zuul-cloner to check out the appropriate versions of modules
that are participating in the integration test.

Add jobs that run this to the experimental pipeline.

Change-Id: I16055cdd45f6e05aa8ebceb500c5709f265268df
2014-09-03 15:03:09 -07:00
Jenkins
c7e5b0d19c Merge "Update git puppet modules properly." 2014-09-02 19:19:01 +00:00
Jenkins
15910f883f Merge "Bump the apt module version from 1.1 to 1.4" 2014-08-29 19:49:14 +00:00
Colleen Murphy
fbe1dbe611 Update version of puppetlabs-stdlib in install_modules.sh
Without this patch, install_modules.sh will attempt to install
puppetlabs-stdlib 3.2.0 and fail because puppetlabs-concat installs
puppetlabs-stdlib > 4. This is a problem because it causes inconsistency
between the version listed in install_modules.sh and the actual installed
version, and it causes errors during subsequent runs of the shell script. This
patch resolves the problem by updating the version listed in
install_modules.sh to the latest version.

Closes-Bug: 1324645
Change-Id: I7822597785946330346c476753ff60ae1b345014
2014-08-28 18:40:44 -05:00
Clark Boylan
e98f519fb9 Update git puppet modules properly.
When using git sourced puppet modules we need to specify the desired ref
as a tag, sha1, or remote branch because install_modules.sh does not
reset local branches to match remote branches.

Do a git remote update instaed of a git fetch to make sure that all
remotes are updated.

Change-Id: I1785af6fd02e6cda8f5f62b54bdb83efb47b7197
2014-08-28 15:39:36 -07:00
Jenkins
26cf0f7d28 Merge "Use puppet-storyboard@master" 2014-08-20 20:32:18 +00:00
Jenkins
4a44f9931b Merge "Bump postgres module version" 2014-08-19 19:42:47 +00:00
Jenkins
e6e2f774ab Merge "Git rev-list --max-count needs an = for assignment" 2014-08-15 20:51:02 +00:00
Michael Krotscheck
2ab45223e3 Use puppet-storyboard@master
This change makes sure that the version of puppet-storyboard in use
is always the most recent one. It also removes example42::puppi, as
as of https://review.openstack.org/#/c/114383/ it is no longer
a requirement.

Depends on https://review.openstack.org/#/c/114383/

Change-Id: I7024476a8eea9c1bb61f082782fc1149eda37a6e
2014-08-15 09:42:05 -07:00
Michael Krotscheck
9223b79612 Update storyboard version to include rabbitmq
This updates the version of storyboard that is used by infra to include
rabbitmq.

Change-Id: I8cbfb17715a75cf63dd60ad4bae24bc40b3a6944
2014-08-13 14:48:30 -07:00
Spencer Krum
74bd3d233e Use split-out storyboard module
This uses the source install modules functionality to grab the storyboard
module. It pins to the commit hash right now so we will need to update that
periodically.

This removes the storyboard module directory entirely since we no longer
need it.

Change-Id: Iceab5501fa0c177f55db1a6b723b6b215fca9ed0
2014-08-13 00:34:50 +00:00
Spencer Krum
9dc552a8a6 Bump postgres module version
The key benefit here is that trusty can autodetect postgres version
without manually setting things.

Change-Id: I066c39e1437dfadf06047a714c716adea30bdb5d
2014-08-12 08:36:56 -07:00
Michael Krotscheck
682019f1fa Modularized StoryBoard Module
In order to get the puppet module for storyboard up to a level where
we can publish it to puppetforge, I did some work on it to create
separate modules which can be used by anyone to install storyboard.

- API and Webclient are now installed via storyboard::application,
  which assumes that you can provide the DB connection criteria.
- storyboard::cert is now a separate class, which accepts either
  files or strings, which generates the SSL certificate and chain
  files for storyboard.
- storyboard::params is our dependency checker.
- storyboard::init will install a standalone, entirely
  self-contained instance of storyboard.
- Added various puppet module files necessary for eventual
  deployment to puppetforge.
- Added README.md documentation for later puppetforge addition.

This patch also includes a new module: example42-puppi, which is a
series of convenience utilities useful for deployment. For example,
puppi::netinstall (used here) will fetch tarballs and zip files and
extract them into a provided directory. It also contains changes to
the storyboard configuration for the new refresh token support patch
in #94363

Change-Id: I6ab8c24b308df38774fc0694d218dcb5022cd899
2014-08-08 10:58:17 -07:00
Jeremy Stanley
3f1491d1e4 Git rev-list --max-count needs an = for assignment
* install_modules.sh: When invoking git rev-list with --max-count it
only supports = and not space before the value. The previous
implementation was flawed and only ever resulted in git errors about
an unrecognized ref for which it was misinterpreting the --max-count
value as representing.

Change-Id: I11f8f5b6e13d4640d6558f423292ad7e7b89961f
2014-08-04 16:18:53 -07:00
Spencer Krum
671b282f2a Bump the apt module version from 1.1 to 1.4
At this version the apt module also includes a
apt::unattended_upgrades class that we might be able to use
to deprecate our home-spun unattended_upgrades module.

Change-Id: Idd6f3a935cf4d543ee10d909900c8e01947e8ced
2014-07-10 13:29:10 -07:00
K Jonathan Harker
a0b5912853 Update git refs before comparing refs
The current check will work since an known tag is already at the right
ref and an unknown tag will give an unequal value from git rev-list. But
this fails to work if the version is, for example, 'origin/master'
rather than a git tag.

Change-Id: I746624649acaa7fab217b8ff5570f5822003b2f3
2014-07-09 14:30:16 -07:00
K Jonathan Harker
7af40db7f2 Revert "Revert "Downgrade puppetlabs-apache to version 0.0.4.""
This reverts commit 3afc75132aa3a7d3cae911ae5e7166383b9ee4ba.

The new apache module has started managing /etc/httpd/conf/httpd.conf
with a template that has some significant differences than our template
in the cgit module.

Change-Id: I99795d35596f35dfc34e89891155dd2b83e465fe
2014-07-01 17:38:58 -07:00
K Jonathan Harker
3afc75132a Revert "Downgrade puppetlabs-apache to version 0.0.4."
This reverts commit 7b9ea298cf4ae941af74381925b17a4b29337eb9.

Upgrading puppetlabs-apache to version 0.4.0 will give us important
variables in apache::params (specifically $conf_dir and $vdir) which
will allow us to modify the cgit module to run on both Debuntu and RHEL
systems -- which is useful because test.sh should be testing all modules
on both systems.

Of the two issues that 7b9ea298 addresses, I believe that our Oneiric
hosts have all gone the way of the dinosaur and we can work around the
docroot check by replacing 'MEANINGLESS ARGUMENT' with
'/tmp/meaningless_docroot'.

Conflicts:
	install_modules.sh

Change-Id: I7c08f85db6810ab28fa044f1923833359271e8ec
2014-06-27 15:29:03 -07:00
Dan
ba3eb9f791 Allow modules to be installed from source
This patch adds logic that can be used to clone
modules from remote git repositories and set the
resulting local repos to a know revision.

This logic has been added to make it easier to
decouple modules that are currently included as
a part of config into their own repos. Allowing
source installation will allow for edits of these
modules to be consumed more easily (without requiring
an additional release to the forge).

Additionally, this patch removes the puppet forge
installation logic that was installing without deps.
If modules are needed without their dependencies, they
can be installed from source to limit the amounts of ways
that modules can be installed.

Change-Id: I6e25cd65569f82bd5b0ef7d00b32e93ecdda3d71
2014-06-10 17:07:04 -07:00
Clark Boylan
30c2f7d462 Remove puppet dashboard.
Puppet board has replaced puppet dashboard. Remove reference to puppet
dashboard as it shouldn't be used.

Change-Id: I5eeee2984729ef5d1b883b4762347d19786e28ed
2014-04-17 14:48:19 -07:00
Spencer Krum
101e58991c Further changes to bring puppetboard online
Use inherits keyword to bring puppetboard::params into scope
Use 2>&1 in install_modules to silence warnings
Set puppetdb class to open port 80 when puppetboard is used
Remove inherits from params check from puppet lint

Change-Id: I75fc35542e9f2641bcab613d7e1aef83d9300217
2014-03-22 12:54:38 -07:00
Spencer Krum
5740ed3118 Add PuppetBoard to openstack infra
This places the puppet web dashboard puppetboard from
https://github.com/nedap/puppetboard onto the puppetdb server.

This enables anyone to see the output of puppet runs, which nodes
are failing and which nodes have not checked in over time. We can
also quickly and interactively query the facts database.

This also modifies install_modules.sh to support a list of modules
that will be installed without dependency resolution.

Change-Id: I09d09410dd8fa71759ce6b2f9c7f34f78eefe204
2014-03-20 00:50:54 -07:00
Spencer Krum
04b114ba82 Add puppetdb to a new puppetdb host
This brings in the puppetdb module via install_modules.sh and
creates a new class, openstack_project::puppetdb. It was modeled
on openstack_project::puppetmaster. Note that this will not enable
puppetdb to participate in the Puppet ecosystem yet. An additional
class will have to be added to the master:

  class { 'puppetdb::master::config':
    puppetdb_server => 'puppetdb',
  }

I will leave this out of this change so we can bring up puppetdb in
stages.

This paves the way for puppetboard to be brought online.

Change-Id: I8194372bd31e08f12a815fd04dcdf338565ed911
2014-02-15 16:14:33 -08:00
Dan Prince
0f445410ce Drop the saz-gearman module (we don't use it)
This module confused me in that I though we were actually
using gearmand somewhere. Infra has its own version of
gearman fork in python... so no need to keep this around anymore.

This change adds a new remove_module function to install_modules.sh
and updates the script to use that function to remove gearman
and the associated dependency modules.

Change-Id: Ifc3808407eba9853b6991ef9744adb96453d46a8
2013-11-21 09:21:53 -05:00
Clark Boylan
f2bfd4b0e0 Upgrade puppetlabs-postgresql to fix concat module
* install_modules.sh: Upgrade puppetlabs-postgresql to version 3.0.0 to
fix conflicts in the concat module. Old postgresql uses ripienaar's
module other modules want puppetlabs' concat module. This upgrade resolves
that conflict.

Change-Id: I3813abea48d933421ebb4f81e84e783fa51618db
2013-10-18 15:07:52 -07:00
Clark Boylan
19d6de12ad Upgrade puppetlabs-haproxy to fix concat module.
install_modules.sh: Upgrade puppetlabs-haproxy to version 0.4.1 to fix
conflicts in the concat module. Old haproxy uses ripienaar's module
other modules want puppetlabs' concat module. This upgrade resolves
that conflict.

Change-Id: Ie036091088a6c325264f8a0b4400dcf601827d50
2013-10-18 14:10:24 -07:00
Jenkins
c164abc879 Merge "Upgrade puppetlabs-mysql to 0.6.1." 2013-10-12 20:57:34 +00:00
Clark Boylan
9a30d3a309 Upgrade puppetlabs-mysql to 0.6.1.
Puppetlabs-mysql 0.6.1 correctly removes the local ::1 root user in the
account_security manifest. Upgrade to this version to take advantage of
that. Do not upgrade to latest version (0.9.0) to minimize delta that
needs testing.

Change-Id: Ic8265733f1159f34ae0afcccdea4c7d8cd44e3cb
2013-10-12 17:37:24 +00:00
Marton Kiss
648a8af52f Add groups-dev.openstack.org node
Add a groups-dev node to openstack infra, based on drupal
environment. This node checks out the repository:
https://git.openstack.org/openstack-infra/groups

Build a new deployment from scratch, including drush make
distbuild and drush si auto-installation features.

Change-Id: I71eef1c14724ceb13a896ff768923148dedcc2ef
2013-09-17 11:43:00 +02:00
Clark Boylan
dea75ed695 Load balance git requests.
* install_modules.sh: Add puppetlabs-haproxy forge module.

* modules/cgit/manifests/init.pp: Add haproxy config to load balance
https, https and git protocol git access. Each git server will host git
http on port 8080, https on port 4443 and git protocol on 29418. These
endpoints will then be load balanced by a single haproxy instance
listening on ports 80, 443 and 9418. The use of haproxy and having
services listen on offset ports to accomodate haproxy is toggleable
using the $balance_git and $behind_proxy boolean flags.
Additionally, configure rsyslog for haproxy.

* modules/cgit/files/rsyslog.haproxy.conf: Enable syslog over UDP on
port 514. This is needed by haproxy to perform logging. Send local0
messages to /var/log/haproxy.log.

* modules/cgit/templates/ssl.conf.erb: Make Apache https listen port
configurable. Remove default virtualhost.

* modules/cgit/templates/httpd.conf.erb: Make Apache http listen port
configurable.

* modules/cgit/templates/git.vhost..erb: Make Apache http(s) listen
ports configuruable. Allow http without redirecting to https as a
fallback option to accomodate CentOS clients.

* modules/openstack_project/manifests/git.pp: Pass load balancer
variables through to the cgit manifest.

* manifests/site.pp: Configure git.o.o to run the load balancer haproxy
and balance across the new gitXX.o.o nodes.

Change-Id: Icefc5923cff9a7c6ce62c1923ec2ea87ebc6474a
2013-08-22 10:52:04 -07:00
Elizabeth Krumbach
fe60702996 Bump version of selinux module
The 0.5.0 selinux module we initially installed had a dangling
symlink bug. Upstream has fixed this in 0.5.1.

Change-Id: Ia286eb39de3ded7d407f2875fcce79f834a82aca
Reviewed-on: https://review.openstack.org/36569
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-07-12 16:53:00 +00:00
Elizabeth Krumbach
25120b12de Add selinux puppet module
The selinux puppet module makes it easy to change status of selinux
activation (enforcing/permissive/disabled) for CentOS/Red Hat.

Change-Id: I6e57ebbc6d78051021a2f4062edab09b92c6e03d
Reviewed-on: https://review.openstack.org/35542
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-07-09 00:03:28 +00:00
Dan Prince
7ed9216630 Bump puppetlabs/postgres module to 2.3.0.
I got errors when I tried to use 2.0.1 on Fedora 18. Upgrading
seems to have fixed the issues.

Change-Id: I90fbf45a90e42481f48643e7fca91ba105470a7a
Reviewed-on: https://review.openstack.org/32450
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-06-24 22:02:38 +00:00
Jeremy Stanley
b9b510ba6a Connectivity from workers to gearmand on zuul.
* manifests/site.pp: Pass both jenkins.openstack.org and
jenkins-dev.openstack.org to openstack_project::zuul as the list of
gearman workers for use in iptables rules.

* modules/openstack_project/manifests/jenkins.pp
* modules/openstack_project/manifests/jenkins_dev.pp: Remove unused
4155/tcp from public allowed ports list, previously for a bzr
service which is no longer running on these servers.

* modules/openstack_project/manifests/zuul.pp: Add iptables rules
allowing access from gearman workers to the gearmand, and also
configure gearmand to listen on all addresses including IPv6, as
opposed to its IPv4-only default.

* install_modules.sh: Add puppetlabs-stdlib version 3.2.0 to the list of
puppet dependencies. This is required to pull in the "flatten" function
used in zuul.pp above.

Change-Id: I05d4abc92b3050884a7e9f3c2b6cd9dcc81bfa36
Reviewed-on: https://review.openstack.org/26245
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2013-04-05 20:11:04 +00:00
Jeremy Stanley
bd8ef6486b Upgrade to openstackci-dashboard 0.0.8.
Change-Id: I574bdbf3dddd5360368aa3d08ff6b25bc0beac1f
Reviewed-on: https://review.openstack.org/23463
Reviewed-by: Anita Kuno <akuno@lavabit.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2013-03-05 00:48:25 +00:00
Dan Prince
e52c8739ce Switch to puppetlabs-ntp.
Updates our install_modules.sh file so that it uses
puppetlabs-ntp instead of kickstandproject-ntp. This provides
better multi distro NTP support.

Change-Id: I7b11e8c67a2e38653be281b5e6aa1d08768cc617
Reviewed-on: https://review.openstack.org/23204
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-03-04 17:59:56 +00:00
Khai Do
2e98ade518 install gearman module from puppet forge
Install gearman module in preparation for building out gearman servers

Change-Id: Ic2f5ec7e4506784dbb8a92068961c4e8a2fc775d
Reviewed-on: https://review.openstack.org/21147
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
2013-02-21 23:24:00 +00:00
Jeremy Stanley
163a54731e Increase puppetlabs-apt to 1.1.0.
* install_modules.sh: The puppetlabs-postgresql 2.0.1 module wants
puppetlabs-apt 1.1.0, so this should stop the schizophrenic cronspam
from repeated failed attempts to install the old version.

Change-Id: Idad5951bb7b5c941a5239a4f20b80a90ee65ac1a
Reviewed-on: https://review.openstack.org/20257
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-01-22 20:02:13 +00:00
Clark Boylan
ff5a1401ef Add postgres server to Jenkins slaves.
Add postgres servers to each non bare Jenkins slave so that tests can be
performed against postgres.

Change-Id: I374afdf2443086e2c5ad42d2e163adbf2d3df74b
Reviewed-on: https://review.openstack.org/20041
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Kurt Taylor <krtaylor@us.ibm.com>
Reviewed-by: Sean Dague <sdague@linux.vnet.ibm.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-01-21 23:36:20 +00:00
Dan Prince
fb4af5242a Drop clone_git function from install_modules.sh.
We no longer use/need this.

Change-Id: I3474ccf8615d5392f4df514b3171e71551927fe3
Reviewed-on: https://review.openstack.org/18908
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-01-17 18:25:49 +00:00