This helps when reviewing the log file within Jenkins.
Change-Id: I9855fcc628cf8764cbe53b01247613359e731cd4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This duplicates the behavior of puppet module install for the git
clone operations.
Puppet module install will use the first element of the modulepath.
This makes the git clone operations go into the same directory as
the modules installed via the puppet module tool.
This helps us downstream because we are moving the directory that
holds these modules out of /etc/ and into /opt/.
Change-Id: I0c7833a6d9a79fa613efc9daebee916f3192306a
This causes apply-test.sh to get the list of puppet modules to do
integration test with from modules.env. Modules.env has been
refactored to have three lists of modules: Package modules(MODULES),
SOURCE_MODULES, and INTEGRATION_MODULES.
When PUPPET_INTEGRATION_TEST is not set, INTEGRATION_MODULES is
folded into SOURCE_MODULES.
In apply-test.sh, INTEGRATION_MODULES is sourced from modules.env
and some string/array manipulation is performed to pass each entry
to zuul-cloner.
Change-Id: I47302c5c7e8c41b985f16a05c4e9b8078ea867a3
This ends up putting a bunch of ansi escapes in the apply test
output. We could conditionally inject it form the apply test
if people want color when this is run normally.
Change-Id: I004bcd29671344430d271aad7f54186bc725215c
Use case: Our team would like to use install_modules.sh
in other places beyond openstack-infra/config, we would
also like to provide some control over what install_modules.sh
installs beyond what is given as defaults.
* Added a copyright notice so we can use it in other projects
* if a modules.env file is found in the same directory
as install_modules.sh then source the modules.env
* an export called MODULE_ENV_FILE can be included to
augment the name of the file from modules.env
to an alternative file name like modules.test.env.
This gives users of the script an option to provide
different modules on different test environments.
* as export to change the path from the default
directory to an alternate directory can be
specified with the export MODULE_ENV_PATH var.
The default directory is the same location as
the install_modules.sh script.
Change-Id: I0c3f9bcb3e91236ceca492426ee6105242a2c8eb
Fix all of the bashate failures. Also, bashate will install its negative
tests in its virtualenv, so we need to generate a list of files to pass to
bashate that excludes .tox/
Change-Id: I55559bf6137f705aec9a7f277928ad8d4cadc2ca
Make module installation conditional in install_modules.sh so that
we can specify that we _do not_ want it to install the modules
that will be participating in the integration test.
Add a variant of the apply test script (currently test.sh) that
uses zuul-cloner to check out the appropriate versions of modules
that are participating in the integration test.
Add jobs that run this to the experimental pipeline.
Change-Id: I16055cdd45f6e05aa8ebceb500c5709f265268df
Without this patch, install_modules.sh will attempt to install
puppetlabs-stdlib 3.2.0 and fail because puppetlabs-concat installs
puppetlabs-stdlib > 4. This is a problem because it causes inconsistency
between the version listed in install_modules.sh and the actual installed
version, and it causes errors during subsequent runs of the shell script. This
patch resolves the problem by updating the version listed in
install_modules.sh to the latest version.
Closes-Bug: 1324645
Change-Id: I7822597785946330346c476753ff60ae1b345014
When using git sourced puppet modules we need to specify the desired ref
as a tag, sha1, or remote branch because install_modules.sh does not
reset local branches to match remote branches.
Do a git remote update instaed of a git fetch to make sure that all
remotes are updated.
Change-Id: I1785af6fd02e6cda8f5f62b54bdb83efb47b7197
This uses the source install modules functionality to grab the storyboard
module. It pins to the commit hash right now so we will need to update that
periodically.
This removes the storyboard module directory entirely since we no longer
need it.
Change-Id: Iceab5501fa0c177f55db1a6b723b6b215fca9ed0
The key benefit here is that trusty can autodetect postgres version
without manually setting things.
Change-Id: I066c39e1437dfadf06047a714c716adea30bdb5d
In order to get the puppet module for storyboard up to a level where
we can publish it to puppetforge, I did some work on it to create
separate modules which can be used by anyone to install storyboard.
- API and Webclient are now installed via storyboard::application,
which assumes that you can provide the DB connection criteria.
- storyboard::cert is now a separate class, which accepts either
files or strings, which generates the SSL certificate and chain
files for storyboard.
- storyboard::params is our dependency checker.
- storyboard::init will install a standalone, entirely
self-contained instance of storyboard.
- Added various puppet module files necessary for eventual
deployment to puppetforge.
- Added README.md documentation for later puppetforge addition.
This patch also includes a new module: example42-puppi, which is a
series of convenience utilities useful for deployment. For example,
puppi::netinstall (used here) will fetch tarballs and zip files and
extract them into a provided directory. It also contains changes to
the storyboard configuration for the new refresh token support patch
in #94363
Change-Id: I6ab8c24b308df38774fc0694d218dcb5022cd899
* install_modules.sh: When invoking git rev-list with --max-count it
only supports = and not space before the value. The previous
implementation was flawed and only ever resulted in git errors about
an unrecognized ref for which it was misinterpreting the --max-count
value as representing.
Change-Id: I11f8f5b6e13d4640d6558f423292ad7e7b89961f
At this version the apt module also includes a
apt::unattended_upgrades class that we might be able to use
to deprecate our home-spun unattended_upgrades module.
Change-Id: Idd6f3a935cf4d543ee10d909900c8e01947e8ced
The current check will work since an known tag is already at the right
ref and an unknown tag will give an unequal value from git rev-list. But
this fails to work if the version is, for example, 'origin/master'
rather than a git tag.
Change-Id: I746624649acaa7fab217b8ff5570f5822003b2f3
This reverts commit 3afc75132aa3a7d3cae911ae5e7166383b9ee4ba.
The new apache module has started managing /etc/httpd/conf/httpd.conf
with a template that has some significant differences than our template
in the cgit module.
Change-Id: I99795d35596f35dfc34e89891155dd2b83e465fe
This reverts commit 7b9ea298cf4ae941af74381925b17a4b29337eb9.
Upgrading puppetlabs-apache to version 0.4.0 will give us important
variables in apache::params (specifically $conf_dir and $vdir) which
will allow us to modify the cgit module to run on both Debuntu and RHEL
systems -- which is useful because test.sh should be testing all modules
on both systems.
Of the two issues that 7b9ea298 addresses, I believe that our Oneiric
hosts have all gone the way of the dinosaur and we can work around the
docroot check by replacing 'MEANINGLESS ARGUMENT' with
'/tmp/meaningless_docroot'.
Conflicts:
install_modules.sh
Change-Id: I7c08f85db6810ab28fa044f1923833359271e8ec
This patch adds logic that can be used to clone
modules from remote git repositories and set the
resulting local repos to a know revision.
This logic has been added to make it easier to
decouple modules that are currently included as
a part of config into their own repos. Allowing
source installation will allow for edits of these
modules to be consumed more easily (without requiring
an additional release to the forge).
Additionally, this patch removes the puppet forge
installation logic that was installing without deps.
If modules are needed without their dependencies, they
can be installed from source to limit the amounts of ways
that modules can be installed.
Change-Id: I6e25cd65569f82bd5b0ef7d00b32e93ecdda3d71
Puppet board has replaced puppet dashboard. Remove reference to puppet
dashboard as it shouldn't be used.
Change-Id: I5eeee2984729ef5d1b883b4762347d19786e28ed
Use inherits keyword to bring puppetboard::params into scope
Use 2>&1 in install_modules to silence warnings
Set puppetdb class to open port 80 when puppetboard is used
Remove inherits from params check from puppet lint
Change-Id: I75fc35542e9f2641bcab613d7e1aef83d9300217
This places the puppet web dashboard puppetboard from
https://github.com/nedap/puppetboard onto the puppetdb server.
This enables anyone to see the output of puppet runs, which nodes
are failing and which nodes have not checked in over time. We can
also quickly and interactively query the facts database.
This also modifies install_modules.sh to support a list of modules
that will be installed without dependency resolution.
Change-Id: I09d09410dd8fa71759ce6b2f9c7f34f78eefe204
This brings in the puppetdb module via install_modules.sh and
creates a new class, openstack_project::puppetdb. It was modeled
on openstack_project::puppetmaster. Note that this will not enable
puppetdb to participate in the Puppet ecosystem yet. An additional
class will have to be added to the master:
class { 'puppetdb::master::config':
puppetdb_server => 'puppetdb',
}
I will leave this out of this change so we can bring up puppetdb in
stages.
This paves the way for puppetboard to be brought online.
Change-Id: I8194372bd31e08f12a815fd04dcdf338565ed911
This module confused me in that I though we were actually
using gearmand somewhere. Infra has its own version of
gearman fork in python... so no need to keep this around anymore.
This change adds a new remove_module function to install_modules.sh
and updates the script to use that function to remove gearman
and the associated dependency modules.
Change-Id: Ifc3808407eba9853b6991ef9744adb96453d46a8
* install_modules.sh: Upgrade puppetlabs-postgresql to version 3.0.0 to
fix conflicts in the concat module. Old postgresql uses ripienaar's
module other modules want puppetlabs' concat module. This upgrade resolves
that conflict.
Change-Id: I3813abea48d933421ebb4f81e84e783fa51618db
install_modules.sh: Upgrade puppetlabs-haproxy to version 0.4.1 to fix
conflicts in the concat module. Old haproxy uses ripienaar's module
other modules want puppetlabs' concat module. This upgrade resolves
that conflict.
Change-Id: Ie036091088a6c325264f8a0b4400dcf601827d50
Puppetlabs-mysql 0.6.1 correctly removes the local ::1 root user in the
account_security manifest. Upgrade to this version to take advantage of
that. Do not upgrade to latest version (0.9.0) to minimize delta that
needs testing.
Change-Id: Ic8265733f1159f34ae0afcccdea4c7d8cd44e3cb
Add a groups-dev node to openstack infra, based on drupal
environment. This node checks out the repository:
https://git.openstack.org/openstack-infra/groups
Build a new deployment from scratch, including drush make
distbuild and drush si auto-installation features.
Change-Id: I71eef1c14724ceb13a896ff768923148dedcc2ef
* install_modules.sh: Add puppetlabs-haproxy forge module.
* modules/cgit/manifests/init.pp: Add haproxy config to load balance
https, https and git protocol git access. Each git server will host git
http on port 8080, https on port 4443 and git protocol on 29418. These
endpoints will then be load balanced by a single haproxy instance
listening on ports 80, 443 and 9418. The use of haproxy and having
services listen on offset ports to accomodate haproxy is toggleable
using the $balance_git and $behind_proxy boolean flags.
Additionally, configure rsyslog for haproxy.
* modules/cgit/files/rsyslog.haproxy.conf: Enable syslog over UDP on
port 514. This is needed by haproxy to perform logging. Send local0
messages to /var/log/haproxy.log.
* modules/cgit/templates/ssl.conf.erb: Make Apache https listen port
configurable. Remove default virtualhost.
* modules/cgit/templates/httpd.conf.erb: Make Apache http listen port
configurable.
* modules/cgit/templates/git.vhost..erb: Make Apache http(s) listen
ports configuruable. Allow http without redirecting to https as a
fallback option to accomodate CentOS clients.
* modules/openstack_project/manifests/git.pp: Pass load balancer
variables through to the cgit manifest.
* manifests/site.pp: Configure git.o.o to run the load balancer haproxy
and balance across the new gitXX.o.o nodes.
Change-Id: Icefc5923cff9a7c6ce62c1923ec2ea87ebc6474a
The 0.5.0 selinux module we initially installed had a dangling
symlink bug. Upstream has fixed this in 0.5.1.
Change-Id: Ia286eb39de3ded7d407f2875fcce79f834a82aca
Reviewed-on: https://review.openstack.org/36569
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
The selinux puppet module makes it easy to change status of selinux
activation (enforcing/permissive/disabled) for CentOS/Red Hat.
Change-Id: I6e57ebbc6d78051021a2f4062edab09b92c6e03d
Reviewed-on: https://review.openstack.org/35542
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Khai Do <zaro0508@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
I got errors when I tried to use 2.0.1 on Fedora 18. Upgrading
seems to have fixed the issues.
Change-Id: I90fbf45a90e42481f48643e7fca91ba105470a7a
Reviewed-on: https://review.openstack.org/32450
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
* manifests/site.pp: Pass both jenkins.openstack.org and
jenkins-dev.openstack.org to openstack_project::zuul as the list of
gearman workers for use in iptables rules.
* modules/openstack_project/manifests/jenkins.pp
* modules/openstack_project/manifests/jenkins_dev.pp: Remove unused
4155/tcp from public allowed ports list, previously for a bzr
service which is no longer running on these servers.
* modules/openstack_project/manifests/zuul.pp: Add iptables rules
allowing access from gearman workers to the gearmand, and also
configure gearmand to listen on all addresses including IPv6, as
opposed to its IPv4-only default.
* install_modules.sh: Add puppetlabs-stdlib version 3.2.0 to the list of
puppet dependencies. This is required to pull in the "flatten" function
used in zuul.pp above.
Change-Id: I05d4abc92b3050884a7e9f3c2b6cd9dcc81bfa36
Reviewed-on: https://review.openstack.org/26245
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Change-Id: I574bdbf3dddd5360368aa3d08ff6b25bc0beac1f
Reviewed-on: https://review.openstack.org/23463
Reviewed-by: Anita Kuno <akuno@lavabit.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
Updates our install_modules.sh file so that it uses
puppetlabs-ntp instead of kickstandproject-ntp. This provides
better multi distro NTP support.
Change-Id: I7b11e8c67a2e38653be281b5e6aa1d08768cc617
Reviewed-on: https://review.openstack.org/23204
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
Install gearman module in preparation for building out gearman servers
Change-Id: Ic2f5ec7e4506784dbb8a92068961c4e8a2fc775d
Reviewed-on: https://review.openstack.org/21147
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Tested-by: Jenkins
* install_modules.sh: The puppetlabs-postgresql 2.0.1 module wants
puppetlabs-apt 1.1.0, so this should stop the schizophrenic cronspam
from repeated failed attempts to install the old version.
Change-Id: Idad5951bb7b5c941a5239a4f20b80a90ee65ac1a
Reviewed-on: https://review.openstack.org/20257
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
Add postgres servers to each non bare Jenkins slave so that tests can be
performed against postgres.
Change-Id: I374afdf2443086e2c5ad42d2e163adbf2d3df74b
Reviewed-on: https://review.openstack.org/20041
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: Kurt Taylor <krtaylor@us.ibm.com>
Reviewed-by: Sean Dague <sdague@linux.vnet.ibm.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
We no longer use/need this.
Change-Id: I3474ccf8615d5392f4df514b3171e71551927fe3
Reviewed-on: https://review.openstack.org/18908
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins