Commit Graph

544 Commits

Author SHA1 Message Date
James E. Blair
093e9c525d Remove update_slave from puppetmaster
This was added for the puppet3 transition but is no longer needed
with the way we are using ansible.

Change-Id: Iac45771b64434fb36322dd91cc73e13e859724db
2014-09-11 14:12:36 -07:00
Spencer Krum
b16ea8906e puppetmaster is a CA
Change-Id: Ifa448e845259e7de8f13bfef420c3ea481cbda24
2014-09-10 12:48:33 -07:00
Spencer Krum
9dcbfe6fbb Updating node defn for puppet 3 master
This updates puppet version to 3.6

Change-Id: I37879153c68011c752919675a2d3416383aaf8e5
2014-09-10 09:49:01 -07:00
Monty Taylor
23dff8a2c6 Add entry for new PyPI mirrors
We want to actually run PyPI mirrors on a per-region basis. To do that,
we need to configure machines to run such things.

Change-Id: I96955d78783c0db31ace52146834f4e6b8e1fde4
2014-09-05 18:05:29 +00:00
Jenkins
a3af99c14e Merge "Add #openstack-stable IRC channel" 2014-09-03 23:13:49 +00:00
James E. Blair
baa13e435e Use zm04
Added to the gearman firewall and cacti.  Merge this after the
host exists.

Change-Id: Ic2220cbe4985031a1b86c26763e64e52b427cbda
2014-08-29 16:16:32 -07:00
James E. Blair
3c405400ea Add zm04
Just add the host def.  Adding it to firewalls, etc, in a
subsequent change.

Change-Id: I0e48f109f30c8ed7f5fa0c2c3bcca2731497377d
2014-08-29 16:15:41 -07:00
Jeremy Stanley
ce3bf5056a Add #openstack-stable IRC channel
This adds the #openstack-stable IRC channel to typical bots.

Change-Id: Iae98e69ad62086baa02f0cac191e3ad7694ace80
Related-Bug: #1360324
2014-08-24 21:47:09 +00:00
James E. Blair
1e2c4723a8 Use zm03
Add zm03 to firewalls and cacti.  Merge this after the node actually
exists.

Change-Id: If5aba999edd434baa96475dc846d6b2b315e33e4
2014-08-19 15:30:46 -07:00
James E. Blair
23f502f210 Add zm03
Just the node definition; add it to firewalls, etc, after it's
actually spun up.

Change-Id: I8b056bf446a0c6e3c42e40bd46838bed2c2a3964
2014-08-19 15:30:05 -07:00
Jenkins
a47229d742 Merge "Rename Marconi to Zaqar" 2014-08-16 16:49:55 +00:00
Flavio Percoco
b8a321ee54 Rename Marconi to Zaqar
Due to trademark issues, Marconi can't be used as a codename for the
queuing service. A new name has been voted by the team, Zaqar.

This patch replaces Marconi with Zaqar everywhere in the infrastructure
configs. The new channel has been created and openstackinfra has been
granted with all the required permissions in that channel.

Change-Id: I8e587af588d9be0b5ebbab4b0f729b106a2ae537
2014-08-16 16:11:03 +00:00
Michael Krotscheck
9223b79612 Update storyboard version to include rabbitmq
This updates the version of storyboard that is used by infra to include
rabbitmq.

Change-Id: I8cbfb17715a75cf63dd60ad4bae24bc40b3a6944
2014-08-13 14:48:30 -07:00
Jenkins
187f7d38c6 Merge "Cleanup elasticsearch related config" 2014-08-12 19:31:49 +00:00
Jeremy Stanley
aef7867cc0 Update and normalize OpenStackID dev DB creds
* manifests/site.pp: Start using hiera for openstackid-dev's
id_mysql_user parameter for consistency.

* modules/openstack_project/manifests/openstackid_dev.pp: TipIt is
moving the openstackid-dev server to new Trove databases, so update
the database names accordingly. Also null out the lingering default
values which we're filling from hiera in site.pp.

Change-Id: Id1b3b17751a34d3e760a789bf23c3ebea9c8c606
2014-08-11 18:49:35 +00:00
Michael Krotscheck
682019f1fa Modularized StoryBoard Module
In order to get the puppet module for storyboard up to a level where
we can publish it to puppetforge, I did some work on it to create
separate modules which can be used by anyone to install storyboard.

- API and Webclient are now installed via storyboard::application,
  which assumes that you can provide the DB connection criteria.
- storyboard::cert is now a separate class, which accepts either
  files or strings, which generates the SSL certificate and chain
  files for storyboard.
- storyboard::params is our dependency checker.
- storyboard::init will install a standalone, entirely
  self-contained instance of storyboard.
- Added various puppet module files necessary for eventual
  deployment to puppetforge.
- Added README.md documentation for later puppetforge addition.

This patch also includes a new module: example42-puppi, which is a
series of convenience utilities useful for deployment. For example,
puppi::netinstall (used here) will fetch tarballs and zip files and
extract them into a provided directory. It also contains changes to
the storyboard configuration for the new refresh token support patch
in #94363

Change-Id: I6ab8c24b308df38774fc0694d218dcb5022cd899
2014-08-08 10:58:17 -07:00
Clark Boylan
38b53a4bee Cleanup elasticsearch related config
Remove elasticsearch01 as it has been turned off.

Allow all elasticsearch running nodes to talk to each other over ports
9200 to 9400. This includes the logstash-worker nodes running non data
elasticsearch clients. This appears to be necessary to keep logs from
filling up with gigs of NoRouteToHost exceptions on the worker nodes.

Change-Id: I3e3365a717d22d881654206692d17dd05ed32c98
2014-08-06 18:03:04 -07:00
Tong Li
4576ca25d8 add channel #openstack-monasca to eavesdrop log server
Change-Id: I9821a2dce3b2690fab0d9694a905c0ad69f9806c
2014-07-31 22:03:26 -04:00
Jenkins
f88f5d5139 Merge "Remove the mirror slaves" 2014-07-29 23:02:19 +00:00
Monty Taylor
290d7f7435 Make the fallback sysadmins value the same
The fallback value for sysadmins was was admins in some places
and admin in others. However, re: jeblair:

The fallback value for sysadmins should be "[]", which is actually the
default value. Configuring the mail system to send root's mail to an
account called 'admin' which may or may not exist and if it does exist
may send mail to root is not a good idea.

The default of [] will deliver root's mail normally.

Change-Id: Iaeca86ef135b6a3210541618d48caf4058dc7966
2014-07-13 08:53:25 -07:00
Marton Kiss
bd9609e748 Add groups.openstack.org instance
site.pp and groups.pp update for groups portal. To make it alive
we need to do the following preliminary tasks:
- create remote database in trove for groups
- add hiera variables similar to groups-dev
- launch the instance with those manifests

Change-Id: I91d7b35ad9aabc2237d5516918fa6ce8d92ee5af
2014-07-09 20:11:39 +00:00
Monty Taylor
c8a29bea6d Delete the old cron-based puppet thing
Also, delete the legacy non-nodepool slave references.

Change-Id: I995576ab240c6f228716fa075d3513b47c41e7c1
2014-07-05 14:26:30 -07:00
Monty Taylor
73c94f4c86 Remove the mirror slaves
Now that we're using bandersnatch and cron, we do not need the
mirror slaves any longer.

Change-Id: I637dd7f32580c59ca00ff20f29ebba4d4bcf8e94
2014-07-05 11:05:49 -07:00
Monty Taylor
0fb121893d Remove salt
We don't really use the salt infrastructure that we set up, which means
it's cruft. Go ahead and admit that we don't use it and remove it from
our systems.

Change-Id: Ic43695719cbad30aded16ac480deb3dfd9b2a110
2014-07-04 21:44:44 -07:00
Monty Taylor
034f37c32a Use ansible instead of direct ssh calls
Instead of a shell script looping over ssh calls, use a simple
ansible playbook. The benefit this gets is that we can then also
script ad-hoc admin tasks either via playbooks or on the command
line. We can also then get rid of the almost entirely unused
salt infrastructure.

Change-Id: I53112bd1f61d94c0521a32016c8a47c8cf9e50f7
2014-07-04 10:01:08 -07:00
Jenkins
3a63262fa9 Merge "Remove obsolete #openstack-climate channel" 2014-07-04 11:30:23 +00:00
Jenkins
1105933c8f Merge "Only run relevant puppet apply tests" 2014-07-03 22:45:19 +00:00
Jenkins
629d146fea Merge "Add in wsgi.conf for os-loganalyze" 2014-07-03 22:15:46 +00:00
James E. Blair
38d62b3030 Only run relevant puppet apply tests
Associate an OS with each node defined in site.pp so that the puppet
apply test will only test nodes apprapriate to the node the test
is running on.

Fix remaining errors related to this test, and make it vote.

Also, start running the centos6 version regularly
(which is still non-voting).

Change-Id: I26722dce15589c982af5de36f8ccf42666d29f2b
2014-07-03 15:11:23 -07:00
Joshua Hesketh
df4f93b891 Add in wsgi.conf for os-loganalyze
Configure to use the read only swift creds that pair up with the read
write creds used to push the files.

Change-Id: I53252b3ed0d596b3fe36caef179f253bde1739cb
2014-07-03 14:24:25 -07:00
Spencer Krum
b65a2d3afc Allow site.pp to manage ca and ca_sever in puppet.conf
This allows us to set ca = false and ca_server = <fqdn> on the
new puppet 3 master.

Change-Id: Iba189bdc4bfb22fd23052f2570f52133ea184126
2014-07-02 15:01:17 -07:00
Spencer Krum
6adda92be8 Add node def for puppet3 master
This change modifies install_puppet.sh to accept a --three option
setting it to install the latest puppet available. It also creates
a node definition for the puppetmaster.o.o node, the new 3 master,
and the master of the future. Changes were made to various classes
to allow the pinning to version 2.x to be turned off.

Change-Id: I805d6dc50b9de0d8a99cf818d22d06c2dea6090a
2014-07-02 13:25:14 -07:00
Sergey Lukjanov
28e8be0700 Remove obsolete #openstack-climate channel
Transition period for climate-blazar renaming ended.

Change-Id: Ie9fcca1666f4a6f96e13d420044abee07826fada
2014-07-01 12:24:59 +04:00
Jenkins
c74e6edbc6 Merge "Groups portal infra code refactor" 2014-06-24 00:34:41 +00:00
K Jonathan Harker
ad431b7700 Move variables in manifests/site.pp to top of file
Put the variables defined in manifests/site.pp at the top of the file so
that they are in a known location after running csplit in test.sh and we
can prepend them to the puppet-apply top files.

A better solution would be to move this data into hiera, but this is not
sensitive data, so a move to hiera should wait until we a have solution
for a public hiera data repo separate from the private hiera data.

Change-Id: I509a8266462dfdf53e1727938e4fb043241166b6
2014-06-17 17:05:56 -07:00
Monty Taylor
73d0474196 Add lifeless to statusbot list
He's in APAC and also knows all of the Launchpad folks, so
communicating status is helpful.

Change-Id: Ibb43981b10abc93c947f20c48315fedce5fb6608
2014-06-14 19:38:38 -07:00
Monty Taylor
ec428260cc Add jhesketh to the magic statusbot acl list
He's in core now, he should be able to set status.

Change-Id: I673e27ca52c2b77c108d5b78a48ab91725c7b6bd
2014-06-14 19:37:13 -07:00
Clark Boylan
ce03505892 Add elasticsearch07
Add elasticsearch07 node. Move the elasticsearch discover node to
elasticsearch02 instead of 01 as we are moving away from 01 as part of
the 07 addition.

Change-Id: I2aa857ec4984ae1fc2f8e27f437f8ecc61d24fbd
2014-06-13 11:19:34 -07:00
Clark Boylan
327f1690da Be specific about which ES nodes are puppetable
Our Elasticsearch nodes need cinder volumes attached to them prior to
being puppeted. Make the regex for the elasticsearch nodes in site.pp
match only existing nodes so that new nodes can be created and have
their volumes attached before proper puppeting.

Change-Id: If12c67174315b8480998148e1325de59ecdba372
2014-06-12 16:21:56 -07:00
Marton Kiss
a33ba74399 Groups portal infra code refactor
This patch removes the drush make site building function from
groups-dev instance, and now directly fetch release tarballs from
http://tarballs.openstack.org/groups repository. With an advanced
multi-slot deployment architecture it prevents the typical Drupal
WSOD issues that randomly caused site malfunction when a request
arrived during installation. It also simplifies the
deployment steps using the standard drush aliases and
drush-dsd extension and supports local configuration variables
in local_settings.php file.

Change-Id: I73976a60e080d15b6f513db79fee46bcf468e302
2014-06-11 23:12:12 +02:00
Clark Boylan
2c4cdcd317 Add four new logstash-worker nodes.
We have fallen behind on our ability to index all the things. Logstash
itself appears to be the current bottleneck. Add four more nodes to help
combat the backlog.

Change-Id: I9d5a7474a801c58c1e8933b264172e72f2d0bb86
2014-06-09 17:19:25 -07:00
Sergey Lukjanov
1678da69c5 Add #openstack-sahara to status bot
Change-Id: Id97e9c97fec5393b8d1b09a48ce61c871f633867
2014-06-02 18:25:48 +04:00
Jonathan Harker
4f25a69a89 Run the puppet apply test (requires sudo)
The test.sh script is not currently being run in any jobs, this change
removes the redundant validation code that's also in the puppet-syntax
job and creates a puppet-apply-test job that runs the test.sh script.

Running `puppet apply --noop` requires sudo, otherwise it will give
errors about refusing to run commands as other users.

Change-Id: Ie6b278d98390a8a5dd8bb24899c8c4083f5755c9
2014-05-28 17:02:16 -07:00
Clark Boylan
dd33ac2e10 Pass tenant_name to zuul config.
Turns out that while keystoneclient does not require a tenant_name be
set swiftclient does require it so the fix to not pass a tenant_name to
zuul did not work.

Alex_Gaynor pointed out the reason using the normal tenant_name doesn't
work with Rax object store is that the tenant_name for object store in
Rax is different than the normal tenant_name. Use that different and
correct tenant_name instead.

Change-Id: I13d6b8dfc190d35178e7fc979d7feb58e7e1f8dd
2014-05-23 16:00:50 -07:00
Sergey Lukjanov
9b26f18aaa Rename climate to blazar
Keep old channel #openstack-climate for some transition period.

Change-Id: Iea44afdbd69a43e0d6c349587ecb50318130b8b4
2014-05-23 21:54:07 +00:00
Jenkins
6d2af6e0cb Merge "Adding #openstack-containers IRC bot" 2014-05-23 10:12:42 +00:00
Joshua Hesketh
564f8ab237 Configure swift credentials for workers to push to
This change adds credentials as set up by fungi (2014-04-05T03:16:46)
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-04-05.log

This allows zuul to send signed URL's for workers to push files to on
a per-job basis.

This change will require a zuul restart, not just a reload!

Note this is unreverts the revert in
c1b98eaff6 but is slightly edited hence
the lack of a proper revert commit. The reason for the revert has been
debugged and was due to rax identity service not providing a catalog
entry for swift when a tenant name is provided.

Change-Id: I04d3207002f7422b9851515ee88a74b19dd2f248
2014-05-20 13:36:36 -07:00
Adrian Otto
eaf54f1bb3 Adding #openstack-containers IRC bot
This is a dedicated IRC channel for discussion
about Operating System container support in
OpenStack. The members of the OpenStack
Containers Team will idle in this channel:

https://wiki.openstack.org/wiki/Teams/Containers

Change-Id: I9ec77969f926e8ced912ce194f7d6fae6a0e2e4d
2014-05-17 10:10:28 -07:00
Clark Boylan
c1b98eaff6 Revert "Configure swift credentials for workers to push to"
This reverts commit 176f15cd48.

Puppet and zuul aren't ready for this. The swift client throws an
exception. We need to restart zuul to fix a more pressing issue hence
this revert. Will debug when able to.

Change-Id: Ie4487d84b7f1495c7ec6fd31e4b5b90d6699a31d
2014-05-05 18:49:15 -07:00
Jenkins
30d8b49457 Merge "Configure swift credentials for workers to push to" 2014-05-05 22:14:52 +00:00