Commit Graph

25 Commits

Author SHA1 Message Date
Khai Do
0e89f115a3 run git gc instead of repack for git repos
Run git gc instead of repack on all repos.  Testing[1] indicates that
gc-ing reduces repo size and helps maintain better performance for
apps that host the repos.

remove git pack-refs because a git gc enables packRefs by default[2]

[1] http://lists.openstack.org/pipermail/openstack-infra/2016-June/004403.html
[2] https://git-scm.com/docs/git-gc

Change-Id: I4e171b17a7b4107c992d0e3cc0e7bff3d9599526
2016-06-17 15:12:51 -07:00
Elizabeth K. Joseph
359b7d48b9 Customize cgit max-repo-count and set it to 1500
The max-repo-count setting in the cgitrc adjusts how many repos should
be displayed on each page. In the puppet-cgit module we have a default
of 600, which was intended to be enough for all the repos to be shown
on git.openstack.org without a second page. We're now at over 1,000
repos, so you now need to page through them and this is not an optimal
experience. Since cgit can handle loading thousands of repos on the
main page, add the setting to our local configuration and bump it up
to 1500.

Change-Id: I7a2ea185f0876806ecfd17bedd84dbcd9769b124
2016-03-04 13:12:40 -08:00
Timothy Chavez
e130792ed9 Parameterize setting the selinux mode
By parameterizing the selinux mode we can set the mode and pass the mode
to other classes so they can make decisions on whether or not to run
selinux commands as part of their configuration.

Depends-On: I21add092d9d09077f2b23760a384f5a5cb91d86a
Change-Id: I86a5bb006b6ab9d64f7fbd894c967428cfaed8f8
2015-09-10 13:38:53 -05:00
Timothy Chavez
744de231fe Use valid cgitrc configuration key
The cgitrc configuration should be 'clone-prefix' not 'clone_prefix'[1]
This fix will correctly override the default set in puppet-cgit.

[1] http://git.zx2c4.com/cgit/tree/cgitrc.5.txt

Change-Id: Ie96e046febcacc5359fe449b82ee6f32bcac9287
2015-07-21 18:52:03 -05:00
Jenkins
6c1b600157 Merge "Start using puppet-cgit to manage cgitrc" 2015-07-16 21:48:26 +00:00
Yolanda Robla
f2d0e3d799 Start using puppet-cgit to manage cgitrc
Instead of taking care of cgitrc configuration in system-config,
use the new embedded logic for puppet-cgit to set that
configuration.

Depends on puppet-cgit module, as the configuration was moved
there.

Depends-On: I5c8ef0e7036c4b7a33aa12dc502c048af0499dda
Change-Id: I0bdbbdc3689cc33981c9c74af4b6c3080acf2dbe
2015-07-09 06:26:33 +00:00
Yolanda Robla
8ebf8e2953 Fixes in git_backend class
Two minor changes here:
- vhost_name need to be double quoted for vhost_name to be
interpreted
- create-cgitrepos script does not exist until jeepyb is
installed, so requiring jeepyb class before calling that.

Change-Id: I1596a76a67014d7a18fa85ff5fadb5cb0b8e2fb0
2015-06-15 13:24:47 +02:00
Jenkins
e9fb6f09cf Merge "Move server outside git_backend class" 2015-05-07 16:40:06 +00:00
Yolanda Robla
767a7a0cb9 Don't hardcode names on certs
It should be more flexible to just add use vhost_name
as part of the name, instead of hardcoding to openstack.

Change-Id: I0f3e9216480e4ef008b965481e669b12597566d1
2015-04-30 15:38:55 +02:00
Yolanda Robla
ba9c487e6e Move server outside git_backend class
Change-Id: I00d3ce8970e0a0dacbca93fde918e406e99e1600
Story: 2000172
Spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/server_base_template_refactor.html
2015-04-30 15:25:24 +02:00
Clark Boylan
6ab8bd166a Install lsof on the git mirror nodes
lsof is a very valuable tool for debugging connectivity issues. It is
not installed on our centos6 nodes by default. Install it here so that
we have it available for debugging connections issues.

Change-Id: I836b6bb4e852a2fbbff8d3a1620920de8d03b112
2014-11-20 14:58:27 -08:00
James E. Blair
e98e3d49fb Switch git and storyboard to project-config
Change-Id: I78a5ac024bbc44504529233804288ccc81829ede
2014-09-25 13:11:54 -07:00
K Jonathan Harker
c29a9dd8df Only include selinux bits on rhel-based systems
The gate-config-puppet-apply test will try to run everything on a
bare-precise node, so even though this doesn't occur in production it is
being tested.

Change-Id: I13b03a7476e30cbb26dc820bb4cf5b56c4018669
2014-05-30 15:36:07 -07:00
Monty Taylor
8895d64289 Fix up ssh key usage
We need for the keys to only have the key content. Also, we've learned
that the ssh_authorized_key title is important.

Change-Id: Ie772572e041ea6c1587dddb02937b89541a37e1b
2014-04-25 12:35:25 -07:00
Jenkins
b6f3fd8d6e Merge "Move to using ssh_authorized_key type" 2014-04-25 09:03:46 +00:00
Jenkins
dd5c97352d Merge "Clarify comment about variables in git_backend.pp" 2014-04-22 19:45:34 +00:00
Monty Taylor
6c1486ea92 Move to using ssh_authorized_key type
There is a builtin type for adding authorized_keys. We started
using it for the restricted access for the puppet triggering key,
and it seems to make sense to be consistent across the board.

Change-Id: I76ffb0136c58c05fa7c8abb793cb8e3b03cb5f6c
2014-04-15 20:24:23 -07:00
Clark Boylan
943ed2315f Zuul use dedicated ssh key.
Zuul should use a dedicated ssh key rather than piggy backing off of the
jenkins ssh key. This change makes this distinction clear and removes
one zuul use of the jenkins ssh key that is not necessary.

Change-Id: I74c811a8bf94838b285791e158f4e468513eaa3e
2014-04-15 10:54:52 -07:00
Elizabeth Krumbach Joseph
d3e5eba820 Clarify comment about variables in git_backend.pp
A couple of varibles that are not used by the git_backend.pp
manifest are defined because they're required by jeepyb.
Rewrote inline comment to make this more clear.

Change-Id: Ic96df55c40847bafcf5ce6197f1e627d7aef26aa
2014-04-04 12:25:33 -07:00
Jeremy Stanley
2f4631cc0e Repack git repositories daily
The volume of loose refs which get replicated to our git backends
pile up quickly during periods of heavy use, which increases load
and network traffic on the servers, slowing down updates and raising
the risk from network errors. Reduce the scheduled repack frequency
from weekly to daily, which should also result in quicker repacks.

Change-Id: I6380e67444c54f5b1a1c1b5b631e95c5a20f119a
2014-03-04 15:45:00 +00:00
Monty Taylor
bc794da31e Split config from projects list
The projects list is a common list for devs to interact with. The config
in the list is not, but the config in the list means the file needs to
be in an erb template.

Split the two concerns, similar to zuul. Put the config in a config file
and the project data in a yaml file.

Change-Id: I708b8655b4b1ce377f3b7369e987418c1d72d977
2014-01-28 09:32:09 -08:00
Jeremy Stanley
73c0175819 Allow zuul to push to git servers
* manifests/site.pp
* modules/openstack_project/manifests/git_backend.pp: Add the zuul
server's public SSH key to a dedicated zuul account on the git
server farm. This will allow zuul to push arbitrary references into
its scratch repositories. Also set ownership on its scratch
repository hierarchy and pack them, properly skipping it in the
existing cgit repo repack since ownership differs.

Change-Id: I4b5b583847c828aadc903270433882531e5b1ea7
2014-01-08 16:43:36 +08:00
Jeremy Stanley
09b1e91b63 Pass a zuul scratch subpath to create-cgitrepos
* modules/openstack_project/manifests/git_backend.pp: Pass "zuul" as
the SCRATCH_SUBPATH environment variable for the create-cgitrepos
utility. With a new enough jeepyb to support this, it will generate
empty shadow git repositories where zuul can stow its throwaway job
references.

Change-Id: I26828ab9935e47382f6706a5c590f32b806d1713
2014-01-08 01:31:57 +00:00
Mathieu Gagné
6bd2888255 Add commit-filter for cgit
A commit-filter adds the ability to format commit messages.

This commit-filter automatically adds links for:
- Change-id: Link to the change on review.o.o
- SHA1 hash: Link to the git commit on git.o.o
- Bug: Link to the bug on launchpad.net
- Blueprint: Link to the blueprint on launchpad.net

Change-Id: If8916948ca154e6957efb005d35812ae097721b7
2013-08-28 15:47:09 -04:00
Clark Boylan
f22f8f5aa1 Split git.o.o haproxy out of cgit module.
* manifests/site.pp: Use distinct manifests for git.o.o and git.o.o
backends. This allows for the haproxy server to not serve git content
and purely be a load balancer.

* modules/cgit/manifests/init.pp: Remove haproxy from cgit module.
Remove stale xinetd cleanup. Select git daemon port when selecting
HTTP(S) ports.

* modules/openstack_project/manifests/git.pp: Make git.pp a manifest to
load balance git servers with haproxy.

* modules/openstack_project/manifests/git_backend.pp: New manifest to
manage servers that actually serve git content. They sit behind a load
balancer.

* modules/openstack_project/manifests/review.pp: Stop replicating repos
to git load balancer.

Change-Id: I343a0d1e0a7b93874c2e2299ed974a3304957efb
2013-08-26 12:05:26 -07:00