Rename CI to Infra in MANY MANY places.

Change-Id: I409bd50ae374e0288531f07cfeea34856c5f8067
Reviewed-on: https://review.openstack.org/17319
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: James E. Blair <corvus@inaugust.com>
This commit is contained in:
Monty Taylor 2012-12-02 11:52:32 -08:00 committed by James E. Blair
parent 5b9461b5e1
commit 74460853d1
52 changed files with 228 additions and 584 deletions

View File

@ -1,4 +1,4 @@
[gerrit]
host=review.openstack.org
port=29418
project=openstack/openstack-ci-puppet.git
project=openstack-infra/config.git

View File

@ -1,5 +1,5 @@
These are a set of puppet manifests and modules that are currently being
used to manage the OpenStack CI infrastructure.
used to manage the OpenStack Project infrastructure.
The main entry point is in manifests/site.py.

View File

@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
#
# OpenStack CI documentation build configuration file, created by
# sphinx-quickstart on Mon Jul 18 13:42:23 2011.
# OpenStack Project Infrastructure documentation build configuration file,
# created by sphinx-quickstart on Mon Jul 18 13:42:23 2011.
#
# This file is execfile()d with the current directory set to its containing
# dir.
@ -41,8 +41,8 @@ source_suffix = '.rst'
master_doc = 'index'
# General information about the project.
project = u'OpenStack CI'
copyright = u'2011, Monty Taylor, James Blair and Andrew Hutchings'
project = u'OpenStack Project Infrastructure'
copyright = u'2012, OpenStack Infastructure Team - see git repo for details'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@ -169,7 +169,7 @@ html_static_path = ['_static']
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'OpenStackCIdoc'
htmlhelp_basename = 'OpenStackInfradoc'
# -- Options for LaTeX output -------------------------------------------------
@ -184,8 +184,8 @@ htmlhelp_basename = 'OpenStackCIdoc'
# (source start file, target name, title, author, documentclass
# [howto/manual]).
latex_documents = [
('index', 'OpenStackCI.tex', u'OpenStack CI Documentation',
u'Monty Taylor and James Blair', 'manual'),
('index', 'OpenStackInfra.tex', u'OpenStack Infrastructure Documentation',
u'OpenStack Infrastructure Team', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
@ -217,6 +217,6 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'openstackci', u'OpenStack CI Documentation',
[u'Monty Taylor, James Blair and Andrew Hutchings'], 1)
('index', 'openstackinfra', u'OpenStack Infrastructure Documentation',
[u'OpenStack Infrastructure Team'], 1)
]

View File

@ -41,8 +41,8 @@ host for use by the OpenStack project.
.. code-block:: bash
sudo apt-get install puppet git openjdk-6-jre-headless mysql-server
git clone git://github.com/openstack/openstack-ci-puppet.git
cd openstack-ci-puppet/
git clone git://github.com/openstack-infra/config.git
cd config/
sudo bash run_puppet.sh
Install MySQL
@ -462,59 +462,18 @@ onto the gerrit servers. This script follows two rules:
If your review gets touched by either of these rules it is possible to
unabandon a review on the gerrit web interface.
Launchpad Sync
==============
The launchpad user sync process consists of two scripts which are in
openstack/openstack-ci on github: sync_launchpad_gerrit.py and
insert_gerrit.py.
Both scripts should be run as gerrit2 on review.openstack.org
sync_launchpad_users.py runs and creates a python pickle file, users.pickle,
with all of the user and group information. This is a long process. (12
minutes)
insert_gerrit.py reads the pickle file and applies it to the MySQL database.
The gerrit caches must then be flushed.
Depends
-------
::
apt-get install python-mysqldb python-openid python-launchpadlib
Keys
----
The key for the launchpad sync user is in ~/.ssh/launchpad_rsa. Connecting
to Launchpad requires oauth authentication - so the first time
sync_launchpad_gerrit.py is run, it will display a URL. Open this URL in a
browser and log in to launchpad as the hudson-openstack user. Subsequent
runs will run with cached credentials.
Running
-------
::
cd openstack-ci
git pull
python sync_launchpad_gerrit.py
python insert_gerrit.py
ssh -i /home/gerrit2/.ssh/launchpadsync_rsa -p29418 review.openstack.org gerrit flush-caches
Gerrit IRC Bot
==============
Installation
------------
Ensure there is an up-to-date checkout of openstack-ci in ~gerrit2.
Ensure there is an up-to-date checkout of openstack-infra/config in ~gerrit2.
::
apt-get install python-irclib python-daemon python-yaml
cp ~gerrit2/openstack-ci/gerritbot.init /etc/init.d
cp ~gerrit2/openstack-infra/config/gerritbot.init /etc/init.d
chmod a+x /etc/init.d/gerritbot
update-rc.d gerritbot defaults
su - gerrit2
@ -560,7 +519,7 @@ them are applied.
Installation
------------
Ensure an up-to-date checkout of openstack-ci is in ~gerrit2.
Ensure an up-to-date checkout of openstack-infra/config is in ~gerrit2.
::
@ -650,7 +609,7 @@ Creating a new Gerrit Project with Puppet
=========================================
Gerrit project creation is now managed through changes to the
openstack-ci-puppet repository. The old manual processes are documented
openstack-infra/config repository. The old manual processes are documented
below as the processes are still valid and documentation of them may
still be useful when dealing with corner cases. That said, you should
use this method whenever possible.
@ -665,7 +624,7 @@ also want to configure Zuul and Jenkins to run tests on the new project.
The details for that process are in the next section.
Gerrit projects are configured in the
``openstack-ci-puppet/modules/openstack_project/templates/review.projects.yaml.erb``.
``openstack-infra/config:modules/openstack_project/templates/review.projects.yaml.erb``.
file. This file contains two sections, the first is a set of default
config values that each project can override, and the second is a list
of projects (each may contain their own overrides).
@ -734,7 +693,7 @@ a single project you will want to do the following:
mergeContent = true
#. Add a project entry for the project in
``openstack-ci-puppet/modules/openstack_project/templates/review.projects.yaml.erb``.::
``openstack-infra/config:modules/openstack_project/templates/review.projects.yaml.erb``.::
- project: openstack/project-name
acl_config: /home/gerrit2/acls/project-name.config
@ -756,7 +715,7 @@ Have Zuul Monitor a Gerrit Project
=====================================
Define the required jenkins jobs for this project using the Jenkins Job
Builder. Edit openstack/openstack-ci-puppet:modules/openstack_project/files/jenkins_jobs/config/projects.yaml
Builder. Edit openstack-infra/config:modules/openstack_project/files/jenkins_jobs/config/projects.yaml
and add the desired jobs. Most projects will use the python jobs template.
A minimum config::
@ -787,7 +746,7 @@ Full example config for nova::
- openstack-publish-jobs
- gate-{name}-pylint
Edit openstack/openstack-ci-puppet:modules/openstack_project/files/zuul/layout.yaml
Edit openstack-infra/config:modules/openstack_project/files/zuul/layout.yaml
and add the required jenkins jobs to this project. At a minimum you will
probably need the gate-PROJECT-merge test in the check and gate queues.
@ -867,7 +826,7 @@ Pull requests can not be disabled for a project in Github, so instead
we have a script that runs from cron to close any open pull requests
with instructions to use Gerrit.
* Edit openstack/openstack-ci-puppet:modules/openstack_project/templates/review.projects.yaml.erb
* Edit openstack-infra/config:modules/openstack_project/templates/review.projects.yaml.erb
and add the project to the list of projects in the yaml file
@ -911,14 +870,8 @@ inside of git in gerrit. Check out the branch from git::
There will be two interesting files, `groups` and `project.config`. `groups`
contains UUIDs and names of groups that will be referenced in
`project.config`. There is a helper script in the openstack-ci repo called
`get_group_uuid.py` which will fetch the UUID for a given group. For
$PROJECT-core and $PROJECT-drivers::
openstack-ci/gerrit/get_group_uuid.py $GROUP_NAME
And make entries in `groups` for each one of them. Next, edit
`project.config` to look like::
`project.config`. UUIDs can be found on the group page in gerrit.
Next, edit `project.config` to look like::
[access "refs/*"]
owner = group Administrators

View File

@ -1,17 +1,17 @@
.. OpenStack CI documentation master file, created by
.. OpenStack Infrastructure documentation master file, created by
sphinx-quickstart on Mon Jul 18 13:42:23 2011.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
OpenStack Continuous Integration
OpenStack Project Infrastructure
================================
This documentation covers the installation and maintenance of the
Continuous Integration (CI) infrastructure used by OpenStack. It
project infrastructure used by OpenStack. It
may be of interest to people who may want to help develop this
infrastructure or integrate their tools into it. Some instructions
may be useful to other projects that want to set up similar CI
systems.
may be useful to other projects that want to set up similar infrastrucutre
systems for their developers.
OpenStack developers or users do not need to read this documentation.
Instead, see http://wiki.openstack.org/ to learn how contribute to or

View File

@ -53,305 +53,7 @@ Currently OpenStack integration testing is performed by the devstack
gate test framework. This framework runs the devstack exercises and
Tempest smoketests against a devstack install on single use cloud
servers. The devstack gate source can be found on
`Github <https://github.com/openstack-ci/devstack-gate>`_ and the
`Readme <https://github.com/openstack-ci/devstack-gate/blob/master/README.md>`_
`Github <https://github.com/openstack-infra/devstack-gate>`_ and the
`Readme <https://github.com/openstack-infra/devstack-gate/blob/master/README.md>`_
describes the process of using devstack gate to run your own devstack
based tests.
Rackspace Bare-Metal Testing Cluster
====================================
The Rackspace Bare-Metal Testing Cluster is no longer in place and is
not used for OpenStack integration testing. The following documentation
is retained as it is potentially useful should you want to do something
similar and it is historically significant.
The CI team mantains a cluster of machines supplied by Rackspace to
perform bare-metal deployment and testing of OpenStack as a whole.
This installation is intended as a reference implementation of just
one of many possible testing platforms, all of which can be integrated
with the OpenStack Jenkins system. This is a cluster of several
physical machines meaning the test environment has access to all of
the native processor features, and real-world networking, including
tagged VLANs.
Each time the trunk repo is updated, a Jenkins job will deploy an
OpenStack cluster using devstack and then run the openstack-test-rax
test suite against the cluster.
Deployment and Testing Process
------------------------------
The cluster deployment is divided into two phases: base operating
system installation, and OpenStack installation. Because the
operating system install takes considerable time (15 to 30 minutes),
has external network resource dependencies (the distribution mirror),
and has no bearing on the outcome of the OpenStack tests themselves,
the process used here effectively snapshots the machines immediately
after the base OS install and before OpenStack is installed. LVM
snapshots and kexec are used to immediately return the cluster to a
newly installed state without incurring the additional time it would
take to install from scratch. The Jenkins testing job invokes the
process starting at :ref:`rax_openstack_install`.
Installation Server Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The CI team runs the Ubuntu Orchestra server (based on cobbler) on our
Jenkins slave node to manage the OS installation on the test machines.
The configuration for the Orchestra server is kept in the CI team's
puppet modules. If you want to set up your own system, Orchestra is
not required, any system capable of performing the following steps is
suitable. However, if you want to stand up a test system as quickly
and simply as possible, you may find it easiest to base your system on
the one the CI team uses. You may use the puppet modules yourself, or
follow the instructions below.
The CI team's Orchestra configuration module is at:
https://github.com/openstack/openstack-ci-puppet/tree/master/modules/orchestra
Install Orchestra
"""""""""""""""""
Install Ubuntu 11.10 (Oneiric) and Orchestra::
sudo apt-get install ubuntu-orchestra-server ipmitool
The install process will prompt you to enter a password for Cobbler.
Have one ready and keep it in a safe place. The procedure here will
not use it, but if you later want to use the Cobbler web interface,
you will need it.
Configure Orchestra
"""""""""""""""""""
Install the following files on the Orchestra server so that it deploys
machines with our LVM/kexec test framework.
We update the dnsmasq.conf cobbler template to add
"dhcp-ignore=tag:!known", and some site-specific network
configuration::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/dnsmasq.template \
-O /etc/cobbler/dnsmasq.template
Our servers need a kernel module blacklisted in order to boot
correctly. If you don't need to blacklist any modules, you should
either create an empty file here, or remove the reference to this file
from the preseed file later::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/openstack_module_blacklist \
-O /var/lib/cobbler/snippets/openstack_module_blacklist
This cobbler snippet uses cloud-init to set up the LVM/kexec
environment and configures TCP syslogging to the installation
server/Jenkins slave::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/openstack_cloud_init \
-O /var/lib/cobbler/snippets/openstack_cloud_init
This snippet holds the mysql root password that will be configured at
install time. It's currently a static string, but you could
dynamically write this file, or simply replace it with something more
secure::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/openstack_mysql_password \
-O /var/lib/cobbler/snippets/openstack_mysql_password
This preseed file manages the OS install on the test nodes. It
includes the snippets installed above::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/openstack-test.preseed \
-O /var/lib/cobbler/kickstarts/openstack-test.preseed
The following sudoers configuration is needed to allow Jenkins to
control cobbler, remove syslog files from the test hosts before
starting new tests, and restart rsyslog::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/orchestra-jenkins-sudoers -O /etc/sudoers.d/orchestra-jenkins
Replace the Orchestra rsyslog config file with a simpler one that logs
all information from remote hosts in one file per host::
wget https://raw.github.com/openstack/openstack-ci-puppet/master/modules/orchestra/files/99-orchestra.conf -O /etc/rsyslog.d/99-orchestra.conf
Make sure the syslog directories exist and restart rsyslog::
mkdir -p /var/log/orchestra/rsyslog/
chown -R syslog.syslog /var/log/orchestra/
restart rsyslog
Add an "OpenStack Test" system profile to cobbler that uses the
preseed file above::
cobbler profile add \
--name=natty-x86_64-ostest \
--parent=natty-x86_64 \
--kickstart=/var/lib/cobbler/kickstarts/openstack-test.preseed \
--kopts="priority=critical locale=en_US"
Add each of your systems to cobbler with a command similar to this
(you may need different kernel options)::
cobbler system add \
--name=baremetal1 \
--hostname=baremetal1 \
--profile=natty-x86_64-ostest \
--mac=00:11:22:33:44:55 \
--power-type=ipmitool \
--power-user=IPMI_USERNAME \
--power-pass=IPMI_PASS \
--power-address=IPMI_IP_ADDR \
--ip-address=SYSTEM_IP_ADDRESS \
--subnet=SYSTEM_SUBNET \
--kopts="netcfg/choose_interface=auto netcfg/dhcp_timeout=60 auto=true priority=critical"
When complete, have cobbler write out its configuration files::
cobbler sync
Set Up Jenkins Jobs
"""""""""""""""""""
We have Jenkins jobs to handle all of the tasks after the initial
Orchestra configuration so that we can easily run them at any time.
This includes the OS installation on the test nodes, even though we
don't run that often because the state is preserved in an LVM
snapshot, we may want to change the configuration used and make a new
snapshot. In that case we just need to trigger the Jenkins job again.
The Jenkins job that kicks off the operating system installation calls
the "baremetal-os-install.sh" script from the openstack-ci repo:
https://github.com/openstack/openstack-ci/blob/master/slave_scripts/baremetal-os-install.sh
That script instructs cobbler to install the OS on each of the test
nodes.
To speed up the devstack installation and avoid excessive traffic to
the pypi server, we build a PIP package cache on the installation
server. That is also an infrequent task that we configure as a
jenkins job. That calls:
https://github.com/openstack/openstack-ci/blob/master/slave_scripts/update-pip-cache.sh
That builds a PIP package cache that the test script later copies to
the test servers for use by devstack.
Run those two jobs, and once complete, the test nodes are ready to go.
This is the end of the operating system installation, and the system
is currently in the pristine state that will be used by the test
procedure (which is stored in the LVM volume "orig_root").
.. _rax_openstack_install:
OpenStack Installation
~~~~~~~~~~~~~~~~~~~~~~
When the deployment and integration test job runs, it does the
following, each time starting from the pristine state arrived at the
end of the previous section.
Reset the Test Nodes
""""""""""""""""""""
The Jenkins deployment and test job first runs the deployment script:
https://github.com/openstack/openstack-ci/blob/master/slave_scripts/baremetal-deploy.sh
Which invokes the following script on each host to reset it to the
pristine state:
https://github.com/openstack/openstack-ci/blob/master/slave_scripts/lvm-kexec-reset.sh
Because kexec is in use, resetting the environment and rebooting into
the pristine state takes only about 3 seconds.
The deployment script then removes the syslog files from the previous
run and restarts rsyslog to re-open them. Once the first test host
finishes booting and brings up its network, OpenStack installation
starts.
Run devstack on the Test Nodes
""""""""""""""""""""""""""""""
Devstack's build_bm_multi script is run, which invokes devstack on
each of the test nodes. First on the "head" node which runs all of
the OpenStack services for the remaining "compute" nodes.
Run Test Suite
""""""""""""""
Once devstack is complete, the test suite is run. All logs from the
test nodes should be sent via syslog to the Jenkins slave, and at the
end of the test, the logs are archived with the Job for developers to
inspect in case of problems.
Cluster Configuration
---------------------
Here are the configuration parameters of the CI team's test cluster.
The cluster is currently divided into three mini-clusters so that
independent Jenkins jobs can run in parallel on the different
clusters.
VLANs
~~~~~
+----+--------------------------------+
|VLAN| Description |
+====+================================+
|90 | Native VLAN |
+----+--------------------------------+
|91 | Internal cluster communication |
| | network: 192.168.91.0/24 |
+----+--------------------------------+
|92 | Public Internet (fake) |
| | network: 192.168.92.0/24 |
+----+--------------------------------+
Servers
~~~~~~~
The servers are located on the Rackspace network, only accessible via
VPN.
+-----------+--------------+---------------+
| Server | Primary IP | Management IP |
+===========+==============+===============+
|deploy-rax | 10.14.247.36 | 10.14.247.46 |
+-----------+--------------+---------------+
|baremetal1 | 10.14.247.37 | 10.14.247.47 |
+-----------+--------------+---------------+
|baremetal2 | 10.14.247.38 | 10.14.247.48 |
+-----------+--------------+---------------+
|baremetal3 | 10.14.247.39 | 10.14.247.49 |
+-----------+--------------+---------------+
|baremetal4 | 10.14.247.40 | 10.14.247.50 |
+-----------+--------------+---------------+
|baremetal5 | 10.14.247.41 | 10.14.247.51 |
+-----------+--------------+---------------+
|baremetal6 | 10.14.247.42 | 10.14.247.52 |
+-----------+--------------+---------------+
|baremetal7 | 10.14.247.43 | 10.14.247.53 |
+-----------+--------------+---------------+
|baremetal8 | 10.14.247.44 | 10.14.247.54 |
+-----------+--------------+---------------+
|baremetal9 | 10.14.247.45 | 10.14.247.55 |
+-----------+--------------+---------------+
deploy-rax
The deployment server and Jenkins slave. It deploys the servers
using Orchestra and Devstack, and runs the test framework. It
should not run any OpenStack components, but we can install
libraries or anything else needed to run tests.
baremetal1, baremetal4, baremetal7
Configured as "head" nodes to run nova, mysql, and glance. Each one
is the head node of a three node cluster including the two compute
nodes following it
baremetal2-3, baremtal5-6, baremetal8-9
Configured as compute nodes for each of the three mini-clusters.

View File

@ -7,17 +7,17 @@ Overview
In order to make the process of managing hundreds of Jenkins Jobs easier a
Python based utility was designed to take YAML based configurations and convert
those into jobs that are injected into Jenkins. The source for this utility can
be found on `github <https://github.com/openstack-ci/jenkins-job-builder>`_ and
be found on `github <https://github.com/openstack-infra/jenkins-job-builder>`_ and
it comes with its own
`documentation <http://ci.openstack.org/jenkins-job-builder/>`_.
The documentation below describes how the OpenStack CI team uses the Jenkins
Job Builder in their environment.
The documentation below describes how the OpenStack Infrastructure team uses
the Jenkins Job Builder in their environment.
Configuring Projects
--------------------
The YAML scripts to make this work are stored in the ``openstack-ci-puppet``
The YAML scripts to make this work are stored in the ``openstack-infra/config``
repository in the
``modules/openstack_project/files/jenkins_job_builder/config/`` directory.
In this directory you can have four different types of yaml config files:
@ -49,8 +49,8 @@ Example defaults config:
<p>If you would like to make changes to this job, please see:
<a href="https://github.com/openstack/openstack-ci-puppet">
https://github.com/openstack/openstack-ci-puppet
<a href="https://github.com/openstack-infra/config">
https://github.com/openstack-infra/config
</a>
In modules/openstack_project/files/jenkins_jobs
@ -252,7 +252,7 @@ The contents of this erb are:
url=<%= url %>
The values for user and url are hardcoded in the Puppet repo in
`modules/openstack_project/manifests/jenkins.pp <https://github.com/openstack/openstack-ci-puppet/blob/master/modules/openstack_project/manifests/jenkins.pp>`_,
`modules/openstack_project/manifests/jenkins.pp <https://github.com/openstack-infra/config/blob/master/modules/openstack_project/manifests/jenkins.pp>`_,
but the password is stored in hiera. Make sure you have it defined as
``jenkins_jobs_password`` in the hiera DB.

View File

@ -4,7 +4,7 @@ Meetbot
Overview
--------
The OpenStack CI team run a slightly modified
The OpenStack Infrastructure team run a slightly modified
`Meetbot <http://wiki.debian.org/MeetBot>`_ to log IRC channel activity and
meeting minutes. Meetbot is a plugin for
`Supybot <http://sourceforge.net/projects/supybot/>`_ which adds meeting
@ -21,8 +21,8 @@ get you going, but there are other goodies in ``doc/``.
Once you have Supybot installed you will need to configure a bot. The
``supybot-wizard`` command can get you started with a basic config, or you can
have Puppet do the heavy lifting. The OpenStack CI Meetbot Puppet module creates
a configuration and documentation for that module is at
have Puppet do the heavy lifting. The OpenStack Infrastructure Meetbot Puppet
module creates a configuration and documentation for that module is at
:ref:`Meetbot_Puppet_Module`.
One important config setting is ``supybot.reply.whenAddressedBy.chars``, which
@ -33,20 +33,21 @@ if you don't want a prefix character).
Meetbot
-------
The OpenStack CI Meetbot fork can be found at
https://github.com/openstack-ci/meetbot. Manual installation of the Meetbot
The OpenStack Infrastructure Meetbot fork can be found at
https://github.com/openstack-infra/meetbot. Manual installation of the Meetbot
plugin is straightforward and documented in that repository's README.
OpenStack CI installs and configures Meetbot through Puppet. Documentation for
the Puppet module that does that can be found at :ref:`Meetbot_Puppet_Module`.
OpenStack Infrastructure installs and configures Meetbot through Puppet.
Documentation for the Puppet module that does that can be found at
:ref:`Meetbot_Puppet_Module`.
Voting
^^^^^^
The OpenStack CI Meetbot fork adds simple voting features. After a meeting has
been started a meeting chair can begin a voting block with the ``#startvote``
command. The command takes two arguments, a question posed to voters (ending
with a ``?``), and the valid voting options. If the second argument is missing
the default options are "Yes" and "No". For example:
The OpenStack Infrastructure Meetbot fork adds simple voting features. After
a meeting has been started a meeting chair can begin a voting block with the
``#startvote`` command. The command takes two arguments, a question posed to
voters (ending with a ``?``), and the valid voting options. If the second
argument is missing the default options are "Yes" and "No". For example:
``#startvote Should we vote now? Yes, No, Maybe``

View File

@ -20,12 +20,12 @@ ship the data to the clients. To install this:
sudo apt-get install puppet puppetmaster-passenger
Files for puppet master are stored in a git repo clone at
``/opt/openstack-ci-puppet``. We have a ``root`` cron job that
``/opt/openstack-infra/config``. We have a ``root`` cron job that
automatically populates these from our puppet git repository as follows:
.. code-block:: bash
\*/15 * * * * sleep $((RANDOM\%600)) && cd /opt/openstack-ci-puppet && /usr/bin/git pull -q
\*/15 * * * * sleep $((RANDOM\%600)) && cd /opt/openstack-infra/config && /usr/bin/git pull -q
The ``/etc/puppet/puppet.conf`` file then needs updating to point to the
manifest and modules as follows:
@ -37,8 +37,8 @@ manifest and modules as follows:
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
manifestdir=/opt/openstack-ci-puppet/manifests
modulepath=/opt/openstack-ci-puppet/modules
manifestdir=/opt/openstack-infra/config/manifests
modulepath=/opt/openstack-infra/config/modules
manifest=$manifestdir/site.pp
Hiera
@ -72,9 +72,9 @@ which tells is where to find subsequent configuration files.
:datadir: '/etc/puppet/hieradata/%{environment}'
This setup supports multiple configuration. The two sets of environments
that OpenStack CI users are ``production`` and ``development``. ``production``
is the default is and the environment used when nothing else is specified.
Then the configuration needs to be placed into common.yaml in
that OpenStack Infrastructure uses are ``production`` and ``development``.
``production`` is the default is and the environment used when nothing else
is specified. Then the configuration needs to be placed into common.yaml in
``/etc/puppet/hieradata/production`` and ``/etc/puppet/hieradata/development``.
The values are simple key-value pairs in yaml format.

View File

@ -6,13 +6,13 @@ Overview
Much of the OpenStack project infrastructure is deployed and managed using
puppet.
The OpenStack CI team manage a number of custom puppet modules outlined in this
document.
The OpenStack Infrastructure team manages a number of custom puppet modules
outlined in this document.
Doc Server
----------
The doc_server module configures nginx [3]_ to serve the documentation for
The doc_server module configures apache to serve the documentation for
several specified OpenStack projects. At the moment to add a site to this
you need to edit ``modules/doc_server/manifests/init.pp`` and add a line as
follows:
@ -31,13 +31,13 @@ Lodgeit
The lodgeit module installs and configures lodgeit [1]_ on required servers to
be used as paste installations. For OpenStack we use
`a fork <https://github.com/openstack-ci/lodgeit>`_ of this which is based on
`a fork <https://github.com/openstack-infra/lodgeit>`_ of this which is based on
one with bugfixes maintained by
`dcolish <https://bitbucket.org/dcolish/lodgeit-main>`_ but adds back missing
anti-spam features required by Openstack.
Puppet will configure lodgeit to use drizzle [2]_ as a database backend,
nginx [3]_ as a front-end proxy and upstart scripts to run the lodgeit
apache as a front-end proxy and upstart scripts to run the lodgeit
instances. It will store and maintain local branch of the the mercurial
repository for lodgeit in ``/tmp/lodgeit-main``.
@ -92,7 +92,7 @@ The planet module installs Planet Venus [4]_ along with required dependancies
on a server. It also configures specified planets based on options given.
Planet Venus works by having a cron job which creates static files. In this
module the static files are served using nginx [3]_.
module the static files are served using apache.
To use this module you need to add something similar to the following into the
main ``site.pp`` manifest:
@ -125,9 +125,10 @@ Meetbot
The meetbot module installs and configures meetbot [5]_ on a server. The
meetbot version installed by this module is pulled from the
`Openstack CI fork <https://github.com/openstack-ci/meetbot/>`_ of the project.
`OpenStack Infrastructure fork <https://github.com/openstack-infra/meetbot/>`_
of the project.
It also configures nginix [3]_ to be used for accessing the public IRC logs of
It also configures apache to be used for accessing the public IRC logs of
the meetings.
To use this module simply add a section to the site manifest as follows:
@ -271,7 +272,7 @@ in the ``JS`` box enter
Etherpad Lite
-------------
This Puppet module installs Etherpad Lite [6]_ and its dependencies (including
This Puppet module installs Etherpad Lite [3]_ and its dependencies (including
node.js). This Puppet module also configures Etherpad Lite to be started at
boot with Nginx running in front of it as a reverse proxy and MySQL running as
the database backend.
@ -381,7 +382,6 @@ the ``etherpad-lite`` DB daily and keep a rotation of 30 days of backups.
.. rubric:: Footnotes
.. [1] `Lodgeit homepage <http://www.pocoo.org/projects/lodgeit/>`_
.. [2] `Drizzle homepage <http://www.drizzle.org/>`_
.. [3] `nginx homepage <http://nginx.org/en/>`_
.. [3] `Etherpad Lite homepage <https://github.com/Pita/etherpad-lite>`_
.. [4] `Planet Venus homepage <http://intertwingly.net/code/venus/docs/index.html>`_
.. [5] `Meetbot homepage <http://wiki.debian.org/MeetBot>`_
.. [6] `Etherpad Lite homepage <https://github.com/Pita/etherpad-lite>`_

View File

@ -40,13 +40,13 @@ Create a new StackForge Project with Puppet
===========================================
OpenStack uses Puppet and a management script to create Gerrit projects
with simple changes to the openstack-ci-puppet repository. To start make
sure you have cloned the openstack-ci-puppet repository
``git clone https://github.com/openstack/openstack-ci-puppet``.
with simple changes to the openstack-infra/config repository. To start make
sure you have cloned the openstack-infra/config repository
``git clone https://github.com/openstack-infra/config``.
First you need to add your StackForge project to the master project
list. Edit
``openstack-ci-puppet/modules/openstack_project/templates/review.projects.yaml.erb``
``openstack-infra/config/modules/openstack_project/templates/review.projects.yaml.erb``
and add a new section for your project at the end of the file. It should
look something like::
@ -62,7 +62,7 @@ are optional, but you must have an acl_config. Note that the current
tools assume that the upstream repo will have a master branch.
The next step is to add a Gerrit ACL config file. Edit
``openstack-ci-puppet/modules/openstack_project/files/gerrit/acls/stackforge/project-name.config``
``openstack-infra/config/modules/openstack_project/files/gerrit/acls/stackforge/project-name.config``
and make it look like::
[access "refs/heads/*"]
@ -85,12 +85,12 @@ these additional tools.
Add Jenkins Jobs to StackForge Projects
=======================================
In the same openstack-ci-puppet repository (and in the same change if
In the same openstack-infra/config repository (and in the same change if
you like) we need to edit two additional files to setup Jenkins jobs
and Zuul for the new StackForge project.
Edit
``openstack-ci-puppet/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml``
``openstack-infra/config/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml``
and add a new section for your project at the end of the file. It should
look something like::
@ -124,7 +124,7 @@ python 2.6 and 2.7 unittests, and coverage) your entry in
Now that we have a Jenkins job we need to tell Zuul to run that job when
appropriate. Edit
``openstack-ci-puppet/modules/openstack_project/files/zuul/layout.yaml``
``openstack-infra/config/modules/openstack_project/files/zuul/layout.yaml``
and add a new section for your project at the end of the file. It should
look something like::
@ -160,7 +160,7 @@ look like this instead::
publish:
- project-name-docs
That concludes the bare minimum openstack-ci-puppet changes necessary to
That concludes the bare minimum openstack-infra/config changes necessary to
add a project to StackForge. You can commit these changes and submit
them to review.openstack.org at this point, or you can wait a little
longer and add your project to GerritBot first.
@ -170,7 +170,7 @@ Configure StackForge Project to use GerritBot
To have GerritBot send Gerrit events for your project to a Freenode IRC
channel edit
``openstack-ci-puppet/modules/gerritbot/files/gerritbot_channel_config.yaml``.
``openstack-infra/config/modules/gerritbot/files/gerritbot_channel_config.yaml``.
If you want to configure GerritBot to leave alerts in a channel
GerritBot has always joined just add your project to the project list
for that channel::

View File

@ -3,8 +3,8 @@
Infrastructure Systems
######################
The OpenStack CI team maintains a number of systems that are critical
to the operation of the OpenStack project, such as gerrit, jenkins,
The OpenStack Infrastructure team maintains a number of systems that are
critical to the operation of the OpenStack project, such as gerrit, jenkins,
mailman, meetbot, etherpad, paste, and others.
Additionally the team maintains the project sites on Launchpad and
@ -14,8 +14,8 @@ continued and secure operation of the project.
SSH Access
**********
For any of the systems managed by the CI team, the following practices
must be observed for SSH access:
For any of the systems managed by the OpenStack Infrastructure team, the
following practices must be observed for SSH access:
* SSH access is only permitted with SSH public/private key
authentication.
@ -40,9 +40,9 @@ must be observed for SSH access:
the confirmation constraint ('ssh-add -c').
* The number of SSH keys that are configured to permit access to
OpenStack machines should be kept to a minimum.
* OpenStack CI machines must use puppet to centrally manage and
* OpenStack Infrastructure machines must use puppet to centrally manage and
configure user accounts, and the SSH authorized_keys files from the
openstack-ci-puppet repository.
openstack-infra/config repository.
* SSH keys should be periodically rotated (at least once per year).
During rotation, a new key can be added to puppet for a time, and
then the old one removed. Be sure to run puppet on the backup
@ -59,7 +59,7 @@ project infrastructure servers should be carried out entirely through
puppet so that anyone can contribute.
The Git repository with the puppet configuration may be cloned from
https://github.com/openstack/openstack-ci-puppet and changes submitted
https://github.com/openstack-infra/config and changes submitted
with `git-review`.
In order to ensure that it is easy for both the OpenStack project as

View File

@ -84,12 +84,14 @@ To request a sevice acconut for your system you first need to create a new
account in LaunchPad. This account needs to be joined to the
`OpenStack Team <https://launchpad.net/~openstack>`_ or one of the related teams
so that Gerrit can pick it up. You can then contact the
OpenStack CI Admins via `email <mailto:openstack-ci-admins@lists.launchpad.net>`_
OpenStack Infrastructure Team via
`email <mailto:openstack-infra@lists.openstack.org>`_
or the #openstack-infra IRC channel. We will set things up on Gerrit to
receive your system's votes.
Feel free to contact the CI team to arrange setting up a dedicated user so your
system can post reviews up using a system name rather than your user name.
Feel free to contact the Infrastructure team to arrange setting up a dedicated
user so your system can post reviews up using a system name rather than your
user name.
The Jenkins Gerrit Trigger Plugin Way
-------------------------------------

View File

@ -14,7 +14,7 @@ dpkg -i $puppet_deb
apt-get update
apt-get install -y puppet git rubygems
git clone https://github.com/openstack/openstack-ci-puppet
bash openstack-ci-puppet/install_modules.sh
git clone https://github.com/openstack-infra/config
bash config/install_modules.sh
puppet apply --modulepath=`pwd`/openstack-ci-puppet/modules:/etc/puppet/modules -e 'node default {class { "openstack_project::bare_slave": install_users => false }}'
puppet apply --modulepath=`pwd`/config/modules:/etc/puppet/modules -e 'node default {class { "openstack_project::bare_slave": install_users => false }}'

View File

@ -1,5 +1,5 @@
# This file is managed by puppet.
# https://github.com/openstack/openstack-ci-puppet
# https://github.com/openstack-infra/config
[gerrit]
basePath = git

View File

@ -1,5 +1,5 @@
# This file is managed by puppet.
# https://github.com/openstack/openstack-ci-puppet
# https://github.com/openstack-infra/config
<% if replicate_github -%>
[remote "github"]

View File

@ -4,22 +4,21 @@ openstack-infra:
- change-merged
- x-vrif-minus-2
projects:
- openstack/openstack-ci-puppet
- openstack-ci/config
- openstack-ci/devstack-gate
- openstack-ci/gerrit
- openstack-ci/gerrit-trigger-plugin
- openstack-ci/gerrit-verification-status-plugin
- openstack-ci/gerritbot
- openstack-ci/gerritlib
- openstack-ci/git-review
- openstack-ci/jeepyb
- openstack-ci/jenkins-job-builder
- openstack-ci/lodgeit
- openstack-ci/meetbot
- openstack-ci/nose-html-output
- openstack-ci/pypi-mirror
- openstack-ci/zuul
- openstack-infra/config
- openstack-infra/devstack-gate
- openstack-infra/gerrit
- openstack-infra/gerrit-trigger-plugin
- openstack-infra/gerrit-verification-status-plugin
- openstack-infra/gerritbot
- openstack-infra/gerritlib
- openstack-infra/git-review
- openstack-infra/jeepyb
- openstack-infra/jenkins-job-builder
- openstack-infra/lodgeit
- openstack-infra/meetbot
- openstack-infra/nose-html-output
- openstack-infra/pypi-mirror
- openstack-infra/zuul
branches:
- master

View File

@ -1,7 +1,7 @@
# == Class: jeepyb
#
class jeepyb (
$git_source_repo = 'https://github.com/openstack-ci/jeepyb.git',
$git_source_repo = 'https://github.com/openstack-infra/jeepyb.git',
) {
include mysql::python

View File

@ -24,7 +24,7 @@ class jenkins::job_builder (
ensure => latest,
provider => git,
revision => 'master',
source => 'https://github.com/openstack-ci/jenkins-job-builder.git',
source => 'https://github.com/openstack-infra/jenkins-job-builder.git',
}
exec { 'install_jenkins_job_builder':

View File

@ -221,22 +221,22 @@ class Sync(object):
projects = [
'heat-api/heat',
'heat-api/python-heatclient',
'openstack-ci/devstack-gate',
'openstack-ci/gerrit',
'openstack-ci/gerrit-verification-status-plugin',
'openstack-ci/gerritbot',
'openstack-ci/gerritlib',
'openstack-ci/git-review',
'openstack-ci/jeepyb',
'openstack-ci/jenkins-job-builder',
'openstack-ci/lodgeit',
'openstack-ci/meetbot',
'openstack-ci/nose-html-output',
'openstack-ci/puppet-apparmor',
'openstack-ci/puppet-dashboard',
'openstack-ci/puppet-vcsrepo',
'openstack-ci/pypi-mirror',
'openstack-ci/zuul',
'openstack-infra/devstack-gate',
'openstack-infra/gerrit',
'openstack-infra/gerrit-verification-status-plugin',
'openstack-infra/gerritbot',
'openstack-infra/gerritlib',
'openstack-infra/git-review',
'openstack-infra/jeepyb',
'openstack-infra/jenkins-job-builder',
'openstack-infra/lodgeit',
'openstack-infra/meetbot',
'openstack-infra/nose-html-output',
'openstack-infra/puppet-apparmor',
'openstack-infra/puppet-dashboard',
'openstack-infra/puppet-vcsrepo',
'openstack-infra/pypi-mirror',
'openstack-infra/zuul',
'openstack-dev/devstack',
'openstack-dev/openstack-nose',
'openstack-dev/openstack-qa',
@ -256,8 +256,7 @@ class Sync(object):
'openstack/nova',
'openstack/object-api',
'openstack/openstack-chef',
'openstack/openstack-ci',
'openstack/openstack-ci-puppet',
'openstack-infra/config',
'openstack/openstack-manuals',
'openstack/openstack-planet',
'openstack/openstack-puppet',

View File

@ -47,7 +47,7 @@ class lodgeit {
vcsrepo { '/tmp/lodgeit-main':
ensure => latest,
provider => git,
source => 'https://github.com/openstack-ci/lodgeit.git',
source => 'https://github.com/openstack-infra/lodgeit.git',
}
# create initial git DB backup location

View File

@ -4,7 +4,7 @@ class meetbot {
vcsrepo { '/opt/meetbot':
ensure => latest,
provider => git,
source => 'https://github.com/openstack-ci/meetbot.git',
source => 'https://github.com/openstack-infra/meetbot.git',
}
user { 'meetbot':

View File

@ -1 +1 @@
This file stores config files specific to the OpenStack CI project.
This file stores config files specific to the OpenStack Infrastructure project.

View File

@ -1,10 +0,0 @@
[access "refs/heads/*"]
label-Code-Review = -2..+2 group openstack-ci-admins
label-Approved = +0..+1 group openstack-ci-admins
[project]
state = active
[receive]
requireChangeId = true
requireContributorAgreement = true
[submit]
mergeContent = true

View File

@ -7,8 +7,8 @@
<p>If you would like to make changes to this job, please see:
<a href="https://github.com/openstack/openstack-ci-puppet">
https://github.com/openstack/openstack-ci-puppet
<a href="https://github.com/openstack-infra/config">
https://github.com/openstack-infra/config
</a>
In modules/openstack_project/files/jenkins_job_builder/config

View File

@ -109,7 +109,7 @@
- shell: |
#!/bin/bash -xe
if [[ ! -e devstack-gate ]]; then
git clone http://zuul.openstack.org/p/openstack-ci/devstack-gate
git clone http://zuul.openstack.org/p/openstack-infra/devstack-gate
else
cd devstack-gate
git remote update

View File

@ -1,5 +1,5 @@
- job:
name: gate-ci-puppet-syntax
name: gate-openstack-infra-config-syntax
node: precise
triggers:
@ -15,7 +15,7 @@
- job:
name: gate-ci-puppet-lint
name: gate-openstack-infra-config-lint
node: precise
triggers:
@ -30,7 +30,7 @@
- job:
name: ci-puppet-compare-xml
name: openstack-infra-config-compare-xml
node: precise
triggers:

View File

@ -45,7 +45,7 @@
- project:
name: devstack-gate
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -73,7 +73,7 @@
- project:
name: gerrit-verification-status-plugin
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -82,7 +82,7 @@
- project:
name: gerrit
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -91,7 +91,7 @@
- project:
name: gerritbot
github-org: openstack-ci
github-org: openstack-infra
node: precise
doc-publisher-site: ci.openstack.org
tarball-publisher-site: 173.203.107.207
@ -107,7 +107,7 @@
- project:
name: gerritlib
github-org: openstack-ci
github-org: openstack-infra
node: precise
doc-publisher-site: ci.openstack.org
tarball-publisher-site: nova.openstack.org
@ -122,7 +122,7 @@
- project:
name: jeepyb
github-org: openstack-ci
github-org: openstack-infra
node: precise
doc-publisher-site: ci.openstack.org
tarball-publisher-site: nova.openstack.org
@ -137,7 +137,7 @@
- project:
name: git-review
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
- gate-{name}-merge
@ -221,7 +221,7 @@
- project:
name: jenkins-job-builder
github-org: openstack-ci
github-org: openstack-infra
node: precise
tarball-publisher-site: nova.openstack.org
@ -252,7 +252,7 @@
- project:
name: lodgeit
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -261,7 +261,7 @@
- project:
name: meetbot
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -302,7 +302,7 @@
- project:
name: nose-html-output
github-org: openstack-ci
github-org: openstack-infra
node: precise
doc-publisher-site: ci.openstack.org
tarball-publisher-site: 173.203.107.207
@ -332,8 +332,8 @@
- project:
name: ci-puppet
github-org: openstack
name: config
github-org: openstack-infra
node: precise
jobs:
@ -419,7 +419,7 @@
- project:
name: puppet-apparmor
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -428,7 +428,7 @@
- project:
name: puppet-dashboard
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -437,7 +437,7 @@
- project:
name: puppet-vcsrepo
github-org: openstack-ci
github-org: openstack-infra
node: precise
jobs:
@ -446,7 +446,7 @@
- project:
name: pypi-mirror
github-org: openstack-ci
github-org: openstack-infra
<