In I4e5f803b9d4fb6c2351cf151a085b93a7fd20f60 I put the wrong thing in
the zuul.openstack.org config; for that site we want to cache
/api/status; not the tenant path.
Change-Id: Iffbd870aeff496b9c259206f866af3a90a4349db
mod_mem_cache was removed in Apache 2.4 so all the bits of
configuration gated by the IfModule are currently irrelevant.
The replacement is socache, the in-memory version is "shmcb" (can also
hook up to memcache, etc.). Enable the socache module, and switch the
cache matching parts to use socache and then fall-back to disk cache
(this is what it says this will do in the manual [1])
The other part of this is to turn the CacheQuickHandler off. The
manual says about this [2]
In the default enabled configuration, the cache operates within the
quick handler phase. This phase short circuits the majority of
server processing, and represents the most performant mode of
operation for a typical server. The cache bolts onto the front of
the server, and the majority of server processing is avoided.
I won't claim to fully understand how our mod_rewrite rules and
mod_proxy all hang together with phases and what-not. But emperically
with this turned on (default) we do not seem to get any caching on the
tenant status pages, and with it turned off we do.
I've deliberately removed IfModule gating as well. This actually hid
the problem and made it much more difficult to diagnose; it is much
better if these directives just fail to start Apache if we do not have
the modules we expect to have.
[1] https://httpd.apache.org/docs/2.4/mod/mod_cache_socache.html
[2] https://httpd.apache.org/docs/2.4/mod/mod_cache.html#cachequickhandler
Change-Id: I4e5f803b9d4fb6c2351cf151a085b93a7fd20f60
These two values overwrite each other, move into common configuration.
The "cache-status" is a verbose string, so quote it.
Change-Id: I3cc4627de3d6a0de1adcfed6b424fc3ed0099245
We need a regex to match the url path for zuul statuses. Our existing
setup assumed this would work in a CacheEnable directive but it seems
that it does not. Move this into a LocationMatch which explicitly
supports regexes.
Change-Id: I9df06d2af31ce6550e537f4594640487cca1d735
We attempt to cache things served by zuul-web in our apache proxy. This
is to reduce the load on the zuul-web process which has to query
gearman, the sql database, and eventuall the zookeeper database to
produce its responses.
Things are currently operating slowly and it isn't clear if we're
caching properly. To check that better update our logging format to
record cache hits and misses. Also drop an unnecessary .* in the
CacheEnable url-strings for /static/ as it is unclear if the .* is
treated as a regex here.
Change-Id: Ib57c085fa15365b89b3276e037339dbeddb094e3
This should only land after we've launched a new nb03.opendev.org
running with the new nodepool arm64 docker image. Once that happens and
we are happy with how it is running we can safely stop managing the
existing nb03.openstack.org server with puppet.
Change-Id: I8d224f9775bd461b43a2631897babd9e351ab6ae
We install docker-compose from pypi in order to get newer features
(particularly useful for gerrit). On x86 all the deps for this have
wheels and we don't need build deps but on arm64 wheels don't exist for
things like cffi. Add build-essential, python3-dev, libffi-dev, and
libssl-dev to ensure we can build the necessary deps to install
docker-compose on arm64.
Change-Id: Id9c61dc904d34d2f7cbe17c70ad736a9562bb923
This server is going to be our new arm64 nodepool-builder running on the
new arm64 docker images for nodepool.
Depends-On: https://review.opendev.org/750037
Change-Id: I3b46ff901eb92c7f09b79c22441c3f80bc6f9d15
Update the contents for https://opendev.org/ default index page,
to be less of a transition explanation for existing OpenStack
infrastructure users, and more of an introduction for new users.
In particular:
- prominently display links to main opendev services
- explain what Opendev is and why we do it
- detail differences to expect from other development platforms
Change-Id: Idaefb41590df24e649a4fd5225cc1078f2903696
These don't make any sense in the top-level these days.
Once upon a time we used to use these as node scripts to bring up
testing nodes (I think). The important thing is they're not used now.
Change-Id: Iffa6c6bee647f1a242e9e71241d829c813f2a3e7
Modules are collected on bridge and then synchronized to remote hosts
where puppet is run. This is done to ensure an atomic run of puppet
across affected hosts.
These modules are described in modules.env and cloned by
install_modules.sh. Currently this is done in install-ansible, but
after some recent refactoring
(I3b1cea5a25974f56ea9202e252af7b8420f4adc9) the best home for it
appears to now be in puppet-setup-ansible; just before the script is
run.
Change-Id: I4b1d709d7037e2851d73be4bc7a202f52858ad4f
It turns out you can't use "run_once" with the "free" strategy in
Ansible. It actually warns you about this, if you're looking in the
right place.
The existing run-puppet role calls two things with "run_once:", both
delegated to localhost -- cloning the ansible-role-puppet repo (so we
can include_role: puppet) and installing the puppet modules (via
install-ansible-roles role), which are copied from bridge to the
remote side and run by ansible-role-puppet.
With remote_puppet_else.yaml we are running all the puppet hosts at
once with the "free" strategy. This means that these two tasks, both
delegated to localhost (bridge) are actually running for every host.
install-ansible-roles does a git clone, and thus we often see one of
the clones bailing out with a git locking error, because the other
host is running similtaneously.
I8585a1af2dcc294c0e61fc45d9febb044e42151d tried to stop this with
"run_once:" -- but as noted because it's running under the "free"
strategy this is silently ignored.
To get around this, split out the two copying steps into a new role
"puppet-setup". To maintain the namespace, the "run-puppet" module is
renamed to "puppet-run". Before each call of (now) "puppet-run", make
sure we run "puppet-setup" just on localhost.
Remove the run_once and delegation on "install-ansible-roles"; because
this is now called from the playbook with localhost context.
Change-Id: I3b1cea5a25974f56ea9202e252af7b8420f4adc9
We create a virtualenv to install ansible in which then runs puppet for
us in our puppet apply jobs. This is pulling in setuptools 50 which then
fails due to the problems setuptools 50 has with older pythons. Address
this by pinning back to setuptools <50.
Change-Id: I02ea466319f7cd90f73972bf5a99876d14823ac1
Limestone has updated their self signed cert and in order to properly
verify it we need to update the cert material to check against itself.
Maybe we should confirm with logan- that the new cert material looks
correct before landing this just to be sure we're trusting the correct
thing.
Change-Id: Id528716aecb45ffb263850f697c5fb22db3b7969
I forgot in I5b7106e2263010ff353e8a1de43e73b0c0ec57e1 this is a new
mirror, which needs the LE bits setup.
Change-Id: I3109573b2b03453049a265a829445f88f8a87557
It seems acme.sh might have been rewriting this with quotes, and has
now stopped doing that. Fix the match.
Change-Id: I3c363c498580b79a1a9ed07da6ed3ac72807383b
Collect the tox logs from the testinfra run on bridge.openstack.org.
The dependent change helps if we have errors installing things into
tox, and this change lets us see the results.
Depends-On: https://review.opendev.org/747325
Change-Id: Id3c39d4287d7dc9705890c73a230b1935d349b9f
We have several tox environments that don't really share the same
dependencies. Currently we put everything into test-requirements.txt,
meaning the linter environment gets testinfra and testinfra
environment gets things like ansible-lint it's not using.
Apart from being a bit faster not installing things, this is helpful
for restoring the ansible -devel test. In that case, we have ansible
as a project dependency; when installing from git, ansible now
installs itself as "ansible-base" ("ansible" is the released pypi
package that incorporates collections). So if we install ansible-lint
in the testinfra environment, this pulls in ansible ... and then it
breaks when tox wants to install our Zuul clone of upstream ansible
(you can't have ansible and ansible-base together).
Change-Id: I5f3fc93e6ddd611f319fa269e122ed576c9cb378
We have having constant issues with the bionic arm64 mirror shutting
itself off. Before we go too far down the path of debugging what
appears to be a kernel oops issue, let's rebuild it as focal.
Update the sources list in the base. Update the testing to use a
focal node.
Change-Id: I5b7106e2263010ff353e8a1de43e73b0c0ec57e1
In our beaker rspec testing we ssh into localhost pretending it is a
managed VM because that is how all the config management testing tools
want to work... This is has run into problems with new format ssh keys
which zuul provides. If such a key is present we convert it to PEM
othrewise we generate our own.
Also add ensure-virtualenv to the job as we appear to need it to run
these tests properly.
Change-Id: Ibb6080b5a321a6955866ef9b847c4d00da17f427
The pubmirror[12].math.uh.edu mirrors of Fedora 31 updates for
x86_64 have been sitting stale for several days with a corrupt
index, causing jobs which access our copy of this from our mirror
network to fail. Instead mirror Fedora releases/updates from
mirrors.mit.edu which seems to be updating just fine currently. We
can switch this back if/when the situation with the uh.edu mirrors
is resolved.
We're continuing to mirror EPEL and Fedora Atomic from
pubmirror[12].math.uh.edu for now, as we've had no reports of EPEL
problems on our mirrors (yet anyway), and it's hard to find any
other rsync mirrors of Atomic.
Change-Id: Iefd02602e2f2b39c4b72dc4d95ac62993ca65cdd
