155 Commits

Author SHA1 Message Date
Zuul
736b14d1b0 Merge "Small repo template cleanups in Gitea" 2020-06-22 16:14:35 +00:00
Zuul
6ca6ed8863 Merge "Update to gitea 1.12.0" 2020-06-22 16:14:08 +00:00
Clark Boylan
1770a25c8a Small repo template cleanups in Gitea
This improves the comments around why we've cleaned up some repo
functionality and also removes the activity button. The activity data
gitea is aware of is not a complete picture and is likely to be
misleading. Avoid misleading people by removing that button.

Change-Id: If7546f95421efdd897f2949e4f89a08d53289ce8
2020-06-19 15:26:06 -07:00
Clark Boylan
4f405f13f5 Update to gitea 1.12.0
Gitea 1.12.0 brings much improved git repo performance. Specifically it
caches the most recent commits helping the web views render much more
quickly.

This bumps our golang base image to 1.14-buster as gitea has changed
their default golang verison to 1.14. We also add gnupg to the package
install list as the upstream images did this.

In the templates I've tried to comment out rather than remove content
that we don't want exposed. The reason for this is it makes resolving
future template updates simpler.

For v1.12.0 release notes see:
https://github.com/go-gitea/gitea/blob/v1.12.0/CHANGELOG.md

Change-Id: I8fd6587c8962554023d878266055a2bd9d2499f9
2020-06-19 09:09:16 -07:00
Zuul
3f15016351 Merge "Be explicit about using python3 in docker images" 2020-06-15 23:37:17 +00:00
Zuul
c8f8edd478 Merge "uwsgi-base: drop packages.txt" 2020-06-15 21:29:41 +00:00
Mohammed Naser
7cd63aaded uwsgi-base: drop packages.txt
When we're building an image that is based off uwsgi-base, the
packages.txt file remains from the previous run which means that it only
install uWSGI again and not the package we're trying to build.

Change-Id: Ic71c47d53566b4ea79e03d19fd355c5c71a62802
2020-06-14 13:30:16 -04:00
Clark Boylan
421a71e145 Be explicit about using python3 in docker images
Our docker python-base image is a python3 image where python and python3
point to the same version of python which happens to be a 3.x depending
on the specific base image.

One of the things that came up at the PTG is that we'd like to start
identifying where we use python2 in order to port to python3. That is
made a bit more difficult when we use `python` assuming it is `python3`
as we do in our docker images.

Update the images to use `python3` to make this a lot more explicit and
clear that we don't need to port these items.

Change-Id: I54e8f128f0cb8fbbdf54e3748384470262bef3a9
2020-06-09 09:30:56 -07:00
James E. Blair
6a53d47050 Exclude some regex matches in jitsi-meet web
If we match these explicit paths, don't allow the regex matches to
run.  Otherwise, we might redirect "/http-bind" to "/" which doesn't
work.

Also, restore the default meet.conf to the upstream; we're not
intending to carry any local changes to the rootfs.

Change-Id: I119d8c223291d79bd5fe0977264dec4bae126280
2020-05-13 07:40:18 -07:00
Jeremy Stanley
a48ac23520 Support hyphens and underscores for meetpad rooms
Many people put hyphens (-) and underscores (_) in their pad names,
and Jitsi-Meet seems to support them in its room names too, so add
them to our rewrite regex.

Change-Id: I819b8d56bf839cf743319e9cd823440c7d910001
2020-05-08 16:11:56 +00:00
Zuul
643b1d0541 Merge "python-builder: drop # from line" 2020-05-07 17:10:48 +00:00
Zuul
b2df0af08c Merge "Stop using jemalloc in python base image" 2020-05-05 18:50:04 +00:00
Clark Boylan
e9854d8bdc Stop using jemalloc in python base image
Initial testing seems to show that jemalloc is a probable source of our
memory leaks with zuul-scheduler and zuul-web under python3.7 and
python3.8. While this was desireable under python3.5 with zuul-executor
it doesn't seem like we want to enable this universally. For this reason
remove jemalloc from the base image. It can be added into images in
specific places if desireable but isn't a good default.

Change-Id: I671eef4ef27fdde161b52cf0a4187fac10ac8c4e
2020-05-04 16:31:36 -07:00
Monty Taylor
cee7903827 Add constraints support to python-builder
If someone drops an upper-constraints.txt into the source dir,
add it to the pip interactions.

Change-Id: I6b3c2a178d6fd9ee19918657c809ebf100a7724b
2020-05-04 15:21:33 -05:00
Mohammed Naser
254a4eb9f9 python-builder: drop # from line
If the extras includes a '#' inside of it, it means that it is
probably followed up with a comment such as license for that package and
it will fail to install because pip will try to use that literal #.

Change-Id: I31c5ffe3dda3ced4b6a412cc955c73321d4af507
2020-05-04 15:17:12 -04:00
Andreas Jaeger
90fa9170dd Fix py3 problem in accessbot.py
Change-Id: I4e3b3683b33b600cc4c1e4f09ba02c59e6f5cbb9
Fix:
TypeError: unsupported operand type(s) for +: 'dict_items' and 'dict_items'
2020-05-02 22:11:21 +02:00
Andreas Jaeger
26f1dae81b Fix accessbot/Dockerfile
The command is broken (extra ", misses redirection) and duplicates
content from accessbot.sh. Call accessbot.sh directly and use that one
instead.

Change-Id: Ieb530ef27e5995f2848a3c23a6c04a0717716e14
2020-05-02 18:02:06 +02:00
Zuul
1b2d113c0f Merge "Split eavesdrop into its own playbook" 2020-04-24 15:02:34 +00:00
Monty Taylor
9fd2135a46 Split eavesdrop into its own playbook
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.

Add the ability to override the keys for the zuul user.

Remove openstack_project::server, it doesn't do anything.

Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.

Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
2020-04-23 14:34:28 -05:00
Zuul
2dcc8fc366 Merge "Add a uwsgi-base container image" 2020-04-23 16:14:20 +00:00
Zuul
b568391a4b Merge "Allow passing an arbitrary package list to assemble" 2020-04-23 16:07:41 +00:00
Zuul
e79de4587a Merge "Upgrade to gitea 1.11.4" 2020-04-23 15:37:44 +00:00
Monty Taylor
08a807e188 Add a uwsgi-base container image
For things using the builder/base pattern providing uwsgi services.

Change-Id: Iaf6d31a3d119f6b7e87b54cda1969a9994110dad
2020-04-23 13:11:44 +00:00
Monty Taylor
f3304c38d5 Allow passing an arbitrary package list to assemble
This lets you build an image to install a package or set of packages
without having to build a fake pbr install.

Change-Id: I6c171881283b89a14a532f31aedb33e223a0ea1f
2020-04-22 19:15:51 -05:00
Monty Taylor
ba3a0eb775 Allow requesting a list of extras to install
We produce lists of extras requirements to install, but installing
them takes an extra step. Allow it to be done all at once by accepting
names of extras on the install-from-bindep command line.

Change-Id: Ie75b9a668569c759f78b70617e311ed6f025039a
2020-04-22 14:39:55 -05:00
Monty Taylor
d1c49ff89e Upgrade to gitea 1.11.4
1.11.4 has been released. Nothing much of note, other than an
update to go-git v5.

Change-Id: I7c097b8033116800bbe6d55258f82dbedbf7248f
2020-04-15 08:54:37 -05:00
Zuul
d969cff10d Merge "Update link to etherpad from OpenDev" 2020-04-13 23:17:49 +00:00
Monty Taylor
7657b69387 Build and use our own etherpad image
We need to install the ep_headings plugin. We've got a hack in place
to do it in prod, but it seems like every time we npm install the
plugin in ansible, we need to restart the etherpad container
for $reasons.

It's cleaner to just build an image based on upstream with ep_headings
installed.

Change-Id: Id3b31bbabbbc5f6dcfe56486e48916384292f693
2020-04-13 08:07:09 -05:00
olaph wagoner
2b3f94a650 Fix typo on website
In the 'Join Us' section of the OpenDev home page, OpenDev is
spelt with a lower case 'd'.  This commit correctly capitalizes it.

Change-Id: I8d10cdbc89d9e509fa0c688345f5f462ccb539e6
2020-04-10 13:31:00 -04:00
Monty Taylor
3d7e824ff8 Update link to etherpad from OpenDev
We can link to etherpad.opendev.org now. This is the second to last
mention of openstack.org on opendev.org.

Change-Id: Ie50cf692c5a5cec2f6a6b19d026913d61e0b95e8
2020-04-09 13:12:49 -05:00
Jeremy Stanley
aacbe7a226 Add IRC logs and ML subscribe links to opendev.org
Folks looking at the contact section of our Web site may want to
subscribe to the mailing lists we mention there, so provide handy
hyerlinks enabling them to do that. Also add a link to the IRC
channel logs (partly to balance out the bullet list).

Change-Id: Ib3a1ca47b794f84029b3aff43c7a284fc470b7f1
2020-04-07 18:35:54 +00:00
Jeremy Stanley
8641302459 Mention new mailing lists
The OpenDev community is moving its discussions off the old
openstack-infra mailing list, so make sure to refer to the correct
new address(es).

Change-Id: I558b60ea0aa3421285d46be449f04198441cf285
2020-04-06 18:19:28 +00:00
Thierry Carrez
8ea3d6c8b7 [gitea] Point to newly-split Getting Started content
We recently added a "Get started" link to Gitea, pointing to the
generic Developer's Guide. Now that the content is split between
"Getting started" and the reference "Developer's Guide", point
to Getting started content instead.

This should be a lot less intimidating for our new users.

Depends-On: https://review.opendev.org/715961
Change-Id: Ifa496bc8de1a6a39d7b52363db3ab49b8175ed9a
2020-03-30 16:41:52 +02:00
Zuul
f892f0690e Merge "jitsi-meet: open etherpad on join" 2020-03-27 14:48:18 +00:00
Zuul
bfe1af5747 Merge "Build jitsi-meet images" 2020-03-27 14:48:16 +00:00
Zuul
e71a4e8ada Merge "Remove gitea doc links" 2020-03-27 13:46:57 +00:00
Zuul
7fdc05318b Merge "Add links to dev docs on gitea" 2020-03-27 13:10:56 +00:00
Clark Boylan
8bcc9bba24 Remove gitea doc links
These render as "help" links in the UI and for most of our users will
just be an attractive nuisance since we don't manage accounts and repos
and such directly in gitea.

Change-Id: Iea4bd262dd6304bc1d423f1bf23dacc4333f506b
2020-03-26 10:16:20 -07:00
Clark Boylan
e6639c52c1 Add links to dev docs on gitea
Currently if you end up at opendev.org it isn't clear how to start
contributing code. Add a "Get Started" link to the nav bar header to
will send you to the opendev developer docs.

Note we may want to consider updating the developer docs to be a better
landing page or even create a new doc for that, but this is a start.

Change-Id: Iee43e2552c1be2e4cd46f43f050e476f140530ad
2020-03-26 10:15:39 -07:00
Zuul
2bf5de8f72 Merge "Start making 3.8 python images" 2020-03-26 13:43:27 +00:00
James E. Blair
b478ad2acf jitsi-meet: open etherpad on join
For our meetpad service, modify jitsi-meet to open the etherpad
when a user joins.

There is an upstream PR for this, but until it lands or is rejected,
this clones from a temporary github repo with the patch.

Change-Id: Idd33a68a4fe7a90c75c5988bd1fd6136b8b7db08
2020-03-25 16:37:54 -07:00
Monty Taylor
650392c700 Start making 3.8 python images
base and builder are currently pinned to 3.7, which makes it hard
to consider upgrading to python-3.8. To help with that, make
python-3.7 and python-3.8 images, but point latest as 3.7 for the
time being. Then add version-specific provides and FROM lines
so that we can start being deliberate and clear about our version
choice.

Change-Id: Ibf1d846d5c4a005547785124567ce2900e272a7a
2020-03-25 17:45:39 -05:00
James E. Blair
c79c11c2b7 Build jitsi-meet images
So that we can run jitsi-meet with local modifications, build our
own container images.  This builds the base, prosody, and web images
from the docker-jitsi-meet project.  That project has distinct
Dockerfiles for each image, but for simplicity, this change combines
them into a single multi-stage Dockerfile.  The minor stylistic
differences between the different sections are a result of that, and
are intentional in order to minimise the delta from the source material.

Again, for simplicity, this change does not publish the base image
since it is not anticipated that we will run this build often.  If we do,
we could split this back out.

The upstream images are based on pre-built debian packages hosted by
the jitsi project.  Since our goal is to modify the software, we will
need to rebuild the debian packages as well.  This adds a new builder
image that is used to build the debian packages initially.

The docker-jitsi-meet project also has Dockerfiles for several more
images, but since the immediate need is only for the "web" image (built
from the "jitsi-meet" project), we only build that image and the "prosody"
image (not strictly necessary, but it is also a product of the "jisti-meet"
repository, so it seems a good practice to build it as well).

Change-Id: Ib3177ebfe2b8732a3522a1fa101fe95586dd1e1b
2020-03-25 15:40:50 -07:00
Zuul
66e6c84e40 Merge "Disable recommends in python-base and python-builder" 2020-03-25 22:18:02 +00:00
Clark Boylan
d679a9e197 Note about gitea functionality on opendev.org
We seem to semi regularly get questions about why we don't use features
like Gitea's issue tracker and wiki. Add that to the opendev.org front
page FAQ.

Change-Id: Ie5c4602741dcdb9cc4e87b9286a8f6b8b7ed7934
2020-03-24 09:33:16 -07:00
Monty Taylor
643623ba8a Disable recommends in python-base and python-builder
We disable these on our VM images, so might as well make our
container images follow suit. Could make them a smidge smaller.
Also do it in the gerrit-base image.

Change-Id: Iba60cf5c7009d57c4910f9e4464aff9231598ad6
2020-03-23 16:06:43 -05:00
Monty Taylor
8c7b94436b Cleanup a few docker image related thigns
We don't use the bazel image anymore, so it can go away.

Also, the bouncy castle line in the gerrit base dir is bong.

Change-Id: I58842dd9adee108893c0c953c4bb8361b9117775
2020-03-21 10:21:24 -05:00
Monty Taylor
63d8f7af48 Base 2.13 image on gerrit-base
We install jeepyb and launchpadlib in gerrit-base. Those are
important. We also need to add cgi for gitweb.

The gerrit init command does two things that we don't actually
want it to do at runtime. It extracts the plugins into the
plugins dir, and it downloads the right database library.

We can extract the plugins for it during image creation, and
then we can also download the plugin it would have downloaded.

We can also download the mysql library for it:

https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.13/gerrit-pgm/src/main/resources/com/google/gerrit/pgm/init/libraries.config

Finally, we tell it to not download or expand anything during
init, because we're running in a container and next time we run
the process that dir isn't going to be there.

Our gerrit integration tests don't depend on our gerrit image builds.
Put in image depends between run-review and gerrit builds.

We also need to depend directly on opendev-buildset-registry.

Add java.security.egd setting to java invocation

This tells java to be secure.

https://stackoverflow.com/questions/58991966/what-java-security-egd-option-is-for

Add support for setting heap limit properly

The gerrit init script does this based on the value in
container.javaOptions. We could, but then we'd have to
run an entrypoint script. Instead, set the value via
the JAVA_OPTIONS env var setting based on a value from
ansible.

Finally, make gerrit-master image build non-voting

It looks like there might be a real issue, but debugging that
is not important for us at this moment.

Depends-On: https://review.opendev.org/714216
Change-Id: I01e94c10f470fb3c8ddfce7b0e201357e5050679
2020-03-20 16:37:18 -05:00
Clark Boylan
6c7f7de55f Don't run make generate for gitea
Upstream stopped running make generate in order to fix their static file
builds [0]. Our static file builds have stopped working with our bump to
1.11.3. Apply the corresponding fix to our dockerfile.

Note that we also use clean-all instead of clean to be sure we clean all
the js and css files first.

[0] 48be1889cd (diff-3254677a7917c6c01f55212f86c57fbf)

Change-Id: I40f0449ae29e185ba7082f2f5a27dc96acf58e31
2020-03-18 12:57:38 -07:00
Zuul
e3f7c8cee8 Merge "Update references to IRC channels" 2020-03-18 18:55:57 +00:00