Clark Boylan eeac5467c3 Set up robots.txt on lists servers ... This sets up a robots.txt on our lists servers. To start this file prevents SEMrush bot from indexing our lists as that has been causing lists.openstack.org to OOM with many listinfo processes started by Apache. We've avoided this OOM by manually configuring this robots.txt. Other things we have ruled out are bup and input email causes qrunner's to grow unexpectedly large. Fairly confident this bot is the trigger. Note this fixes testing by adding 'hieradata' to set listpassword var. Depends-On: https://review.opendev.org/724389 Change-Id: Id4f6739a8cf6a01f9796fa54c86ba1af3e31fecf		2020-04-29 17:48:13 +00:00
..
html-templates-en	Update infra-manual location	2020-03-20 22:03:09 +01:00
robots.txt	Set up robots.txt on lists servers	2020-04-29 17:48:13 +00:00