6 Commits (b9d280acce708176ab41541b638b0ccdbc3f1027)

Author SHA1 Message Date
  Christian Berendt 8f05ac1b0a [sitemap] respect the robots.txt policies 5 years ago
  Christian Berendt b7ed3589d3 [sitemap] remove deprecated URL from settings 5 years ago
  Christian Berendt 02d0060144 [sitemap] increase the DOWNLOAD_WARNSIZE 5 years ago
  Christian Berendt 8a2df7b6ef [sitemap] resolve several ScrapyDeprecationWarning exceptions 5 years ago
  Christian Berendt 31c25891c1 script to generate the sitemap.xml for docs.openstack.org 6 years ago