6 Commits (ea541c2e7b6e23edb157911978433fa7d740c3ab)

Author SHA1 Message Date
  Christian Berendt 8f05ac1b0a [sitemap] respect the robots.txt policies 4 years ago
  Christian Berendt b7ed3589d3 [sitemap] remove deprecated URL from settings 4 years ago
  Christian Berendt 02d0060144 [sitemap] increase the DOWNLOAD_WARNSIZE 4 years ago
  Christian Berendt 8a2df7b6ef [sitemap] resolve several ScrapyDeprecationWarning exceptions 5 years ago
  Christian Berendt 31c25891c1 script to generate the sitemap.xml for docs.openstack.org 6 years ago