6 Commits (ea541c2e7b6e23edb157911978433fa7d740c3ab)

Author SHA1 Message Date
Christian Berendt 8f05ac1b0a [sitemap] respect the robots.txt policies 6 years ago
Christian Berendt b7ed3589d3 [sitemap] remove deprecated URL from settings 6 years ago
Christian Berendt 02d0060144 [sitemap] increase the DOWNLOAD_WARNSIZE 6 years ago
Christian Berendt 8a2df7b6ef [sitemap] resolve several ScrapyDeprecationWarning exceptions 6 years ago
Christian Berendt 31c25891c1 script to generate the sitemap.xml for docs.openstack.org 7 years ago