Tools used by OpenStack Documentation
Nie możesz wybrać więcej, niż 25 tematów Tematy muszą się zaczynać od litery lub cyfry, mogą zawierać myślniki ('-') i mogą mieć do 35 znaków.

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778
  1. =================
  2. Sitemap Generator
  3. =================
  4. This script crawls all available sites on https://docs.openstack.org and
  5. extracts all URLs. Based on the URLs the script generates a sitemap for search
  6. engines according to the `sitemaps protocol
  7. <http://www.sitemaps.org/protocol.html>`_.
  8. Installation
  9. ~~~~~~~~~~~~
  10. To install the needed modules you can use pip or the package management system
  11. included in your distribution. When using the package management system maybe
  12. the name of the packages differ. Installation in a virtual environment is
  13. recommended.
  14. .. code-block:: console
  15. $ virtualenv venv
  16. $ . venv/bin/activate
  17. $ pip install Scrapy
  18. When using pip, you may also need to install some development packages. For
  19. example, on Ubuntu 16.04 install the following packages:
  20. .. code-block:: console
  21. $ sudo apt install gcc libssl-dev python-dev python-virtualenv
  22. Usage
  23. ~~~~~
  24. To generate a new sitemap file, change into your local clone of the
  25. ``openstack/openstack-doc-tools`` repository and run the following commands:
  26. .. code-block:: console
  27. $ cd sitemap
  28. $ scrapy crawl sitemap
  29. The script takes several minutes to crawl all available
  30. sites on https://docs.openstack.org. The result is available in the
  31. ``sitemap_docs.openstack.org.xml`` file.
  32. Options
  33. ~~~~~~~
  34. domain=URL
  35. Sets the ``domain`` to crawl. Default is ``docs.openstack.org``.
  36. For example, to crawl https://developer.openstack.org use the following
  37. command:
  38. .. code-block:: console
  39. $ scrapy crawl sitemap -a domain=developer.openstack.org
  40. The result is available in the ``sitemap_developer.openstack.org.xml`` file.
  41. urls=URL
  42. You can define a set of additional start URLs using the ``urls`` attribute.
  43. Separate multiple URLs with ``,``.
  44. For example:
  45. .. code-block:: console
  46. $ scrapy crawl sitemap -a domain=developer.openstack.org -a urls="https://developer.openstack.org/de/api-guide/quick-start/"
  47. LOG_FILE=FILE
  48. Write log messages to the specified file.
  49. For example, to write to ``scrapy.log``:
  50. .. code-block:: console
  51. $ scrapy crawl sitemap -s LOG_FILE=scrapy.log