80 Commits

Author SHA1 Message Date
Jenkins
787b48019f Merge "Changing 'raise e' to 'raise' to prevent exception masking" 2016-09-05 08:03:27 +00:00
KATO Tomoyuki
6842c4b774 [config-ref] support cfg.PortOpt type by the generation tool
Change-Id: I99aabfadcadbadac6c8cfc552dc4f99837df5e81
Partial-Bug: 1603082
2016-08-03 11:21:47 +09:00
KATO Tomoyuki
3725c14c62 [config-ref] skip options marked as disable
The common configurations are described in the common
configurations table. The corresponding configutations
at each table are not necessary.

Change-Id: Ied2d03d1c7f71f5712fcf57d4499e127db327b7d
Implements: blueprint config-ref-common-sections
2016-07-26 04:03:42 +00:00
Matt Smith
307344f7a8 Changing 'raise e' to 'raise' to prevent exception masking
Change-Id: I5786ef33368f251815f6acbf97a15990562ffb85
2016-06-17 09:52:08 -07:00
Jenkins
10c6b0236d Merge "Allow bulleted lists in config reference" 2016-06-16 08:27:16 +00:00
Alexis Lee
f8bff99285 Allow for config option flags
Add a definition list under the help text to detail pertinent flags.
Make 'mutable' one of these flags.

Change-Id: Idc0080be25db12485b33be421c5557d78418823c
2016-06-08 22:31:24 +00:00
John Garbutt
4f0433ffce Allow bulleted lists in config reference
This is very much a hack, but its really just to see if it is possible
to convert the bullet lists into text that renders correctly with rst.

This was tested with the nova-hyperv.rst:
f0607509e3/doc/config-reference/source/tables/nova-hyperv.rst

Change-Id: Iab063774f6732d52cd8e79f82ff0fc93bd6d306b
2016-05-27 08:41:04 +00:00
ZhiQiang Fan
e115fee6a3 safe get deprecated_reason
when run ./autohelp-wrapper rst aodh, it failed with:
AttributeError: 'BoolOpt' object has no attribute 'deprecated_reason'

we need to check the value before we use it

Change-Id: I362606ba186d1582f850bfa4d602503d6ea5079a
Closes-Bug: #1573953
2016-04-24 01:51:38 +08:00
Ronald Bradford
cba7f7f340 Better identify deprecated configuration options
If a configuration option is tagged as deprecated in the code and the
help text does not start with DEPRECATED: add this to better identify
options that are no longer used.

If the code provides an additional deprecated reason, append this to
the provided help text of the configuration option.

Change-Id: I1a7c76c0109a6e59cc87d0ddc942c6d1cfb9dc91
2016-03-16 18:08:07 -04:00
Ronald Bradford
557d0af6d0 Provide a more human identifable option data type
Update the configuration tables option type from an internal meta type
to a more meaningful description.  E.g. Boolean for BoolOpt, String
for StringOpt, Multi-valued for MultiStrOpt etc.

Inspired by the translation used in Developer Docs

Change-Id: I5bd7aec727292ceef4dc0ff56ab8f334ea2f22bc
2016-03-16 17:32:35 -04:00
Ronald Bradford
4bca964706 Catch sqlalchemy.exc.InvalidRequestError exception
Running update of keystone produces the following exception which
teminates processing. This enables processing to continue, like
capture of other types of errors.
* Table 'federation_protocol' is already defined for this MetaData
  instance.  Specify 'extend_existing=True' to redefine options and
  columns on an existing Table object.

Cater for projects that do not use sqlalchemy, e.g. swift.

Change-Id: I87df4bb784f94b7194374b2448e4c34beadb923f
2016-03-16 17:32:08 -04:00
Jenkins
a19eef20ce Merge "autohelp: avoid duplicated overriden entries" 2016-01-24 12:11:29 +00:00
Gauvain Pocentek
d0c8b278de autohelp: fix poor behavior of _sanitize_default
Change-Id: I9a9d41785c7674056a6d553941510ca1e5b30383
2016-01-24 11:53:25 +01:00
Gauvain Pocentek
5dc95bfb6b autohelp: avoid duplicated overriden entries
Change-Id: I99e3f9735ec1baf8d514088ec3942bfe76895a2a
2016-01-24 11:18:56 +01:00
Gauvain Pocentek
e436173ae2 Move the config items in the manuals repository
Modifying configuration options (ignored modules, extra repositories,
hooks) for autohelp requires an update of the doc-tools repository,
which doesn't make much sense since the goal is to update the manuals
repository.

This patch removes the configuration options from the repository and
updates the scripts to use the configuration items in openstack-manuals.

Change-Id: I630d2a2fea690f6360039aa100331678468d18dc
2016-01-11 08:52:33 +01:00
Gauvain Pocentek
e3c42a269d autohelp update can create the flags file
If the flagmappings file doesn't exist the function will create it,
with all the sections set to Unknown. This mimics the behavior of the
`create`, not needed anymore.

Change-Id: I7a920300b784aa8fca6412c292d807beb4d5c8e5
2016-01-08 09:09:58 +01:00
Gauvain Pocentek
658c74dd7e [autohelp] config-ref-rst moves to config-reference
Update the default paths to make the scripts work after the config-ref
publication.

Implements: blueprint config-ref-rst

Change-Id: I55534d08b10af4fb44a7dd22718ae22e44708559
2015-12-20 20:56:08 +01:00
Gauvain Pocentek
e8784d29c7 [autohelp] display information on import error
When a module import fails the wrapper script doesn't display its
path/name, so it is complicated to debug. this patch displays the module
name before re-raising the exception.

Change-Id: Ie9d691a41b0c178b856f19605a8c6434d3ce2c88
2015-12-17 17:54:53 +01:00
Gauvain Pocentek
5c743c3d4a [autohelp] allow overrides of sections
We want to be able to change the section in which an option is
registered in the documentation. This patch uses a <project>.overrides
file to define in which section(s) an option should be moved.

The format of this file is (1 line per option):
[<group>/]<option> <new_group1>[ <new_group2> ... ]

Change-Id: I9fd0b763b6e56cd360694566de7f3acc3172c9c1
Closes-Bug: #1522794
2015-12-05 15:09:53 +01:00
Jenkins
a918809e0e Merge "[autohelp] Add table label for RST reference" 2015-11-23 12:48:26 +00:00
KATO Tomoyuki
d881430418 [autohelp] Add table label for RST reference
Each table need to be markuped up for reference.
In case the label is set at outside of inclusion,
Sphinx can not properly treat the label.

Change-Id: I2257471300ca5752ba717858a14d4fa26f62916f
2015-11-21 22:52:39 +09:00
Gauvain Pocentek
50589f3343 autohelp: avoid help strings on multiple lines
Multiple lines means dealing with indentation, which is complicated and
not a problem for automatically generated tables. Use the same helptext
generation in autohelp than in extract_swift_flags.

Change-Id: Ibd64d3f0387c3e7f1f3aacf57be5f0c97addf5c9
2015-11-20 16:38:40 +01:00
Jenkins
9abc7dc283 Merge "[autohelp] Move ignored modules list to a file" 2015-11-16 05:44:28 +00:00
Gauvain Pocentek
52537d2341 autohelp.py: switch to jinja templates
Change-Id: Ice55c665adf4c0ede819cfc7264ed13feeca14db
2015-11-15 19:40:17 +01:00
Gauvain Pocentek
1d309a0a47 [autohelp] Move ignored modules list to a file
Change-Id: I80a7ed680149e40cbbf84f22fb5735a00666c872
2015-11-15 17:50:55 +01:00
Gauvain Pocentek
a699f8d764 autohelp: properly sanitize paths
Change-Id: Id2449820c5c10eb40be1f688504ec6bf13cc42a2
2015-11-09 09:55:38 +01:00
Atsushi SAKAI
3dbacea2fb Update config reference package list for neutron
This is just for update package list changes.
 Not solved config reference autogeneration problem.

 Remove stackforge package handling in autohelp-wrapper
 Remove modules
   networking-ibm (SDNVE) (It seems no maintained)
   networking-bigswitch (It seems no maintained)
   networking-portforwarding (It seems no maintained)

 Changes from stackforge to openstack
   networking-bagpipe-l2, networking-brocade,
   networking-edge-vpn, networking-hyperv,
   networking-mlnx, networking-nec, networking-ovs-dpdk,
   networking-zvm, networking-6wind

 Add
   octavia

Change-Id: I5ba3b26e110f55b2cc76f37b903a1b07dd6a1202
Related-Bug: #1503967
2015-10-19 16:23:38 +09:00
Tom Fifield
cd74750d4b Increase visibility of the warning in autodoc files
Autogenerate-config-docs creates files that live in the
openstack-manuals repository, but are automatically generated
and should not be edited.

On occasion, contributors miss the warning not to edit the files.

This patch increases the previous warning so that it should take up
enough screen real-estate in a text editor that it would be less
easily skipped, and adds some additional information on what to do
instead.

Change-Id: I800e1ed19427e00f903046731fe8f22f22e4c6db
2015-10-08 16:43:16 +08:00
Atsushi SAKAI
b2f64d0012 Add supported modules for config ref generator
With this patch, it can run on
  designate
  manila
  zaqar

Related-Bug: #1407581

How to add
  see the Trace back and add last item. like follows
    For addin designate.api.v1.extensions.sync case,

====log===
Traceback (most recent call last):

snip

  File "/home/sakaia/work/openstack-doc-tools/autogenerate_config_docs/autohelp.py", line 138, in import_modules
    module = importlib.import_module(modname)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/sakaia/work/openstack-doc-tools/autogenerate_config_docs/sources/designate/designate/api/v1/extensions/sync.py", line 23, in <module>
    central_api = central_rpcapi.CentralAPI()
=====

Change-Id: I76101cba32c4dd09a303a878719db5f544eb3ebf
2015-09-16 16:16:30 +09:00
Gauvain Pocentek
04e4d4bd35 autohelp: add an exception for the cinder wsgi script
This script doesn't test the __main__ variable so the import tries to
run cinder and fails.

Change-Id: Ie61946235bb51e770941b433b56c537382f494d2
2015-09-07 08:54:18 +02:00
Gauvain Pocentek
11e46da154 autohelp: add oslo.service to the list of extensions
Change-Id: Id523dd72c5987f86d22bd5905ed1511b58ae25cf
Closes-Bug: #1487171
2015-08-22 15:12:45 +02:00
Jenkins
0b67e01e79 Merge "autohelp: support reading options from multiple repos" 2015-07-27 03:39:08 +00:00
Gauvain Pocentek
da4d785171 autohelp: add oslo.cache as extension
Change-Id: Ifac03228dfe43719fdc89cfffa2ac53865aa06ee
2015-07-24 09:58:39 +02:00
Gauvain Pocentek
2e63aba8ac autohelp: support reading options from multiple repos
This change modifies the -i option behavior of autohelp.py to add the
possibility to find configuration options in multiple repositories. The
change implies a few related modifications:

- do not check if the source repository is a git repository
- update autohelp-wrapper to clone and use the additional repositories
  (only for neutron at the moment)

Note: diff_branches.py will not work with this change, it'll be fixed in
another patch.

Closes-Bug: #1448107
Change-Id: I4999aca4f7926e19513acdf38f1e2828b52cee13
2015-07-23 16:49:43 +02:00
Gauvain Pocentek
a98a112832 autohelp.py: remove the oslo.i18n hack
All the projects use oslo.i18n in kilo and liberty, the workaround for
projects non using this module is not needed anymore.

Change-Id: I7a096c11eef2ae2f3e9006fa0379f49f18a1cc81
2015-07-23 11:02:08 +02:00
Gauvain Pocentek
b73375f1fe autohelp: fix a typo
This adds back the missing keystone database settings, and most likely
some other settings.

Change-Id: Idd0c327a6e8a95bc800a9abd695ef7a277d52f2f
2015-07-14 22:19:39 +02:00
Ruby Loo
6329eea32a autohelp: use correct option names from extensions
autohelp's OptionsCache.load_extension_options() loads options from
libraries. An oslo_config.cfg.Opt object has a 'name' attribute and
a 'dest' attribute. The option name that we're interested in (that
the user sets) is the 'dest' value, not the 'name' value. The
load_extension_options() method was incorrectly using the option's
'name' attribute.

This fixes it so that the option's 'dest' attribute is used instead.
Eg, from oslo_log library, it now correctly uses 'log_config_append'
instead of 'log-config-append'.

Change-Id: I01422bf33be286cc0176a3e1e4ac1c348c6821ef
Closes-Bug: #1464058
2015-06-11 02:00:51 +00:00
Gauvain Pocentek
b455d12bfb autohelp: generate rst tables
We are mmoving the guides to RST, so we need config-reference tables in
this format. This patch adds a `rst` subcommand to autohelp.py to handle
this new requirement.

RST has limited tables-related features, and openstackdocstheme will
need an update to improve the output.

Change-Id: I5e58d60e2308d208e850f903559da5f5a6a949e3
2015-05-22 09:44:29 -07:00
Gauvain Pocentek
9aa45dda9a autohelp: remove rootwrap discovery code
All the projects are using oslo_rootwrap in kilo, and don't provide
rootwrap options anymore.

Change-Id: I1a0bd98862b0504a723c7aef14f8a3f0c0945ed5
2015-05-21 07:16:19 -07:00
Gauvain Pocentek
e4c8992edb autohelp: load oslo.middleware options
Partial-Bug: #1452454
Change-Id: I38289912445117ed4d2bbee2d6a886fd88fac4c8
2015-05-07 06:00:40 +00:00
Gauvain Pocentek
feba2086be Autohelp: update the scripts for the juno/kilo diff
* include sahara
* remove the keystoneclient workaround
* ignore a problematic neutron import
* handle "None" values

Change-Id: I8f342dcb39f5ddd69b73baec2ecb1f3c6883c01a
2015-04-29 15:08:56 +00:00
Doug Hellmann
d797ef81cd Drop use of 'oslo' namespace package
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.

The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.

Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.

Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages

Change-Id: Ib17a74ada3122ce45c52fdd7dcf8d27fafbb1d70
2015-04-28 22:18:23 +00:00
Gauvain Pocentek
71b8d2175f autohelp: lookup options in more oslo libraries
Change-Id: I24942de43dbe04256f511ee372f203942f555cce
2015-04-24 12:11:58 +02:00
Gauvain Pocentek
044850f8cc autohelp: strip help strings on dump too
Change-Id: I639ffd4352b8e9b09debaca01fa2befd79ecf466
2015-04-02 15:13:42 +02:00
Gauvain Pocentek
05d96ac900 autohelp: update the ignored modules list
This includes 2 glance contrib modules, and the experimental pgsql trove
modules.

Change-Id: If18946b1cda5c5742f554b0635789415a50c02b8
2015-04-02 13:05:09 +02:00
Gauvain Pocentek
223286cadc Rework the autohelp serialization of options
Using oslo.config objects in pickle leads to multiple import problems on
deserialization. Since we don't actually need oslo objects, we convert
them into dicts before dumping them.

Change-Id: Icbd1a3957e4d4ae05fe6227fbdd1cb58e5a3a98d
2015-03-03 17:13:18 +01:00
Gauvain Pocentek
3d2f91cf8e autohelp: don't ignore cmd/ folders
These folders used to bring command line options that we don't want to
have in the generated tables, but the behavior has changed. This patch
restores the import of files in the cmd/ folders, and handles special
cases where the import is problematic.

Change-Id: I0779e0f689d2f6f64149c3d111a8dd1ecd6cfc20
Partial-Bug: #1363954
2015-02-19 21:06:24 +01:00
Gauvain Pocentek
7a1046484b autohelp: don't specify encoding in the base xml
Specifying the encoding and working with unicode strings in not
supported in lxml. The BASE_XML string is only used to init the XML tree
and does not need a specific encoding. The utf-8 encoding is explicitely
added when the final XML is written to file.

Change-Id: I74e0a316cf718ad9768a690f8ccc4a1cc9a9fc7e
2015-01-31 12:26:14 +01:00
Gauvain Pocentek
1cfa1fb2be Handle possible keystone duplicated opts
We use trickery to import the keystone middleware options, which might
result in duplicated opts errors. This patch ignore such errors on the
middleware import.

Also use keystonemiddleware instead of keystoneclient.middleware.

Change-Id: I4e003ec9f4a7a706a2bbdeea5f874624fc59ea40
2015-01-21 18:39:28 +01:00
Tom Fifield
3363d2e1cb Fix xml tagging for non-swift config tables
A previous patch attempted to introduce semantic tagging for
options and their default values in the automatically
generated configuration reference tables.

While this worked well for swift, incorrect code was used
for the XML generation of the main autohelp.py method.

This patch fixes the string based code to use XML subElements
as was already done for extract_swift_flags.

Change-Id: I93a89b726d66acd0778e8365f47d5ee2f2e0b3c5
2015-01-01 19:14:43 +08:00