Browse Source

Remove 'autogenerate_config_docs'

This tool has been superseded by the 'oslo_config.sphinxext' Sphinx
extension and can finally be put to rest.

[1] https://docs.openstack.org/oslo.config/latest/reference/sphinxext.html

Change-Id: Idae5ba656c3a0da8a26c6f1e63332365c8012c0b
tags/1.7.0
Stephen Finucane 1 year ago
parent
commit
d3c952064a

+ 0
- 6
README.rst View File

@@ -59,9 +59,3 @@ On openSUSE::
59 59
 On Ubuntu::
60 60
 
61 61
     $ apt-get install libxml2-dev libxslt-dev
62
-
63
-
64
-Regenerating config option tables
65
----------------------------------
66
-
67
-See :ref:`autogenerate_config_docs`.

+ 0
- 121
autogenerate_config_docs/README.rst View File

@@ -1,121 +0,0 @@
1
-.. _autogenerate_config_docs:
2
-
3
-autogenerate_config_docs
4
-========================
5
-
6
-Automatically generate configuration tables to document OpenStack.
7
-
8
-Using the wrapper
9
------------------
10
-
11
-``autohelp-wrapper`` is the recommended tool to generate the configuration
12
-tables. Don't bother using ``autohelp.py`` manually.
13
-
14
-The ``autohelp-wrapper`` script installs a virtual environment and all the
15
-needed dependencies, clones or updates the projects and manuals repositories,
16
-then runs the ``autohelp.py`` script in the virtual environment.
17
-
18
-New and updated flagmappings are generated in the ``openstack-manuals``
19
-repository (``tools/autogenerate-config-flagmappings/`` directory).
20
-
21
-Prior to running the following commands, you need to install several development
22
-packages.
23
-
24
-On Ubuntu:
25
-
26
-.. code-block:: console
27
-
28
-    $ sudo apt-get install python-dev python-pip python-virtualenv \
29
-                           libxml2-dev libxslt1-dev zlib1g-dev \
30
-                           libmysqlclient-dev libpq-dev libffi-dev \
31
-                           libsqlite3-dev libldap2-dev libsasl2-dev \
32
-                           libjpeg-dev
33
-
34
-On RHEL 7 and CentOS 7:
35
-
36
-.. code-block:: console
37
-
38
-    $ sudo yum install https://www.rdoproject.org/repos/rdo-release.rpm
39
-    $ sudo yum update
40
-    $ sudo yum install python-devel python-pip python-virtualenv \
41
-                           libxml2-devel libxslt-devel zlib-devel \
42
-                           mariadb-devel postgresql-devel libffi-devel \
43
-                           sqlite-devel openldap-devel cyrus-sasl-devel \
44
-                           libjpeg-turbo-devel gcc git
45
-
46
-.. note::
47
-    * libjpeg is needed for ironic
48
-
49
-The workflow is:
50
-
51
-.. code-block:: console
52
-
53
-    $ pip install -rrequirements.txt
54
-    $ ./autohelp-wrapper update
55
-    $ $EDITOR sources/openstack-manuals/tools/autogenerate-config-flagmappings/*.flagmappings
56
-    $ ./autohelp-wrapper rst
57
-    $ # check the results in sources/openstack-manuals
58
-
59
-This will generate the tables for all the known projects.
60
-Note for neutron project: If the driver/plugin resides outside the neutron
61
-repository, then the driver/plugin has to be explicitly installed within the
62
-virtual environment to generate the configuration options.
63
-
64
-To generate the mappings and tables for a subset of projects, use the code
65
-names as arguments:
66
-
67
-.. code-block:: console
68
-
69
-    $ ./autohelp-wrapper update cinder heat
70
-    $ # edit the mappings files
71
-    $ ./autohelp-wrapper rst cinder heat
72
-
73
-
74
-Flagmappings files
75
-------------------
76
-
77
-The tool uses flagmapping files to map options to custom categories. Flag
78
-mapping files can be found in the ``tools/autogenerate-config-flagmappings``
79
-folder of the openstack-manuals project. Not all projects use flagmapping
80
-files, as those that do not will be disabled by the presence of a
81
-``$project.disable`` file in that folder. For those that do, however, the files
82
-use the following format::
83
-
84
-    OPTION_SECTION/OPTION_NAME group1 [group2, ...]
85
-
86
-Groups need to be defined manually to organize the configuration tables.
87
-
88
-The group values can only contain alphanumeric characters, _ and - (they will
89
-be used as document IDs).
90
-
91
-To make the table titles more user friendly, create or edit the PROJECT.headers
92
-file in the manuals repository. Each line of this file is of the form:
93
-
94
-::
95
-
96
-    GROUP A Nice Title
97
-
98
-Working with branches
99
----------------------
100
-
101
-``autohelp-wrapper`` works on the master branch by default, but you can tell it
102
-to work on another branch:
103
-
104
-.. code-block:: console
105
-
106
-    $ ./autohelp-wrapper -b stable/liberty update
107
-
108
-.. note::
109
-   The ``-b`` switch doesn't apply to the ``openstack-manuals`` repository
110
-   which will be left untouched (no ``git branch``, no ``git update``).
111
-
112
-
113
-Generate configuration difference
114
----------------------------------
115
-
116
-To generate "New, updated, and deprecated options" for each service,
117
-run ``diff_branches.py``. For example:
118
-
119
-.. code-block:: console
120
-
121
-   $ ./diff_branches.py stable/liberty stable/mitaka nova

+ 0
- 18
autogenerate_config_docs/__init__.py View File

@@ -1,18 +0,0 @@
1
-#   Copyright 2012 OpenStack Foundation
2
-#
3
-#   Licensed under the Apache License, Version 2.0 (the "License"); you may
4
-#   not use this file except in compliance with the License. You may obtain
5
-#   a copy of the License at
6
-#
7
-#       http://www.apache.org/licenses/LICENSE-2.0
8
-#
9
-#   Unless required by applicable law or agreed to in writing, software
10
-#   distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
11
-#   WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
12
-#   License for the specific language governing permissions and limitations
13
-#   under the License.
14
-
15
-import pbr.version
16
-
17
-
18
-__version__ = pbr.version.VersionInfo('openstack-doc-tools').version_string()

+ 0
- 273
autogenerate_config_docs/autohelp-wrapper View File

@@ -1,273 +0,0 @@
1
-#!/bin/bash
2
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
3
-# not use this file except in compliance with the License. You may obtain
4
-# a copy of the License at
5
-#
6
-#      http://www.apache.org/licenses/LICENSE-2.0
7
-#
8
-# Unless required by applicable law or agreed to in writing, software
9
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11
-# License for the specific language governing permissions and limitations
12
-# under the License.
13
-
14
-set -e
15
-
16
-HERE=$(pwd)
17
-VENVDIR=$HERE/venv
18
-SOURCESDIR=$HERE/sources
19
-MANUALSREPO=$SOURCESDIR/openstack-manuals
20
-MAPPINGS_DIR=$MANUALSREPO/tools/autogenerate-config-flagmappings
21
-AUTOHELP="python $HERE/autohelp.py"
22
-GITBASE=${GITBASE:-git://git.openstack.org/openstack}
23
-GITPROJ=${GITPROJ:-git://git.openstack.org/openstack}
24
-PROJECTS="aodh ceilometer cinder glance heat ironic keystone manila \
25
-            murano neutron nova sahara senlin trove zaqar"
26
-MANUALS_PROJECTS="openstack-manuals"
27
-BRANCH=master
28
-FAST=0
29
-QUIET=0
30
-CLONE_MANUALS=1
31
-
32
-usage() {
33
-    echo "Wrapper for autohelp.py"
34
-    echo "Usage:"
35
-    echo "  $(basename $0) [ OPTIONS ] dump|update|rst|setup [ project... ]"
36
-    echo
37
-    echo "Subcommands:"
38
-    echo "  dump:       Dumps the list of options with their attributes"
39
-    echo "  update:     Update or create the flagmapping files"
40
-    echo "  rst:        Generate the options tables in RST format"
41
-    echo "  setup:      Install the environment only"
42
-    echo
43
-    echo "Options:"
44
-    echo "  -b BRANCH:  Work on this branch (defaults to master)"
45
-    echo "  -g GITPROJ: Use this location for the project git repos "
46
-    echo "              (defaults to git://git.openstack.org/openstack)"
47
-    echo "  -c:         Recreate the virtual environment"
48
-    echo "  -f:         Work offline: Do not change environment or sources"
49
-    echo "  -e PATH:    Create the virtualenv in PATH"
50
-    echo "  -v LEVEL:   Verbose message (1 or 2)"
51
-    echo "              (check various python modules imported or not)"
52
-    echo "  -o OUTDIR:  Path to output openstack-manuals directory "
53
-    echo "              (defaults to ./sources/openstack-manuals)"
54
-}
55
-
56
-setup_venv() {
57
-    project=$1
58
-
59
-    if [ ! -e $VENVDIR/$project/bin/activate ]; then
60
-        mkdir -p $VENVDIR/$project
61
-        virtualenv $VENVDIR/$project
62
-    fi
63
-    activate_venv $project
64
-}
65
-
66
-activate_venv() {
67
-    project=$1
68
-
69
-    . $VENVDIR/$project/bin/activate
70
-    pip install --upgrade pip setuptools
71
-}
72
-
73
-get_project() {
74
-    project=$1
75
-    git_url=$GITPROJ
76
-
77
-    if [ ! -e $SOURCESDIR/$project ]; then
78
-        if [[ $MANUALS_PROJECTS =~ (^| )$project($| ) ]]; then
79
-            git_url=$GITBASE
80
-        fi
81
-        git clone $git_url/$project $SOURCESDIR/$project
82
-
83
-        if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
84
-            while read extra; do
85
-                git clone $git_url/$extra $SOURCESDIR/$extra
86
-            done < $MAPPINGS_DIR/$project.extra_repos
87
-        fi
88
-
89
-    else
90
-        if [ $project != openstack-manuals ]; then
91
-            (cd $SOURCESDIR/$project && git pull)
92
-        fi
93
-
94
-        if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
95
-            while read extra; do
96
-                (cd $SOURCESDIR/$extra && git pull)
97
-            done < $MAPPINGS_DIR/$project.extra_repos
98
-        fi
99
-    fi
100
-}
101
-
102
-setup_tools() {
103
-    pip install -rrequirements.txt
104
-}
105
-
106
-while getopts :b:g:e:o:v:cfq opt; do
107
-    case $opt in
108
-    b)
109
-        BRANCH=$OPTARG
110
-        ;;
111
-    g)
112
-        GITPROJ=$OPTARG
113
-        ;;
114
-    c)
115
-        rm -rf $VENVDIR
116
-        ;;
117
-    e)
118
-        VENVDIR=$OPTARG
119
-        ;;
120
-    o)
121
-        MANUALSREPO=$OPTARG
122
-        MAPPINGS_DIR=$OPTARG/tools/autogenerate-config-flagmappings
123
-        CLONE_MANUALS=0
124
-        ;;
125
-    f)
126
-        FAST=1
127
-        ;;
128
-    q)
129
-        QUIET=1
130
-        ;;
131
-    v)
132
-        AUTOOPT="-v"
133
-        if [ $OPTARG = 2 ]; then
134
-            AUTOOPT="-vv"
135
-        fi
136
-        ;;
137
-    \?)
138
-        usage
139
-        exit 1
140
-        ;;
141
-    esac
142
-done
143
-shift $(($OPTIND - 1))
144
-
145
-if [ $# -lt 1 ]; then
146
-    usage
147
-    exit 1
148
-fi
149
-
150
-ACTION=$1
151
-shift
152
-
153
-if [ $QUIET -eq 1 ]; then
154
-    exec 3>&1 >/dev/null
155
-    exec 4>&2 2>/dev/null
156
-fi
157
-
158
-case $ACTION in
159
-    update|rst|dump|setup) ;;
160
-    *)
161
-        usage
162
-        exit 1
163
-        ;;
164
-esac
165
-
166
-if [ ! -e autohelp.py ]; then
167
-    echo "Execute this script in the autogenerate_config_docs directory."
168
-    exit 1
169
-fi
170
-
171
-[ $# != 0 ] && PROJECTS="$*"
172
-
173
-RELEASE=$(echo $BRANCH | sed 's,^stable.,,')
174
-
175
-if [ "$FAST" -eq 0 ] ; then
176
-    if [ "$CLONE_MANUALS" -eq 1 ] ; then
177
-        get_project openstack-manuals
178
-    fi
179
-
180
-    for project in $PROJECTS; do
181
-        setup_venv $project
182
-        setup_tools
183
-        if [ -e $MAPPINGS_DIR/$project.requirements ]; then
184
-            pip install -r $MAPPINGS_DIR/$project.requirements \
185
-                --allow-all-external
186
-        fi
187
-        get_project $project
188
-
189
-        (
190
-            pushd $SOURCESDIR/$project
191
-            module=$(echo $project | tr - _ )
192
-            find $module -name "*.pyc" -delete
193
-            GIT_CMD="git show-ref --verify --quiet refs/heads/$BRANCH"
194
-            if $GIT_CMD; then
195
-                git checkout $BRANCH
196
-            else
197
-                git checkout -b $BRANCH remotes/origin/$BRANCH
198
-            fi
199
-            pip install -rrequirements.txt
200
-            [ -e "test-requirements.txt" ] && \
201
-                pip install -rtest-requirements.txt
202
-            python setup.py install
203
-            popd
204
-
205
-            if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
206
-                while read extra; do
207
-                    (
208
-                        cd $SOURCESDIR/$extra
209
-                        pip install -rrequirements.txt
210
-                        [ -e "test-requirements.txt" ] && \
211
-                            pip install -rtest-requirements.txt
212
-                        python setup.py install
213
-                    )
214
-                done < $MAPPINGS_DIR/$project.extra_repos
215
-            fi
216
-        )
217
-    done
218
-fi
219
-
220
-for project in $PROJECTS; do
221
-    echo "Working on $project..."
222
-    activate_venv $project
223
-    if [ "$ACTION" = "setup" ]; then
224
-        break
225
-    fi
226
-
227
-    if [ -e $MAPPINGS_DIR/$project.extra_repos ]; then
228
-        extra_flags=
229
-        while read extra; do
230
-            package=$(echo $extra | tr - _)
231
-            if [ $package = "networking_midonet" ]; then
232
-                package="midonet"
233
-            fi
234
-            if [ $package = "networking_hyperv" ]; then
235
-                package="hyperv"
236
-            fi
237
-            if [ $package = "networking_edge_vpn" ]; then
238
-                package="networking-edge-vpn"
239
-            fi
240
-            if [ $package = "networking_zvm" ]; then
241
-                package="neutron"
242
-                cp -r $SOURCESDIR/networking-zvm/neutron/plugins/zvm \
243
-                    $SOURCESDIR/neutron/neutron/plugins
244
-                cp -r \
245
-                    $SOURCESDIR/networking-zvm/neutron/plugins/ml2/drivers/zvm\
246
-                    $SOURCESDIR/neutron/neutron/plugins/ml2/drivers
247
-            fi
248
-            extra_flags="$extra_flags -i $SOURCESDIR/$extra/$package"
249
-        done < $MAPPINGS_DIR/$project.extra_repos
250
-    fi
251
-
252
-    cd $MAPPINGS_DIR
253
-
254
-    case $ACTION in
255
-    update)
256
-        $AUTOHELP update $project -i $SOURCESDIR/$project/$project \
257
-            $extra_flags $AUTOOPT
258
-        mv $project.flagmappings.new $project.flagmappings
259
-        ;;
260
-    rst)
261
-        $AUTOHELP rst $project -i $SOURCESDIR/$project/$project \
262
-            $extra_flags $AUTOOPT
263
-        ;;
264
-    dump)
265
-        if [ $QUIET -eq 1 ]; then
266
-            exec 1>&3
267
-            exec 2>&4
268
-        fi
269
-        $AUTOHELP dump $project -i $SOURCESDIR/$project/$project \
270
-            $extra_flags $AUTOOPT
271
-        ;;
272
-    esac
273
-done

+ 0
- 696
autogenerate_config_docs/autohelp.py View File

@@ -1,696 +0,0 @@
1
-#!/usr/bin/env python
2
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
3
-# not use this file except in compliance with the License. You may obtain
4
-# a copy of the License at
5
-#
6
-#      http://www.apache.org/licenses/LICENSE-2.0
7
-#
8
-# Unless required by applicable law or agreed to in writing, software
9
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11
-# License for the specific language governing permissions and limitations
12
-# under the License.
13
-#
14
-# A collection of tools for working with flags from OpenStack
15
-# packages and documentation.
16
-#
17
-# For an example of usage, run this program with the -h switch.
18
-#
19
-
20
-# Must import this before argparse
21
-from oslo_config import cfg
22
-
23
-import argparse
24
-import importlib
25
-import os
26
-import pickle
27
-import re
28
-import sys
29
-
30
-import jinja2
31
-import stevedore
32
-
33
-try:
34
-    from sqlalchemy import exc
35
-except Exception:
36
-    pass
37
-
38
-sys.path.insert(0, '.')
39
-try:
40
-    from hooks import HOOKS  # noqa
41
-except ImportError:
42
-    pass
43
-
44
-
45
-EXTENSIONS = ['oslo.cache',
46
-              'oslo.concurrency',
47
-              'oslo.db',
48
-              'oslo.log',
49
-              'oslo.messaging',
50
-              'oslo.middleware',
51
-              'oslo.policy',
52
-              'oslo.service']
53
-
54
-_TYPE_DESCRIPTIONS = {
55
-    cfg.StrOpt: 'String',
56
-    cfg.BoolOpt: 'Boolean',
57
-    cfg.IntOpt: 'Integer',
58
-    cfg.FloatOpt: 'Floating point',
59
-    cfg.ListOpt: 'List',
60
-    cfg.DictOpt: 'Dict',
61
-    cfg.MultiStrOpt: 'Multi-valued',
62
-    cfg.IPOpt: 'IP address',
63
-    cfg.PortOpt: 'Port number',
64
-    cfg.HostnameOpt: 'Hostname',
65
-    cfg.URIOpt: 'URI',
66
-    cfg.HostAddressOpt: 'Host address',
67
-    cfg._ConfigFileOpt: 'List of filenames',
68
-    cfg._ConfigDirOpt: 'List of directory names',
69
-}
70
-
71
-register_re = re.compile(r'''^ +.*\.register_opts\((?P<opts>[^,)]+)'''
72
-                         r'''(, (group=)?["'](?P<group>.*)["'])?\)''')
73
-
74
-
75
-def import_modules(repo_location, package_name, verbose=0):
76
-    """Import modules.
77
-
78
-    Loops through the repository, importing module by module to
79
-    populate the configuration object (cfg.CONF) created from Oslo.
80
-    """
81
-
82
-    with open('ignore.list') as fd:
83
-        ignore_list = [l for l in fd.read().split('\n')
84
-                       if l and (l[0] != '#')]
85
-
86
-    pkg_location = os.path.join(repo_location, package_name)
87
-    for root, dirs, files in os.walk(pkg_location):
88
-        skipdir = False
89
-        for excludedir in ('tests', 'locale',
90
-                           os.path.join('db', 'migration'), 'transfer'):
91
-            if ((os.path.sep + excludedir + os.path.sep) in root or (
92
-                    root.endswith(os.path.sep + excludedir))):
93
-                skipdir = True
94
-                break
95
-        if skipdir:
96
-            continue
97
-        for pyfile in files:
98
-            if pyfile.endswith('.py'):
99
-                abs_path = os.path.join(root, pyfile)
100
-                modfile = abs_path.split(repo_location, 1)[1]
101
-                modname = os.path.splitext(modfile)[0].split(os.path.sep)
102
-                modname = [m for m in modname if m != '']
103
-                modname = '.'.join(modname)
104
-                if modname.endswith('.__init__'):
105
-                    modname = modname[:modname.rfind(".")]
106
-                if modname in ignore_list:
107
-                    continue
108
-                try:
109
-                    module = importlib.import_module(modname)
110
-                    if verbose >= 1:
111
-                        print("imported %s" % modname)
112
-                except ImportError as e:
113
-                    """
114
-                    work around modules that don't like being imported in
115
-                    this way FIXME This could probably be better, but does
116
-                    not affect the configuration options found at this stage
117
-                    """
118
-                    if verbose >= 2:
119
-                        print("Failed to import: %s (%s)" % (modname, e))
120
-                    continue
121
-                except cfg.DuplicateOptError as e:
122
-                    """
123
-                    oslo.cfg doesn't allow redefinition of a config option, but
124
-                    we don't mind. Don't fail if this happens.
125
-                    """
126
-                    if verbose >= 2:
127
-                        print(e)
128
-                    continue
129
-                except cfg.NoSuchGroupError as e:
130
-                    """
131
-                    If a group doesn't exist, we ignore the import.
132
-                    """
133
-                    if verbose >= 2:
134
-                        print(e)
135
-                    continue
136
-                except exc.InvalidRequestError as e:
137
-                    if verbose >= 2:
138
-                        print(e)
139
-                    continue
140
-                except Exception as e:
141
-                    print("Impossible to import module %s" % modname)
142
-                    raise
143
-
144
-                _register_runtime_opts(module, abs_path, verbose)
145
-                _run_hook(modname)
146
-
147
-    # All the components provide keystone token authentication, usually using a
148
-    # pipeline. Since the auth_token options can only be discovered at runtime
149
-    # in this configuration, we force their discovery by importing the module.
150
-    try:
151
-        import keystonemiddleware.auth_token  # noqa
152
-    except cfg.DuplicateOptError:
153
-        pass
154
-
155
-
156
-def _run_hook(modname):
157
-    try:
158
-        HOOKS[modname]()
159
-    except KeyError:
160
-        pass
161
-
162
-
163
-def _register_runtime_opts(module, abs_path, verbose):
164
-    """Handle options not registered on module import.
165
-
166
-    This function parses the .py files to discover calls to register_opts in
167
-    functions and methods. It then explicitly call cfg.register_opt on each
168
-    option to register (most of) them.
169
-    """
170
-
171
-    with open(abs_path) as fd:
172
-        for line in fd:
173
-            m = register_re.search(line)
174
-            if not m:
175
-                continue
176
-
177
-            opts_var = m.group('opts')
178
-            opts_group = m.group('group')
179
-
180
-            # Get the object (an options list) from the opts_var string.
181
-            # This requires parsing the string which can be of the form
182
-            # 'foo.bar'. We treat each element as an attribute of the previous.
183
-            register = True
184
-            obj = module
185
-            for item in opts_var.split('.'):
186
-                try:
187
-                    obj = getattr(obj, item)
188
-                except AttributeError:
189
-                    # FIXME(gpocentek): AttributeError is raised when a part of
190
-                    # the opts_var string is not an actual attribute. This will
191
-                    # need more parsing tricks.
192
-                    register = False
193
-                    if verbose >= 2:
194
-                        print("Ignoring %(obj)s in %(module)s" %
195
-                              {'obj': opts_var, 'module': module})
196
-                    break
197
-
198
-            if register and isinstance(obj, list):
199
-                for opt in obj:
200
-                    if not isinstance(opt, cfg.Opt):
201
-                        continue
202
-                    try:
203
-                        cfg.CONF.register_opt(opt, opts_group)
204
-                    except cfg.DuplicateOptError:
205
-                        # ignore options that have already been registered
206
-                        pass
207
-
208
-
209
-def _sanitize_default(opt):
210
-    """Adapts unrealistic default values."""
211
-
212
-    # If the Oslo version is recent enough, we can use the 'sample_default'
213
-    # attribute
214
-    if (hasattr(opt, 'sample_default') and opt.sample_default is not None):
215
-        return str(opt.sample_default)
216
-
217
-    if ((type(opt).__name__ == "ListOpt") and (type(opt.default) == list)):
218
-        return ", ".join(str(item) for item in opt.default)
219
-
220
-    default = str(opt.default)
221
-
222
-    if default == os.uname()[1]:
223
-        return 'localhost'
224
-
225
-    if opt.name == 'bindir':
226
-        return '/usr/local/bin'
227
-
228
-    if opt.name == 'my_ip':
229
-        return '10.0.0.1'
230
-
231
-    if isinstance(opt, cfg.StrOpt) and default.strip() != default:
232
-        return '"%s"' % default
233
-
234
-    for pathelm in sys.path:
235
-        if pathelm in ('.', ''):
236
-            continue
237
-        if pathelm.endswith('/'):
238
-            pathelm = pathelm[:-1]
239
-        if pathelm in default:
240
-            default = re.sub(r'%s(/sources)?' % pathelm,
241
-                             '/usr/lib/python/site-packages', default)
242
-
243
-    return default
244
-
245
-
246
-def _get_overrides(package_name):
247
-    overrides_file = '%s.overrides' % package_name
248
-    if not os.path.exists(overrides_file):
249
-        return {}
250
-    overrides = {}
251
-    with open(overrides_file) as fd:
252
-        for line in fd:
253
-            if line == '#':
254
-                continue
255
-            try:
256
-                opt, sections = line.strip().split(' ', 1)
257
-                sections = [x.strip() for x in sections.split(' ')]
258
-            except ValueError:
259
-                continue
260
-
261
-            overrides[opt] = sections
262
-
263
-    return overrides
264
-
265
-
266
-class OptionsCache(object):
267
-    def __init__(self, overrides={}, verbose=0):
268
-        self._verbose = verbose
269
-        self._opts_by_name = {}
270
-        self._opts_by_group = {}
271
-        self._opt_names = []
272
-        self._overrides = overrides
273
-
274
-        for optname in cfg.CONF._opts:
275
-            opt = cfg.CONF._opts[optname]['opt']
276
-            # We ignore some CLI opts by excluding SubCommandOpt objects
277
-            if not isinstance(opt, cfg.SubCommandOpt):
278
-                self._add_opt(optname, 'DEFAULT', opt)
279
-
280
-        for group in cfg.CONF._groups:
281
-            for optname in cfg.CONF._groups[group]._opts:
282
-                self._add_opt(group + '/' + optname, group,
283
-                              cfg.CONF._groups[group]._opts[optname]['opt'])
284
-
285
-        self._opt_names.sort(OptionsCache._cmpopts)
286
-
287
-    def _add_opt(self, optname, group, opt):
288
-        if optname in self._opts_by_name:
289
-            if self._verbose >= 2:
290
-                print("Duplicate option name %s" % optname)
291
-            return
292
-
293
-        opt.default = _sanitize_default(opt)
294
-
295
-        def fill(optname, group, opt):
296
-            if optname in self._opts_by_name:
297
-                return
298
-            self._opts_by_name[optname] = (group, opt)
299
-            self._opt_names.append(optname)
300
-
301
-            if group not in self._opts_by_group:
302
-                self._opts_by_group[group] = []
303
-
304
-            self._opts_by_group[group].append(opt)
305
-
306
-        if optname in self._overrides:
307
-            for new_group in self._overrides[optname]:
308
-                if new_group == 'DEFAULT':
309
-                    new_optname = opt.name
310
-                else:
311
-                    new_optname = new_group + '/' + opt.name
312
-                fill(new_optname, new_group, opt)
313
-
314
-        else:
315
-            fill(optname, group, opt)
316
-
317
-    def __len__(self):
318
-        return len(self._opt_names)
319
-
320
-    def load_extension_options(self, module):
321
-        # Note that options loaded this way aren't added to _opts_by_module
322
-        loader = stevedore.named.NamedExtensionManager(
323
-            'oslo.config.opts',
324
-            names=(module,),
325
-            invoke_on_load=False
326
-        )
327
-        for ext in loader:
328
-            for group, opts in ext.plugin():
329
-                for opt in opts:
330
-                    if group is None:
331
-                        self._add_opt(opt.dest, 'DEFAULT', opt)
332
-                    else:
333
-                        self._add_opt(group + '/' + opt.dest, group, opt)
334
-
335
-        self._opt_names.sort(OptionsCache._cmpopts)
336
-
337
-    def maybe_load_extensions(self, repositories):
338
-        # Use the requirements.txt of the project to guess if an oslo module
339
-        # needs to be imported
340
-        needed_exts = set()
341
-        for repo in repositories:
342
-            base_path = os.path.dirname(repo)
343
-            for ext in EXTENSIONS:
344
-                requirements = os.path.join(base_path, 'requirements.txt')
345
-                with open(requirements) as fd:
346
-                    for line in fd:
347
-                        if line.startswith(ext):
348
-                            needed_exts.add(ext)
349
-
350
-        for ext in needed_exts:
351
-            self.load_extension_options(ext)
352
-
353
-    def get_group_names(self):
354
-        return self._opts_by_group.keys()
355
-
356
-    def get_option_names(self):
357
-        return self._opt_names
358
-
359
-    def get_group(self, name):
360
-        return self._opts_by_group[name]
361
-
362
-    def get_option(self, name):
363
-        return self._opts_by_name[name]
364
-
365
-    def dump(self):
366
-        """Dumps the list of options with their attributes.
367
-
368
-        This output is consumed by the diff_branches script.
369
-        """
370
-        for name, (group, option) in self._opts_by_name.items():
371
-            deprecated_opts = [{'group': deprecated.group,
372
-                                'name': deprecated.name}
373
-                               for deprecated in option.deprecated_opts]
374
-            help_str = option.help.strip() if option.help else "None"
375
-            new_option = {
376
-                'default': option.default,
377
-                'help': help_str,
378
-                'deprecated_opts': deprecated_opts,
379
-                'type': option.__class__.__name__.split('.')[-1]
380
-            }
381
-            self._opts_by_name[name] = (group, new_option)
382
-        print(pickle.dumps(self._opts_by_name))
383
-
384
-    @staticmethod
385
-    def _cmpopts(x, y):
386
-        if '/' in x and '/' in y:
387
-            prex = x[:x.find('/')]
388
-            prey = y[:y.find('/')]
389
-            if prex != prey:
390
-                return cmp(prex, prey)
391
-            return cmp(x, y)
392
-        elif '/' in x:
393
-            return 1
394
-        elif '/' in y:
395
-            return -1
396
-        else:
397
-            return cmp(x, y)
398
-
399
-
400
-def _use_categories(package_name):
401
-    return not os.path.isfile(package_name + '.disable')
402
-
403
-
404
-def _get_options_by_cat(package_name):
405
-    options_by_cat = {}
406
-
407
-    with open(package_name + '.flagmappings') as f:
408
-        for line in f:
409
-            if not line.strip() or line.startswith('#'):
410
-                continue
411
-            opt, categories = line.split(' ', 1)
412
-            for category in categories.split():
413
-                options_by_cat.setdefault(category, []).append(opt)
414
-
415
-    return options_by_cat
416
-
417
-
418
-def _get_category_names(package_name):
419
-    package_headers = package_name + '.headers'
420
-    category_names = {}
421
-    for headers_file in ('shared.headers', package_headers):
422
-        try:
423
-            with open(headers_file) as f:
424
-                for line in f:
425
-                    if not line.strip() or line.startswith('#'):
426
-                        continue
427
-                    cat, nice_name = line.split(' ', 1)
428
-                    category_names[cat] = nice_name.strip()
429
-        except IOError:
430
-            print("Cannot open %s (ignored)" % headers_file)
431
-
432
-    return category_names
433
-
434
-
435
-def _format_opt(option):
436
-
437
-    def _remove_prefix(text, prefix):
438
-        if text.startswith(prefix):
439
-            return text[len(prefix):].lstrip(':')
440
-        return text
441
-
442
-    def _reflow_text(text):
443
-        text = re.sub(r'\n+\s*\* ', '$sentinal$* ', text)
444
-        text = text.replace('\n\n', '$sentinal$')
445
-        text = text.replace('\n', ' ')
446
-        text = ' '.join(text.split())
447
-        return text.split('$sentinal$')
448
-
449
-    def _strip_indentation(text):
450
-        return ' '.join([x.strip() for x in text.split('\n')]).strip()
451
-
452
-    help_text = option.help or "No help text available for this option."
453
-    help_text = _remove_prefix(help_text.strip(), 'DEPRECATED')
454
-    help_text = _reflow_text(help_text)
455
-
456
-    deprecated_text = (option.deprecated_reason or
457
-                       'No deprecation reason provided for this option.')
458
-    deprecated_text = _strip_indentation(deprecated_text)
459
-
460
-    opt_type = _TYPE_DESCRIPTIONS.get(type(option), 'Unknown')
461
-
462
-    flags = []
463
-
464
-    if (option.deprecated_for_removal or
465
-            (option.help and option.help.startswith('DEPRECATED'))):
466
-        flags.append(('Deprecated', deprecated_text))
467
-    if option.mutable:
468
-        flags.append(('Mutable', 'This option can be changed without'
469
-                      ' restarting.'))
470
-
471
-    return {
472
-        'name': option.dest,
473
-        'type': opt_type,
474
-        'default': _sanitize_default(option),
475
-        'help': help_text,
476
-        'flags': flags
477
-    }
478
-
479
-
480
-def write_files(package_name, options, target):
481
-    """Write tables.
482
-
483
-    Some projects make use of flagmapping files, while others make use
484
-    of oslo.config's OptGroup to do the same in code. The function will
485
-    handle both, printing a list of options by either category or group.
486
-    """
487
-    target = target or '../../doc/config-reference/source/tables'
488
-    if not os.path.isdir(target):
489
-        os.makedirs(target)
490
-
491
-    if _use_categories(package_name):
492
-        _write_files_by_category(package_name, options, target)
493
-    else:
494
-        _write_files_by_group(package_name, options, target)
495
-
496
-
497
-def _write_files_by_category(package_name, options, target):
498
-    options_by_cat = _get_options_by_cat(package_name)
499
-    category_names = _get_category_names(package_name)
500
-
501
-    for cat in options_by_cat.keys():
502
-        env = {
503
-            'pkg': package_name,
504
-            'cat': cat,
505
-            'label': '-'.join([package_name, cat]),
506
-            'groups': [],
507
-            'items': [],
508
-        }
509
-
510
-        # Skip the options that is explicitly marked as disabled,
511
-        # which is used for common configuration options.
512
-        if cat == 'disable':
513
-            continue
514
-
515
-        if cat in category_names:
516
-            env['nice_cat'] = category_names[cat]
517
-        else:
518
-            env['nice_cat'] = cat
519
-            print("No nicename for %s" % cat)
520
-
521
-        curgroup = None
522
-        items = None
523
-        for optname in options_by_cat[cat]:
524
-            group, option = options.get_option(optname)
525
-
526
-            if group != curgroup:
527
-                if group is not None:
528
-                    curgroup = group
529
-                    env['groups'].append(group)
530
-                    if items is not None:
531
-                        env['items'].append(items)
532
-                items = []
533
-
534
-            items.append(_format_opt(option))
535
-
536
-        env['items'].append(items)
537
-
538
-        file_path = ("%(target)s/%(package_name)s-%(cat)s.rst" %
539
-                     {'target': target, 'package_name': package_name,
540
-                      'cat': cat})
541
-        tmpl_file = os.path.join(os.path.dirname(__file__),
542
-                                 'templates/autohelp-category.rst.j2')
543
-        _write_template(tmpl_file, file_path, env)
544
-
545
-
546
-def _write_files_by_group(package_name, options, target):
547
-    for group in options.get_group_names():
548
-        env = {
549
-            'pkg': package_name,
550
-            'group': group,
551
-            'label': '-'.join([package_name, group]),
552
-            'items': [],
553
-        }
554
-
555
-        for option in options.get_group(group):
556
-            env['items'].append(_format_opt(option))
557
-
558
-        file_path = ("%(target)s/%(package_name)s-%(group)s.rst" %
559
-                     {'target': target, 'package_name': package_name,
560
-                      'group': group})
561
-        tmpl_file = os.path.join(os.path.dirname(__file__),
562
-                                 'templates/autohelp-group.rst.j2')
563
-        _write_template(tmpl_file, file_path, env)
564
-
565
-
566
-def _write_template(template_path, output_path, env):
567
-    with open(template_path) as fd:
568
-        template = jinja2.Template(fd.read(), trim_blocks=True)
569
-        output = template.render(filename=output_path, **env)
570
-
571
-    with open(output_path, 'w') as fd:
572
-        fd.write(output)
573
-
574
-
575
-def update_flagmappings(package_name, options, verbose=0):
576
-    """Update flagmappings file.
577
-
578
-    Update a flagmappings file, adding or removing entries as needed.
579
-    This will create a new file $package_name.flagmappings.new with
580
-    category information merged from the existing $package_name.flagmappings.
581
-    """
582
-    if not _use_categories:
583
-        print("This project does not use flagmappings. Nothing to update.")
584
-        return
585
-
586
-    original_flags = {}
587
-    try:
588
-        with open(package_name + '.flagmappings') as f:
589
-            for line in f:
590
-                try:
591
-                    flag, category = line.split(' ', 1)
592
-                except ValueError:
593
-                    flag = line.strip()
594
-                    category = "Unknown"
595
-                original_flags.setdefault(flag, []).append(category.strip())
596
-    except IOError:
597
-        # If the flags file doesn't exist we'll create it
598
-        pass
599
-
600
-    updated_flags = []
601
-    for opt in options.get_option_names():
602
-        if len(original_flags.get(opt, [])) == 1:
603
-            updated_flags.append((opt, original_flags[opt][0]))
604
-            continue
605
-
606
-        updated_flags.append((opt, 'Unknown'))
607
-
608
-    with open(package_name + '.flagmappings.new', 'w') as f:
609
-        for flag, category in updated_flags:
610
-            f.write(flag + ' ' + category + '\n')
611
-
612
-    if verbose >= 1:
613
-        removed_flags = (set(original_flags.keys()) -
614
-                         set([x[0] for x in updated_flags]))
615
-        added_flags = (set([x[0] for x in updated_flags]) -
616
-                       set(original_flags.keys()))
617
-
618
-        print("\nRemoved Flags\n")
619
-        for line in sorted(removed_flags, OptionsCache._cmpopts):
620
-            print(line)
621
-
622
-        print("\nAdded Flags\n")
623
-        for line in sorted(added_flags, OptionsCache._cmpopts):
624
-            print(line)
625
-
626
-
627
-def main():
628
-    parser = argparse.ArgumentParser(
629
-        description='Manage flag files, to aid in updating documentation.',
630
-        usage='%(prog)s <cmd> <package> [options]')
631
-    parser.add_argument('subcommand',
632
-                        help='Action (update, rst, dump).',
633
-                        choices=['update', 'rst', 'dump'])
634
-    parser.add_argument('package',
635
-                        help='Name of the top-level package.')
636
-    parser.add_argument('-v', '--verbose',
637
-                        action='count',
638
-                        default=0,
639
-                        dest='verbose',
640
-                        required=False,)
641
-    parser.add_argument('-i', '--input',
642
-                        dest='repos',
643
-                        help='Path to a python package in which options '
644
-                             'should be discoverd. Can be used multiple '
645
-                             'times.',
646
-                        required=False,
647
-                        type=str,
648
-                        action='append')
649
-    parser.add_argument('-o', '--output',
650
-                        dest='target',
651
-                        help='Directory or file in which data will be saved.\n'
652
-                             'Defaults to ../../doc/common/tables/ '
653
-                             'for "rst".\n'
654
-                             'Defaults to stdout for "dump"',
655
-                        required=False,
656
-                        type=str,)
657
-    args = parser.parse_args()
658
-
659
-    if args.repos is None:
660
-        args.repos = ['./sources/%s/%s' % args.package]
661
-
662
-    for repository in args.repos:
663
-        package_name = os.path.basename(repository)
664
-        base_path = os.path.dirname(repository)
665
-
666
-        sys.path.insert(0, base_path)
667
-        try:
668
-            __import__(package_name)
669
-        except ImportError as e:
670
-            if args.verbose >= 1:
671
-                print(str(e))
672
-                print("Failed to import: %s (%s)" % (package_name, e))
673
-
674
-        import_modules(base_path, package_name, verbose=args.verbose)
675
-        sys.path.pop(0)
676
-
677
-    overrides = _get_overrides(package_name)
678
-    options = OptionsCache(overrides, verbose=args.verbose)
679
-    options.maybe_load_extensions(args.repos)
680
-
681
-    if args.verbose > 0:
682
-        print("%s options imported from package %s." % (len(options),
683
-                                                        str(package_name)))
684
-
685
-    if args.subcommand == 'update':
686
-        update_flagmappings(args.package, options, verbose=args.verbose)
687
-
688
-    elif args.subcommand == 'rst':
689
-        write_files(args.package, options, args.target)
690
-
691
-    elif args.subcommand == 'dump':
692
-        options.dump()
693
-
694
-
695
-if __name__ == "__main__":
696
-    main()

+ 0
- 289
autogenerate_config_docs/diff_branches.py View File

@@ -1,289 +0,0 @@
1
-#!/usr/bin/env python
2
-# Licensed under the Apache License, Version 2.0 (the "License"); you may
3
-# not use this file except in compliance with the License. You may obtain
4
-# a copy of the License at
5
-#
6
-#      http://www.apache.org/licenses/LICENSE-2.0
7
-#
8
-# Unless required by applicable law or agreed to in writing, software
9
-# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10
-# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11
-# License for the specific language governing permissions and limitations
12
-# under the License.
13
-#
14
-# A collection of tools for working with flags from OpenStack
15
-# packages and documentation.
16
-#
17
-# For an example of usage, run this program with the -h switch.
18
-#
19
-import argparse
20
-import os
21
-import pickle
22
-import subprocess
23
-import sys
24
-
25
-import jinja2
26
-
27
-
28
-PROJECTS = ['aodh', 'ceilometer', 'cinder', 'glance', 'heat', 'ironic',
29
-            'keystone', 'manila', 'neutron', 'nova', 'sahara', 'trove']
30
-MASTER_RELEASE = 'Ocata'
31
-CODENAME_TITLE = {'aodh': 'Alarming',
32
-                  'ceilometer': 'Telemetry',
33
-                  'cinder': 'Block Storage',
34
-                  'glance': 'Image service',
35
-                  'heat': 'Orchestration',
36
-                  'ironic': 'Bare Metal service',
37
-                  'keystone': 'Identity service',
38
-                  'manila': 'Shared File Systems service',
39
-                  'murano': 'Application Catalog service',
40
-                  'neutron': 'Networking',
41
-                  'nova': 'Compute',
42
-                  'sahara': 'Data Processing service',
43
-                  'senlin': 'Clustering service',
44
-                  'trove': 'Database service',
45
-                  'zaqar': 'Message service'}
46
-
47
-
48
-def setup_venv(projects, branch, novenvupdate):
49
-    """Setup a virtual environment for `branch`."""
50
-    dirname = os.path.join('venv', branch.replace('/', '_'))
51
-    if novenvupdate and os.path.exists(dirname):
52
-        return
53
-    if not os.path.exists('venv'):
54
-        os.mkdir('venv')
55
-    args = ["./autohelp-wrapper", "-b", branch, "-e", dirname, "setup"]
56
-    args.extend(projects)
57
-    if subprocess.call(args) != 0:
58
-        print("Impossible to create the %s environment." % branch)
59
-        sys.exit(1)
60
-
61
-
62
-def _get_packages(project, branch):
63
-    release = branch if '/' not in branch else branch.split('/')[1]
64
-    packages = [project]
65
-    try:
66
-        with open('extra_repos/%s-%s.txt' % (project, release)) as f:
67
-            packages.extend([p.strip() for p in f])
68
-    except IOError:
69
-        pass
70
-
71
-    return packages
72
-
73
-
74
-def get_options(project, branch):
75
-    """Get the list of known options for a project."""
76
-    print("Working on %(project)s (%(branch)s)" % {'project': project,
77
-                                                   'branch': branch})
78
-    # And run autohelp script to get a serialized dict of the discovered
79
-    # options
80
-    dirname = os.path.join('venv', branch.replace('/', '_'))
81
-    args = ["./autohelp-wrapper", "-q", "-b", branch, "-e", dirname,
82
-            "dump", project]
83
-
84
-    path = os.environ.get("PATH")
85
-    bin_path = os.path.abspath(os.path.join(dirname, "bin"))
86
-    path = "%s:%s" % (bin_path, path)
87
-    serialized = subprocess.check_output(args,
88
-                                         env={'VIRTUAL_ENV': dirname,
89
-                                              'PATH': path})
90
-    ret = pickle.loads(serialized)
91
-    return ret
92
-
93
-
94
-def _cmpopts(x, y):
95
-    """Compare to option names.
96
-
97
-    The options can be of 2 forms: option_name or group/option_name. Options
98
-    without a group always comes first. Options are sorted alphabetically
99
-    inside a group.
100
-    """
101
-    if '/' in x and '/' in y:
102
-        prex = x[:x.find('/')]
103
-        prey = y[:y.find('/')]
104
-        if prex != prey:
105
-            return cmp(prex, prey)
106
-        return cmp(x, y)
107
-    elif '/' in x:
108
-        return 1
109
-    elif '/' in y:
110
-        return -1
111
-    else:
112
-        return cmp(x, y)
113
-
114
-
115
-def diff(old_list, new_list):
116
-    """Compare the old and new lists of options."""
117
-    new_opts = []
118
-    new_defaults = []
119
-    deprecated_opts = []
120
-    for name, (group, option) in new_list.items():
121
-        # Find the new options
122
-        if name not in old_list.viewkeys():
123
-            new_opts.append(name)
124
-
125
-        # Find the options for which the default value has changed
126
-        elif option['default'] != old_list[name][1]['default']:
127
-            new_defaults.append(name)
128
-
129
-        # Find options that have been deprecated in the new release.
130
-        # If an option name is a key in the old_list dict, it means that it
131
-        # wasn't deprecated.
132
-
133
-        # Some options are deprecated, but not replaced with a new option.
134
-        # These options usually contain 'DEPRECATED' in their help string.
135
-        if 'DEPRECATED' in option['help']:
136
-            deprecated_opts.append((name, None))
137
-
138
-        for deprecated in option['deprecated_opts']:
139
-            # deprecated_opts is a list which always holds at least 1 invalid
140
-            # dict. Forget it.
141
-            if deprecated['name'] is None:
142
-                continue
143
-
144
-            if deprecated['group'] in [None, 'DEFAULT']:
145
-                full_name = deprecated['name']
146
-            else:
147
-                full_name = deprecated['group'] + '/' + deprecated['name']
148
-
149
-            if full_name in old_list.viewkeys():
150
-                deprecated_opts.append((full_name, name))
151
-
152
-    return new_opts, new_defaults, deprecated_opts
153
-
154
-
155
-def format_option_name(name):
156
-    """Return a formatted string for the option path."""
157
-    if name is None:
158
-        return "None"
159
-
160
-    try:
161
-        section, name = name.split('/')
162
-    except ValueError:
163
-        # name without a section ('log_dir')
164
-        return "[DEFAULT] %s" % name
165
-
166
-    return "[%s] %s" % (section, name)
167
-
168
-
169
-def release_from_branch(branch):
170
-    if branch == 'master':
171
-        return MASTER_RELEASE
172
-    else:
173
-        return branch.replace('stable/', '').title()
174
-
175
-
176
-def get_env(project, new_branch, old_list, new_list):
177
-    """Generate the jinja2 environment for the defined branch and project."""
178
-    new_opts, new_defaults, deprecated_opts = diff(old_list, new_list)
179
-    release = release_from_branch(new_branch)
180
-
181
-    env = {
182
-        'release': release,
183
-        'project': project,
184
-        'codename': CODENAME_TITLE[project],
185
-        'new_opts': [],
186
-        'new_defaults': [],
187
-        'deprecated_opts': []
188
-    }
189
-
190
-    # New options
191
-    if new_opts:
192
-        for name in sorted(new_opts, _cmpopts):
193
-            opt = new_list[name][1]
194
-            name = format_option_name(name)
195
-            helptext = opt['help'].strip().replace('\n', ' ')
196
-            helptext = ' '.join(helptext.split())
197
-            cells = (("%(name)s = %(default)s" %
198
-                      {'name': name,
199
-                       'default': opt['default']}).strip(),
200
-                     "(%(type)s) %(help)s" % {'type': opt['type'],
201
-                                              'help': helptext})
202
-            env['new_opts'].append(cells)
203
-
204
-    # New defaults
205
-    if new_defaults:
206
-        for name in sorted(new_defaults, _cmpopts):
207
-            old_default = old_list[name][1]['default']
208
-            new_default = new_list[name][1]['default']
209
-            if isinstance(old_default, list):
210
-                old_default = ", ".join(old_default)
211
-            if isinstance(new_default, list):
212
-                new_default = ", ".join(new_default)
213
-            name = format_option_name(name)
214
-            cells = (name, old_default, new_default)
215
-            env['new_defaults'].append(cells)
216
-
217
-    # Deprecated options
218
-    if deprecated_opts:
219
-        for deprecated, new in sorted(deprecated_opts, cmp=_cmpopts,
220
-                                      key=lambda tup: tup[0]):
221
-            deprecated = format_option_name(deprecated)
222
-            new = format_option_name(new)
223
-            env['deprecated_opts'].append((deprecated, new))
224
-
225
-    return env
226
-
227
-
228
-def main():
229
-    parser = argparse.ArgumentParser(
230
-        description='Generate a summary of configuration option changes.',
231
-        usage='%(prog)s [options] <old_branch> <new_branch> [projects]')
232
-    parser.add_argument('old_branch',
233
-                        help='Name of the old branch.')
234
-    parser.add_argument('new_branch',
235
-                        help='Name of the new branch.')
236
-    parser.add_argument('projects',
237
-                        help='List of projects to work on.',
238
-                        nargs='*',
239
-                        default=PROJECTS)
240
-    parser.add_argument('-i', '--input',
241
-                        dest='sources',
242
-                        help='Path to a folder containing the git '
243
-                             'repositories.',
244
-                        required=False,
245
-                        default='./sources',
246
-                        type=str,)
247
-    parser.add_argument('-o', '--output',
248
-                        dest='target',
249
-                        help='Directory or file in which data will be saved.\n'
250
-                             'Defaults to "."',
251
-                        required=False,
252
-                        default='.',
253
-                        type=str,)
254
-    parser.add_argument('-n', '--no-venv-update',
255
-                        dest='novenvupdate',
256
-                        help='Don\'t update the virtual envs.',
257
-                        required=False,
258
-                        action='store_true',
259
-                        default=False,)
260
-    args = parser.parse_args()
261
-
262
-    setup_venv(args.projects, args.old_branch, args.novenvupdate)
263
-    setup_venv(args.projects, args.new_branch, args.novenvupdate)
264
-
265
-    for project in args.projects:
266
-        old_list = get_options(project, args.old_branch)
267
-        new_list = get_options(project, args.new_branch)
268
-
269
-        release = args.new_branch.replace('stable/', '')
270
-        env = get_env(project, release, old_list, new_list)
271
-        filename = ("%(project)s-conf-changes.rst" %
272
-                    {'project': project})
273
-        tmpl_file = 'templates/changes.rst.j2'
274
-        if not os.path.exists(args.target):
275
-            os.makedirs(args.target)
276
-        dest = os.path.join(args.target, filename)
277
-
278
-        with open(tmpl_file) as fd:
279
-            template = jinja2.Template(fd.read(), trim_blocks=True)
280
-            output = template.render(**env)
281
-
282
-        with open(dest, 'w') as fd:
283
-            fd.write(output)
284
-
285
-    return 0
286
-
287
-
288
-if __name__ == "__main__":
289
-    sys.exit(main())

+ 0
- 5
autogenerate_config_docs/requirements.txt View File

@@ -1,5 +0,0 @@
1
-docutils
2
-jinja2
3
-lxml
4
-oslo.config
5
-oslo.i18n

+ 0
- 45
autogenerate_config_docs/templates/autohelp-category.rst.j2 View File

@@ -1,45 +0,0 @@
1
-..
2
-    Warning: Do not edit this file. It is automatically generated from the
3
-    software project's code and your changes will be overwritten.
4
-
5
-    The tool to generate this file lives in openstack-doc-tools repository.
6
-
7
-    Please make any changes needed in the code, then run the
8
-    autogenerate-config-doc tool from the openstack-doc-tools repository, or
9
-    ask for help on the documentation mailing list, IRC channel or meeting.
10
-
11
-.. _{{ label }}:
12
-
13
-.. list-table:: Description of {{ nice_cat }} configuration options
14
-   :header-rows: 1
15
-   :class: config-ref-table
16
-
17
-   * - Configuration option = Default value
18
-     - Description
19
-{% for group in groups %}
20
-
21
-   * - **[{{ group }}]**
22
-     -
23
-{% for item in items[loop.index0] %}
24
-
25
-{% if item['default'] is equalto '' %}
26
-   * - ``{{ item['name'] }}`` =
27
-{% else %}
28
-   * - ``{{ item['name'] }}`` = ``{{ item['default'] }}``
29
-{% endif %}
30
-{% for paragraph in item['help'] %}
31
-
32
-{% if loop.first %}
33
-     - ({{ item['type'] }}) {{ paragraph }}
34
-{% else %}
35
-       {{ paragraph }}
36
-{% endif %}
37
-{% endfor %}
38
-{% for flagname, flagdesc in item['flags'] %}
39
-
40
-       - **{{ flagname }}**
41
-
42
-         {{ flagdesc }}
43
-{% endfor %}
44
-{% endfor %}
45
-{% endfor %}

+ 0
- 40
autogenerate_config_docs/templates/autohelp-group.rst.j2 View File

@@ -1,40 +0,0 @@
1
-..
2
-    Warning: Do not edit this file. It is automatically generated from the
3
-    software project's code and your changes will be overwritten.
4
-
5
-    The tool to generate this file lives in openstack-doc-tools repository.
6
-
7
-    Please make any changes needed in the code, then run the
8
-    autogenerate-config-doc tool from the openstack-doc-tools repository, or
9
-    ask for help on the documentation mailing list, IRC channel or meeting.
10
-
11
-.. _{{ label }}:
12
-
13
-.. list-table:: Description of {{ group }} configuration options
14
-   :header-rows: 1
15
-   :class: config-ref-table
16
-
17
-   * - Configuration option = Default value
18
-     - Description
19
-{% for item in items %}
20
-
21
-{% if item['default'] is equalto '' %}
22
-   * - ``{{ item['name'] }}`` =
23
-{% else %}
24
-   * - ``{{ item['name'] }}`` = ``{{ item['default'] }}``
25
-{% endif %}
26
-{% for paragraph in item['help'] %}
27
-
28
-{% if loop.first %}
29
-     - ({{ item['type'] }}) {{ paragraph }}
30
-{% else %}
31
-       {{ paragraph }}
32
-{% endif %}
33
-{% endfor %}
34
-{% for flagname, flagdesc in item['flags'] %}
35
-
36
-       - **{{ flagname }}**
37
-
38
-         {{ flagdesc }}
39
-{% endfor %}
40
-{% endfor %}

+ 0
- 61
autogenerate_config_docs/templates/changes.rst.j2 View File

@@ -1,61 +0,0 @@
1
-New, updated, and deprecated options in {{ release }} for {{ codename }}
2
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{{ '~' * release|length }}~~~~~{{ '~' * codename|length }}
3
-
4
-..
5
-  Warning: Do not edit this file. It is automatically generated and your
6
-  changes will be overwritten. The tool to do so lives in the
7
-  openstack-doc-tools repository.
8
-
9
-{% if new_opts %}
10
-.. list-table:: New options
11
-   :header-rows: 1
12
-   :class: config-ref-table
13
-
14
-   * - Option = default value
15
-     - (Type) Help string
16
-{% for cells in new_opts %}
17
-   * - ``{{ cells[0] }}``
18
-     - {{ cells[1] }}
19
-{% endfor %}
20
-{% endif %}
21
-
22
-{% if new_defaults %}
23
-.. list-table:: New default values
24
-   :header-rows: 1
25
-   :class: config-ref-table
26
-
27
-   * - Option
28
-     - Previous default value
29
-     - New default value
30
-{% for cells in new_defaults %}
31
-   * - ``{{ cells[0] }}``
32
-{% if cells[1] is equalto '' %}
33
-     -
34
-{% else %}
35
-     - ``{{ cells[1] }}``
36
-{% endif %}
37
-{% if cells[2] is equalto '' %}
38
-     -
39
-{% else %}
40
-     - ``{{ cells[2] }}``
41
-{% endif %}
42
-{% endfor %}
43
-{% endif %}
44
-
45
-{% if deprecated_opts %}
46
-.. list-table:: Deprecated options
47
-   :header-rows: 1
48
-   :class: config-ref-table
49
-
50
-   * - Deprecated option
51
-     - New Option
52
-{% for cells in deprecated_opts %}
53
-   * - ``{{ cells[0] }}``
54
-     - ``{{ cells[1] }}``
55
-{% endfor %}
56
-{% endif %}
57
-
58
-{% if not new_opts and not new_defaults and not deprecated_opts %}
59
-There are no new, updated, and deprecated options
60
-in {{ release }} for {{ codename }}.
61
-{% endif %}

+ 0
- 1
doc/source/autogenerate_config_docs.rst View File

@@ -1 +0,0 @@
1
-.. include:: ../../autogenerate_config_docs/README.rst

+ 0
- 1
doc/source/index.rst View File

@@ -10,7 +10,6 @@ Contents:
10 10
    doc-tools-readme
11 11
    installation
12 12
    usage
13
-   autogenerate_config_docs
14 13
    man/openstack-doc-test
15 14
    sitemap-readme
16 15
    release_notes

+ 8
- 0
releasenotes/notes/the-great-doc-tools-cleanup-1a79e2c200232489.yaml View File

@@ -0,0 +1,8 @@
1
+---
2
+upgrade:
3
+  - |
4
+    The `autogenerate_config_docs` set of tools has been removed. These were
5
+    always buggy and hard to maintain, and they have been superseded by the
6
+    ``oslo_config.sphinxext`` `Sphinx extension`__.
7
+
8
+    __ https://docs.openstack.org/oslo.config/latest/reference/sphinxext.html

+ 0
- 1
setup.cfg View File

@@ -22,7 +22,6 @@ classifier =
22 22
 [files]
23 23
 packages =
24 24
     os_doc_tools
25
-    autogenerate_config_docs
26 25
 data_files =
27 26
     share/openstack-doc-tools/sitemap = sitemap/*
28 27
     share/openstack-doc-tools/cleanup = cleanup/*

+ 2
- 3
tox.ini View File

@@ -25,8 +25,7 @@ commands =
25 25
     doc8 -e txt -e rst doc/source/ HACKING.rst
26 26
     # Run bashate during pep8 runs to ensure violations are caught by
27 27
     # the check and gate queues.
28
-    bashate autogenerate_config_docs/autohelp-wrapper \
29
-        bin/doc-tools-check-languages \
28
+    bashate bin/doc-tools-check-languages \
30 29
         cleanup/remove_trailing_whitespaces.sh
31 30
 
32 31
 [testenv:pylint]
@@ -58,7 +57,7 @@ usedevelop = False
58 57
 [flake8]
59 58
 show-source = True
60 59
 builtins = _
61
-exclude=.venv,.git,.tox,dist,*lib/python*,*egg,build,*autogenerate_config_docs/venv,*autogenerate_config_docs/sources,doc/source/conf.py
60
+exclude=.venv,.git,.tox,dist,*lib/python*,*egg,build,doc/source/conf.py
62 61
 # 28 is currently the most complex thing we have
63 62
 max-complexity=29
64 63
 ignore = H101

Loading…
Cancel
Save