Browse Source

add spec for refactoring configuration templates

Change-Id: I3b00bfea2d7cff9cc00fe0e56e5375b153658b1b
changes/42/213042/16
Jan Klare 3 years ago
parent
commit
994f8f220c
1 changed files with 232 additions and 0 deletions
  1. 232
    0
      specs/liberty/all/refactor_config_templates.rst

+ 232
- 0
specs/liberty/all/refactor_config_templates.rst View File

@@ -0,0 +1,232 @@
1
+..
2
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
3
+ License.
4
+
5
+ http://creativecommons.org/licenses/by/3.0/legalcode
6
+
7
+=================================================
8
+Refactor configuration templates in all cookbooks
9
+=================================================
10
+
11
+URL of launchpad blueprint:
12
+
13
+https://blueprints.launchpad.net/openstack-chef/+spec/refactor-configuration-templates
14
+
15
+Problem description
16
+===================
17
+
18
+Our templates for the configurations files for all services have grown through
19
+the years and it is getting harder and harder to figure out where to set a
20
+specific option that goes into the config files. The same is true for reading
21
+these configuration files after they have been rendered, since they contain a
22
+lot more options and code than actually needed or used. Additionally every time
23
+one of the OpenStack service projects adds or removes an configuration option,
24
+we manually need to add an attribute, some conditions and an entry in the
25
+template for this. Same goes for every change in the defaults, that is either
26
+not reflected at all in our config files, or set explicitly to the same thing,
27
+which is just unneeded code after all.  Finally, the fact that all of our
28
+templates are quite big blocks of code and even contain some hardcoded values
29
+for specific configuration options, which came from the original configuration
30
+files imported years ago, does not help with keeping up with the multiple changes
31
+in the service projects.
32
+
33
+Proposed change
34
+===============
35
+
36
+The templates for OpenStack service configurations files should implement or use
37
+a logic of setting the needed key-value configuration options in the appropriate
38
+section automatically from properly defined chef attributes.
39
+
40
+* logic to render proper configuration files needs to be defined and added
41
+  (example given below)
42
+* attributes used for or in configurations templates need to be checked and
43
+  refactored to fit this logic
44
+* commonly used configuration options should be handled in one place
45
+* only needed configuration options should be set in the configuration files
46
+  after rendering the template
47
+* configuration options that are specific to only a few setups should be removed
48
+  from the default attributes and recipes (especially the switchcases for them)
49
+  and moved to the documentation. Although this will mean to remove
50
+  functionality in the sense of removing some switchcases and attributes in the
51
+  original cookbooks, this should not mean to loose these completely, but rather
52
+  move them to the documentation to allow an easy implementation in wrapper
53
+  cookbooks.  
54
+
55
+Scope
56
+-----
57
+
58
+This spec tries to define a refactoring process that implements a more elegant
59
+handling of template files for OpenStack service configurations throughout all
60
+cookbooks used to deploy OpenStack.
61
+
62
+
63
+End user impact
64
+---------------
65
+
66
+* Users of the cookbooks will be able to wrap all OpenStack chef cookbook more
67
+  easily and set configuration options if needed without patching the actual
68
+  templates.
69
+* Old wrapper cookbooks before this change will stop working partially and
70
+  would need to be refactored accordingly. The same is true for all environment
71
+  files that include configuration options for the services via attributes.
72
+* Reading configuration files will become a lot easier, since only specifically
73
+  wanted and needed options will be included in them.
74
+* The user can add environment and company specific comments in the service
75
+  configuration files via attributes.
76
+
77
+
78
+Developer impact
79
+----------------
80
+
81
+Cookbook developers will not need to add attributes for configuration templates,
82
+nor should there be any need to change existing attributes to fit the changing
83
+defaults in OpenStack service projects. It should become easier to read the
84
+overall cookbook, since a big part of the code in the templates and attribute
85
+files will be cut or moved with this feature. Therefore it might be easier to
86
+contribute as a newcomer without spending hours to understand how all the
87
+attributes work with each other (given that the logic used to render the
88
+templates is easier to understand of course).
89
+
90
+Implementation
91
+==============
92
+
93
+Assignee(s)
94
+-----------
95
+
96
+Primary assignee:
97
+  <j-klare>
98
+
99
+Other contributors:
100
+
101
+Work Items
102
+----------
103
+
104
+* define a generic structure for templates, that can be used for most of the
105
+  configuration files
106
+* main config files for services look like this for all configuration
107
+  options:
108
+
109
+    [section]
110
+    # optional comment or description
111
+    key = value (usually simple boolean, numeric or string)
112
+    # optional comment or description
113
+    key = [ array which could be large ]
114
+
115
+* create a logic to render configuration files from properly set chef attributes
116
+* attributes could look like this for main configuration files (e.g. nova
117
+  or neutron.conf):
118
+
119
+::
120
+
121
+  default['openstack'][service]['conf'][section][key] =
122
+    value
123
+
124
+without a comment:
125
+
126
+::
127
+
128
+  default['openstack'][service]['conf'][section]['debug'] =
129
+    true
130
+
131
+with a comment:
132
+
133
+::
134
+
135
+  default['openstack'][service]['conf'][section]['debug'] =
136
+    { set_to: true, comment: 'this option is the switch to enable/disable debug loggging' }
137
+
138
+cookbook-openstack-common/templates/common.conf.erb :
139
+
140
+::
141
+
142
+  <% @service_config.each do |section, values| -%>
143
+  [<%= section %>]
144
+    <% values.each do |key, value| -%>
145
+      <% if value.class == Hash -%>
146
+  <%= "# {value['comment']}" -%>
147
+  <%= key %> = <%= value['set_to'] %>
148
+      <% else -%>
149
+  <%= key %> = <%= value %>
150
+      <% end -%>
151
+    <% end -%>
152
+  <% end -%>
153
+
154
+cookbook-openstack-compute/recipes/common.rb :
155
+
156
+::
157
+
158
+  ...
159
+  # activesupport currently needs to be upgraded, since the one used in
160
+  # chef 12 is too old (3.2.19) and does not include the needed deep_merge
161
+  chef_gem 'activesupport' do
162
+    action :upgrade
163
+    version '4.2.4'
164
+  end
165
+  require 'active_support'
166
+  ...
167
+
168
+  neutron_admin_password = get_password 'service', 'openstack-network'
169
+  identity_admin_endpoint = admin_endpoint 'identity-admin'
170
+  identity_uri = identity_uri_transform(identity_admin_endpoint)
171
+  ...
172
+
173
+  secrets = {
174
+              neutron: {
175
+                admin_password: neutron_admin_password,
176
+                ...
177
+              },
178
+              keystone_authtoken: {
179
+                identity_uri: identity_uri,
180
+                ...
181
+              },
182
+              ...
183
+            }
184
+
185
+  # merge node attributes with secrets like passwords etc. for
186
+  # usage in template['/etc/nova/nova.conf']
187
+  nova_config_options =
188
+    node['openstack']['compute']['conf'].deep_merge(secrets)
189
+
190
+  template '/etc/nova/nova.conf' do
191
+    source 'common.conf.erb'
192
+    cookbook 'openstack-common'
193
+    owner node['openstack']['compute']['user']
194
+    group node['openstack']['compute']['group']
195
+    mode 00640
196
+    variables(
197
+      service_config: nova_config_options
198
+    )
199
+  end
200
+  ...
201
+
202
+cookbook-openstack-compute/templates/nova.conf.erb -- not needed anymore
203
+
204
+* add a link to documentation/config reference to all config files
205
+* refactor currently used attributes to fit into that logic
206
+* adapt specs
207
+* define a set of minimal needed attributes to create a working stack and move
208
+  the rest of the attributes into documentation
209
+* remove attributes that would set configuration options that are equal to the
210
+  defaults
211
+* propagate not backward compatible change at a fitting point in time without
212
+  making a lot of people angry
213
+
214
+
215
+Testing
216
+=======
217
+
218
+* lint and style tests with rubocop (as is)
219
+* unit tests with chefspec with special focus on testing the proper rendering of
220
+  the configuration templates (including the sections)
221
+* integration testing with chef provisioning
222
+
223
+
224
+Documentation Impact
225
+====================
226
+
227
+These patches should move a good part of the currently defined attributes to the
228
+documentation as examples for specific setups.
229
+
230
+
231
+References
232
+==========

Loading…
Cancel
Save