Enable the merging.txt to be in .rst format for public viewing
This commit is contained in:
Joshua Harlow
@@ -1,6 +1,3 @@
|
||||
Arriving in 0.7.2 is a new way to handle dictionary merging in cloud-init.
|
||||
---
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
@@ -16,32 +13,38 @@ to determine exactly how there objects will be merged.
|
||||
|
||||
For example.
|
||||
|
||||
#cloud-config (1)
|
||||
run_cmd:
|
||||
- bash1
|
||||
- bash2
|
||||
.. code-block:: yaml
|
||||
|
||||
#cloud-config (2)
|
||||
run_cmd:
|
||||
- bash3
|
||||
- bash4
|
||||
#cloud-config (1)
|
||||
run_cmd:
|
||||
- bash1
|
||||
- bash2
|
||||
|
||||
#cloud-config (2)
|
||||
run_cmd:
|
||||
- bash3
|
||||
- bash4
|
||||
|
||||
The previous way of merging the following 2 objects would result in a final
|
||||
cloud-config object that contains the following.
|
||||
|
||||
#cloud-config (merged)
|
||||
run_cmd:
|
||||
- bash3
|
||||
- bash4
|
||||
.. code-block:: yaml
|
||||
|
||||
#cloud-config (merged)
|
||||
run_cmd:
|
||||
- bash3
|
||||
- bash4
|
||||
|
||||
Typically this is not what users want, instead they would likely prefer:
|
||||
|
||||
#cloud-config (merged)
|
||||
run_cmd:
|
||||
- bash1
|
||||
- bash2
|
||||
- bash3
|
||||
- bash4
|
||||
.. code-block:: yaml
|
||||
|
||||
#cloud-config (merged)
|
||||
run_cmd:
|
||||
- bash1
|
||||
- bash2
|
||||
- bash3
|
||||
- bash4
|
||||
|
||||
This way makes it easier to combine the various cloud-config objects you have
|
||||
into a more useful list, thus reducing duplication that would have had to
|
||||
@@ -51,7 +54,7 @@ Customizability
|
||||
---------------
|
||||
|
||||
Since the above merging algorithm may not always be the desired merging
|
||||
algorithm (like how the merging algorithm in < 0.7.2 was not always the preferred
|
||||
algorithm (like how the previous merging algorithm was not always the preferred
|
||||
one) the concept of customizing how merging can be done was introduced through
|
||||
a new concept call 'merge classes'.
|
||||
|
||||
@@ -60,42 +63,44 @@ to merge a given type with another given type.
|
||||
|
||||
An example of one of these merging classes is the following:
|
||||
|
||||
class Merger(object):
|
||||
def __init__(self, merger, opts):
|
||||
self._merger = merger
|
||||
self._overwrite = 'overwrite' in opts
|
||||
.. code-block:: python
|
||||
|
||||
# This merging algorithm will attempt to merge with
|
||||
# another dictionary, on encountering any other type of object
|
||||
# it will not merge with said object, but will instead return
|
||||
# the original value
|
||||
#
|
||||
# On encountering a dictionary, it will create a new dictionary
|
||||
# composed of the original and the one to merge with, if 'overwrite'
|
||||
# is enabled then keys that exist in the original will be overwritten
|
||||
# by keys in the one to merge with (and associated values). Otherwise
|
||||
# if not in overwrite mode the 2 conflicting keys themselves will
|
||||
# be merged.
|
||||
def _on_dict(self, value, merge_with):
|
||||
if not isinstance(merge_with, (dict)):
|
||||
return value
|
||||
merged = dict(value)
|
||||
for (k, v) in merge_with.items():
|
||||
if k in merged:
|
||||
if not self._overwrite:
|
||||
merged[k] = self._merger.merge(merged[k], v)
|
||||
else:
|
||||
merged[k] = v
|
||||
else:
|
||||
merged[k] = v
|
||||
return merged
|
||||
class Merger(object):
|
||||
def __init__(self, merger, opts):
|
||||
self._merger = merger
|
||||
self._overwrite = 'overwrite' in opts
|
||||
|
||||
# This merging algorithm will attempt to merge with
|
||||
# another dictionary, on encountering any other type of object
|
||||
# it will not merge with said object, but will instead return
|
||||
# the original value
|
||||
#
|
||||
# On encountering a dictionary, it will create a new dictionary
|
||||
# composed of the original and the one to merge with, if 'overwrite'
|
||||
# is enabled then keys that exist in the original will be overwritten
|
||||
# by keys in the one to merge with (and associated values). Otherwise
|
||||
# if not in overwrite mode the 2 conflicting keys themselves will
|
||||
# be merged.
|
||||
def _on_dict(self, value, merge_with):
|
||||
if not isinstance(merge_with, (dict)):
|
||||
return value
|
||||
merged = dict(value)
|
||||
for (k, v) in merge_with.items():
|
||||
if k in merged:
|
||||
if not self._overwrite:
|
||||
merged[k] = self._merger.merge(merged[k], v)
|
||||
else:
|
||||
merged[k] = v
|
||||
else:
|
||||
merged[k] = v
|
||||
return merged
|
||||
|
||||
As you can see there is a '_on_dict' method here that will be given a source value
|
||||
and a value to merge with. The result will be the merged object. This code itself
|
||||
is called by another merging class which 'directs' the merging to happen by
|
||||
analyzing the types of the objects to merge and attempting to find a know object
|
||||
that will merge that type. I will avoid pasting that here, but it can be found
|
||||
in the mergers/__init__.py file (see LookupMerger and UnknownMerger).
|
||||
in the `mergers/__init__.py` file (see `LookupMerger` and `UnknownMerger`).
|
||||
|
||||
So following the typical cloud-init way of allowing source code to be downloaded
|
||||
and used dynamically, it is possible for users to inject there own merging files
|
||||
@@ -123,11 +128,14 @@ for your own usage.
|
||||
or as a dictionary (see format below). The keys that are looked up for this
|
||||
definition are the following (in order), 'merge_how', 'merge_type'.
|
||||
|
||||
*String format*
|
||||
String format
|
||||
********
|
||||
|
||||
The string format that is expected is the following.
|
||||
|
||||
"classname(option1,option2)+classname2(option3,option4)" (and so on)
|
||||
::
|
||||
|
||||
classname1(option1,option2)+classname2(option3,option4)....
|
||||
|
||||
The class name there will be connected to class names used when looking for the
|
||||
class that can be used to merge and options provided will be given to the class
|
||||
@@ -135,20 +143,21 @@ on construction of that class.
|
||||
|
||||
For example, the default string that is used when none is provided is the following:
|
||||
|
||||
"list(extend)+dict()+str(append)"
|
||||
::
|
||||
|
||||
list(extend)+dict()+str(append)
|
||||
|
||||
*Dictionary format*
|
||||
Dictionary format
|
||||
********
|
||||
|
||||
In cases where a dictionary can be used to specify the same information as the
|
||||
string format (ie option #2 of above) it can be used, for example.
|
||||
|
||||
merge_how:
|
||||
- name: list
|
||||
settings: [extend]
|
||||
- name: dict
|
||||
settings: []
|
||||
- name: str
|
||||
settings: [append]
|
||||
.. code-block:: python
|
||||
|
||||
{'merge_how': [{'name': 'list', 'settings': ['extend']},
|
||||
{'name': 'dict', 'settings': []},
|
||||
{'name': 'str', 'settings': ['append']}]}
|
||||
|
||||
This would be the equivalent format for default string format but in dictionary
|
||||
form instead of string form.
|
||||
@@ -172,7 +181,7 @@ cloud-config dictionary coming after it.
|
||||
Other uses
|
||||
----------
|
||||
|
||||
The default merging algorithm for merging conf.d yaml files (which form a initial
|
||||
The default merging algorithm for merging 'conf.d' yaml files (which form a initial
|
||||
yaml config for cloud-init) was also changed to use this mechanism so its full
|
||||
benefits (and customization) can also be used there as well. Other places that
|
||||
used the previous merging are also similar now extensible (metadata merging for
|
||||
@@ -24,6 +24,7 @@ Summary
|
||||
topics/examples
|
||||
topics/datasources
|
||||
topics/modules
|
||||
topics/merging
|
||||
topics/moreinfo
|
||||
topics/hacking
|
||||
|
||||
|
||||
5
doc/rtd/topics/merging.rst
Normal file
5
doc/rtd/topics/merging.rst
Normal file
@@ -0,0 +1,5 @@
|
||||
=========
|
||||
Merging
|
||||
=========
|
||||
|
||||
.. include:: ../../merging.rst
|
||||
Reference in New Issue
Block a user