Promote HEAT_ENV
At the moment we're in a conflicted situation where certain config values must be specified in the HEAT_ENV file, but others must be set via env vars. This spec is intended to start a discussion around resolving that. Design for blueprint tripleo-juno-promote-heat-env. Change-Id: Idb49477f4b37474aac11c680a837be17b696a18d
This commit is contained in:
Alexis Lee
parent
f0d02400a1
commit
5c102d440a
|
|
@ -0,0 +1,258 @@
|
|||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
================
|
||||
Promote HEAT_ENV
|
||||
================
|
||||
|
||||
https://blueprints.launchpad.net/tripleo/+spec/tripleo-juno-promote-heat-env
|
||||
|
||||
Promote values set in the Heat environment file to take precedence over
|
||||
input environment variables.
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
Historically TripleO scripts have consulted the environment for many items of
|
||||
configuration. This raises risks of scope leakage and the number of environment
|
||||
variables required often forces users to manage their environment with scripts.
|
||||
Consequently, there's a push to prefer data files like the Heat environment
|
||||
file (HEAT_ENV) which may be set by passing -e to Heat. To allow this file to
|
||||
provide an unambiguous source of truth, the environment must not be allowed to
|
||||
override the values from this file. That is to say, precedence must be
|
||||
transferred.
|
||||
|
||||
A key distinction is whether the value of an environment variable is obtained
|
||||
from the environment passed to it by its parent process (either directly or
|
||||
through derivation). Those which are will be referred to as "input variables"
|
||||
and are deprecated by this spec. Those which are not will be called "local
|
||||
variables" and may be introduced freely. Variables containing values
|
||||
synthesised from multiple sources must be handled on a case-by-case basis.
|
||||
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
||||
Since changes I5b7c8a27a9348d850d1a6e4ab79304cf13697828 and
|
||||
I42a9d4b85edcc99d13f7525e964baf214cdb7cbf, ENV_JSON (the contents of the file
|
||||
named by HEAT_ENV) is constructed in devtest_undercloud.sh like so::
|
||||
|
||||
ENV_JSON=$(jq '.parameters = {
|
||||
"MysqlInnodbBufferPoolSize": 100
|
||||
} + .parameters + {
|
||||
"AdminPassword": "'"${UNDERCLOUD_ADMIN_PASSWORD}"'",
|
||||
"AdminToken": "'"${UNDERCLOUD_ADMIN_TOKEN}"'",
|
||||
"CeilometerPassword": "'"${UNDERCLOUD_CEILOMETER_PASSWORD}"'",
|
||||
"GlancePassword": "'"${UNDERCLOUD_GLANCE_PASSWORD}"'",
|
||||
"HeatPassword": "'"${UNDERCLOUD_HEAT_PASSWORD}"'",
|
||||
"NovaPassword": "'"${UNDERCLOUD_NOVA_PASSWORD}"'",
|
||||
"NeutronPassword": "'"${UNDERCLOUD_NEUTRON_PASSWORD}"'",
|
||||
"NeutronPublicInterface": "'"${NeutronPublicInterface}"'",
|
||||
"undercloudImage": "'"${UNDERCLOUD_ID}"'",
|
||||
"BaremetalArch": "'"${NODE_ARCH}"'",
|
||||
"PowerSSHPrivateKey": "'"${POWER_KEY}"'",
|
||||
"NtpServer": "'"${UNDERCLOUD_NTP_SERVER}"'"
|
||||
}' <<< $ENV_JSON)
|
||||
|
||||
This is broadly equivalent to "A + B + C", where values from B override those
|
||||
from A and values from C override those from either. Currently section C
|
||||
contains a mix of input variables and local variables. It is proposed that
|
||||
current and future environment variables are allocated such that:
|
||||
|
||||
* A only contains default values.
|
||||
* B is the contents of the HEAT_ENV file (from either the user or a prior run).
|
||||
* C only contains computed values (from local variables).
|
||||
|
||||
The following are currently in section C but are not local vars::
|
||||
|
||||
NeutronPublicInterface (default 'eth0')
|
||||
UNDERCLOUD_NTP_SERVER (default '')
|
||||
|
||||
The input variables will be ignored and the defaults moved into section A::
|
||||
|
||||
ENV_JSON=$(jq '.parameters = {
|
||||
"MysqlInnodbBufferPoolSize": 100,
|
||||
"NeutronPublicInterface": "eth0",
|
||||
"NtpServer": ""
|
||||
} + .parameters + {
|
||||
... elided ...
|
||||
}' <<< $ENV_JSON)
|
||||
|
||||
devtest_overcloud.sh will be dealt with similarly. These are the variables
|
||||
which need to be removed and their defaults added to section A::
|
||||
|
||||
OVERCLOUD_NAME (default '')
|
||||
OVERCLOUD_HYPERVISOR_PHYSICAL_BRIDGE (default '')
|
||||
OVERCLOUD_HYPERVISOR_PUBLIC_INTERFACE (default '')
|
||||
OVERCLOUD_BRIDGE_MAPPINGS (default '')
|
||||
OVERCLOUD_FLAT_NETWORKS (default '')
|
||||
NeutronPublicInterface (default 'eth0')
|
||||
OVERCLOUD_LIBVIRT_TYPE (default 'qemu')
|
||||
OVERCLOUD_NTP_SERVER (default '')
|
||||
|
||||
Only one out of all these input variables is used outside of these two scripts
|
||||
and consequently the rest are safe to remove.
|
||||
|
||||
The exception is OVERCLOUD_LIBVIRT_TYPE. This is saved by the script
|
||||
'write-tripleorc'. As it will now be preserved in HEAT_ENV, it does not need to
|
||||
also be preserved by write-tripleorc and can be removed from there.
|
||||
|
||||
----
|
||||
|
||||
So that users know they need to start setting these values through HEAT_ENV
|
||||
rather than input variables, it is further proposed that for an interim period
|
||||
each script echo a message to STDERR if deprecated input variables are set. For
|
||||
example::
|
||||
|
||||
for OLD_VAR in OVERCLOUD_NAME; do
|
||||
if [ ! -z "${!OLD_VAR}" ]; then
|
||||
echo "WARNING: ${OLD_VAR} is deprecated, please set this in the" \
|
||||
"HEAT_ENV file (${HEAT_ENV})" 1>&2
|
||||
fi
|
||||
done
|
||||
|
||||
----
|
||||
|
||||
To separate user input from generated values further, it is proposed that user
|
||||
values be read from a new file - USER_HEAT_ENV. This will default to
|
||||
{under,over}cloud-user-env.json. A new commandline parameter, --user-heat-env,
|
||||
will be added to both scripts so that this can be changed.
|
||||
|
||||
#. ENV_JSON is initialised with default values.
|
||||
#. ENV_JSON is overlaid by HEAT_ENV.
|
||||
#. ENV_JSON is overlaid by USER_HEAT_ENV.
|
||||
#. ENV_JSON is overlaid by computed values.
|
||||
#. ENV_JSON is saved to HEAT_ENV.
|
||||
|
||||
See http://paste.openstack.org/show/83551/ for an example of how to accomplish
|
||||
this. In short::
|
||||
|
||||
ENV_JSON=$(cat ${HEAT_ENV} ${USER_HEAT_ENV} | jq -s '
|
||||
.[0] + .[1] + {"parameters":
|
||||
({..defaults..} + .[0].parameters + {..computed..} + .[1].parameters)}')
|
||||
cat > "${HEAT_ENV}" <<< ${ENV_JSON}
|
||||
|
||||
Choosing to move user data into a new file, compared to moving the merged data,
|
||||
makes USER_HEAT_ENV optional. If users wish, they can continue providing their
|
||||
values in HEAT_ENV. The complementary solution requires users to clean
|
||||
precomputed values out of HEAT_ENV, or they risk unintentionally preventing the
|
||||
values from being recomputed.
|
||||
|
||||
Loading computed values after user values sacrifices user control in favour of
|
||||
correctness. Considering that any devtest user must be rather technical, if a
|
||||
computation is incorrect they can fix or at least hack the computation
|
||||
themselves.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
Instead of removing the input variables entirely, an interim form could be
|
||||
used::
|
||||
|
||||
ENV_JSON=$(jq '.parameters = {
|
||||
"MysqlInnodbBufferPoolSize": 100,
|
||||
"NeutronPublicInterface": "'"${NeutronPublicInterface}"'",
|
||||
"NtpServer": "'"${UNDERCLOUD_NTP_SERVER}"'"
|
||||
} + .parameters + {
|
||||
...
|
||||
}
|
||||
|
||||
However, the input variables would only have an effect if the keys they affect
|
||||
are not present in HEAT_ENV. As HEAT_ENV is written each time devtest runs, the
|
||||
keys will usually be present unless the file is deleted each time (rendering it
|
||||
pointless). So this form is more likely to cause confusion than aid
|
||||
transition.
|
||||
|
||||
----
|
||||
|
||||
jq includes an 'alternative operator', ``//``, which is intended for providing
|
||||
defaults::
|
||||
|
||||
A filter of the form a // b produces the same results as a, if a produces
|
||||
results other than false and null. Otherwise, a // b produces the same
|
||||
results as b.
|
||||
|
||||
This has not been used in the proposal for two reasons:
|
||||
|
||||
#. It only works on individual keys, not whole maps.
|
||||
#. It doesn't work in jq 1.2, still included by Ubuntu 13.04 (Saucy).
|
||||
|
||||
Security Impact
|
||||
---------------
|
||||
|
||||
None.
|
||||
|
||||
Other End User Impact
|
||||
---------------------
|
||||
|
||||
An announcement will be made on the mailing list when this change merges. This
|
||||
coupled with the warnings given if the deprecated variables are set should
|
||||
provide sufficient notice.
|
||||
|
||||
As HEAT_ENV is rewritten every time devtest executes, we can safely assume it
|
||||
matches the last environment used. However users who use scripts to switch
|
||||
their environment may be surprised. Overall the change should be a benefit to
|
||||
these users, as they can use two separate HEAT_ENV files (passing --heat-env to
|
||||
specify which to activate) instead of needing to maintain scripts to set up
|
||||
their environment and risking settings leaking from one to the other.
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
None.
|
||||
|
||||
Other Deployer Impact
|
||||
---------------------
|
||||
|
||||
None.
|
||||
|
||||
Developer Impact
|
||||
----------------
|
||||
|
||||
None.
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
lxsli
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Add USER_HEAT_ENV to both scripts.
|
||||
* Move variables in both scripts.
|
||||
* Add deprecated variables warning to both scripts.
|
||||
* Remove OVERCLOUD_LIBVIRT_TYPE from write-tripleorc.
|
||||
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None.
|
||||
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
The change will be tested in isolation from the rest of the script.
|
||||
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
* Update usage docs with env var deprecation warnings.
|
||||
* Update usage docs to recommend HEAT_ENV.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
#. http://stedolan.github.io/jq/manual/ - JQ manual
|
||||
#. http://jqplay.herokuapp.com/ - JQ interactive demo
|
||||
Loading…
Reference in New Issue