config: Add support for config from environment variables
The spec proposes a model for getting configuration/option settings/values from environment variables in the execution environment. blueprint config-from-environment Change-Id: I56acd64a29dc4eb1c4aeb82669b537efedcbe556
This commit is contained in:
parent
9e3edb2500
commit
81484e3591
|
@ -0,0 +1,223 @@
|
||||||
|
|
||||||
|
==============================
|
||||||
|
Config Opts From Environment
|
||||||
|
==============================
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/oslo.config/+spec/config-from-environment
|
||||||
|
|
||||||
|
It is common and convenient in service management, especially container
|
||||||
|
orchestration, to manage configuration settings using environment variables set
|
||||||
|
by the controlling environment. This allows the service being managed to avoid
|
||||||
|
extraneous files and other artifacts, allowing service immutability and thus
|
||||||
|
straightforward elasticity. This specification proposes using the new
|
||||||
|
`drivers`_ functionality provided by ``oslo.config`` to automatically support
|
||||||
|
overriding or setting configuration from environment variables with standard
|
||||||
|
and predictable names.
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
A common way to pass instance specific settings to a container is to use
|
||||||
|
shell environment variables. For example, in docker::
|
||||||
|
|
||||||
|
docker run -t -p 127.0.0.1:8081:80 \
|
||||||
|
-e "DB_SYNC=True" \
|
||||||
|
-e "AUTH_STRATEGY=noauth2" \
|
||||||
|
myservice
|
||||||
|
|
||||||
|
or in a (fragment of a) Kubernetes deployment::
|
||||||
|
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: myservice
|
||||||
|
image: myservice
|
||||||
|
env:
|
||||||
|
- name: DB_SYNC
|
||||||
|
value: "True"
|
||||||
|
- name: AUTH_STRATEGY
|
||||||
|
value: noauth2
|
||||||
|
ports:
|
||||||
|
- containerPort: 80
|
||||||
|
|
||||||
|
In OpenStack these settings are usually managed by config files read by
|
||||||
|
``oslo.config`` into instances of ``ConfigOpts`` classes. At this time there is
|
||||||
|
no unified way to use environment variables for these settings, meaning that in
|
||||||
|
many cases if a deployer would like to run an OpenStack service in a container,
|
||||||
|
they must provide a configuration file for that container, or provide some way
|
||||||
|
for the service to gather configuration at run time.
|
||||||
|
|
||||||
|
The `drivers`_ functionality being developed for ``oslo.config`` provides one
|
||||||
|
model for that run time configuration, but does not specifically address
|
||||||
|
the case of using simple environment variables.
|
||||||
|
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
The spec proposes using the drivers model to create a ``ConfigurationSource``
|
||||||
|
for ``oslo.config`` that looks to the running environment for variables that
|
||||||
|
match expected names for registered configuration settings. The source will be
|
||||||
|
available by default, ideally coming second in the stack, after individual
|
||||||
|
command line overrides, but before any files or other sources. If this proves
|
||||||
|
impossible because of inter-dependencies in the file and command line handling,
|
||||||
|
first in the stack is the second-best option.
|
||||||
|
|
||||||
|
The ``ConfigurationSource`` will translate config option names to candidate
|
||||||
|
variable names. If those variables are set in the local environment (a member
|
||||||
|
of ``os.environ`` in Python) the values will be returned. If they are not set,
|
||||||
|
further processing of other sources will proceed.
|
||||||
|
|
||||||
|
There are two main challenges with this proposal. Resolving them in a
|
||||||
|
satisfactory fashion is why this spec is being written. They are:
|
||||||
|
|
||||||
|
#. Determining a satisfactory scheme for translating configuration option names
|
||||||
|
to environment variable names in a way that is both predictable for humans
|
||||||
|
and highly unlikely to collide with variable names that might otherwise be
|
||||||
|
used. A strawman proposal is as follows:
|
||||||
|
|
||||||
|
#. Prefix each variable with ``OS_``
|
||||||
|
#. Followed by the group: ``DEFAULT``
|
||||||
|
#. Separated from the name by a double-under ``__`` (to allow unders in the
|
||||||
|
group)
|
||||||
|
#. Followed by the name, resulting in something like:
|
||||||
|
``OS_PLACEMENT_DATABASE__CONNECTION``
|
||||||
|
|
||||||
|
This will result in quite long names in some cases, but that's likely going
|
||||||
|
to be the case with any solution that satisfies both requirements. Note that
|
||||||
|
this format can be transformed in both directions: from option group and
|
||||||
|
name to environment variable name and vice versa.
|
||||||
|
|
||||||
|
#. Individual options in a ``ConfigOpts`` have types. Environment variables, on
|
||||||
|
the other hand, are strings. We either need to establish a way of coercing
|
||||||
|
string-based representations of non-string types to the desired type
|
||||||
|
(`keystonemiddleware`_ has some code that does some of this) or we perhaps
|
||||||
|
initially only support ``StrOpt``.
|
||||||
|
|
||||||
|
Here is a contrived example that describes two options which have similar names
|
||||||
|
that demonstrates the importance of group handling. Configuration items as
|
||||||
|
follows::
|
||||||
|
|
||||||
|
[placement]
|
||||||
|
database_connection = foo
|
||||||
|
|
||||||
|
[placement_database]
|
||||||
|
connection = bar
|
||||||
|
|
||||||
|
would result in two environment variables ``OS_PLACEMENT__DATABASE_CONNECTION``
|
||||||
|
and ``OS_PLACEMENT_DATABASE__CONNECTION``.
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
This is not something that has to be done in ``oslo.config``. Individual
|
||||||
|
services could manage their own environment checking, but that's not really
|
||||||
|
in keeping with the oslo principles nor the consistency goals of OpenStack.
|
||||||
|
|
||||||
|
In some circumstances an ``Opt`` could set a default that reads from the
|
||||||
|
environment using ``default=os.environ.get('SOMETHING')`` but this has a
|
||||||
|
critical disadvantage: The environment variable is only used if the option is
|
||||||
|
not already set in the configuration. Ideally the environment variable should
|
||||||
|
override configuration.
|
||||||
|
|
||||||
|
Another way to do this would be to add another parameter to ``Opt`` named
|
||||||
|
``envvar``. If set, the value would be used to override the automatically
|
||||||
|
generated environment variable name (as described above). This is considered
|
||||||
|
undesirable as it made lead to published inconsistencies in variable naming.
|
||||||
|
|
||||||
|
|
||||||
|
Impact on Existing APIs
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
See above.
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
There's a slim chance that service behavior could change if pre-existing
|
||||||
|
environment variables are present that happen to match the naming scheme
|
||||||
|
described here.
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
No significant performance impact is expected. Querying the environment is
|
||||||
|
quick and the querying is done as needed, not for all possible configuration
|
||||||
|
values.
|
||||||
|
|
||||||
|
Configuration Impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
No new options will be added when registering options. Support for environment
|
||||||
|
variable-based overrides will be automatic.
|
||||||
|
|
||||||
|
Developer Impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Developers will have an additional customization option available.
|
||||||
|
|
||||||
|
Testing Impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
Additional unit tests will be required to cover the added functionality.
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
cdent
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
volunteers?
|
||||||
|
|
||||||
|
Milestones
|
||||||
|
----------
|
||||||
|
|
||||||
|
..TODO(cdent): figure this out
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Implement the new ``ConfigurationSource`` driver.
|
||||||
|
* Integrate it as a default driver.
|
||||||
|
* Update documentation.
|
||||||
|
* Update the sample configuration generator to include the variable names.
|
||||||
|
* Update the documentation generator to include the variable names.
|
||||||
|
|
||||||
|
|
||||||
|
Incubation
|
||||||
|
==========
|
||||||
|
|
||||||
|
N/A
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
The documentation will need to be updated to indicate that each option can be
|
||||||
|
overridden with an environment variable and to describe how the name of the
|
||||||
|
variable will be generated.
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
This implementation is dependent on the emerging `drivers`_ functionality in
|
||||||
|
``oslo.config``.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
* Backend Drivers for oslo.config:
|
||||||
|
`<http://specs.openstack.org/openstack/oslo-specs/specs/queens/oslo-config-drivers.html>`_
|
||||||
|
* Keystone Middleware option type coercing:
|
||||||
|
`<http://git.openstack.org/cgit/openstack/keystonemiddleware/tree/keystonemiddleware/_common/config.py>`_
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0
|
||||||
|
Unported License.
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
.. _drivers: http://specs.openstack.org/openstack/oslo-specs/specs/queens/oslo-config-drivers.html
|
||||||
|
.. _keystonemiddleware: http://git.openstack.org/cgit/openstack/keystonemiddleware/tree/keystonemiddleware/_common/config.py
|
Loading…
Reference in New Issue