Configurable /mnt/state
This spec covers making the top level directory where persistent and stateful data is written to configurable. Currently this top level path is hardcoded to /mnt/state in our os-apply-config templates, os-refresh-config scripts, and other element scripts. Change-Id: Ifdaf31a97a76d69d89c0b5f48714cedfb3cf7b06
This commit is contained in:
James Slagle
Alexis Lee
parent
91e9a49208
commit
47e2460d4c
|
|
@ -0,0 +1,238 @@
|
|||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
=======================================================
|
||||
Configurable directory for persistent and stateful data
|
||||
=======================================================
|
||||
|
||||
https://blueprints.launchpad.net/tripleo/+spec/tripleo-juno-configurable-mnt-state
|
||||
|
||||
Make the hardcoded /mnt/state path for stateful data be configurable.
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
1. A hard coded directory of /mnt/state for persistent data is incompatible
|
||||
with Red Hat based distros available mechanism for a stateful data path. Red
|
||||
Hat based distros, such as Fedora, RHEL, and CentOS, have a feature that uses
|
||||
bind mounts for mounting paths onto a stateful data partition and does not
|
||||
require manually reconfiguring software to use /mnt/state.
|
||||
|
||||
2. Distros that use SELinux have pre-existing policy that allows access to
|
||||
specific paths. Reconfiguring these paths to be under /mnt/state, results
|
||||
in SELinux denials for existing services, requiring additional policy to be
|
||||
written and maintained.
|
||||
|
||||
3. Some operators and administrators find the reconfiguring of many services to
|
||||
not use well known default values for filesystem paths to be disruptive and
|
||||
inconsistent. They do not expect these changes when using a distro that they've
|
||||
come to learn and anticipate certain configurations. These types of changes
|
||||
also require documentation changes to existing documents and processes.
|
||||
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
Deployers will be able to choose a configurable path instead of the hardcoded
|
||||
value of /mnt/state for the stateful path.
|
||||
|
||||
A new element, stateful-path will be added that defines the value for the
|
||||
stateful path. The default will be /mnt/state.
|
||||
|
||||
There are 3 areas that need to respect the configurable path:
|
||||
|
||||
os-apply-config template generation
|
||||
The stateful-path element will set the stateful path value by installing a
|
||||
JSON file to a well known location for os-collect-config to use as a local
|
||||
data source. This will require a new local data source collector to be added
|
||||
to os-collect-config (See `Dependencies`_).
|
||||
|
||||
The JSON file's contents will be based on $STATEFUL_PATH, e.g.:
|
||||
|
||||
{'stateful-path': '/mnt/state'}
|
||||
|
||||
File templates (files under os-apply-config in an element) will then be
|
||||
updated to replace the hard coded /mnt/state with {{stateful-path}}.
|
||||
|
||||
Currently, there is a mix of root locations of the os-apply-config templates.
|
||||
Most are written under /, although some are written under /mnt/state. The
|
||||
/mnt/state is hard coded in the directory tree under os-apply-config in these
|
||||
elements, so this will be removed to have the templates just written under /.
|
||||
Symlinks could instead be used in these elements to setup the correct paths.
|
||||
Support can also be added to os-apply-config's control file mechanism to
|
||||
indicate these files should be written under the stateful path. An example
|
||||
patch that does this is at: https://review.openstack.org/#/c/113651/
|
||||
|
||||
os-refresh-config scripts run at boot time
|
||||
In order to make the stateful path configurable, all of the hard coded
|
||||
references to /mnt/state in os-refresh-config scripts will be replaced with an
|
||||
environment variable, $STATEFUL_PATH.
|
||||
|
||||
The stateful-path element will provide an environment.d script for
|
||||
os-refresh-config that reads the value from os-apply-config:
|
||||
|
||||
export STATEFUL_PATH=$(os-apply-config --key stateful-path --type raw)
|
||||
|
||||
Hook scripts run at image build time
|
||||
The stateful-path element will provide an environment.d script for use at
|
||||
image build time:
|
||||
|
||||
export STATEFUL_PATH=${STATEFUL_PATH:-"/mnt/state"}
|
||||
|
||||
The use-ephemeral element will depend on the stateful-path element, effectively
|
||||
making the default stateful path remain /mnt/state.
|
||||
|
||||
The stateful path can be reconfigured by defining $STATEFUL_PATH either A) in
|
||||
the environment before an image build; or B) in an element with an
|
||||
environment.d script which runs earlier than the stateful-path environment.d
|
||||
script.
|
||||
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
None come to mind, the point of this spec is to enable an alternative to what's
|
||||
already existing. There may be additional alternatives out there other folks
|
||||
may wish to add support for.
|
||||
|
||||
Security Impact
|
||||
---------------
|
||||
None
|
||||
|
||||
Other End User Impact
|
||||
---------------------
|
||||
End users using elements that change the stateful path location from /mnt/state
|
||||
to something else will see this change reflected in configuration files and in
|
||||
the directories used for persistent and stateful data. They will have to know
|
||||
how the stateful path is configured and accessed.
|
||||
|
||||
Different TripleO installs would appear different if used with elements that
|
||||
configured the stateful path differently.
|
||||
|
||||
This also adds some complexity when reading TripleO code, because instead of
|
||||
there being an explicit path, there would instead be a reference to a
|
||||
configurable value.
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
There will be additional logic in os-refresh-config to determine and set the
|
||||
stateful path, and an additional local collector that os-collect-config would
|
||||
use. However, these are negligible in terms of negatively impacting
|
||||
performance.
|
||||
|
||||
|
||||
Other Deployer Impact
|
||||
---------------------
|
||||
Deployers will be able to choose different elements that may reconfigure the
|
||||
stateful path or change the value for $STATEFUL_PATH. The default will remain
|
||||
unchanged however.
|
||||
|
||||
Deployers would have to know what the stateful path is, and if it's different
|
||||
across their environment, this could be confusing. However, this seems unlikely
|
||||
as deployers are likely to be standardizing on one set of common elements,
|
||||
distro, etc.
|
||||
|
||||
In the future, if TripleO CI and CD clouds that are based on Red Hat distros
|
||||
make use of this feature to enable Red Hat read only root support, then these
|
||||
clouds would be configured differently from clouds that are configured to use
|
||||
/mnt/state. As a team, the tripleo-cd-admins will have to know which
|
||||
configuration has been used.
|
||||
|
||||
Developer Impact
|
||||
----------------
|
||||
1. Developers need to use the $STATEFUL_PATH and {{stateful-path}}
|
||||
substitutions when they intend to refer to the stateful path.
|
||||
|
||||
2. Code that needs to know the stateful path will need access to the variable
|
||||
defining the path, it won't be able to assume the path is /mnt/state. A call to
|
||||
os-apply-config to query the key defining the path could be done to get
|
||||
the value, as long as os-collect-config has already run at least once.
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
james-slagle
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
tripleo-incubator
|
||||
^^^^^^^^^^^^^^^^^
|
||||
* Update troubleshooting docs to mention that /mnt/state is a configurable
|
||||
path, and could be different in local environments.
|
||||
|
||||
tripleo-image-elements
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
* Add a new stateful-path element that configures stateful-path and $STATEFUL_PATH
|
||||
to /mnt/state
|
||||
* Update os-apply-config templates to replace /mnt/state with {{stateful-path}}
|
||||
* Update os-refresh-config scripts to replace /mnt/state with $STATEFUL_PATH
|
||||
* Update all elements that have os-apply-config template files under /mnt/state
|
||||
to just be under /.
|
||||
|
||||
* update os-apply-config element to call os-apply-config with a --root
|
||||
$STATEFUL_PATH option
|
||||
* update elements that have paths to os-apply-config generated files (such
|
||||
as /etc/nova/nova.conf) to refer to those paths as
|
||||
$STATEFUL_PATH/path/to/file.
|
||||
|
||||
* make use-ephemeral element depend on stateful-path element
|
||||
|
||||
Dependencies
|
||||
============
|
||||
1. os-collect-config will need a new feature to read from a local data source
|
||||
directory that elements can install JSON files into, such as a source.d. There
|
||||
will be a new spec filed on this feature.
|
||||
https://review.openstack.org/#/c/100965/
|
||||
|
||||
2. os-apply-config will need an option in its control file to support
|
||||
generating templates under the configurable stateful path. There is a patch
|
||||
here: https://review.openstack.org/#/c/113651/
|
||||
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
There is currently no testing that all stateful and persistent data is actually
|
||||
written to a stateful partition.
|
||||
|
||||
We should add tempest tests that directly exercise the preserve_ephemeral
|
||||
option, and have tests that check that all stateful data has been preserved
|
||||
across a "nova rebuild". Tempest seems like a reasonable place to add these
|
||||
tests since preserve_ephemeral is a Nova OpenStack feature. Plus, once TripleO
|
||||
CI is running tempest against the deployed OverCloud, we will be testing this
|
||||
feature.
|
||||
|
||||
We should also test in TripleO CI that state is preserved across a rebuild by
|
||||
adding stateful data before a rebuild and verifying it is still present after a
|
||||
rebuild.
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
We will document the new stateful-path element.
|
||||
|
||||
TripleO documentation will need to mention the potential difference in
|
||||
configuration files and the location of persistent data if a value other than
|
||||
/mnt/state is used.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
os-collect-config local datasource collector spec:
|
||||
|
||||
* https://review.openstack.org/100965
|
||||
|
||||
Red Hat style stateful partition support this will enable:
|
||||
|
||||
* https://git.fedorahosted.org/cgit/initscripts.git/tree/systemd/fedora-readonly
|
||||
* https://git.fedorahosted.org/cgit/initscripts.git/tree/sysconfig/readonly-root
|
||||
* https://git.fedorahosted.org/cgit/initscripts.git/tree/statetab
|
||||
* https://git.fedorahosted.org/cgit/initscripts.git/tree/rwtab
|
||||
Loading…
Reference in New Issue