Configurable /mnt/state
This spec covers making the top level directory where persistent and stateful data is written to configurable. Currently this top level path is hardcoded to /mnt/state in our os-apply-config templates, os-refresh-config scripts, and other element scripts. Change-Id: Ifdaf31a97a76d69d89c0b5f48714cedfb3cf7b06
This commit is contained in:
James Slagle
Alexis Lee
parent
91e9a49208
commit
47e2460d4c
238
specs/juno/tripleo-juno-configurable-mnt-state.rst
Normal file
238
specs/juno/tripleo-juno-configurable-mnt-state.rst
Normal file
@ -0,0 +1,238 @@
|
|||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
=======================================================
|
||||||
|
Configurable directory for persistent and stateful data
|
||||||
|
=======================================================
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/tripleo/+spec/tripleo-juno-configurable-mnt-state
|
||||||
|
|
||||||
|
Make the hardcoded /mnt/state path for stateful data be configurable.
|
||||||
|
|
||||||
|
Problem Description
|
||||||
|
===================
|
||||||
|
|
||||||
|
1. A hard coded directory of /mnt/state for persistent data is incompatible
|
||||||
|
with Red Hat based distros available mechanism for a stateful data path. Red
|
||||||
|
Hat based distros, such as Fedora, RHEL, and CentOS, have a feature that uses
|
||||||
|
bind mounts for mounting paths onto a stateful data partition and does not
|
||||||
|
require manually reconfiguring software to use /mnt/state.
|
||||||
|
|
||||||
|
2. Distros that use SELinux have pre-existing policy that allows access to
|
||||||
|
specific paths. Reconfiguring these paths to be under /mnt/state, results
|
||||||
|
in SELinux denials for existing services, requiring additional policy to be
|
||||||
|
written and maintained.
|
||||||
|
|
||||||
|
3. Some operators and administrators find the reconfiguring of many services to
|
||||||
|
not use well known default values for filesystem paths to be disruptive and
|
||||||
|
inconsistent. They do not expect these changes when using a distro that they've
|
||||||
|
come to learn and anticipate certain configurations. These types of changes
|
||||||
|
also require documentation changes to existing documents and processes.
|
||||||
|
|
||||||
|
|
||||||
|
Proposed Change
|
||||||
|
===============
|
||||||
|
Deployers will be able to choose a configurable path instead of the hardcoded
|
||||||
|
value of /mnt/state for the stateful path.
|
||||||
|
|
||||||
|
A new element, stateful-path will be added that defines the value for the
|
||||||
|
stateful path. The default will be /mnt/state.
|
||||||
|
|
||||||
|
There are 3 areas that need to respect the configurable path:
|
||||||
|
|
||||||
|
os-apply-config template generation
|
||||||
|
The stateful-path element will set the stateful path value by installing a
|
||||||
|
JSON file to a well known location for os-collect-config to use as a local
|
||||||
|
data source. This will require a new local data source collector to be added
|
||||||
|
to os-collect-config (See `Dependencies`_).
|
||||||
|
|
||||||
|
The JSON file's contents will be based on $STATEFUL_PATH, e.g.:
|
||||||
|
|
||||||
|
{'stateful-path': '/mnt/state'}
|
||||||
|
|
||||||
|
File templates (files under os-apply-config in an element) will then be
|
||||||
|
updated to replace the hard coded /mnt/state with {{stateful-path}}.
|
||||||
|
|
||||||
|
Currently, there is a mix of root locations of the os-apply-config templates.
|
||||||
|
Most are written under /, although some are written under /mnt/state. The
|
||||||
|
/mnt/state is hard coded in the directory tree under os-apply-config in these
|
||||||
|
elements, so this will be removed to have the templates just written under /.
|
||||||
|
Symlinks could instead be used in these elements to setup the correct paths.
|
||||||
|
Support can also be added to os-apply-config's control file mechanism to
|
||||||
|
indicate these files should be written under the stateful path. An example
|
||||||
|
patch that does this is at: https://review.openstack.org/#/c/113651/
|
||||||
|
|
||||||
|
os-refresh-config scripts run at boot time
|
||||||
|
In order to make the stateful path configurable, all of the hard coded
|
||||||
|
references to /mnt/state in os-refresh-config scripts will be replaced with an
|
||||||
|
environment variable, $STATEFUL_PATH.
|
||||||
|
|
||||||
|
The stateful-path element will provide an environment.d script for
|
||||||
|
os-refresh-config that reads the value from os-apply-config:
|
||||||
|
|
||||||
|
export STATEFUL_PATH=$(os-apply-config --key stateful-path --type raw)
|
||||||
|
|
||||||
|
Hook scripts run at image build time
|
||||||
|
The stateful-path element will provide an environment.d script for use at
|
||||||
|
image build time:
|
||||||
|
|
||||||
|
export STATEFUL_PATH=${STATEFUL_PATH:-"/mnt/state"}
|
||||||
|
|
||||||
|
The use-ephemeral element will depend on the stateful-path element, effectively
|
||||||
|
making the default stateful path remain /mnt/state.
|
||||||
|
|
||||||
|
The stateful path can be reconfigured by defining $STATEFUL_PATH either A) in
|
||||||
|
the environment before an image build; or B) in an element with an
|
||||||
|
environment.d script which runs earlier than the stateful-path environment.d
|
||||||
|
script.
|
||||||
|
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
None come to mind, the point of this spec is to enable an alternative to what's
|
||||||
|
already existing. There may be additional alternatives out there other folks
|
||||||
|
may wish to add support for.
|
||||||
|
|
||||||
|
Security Impact
|
||||||
|
---------------
|
||||||
|
None
|
||||||
|
|
||||||
|
Other End User Impact
|
||||||
|
---------------------
|
||||||
|
End users using elements that change the stateful path location from /mnt/state
|
||||||
|
to something else will see this change reflected in configuration files and in
|
||||||
|
the directories used for persistent and stateful data. They will have to know
|
||||||
|
how the stateful path is configured and accessed.
|
||||||
|
|
||||||
|
Different TripleO installs would appear different if used with elements that
|
||||||
|
configured the stateful path differently.
|
||||||
|
|
||||||
|
This also adds some complexity when reading TripleO code, because instead of
|
||||||
|
there being an explicit path, there would instead be a reference to a
|
||||||
|
configurable value.
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
There will be additional logic in os-refresh-config to determine and set the
|
||||||
|
stateful path, and an additional local collector that os-collect-config would
|
||||||
|
use. However, these are negligible in terms of negatively impacting
|
||||||
|
performance.
|
||||||
|
|
||||||
|
|
||||||
|
Other Deployer Impact
|
||||||
|
---------------------
|
||||||
|
Deployers will be able to choose different elements that may reconfigure the
|
||||||
|
stateful path or change the value for $STATEFUL_PATH. The default will remain
|
||||||
|
unchanged however.
|
||||||
|
|
||||||
|
Deployers would have to know what the stateful path is, and if it's different
|
||||||
|
across their environment, this could be confusing. However, this seems unlikely
|
||||||
|
as deployers are likely to be standardizing on one set of common elements,
|
||||||
|
distro, etc.
|
||||||
|
|
||||||
|
In the future, if TripleO CI and CD clouds that are based on Red Hat distros
|
||||||
|
make use of this feature to enable Red Hat read only root support, then these
|
||||||
|
clouds would be configured differently from clouds that are configured to use
|
||||||
|
/mnt/state. As a team, the tripleo-cd-admins will have to know which
|
||||||
|
configuration has been used.
|
||||||
|
|
||||||
|
Developer Impact
|
||||||
|
----------------
|
||||||
|
1. Developers need to use the $STATEFUL_PATH and {{stateful-path}}
|
||||||
|
substitutions when they intend to refer to the stateful path.
|
||||||
|
|
||||||
|
2. Code that needs to know the stateful path will need access to the variable
|
||||||
|
defining the path, it won't be able to assume the path is /mnt/state. A call to
|
||||||
|
os-apply-config to query the key defining the path could be done to get
|
||||||
|
the value, as long as os-collect-config has already run at least once.
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
james-slagle
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
tripleo-incubator
|
||||||
|
^^^^^^^^^^^^^^^^^
|
||||||
|
* Update troubleshooting docs to mention that /mnt/state is a configurable
|
||||||
|
path, and could be different in local environments.
|
||||||
|
|
||||||
|
tripleo-image-elements
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
* Add a new stateful-path element that configures stateful-path and $STATEFUL_PATH
|
||||||
|
to /mnt/state
|
||||||
|
* Update os-apply-config templates to replace /mnt/state with {{stateful-path}}
|
||||||
|
* Update os-refresh-config scripts to replace /mnt/state with $STATEFUL_PATH
|
||||||
|
* Update all elements that have os-apply-config template files under /mnt/state
|
||||||
|
to just be under /.
|
||||||
|
|
||||||
|
* update os-apply-config element to call os-apply-config with a --root
|
||||||
|
$STATEFUL_PATH option
|
||||||
|
* update elements that have paths to os-apply-config generated files (such
|
||||||
|
as /etc/nova/nova.conf) to refer to those paths as
|
||||||
|
$STATEFUL_PATH/path/to/file.
|
||||||
|
|
||||||
|
* make use-ephemeral element depend on stateful-path element
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
1. os-collect-config will need a new feature to read from a local data source
|
||||||
|
directory that elements can install JSON files into, such as a source.d. There
|
||||||
|
will be a new spec filed on this feature.
|
||||||
|
https://review.openstack.org/#/c/100965/
|
||||||
|
|
||||||
|
2. os-apply-config will need an option in its control file to support
|
||||||
|
generating templates under the configurable stateful path. There is a patch
|
||||||
|
here: https://review.openstack.org/#/c/113651/
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
There is currently no testing that all stateful and persistent data is actually
|
||||||
|
written to a stateful partition.
|
||||||
|
|
||||||
|
We should add tempest tests that directly exercise the preserve_ephemeral
|
||||||
|
option, and have tests that check that all stateful data has been preserved
|
||||||
|
across a "nova rebuild". Tempest seems like a reasonable place to add these
|
||||||
|
tests since preserve_ephemeral is a Nova OpenStack feature. Plus, once TripleO
|
||||||
|
CI is running tempest against the deployed OverCloud, we will be testing this
|
||||||
|
feature.
|
||||||
|
|
||||||
|
We should also test in TripleO CI that state is preserved across a rebuild by
|
||||||
|
adding stateful data before a rebuild and verifying it is still present after a
|
||||||
|
rebuild.
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
We will document the new stateful-path element.
|
||||||
|
|
||||||
|
TripleO documentation will need to mention the potential difference in
|
||||||
|
configuration files and the location of persistent data if a value other than
|
||||||
|
/mnt/state is used.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
os-collect-config local datasource collector spec:
|
||||||
|
|
||||||
|
* https://review.openstack.org/100965
|
||||||
|
|
||||||
|
Red Hat style stateful partition support this will enable:
|
||||||
|
|
||||||
|
* https://git.fedorahosted.org/cgit/initscripts.git/tree/systemd/fedora-readonly
|
||||||
|
* https://git.fedorahosted.org/cgit/initscripts.git/tree/sysconfig/readonly-root
|
||||||
|
* https://git.fedorahosted.org/cgit/initscripts.git/tree/statetab
|
||||||
|
* https://git.fedorahosted.org/cgit/initscripts.git/tree/rwtab
|
||||||
Loading…
Reference in New Issue
Block a user