Add spec for simple container generation
This spec has been drafted to highlight an effort to improve the container generation process and life cycle management of our applications. Change-Id: I369c9a9f450c220d8cdc2827460de56ff1e4fb57 Co-authored-by: Emilien Macchi <emilien@redhat.com> Signed-off-by: Kevin Carter <kecarter@redhat.com>
This commit is contained in:
parent
40c480da01
commit
273b524df8
|
@ -0,0 +1,427 @@
|
||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
|
||||||
|
===========================
|
||||||
|
Simple Container Generation
|
||||||
|
===========================
|
||||||
|
|
||||||
|
Simple container generation is an initiative to reduce complexity in the
|
||||||
|
TripleO container build, deployment, and distribution process by reducing the
|
||||||
|
size and scope of the TripleO container build tools.
|
||||||
|
|
||||||
|
The primary objective of this initiative is to replace Kolla, and our
|
||||||
|
associated Kolla customization tools, as the selected container generation
|
||||||
|
tool-kit. The TripleO community has long desired an easier solution for
|
||||||
|
deployers and integrators alike and this initiative is making that desire a
|
||||||
|
reality.
|
||||||
|
|
||||||
|
The Simple container generation initiative is wanting to pivot from a
|
||||||
|
tool-chain mired between a foundational component of Kolla-Ansible and a
|
||||||
|
general purpose container build system, to a vertically integrated solution
|
||||||
|
that is only constructing what TripleO needs, in a minimally invasive, and
|
||||||
|
simple to understand way.
|
||||||
|
|
||||||
|
[#f3]_
|
||||||
|
|
||||||
|
|
||||||
|
Problem Description
|
||||||
|
===================
|
||||||
|
|
||||||
|
TripleO currently leverages Kolla to produce container images. These images are
|
||||||
|
built for Kolla-Ansible using an opinionated build process which has general
|
||||||
|
purpose features. While our current images work, they're large and not well
|
||||||
|
suited for the TripleO use-case, especially in distributed data-centers. The
|
||||||
|
issue of container complexity and size impacts three major groups, deployers,
|
||||||
|
third party integrators, and maintainers. As the project is aiming to simplify
|
||||||
|
interactions across the stack, the container life cycle and build process has
|
||||||
|
been identified as something that needs to evolve. The TripleO project needs
|
||||||
|
something vertically integrated which produces smaller images, that are easier
|
||||||
|
to maintain, with far fewer gyrations required to tailor images to our needs.
|
||||||
|
|
||||||
|
|
||||||
|
Proposed Change
|
||||||
|
===============
|
||||||
|
|
||||||
|
Overview
|
||||||
|
--------
|
||||||
|
|
||||||
|
Implement a container file generation role, and a set of statically defined
|
||||||
|
override variable files which are used to generate our required
|
||||||
|
container files. [#f2]_
|
||||||
|
|
||||||
|
Layering
|
||||||
|
^^^^^^^^
|
||||||
|
|
||||||
|
.. code-block:: text
|
||||||
|
|
||||||
|
tripleo-base+---+
|
||||||
|
|
|
||||||
|
|
|
||||||
|
+---+-openstack-${SERVICE}-1-common-+-->openstack-${SERVICE}-1-a
|
||||||
|
| |
|
||||||
|
| +-->openstack-${SERVICE}-1-b
|
||||||
|
| |
|
||||||
|
| +-->openstack-${SERVICE}-1-c
|
||||||
|
+-->openstack-${SERVICE}-2
|
||||||
|
|
|
||||||
|
+-->ancillary-${SERVICE}-1
|
||||||
|
|
|
||||||
|
+-->ancillary-${SERVICE}-2
|
||||||
|
|
||||||
|
|
||||||
|
User Experience
|
||||||
|
^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Building the standard set of images will be done through a simple command line
|
||||||
|
interface using the TripleO python client.
|
||||||
|
|
||||||
|
.. code-block:: shell
|
||||||
|
|
||||||
|
$ openstack tripleo container image build [opts] <args>
|
||||||
|
|
||||||
|
|
||||||
|
This simple sub-command will provide users the ability to construct images as
|
||||||
|
needed, generate container files, and debug runtime issues.
|
||||||
|
|
||||||
|
|
||||||
|
CLI Options
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
The python TripleO client options for the new container image build entry point.
|
||||||
|
|
||||||
|
=========== =============================== =================================================================
|
||||||
|
Option Default Description
|
||||||
|
=========== =============================== =================================================================
|
||||||
|
config-file $PATH/overcloud_containers.yaml Configuration file setting the list of containers to build.
|
||||||
|
exclude [] Container type exclude. Can be specified multiple times.
|
||||||
|
work-dir /tmp/container-builds Container builds directory, storing the container files and
|
||||||
|
logs for each image and its dependencies.
|
||||||
|
skip-push False Skip pushing images to the registry
|
||||||
|
skip-build False Only generates container files without producing a local build.
|
||||||
|
base centos Base image name.
|
||||||
|
type binary Image type.
|
||||||
|
tag latest Image tag.
|
||||||
|
registry localhost Container registry URL.
|
||||||
|
namespace tripleomaster Container registry namespace.
|
||||||
|
volume [] Container bind mount used when building the image. Should be
|
||||||
|
specified multiple times if multiple volumes.
|
||||||
|
=========== =============================== =================================================================
|
||||||
|
|
||||||
|
|
||||||
|
Container Image Build Tools
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Container images will be built using Buildah_, The required Buildah
|
||||||
|
functionality will leverage `BuildahBuilder` via `python-tripleoclient`
|
||||||
|
integration and be exposed though CLI options.
|
||||||
|
|
||||||
|
.. _Buildah: https://buildah.io
|
||||||
|
|
||||||
|
|
||||||
|
Image layout
|
||||||
|
^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Each image will have its own YAML file which has access to the following
|
||||||
|
parameters. Each YAML file will have one required parameter (tcib_from for the
|
||||||
|
source image to build from) and optional parameters.
|
||||||
|
|
||||||
|
================= ============================= ==================== ======== ===================================================
|
||||||
|
Option Default Type Required Description
|
||||||
|
================= ============================= ==================== ======== ===================================================
|
||||||
|
tcib_path `{{ lookup('env', 'HOME') }}` String Path to generated the container file(s) for a given
|
||||||
|
image.
|
||||||
|
tcib_args Dict[str, str] Single level `key:value` pairs. Implements arg_.
|
||||||
|
tcib_from `centos:8` Str True Container image to deploy from. Implements from_.
|
||||||
|
tcib_labels Dict[str, str] Single level `key:value` pairs. Implements label_.
|
||||||
|
tcib_envs Dict[str, str] Single level `key:value` pairs. Implements env_.
|
||||||
|
tcib_onbuilds List[str] <item>=String. Implements onbuild_.
|
||||||
|
tcib_volumes List[str] <item>=String. Implements volume_.
|
||||||
|
tcib_workdir Str Implements workdir_.
|
||||||
|
tcib_adds List[str] <item>=String. Implements add_.
|
||||||
|
tcib_copies List[str] <item>=String. Implements copy_.
|
||||||
|
tcib_exposes List[str] <item>=String. Implements expose_.
|
||||||
|
tcib_user Str Implements user_.
|
||||||
|
tcib_shell Str Implements shell_.
|
||||||
|
tcib_runs List[str] <item>=String. Implements run_.
|
||||||
|
tcib_healthcheck Str Implements healthcheck_.
|
||||||
|
tcib_stopsignal Str Implements stopsignal_.
|
||||||
|
tcib_entrypoint Str Implements entrypoint_.
|
||||||
|
tcib_cmd Str Implements cmd_.
|
||||||
|
tcib_actions List[Dict[str, str]] Each item is a Single level Dictionary `key:value`
|
||||||
|
pairs. Allows for arbitrary verbs which maintains
|
||||||
|
ordering.
|
||||||
|
tcib_gather_files List[str] Each item is a String. Collects files from the
|
||||||
|
host and stores them in the build directory.
|
||||||
|
================= ============================= ==================== ======== ===================================================
|
||||||
|
|
||||||
|
.. _arg: https://docs.docker.com/engine/reference/builder/#arg
|
||||||
|
.. _from: https://docs.docker.com/engine/reference/builder/#from
|
||||||
|
.. _label: https://docs.docker.com/engine/reference/builder/#label
|
||||||
|
.. _env: https://docs.docker.com/engine/reference/builder/#env
|
||||||
|
.. _onbuild: https://docs.docker.com/engine/reference/builder/#onbuild
|
||||||
|
.. _volume: https://docs.docker.com/engine/reference/builder/#volume
|
||||||
|
.. _workdir: https://docs.docker.com/engine/reference/builder/#workdir
|
||||||
|
.. _add: https://docs.docker.com/engine/reference/builder/#add
|
||||||
|
.. _copy: https://docs.docker.com/engine/reference/builder/#copy
|
||||||
|
.. _expose: https://docs.docker.com/engine/reference/builder/#expose
|
||||||
|
.. _user: https://docs.docker.com/engine/reference/builder/#user
|
||||||
|
.. _shell: https://docs.docker.com/engine/reference/builder/#shell
|
||||||
|
.. _run: https://docs.docker.com/engine/reference/builder/#run
|
||||||
|
.. _healthcheck: https://docs.docker.com/engine/reference/builder/#healthcheck
|
||||||
|
.. _stopsignal: https://docs.docker.com/engine/reference/builder/#stopsignal
|
||||||
|
.. _entrypoint: https://docs.docker.com/engine/reference/builder/#entrypoint
|
||||||
|
.. _cmd: https://docs.docker.com/engine/reference/builder/#cmd
|
||||||
|
|
||||||
|
|
||||||
|
Application packages are sorted within each container configuration file.
|
||||||
|
This provides a programmatic interface to derive package sets, allows
|
||||||
|
overrides, and is easily visualized. While the package option is not
|
||||||
|
processes by the `tripleo_container_image_build` role, it will serve as a
|
||||||
|
standard within our templates.
|
||||||
|
|
||||||
|
================ ====================================================
|
||||||
|
Option Description
|
||||||
|
================ ====================================================
|
||||||
|
tcib_packages Dictionary of packages to install.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
common:
|
||||||
|
- openstack-${SERVICE}-common
|
||||||
|
distro-1:
|
||||||
|
common:
|
||||||
|
- openstack-${SERVICE}-proprietary
|
||||||
|
x86_64:
|
||||||
|
- $dep-x86_64
|
||||||
|
power:
|
||||||
|
- $dep-power
|
||||||
|
distro-2:
|
||||||
|
common:
|
||||||
|
- openstack-${SERVICE}
|
||||||
|
- $dep
|
||||||
|
================ ====================================================
|
||||||
|
|
||||||
|
This option is then captured and processed by a simple `RUN` action.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
tcib_actions:
|
||||||
|
- run: "dnf install -y {{ tcib_packages['common'] }} {{ tcib_packages[ansible_distribution][ansible_architecture] }}"
|
||||||
|
|
||||||
|
|
||||||
|
Example Container Variable File
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
tcib_from: ubi8
|
||||||
|
tcib_path: "{{ lookup('env', 'HOME') }}/example-image"
|
||||||
|
tcib_labels:
|
||||||
|
maintainer: MaintainerX
|
||||||
|
tcib_entrypoint: dumb-init --single-child --
|
||||||
|
tcib_stopsignal: SIGTERM
|
||||||
|
tcib_envs:
|
||||||
|
LANG: en_US.UTF-8
|
||||||
|
tcib_runs:
|
||||||
|
- mkdir -p /etc/ssh && touch /etc/ssh/ssh_known_host
|
||||||
|
tcib_copies:
|
||||||
|
- /etc/hosts /opt/hosts
|
||||||
|
tcib_gather_files:
|
||||||
|
- /etc
|
||||||
|
tcib_packages:
|
||||||
|
common:
|
||||||
|
- curl
|
||||||
|
centos:
|
||||||
|
x86_64:
|
||||||
|
- wget
|
||||||
|
tcib_actions:
|
||||||
|
- run: "dnf install -y {{ tcib_packages['common'] }} {{ tcib_packages[ansible_distribution][ansible_architecture] }}"
|
||||||
|
- copy: /etc/resolv.conf /resolv.conf
|
||||||
|
- run: ["/bin/bash", "-c", "echo hello world"]
|
||||||
|
|
||||||
|
|
||||||
|
Container File Structure
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The generated container file(s) will follow a simple directory structure
|
||||||
|
which provide an easy way to view, and understand, build relationships and
|
||||||
|
dependencies throughout the stack.
|
||||||
|
|
||||||
|
.. code-block:: shell
|
||||||
|
|
||||||
|
tripleo-base/${CONTAINERFILE}
|
||||||
|
tripleo-base/ancillary-${SERVICE}-1/${CONTAINERFILE}
|
||||||
|
tripleo-base/ancillary-${SERVICE}-2/${CONTAINERFILE}
|
||||||
|
tripleo-base/openstack-${SERVICE}-1-common/${CONTAINERFILE}
|
||||||
|
tripleo-base/openstack-${SERVICE}-1-common/openstack-${SERVICE}-1-a/${CONTAINERFILE}
|
||||||
|
tripleo-base/openstack-${SERVICE}-1-common/openstack-${SERVICE}-1-b/${CONTAINERFILE}
|
||||||
|
tripleo-base/openstack-${SERVICE}-1-common/openstack-${SERVICE}-1-c/${CONTAINERFILE}
|
||||||
|
tripleo-base/openstack-${SERVICE}-2/${CONTAINERFILE}
|
||||||
|
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
* Use Ansible Bender
|
||||||
|
|
||||||
|
Ansible Bender was evaluated as a tool which could help to build the container
|
||||||
|
images. However it has not been productized downstream; which would make it
|
||||||
|
difficult to consume. It doesn't generate Dockerfiles and there is a strong
|
||||||
|
dependency on Bender tool; the container image build process would therefore be
|
||||||
|
more difficult to do in a standalone environment where Bender isn't available.
|
||||||
|
[#f1]_
|
||||||
|
|
||||||
|
* Leave the container image build process untouched.
|
||||||
|
|
||||||
|
We could leave the container image generate process untouched. This keeps us a
|
||||||
|
consumer of Kolla and requires we maintain our complex ancillary tooling to
|
||||||
|
ensure Kolla containers work for TripleO.
|
||||||
|
|
||||||
|
|
||||||
|
Security Impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
While security is not a primary virtue in the simple container generation
|
||||||
|
initiative, security will be improved by moving to simplified containers. If
|
||||||
|
the simple container generation initiative is ratified, all containers used
|
||||||
|
within TripleO will be vertically integrated into the stack, making it possible
|
||||||
|
to easily audit the build tools and all applications, services, and files
|
||||||
|
installed into our containerized runtimes. With simplification we'll improve
|
||||||
|
the ease of understanding and transparency which makes our project more
|
||||||
|
sustainable, thereby more secure. The proposed solution must provide layers
|
||||||
|
where we know what command has been run exactly; so we can quickly figure out
|
||||||
|
how an image was built.
|
||||||
|
|
||||||
|
|
||||||
|
Upgrade Impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
There is no upgrade impact because the new container images will provide
|
||||||
|
feature parity with the previous ones; they will have the same or similar
|
||||||
|
injected scripts that are used when the containers start.
|
||||||
|
|
||||||
|
|
||||||
|
Other End User Impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
We should expect better performance out of our containers, as they will be
|
||||||
|
smaller. While the runtime will act the same, the software delivery will be
|
||||||
|
faster as the size of each container will smaller, with better constructed
|
||||||
|
layers. Smaller containers will decrease the mean time to ready which will have
|
||||||
|
a positive performance impact and generally improve the user experience.
|
||||||
|
|
||||||
|
|
||||||
|
Other Deployer Impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
The simplified container generation initiative will massively help third party
|
||||||
|
integrators. With simplified container build tools we will be able to easily
|
||||||
|
articulate requirements to folks looking to build on-top of TripleO. Our
|
||||||
|
tool-chain will be capable of bootstrapping applications where required, and
|
||||||
|
simple enough to integrate with a wide variety of custom applications
|
||||||
|
constructed in bespoke formats.
|
||||||
|
|
||||||
|
|
||||||
|
Developer Impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
In the first phase, there won't be any developer impact because the produced
|
||||||
|
images will be providing the same base layers as before. For example, they will
|
||||||
|
contain all the Kolla scripts that are required to merge configuration files or
|
||||||
|
initialize the container at startup.
|
||||||
|
|
||||||
|
These scripts will be injected in the container images for backward
|
||||||
|
compatibility:
|
||||||
|
|
||||||
|
* kolla_extend_start
|
||||||
|
* set_configs.py
|
||||||
|
* start.sh
|
||||||
|
* copy_cacerts.sh
|
||||||
|
* httpd_setup.sh
|
||||||
|
|
||||||
|
In a second phase, we will simplify these scripts to remove what isn't needed
|
||||||
|
by TripleO. The interface in the composable services will likely evolve over
|
||||||
|
time. For example kolla_config will become container_config. There is no plan
|
||||||
|
at this time to rewrite the configuration file merge logic.
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
* Cloudnull
|
||||||
|
* EmilienM
|
||||||
|
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
First phase
|
||||||
|
^^^^^^^^^^^
|
||||||
|
|
||||||
|
* Ansible role to generate container file(s) - https://review.opendev.org/#/c/722557
|
||||||
|
* Container images layouts - https://review.opendev.org/#/c/722486
|
||||||
|
* Deprecate "openstack overcloud container image build"
|
||||||
|
* Implement "openstack tripleo container image build" which will reuse the
|
||||||
|
`BuildahBuilder` and the same logic as the deprecated command but without Kolla.
|
||||||
|
* Build new images and publish them.
|
||||||
|
* Switch the upstream CI to use the new images.
|
||||||
|
|
||||||
|
Second phase:
|
||||||
|
|
||||||
|
* Simplifying the injected scripts to only do what we need in TripleO.
|
||||||
|
* Rename the configuration interfaces in TripleO Heat Templates.
|
||||||
|
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
The tooling will be in existing repositories so there is no new dependency. It
|
||||||
|
will mainly be in tripleo-ansible, tripleo-common, python-tripleoclient and
|
||||||
|
tripleo-heat-templates. Like before, Buildah will be required to build the
|
||||||
|
images.
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
* The tripleo-build-containers-centos-8 job will be switched to be using
|
||||||
|
the new "openstack tripleo container image build" command.
|
||||||
|
|
||||||
|
* A molecule job will exercise the container image build process using
|
||||||
|
the new role.
|
||||||
|
|
||||||
|
* Some end-to-end job will also be investigated to build and deploy
|
||||||
|
a container into a running deployment.
|
||||||
|
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
Much of the documentation impact will be focused on cleanup of the existing
|
||||||
|
documentation which references Kolla, and the creation of documentation that
|
||||||
|
highlights the use of the vertically integrated stack.
|
||||||
|
|
||||||
|
Since the changes should be transparent for the end-users who just pull images
|
||||||
|
without rebuilding it, the manuals will still be updated with the new command
|
||||||
|
and options if anyone wants to build the images themselves.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [#f1] https://review.opendev.org/#/c/722136/
|
||||||
|
.. [#f2] https://review.opendev.org/#/c/722557/
|
||||||
|
.. [#f3] https://blueprints.launchpad.net/tripleo/+spec/simplified-containers
|
Loading…
Reference in New Issue