Update docs: not require cloning git repos
Change-Id: I2b44a6992f8bcc79a8f6f8426427d335ece32789
This commit is contained in:
Vladimir Kozhukalov
parent
1403dccc3f
commit
159adb3859
|
|
@ -1,41 +0,0 @@
|
|||
Before deployment
|
||||
=================
|
||||
|
||||
Before proceeding with the steps outlined in the following
|
||||
sections and executing the actions detailed therein, it is
|
||||
imperative that you clone the essential Git repositories
|
||||
containing all the required Helm charts, deployment scripts,
|
||||
and Ansible roles. This preliminary step will ensure that
|
||||
you have access to the necessary assets for a seamless
|
||||
deployment process.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
mkdir ~/osh
|
||||
cd ~/osh
|
||||
git clone https://opendev.org/openstack/openstack-helm.git
|
||||
git clone https://opendev.org/openstack/openstack-helm-infra.git
|
||||
|
||||
|
||||
All further steps assume these two repositories are cloned into the
|
||||
`~/osh` directory.
|
||||
|
||||
Next, you need to update the dependencies for all the charts in both OpenStack-Helm
|
||||
repositories. This can be done by running the following commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/common/prepare-charts.sh
|
||||
|
||||
Also before deploying the OpenStack cluster you have to specify the
|
||||
OpenStack and the operating system version that you would like to use
|
||||
for deployment. For doing this export the following environment variables
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export OPENSTACK_RELEASE=2024.1
|
||||
export FEATURES="${OPENSTACK_RELEASE} ubuntu_jammy"
|
||||
|
||||
.. note::
|
||||
The list of supported versions can be found :doc:`here </readme>`.
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
Before starting
|
||||
===============
|
||||
|
||||
The OpenStack-Helm charts are published in the `openstack-helm`_ and
|
||||
`openstack-helm-infra`_ helm repositories. Let's enable them:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm repo add openstack-helm https://tarballs.opendev.org/openstack/openstack-helm
|
||||
helm repo add openstack-helm-infra https://tarballs.opendev.org/openstack/openstack-helm-infra
|
||||
|
||||
The OpenStack-Helm `plugin`_ provides some helper commands used later on.
|
||||
So, let's install it:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm plugin install https://opendev.org/openstack/openstack-helm-plugin
|
||||
|
||||
.. _openstack-helm: https://tarballs.opendev.org/openstack/openstack-helm
|
||||
.. _openstack-helm-infra: https://tarballs.opendev.org/openstack/openstack-helm-infra
|
||||
.. _plugin: https://opendev.org/openstack/openstack-helm-plugin.git
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
Deploy Ceph
|
||||
===========
|
||||
|
||||
Ceph is a highly scalable and fault-tolerant distributed storage
|
||||
system designed to store vast amounts of data across a cluster of
|
||||
commodity hardware. It offers object storage, block storage, and
|
||||
file storage capabilities, making it a versatile solution for
|
||||
various storage needs. Ceph's architecture is based on a distributed
|
||||
object store, where data is divided into objects, each with its
|
||||
unique identifier, and distributed across multiple storage nodes.
|
||||
It uses a CRUSH algorithm to ensure data resilience and efficient
|
||||
data placement, even as the cluster scales. Ceph is widely used
|
||||
in cloud computing environments and provides a cost-effective and
|
||||
flexible storage solution for organizations managing large volumes of data.
|
||||
|
||||
Kubernetes introduced the CSI standard to allow storage providers
|
||||
like Ceph to implement their drivers as plugins. Kubernetes can
|
||||
use the CSI driver for Ceph to provision and manage volumes
|
||||
directly. By means of CSI stateful applications deployed on top
|
||||
of Kubernetes can use Ceph to store their data.
|
||||
|
||||
At the same time, Ceph provides the RBD API, which applications
|
||||
can utilize to create and mount block devices distributed across
|
||||
the Ceph cluster. The OpenStack Cinder service utilizes this Ceph
|
||||
capability to offer persistent block devices to virtual machines
|
||||
managed by the OpenStack Nova.
|
||||
|
||||
The recommended way to deploy Ceph on top of Kubernetes is by means
|
||||
of `Rook`_ operator. Rook provides Helm charts to deploy the operator
|
||||
itself which extends the Kubernetes API adding CRDs that enable
|
||||
managing Ceph clusters via Kuberntes custom objects. For details please
|
||||
refer to the `Rook`_ documentation.
|
||||
|
||||
To deploy the Rook Ceph operator and a Ceph cluster you can use the script
|
||||
`ceph-rook.sh`_. Then to generate the client secrets to interface with the Ceph
|
||||
RBD API use this script `ceph-adapter-rook.sh`_
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm-infra
|
||||
./tools/deployment/ceph/ceph-rook.sh
|
||||
./tools/deployment/ceph/ceph-adapter-rook.sh
|
||||
|
||||
.. note::
|
||||
Please keep in mind that these are the deployment scripts that we
|
||||
use for testing. For example we place Ceph OSD data object on loop devices
|
||||
which are slow and are not recommended to use in production.
|
||||
|
||||
|
||||
.. _Rook: https://rook.io/
|
||||
.. _ceph-rook.sh: https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/deployment/ceph/ceph-rook.sh
|
||||
.. _ceph-adapter-rook.sh: https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/deployment/ceph/ceph-adapter-rook.sh
|
||||
|
|
@ -1,51 +0,0 @@
|
|||
Deploy ingress controller
|
||||
=========================
|
||||
|
||||
Deploying an ingress controller when deploying OpenStack on Kubernetes
|
||||
is essential to ensure proper external access and SSL termination
|
||||
for your OpenStack services.
|
||||
|
||||
In the OpenStack-Helm project, we usually deploy multiple `ingress-nginx`_
|
||||
controller instances to optimize traffic routing:
|
||||
|
||||
* In the `kube-system` namespace, we deploy an ingress controller that
|
||||
monitors ingress objects across all namespaces, primarily focusing on
|
||||
routing external traffic into the OpenStack environment.
|
||||
|
||||
* In the `openstack` namespace, we deploy an ingress controller that
|
||||
handles traffic exclusively within the OpenStack namespace. This instance
|
||||
plays a crucial role in SSL termination for enhanced security between
|
||||
OpenStack services.
|
||||
|
||||
* In the `ceph` namespace, we deploy an ingress controller that is dedicated
|
||||
to routing traffic specifically to the Ceph Rados Gateway service, ensuring
|
||||
efficient communication with Ceph storage resources.
|
||||
|
||||
You can utilize any other ingress controller implementation that suits your
|
||||
needs best. See for example the list of available `ingress controllers`_.
|
||||
Ensure that the ingress controller pods are deployed with the `app: ingress-api`
|
||||
label which is used by the OpenStack-Helm as a selector for the Kubernetes
|
||||
services that are exposed as OpenStack endpoints.
|
||||
|
||||
For example, the OpenStack-Helm `keystone` chart by default deploys a service
|
||||
that routes traffic to the ingress controller pods selected using the
|
||||
`app: ingress-api` label. Then it also deploys an ingress object that references
|
||||
the **IngressClass** named `nginx`. This ingress object corresponds to the HTTP
|
||||
virtual host routing the traffic to the Keystone API service which works as an
|
||||
endpoint for Keystone pods.
|
||||
|
||||
.. image:: deploy_ingress_controller.jpg
|
||||
:width: 100%
|
||||
:align: center
|
||||
:alt: deploy-ingress-controller
|
||||
|
||||
To deploy these three ingress controller instances you can use the script `ingress.sh`_
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/common/ingress.sh
|
||||
|
||||
.. _ingress.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/common/ingress.sh
|
||||
.. _ingress-nginx: https://github.com/kubernetes/ingress-nginx/blob/main/charts/ingress-nginx/README.md
|
||||
.. _ingress controllers: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
|
||||
|
|
@ -1,143 +0,0 @@
|
|||
Deploy Kubernetes
|
||||
=================
|
||||
|
||||
OpenStack-Helm provides charts that can be deployed on any Kubernetes cluster if it meets
|
||||
the supported version requirements. However, deploying the Kubernetes cluster itself is beyond
|
||||
the scope of OpenStack-Helm.
|
||||
|
||||
You can use any Kubernetes deployment tool for this purpose. In this guide, we detail how to set up
|
||||
a Kubernetes cluster using Kubeadm and Ansible. While not production-ready, this cluster is ideal
|
||||
as a starting point for lab or proof-of-concept environments.
|
||||
|
||||
All OpenStack projects test their code through an infrastructure managed by the CI
|
||||
tool, Zuul, which executes Ansible playbooks on one or more test nodes. Therefore, we employ Ansible
|
||||
roles/playbooks to install required packages, deploy Kubernetes, and then execute tests on it.
|
||||
|
||||
To establish a test environment, the Ansible role deploy-env_ is employed. This role establishes
|
||||
a basic single/multi-node Kubernetes cluster, ensuring the functionality of commonly used
|
||||
deployment configurations. The role is compatible with Ubuntu Focal and Ubuntu Jammy distributions.
|
||||
|
||||
Install Ansible
|
||||
---------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install ansible
|
||||
|
||||
Prepare Ansible roles
|
||||
---------------------
|
||||
|
||||
Here is the Ansible `playbook`_ that is used to deploy Kubernetes. The roles used in this playbook
|
||||
are defined in different repositories. So in addition to OpenStack-Helm repositories
|
||||
that we assume have already been cloned to the `~/osh` directory you have to clone
|
||||
yet another one
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh
|
||||
git clone https://opendev.org/zuul/zuul-jobs.git
|
||||
|
||||
Now let's set the environment variable ``ANSIBLE_ROLES_PATH`` which specifies
|
||||
where Ansible will lookup roles
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export ANSIBLE_ROLES_PATH=~/osh/openstack-helm-infra/roles:~/osh/zuul-jobs/roles
|
||||
|
||||
To avoid setting it every time when you start a new terminal instance you can define this
|
||||
in the Ansible configuration file. Please see the Ansible documentation.
|
||||
|
||||
Prepare Ansible inventory
|
||||
-------------------------
|
||||
|
||||
We assume you have three nodes, usually VMs. Those nodes must be available via
|
||||
SSH using the public key authentication and a ssh user (let say `ubuntu`)
|
||||
must have passwordless sudo on the nodes.
|
||||
|
||||
Create the Ansible inventory file using the following command
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cat > ~/osh/inventory.yaml <<EOF
|
||||
all:
|
||||
vars:
|
||||
kubectl:
|
||||
user: ubuntu
|
||||
group: ubuntu
|
||||
calico_version: "v3.25"
|
||||
crictl_version: "v1.26.1"
|
||||
helm_version: "v3.6.3"
|
||||
kube_version: "1.26.3-00"
|
||||
yq_version: "v4.6.0"
|
||||
children:
|
||||
primary:
|
||||
hosts:
|
||||
primary:
|
||||
ansible_port: 22
|
||||
ansible_host: 10.10.10.10
|
||||
ansible_user: ubuntu
|
||||
ansible_ssh_private_key_file: ~/.ssh/id_rsa
|
||||
ansible_ssh_extra_args: -o StrictHostKeyChecking=no
|
||||
nodes:
|
||||
hosts:
|
||||
node-1:
|
||||
ansible_port: 22
|
||||
ansible_host: 10.10.10.11
|
||||
ansible_user: ubuntu
|
||||
ansible_ssh_private_key_file: ~/.ssh/id_rsa
|
||||
ansible_ssh_extra_args: -o StrictHostKeyChecking=no
|
||||
node-2:
|
||||
ansible_port: 22
|
||||
ansible_host: 10.10.10.12
|
||||
ansible_user: ubuntu
|
||||
ansible_ssh_private_key_file: ~/.ssh/id_rsa
|
||||
ansible_ssh_extra_args: -o StrictHostKeyChecking=no
|
||||
EOF
|
||||
|
||||
If you have just one node then it must be `primary` in the file above.
|
||||
|
||||
.. note::
|
||||
If you would like to set up a Kubernetes cluster on the local host,
|
||||
configure the Ansible inventory to designate the `primary` node as the local host.
|
||||
For further guidance, please refer to the Ansible documentation.
|
||||
|
||||
Deploy Kubernetes
|
||||
-----------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh
|
||||
ansible-playbook -i inventory.yaml ~/osh/openstack-helm/tools/gate/playbooks/deploy-env.yaml
|
||||
|
||||
The playbook only changes the state of the nodes listed in the Ansible inventory.
|
||||
|
||||
It installs necessary packages, deploys and configures Containerd and Kubernetes. For
|
||||
details please refer to the role `deploy-env`_ and other roles (`ensure-python`_, `ensure-pip`_, `clear-firewall`_)
|
||||
used in the playbook.
|
||||
|
||||
.. note::
|
||||
The role `deploy-env`_ by default will use Google DNS servers, 8.8.8.8 or 8.8.4.4
|
||||
and update `/etc/resolv.conf` on the nodes. These DNS nameserver entries can be changed by
|
||||
updating the file ``~/osh/openstack-helm-infra/roles/deploy-env/files/resolv.conf``.
|
||||
|
||||
It also configures internal Kubernetes DNS server (Coredns) to work as a recursive DNS server
|
||||
and adds its IP address (10.96.0.10 by default) to the `/etc/resolv.conf` file.
|
||||
|
||||
Programs running on those nodes will be able to resolve names in the
|
||||
default Kubernetes domain `.svc.cluster.local`. E.g. if you run OpenStack command line
|
||||
client on one of those nodes it will be able to access OpenStack API services via
|
||||
these names.
|
||||
|
||||
.. note::
|
||||
The role `deploy-env`_ installs and confiugres Kubectl and Helm on the `primary` node.
|
||||
You can login to it via SSH, clone `openstack-helm`_ and `openstack-helm-infra`_ repositories
|
||||
and then run the OpenStack-Helm deployment scipts which employ Kubectl and Helm to deploy
|
||||
OpenStack.
|
||||
|
||||
.. _deploy-env: https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/deploy-env
|
||||
.. _ensure-python: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-python
|
||||
.. _ensure-pip: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-pip
|
||||
.. _clear-firewall: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/clear-firewall
|
||||
.. _openstack-helm: https://opendev.org/openstack/openstack-helm.git
|
||||
.. _openstack-helm-infra: https://opendev.org/openstack/openstack-helm-infra.git
|
||||
.. _playbook: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/gate/playbooks/deploy-env.yaml
|
||||
|
|
@ -1,130 +0,0 @@
|
|||
Deploy OpenStack
|
||||
================
|
||||
|
||||
Now we are ready for the deployment of OpenStack components.
|
||||
Some of them are mandatory while others are optional.
|
||||
|
||||
Keystone
|
||||
--------
|
||||
|
||||
OpenStack Keystone is the identity and authentication service
|
||||
for the OpenStack cloud computing platform. It serves as the
|
||||
central point of authentication and authorization, managing user
|
||||
identities, roles, and access to OpenStack resources. Keystone
|
||||
ensures secure and controlled access to various OpenStack services,
|
||||
making it an integral component for user management and security
|
||||
in OpenStack deployments.
|
||||
|
||||
This is a ``mandatory`` component of any OpenStack cluster.
|
||||
|
||||
To deploy the Keystone service run the script `keystone.sh`_
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/keystone/keystone.sh
|
||||
|
||||
|
||||
Heat
|
||||
----
|
||||
|
||||
OpenStack Heat is an orchestration service that provides templates
|
||||
and automation for deploying and managing cloud resources. It enables
|
||||
users to define infrastructure as code, making it easier to create
|
||||
and manage complex environments in OpenStack through templates and
|
||||
automation scripts.
|
||||
|
||||
Here is the script `heat.sh`_ for the deployment of Heat service.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/heat/heat.sh
|
||||
|
||||
Glance
|
||||
------
|
||||
|
||||
OpenStack Glance is the image service component of OpenStack.
|
||||
It manages and catalogs virtual machine images, such as operating
|
||||
system images and snapshots, making them available for use in
|
||||
OpenStack compute instances.
|
||||
|
||||
This is a ``mandatory`` component.
|
||||
|
||||
The Glance deployment script is here `glance.sh`_.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/glance/glance.sh
|
||||
|
||||
Cinder
|
||||
------
|
||||
|
||||
OpenStack Cinder is the block storage service component of the
|
||||
OpenStack cloud computing platform. It manages and provides persistent
|
||||
block storage to virtual machines, enabling users to attach and detach
|
||||
persistent storage volumes to their VMs as needed.
|
||||
|
||||
To deploy the OpenStack Cinder service use the script `cinder.sh`_
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/cinder/cinder.sh
|
||||
|
||||
Placement, Nova, Neutron
|
||||
------------------------
|
||||
|
||||
OpenStack Placement is a service that helps manage and allocate
|
||||
resources in an OpenStack cloud environment. It helps Nova (compute)
|
||||
find and allocate the right resources (CPU, memory, etc.)
|
||||
for virtual machine instances.
|
||||
|
||||
OpenStack Nova is the compute service responsible for managing
|
||||
and orchestrating virtual machines in an OpenStack cloud.
|
||||
It provisions and schedules instances, handles their lifecycle,
|
||||
and interacts with underlying hypervisors.
|
||||
|
||||
OpenStack Neutron is the networking service that provides network
|
||||
connectivity and enables users to create and manage network resources
|
||||
for their virtual machines and other services.
|
||||
|
||||
These three services are ``mandatory`` and together constitue
|
||||
so-called ``compute kit``.
|
||||
|
||||
To set up the compute service, the first step involves deploying the
|
||||
hypervisor backend using the `libvirt.sh`_ script. By default, the
|
||||
networking service is deployed with OpenvSwitch as the networking
|
||||
backend, and the deployment script for OpenvSwitch can be found
|
||||
here: `openvswitch.sh`_. And finally the deployment script for
|
||||
Placement, Nova and Neutron is here: `compute-kit.sh`_.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/compute-kit/openvswitch.sh
|
||||
./tools/deployment/component/compute-kit/libvirt.sh
|
||||
./tools/deployment/component/compute-kit/compute-kit.sh
|
||||
|
||||
Horizon
|
||||
-------
|
||||
|
||||
OpenStack Horizon is the web application that is intended to provide a graphic
|
||||
user interface to Openstack services.
|
||||
|
||||
To deploy the OpenStack Horizon use the following script `horizon.sh`_
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/horizon/horizon.sh
|
||||
|
||||
.. _keystone.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/keystone/keystone.sh
|
||||
.. _heat.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/heat/heat.sh
|
||||
.. _glance.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/glance/glance.sh
|
||||
.. _libvirt.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/compute-kit/libvirt.sh
|
||||
.. _openvswitch.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/compute-kit/openvswitch.sh
|
||||
.. _compute-kit.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/compute-kit/compute-kit.sh
|
||||
.. _cinder.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/cinder/cinder.sh
|
||||
.. _horizon.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/horizon/horizon.sh
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
Deploy OpenStack backend
|
||||
========================
|
||||
|
||||
OpenStack is a cloud computing platform that consists of a variety of
|
||||
services, and many of these services rely on backend services like RabbitMQ,
|
||||
MariaDB, and Memcached for their proper functioning. These backend services
|
||||
play crucial roles in OpenStack's architecture.
|
||||
|
||||
RabbitMQ
|
||||
~~~~~~~~
|
||||
RabbitMQ is a message broker that is often used in OpenStack to handle
|
||||
messaging between different components and services. It helps in managing
|
||||
communication and coordination between various parts of the OpenStack
|
||||
infrastructure. Services like Nova (compute), Neutron (networking), and
|
||||
Cinder (block storage) use RabbitMQ to exchange messages and ensure
|
||||
proper orchestration.
|
||||
|
||||
MariaDB
|
||||
~~~~~~~
|
||||
Database services like MariaDB are used as the backend database for several
|
||||
OpenStack services. These databases store critical information such as user
|
||||
credentials, service configurations, and data related to instances, networks,
|
||||
and volumes. Services like Keystone (identity), Nova, Glance (image), and
|
||||
Cinder rely on MariaDB for data storage.
|
||||
|
||||
Memcached
|
||||
~~~~~~~~~
|
||||
Memcached is a distributed memory object caching system that is often used
|
||||
in OpenStack to improve performance and reduce database load. OpenStack
|
||||
services cache frequently accessed data in Memcached, which helps in faster
|
||||
data retrieval and reduces the load on the database backend. Services like
|
||||
Keystone and Nova can benefit from Memcached for caching.
|
||||
|
||||
Deployment
|
||||
----------
|
||||
|
||||
The following scripts `rabbitmq.sh`_, `mariadb.sh`_, `memcached.sh`_ can be used to
|
||||
deploy the backend services.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/component/common/rabbitmq.sh
|
||||
./tools/deployment/component/common/mariadb.sh
|
||||
./tools/deployment/component/common/memcached.sh
|
||||
|
||||
.. note::
|
||||
These scripts use Helm charts from the `openstack-helm-infra`_ repository. We assume
|
||||
this repo is cloned to the `~/osh` directory. See this :doc:`section </install/before_deployment>`.
|
||||
|
||||
.. _rabbitmq.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/common/rabbitmq.sh
|
||||
.. _mariadb.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/common/mariadb.sh
|
||||
.. _memcached.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/component/common/memcached.sh
|
||||
.. _openstack-helm-infra: https://opendev.org/openstack/openstack-helm-infra.git
|
||||
|
|
@ -1,17 +1,12 @@
|
|||
Installation
|
||||
============
|
||||
|
||||
Contents:
|
||||
Here are sections that describe how to install OpenStack using OpenStack-Helm:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
before_deployment
|
||||
deploy_kubernetes
|
||||
prepare_kubernetes
|
||||
deploy_ceph
|
||||
setup_openstack_client
|
||||
deploy_ingress_controller
|
||||
deploy_openstack_backend
|
||||
deploy_openstack
|
||||
|
||||
before_starting
|
||||
kubernetes
|
||||
prerequisites
|
||||
openstack
|
||||
|
|
|
|||
|
Before Width: | Height: | Size: 108 KiB After Width: | Height: | Size: 108 KiB |
|
|
@ -0,0 +1,182 @@
|
|||
Kubernetes
|
||||
==========
|
||||
|
||||
OpenStack-Helm provides charts that can be deployed on any Kubernetes cluster if it meets
|
||||
the version :doc:`requirements </readme>`. However, deploying the Kubernetes cluster itself is beyond
|
||||
the scope of OpenStack-Helm.
|
||||
|
||||
You can use any Kubernetes deployment tool for this purpose. In this guide, we detail how to set up
|
||||
a Kubernetes cluster using Kubeadm and Ansible. While not production-ready, this cluster is ideal
|
||||
as a starting point for lab or proof-of-concept environments.
|
||||
|
||||
All OpenStack projects test their code through an infrastructure managed by the CI
|
||||
tool, Zuul, which executes Ansible playbooks on one or more test nodes. Therefore, we employ Ansible
|
||||
roles/playbooks to install required packages, deploy Kubernetes, and then execute tests on it.
|
||||
|
||||
To establish a test environment, the Ansible role `deploy-env`_ is employed. This role deploys
|
||||
a basic single/multi-node Kubernetes cluster, used to prove the functionality of commonly used
|
||||
deployment configurations. The role is compatible with Ubuntu Focal and Ubuntu Jammy distributions.
|
||||
|
||||
.. note::
|
||||
The role `deploy-env`_ is not idempotent and assumed to be applied to a clean environment.
|
||||
|
||||
Clone roles git repositories
|
||||
----------------------------
|
||||
|
||||
Before proceeding with the steps outlined in the following sections, it is
|
||||
imperative that you clone the git repositories containing the required Ansible roles.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
mkdir ~/osh
|
||||
cd ~/osh
|
||||
git clone https://opendev.org/openstack/openstack-helm-infra.git
|
||||
git clone https://opendev.org/zuul/zuul-jobs.git
|
||||
|
||||
Install Ansible
|
||||
---------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install ansible
|
||||
|
||||
Set roles lookup path
|
||||
---------------------
|
||||
|
||||
Now let's set the environment variable ``ANSIBLE_ROLES_PATH`` which specifies
|
||||
where Ansible will lookup roles
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export ANSIBLE_ROLES_PATH=~/osh/openstack-helm-infra/roles:~/osh/zuul-jobs/roles
|
||||
|
||||
To avoid setting it every time when you start a new terminal instance you can define this
|
||||
in the Ansible configuration file. Please see the Ansible documentation.
|
||||
|
||||
Prepare inventory
|
||||
-----------------
|
||||
|
||||
The example below assumes that there are four nodes which must be available via
|
||||
SSH using the public key authentication and a ssh user (let say ``ubuntu``)
|
||||
must have passwordless sudo on the nodes.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cat > ~/osh/inventory.yaml <<EOF
|
||||
---
|
||||
all:
|
||||
vars:
|
||||
ansible_port: 22
|
||||
ansible_user: ubuntu
|
||||
ansible_ssh_private_key_file: /home/ubuntu/.ssh/id_rsa
|
||||
ansible_ssh_extra_args: -o StrictHostKeyChecking=no
|
||||
# The user and group that will be used to run Kubectl and Helm commands.
|
||||
kubectl:
|
||||
user: ubuntu
|
||||
group: ubuntu
|
||||
# The user and group that will be used to run Docker commands.
|
||||
docker_users:
|
||||
- ununtu
|
||||
# The MetalLB controller will be installed on the Kubernetes cluster.
|
||||
metallb_setup: true
|
||||
# Loopback devices will be created on all cluster nodes which then can be used
|
||||
# to deploy a Ceph cluster which requires block devices to be provided.
|
||||
# Please use loopback devices only for testing purposes. They are not suitable
|
||||
# for production due to performance reasons.
|
||||
loopback_setup: true
|
||||
loopback_device: /dev/loop100
|
||||
loopback_image: /var/lib/openstack-helm/ceph-loop.img
|
||||
loopback_image_size: 12G
|
||||
children:
|
||||
# The primary node where Kubectl and Helm will be installed. If it is
|
||||
# the only node then it must be a member of the groups k8s_cluster and
|
||||
# k8s_control_plane. If there are more nodes then the wireguard tunnel
|
||||
# will be established between the primary node and the k8s_control_plane node.
|
||||
primary:
|
||||
hosts:
|
||||
primary:
|
||||
ansible_host: 10.10.10.10
|
||||
# The nodes where the Kubernetes components will be installed.
|
||||
k8s_cluster:
|
||||
hosts:
|
||||
node-1:
|
||||
ansible_host: 10.10.10.11
|
||||
node-2:
|
||||
ansible_host: 10.10.10.12
|
||||
node-3:
|
||||
ansible_host: 10.10.10.13
|
||||
# The control plane node where the Kubernetes control plane components will be installed.
|
||||
# It must be the only node in the group k8s_control_plane.
|
||||
k8s_control_plane:
|
||||
hosts:
|
||||
node-1:
|
||||
ansible_host: 10.10.10.11
|
||||
# These are Kubernetes worker nodes. There could be zero such nodes.
|
||||
# In this case the Openstack workloads will be deployed on the control plane node.
|
||||
k8s_nodes:
|
||||
hosts:
|
||||
node-2:
|
||||
ansible_host: 10.10.10.12
|
||||
node-3:
|
||||
ansible_host: 10.10.10.13
|
||||
EOF
|
||||
|
||||
.. note::
|
||||
If you would like to set up a Kubernetes cluster on the local host,
|
||||
configure the Ansible inventory to designate the ``primary`` node as the local host.
|
||||
For further guidance, please refer to the Ansible documentation.
|
||||
|
||||
.. note::
|
||||
The full list of variables that you can define in the inventory file can be found in the
|
||||
file `deploy-env/defaults/main.yaml`_.
|
||||
|
||||
Prepare playbook
|
||||
----------------
|
||||
|
||||
Create an Ansible playbook that will deploy the environment
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cat > ~/osh/deploy-env.yaml <<EOF
|
||||
---
|
||||
- hosts: all
|
||||
become: true
|
||||
gather_facts: true
|
||||
roles:
|
||||
- ensure-python
|
||||
- ensure-pip
|
||||
- clear-firewall
|
||||
- deploy-env
|
||||
EOF
|
||||
|
||||
Run the playbook
|
||||
-----------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh
|
||||
ansible-playbook -i inventory.yaml deploy-env.yaml
|
||||
|
||||
The playbook only changes the state of the nodes listed in the inventory file.
|
||||
|
||||
It installs necessary packages, deploys and configures Containerd and Kubernetes. For
|
||||
details please refer to the role `deploy-env`_ and other roles (`ensure-python`_,
|
||||
`ensure-pip`_, `clear-firewall`_) used in the playbook.
|
||||
|
||||
.. note::
|
||||
The role `deploy-env`_ configures cluster nodes to use Google DNS servers (8.8.8.8).
|
||||
|
||||
By default, it also configures internal Kubernetes DNS server (Coredns) to work
|
||||
as a recursive DNS server and adds its IP address (10.96.0.10 by default) to the
|
||||
``/etc/resolv.conf`` file.
|
||||
|
||||
Processes running on the cluster nodes will be able to resolve internal
|
||||
Kubernetes domain names ``*.svc.cluster.local``.
|
||||
|
||||
.. _deploy-env: https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/deploy-env
|
||||
.. _deploy-env/defaults/main.yaml: https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/deploy-env/defaults/main.yaml
|
||||
.. _zuul-jobs: https://opendev.org/zuul/zuul-jobs.git
|
||||
.. _ensure-python: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-python
|
||||
.. _ensure-pip: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-pip
|
||||
.. _clear-firewall: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/clear-firewall
|
||||
.. _openstack-helm-infra: https://opendev.org/openstack/openstack-helm-infra.git
|
||||
|
|
@ -0,0 +1,415 @@
|
|||
Deploy OpenStack
|
||||
================
|
||||
|
||||
Check list before deployment
|
||||
----------------------------
|
||||
|
||||
At this point we assume all the prerequisites listed below are met:
|
||||
|
||||
- Kubernetes cluster is up and running.
|
||||
- `kubectl`_ and `helm`_ command line tools are installed and
|
||||
configured to access the cluster.
|
||||
- The OpenStack-Helm repositories are enabled, OpenStack-Helm
|
||||
plugin is installed and necessary environment variables are set.
|
||||
- The ``openstack`` namespace is created.
|
||||
- Ingress controller is deployed in the ``openstack`` namespace.
|
||||
- MetalLB is deployed and configured. The service of type
|
||||
``LoadBalancer`` is created and DNS is configured to resolve the
|
||||
Openstack endpoint names to the IP address of the service.
|
||||
- Ceph is deployed and enabled for using by OpenStack-Helm.
|
||||
|
||||
.. _kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/
|
||||
.. _helm: https://helm.sh/docs/intro/install/
|
||||
|
||||
|
||||
Environment variables
|
||||
---------------------
|
||||
|
||||
First let's set environment variables that are later used in the subsequent sections:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export OPENSTACK_RELEASE=2024.1
|
||||
# Features enabled for the deployment. This is used to look up values overrides.
|
||||
export FEATURES="${OPENSTACK_RELEASE} ubuntu_jammy"
|
||||
# Directory where values overrides are looked up or downloaded to.
|
||||
export OVERRIDES_DIR=$(pwd)/overrides
|
||||
|
||||
Get values overrides
|
||||
--------------------
|
||||
|
||||
OpenStack-Helm provides values overrides for predefined feature sets and various
|
||||
OpenStack/platform versions. The overrides are stored in the OpenStack-Helm
|
||||
git repositories and OpenStack-Helm plugin provides a command to look them up
|
||||
locally and download (optional) if not found.
|
||||
|
||||
Please read the help:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm osh get-values-overrides --help
|
||||
|
||||
For example, if you pass the feature set ``2024.1 ubuntu_jammy`` it will try to
|
||||
look up the following files:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
2024.1.yaml
|
||||
ubuntu_jammy.yaml
|
||||
2024.1-ubuntu_jammy.yaml
|
||||
|
||||
Let's download the values overrides for the feature set defined above:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
INFRA_OVERRIDES_URL=https://opendev.org/openstack/openstack-helm-infra/raw/branch/master
|
||||
for chart in rabbitmq mariadb memcached openvswitch libvirt; do
|
||||
helm osh get-values-overrides -d -u ${INFRA_OVERRIDES_URL} -p ${OVERRIDES_DIR} -c ${chart} ${FEATURES}
|
||||
done
|
||||
|
||||
OVERRIDES_URL=https://opendev.org/openstack/openstack-helm/raw/branch/master
|
||||
for chart in keystone heat glance cinder placement nova neutron horizon; do
|
||||
helm osh get-values-overrides -d -u ${OVERRIDES_URL} -p ${OVERRIDES_DIR} -c ${chart} ${FEATURES}
|
||||
done
|
||||
|
||||
Now you can inspect the downloaded files in the ``${OVERRIDES_DIR}`` directory and
|
||||
adjust them if needed.
|
||||
|
||||
OpenStack backend
|
||||
-----------------
|
||||
|
||||
OpenStack is a cloud computing platform that consists of a variety of
|
||||
services, and many of these services rely on backend services like RabbitMQ,
|
||||
MariaDB, and Memcached for their proper functioning. These backend services
|
||||
play crucial role in OpenStack architecture.
|
||||
|
||||
RabbitMQ
|
||||
~~~~~~~~
|
||||
RabbitMQ is a message broker that is often used in OpenStack to handle
|
||||
messaging between different components and services. It helps in managing
|
||||
communication and coordination between various parts of the OpenStack
|
||||
infrastructure. Services like Nova (compute), Neutron (networking), and
|
||||
Cinder (block storage) use RabbitMQ to exchange messages and ensure
|
||||
proper orchestration.
|
||||
|
||||
Use the following script to deploy RabbitMQ service:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install rabbitmq openstack-helm-infra/rabbitmq \
|
||||
--namespace=openstack \
|
||||
--set pod.replicas.server=1 \
|
||||
--timeout=600s \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c rabbitmq ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
MariaDB
|
||||
~~~~~~~
|
||||
Database services like MariaDB are used as a backend database for majority of
|
||||
OpenStack projects. These databases store critical information such as user
|
||||
credentials, service configurations, and data related to instances, networks,
|
||||
and volumes. Services like Keystone (identity), Nova, Glance (image), and
|
||||
Cinder rely on MariaDB for data storage.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install mariadb openstack-helm-infra/mariadb \
|
||||
--namespace=openstack \
|
||||
--set pod.replicas.server=1 \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c mariadb ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
Memcached
|
||||
~~~~~~~~~
|
||||
Memcached is a distributed memory object caching system that is often used
|
||||
in OpenStack to improve performance. OpenStack services cache frequently
|
||||
accessed data in Memcached, which helps in faster
|
||||
data retrieval and reduces the load on the database backend.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install memcached openstack-helm-infra/memcached \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c memcached ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
OpenStack
|
||||
---------
|
||||
|
||||
Now we are ready for the deployment of OpenStack components.
|
||||
Some of them are mandatory while others are optional.
|
||||
|
||||
Keystone
|
||||
~~~~~~~~
|
||||
|
||||
OpenStack Keystone is the identity and authentication service
|
||||
for the OpenStack cloud computing platform. It serves as the
|
||||
central point of authentication and authorization, managing user
|
||||
identities, roles, and access to OpenStack resources. Keystone
|
||||
ensures secure and controlled access to various OpenStack services,
|
||||
making it an integral component for user management and security
|
||||
in OpenStack deployments.
|
||||
|
||||
This is a ``mandatory`` component of any OpenStack cluster.
|
||||
|
||||
To deploy the Keystone service run the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install keystone openstack-helm/keystone \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c keystone ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
Heat
|
||||
~~~~
|
||||
|
||||
OpenStack Heat is an orchestration service that provides templates
|
||||
and automation for deploying and managing cloud resources. It enables
|
||||
users to define infrastructure as code, making it easier to create
|
||||
and manage complex environments in OpenStack through templates and
|
||||
automation scripts.
|
||||
|
||||
Here are the commands for the deployment of Heat service.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install heat openstack-helm/heat \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c heat ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
Glance
|
||||
~~~~~~
|
||||
|
||||
OpenStack Glance is the image service component of OpenStack.
|
||||
It manages and catalogs virtual machine images, such as operating
|
||||
system images and snapshots, making them available for use in
|
||||
OpenStack compute instances.
|
||||
|
||||
This is a ``mandatory`` component.
|
||||
|
||||
The Glance deployment commands are as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tee ${OVERRIDES_DIR}/glance/values_overrides/glance_pvc_storage.yaml <<EOF
|
||||
storage: pvc
|
||||
volume:
|
||||
class_name: general
|
||||
size: 10Gi
|
||||
EOF
|
||||
|
||||
helm upgrade --install glance openstack-helm/glance \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c glance glance_pvc_storage ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
.. note::
|
||||
|
||||
In the above we prepare a values override file for ``glance`` chart which
|
||||
makes it use a Persistent Volume Claim (PVC) for storing images. We put
|
||||
the values in the ``${OVERRIDES_DIR}/glance/values_overrides/glance_pvc_storage.yaml``
|
||||
so the OpenStack-Helm plugin can pick it up if we pass the feature
|
||||
``glance_pvc_storage`` to it.
|
||||
|
||||
Cinder
|
||||
~~~~~~
|
||||
|
||||
OpenStack Cinder is the block storage service component of the
|
||||
OpenStack cloud computing platform. It manages and provides persistent
|
||||
block storage to virtual machines, enabling users to attach and detach
|
||||
persistent storage volumes to their VMs as needed.
|
||||
|
||||
To deploy the OpenStack Cinder use the following
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install cinder openstack-helm/cinder \
|
||||
--namespace=openstack \
|
||||
--timeout=600s \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c cinder ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
Compute kit backend: Openvswitch and Libvirt
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
OpenStack-Helm recommends using OpenvSwitch as the networking backend
|
||||
for the OpenStack cloud. OpenvSwitch is a software-based, open-source
|
||||
networking solution that provides virtual switching capabilities.
|
||||
|
||||
To deploy the OpenvSwitch service use the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install openvswitch openstack-helm-infra/openvswitch \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c openvswitch ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
Libvirt is a toolkit that provides a common API for managing virtual
|
||||
machines. It is used in OpenStack to interact with hypervisors like
|
||||
KVM, QEMU, and Xen.
|
||||
|
||||
Let's deploy the Libvirt service using the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install libvirt openstack-helm-infra/libvirt \
|
||||
--namespace=openstack \
|
||||
--set conf.ceph.enabled=true \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c libvirt ${FEATURES})
|
||||
|
||||
.. note::
|
||||
Here we don't need to run ``helm osh wait-for-pods`` because the Libvirt pods
|
||||
depend on Neutron OpenvSwitch agent pods which are not yet deployed.
|
||||
|
||||
Compute kit: Placement, Nova, Neutron
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
OpenStack Placement is a service that helps manage and allocate
|
||||
resources in an OpenStack cloud environment. It helps Nova (compute)
|
||||
find and allocate the right resources (CPU, memory, etc.)
|
||||
for virtual machine instances.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install placement openstack-helm/placement
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c placement ${FEATURES})
|
||||
|
||||
OpenStack Nova is the compute service responsible for managing
|
||||
and orchestrating virtual machines in an OpenStack cloud.
|
||||
It provisions and schedules instances, handles their lifecycle,
|
||||
and interacts with underlying hypervisors.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install nova openstack-helm/nova \
|
||||
--namespace=openstack \
|
||||
--set bootstrap.wait_for_computes.enabled=true \
|
||||
--set conf.ceph.enabled=true \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c nova ${FEATURES})
|
||||
|
||||
OpenStack Neutron is the networking service that provides network
|
||||
connectivity and enables users to create and manage network resources
|
||||
for their virtual machines and other services.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
PROVIDER_INTERFACE=<provider_interface_name>
|
||||
tee ${OVERRIDES_DIR}/neutron/values_overrides/neutron_simple.yaml << EOF
|
||||
conf:
|
||||
neutron:
|
||||
DEFAULT:
|
||||
l3_ha: False
|
||||
max_l3_agents_per_router: 1
|
||||
# <provider_interface_name> will be attached to the br-ex bridge.
|
||||
# The IP assigned to the interface will be moved to the bridge.
|
||||
auto_bridge_add:
|
||||
br-ex: ${PROVIDER_INTERFACE}
|
||||
plugins:
|
||||
ml2_conf:
|
||||
ml2_type_flat:
|
||||
flat_networks: public
|
||||
openvswitch_agent:
|
||||
ovs:
|
||||
bridge_mappings: public:br-ex
|
||||
EOF
|
||||
|
||||
helm upgrade --install neutron openstack-helm/neutron \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c neutron neutron_simple ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
Horizon
|
||||
~~~~~~~
|
||||
|
||||
OpenStack Horizon is the web application that is intended to provide a graphic
|
||||
user interface to Openstack services.
|
||||
|
||||
Let's deploy it:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm upgrade --install neutron openstack-helm/neutron \
|
||||
--namespace=openstack \
|
||||
$(helm osh get-values-overrides -p ${OVERRIDES_DIR} -c horizon ${FEATURES})
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
OpenStack client
|
||||
----------------
|
||||
|
||||
Installing the OpenStack client on the developer's machine is a vital step.
|
||||
The easiest way to install the OpenStack client is to create a Python
|
||||
virtual environment and install the client using ``pip``.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python3 -m venv ~/openstack-client
|
||||
source ~/openstack-client/bin/activate
|
||||
pip install python-openstackclient
|
||||
|
||||
Now let's prepare the OpenStack client configuration file:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
mkdir -p ~/.config/openstack
|
||||
tee ~/.config/openstack/clouds.yaml << EOF
|
||||
clouds:
|
||||
openstack_helm:
|
||||
region_name: RegionOne
|
||||
identity_api_version: 3
|
||||
auth:
|
||||
username: 'admin'
|
||||
password: 'password'
|
||||
project_name: 'admin'
|
||||
project_domain_name: 'default'
|
||||
user_domain_name: 'default'
|
||||
auth_url: 'http://keystone.openstack.svc.cluster.local/v3'
|
||||
|
||||
That is it! Now you can use the OpenStack client. Try to run this:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack --os-cloud openstack_helm endpoint list
|
||||
|
||||
.. note::
|
||||
|
||||
In some cases it is more convenient to use the OpenStack client
|
||||
inside a Docker container. OpenStack-Helm provides the
|
||||
`openstackhelm/openstack-client`_ image. The below is an example
|
||||
of how to use it.
|
||||
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -it --rm --network host \
|
||||
-v ~/.config/openstack/clouds.yaml:/etc/openstack/clouds.yaml \
|
||||
-e OS_CLOUD=openstack_helm \
|
||||
docker.io/openstackhelm/openstack-client:${OPENSTACK_RELEASE} \
|
||||
openstack endpoint list
|
||||
|
||||
Remember that the container file system is ephemeral and is destroyed
|
||||
when you stop the container. So if you would like to use the
|
||||
Openstack client capabilities interfacing with the file system then you have to mount
|
||||
a directory from the host file system where necessary files are located.
|
||||
For example, this is useful when you create a key pair and save the private key in a file
|
||||
which is then used for ssh access to VMs. Or it could be Heat templates
|
||||
which you prepare in advance and then use with Openstack client.
|
||||
|
||||
For convenience, you can create an executable entry point that runs the
|
||||
Openstack client in a Docker container. See for example `setup-client.sh`_.
|
||||
|
||||
.. _setup-client.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/common/setup-client.sh
|
||||
.. _openstackhelm/openstack-client: https://hub.docker.com/r/openstackhelm/openstack-client/tags?page=&page_size=&ordering=&name=
|
||||
|
|
@ -1,28 +0,0 @@
|
|||
Prepare Kubernetes
|
||||
==================
|
||||
|
||||
In this section we assume you have a working Kubernetes cluster and
|
||||
Kubectl and Helm properly configured to interact with the cluster.
|
||||
|
||||
Before deploying OpenStack components using OpenStack-Helm you have to set
|
||||
labels on Kubernetes worker nodes which are used as node selectors.
|
||||
|
||||
Also necessary namespaces must be created.
|
||||
|
||||
You can use the `prepare-k8s.sh`_ script as an example of how to prepare
|
||||
the Kubernetes cluster for OpenStack deployment. The script is assumed to be run
|
||||
from the openstack-helm repository
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/common/prepare-k8s.sh
|
||||
|
||||
|
||||
.. note::
|
||||
Pay attention that in the above script we set labels on all Kubernetes nodes including
|
||||
Kubernetes control plane nodes which are usually not aimed to run workload pods
|
||||
(OpenStack in our case). So you have to either untaint control plane nodes or modify the
|
||||
`prepare-k8s.sh`_ script so it sets labels only on the worker nodes.
|
||||
|
||||
.. _prepare-k8s.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/common/prepare-k8s.sh
|
||||
|
|
@ -0,0 +1,328 @@
|
|||
Kubernetes prerequisites
|
||||
========================
|
||||
|
||||
Ingress controller
|
||||
------------------
|
||||
|
||||
Ingress controller when deploying OpenStack on Kubernetes
|
||||
is essential to ensure proper external access for the OpenStack services.
|
||||
|
||||
We recommend using the `ingress-nginx`_ because it is simple and provides
|
||||
all necessary features. It utilizes Nginx as a reverse proxy backend.
|
||||
Here is how to deploy it.
|
||||
|
||||
First, let's create a namespace for the OpenStack workloads. The ingress
|
||||
controller must be deployed in the same namespace because OpenStack-Helm charts
|
||||
create service resources pointing to the ingress controller pods which
|
||||
in turn redirect traffic to particular Openstack API pods.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tee > /tmp/openstack_namespace.yaml <<EOF
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: openstack
|
||||
EOF
|
||||
kubectl apply -f /tmp/openstack_namespace.yaml
|
||||
|
||||
Next, deploy the ingress controller in the ``openstack`` namespace:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
|
||||
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
|
||||
--version="4.8.3" \
|
||||
--namespace=openstack \
|
||||
--set controller.kind=Deployment \
|
||||
--set controller.admissionWebhooks.enabled="false" \
|
||||
--set controller.scope.enabled="true" \
|
||||
--set controller.service.enabled="false" \
|
||||
--set controller.ingressClassResource.name=nginx \
|
||||
--set controller.ingressClassResource.controllerValue="k8s.io/ingress-nginx" \
|
||||
--set controller.ingressClassResource.default="false" \
|
||||
--set controller.ingressClass=nginx \
|
||||
--set controller.labels.app=ingress-api
|
||||
|
||||
You can deploy any other ingress controller that suits your needs best.
|
||||
See for example the list of available `ingress controllers`_.
|
||||
Ensure that the ingress controller pods are deployed with the ``app: ingress-api``
|
||||
label which is used by the OpenStack-Helm as a selector for the Kubernetes
|
||||
service resources.
|
||||
|
||||
For example, the OpenStack-Helm ``keystone`` chart by default creates a service
|
||||
that redirects traffic to the ingress controller pods selected using the
|
||||
``app: ingress-api`` label. Then it also creates an ``Ingress`` resource which
|
||||
the ingress controller then uses to configure its reverse proxy
|
||||
backend (Nginx) which eventually routes the traffic to the Keystone API
|
||||
service which works as an endpoint for Keystone API pods.
|
||||
|
||||
.. image:: ingress.jpg
|
||||
:width: 100%
|
||||
:align: center
|
||||
:alt: ingress scheme
|
||||
|
||||
.. note::
|
||||
For exposing the OpenStack services to the external world, we can a create
|
||||
service of type ``LoadBalancer`` or ``NodePort`` with the selector pointing to
|
||||
the ingress controller pods.
|
||||
|
||||
.. _ingress-nginx: https://github.com/kubernetes/ingress-nginx/blob/main/charts/ingress-nginx/README.md
|
||||
.. _ingress controllers: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
|
||||
|
||||
MetalLB
|
||||
-------
|
||||
|
||||
MetalLB is a load-balancer for bare metal Kubernetes clusters levereging
|
||||
L2/L3 protocols. This is a popular way of exposing the web
|
||||
applications running in Kubernetes to the external world.
|
||||
|
||||
The following commands can be used to deploy MetalLB:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tee > /tmp/metallb_system_namespace.yaml <<EOF
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: metallb-system
|
||||
EOF
|
||||
kubectl apply -f /tmp/metallb_system_namespace.yaml
|
||||
|
||||
helm repo add metallb https://metallb.github.io/metallb
|
||||
helm install metallb metallb/metallb -n metallb-system
|
||||
|
||||
Now it is necessary to configure the MetalLB IP address pool and the IP address
|
||||
advertisement. The MetalLB custom resources are used for this:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tee > /tmp/metallb_ipaddresspool.yaml <<EOF
|
||||
---
|
||||
apiVersion: metallb.io/v1beta1
|
||||
kind: IPAddressPool
|
||||
metadata:
|
||||
name: public
|
||||
namespace: metallb-system
|
||||
spec:
|
||||
addresses:
|
||||
- "172.24.128.0/24"
|
||||
EOF
|
||||
|
||||
kubectl apply -f /tmp/metallb_ipaddresspool.yaml
|
||||
|
||||
tee > /tmp/metallb_l2advertisement.yaml <<EOF
|
||||
---
|
||||
apiVersion: metallb.io/v1beta1
|
||||
kind: L2Advertisement
|
||||
metadata:
|
||||
name: public
|
||||
namespace: metallb-system
|
||||
spec:
|
||||
ipAddressPools:
|
||||
- public
|
||||
EOF
|
||||
|
||||
kubectl apply -f /tmp/metallb_l2advertisement.yaml
|
||||
|
||||
Next, let's create a service of type ``LoadBalancer`` which will the
|
||||
public endpoint for all OpenStack services that we will later deploy.
|
||||
The MetalLB will assign an IP address to it (we can assinged a dedicated
|
||||
IP using annotations):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tee > /tmp/openstack_endpoint_service.yaml <<EOF
|
||||
---
|
||||
kind: Service
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: public-openstack
|
||||
namespace: openstack
|
||||
annotations:
|
||||
metallb.universe.tf/loadBalancerIPs: "172.24.128.100"
|
||||
spec:
|
||||
externalTrafficPolicy: Cluster
|
||||
type: LoadBalancer
|
||||
selector:
|
||||
app: ingress-api
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
- name: https
|
||||
port: 443
|
||||
EOF
|
||||
|
||||
This service will redirect the traffic to the ingress controller pods
|
||||
(see the ``app: ingress-api`` selector). OpenStack-Helm charts create
|
||||
``Ingress`` resources which are used by the ingress controller to configure the
|
||||
reverse proxy backend so that the traffic eventually goes to particular
|
||||
Openstack API pods.
|
||||
|
||||
By default, the ``Ingress`` objects will only contain rules for the
|
||||
``openstack.svc.cluster.local`` DNS domain. This is the internal Kubernetes domain
|
||||
and it is not supposed to be used outside the cluster. However, we can use
|
||||
the Dnsmasq to resolve the ``*.openstack.svc.cluster.local`` names to the
|
||||
``LoadBalancer`` service IP address.
|
||||
|
||||
The following command will start the Dnsmasq container with the necessary configuration:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -d --name dnsmasq --restart always \
|
||||
--cap-add=NET_ADMIN \
|
||||
--network=host \
|
||||
--entrypoint dnsmasq \
|
||||
docker.io/openstackhelm/neutron:2024.1-ubuntu_jammy \
|
||||
--keep-in-foreground \
|
||||
--no-hosts \
|
||||
--bind-interfaces \
|
||||
--address="/openstack.svc.cluster.local/172.24.128.100" \
|
||||
--listen-address="172.17.0.1" \
|
||||
--no-resolv \
|
||||
--server=8.8.8.8
|
||||
|
||||
The ``--network=host`` option is used to start the Dnsmasq container in the
|
||||
host network namespace and the ``--listen-address`` option is used to bind the
|
||||
Dnsmasq to a specific IP. Please use the configuration that suits your environment.
|
||||
|
||||
Now we can add the Dnsmasq IP to the ``/etc/resolv.conf`` file
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
echo "nameserver 172.17.0.1" > /etc/resolv.conf
|
||||
|
||||
or alternatively the ``resolvectl`` command can be used to configure the systemd-resolved.
|
||||
|
||||
.. note::
|
||||
In production environments you probably choose to use a different DNS
|
||||
domain for public Openstack endpoints. This is easy to achieve by setting
|
||||
the necessary chart values. All Openstack-Helm charts values have the
|
||||
``endpoints`` section where you can specify the ``host_fqdn_override``.
|
||||
In this case a chart will create additional ``Ingress`` resources to
|
||||
handle the external domain name and also the Keystone endpoint catalog
|
||||
will be updated.
|
||||
|
||||
Here is an example of how to set the ``host_fqdn_override`` for the Keystone chart:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
endpoints:
|
||||
identity:
|
||||
host_fqdn_override:
|
||||
public: "keystone.example.com"
|
||||
|
||||
Ceph
|
||||
----
|
||||
|
||||
Ceph is a highly scalable and fault-tolerant distributed storage
|
||||
system. It offers object storage, block storage, and
|
||||
file storage capabilities, making it a versatile solution for
|
||||
various storage needs.
|
||||
|
||||
Kubernetes CSI (Container Storage Interface) allows storage providers
|
||||
like Ceph to implement their drivers, so that Kubernetes can
|
||||
use the CSI driver to provision and manage volumes which can be
|
||||
used by stateful applications deployed on top of Kubernetes
|
||||
to store their data. In the context of OpenStack running in Kubernetes,
|
||||
the Ceph is used as a storage backend for services like MariaDB, RabbitMQ and
|
||||
other services that require persistent storage. By default OpenStack-Helm
|
||||
stateful sets expect to find a storage class named **general**.
|
||||
|
||||
At the same time, Ceph provides the RBD API, which applications
|
||||
can utilize directly to create and mount block devices distributed across
|
||||
the Ceph cluster. For example the OpenStack Cinder utilizes this Ceph
|
||||
capability to offer persistent block devices to virtual machines
|
||||
managed by the OpenStack Nova.
|
||||
|
||||
The recommended way to manage Ceph on top of Kubernetes is by means
|
||||
of the `Rook`_ operator. The Rook project provides the Helm chart
|
||||
to deploy the Rook operator which extends the Kubernetes API
|
||||
adding CRDs that enable managing Ceph clusters via Kuberntes custom objects.
|
||||
There is also another Helm chart that facilitates deploying Ceph clusters
|
||||
using Rook custom resources.
|
||||
|
||||
For details please refer to the `Rook`_ documentation and the `charts`_.
|
||||
|
||||
.. note::
|
||||
The following script `ceph-rook.sh`_ (recommended for testing only) can be used as
|
||||
an example of how to deploy the Rook Ceph operator and a Ceph cluster using the
|
||||
Rook `charts`_. Please note that the script places Ceph OSDs on loopback devices
|
||||
which is **not recommended** for production. The loopback devices must exist before
|
||||
using this script.
|
||||
|
||||
Once the Ceph cluster is deployed, the next step is to enable using it
|
||||
for services depoyed by OpenStack-Helm charts. The ``ceph-adapter-rook`` chart
|
||||
provides the necessary functionality to do this. The chart will
|
||||
prepare Kubernetes secret resources containing Ceph client keys/configs
|
||||
that are later used to interface with the Ceph cluster.
|
||||
|
||||
Here we assume the Ceph cluster is deployed in the ``ceph`` namespace.
|
||||
|
||||
The procedure consists of two steps: 1) gather necessary entities from the Ceph cluster
|
||||
2) copy them to the ``openstack`` namespace:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tee > /tmp/ceph-adapter-rook-ceph.yaml <<EOF
|
||||
manifests:
|
||||
configmap_bin: true
|
||||
configmap_templates: true
|
||||
configmap_etc: false
|
||||
job_storage_admin_keys: true
|
||||
job_namespace_client_key: false
|
||||
job_namespace_client_ceph_config: false
|
||||
service_mon_discovery: true
|
||||
EOF
|
||||
|
||||
helm upgrade --install ceph-adapter-rook openstack-helm-infra/ceph-adapter-rook \
|
||||
--namespace=ceph \
|
||||
--values=/tmp/ceph-adapter-rook-ceph.yaml
|
||||
|
||||
helm osh wait-for-pods ceph
|
||||
|
||||
tee > /tmp/ceph-adapter-rook-openstack.yaml <<EOF
|
||||
manifests:
|
||||
configmap_bin: true
|
||||
configmap_templates: false
|
||||
configmap_etc: true
|
||||
job_storage_admin_keys: false
|
||||
job_namespace_client_key: true
|
||||
job_namespace_client_ceph_config: true
|
||||
service_mon_discovery: false
|
||||
EOF
|
||||
|
||||
helm upgrade --install ceph-adapter-rook openstack-helm-infra/ceph-adapter-rook \
|
||||
--namespace=openstack \
|
||||
--values=/tmp/ceph-adapter-rook-openstack.yaml
|
||||
|
||||
helm osh wait-for-pods openstack
|
||||
|
||||
.. _Rook: https://rook.io/
|
||||
.. _charts: https://rook.io/docs/rook/latest-release/Helm-Charts/helm-charts/
|
||||
.. _ceph-rook.sh: https://opendev.org/openstack/openstack-helm-infra/src/branch/master/tools/deployment/ceph/ceph-rook.sh
|
||||
|
||||
Node labels
|
||||
-----------
|
||||
|
||||
Openstack-Helm charts rely on Kubernetes node labels to determine which nodes
|
||||
are suitable for running specific OpenStack components.
|
||||
|
||||
The following sets labels on all the Kubernetes nodes in the cluster
|
||||
including control plane nodes but you can choose to label only a subset of nodes
|
||||
where you want to run OpenStack:
|
||||
|
||||
.. code-block::
|
||||
|
||||
kubectl label --overwrite nodes --all openstack-control-plane=enabled
|
||||
kubectl label --overwrite nodes --all openstack-compute-node=enabled
|
||||
kubectl label --overwrite nodes --all openvswitch=enabled
|
||||
kubectl label --overwrite nodes --all linuxbridge=enabled
|
||||
|
||||
.. note::
|
||||
The control plane nodes are tainted by default to prevent scheduling
|
||||
of pods on them. You can untaint the control plane nodes using the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
kubectl taint nodes -l 'node-role.kubernetes.io/control-plane' node-role.kubernetes.io/control-plane-
|
||||
|
|
@ -1,56 +0,0 @@
|
|||
Setup OpenStack client
|
||||
======================
|
||||
|
||||
The OpenStack client software is a crucial tool for interacting
|
||||
with OpenStack services. In certain OpenStack-Helm deployment
|
||||
scripts, the OpenStack client software is utilized to conduct
|
||||
essential checks during deployment. Therefore, installing the
|
||||
OpenStack client on the developer's machine is a vital step.
|
||||
|
||||
The script `setup-client.sh`_ can be used to setup the OpenStack
|
||||
client.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/osh/openstack-helm
|
||||
./tools/deployment/common/setup-client.sh
|
||||
|
||||
Please keep in mind that the above script configures
|
||||
OpenStack client so it uses internal Kubernetes FQDNs like
|
||||
`keystone.openstack.svc.cluster.local`. In order to be able to resolve these
|
||||
internal names you have to configure the Kubernetes authoritative DNS server
|
||||
(CoreDNS) to work as a recursive resolver and then add its IP (`10.96.0.10` by default)
|
||||
to `/etc/resolv.conf`. This is only going to work when you try to access
|
||||
to OpenStack services from one of Kubernetes nodes because IPs from the
|
||||
Kubernetes service network are routed only between Kubernetes nodes.
|
||||
|
||||
If you wish to access OpenStack services from outside the Kubernetes cluster,
|
||||
you need to expose the OpenStack Ingress controller using an IP address accessible
|
||||
from outside the Kubernetes cluster, typically achieved through solutions like
|
||||
`MetalLB`_ or similar tools. In this scenario, you should also ensure that you
|
||||
have set up proper FQDN resolution to map to the external IP address and
|
||||
create the necessary Ingress objects for the associated FQDN.
|
||||
|
||||
It is also important to note that the above script does not actually installs
|
||||
the Openstack client package on the host but instead it creates a bash
|
||||
script `/usr/local/bin/openstack` that runs the Openstack client in a
|
||||
Docker container. If you need to pass extra command line parameters to the
|
||||
`docker run` command use the environment variable
|
||||
`OPENSTACK_CLIENT_CONTAINER_EXTRA_ARGS`. For example if you need to mount a
|
||||
directory from the host file system, you can do the following
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export OPENSTACK_CLIENT_CONTAINER_EXTRA_ARGS="-v /data:/data"
|
||||
/usr/local/bin/openstack <subcommand> <options>
|
||||
|
||||
Remember that the container file system is ephemeral and is destroyed
|
||||
when you stop the container. So if you would like to use the
|
||||
Openstack client capabilities interfacing with the file system then you have to mount
|
||||
a directory from the host file system where you will read/write necessary files.
|
||||
For example, this is useful when you create a key pair and save the private key in a file
|
||||
which is then used for ssh access to VMs. Or it could be Heat recipes
|
||||
which you prepare in advance and then use with Openstack client.
|
||||
|
||||
.. _setup-client.sh: https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deployment/common/setup-client.sh
|
||||
.. _MetalLB: https://metallb.universe.tf
|
||||
Loading…
Reference in New Issue