diff --git a/doc/source/install/multinode.rst b/doc/source/install/multinode.rst index c3b5de5f1c..33e9ba6fb0 100644 --- a/doc/source/install/multinode.rst +++ b/doc/source/install/multinode.rst @@ -390,6 +390,12 @@ created, using the following commands: kubectl exec -n ceph ceph-mon-0 -- ceph osd pool create images 8 kubectl exec -n ceph ceph-mon-0 -- ceph osd pool create vms 8 +The number of placement groups can be altered by replacing the 8 +to meet your needs. It is important to note that using too large +of a number for your placement groups may result in Ceph +becoming unhealthy. For more information on this topic, reference +Ceph's documentation `here `_. + MariaDB Installation and Verification ------------------------------------- diff --git a/doc/source/troubleshooting/database.rst b/doc/source/troubleshooting/database.rst index 62da6b1f65..b8c6ac4da9 100644 --- a/doc/source/troubleshooting/database.rst +++ b/doc/source/troubleshooting/database.rst @@ -8,34 +8,6 @@ deploying Charts in this repository. Galera Cluster ============== -**CHART:** openstack-helm/mariadb (when ``developer-mode: false``) - -MariaDB is a ``StatefulSet`` (``PetSets`` have been retired in -Kubernetes v1.5.0). As such, it initiates a 'seed' which is used to -deploy MariaDB members via `affinity/anti-affinity -`__ -features. Ceph uses this as well. So what you will notice is the -following behavior: - -:: - - openstack mariadb-0 0/1 Running 0 28s 10.25.49.199 kubenode05 - openstack mariadb-seed-0ckf4 1/1 Running 0 48s 10.25.162.197 kubenode01 - - - NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE - openstack mariadb-0 1/1 Running 0 1m 10.25.49.199 kubenode05 - openstack mariadb-1 0/1 Pending 0 0s - openstack mariadb-1 0/1 Pending 0 0s kubenode04 - openstack mariadb-1 0/1 ContainerCreating 0 0s kubenode04 - openstack mariadb-1 0/1 Running 0 3s 10.25.178.74 kubenode04 - -What you're seeing is the output of -``kubectl get pods -o wide --all-namespaces``, which is used to monitor -the seed host preparing each of the MariaDB/Galera members in order: -mariadb-0, then mariadb-1, then mariadb-2. This process can take up to a -few minutes, so be patient. - To test MariaDB, do the following: :: @@ -50,32 +22,3 @@ To test MariaDB, do the following: | performance_schema | +--------------------+ admin@kubenode01:~/projects/openstack-helm$ - -Now you can see that MariaDB is loaded, with databases intact! If you're -at this point, the rest of the installation is easy. You can run the -following to check on Galera: - -:: - - admin@kubenode01:~/projects/openstack-helm$ kubectl describe po/mariadb-0 -n openstack - Name: mariadb-0 - Namespace: openstack - Node: kubenode05/192.168.3.25 - Start Time: Fri, 23 Dec 2016 16:15:49 -0500 - Labels: app=mariadb - galera=enabled - Status: Running - IP: 10.25.49.199 - Controllers: StatefulSet/mariadb - ... - ... - ... - FirstSeen LastSeen Count From SubObjectPath Type Reason Message - --------- -------- ----- ---- ------------- -------- ------ ------- - 5s 5s 1 {default-scheduler } Normal Scheduled Successfully assigned mariadb-0 to kubenode05 - 3s 3s 1 {kubelet kubenode05} spec.containers{mariadb} Normal Pulling pulling image "quay.io/stackanetes/stackanetes-mariadb:newton" - 2s 2s 1 {kubelet kubenode05} spec.containers{mariadb} Normal Pulled Successfully pulled image "quay.io/stackanetes/stackanetes-mariadb:newton" - 2s 2s 1 {kubelet kubenode05} spec.containers{mariadb} Normal Created Created container with docker id f702bd7c11ef; Security:[seccomp=unconfined] - 2s 2s 1 {kubelet kubenode05} spec.containers{mariadb} Normal Started Started container with docker id f702bd7c11ef - -So you can see that galera is enabled. diff --git a/doc/source/troubleshooting/development.rst b/doc/source/troubleshooting/development.rst deleted file mode 100644 index 6585bab466..0000000000 --- a/doc/source/troubleshooting/development.rst +++ /dev/null @@ -1,64 +0,0 @@ -Diagnosing the problem ----------------------- - -In order to protect your general sanity, we've included a curated list -of verification and troubleshooting steps that may help you avoid some -potential issues while developing Openstack-Helm. - -MariaDB -~~~~~~~ - -To verify the state of MariaDB, use the following command: - -:: - - $ kubectl exec mariadb-0 -it -n openstack -- mysql -u root -p password -e 'show databases;' - +--------------------+ - | Database | - +--------------------+ - | information_schema | - | mysql | - | performance_schema | - +--------------------+ - $ - -Helm Server/Repository -~~~~~~~~~~~~~~~~~~~~~~ - -Sometimes you will run into Helm server or repository issues. For our -purposes, it's mostly safe to whack these. If you are developing -charts for other projects, use at your own risk (you most likely know -how to resolve these issues already). - -To check for a running instance of Helm Server: - -:: - - $ ps -a | grep "helm serve" - 29452 ttys004 0:00.23 helm serve . - 35721 ttys004 0:00.00 grep --color=auto helm serve - -Kill the "helm serve" running process: - -:: - - $ kill 29452 - -To clear out previous Helm repositories, and reinstall a local -repository: - -:: - - $ helm repo list - NAME URL - stable https://kubernetes-charts.storage.googleapis.com/ - local http://localhost:8879/charts - $ - $ helm repo remove local - -This allows you to read your local repository, if you ever need to do -these steps: - -:: - - $ helm repo add local http://localhost:8879/charts diff --git a/doc/source/troubleshooting/index.rst b/doc/source/troubleshooting/index.rst index 0f6365b3f1..be2ff1952e 100644 --- a/doc/source/troubleshooting/index.rst +++ b/doc/source/troubleshooting/index.rst @@ -8,8 +8,6 @@ Sometimes things go wrong. These guides will help you solve many common issues w :maxdepth: 2 database - development - networking persistent-storage Getting help diff --git a/doc/source/troubleshooting/networking.rst b/doc/source/troubleshooting/networking.rst deleted file mode 100644 index 6171353cc7..0000000000 --- a/doc/source/troubleshooting/networking.rst +++ /dev/null @@ -1,9 +0,0 @@ -========== -Networking -========== - -This guide is to help users debug any networking issues when deploying -Charts in this repository. - -Diagnosing the problem -====================== diff --git a/doc/source/troubleshooting/persistent-storage.rst b/doc/source/troubleshooting/persistent-storage.rst index fd78247b88..7ba5e6ad20 100644 --- a/doc/source/troubleshooting/persistent-storage.rst +++ b/doc/source/troubleshooting/persistent-storage.rst @@ -3,81 +3,15 @@ Persistent Storage ================== This guide is to help users debug any general storage issues when -deploying Charts in this repository. +deploying charts in this repository. Ceph ==== -**CHART:** openstack-helm/ceph +Ceph Deployment Status +~~~~~~~~~~~~~~~~~~~~~~ -Ceph Validating PVC -~~~~~~~~~~~~~~~~~~~ - -To validate persistent volume claim (PVC) creation, we've placed a test -manifest in the ``./test/`` directory. Deploy this pvc and explore the -deployment: - -:: - - admin@kubenode01:~$ kubectl get pvc -o wide --all-namespaces -w - NAMESPACE NAME STATUS VOLUME CAPACITY ACCESSMODES AGE - ceph pvc-test Bound pvc-bc768dea-c93e-11e6-817f-001fc69c26d1 1Gi RWO 9h - admin@kubenode01:~$ - -The output above indicates that the PVC is 'bound' correctly. Now -digging deeper: - -:: - - admin@kubenode01:~/projects/openstack-helm$ kubectl describe pvc pvc-test -n ceph - Name: pvc-test - Namespace: ceph - StorageClass: general - Status: Bound - Volume: pvc-bc768dea-c93e-11e6-817f-001fc69c26d1 - Labels: - Capacity: 1Gi - Access Modes: RWO - No events. - admin@kubenode01:~/projects/openstack-helm$ - -We can see that we have a VolumeID, and the 'capacity' is 1GB. It is a -'general' storage class. It is just a simple test. You can safely delete -this test by issuing the following: - -:: - - admin@kubenode01:~/projects/openstack-helm$ kubectl delete pvc pvc-test -n ceph - persistentvolumeclaim "pvc-test" deleted - admin@kubenode01:~/projects/openstack-helm$ - -Ceph Validating StorageClass -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Next we can look at the storage class, to make sure that it was created -correctly: - -:: - - admin@kubenode01:~$ kubectl describe storageclass/general - Name: general - IsDefaultClass: No - Annotations: - Provisioner: kubernetes.io/rbd - Parameters: adminId=admin,adminSecretName=pvc-ceph-conf-combined-storageclass,adminSecretNamespace=ceph,monitors=ceph-mon.ceph:6789,pool=rbd,userId=admin,userSecretName=pvc-ceph-client-key - No events. - admin@kubenode01:~$ - -The parameters is what we're looking for here. If we see parameters -passed to the StorageClass correctly, we will see the -``ceph-mon.ceph:6789`` hostname/port, things like ``userid``, and -appropriate secrets used for volume claims. This all looks great, and it -time to Ceph itself. - -Ceph Validation -~~~~~~~~~~~~~~~ - -Most commonly, we want to validate that Ceph is working correctly. This +First, we want to validate that Ceph is working correctly. This can be done with the following ceph command: :: @@ -96,26 +30,55 @@ can be done with the following ceph command: admin@kubenode01:~$ Use one of your Ceph Monitors to check the status of the cluster. A -couple of things to note above; our health is 'HEALTH\_OK', we have 3 -mons, we've established a quorum, and we can see that our active mds is -'ceph-mds-2810413505-gtjgv'. We have a healthy environment. +couple of things to note above; our health is `HEALTH\_OK`, we have 3 +mons, we've established a quorum, and we can see that all of our OSDs +are up and in the OSD map. -For Glance and Cinder to operate, you will need to create some storage -pools for these systems. Additionally, Nova can be configured to use a -pool as well, but this is off by default. +PVC Preliminary Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Before proceeding, it is important to ensure that you have deployed a +client key in the namespace you wish to fulfill ``PersistentVolumeClaims``. +To verify that your deployment namespace has a client key: :: - kubectl exec -n ceph -it ceph-mon-0 ceph osd pool create volumes 128 - kubectl exec -n ceph -it ceph-mon-0 ceph osd pool create images 128 + admin@kubenode01: $ kubectl get secret -n openstack + NAME TYPE DATA AGE + default-token-nvl10 kubernetes.io/service-account-token 3 7d + pvc-ceph-client-key kubernetes.io/rbd 1 6m -Nova storage would be added like this: +Without this, your RBD-backed PVCs will never reach the ``Bound`` state. For +more information, see how to `activate namespace for ceph <../install/multinode.html#activating-control-plane-namespace-for-ceph>`_. + +Note: This step is not relevant for PVCs within the same namespace Ceph +was deployed. + +Ceph Validating PVC Operation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To validate persistent volume claim (PVC) creation, we've placed a test +manifest `here `_. +Deploy this manifest and verify the job completes successfully. + +Ceph Validating StorageClass +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Next we can look at the storage class, to make sure that it was created +correctly: :: - kubectl exec -n ceph -it ceph-mon-0 ceph osd pool create vms 128 + admin@kubenode01:~$ kubectl describe storageclass/general + Name: general + IsDefaultClass: No + Annotations: + Provisioner: kubernetes.io/rbd + Parameters: adminId=admin,adminSecretName=pvc-ceph-conf-combined-storageclass,adminSecretNamespace=ceph,monitors=ceph-mon.ceph:6789,pool=rbd,userId=admin,userSecretName=pvc-ceph-client-key + No events. + admin@kubenode01:~$ -The choosing the amount of storage is up to you and can be changed by -replacing the 128 to meet your needs. - -We are now ready to install our next chart, MariaDB. +The parameters are what we're looking for here. If we see parameters +passed to the StorageClass correctly, we will see the +``ceph-mon.ceph:6789`` hostname/port, things like ``userid``, and +appropriate secrets used for volume claims.