CEPH Mons autodetection doesn't work reliably across all (#156)

We should use known parameters for the custer POD ips and the MON_IP to make MON IP discovery more consistent.

ceph/templates/statefulset-mon.yaml

@@ -72,10 +72,16 @@ spec:
               value: MON
             - name: KV_TYPE
               value: k8s
-            - name: NETWORK_AUTO_DETECT
-              value: "4"
             - name: CLUSTER
               value: ceph
+            - name: NETWORK_AUTO_DETECT
+              value: "0"
+            - name: CEPH_PUBLIC_NETWORK
+              value: {{ .Values.network.public | quote }}
+            - name: MON_IP
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP
           volumeMounts:
             - name: ceph-conf
               mountPath: /etc/ceph

ceph/values.yaml

@@ -25,6 +25,7 @@ labels:
   node_selector_value: enabled
 
 network:
+  public: "10.25.0.0/16"
   port: 
     mon: 6789
     rgw_ingress: 80

docs/installation/getting-started.md

@@ -268,7 +268,7 @@ Please ensure that you use ``--purge`` whenever deleting a project.
 ## Ceph Installation and Verification
 Install the first service, which is Ceph. If all instructions have been followed as mentioned above, this installation should go smoothly. Use the following command to install Ceph:
 ```
-admin@kubenode01:~$ helm install --name=ceph local/ceph --namespace=ceph
+admin@kubenode01:~$ helm install --set network.public=$osd_public_network --name=ceph local/ceph --namespace=ceph
 ```
 
 ## Bootstrap Installation