Now we suggest to use 'openstack' commands to replace individual commands of every project. Change-Id: I47dac03e9da1f00f33bd7caba003de0dd1a110b6
5.3 KiB
OpenShift Origin Highly Available Environment
This nested heat stack deploys a highly-available OpenShift Origin environment.
Resources Deployed
- 6 instances
- Highly available OpenShift broker set (3)
- OpenShift nodes (3)
- 7 floating IPs (includes one for LBaaS VIP)
- LBaaS, consisting of health monitor (HTTPS), pool, virtual IP (VIP)
- Integrated BIND server on broker 1 for dynamic DNS updates
Deployment
zone transferred to
upstream DNS (IT)
\ ----------------------
\ / mongo replica set \
\ / ActiveMQ pool \
--\--------- ------------ ------------
| BIND | | | | | |
-------- |---| broker 2 |---| broker 3 |
| broker 1 | | | | |
------------ ------------ ------------
\ | /
\ | /
LBaaS agent (API) ---------------- developers
/ | \
/ | \
------------ ------------ ------------
| | | | | |
| node 1 |---| node 2 |---| node 3 | ---- application
| | | | | | users
------------ ------------ ------------
Requirements
- Neutron networking: one private and one public network
- Compute quota for six VM instances
- Pool of seven available floating IP addresses. Addresses will be created and assigned at deployment.
- Load Balancer as a Server (LBaaS) configured. See neutron lbaas agent configuration section.
- IP address of upstream (IT) DNS server for zone transfers
Files
These templates are Heat Orchestration Templates (HOT). Environment files are used to reduce CLI parameters and provide a way to reuse resources.
- Templates
- oso_ha_stack.yaml
- oso_node_stack.yaml
- Environments
- oso_ha_env.yaml
- oso_node_env.yaml
How to Deploy
-
git clone https://github.com/openstack/heat-templates.git
this repository -
Change to this directory
cd heat-templates/openshift-origin/centos65/highly-available/
-
Edit heat environment file
oso_ha_env.yaml
according to your environment. -
Launch highly available OpenShift stack
openstack stack create openshift-ha-stack -t oso_ha_stack.yaml -e oso_ha_env.yaml
-
Monitor progress. Options include:
tail -f /var/log/heat/heat-engine.log
tail -f /tmp/openshift.out
openstack stack list
openstack stack resource list openshift-ha-stack
Scaling: Adding Nodes
OpenShift nodes may be manually added as needed using the OpenShift node heat template.
-
From directory
heat-templates/openshift-origin/centos65/highly-available/
edit the heat environment fileoso_node_env.yaml
-
Launch node stack. This will deploy a single node server with attached cinder volume and floating IP address. Be sure to pass in the node hostname parameter to override the default.
openstack stack create openshift-node -t oso_node_stack.yaml -e oso_node_env.yaml --parameter "node_hostname=node4"
-
On broker1 add a DNS record for the new node server in
/var/named/dynamic/<my_domain>.db
. To force a zone transfer to the upstream DNS increment the serial number by 1 and runrndc freeze ; rndc thaw
.
Additional configuration Steps
-
Add brokers to LBaaS pool. On OpenStack:
neutron lb-member-create --address <broker1_fixed_ip> --protocol-port 443 oso_broker_lb_pool neutron lb-member-create --address <broker2_fixed_ip> --protocol-port 443 oso_broker_lb_pool neutron lb-member-create --address <broker3_fixed_ip> --protocol-port 443 oso_broker_lb_pool
-
Add session persistence to LBaaS virtual IP (VIP):
neutron lb-vip-update oso_broker_vip --session-persistence type=dict type='SOURCE_IP'
-
Update upstream DNS server to accept zone transfers from the OpenShift dynamic DNS. An example configuration would be to add a slave zone to /var/named.conf
zone "<openshift_domain_name>" { type slave; file "slaves/<openshift_domain_name>.db"; masters { <broker1_ip_address>; }; };
- If the upstream DNS configuration is not available a test client machine may be pointed to the broker 1 IP address (e.g. edit /etc/resolv.conf).
-
Create districts. The following creates a small district and adds two nodes to the district.
oo-admin-ctl-district -c create -n small_district -p small oo-admin-ctl-district -c add-node -n small_district -i <node1_hostname> oo-admin-ctl-district -c add-node -n small_district -i <node2_hostname>
Troubleshooting
oo-mco ping
on a broker to verify nodes are registeredoo-diagnostics -v
on a broker to run a comprehensive set of testsoo-accept-node -v
on a node- If LBaaS is not set up any broker hostname can be used temporarily as the developer and node API target. Be sure to edit
/etc/openshift/node.conf
.