5.3 KiB

Raw Blame History

OpenShift Origin Highly Available Environment

This nested heat stack deploys a highly-available OpenShift Origin environment.

Resources Deployed

6 instances
- Highly available OpenShift broker set (3)
- OpenShift nodes (3)
7 floating IPs (includes one for LBaaS VIP)
LBaaS, consisting of health monitor (HTTPS), pool, virtual IP (VIP)
Integrated BIND server on broker 1 for dynamic DNS updates

Deployment

   zone transferred to
    upstream DNS (IT)
              \          ----------------------
               \        /   mongo replica set  \
                \      /      ActiveMQ pool     \
               --\---------   ------------   ------------
               | BIND |   |   |          |   |          |
               --------   |---| broker 2 |---| broker 3 |
               | broker 1 |   |          |   |          |
               ------------   ------------   ------------
                           \       |          /
                            \      |         /
                            LBaaS agent (API) ---------------- developers
                            /      |       \
                           /       |        \
               ------------   ------------   ------------
               |          |   |          |   |          |
               |  node 1  |---|  node 2  |---|  node  3 | ---- application
               |          |   |          |   |          |         users
               ------------   ------------   ------------

Requirements

Neutron networking: one private and one public network
Compute quota for six VM instances
Pool of seven available floating IP addresses. Addresses will be created and assigned at deployment.
Load Balancer as a Server (LBaaS) configured. See neutron lbaas agent configuration section.
IP address of upstream (IT) DNS server for zone transfers

Files

These templates are Heat Orchestration Templates (HOT). Environment files are used to reduce CLI parameters and provide a way to reuse resources.

Templates
- oso_ha_stack.yaml
- oso_node_stack.yaml
Environments
- oso_ha_env.yaml
- oso_node_env.yaml

How to Deploy

git clone https://github.com/openstack/heat-templates.git this repository

Change to this directory

 cd heat-templates/openshift-origin/centos65/highly-available/

Edit heat environment file oso_ha_env.yaml according to your environment.

Launch highly available OpenShift stack

 openstack stack create openshift-ha-stack -t oso_ha_stack.yaml -e oso_ha_env.yaml

Monitor progress. Options include:

tail -f /var/log/heat/heat-engine.log
tail -f /tmp/openshift.out
openstack stack list
openstack stack resource list openshift-ha-stack

Scaling: Adding Nodes

OpenShift nodes may be manually added as needed using the OpenShift node heat template.

From directory heat-templates/openshift-origin/centos65/highly-available/ edit the heat environment file oso_node_env.yaml
Launch node stack. This will deploy a single node server with attached cinder volume and floating IP address. Be sure to pass in the node hostname parameter to override the default.
```
 openstack stack create openshift-node -t oso_node_stack.yaml -e oso_node_env.yaml --parameter "node_hostname=node4"
```
On broker1 add a DNS record for the new node server in /var/named/dynamic/<my_domain>.db. To force a zone transfer to the upstream DNS increment the serial number by 1 and run rndc freeze ; rndc thaw.

Additional configuration Steps

Add brokers to LBaaS pool. On OpenStack:

 neutron lb-member-create --address <broker1_fixed_ip> --protocol-port 443 oso_broker_lb_pool
 neutron lb-member-create --address <broker2_fixed_ip> --protocol-port 443 oso_broker_lb_pool
 neutron lb-member-create --address <broker3_fixed_ip> --protocol-port 443 oso_broker_lb_pool

Add session persistence to LBaaS virtual IP (VIP):

 neutron lb-vip-update oso_broker_vip --session-persistence type=dict type='SOURCE_IP'

Update upstream DNS server to accept zone transfers from the OpenShift dynamic DNS. An example configuration would be to add a slave zone to /var/named.conf
```
 zone "<openshift_domain_name>" {
     type slave;
     file "slaves/<openshift_domain_name>.db";
     masters { <broker1_ip_address>; };
 };
```
- If the upstream DNS configuration is not available a test client machine may be pointed to the broker 1 IP address (e.g. edit /etc/resolv.conf).

Create districts. The following creates a small district and adds two nodes to the district.

 oo-admin-ctl-district -c create -n small_district -p small
 oo-admin-ctl-district -c add-node -n small_district -i <node1_hostname>
 oo-admin-ctl-district -c add-node -n small_district -i <node2_hostname>

Troubleshooting

oo-mco ping on a broker to verify nodes are registered
oo-diagnostics -v on a broker to run a comprehensive set of tests
oo-accept-node -v on a node
If LBaaS is not set up any broker hostname can be used temporarily as the developer and node API target. Be sure to edit /etc/openshift/node.conf.

5.3 KiB Raw Blame History