392 lines
16 KiB
HTML
392 lines
16 KiB
HTML
|
|
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
|
|
<title>Using Ryu Network Operating System with OpenStack as OpenFlow controller — Ryu 1.5 documentation</title>
|
|
|
|
<link rel="stylesheet" href="_static/haiku.css" type="text/css" />
|
|
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
|
<link rel="stylesheet" href="_static/print.css" type="text/css" />
|
|
|
|
<script type="text/javascript">
|
|
var DOCUMENTATION_OPTIONS = {
|
|
URL_ROOT: '',
|
|
VERSION: '1.5',
|
|
COLLAPSE_INDEX: false,
|
|
FILE_SUFFIX: '.html',
|
|
HAS_SOURCE: true
|
|
};
|
|
</script>
|
|
<script type="text/javascript" src="_static/jquery.js"></script>
|
|
<script type="text/javascript" src="_static/underscore.js"></script>
|
|
<script type="text/javascript" src="_static/doctools.js"></script>
|
|
<script type="text/javascript" src="_static/theme_extras.js"></script>
|
|
<link rel="top" title="Ryu 1.5 documentation" href="index.html" />
|
|
<link rel="up" title="OpenStack Integration" href="openstack.html" />
|
|
<link rel="next" title="Step-by-step example for testing ryu with OpenStack" href="step_by_step.html" />
|
|
<link rel="prev" title="OpenStack Integration" href="openstack.html" />
|
|
</head>
|
|
<body>
|
|
<div class="header"><h1 class="heading"><a href="index.html">
|
|
<span>Ryu 1.5 documentation</span></a></h1>
|
|
<h2 class="heading"><span>Using Ryu Network Operating System with OpenStack as OpenFlow controller</span></h2>
|
|
</div>
|
|
<div class="topnav">
|
|
|
|
<p>
|
|
«  <a href="openstack.html">OpenStack Integration</a>
|
|
  ::  
|
|
<a class="uplink" href="index.html">Contents</a>
|
|
  ::  
|
|
<a href="step_by_step.html">Step-by-step example for testing ryu with OpenStack</a>  »
|
|
</p>
|
|
|
|
</div>
|
|
<div class="content">
|
|
|
|
|
|
<div class="section" id="using-ryu-network-operating-system-with-openstack-as-openflow-controller">
|
|
<span id="using-with-openstack"></span><h1>Using Ryu Network Operating System with OpenStack as OpenFlow controller<a class="headerlink" href="#using-ryu-network-operating-system-with-openstack-as-openflow-controller" title="Permalink to this headline">¶</a></h1>
|
|
<p>This section describes how to setup openstack (nova, quantum) and
|
|
ryu-manager.
|
|
It is assumed that kvm with libvirt is used and each host machines that run
|
|
nova-compute/nova-network has two physical NICs.
|
|
It would be possible to deploy it with single NIC machines as described at
|
|
the last section.</p>
|
|
<p>NOTE: How to use nova isn’t described in this document.</p>
|
|
<div class="section" id="overview">
|
|
<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h2>
|
|
<p>Ryu is designed/implemented with for production use in mind, so it cooperates
|
|
very well with <a class="reference external" href="http://www.openstack.org/">OpenStack</a> .
|
|
With nova and quantum OVS plugin,
|
|
Ryu provides L2 segregation of Multi-tenants without any switch feature/settings
|
|
like VLAN. So it’s very easy to use/experiment/deploy this segregation as
|
|
the below figure.</p>
|
|
<blockquote>
|
|
<div><img alt="_images/logical-view.png" src="_images/logical-view.png" />
|
|
</div></blockquote>
|
|
<div class="section" id="physical-machine-setup">
|
|
<h3>Physical machine setup<a class="headerlink" href="#physical-machine-setup" title="Permalink to this headline">¶</a></h3>
|
|
<p>The following figure depicts how physical hosts are connected and each daemons
|
|
are deployed.</p>
|
|
<blockquote>
|
|
<div><img alt="_images/physical-view.png" src="_images/physical-view.png" />
|
|
</div></blockquote>
|
|
<p>Although the nova-api, nova-scheduler, nova-network and related openstack
|
|
daemons are installed in each own physical machines in the above picture,
|
|
they can be installed on a physical machine which also runs nova-compute.
|
|
Each host machine has two nics and one is connected to management LAN
|
|
and other is connected to deployment LAN.</p>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="how-to-install-setup">
|
|
<h2>How to install/setup<a class="headerlink" href="#how-to-install-setup" title="Permalink to this headline">¶</a></h2>
|
|
<p>If you are not familiar with installing/setting up nova/quantum/openvswitch
|
|
from the source, please refer to OpenStack document and get back here again.
|
|
[
|
|
<a class="reference external" href="http://docs.openstack.org/">OpenStack docs</a> ,
|
|
<a class="reference external" href="http://www.openstack.org/projects/compute/">Nova</a> ,
|
|
<a class="reference external" href="http://docs.openstack.org/incubation/">Quantum</a> ,
|
|
<a class="reference external" href="http://openvswitch.org/openstack/2011/07/25/openstack-quantum-and-open-vswitch-part-1/">OpenvSwtich and Quantum Part 1</a> ,
|
|
<a class="reference external" href="http://openvswitch.org/openstack/2011/07/25/openstack-quantum-and-open-vswitch-part-1/">OpenvSwtich and Quantum Part 2</a> ,
|
|
<a class="reference external" href="http://openvswitch.org/openstack/documentation/">OVS Quantum Plugin Documentation</a>
|
|
]</p>
|
|
<ul>
|
|
<li><dl class="first docutils">
|
|
<dt>Install ryu and run ryu-manager</dt>
|
|
<dd><ul class="first last">
|
|
<li><dl class="first docutils">
|
|
<dt>install ryu from the source code on the hosts on which you run</dt>
|
|
<dd><ul class="first last simple">
|
|
<li>nova-compute,</li>
|
|
<li>quantum-server and</li>
|
|
<li>ryu-manager.</li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
<p>This is because quantum-server and ova quantum agent which runs on
|
|
nova-compute node needs ryu-client library to communicate ryu-manager.</p>
|
|
<p>Type in ryu source directory:</p>
|
|
<div class="highlight-python"><pre>% python ./setup.py install</pre>
|
|
</div>
|
|
</li>
|
|
<li><p class="first">edit /etc/ryu/ryu.conf on the host on which you run ryu-manager
|
|
if necessary</p>
|
|
<p>No configuration is needed on hosts that runs quantum and ovs quantum
|
|
agent.</p>
|
|
</li>
|
|
<li><p class="first">run ryu network os:</p>
|
|
<div class="highlight-python"><pre>% ryu-manager [----flagfile /etc/ryu/ryu.conf]</pre>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><dl class="first docutils">
|
|
<dt>get nova source and quantum source from github</dt>
|
|
<dd><ul class="first last">
|
|
<li><p class="first">They are a bit modified from openstack master tree. They are available
|
|
at github for convinience</p>
|
|
<blockquote>
|
|
<div><ul class="simple">
|
|
<li><a class="reference external" href="https://github.com/osrg/nova/tree/ryu">https://github.com/osrg/nova/tree/ryu</a></li>
|
|
<li><a class="reference external" href="https://github.com/osrg/quantum/tree/ryu">https://github.com/osrg/quantum/tree/ryu</a></li>
|
|
</ul>
|
|
</div></blockquote>
|
|
<p>clone them by typing the followings in an appropriate directory:</p>
|
|
<div class="highlight-python"><pre>% git clone git://github.com/osrg/nova.git
|
|
% git clone git://github.com/osrg/quantum.git</pre>
|
|
</div>
|
|
<p>If you prefer https, try those:</p>
|
|
<div class="highlight-python"><pre>% git clone https://github.com/osrg/nova.git
|
|
% git clone https://github.com/osrg/quantum.git</pre>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><p class="first">Install nova and quantum as usual.
|
|
(And other Openstack related component if necessary. e.g. glance)</p>
|
|
<p>Each daemons can be installed in a single machine or in different machines.
|
|
Please refer to Openstack documentation for details.
|
|
You may want to set up multiple nova-compute nodes for interesting use case.</p>
|
|
</li>
|
|
<li><p class="first">Setup nova daemons. (Edit nova.conf)
|
|
Specifically configure nova-network and nova-compute</p>
|
|
<blockquote>
|
|
<div><ul>
|
|
<li><dl class="first docutils">
|
|
<dt>configure nova-network</dt>
|
|
<dd><ul class="first simple">
|
|
<li>–fixed_ranges=<setup here></li>
|
|
<li>–network_size=<setup here></li>
|
|
<li>–network_manager=nova.network.quantum.manager.QuantumManager</li>
|
|
<li>–quantum_connection_host=<quantum server ip address></li>
|
|
<li>–firewall_driver=quantum.plugins.ryu.nova.firewall.NopFirewallDriver</li>
|
|
<li>–quantum_use_dhcp</li>
|
|
</ul>
|
|
<blockquote>
|
|
<div><p>NOP firewall driver is newly introduced for demonstrating Ryu
|
|
capability.
|
|
If you want, other existing firewall driver can be specified.
|
|
But such specification don’t have any effect in fact
|
|
because ryu directly controls packets to VM instance via OVS bypassing
|
|
netfilter/iptables.</p>
|
|
</div></blockquote>
|
|
<ul class="last simple">
|
|
<li>–linuxnet_interface_driver=quantum.plugins.ryu.nova.linux_net.LinuxOVSRyuInterfaceDriver</li>
|
|
<li>–linuxnet_ovs_ryu_api_host=<IP address of ryu server>:<Ryu rest API port></li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><p class="first">set up OVS on each nova-compute node</p>
|
|
<p>If Ubuntu is used, you can install it from packages as
|
|
openvswitch-datapath-dkms, openvswitch-common, openvswitch-switch
|
|
If you already use bridge, you may need to edit /etc/modules to load
|
|
openvswitch kernel module, openvswitch_mod and brcompat_mod, before
|
|
bridge module and reboot to unload bridge module.</p>
|
|
</li>
|
|
</ul>
|
|
<blockquote>
|
|
<div><p>And then create ovs bridge:</p>
|
|
<div class="highlight-python"><div class="highlight"><pre><span class="c"># ovs-vsctl add-br <bridge name: usually br-int></span>
|
|
</pre></div>
|
|
</div>
|
|
<p>And if you connect NIC to OVS bridge.:</p>
|
|
<div class="highlight-python"><div class="highlight"><pre><span class="c"># ovs-vsctl add-port <bridge name:br-int> <ether interface: e.g. eth<N>></span>
|
|
</pre></div>
|
|
</div>
|
|
</div></blockquote>
|
|
<ul>
|
|
<li><dl class="first docutils">
|
|
<dt>configure each nova-compute</dt>
|
|
<dd><ul class="first last simple">
|
|
<li>–libvirt_type=kvm</li>
|
|
<li>–libvirt_ovs_integration_bridge=<OVS bridge:br-int></li>
|
|
<li>–libvirt_vif_type=ethernet</li>
|
|
<li>–libvirt_vif_driver=quantum.plugins.ryu.nova.vif.LibvirtOpenVswitchOFPRyuDriver</li>
|
|
<li>–libvirt_ovs_ryu_api_host=<IP address of ryu server>:<Ryu rest API port></li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
</ul>
|
|
</div></blockquote>
|
|
</li>
|
|
<li><dl class="first docutils">
|
|
<dt>install quantum server and have quantum to use OVS pluging</dt>
|
|
<dd><ul class="first last">
|
|
<li><dl class="first docutils">
|
|
<dt>Edit [PLUGIN] section of /etc/quantum/plugins.ini</dt>
|
|
<dd><ul class="first last simple">
|
|
<li>provider = quantum.plugins.ryu.ryu_quantum_plugin.RyuQuantumPlugin</li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><p class="first">Edit [DATABASE] and [OVS] section of /etc/quantum/plugins/ryu/ryu.ini</p>
|
|
<ul class="simple">
|
|
<li>[DATABASE] section<ul>
|
|
<li>sql_connection = <sql connection to your db></li>
|
|
</ul>
|
|
</li>
|
|
<li>[OVS] section<ul>
|
|
<li>integration-bridge = <OVS bridge name: br-int></li>
|
|
<li>openflow-controller = <ryu-manager IP address>:<ryu openflow port: default 6633></li>
|
|
<li>openflow-rest-api = <ryu-manager IP address>:<RYU reset API port: default 8080></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li><p class="first">Run quantum server</p>
|
|
</li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><dl class="first docutils">
|
|
<dt>install quantum OVS agent on each nova-compute node</dt>
|
|
<dd><ul class="first last">
|
|
<li><p class="first">Edit /etc/quantum/plugins/ryu/ryu.ini</p>
|
|
</li>
|
|
<li><p class="first">copy the ryu_quantum_agent.py into nova-compute/network node.</p>
|
|
<p>The agent isn’t installed by setup.py so that you have to copy it manually.
|
|
ryu_quantum_agent.py is located at
|
|
<quantum source base>/quantum/plugins/ryu/agent/ryu_quantum_agent.py</p>
|
|
</li>
|
|
<li><p class="first">Run ryu agent:</p>
|
|
<div class="highlight-python"><div class="highlight"><pre><span class="c"># ryu_quantum_agent.py -v /etc/quantum/plugins/ryu/ryu.ini</span>
|
|
</pre></div>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><p class="first">Then as usual openstack nova operation, create user, project, network and
|
|
run instances.</p>
|
|
</li>
|
|
<li><p class="first">Enjoy!</p>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
<div class="section" id="testing">
|
|
<h2>Testing<a class="headerlink" href="#testing" title="Permalink to this headline">¶</a></h2>
|
|
<p>Yay, now you have ryu network Operating System set up.
|
|
You would want to really they are L2-segregated.</p>
|
|
<ul class="simple">
|
|
<li>create multi projects and run instances.</li>
|
|
<li>ping/traceroute between them.</li>
|
|
<li>tcpdump in the instances</li>
|
|
</ul>
|
|
<p>The routing between gateway(gw-xxx) of each tenants are disabled
|
|
by nova.network.linux_net.LinuxOVSOFInterfaceDriver by installing iptables
|
|
rule on nova-network host:</p>
|
|
<div class="highlight-python"><div class="highlight"><pre><span class="c"># iptable -t filter -A nova-network-FORWARD --in-interface gw-+ --out-interface gw-+</span>
|
|
</pre></div>
|
|
</div>
|
|
<p>Thus pinging/tracerouting between VMs in distinct tenants doesn’t work.
|
|
If you drop the above rule by:</p>
|
|
<div class="highlight-python"><div class="highlight"><pre><span class="c"># iptable -t filter -D nova-network-FORWARD --in-interface gw-+ --out-interface gw-+</span>
|
|
</pre></div>
|
|
</div>
|
|
<p>You will see ping/tracerout works. Please notice that the packets go through
|
|
gw-xxx and gw-yyy, not directly.</p>
|
|
<blockquote>
|
|
<div><img alt="_images/trace-route.png" src="_images/trace-route.png" />
|
|
</div></blockquote>
|
|
</div>
|
|
<div class="section" id="caveats">
|
|
<h2>Caveats<a class="headerlink" href="#caveats" title="Permalink to this headline">¶</a></h2>
|
|
<ul>
|
|
<li><dl class="first docutils">
|
|
<dt>Run the following daemons in this order</dt>
|
|
<dd><ol class="first arabic simple">
|
|
<li>Run Ryu network Operating System</li>
|
|
<li>Run quantum with Ryu plugin</li>
|
|
<li>Run quantum Ryu agent</li>
|
|
<li>run your guest instance</li>
|
|
</ol>
|
|
<p class="last">For now, ryu-manager doesn’t have persistent store, so if it’s rebooted,
|
|
all the necessary information must be told again from quantum server/agent.</p>
|
|
</dd>
|
|
</dl>
|
|
</li>
|
|
<li><p class="first">nova-manage network delete doesn’t work</p>
|
|
<p>At this moment, quantum doesn’t implement network delete fully yet.
|
|
If you issue the command, it fails. And you need to fix nova/quantum DB
|
|
by hand using SQL.</p>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
<div class="section" id="appendix">
|
|
<h2>Appendix<a class="headerlink" href="#appendix" title="Permalink to this headline">¶</a></h2>
|
|
<p>In the above, two physical NIC deployment is described.
|
|
Some people may want to use those settings with single NIC machine or even
|
|
with single machine.
|
|
It would be possible as the following pictures, but we haven’t tested those
|
|
setting. If you success it, please report it.</p>
|
|
<div class="section" id="single-nic-setup">
|
|
<h3>single NIC setup<a class="headerlink" href="#single-nic-setup" title="Permalink to this headline">¶</a></h3>
|
|
<p>If your host machines have only single NIC, it would be possible to use
|
|
Ryu network Operating System with Linux bridge. However we haven’t tested such
|
|
setups.</p>
|
|
<blockquote>
|
|
<div><img alt="_images/compute-node.png" src="_images/compute-node.png" />
|
|
</div></blockquote>
|
|
</div>
|
|
<div class="section" id="all-in-one-setup">
|
|
<h3>All-in-One Setup<a class="headerlink" href="#all-in-one-setup" title="Permalink to this headline">¶</a></h3>
|
|
<p>You can also setup in single physical host as the following picture.</p>
|
|
<blockquote>
|
|
<div><img alt="_images/minimul-setup.png" src="_images/minimul-setup.png" />
|
|
</div></blockquote>
|
|
<p>You can setup the above environment quickly using DevStack.</p>
|
|
<blockquote>
|
|
<div><ol class="arabic">
|
|
<li><p class="first">Install Ubuntu 11.10 (Oneiric)</p>
|
|
</li>
|
|
<li><p class="first">Download Ryu enabled DevStack from github</p>
|
|
<div class="highlight-python"><pre>% git clone git://github.com/osrg/devstack.git</pre>
|
|
</div>
|
|
</li>
|
|
<li><p class="first">Start the install</p>
|
|
<div class="highlight-python"><pre>% cd devstack; ./stack.sh</pre>
|
|
</div>
|
|
<p>It will take a few minutes.</p>
|
|
</li>
|
|
</ol>
|
|
</div></blockquote>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
</div>
|
|
<div class="bottomnav">
|
|
|
|
<p>
|
|
«  <a href="openstack.html">OpenStack Integration</a>
|
|
  ::  
|
|
<a class="uplink" href="index.html">Contents</a>
|
|
  ::  
|
|
<a href="step_by_step.html">Step-by-step example for testing ryu with OpenStack</a>  »
|
|
</p>
|
|
|
|
</div>
|
|
|
|
<div class="footer">
|
|
© Copyright 2011, 2012 Nippon Telegraph and Telephone Corporation.
|
|
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
|
|
</div>
|
|
</body>
|
|
</html> |