publications/tutorial/index.html

818 lines
20 KiB
HTML

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en-US">
<head>
<meta name="generator" content=
"HTML Tidy for Linux/x86 (vers 1st November 2003), see www.w3.org" />
<title>Developing OpenStack with devstack</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="copyright" content=
"Copyright &#169; 2005-2010 W3C (MIT, ERCIM, Keio)" />
<meta name="duration" content="60" />
<meta name="font-size-adjustment" content="0" />
<link rel="stylesheet" href="styles/slidy.css" type="text/css" />
<link rel="stylesheet" href="styles/openstack.css" type="text/css" />
<script src="scripts/slidy.js" charset="utf-8" type="text/javascript">
</script>
<style>
div.slide ul {
font-size: 100%
}
</style>
</head>
<body>
<div class="background"><img alt="" id="head-icon"
src="graphics/open-stack-cloud-computing-logo-2.png" /></div>
<div class="slide cover title">
<!-- hidden style graphics to ensure they are saved with other content -->
<img class="hidden" src="graphics/bullet.png" alt="" />
<img class="hidden" src="graphics/fold.gif" alt="" />
<img class="hidden" src="graphics/unfold.gif" alt="" />
<img class="hidden" src="graphics/fold-dim.gif" alt="" />
<img class="hidden" src="graphics/nofold-dim.gif" alt="" />
<img class="hidden" src="graphics/unfold-dim.gif" alt="" />
<img class="hidden" src="graphics/bullet-fold.gif" alt="" />
<img class="hidden" src="graphics/bullet-unfold.gif" alt="" />
<img class="hidden" src="graphics/bullet-fold-dim.gif" alt="" />
<img class="hidden" src="graphics/bullet-nofold-dim.gif" alt="" />
<img class="hidden" src="graphics/bullet-unfold-dim.gif" alt="" />
<img src="graphics/openstack-cloud-software-vertical-large.png" alt="OpenStack logo"
class="cover" /><br clear="all" />
<h1>OpenStack Overview</h1>
Monty Taylor
&lt;<a href="mailto:mordred@inaugust.com">mordred@inaugust.com</a>&gt;|
twitter:e_monty <br />
James E. Blair
&lt;<a href="mailto:corvus@inaugust.com">corvus@inaugust.com</a>&gt;<br />
<a
href="https://openstack-ci.github.com/publications/tutorial">https://openstack-ci.github.com/publications/tutorial</a>
</div>
<div class="slide">
<h1>Overview</h1>
<ul>
<li>OpenStack Overview</li>
<li>Dev Process Overview</li>
<li>Devstack Install</li>
</ul></div>
<div class="slide">
<h1>Anti-overview</h1>
<ul>
<li>In Depth OpenStack Tutorial</li>
<li>Tutorial on chef/puppet/juju/cfengine/salt</li>
<li>Endorsement of any of the above</li>
<li>End-user Cloud Tutorial</li>
<li>A Duck</li>
</ul></div>
<div class="slide">
<h1>OpenStack Overview</h1>
<ul>
<li>Software to run a cloud</li>
<li>Made up of several (growing) projects</li>
<li>Open Source</li>
</ul></div>
<div class="slide">
<h1>Projects</h1>
<ul>
<li>nova (compute)</li>
<li>swift (object storge)</li>
<li>glance (image service)</li>
<li>keystone (identity service)</li>
<li>quantum (network service)</li>
<li>melange (network information service)</li>
<li>horizon (dashboard)</li>
</ul></div>
<div class="slide">
<h1>Libraries</h1>
<ul>
<li>python-novaclient</li>
<li>python-keystoneclient</li>
<li>python-quantumclient</li>
<li>python-melangeclient</li>
</ul></div>
<div class="slide">
<h1>Tools</h1>
<ul>
<li>tempest</li>
<li>git-review</li>
<li>devstack</li>
<li>openstack.nose_plugin</li>
<li>openstack-ci</li>
<li>openstack-ci-puppet</li>
</ul></div>
<div class="slide">
<h1>Nova Componenets</h1>
<ul>
<li>API</li>
<li>Scheduler</li>
<li>Compute</li>
<li>Network</li>
<li>Volume</li>
<li>messaging</li>
<li>database</li>
</ul></div>
<div class="slide">
<h1>Who is Running It?</h1>
<ul>
<li>Rackspace<ul>
<li>Cloud Files == swift</li>
<li>Cloud Servers (beta) == nova</li></ul>
<li>HP</li><ul>
<li>hpcloud in Private Beta (diablo)</li></ul>
<li>trystack (diablo)</li>
<li>AT&amp;T (diablo)</li>
<li>Cern</li>
<li>Mercado Libre</li>
</ul>
</div>
<div class="slide">
<h1>ALSO</h1>
<ul>
<li>Cisco</li>
<li>Citrix</li>
<li>NTT</li>
<li>Piston</li>
<li>Nebula</li>
<li>Nicira</li>
<li>RedHat</li>
<li>Canonical</li>
</ul>
</div>
<div class="slide">
<h1>Release Management</h1>
<ul>
<li>Time Based Releases</li>
<li>Six Month Cadence<ul>
<li>Tied to Ubuntu Releases</li>
</ul><li>Design summits each cycle</li>
<li>Continuously Open Trunk<ul>
<li>Develop directly on master</li>
</ul><li>One Month Milestone Releases</li>
<li>Post-release Stable Branches</li>
<li>Previous stable: diablo</li>
<li>Current stable: essex</li>
<li>Current devel: folsom</li>
<li>Next devel: G[...]</li>
</ul>
</div>
<div class="slide">
<h1>Development Process Overview</h1>
<ul>
<li>Highly Automated</li>
<li>Developer Run</li>
<li>Democratic Structure</li>
<li>Virtual/Online</li>
</ul>
</div>
<div class="slide">
<h1>Vision</h1>
<ul>
<li>Consistent Tooling</li>
<li>Consistent Process</li>
<li>Consistent Product</li>
<li>Multiplier Effect</li>
</ul>
</div>
<div class="slide">
<h1>Consistent Tooling</h1>
<ul>
<li>Minimize meta-development</li>
<li>Process divergence == wasted developer time</li>
<li>Lowers onboarding time</li>
<li>Consolidate tool development</li>
<li>Minimize project-specific weird build crud</li>
</ul>
</div>
<div class="slide">
<h1>Development Infrastructure Systems</h1>
<ul>
<li>Gerrit (code review, git) <a href="http://review.openstack.org">http://review.openstack.org</a></li>
<li>Jenkins (CI, automation) <a href="http://jenkins.openstack.org">http://jenkins.openstack.org</a><ul>
<li>pre-merge testing</li>
<li>post-merge testing</li>
<li>post-merge artifact management</li>
</ul><li>Orchestra (bare metal deployment)</li>
<li>Launchpad (bugs, blueprints, releases, translations, SSO) <a href="http://launchpad.net/openstack">http://launchpad.net/openstack</a></li>
<li>Blog Aggregator <a
href="http://planet.openstack.org">http://planet.openstack.org</a></li>
<li>Pastebin <a
href="http://paste.openstack.org">http://paste.openstack.org</a></li>
</ul>
</div>
<div class="slide">
<h1>Environment</h1>
<ul>
<li>Ubuntu</li>
<li>Python<ul>
<li>pep8 standards</li>
<li>openstack.common</li>
</ul><li>virtualenv/pip/tox</li>
<li>IRC (#openstack-dev, #openstack-meeting)</li>
<li>devstack</li>
<li>gated trunk based on master</li>
</ul>
</div>
<div class="slide">
<h1>Getting Started With Devstack</h1>
<ul>
<li><a href="http://devstack.org">devstack.org</a></li>
<li>Annotated shell script which installs devstack in a VM</li>
<li><em>NOT FOR PRODUCTION USE</em></li>
<li>For developers to test stuff</li>
<li><em>RUN IN A VM OR A CLOUD SERVER<small>not your personal laptop</small></em></li>
</ul>
</div>
<div class="slide">
<h1>Installing a Cloud using Devstack</h1> <ul>
<li>Get an Ubuntu Oneiric Server Image</li>
<small> Don't do this yet </small>
<pre>
git clone git://github.com/openstack-dev/devstack.git
cd devstack; ./stack.sh
</pre>
</ul> </div>
<div class="slide">
<h1>Slower this time</h1> <ul>
<li>Get an Ubuntu Oneiric Server Image</li>
<p><a
href="https://help.ubuntu.com/community/Installation/MinimalCD">https://help.ubuntu.com/community/Installation/MinimalCD</a></p>
<li>Spin up an HP Cloud 2G Node</li>
<li>Note the lack of swap</li>
</ul> </div>
<div class="slide">
<h1>Get devstack</h1> <ul>
<pre>
git clone git://github.com/openstack-dev/devstack.git
cd devstack
</pre>
</ul> </div>
<div class="slide">
<h1>Customizing Devstack</h1> <ul>
<li>localrc</li>
<pre>
FIXED_RANGE=10.1.0.0/24
FIXED_NETWORK_SIZE=256
</pre>
<li>Look in stackrc and stack.sh for other options</li>
</ul></div>
<div class="slide">
<h1>Run devstack</h1> <ul>
<pre>
./stack.sh
</pre>
</ul> </div>
<div class="slide">
<h1>Gated Trunk</h1>
<ul>
<li>Ensures Code Quality</li>
<li>Protects developers<ul>
<li>Devs always start from working code</li>
</ul><li>Protects tree<ul>
<li>Bad code doesn't land</li>
</ul><li>Egalitarian<ul>
<li>Process is the same for everyone</li>
<li>Process is transparent</li>
<li>Process is automated</li>
</ul></ul>
</ul>
</div>
<div class="slide">
<h1>Everything Is Automated</h1>
<img src="images/jenkins-gate.png" />
</div>
<div class="slide">
<h1>Process Flow</h1>
<ul>
<li>Code is written and locally tested in a virtualenv</li>
<li>Code is submitted for code review to gerrit</li>
<li>Code is peer-reviewed</li>
<li>Code is accepted or rejected by core team</li>
<li>Code is run through pre-merge automated checks</li>
<li>Code is merged or rejected</li>
<li>Code is run through post-merge automated checks</li>
</ul>
</div>
<div class="slide">
<h1>Gerrit</h1>
<ul>
<li>Developed by Google for Android</li>
<li>Stand-alone patch review system</li>
<li>Integration points: hooks, JSON queries, event-stream</li>
<li>Extensible review categories, default: Verified, Code-Review</li>
</ul>
</div>
<div class="slide">
<h1>Pre-merge Check</h1>
<img src="images/gerrit-jenkins.png"/>
</div>
<div class="slide">
<h1>Approved Reviews</h1>
<img src="images/gerrit-approved.png"/>
</div>
<div class="slide">
<h1>Types of Jenkins Gerrit Triggers</h1>
<ul>
<li> Patchset uploaded </li>
<li> Change merged </li>
<li> Comment added (review state) </li>
</ul>
</div>
<div class="slide">
<h1>OpenID SSO Integration</h1>
<img src="images/gerrit-sso.png"/>
</div>
<div class="slide">
<h1>Bug Integration - Gerrit</h1>
<img src="images/gerrit-bug.png"/>
</div>
<div class="slide">
<h1>Bug Integration - Launchpad</h1>
<img src="images/lp-bug.png"/>
</div>
<div class="slide">
<h1>Blueprints - Gerrit</h1>
<img src="images/gerrit-bp.png"/>
</div>
<div class="slide">
<h1>Blueprints - Launchpad</h1>
<img src="images/lp-bp.png"/>
</div>
<div class="slide">
<h1>Blueprints - Gerrit Topics</h1>
<img src="images/gerrit-bp-topic.png"/>
</div>
<div class="slide">
<h1>Git Review</h1>
<ul>
<li>External Git subcommand</li>
<li>Developers can easily incorporate code review into git workflow</li>
<li>Zero-configuration</li>
<li>Can be used for any project, being adopted by other projects</li>
</ul>
<pre>
corvus@shiprock:~/rs/github/quantum$ git commit -a
[new-versionpy ddf1dce] Base version.py on glance.
3 files changed, 28 insertions(+), 107 deletions(-)
delete mode 100644 version.py
corvus@shiprock:~/rs/github/quantum$ git review
remote: Resolving deltas: 0% (0/3)
remote:
remote: New Changes:
remote: https://review.openstack.org/3072
remote:
To ssh://corvus@review.openstack.org:29418/openstack/quantum.git
* [new branch] HEAD -> refs/for/master/bug/916018
</pre>
</div>
<div class="slide">
<h1>Types of Tests</h1>
<ul>
<li> Unit tests </li>
<ul><li>Test the code of a single project</li></ul>
<li> Functional tests
<ul>
<li> Can be run on real or virtual servers </li>
<li> Test the service of a single project </li>
<li> Easy for developers to run </li>
</ul>
</li>
<li> Integration tests
<ul>
<li> May be able to run on virtual servers, should run on real servers </li>
<li> Usually harder for a developer to run </li>
</ul>
</li>
</ul>
</div>
<div class="slide">
<h1>Virtual Integration Tests</h1>
<p>Devstack: https://github.com/openstack-dev/devstack</p>
<p>Easy for developers to recreate locally</p>
<p>On each commit:</p>
<ul>
<li>Provision Cloud Server (surprisingly HARD!)</li>
<li>Run devstack to install code</li>
<li>On success - delete server</li>
<li>On failure:</li>
<ul>
<li>Keep Server</li>
<li>Install developer's ssh key</li>
<li>Give the server to the developer for 24H</li>
</ul>
</ul>
</div>
<div class="slide">
<h1>Getting Started With Developing</h1>
</h3><a
href="http://www.linuxjedi.co.uk/2012/03/real-way-to-start-hacking-on-openstack.html">http://www.linuxjedi.co.uk/2012/03/real-way-to-start-hacking-on-openstack.html</a></h3>
<ul>
<li>Get a launchpad account</li>
<li>Upload your SSH key to Launchpad</li>
<li>Sign the CLA: <a
href="http://wiki.openstack.org/HowToContribute#If_you.27re_a_developer.2C_start_here:">wiki/HowToContribute</a></li>
<li>Join the ~openstack-cla team on Launchpad</li>
<li>One time only:<pre>
git config --global user.name "Firstname Lastname"
git config --global user.email "your_email@youremail.com"
</pre></li>
<li>Install git review</li>
</ul>
</div>
<div class="slide">
<h1>Add your blog to OpenStack Planet</h1>
<pre>git clone git://github.com/openstack/openstack-planet.git
cd openstack-planet
# edit planet.ini
# add your image
git add images/yourusername.png
git add planet.ini
git commit
git review
</pre></div>
<div class="slide">
<h1>tox</h1>
<ul>
<li>Manages python virtualenvs</li>
<li>Runs multiple python versions</li>
</ul>
</div>
<div class="slide">
<h1>Doing something more useful</h1>
<pre>git clone git://github.com/openstack/nova.git
# do things, now, to test it:
tox
# it's perfect - submit!
git commit
git review
</pre>
</div>
<div class="slide">
<h1>Getting Started With Devstack</h1>
<ul>
<li><a href="http://devstack.org">devstack.org</a></li>
<li>Annotated shell script which installs devstack in a VM</li>
<li><em>NOT FOR PRODUCTION USE</em></li>
<li>For developers to test stuff</li>
</ul>
</div>
<div class="slide">
<h1>Installing a Cloud using Devstack</h1> <ul>
<li>Get an Ubuntu Oneiric Server Image</li>
<small> Don't do this yet </small>
<pre>
git clone git://github.com/openstack-dev/devstack.git
cd devstack; ./stack.sh
</pre>
</ul> </div>
<div class="slide">
<h1>Slower this time</h1> <ul>
<li>Get an Ubuntu Oneiric Server Image</li>
<p><a
href="https://help.ubuntu.com/community/Installation/MinimalCD">https://help.ubuntu.com/community/Installation/MinimalCD</a></p>
<li>Spin up an HP Cloud 2G Node</li>
<li>Note the lack of swap</li>
</ul>
<pre>
sudo umount /mnt
sudo mkswap /dev/vdb
sudo swapon /dev/vdb
</pre>
</div>
<div class="slide">
<h1>Get devstack</h1> <ul>
<pre>
git clone git://github.com/openstack-dev/devstack.git
cd devstack
# to run essex:
git checkout stable/essex
</pre>
</ul> </div>
<div class="slide">
<h1>Customizing Devstack</h1> <ul>
<li>localrc</li>
<pre>
FIXED_RANGE=10.1.0.0/24
FIXED_NETWORK_SIZE=256
</pre>
<li>Look in stackrc and stack.sh for other options</li>
</ul></div>
<div class="slide">
<h1>Run devstack</h1> <ul>
<pre>
./stack.sh
</pre>
</ul> </div>
<div class="slide">
<h1>Now What?</h1> <ul>
<li>sudo apt-get remove python-keyring</li>
<li>Services running in screen <em>rejoin-stack.sh</em></li>
<li>Run tests <em>exercise.sh</em></li>
<li>Horizon running on port 80 on localhost</li>
<li>devstack puts credentials in localrc, openrc does useful things. Get them with<pre>
source openrc
</pre></li>
</ul></div>
<div class="slide">
<h1>Images in Glance</h1> <ul>
<pre>
glance -f index
</pre>
<p>Save the UUID you want in a var called IMAGE_ID</p>
</ul></div>
<div class="slide">
<h1>Security Groups</h1> <ul>
<li>Control what ports are open</li>
<pre>
SECGROUP=test_secgroup
nova secgroup-create $SECGROUP "$SECGROUP description"
nova secgroup-add-rule $SECGROUP icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule $SECGROUP tcp 22 22 0.0.0.0/0
</pre>
</ul></div>
<div class="slide">
<h1>Keypairs</h1> <ul>
<li>ssh credentials for logging in</li>
<pre>
KEY_NAME=test2_key
KEY_FILE=key.pem
nova keypair-add $KEY_NAME > $KEY_FILE
</pre>
</ul></div>
<div class="slide">
<h1>Flavors</h1> <ul>
<pre>
nova flavor-list
<small>
+----+-----------+-----------+------+-----------+------+-------+-------------+
| ID | Name | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor |
+----+-----------+-----------+------+-----------+------+-------+-------------+
| 1 | m1.tiny | 512 | 0 | 0 | | 1 | 1.0 |
| 2 | m1.small | 2048 | 10 | 20 | | 1 | 1.0 |
| 3 | m1.medium | 4096 | 10 | 40 | | 2 | 1.0 |
| 4 | m1.large | 8192 | 10 | 80 | | 4 | 1.0 |
| 5 | m1.xlarge | 16384 | 10 | 160 | | 8 | 1.0 |
+----+-----------+-----------+------+-----------+------+-------+-------------+
</small>
</pre>
<p>Save the flavor id as FLAVOR_ID</p>
</ul></div>
<div class="slide">
<h1>Boot an Instance</h1> <ul>
<li>Returns the UUID of the server</li>
<li>Using the values saved previously:
<pre>
nova boot --flavor $FLAVOR_ID --image $IMAGE_ID --security_groups=$SECGROUP \
--key_name $KEY_NAME $INSTANCE_NAME
nova list
</pre>
<p>Save the UUID returned as VM_UUID</p>
</li>
</ul></div>
<div class="slide">
<h1>Add an IP</h1> <ul>
<pre>
nova floating-ip-create
nova floating-ip-list
nova add-floating-ip $VM_UUID $FLOATING_IP
</pre>
</ul></div>
<div class="slide">
<h1>Volumes</h1> <ul>
<pre>
DEVICE=/dev/vdb
VOL_NAME=test_volume
nova volume-create --display_name=$VOL_NAME 1
nova volume-list
nova volume-attach $INSTANCE_NAME $VOLUME_ID $DEVICE
</pre>
</ul></div>
<div class="slide">
<h1>Simulated Production</h1>
<p>Great! Now how about a real deployment</p>
</div>
<div class="slide">
<h1>Bare Metal Process</h1>
<p>Each test:</p>
<ul>
<li> PXE boot </li>
<li> Install Ubuntu </li>
<li> Install OpenStack </li>
<li> Run test suite </li>
</ul>
</div>
<div class="slide">
<h1>Test Process Improved</h1>
<p>One time (Orchestra driven):</p>
<ul>
<li> PXE boot </li>
<li> Install Ubuntu </li>
<li> Install OpenStack dependencies </li>
<li> Snapshot LVM volume </li>
</ul>
<p>Each test:</p>
<ul>
<li> Kexec boot into LVM snapshot</li>
<li> Install OpenStack </li>
<li> Run test suite </li>
</ul>
</div>
<div class="slide">
<h1>Cobbler</h1>
<ul>
<li> Written by RedHat </li>
<li> PXE/net-install </li>
<li> Pluggable power management support (impi, drac, ilo ...) </li>
<li> Manage dns/dhcp/tftp - or use others </li>
<li> Templated kickstart or preseed files </li>
</ul>
</div>
<div class="slide">
<h1>Ubuntu Orchestra</h1>
<ul>
<li> cobbler </li>
<li> squid </li>
<li> dnsmasq </li>
<li> nagios </li>
<li> rsyslog </li>
<li> cloud-init </li>
</ul>
</div>
<div class="slide">
<h1>Walkthrough</h1>
<ul>
<li> https://github.com/openstack/openstack-ci-puppet </li>
<li> https://github.com/openstack/openstack-ci-puppet/tree/master/modules/orchestra </li>
</ul>
</div>
<div class="slide">
<h1>Install Orchestra</h1>
<ul>
<li> apt-get install ubuntu-orchestra-server ipmitool </li>
<li> (install will download ubuntu install images and import them) </li>
</ul>
</div>
<div class="slide">
<h1>Configure dnsmasq</h1>
<ul>
<li> Edit dhcp options in /etc/cobbler/dnsmasq.template </li>
<li> Add "dhcp-ignore=tag:!known" </li>
<li> Insert network range </li>
</ul>
</div>
<div class="slide">
<h1>Cloud Init Script</h1>
<ul>
<li> Ubuntu Orchestra pre-seed files </li>
<li> In the preseed/late_command, use snippets </li>
<li> Run a cloud-init script to bootstrap puppet or chef </li>
</ul>
</div>
<div class="slide">
<h1>Add Profile</h1>
<pre>
cobbler profile add \
--name=natty-x86_64-ostest \
--parent=natty-x86_64 \
--kickstart=/var/lib/cobbler/kickstarts/openstack.preseed \
--kopts="priority=critical locale=en_US"
</pre>
</div>
<div class="slide">
<h1>Add machines manually</h1>
<p>For each machine</p>
<pre>
cobbler system add \
--name=baremetal1 \
--hostname=baremetal1 \
--profile=natty-x86_64-ostest \
--mac=00:11:22:33:44:55 \
--power-type=ipmitool \
--power-user=IPMI_USERNAME \
--power-pass=IPMI_PASS \
--power-address=IPMI_IP_ADDR \
--ip-address=SYSTEM_IP_ADDRESS \
--subnet=SYSTEM_SUBNET \
--kopts="netcfg/choose_interface=auto netcfg/dhcp_timeout=60 auto=true
priority=critical"
</pre>
</div>
<div class="slide">
<h1>Adding systems automatically</h1>
<p>cobbler-enlist</p>
<pre>
d-i preseed/early_command string anna-install cobbler-enlist-udeb
d-i cobbler-enlist/cobbler-server-address http://IP/cobbler_api
d-i cobbler-enlist/cobbler-server-username cobbler
d-i cobbler-enlist/cobbler-server-password password
d-i cobbler-enlist/host-name some-host
d-i cobbler-enlist/host-profile
d-i cobbler-enlist/warning-note note
</pre>
</div>
<div class="slide">
<h1>Reinstall a machine</h1>
<pre>
cobbler sync
cobbler system edit --netboot-enabled=Y --name=baremetal1
cobbler system reboot --name=baremetal1
</pre>
</div>
<div class="slide">
<h1>Thanks!</h1>
Slides are available at https://github.com/openstack-ci/publications in the
tutorial folder.
</div>
</body>
</html>