openstack/octavia
Code Issues Proposed changes

Documenting project direction and design

Browse Source 
This commit adds documentation around project direction and conventions.
Additionally, I've added doc8 checks to make sure or .rst and .txt files
in this repository follow good conventions, and fixed a check related to
graphviz external .dot files.

Change-Id: I1b73b3839b86198f3d56587ca3fb2644dc231f00
This commit is contained in:
Stephen Balukoff 2014-07-30 00:59:02 -07:00
parent 3f281cc457
commit 3324e1219a
9 changed files with 320 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
CONSTITUTION.rst Normal file
View File

@ -0,0 +1,53 @@
====================
Octavia Constitution
====================
This document defines the guiding principles that project leadership will be
following in creating, improving and maintaining the Octavia project.
Octavia is an OpenStack project
-------------------------------
This means we try to run things the same way other "canonized" OpenStack
projects operate from a procedural perspective. This is because we hope that
Octavia will eventually become a standard part of any OpenStack deployment.
Octavia is as open as OpenStack
-------------------------------
Octavia tries to follow the same standards for openness that the OpenStack
project also strives to follow: https://wiki.openstack.org/wiki/Open
We are committed to open design, development, and community.
Octavia is "free"
-----------------
We mean that both in the "beer" and in the "speech" sense. That is to say, the
reference implementation for Octavia should be made up only of open source
components that share the same kind of unencumbered licensing that OpenStack
uses.
Note that this does not mean we are against having vendors develop products
which can replace some of the components within Octavia. (For example, the
Octavia VM images might be replaced by a vendor's proprietary VM image.)
Rather, it means that:
* The reference implementation should always be open source and unencumbered.
* We are typically not interested in making design compromises in order to work
with a vendor's proprietary product. If a vendor wants to develop a component
for Octavia, then the vendor should bend to Octavia's needs, not the other
way around.
Octavia is a load balancer for large operators
----------------------------------------------
That's not to say that small operators can't use it. (In fact, we expect it to
work well for small deployements, too.) But what we mean here is that if in
creating, improving or maintaining Octavia we somehow make it unable to meet
the needs of a typical large operator (or that operator's users), then we have
failed.
Octavia follows the best coding and design conventions
------------------------------------------------------
For the most part, Octavia tries to follow the coding standards set forth for
the OpenStack project in general: http://docs.openstack.org/developer/hacking/
More specific additional standards can be found in the HACKING.rst file in the
same directory as this constitution.
Any exceptions should be well justified and documented. (Comments in or near
the breach in coding standards are usually sufficient documentation.)

101
HACKING.rst
View File

@ -18,3 +18,104 @@ For every new feature, unit tests should be created that both test and
bug that had no unit test, a new passing unit test should be added. If a
submitted bug fix does have a unit test, be sure to add a new one that fails
without the patch and passes with the patch.
Everything is python
--------------------
Although OpenStack apparently allows either python or C++ code, at this time
we don't envision needing anything other than python (and standard, supported
open source modules) for anything we intend to do in Octavia.
Idempotency
-----------
With as much as is going on inside Octavia, its likely that certain messages
and commands will be repeatedly processed. It's important that this doesn't
break the functionality of the load balancing service. Therefore, as much as
possible, algorithms and interfaces should be made as idempotent as possible.
Centralize intelligence, de-centralize workload
-----------------------------------------------
This means that tasks which need to be done relatively infrequently but require
either additional knowledge about the state of other components in the Octavia
system, advanced logic behind decisions, or otherwise a high degree of
intelligence should be done by centralized components (ex. controllers) within
the Octavia system. Examples of this might include:
* Generating haproxy configuration files
* Managing the lifecycle of Octavia VMs
* Moving a loadbalancer instance from one Octavia VM to another.
On the other hand, tasks done extremely often, or which entail a significant
load on the system should be pushed as far out to the most horizontally
scalable components as possible. Examples of this might include:
* Serving actual client requests to end-users (ie. running haproxy)
* Monitoring pool members for failure and sending notifications about this
* Processing log files
There will often be a balance that needs to be struck between these two design
considerations for any given task for which an algorithm needs to be designed.
In considering how to strike this balance, always consider the conditions
that will be present in a large operator environment.
Also, as a secondary benefit of centralizing intelligence, minor feature
additions and bugfixes can often be accomplished in a large operator
environment without having to touch every Octavia VM running in said
environment.
All APIs are versioned
----------------------
This includes "internal" APIs between Octavia components. Experience coding in
the Neutron LBaaS project has taught us that in a large project with many
heterogenous parts, throughout the lifecycle of this project, different parts
will evolve at different rates. It is important that these components are
allowed to do so without hindering or being hindered by parallel development
in other components.
It is also likely that in very large deployments, there might be tens- or
hundreds-of-thousands of individual instances of a given component deployed
(most likely, the Octavia VMs). It is unreasonable to expect a large operator
to update all of these components at once. Therefore it is likely that for a
significant amount of time during a roll-out of a new version, both the old
and new versions of a given component must be able to be controlled or
otherwise interfaced with by the new components.
Both of the above considerations can be allowed for if we use versioning of
APIs where components interact with each other.
Scalability and resilience are as important as functionality
------------------------------------------------------------
Octavia is meant to be an *operator scale* load balancer. As such, it's usually
not enough just to get something working: It also needs to be scalable. For
most components, "scalable" implies horizontally scalable.
In any large operational environment, resilience to failures is a necessity.
Practically speaking, this means that all components of the system that make up
Octavia should be monitored in one way or another, and that where possible
automatic recovery from the most common kinds of failures should become a
standard feature. Where automatic recovery is not an option, then some form
of notification about the failure should be implemented.
Avoid premature optimization
----------------------------
Understand that being "high performance" is often not the same thing as being
"scalable." First get the thing to work in an intelligent way. Only worry about
making it fast if speed becomes an issue.
Don't repeat yourself
---------------------
Octavia strives to follow DRY principles. There should be one source of truth,
and repetition of code should be avoided.
Security is not an afterthought
-------------------------------
The load balancer is often both the most visible public interface to a given
user application, but load balancers themselves often have direct access to
sensitive components and data within the application environment. Security bugs
will happen, but in general we should not approve designs which have known
significant security problems, or which could be made more secure by better
design.
Octavia should follow industry standards
----------------------------------------
By "industry standards" we either mean RFCs or well-established best practices.
We are generally not interested in defining new standards if a prior open
standard already exists. We should also avoid doing things which directly
or indirectly contradict established standards.

7
README.rst
View File

@ -1,6 +1,6 @@
========
Octavia
========
=======
Octavia
=======
Operator-grade open source scalable load balancer.
@ -15,3 +15,4 @@ contributors, please see the CONSTITUTION.rst file in this directory, or
specifications in the specs/ subdirectory. Other documentation can be
found in the docs/ directory.
Please also see the ROADMAP.rst file for our project roadmap.

134
ROADMAP.rst Normal file
View File

@ -0,0 +1,134 @@
===============
Octavia Roadmap
===============
Given the complete lack of any real project management tools in the OpenStack
environment, for the time being we'll be planning and tracking project
progression using this ROADMAP.rst file. (It may be more appropriate to keep
this document in the wiki-- we'll see what people think of this.)
This file consists of three sections:
1. A long-term timeline
2. A short-term timeline (derived from the above, reads like a todo list)
3. Outstanding design questions (things yet to be addressed of near-term to
medium-term importance)
==================
Long-term timeline
==================
Major milestone: Full-featured Neutron LBaaS in Juno
----------------------------------------------------
Description: Neutron LBaaS API and other interfaces are full-featured enough
to allow for a user interface that delivers most of the features Octavia will
be implementing. Work commenced on Octavia coding.
OpenStack release target: Juno
Neutron LBaaS progress:
* New object model support
* TLS support
Octavia progress:
* Consensus on:
* Constitution
* Road map
* Component design
* APIs
* Initial code underway (perhaps alpha release?)
Major milestone: Octavia Version 0.5
------------------------------------
Description: First usable release of Octavia. Delivers load balancing services
on multiple Nova VMs. Single, centralized command and control (not scalable).
OpenStack release target: Kilo
Neutron LBaaS progress:
* Flavor support
* L7 switching support
* Updated horizon UI
* Hooks for Heat integration
Octavia progress:
* Octavia delivers all functionality of Neutron LBaaS user API
* Octavia VMs image building scripts
* Octavia operator API
* Horizon UI for operators
* Neutron LBaaS driver interface for Octavia
* Non-voting Neutron third-party CI for Octavia to ensure Neutron code changes
don't break Octavia
* Command-and-control layer handles:
* Octavia VM lifecycle maangement
* Octavia VM monitoring
* Octavia VM command and control
* Neutron LBaaS service deployment
* Resilient topologies for Octavia VMs (ie. HA for the VMs)
* "Experimental" project status
Major milestone: Octavia Version 1.0
------------------------------------
Description: Operator-scale release of Octavia. Delivers load balancing
services on multiple Nova VMs, and has scalable command and control layer.
OpenStack release target: "L" release
Octavia progress:
* Possibly becomes reference implementation for Neutron LBaaS
* Project becomes incubated
* Fully scalable and HA command-and-control layer
* Improvements to Horizon UI for operators
Major milestone: Octavia Version 2.0
------------------------------------
Description: "Web scale" release of Octavia. Delivers all the features of
1.0, plus allows for horizontal scaling of individual load-balanced services.
(ie. n-node active-active topologies).
OpenStack release target: ???
Octavia progress:
* "Two layer" load balancing topology implemented where layers 1-4 handled by
routing infrastructure, and 4-7 handled by Octavia VMs acting in parallel.
* Improvements to Horizon UI for operators
===================
Short-term timeline
===================
Highest priority:
* See Neutron LBaaS work scheduled for Juno through to completion.
* Import google docs describing v0.5, v1.0 and v2.0 Octavia into specs folder
of this repository
* Get reviews and consensus on the same
Medium priority:
* Define and document Octavia VM <=> Controller RESTful APIs
* Define best practices for credential management between Octavia VM and
controllers (suggested: bi-direction server / client certificat verification)
* Collect requirements for Operator API
* Start work on Octavia VM image
* Start work on Octavia VM agent
* Start work on controllers
* Create Neutron LBaaS driver for Octavia
* Get Octavia to work in devstack
* Flesh out the above items with more detailed checklists as work commences on
them
Lower priority:
* Create mock-ups of and start coding Horizon UI for Octavia operators
* Create non-voting CI interface for testing changes relating to Octavia in
gerrit
============================
Outstanding design questions
============================
* We need to start putting together specifications for the Operator API for
Octavia.

5
specs-tests/tests/test_titles.py
View File

@ -95,10 +95,11 @@ class TestTitles(testtools.TestCase):
self.assertEqual(0, len(titles[refs]))
def test_template(self):
files = glob.glob('specs/*.rst') + glob.glob('specs/*/*')
files = set(glob.glob('specs/*.rst') + glob.glob('specs/*/*'))
files = files - set(glob.glob('specs/*/*.dot'))
for filename in files:
self.assertTrue(filename.endswith(".rst"),
"spec's file must uses 'rst' extension.")
"spec's file must use 'rst' extension.")
with open(filename) as f:
data = f.read()
spec = docutils.core.publish_doctree(data)

9
specs/example.dot
View File

@ -1,7 +1,8 @@
// This work is licensed under a Creative Commons Attribution 3.0
// Unported License.
//
// http://creativecommons.org/licenses/by/3.0/legalcode
/* This work is licensed under a Creative Commons Attribution 3.0
* Unported License.
*
* http://creativecommons.org/licenses/by/3.0/legalcode
*/
digraph G {
label="Sample Graph"

25
specs/template.rst
View File

@ -39,7 +39,8 @@ Some notes about using this template:
For more complicated diagrams that need "real" graphics, yet still should
be in the git revision control system, GraphViz .dot files are acceptable.
If you require an image (screenshot) for your BP, attaching that to the BP
and checking it in is also accepted. However, text representations are prefered.
and checking it in is also accepted. However, text representations are
prefered.
* Diagram examples
@ -186,7 +187,8 @@ proposed changes to the data model.
Questions which need to be addressed by this section include:
* What new data objects and/or database schema changes is this going to require?
* What new data objects and/or database schema changes is this going to
require?
* What database migrations will accompany this change.
@ -324,8 +326,9 @@ changes to an existing notification, or removing a notification.
Other end user impact
---------------------
Aside from the API, are there other ways a user will interact with this feature?
Keep in mind that 'user' in this context could mean either tenant or operator.
Aside from the API, are there other ways a user will interact with this
feature? Keep in mind that 'user' in this context could mean either tenant or
operator.
* Does this change have an impact on python-neutronclient? What does the user
interface there look like?
@ -345,12 +348,12 @@ Examples of things to consider here include:
* A small change in a utility function or a commonly used decorator can have a
large impacts on performance.
* Calls which result in a database queries (whether direct or via conductor) can
have a profound impact on performance when called in critical sections of the
code.
* Calls which result in a database queries (whether direct or via conductor)
can have a profound impact on performance when called in critical sections
of the code.
* Will the change include any locking, and if so what considerations are there on
holding the lock?
* Will the change include any locking, and if so what considerations are there
on holding the lock?
Other deployer impact
---------------------
@ -415,8 +418,8 @@ but we're mostly trying to understand the timeline for implementation.
Dependencies
============
* Include specific references to specs and/or blueprints in octavia, or in other
projects, that this one either depends on or is related to.
* Include specific references to specs and/or blueprints in octavia, or in
other projects, that this one either depends on or is related to.
* If this requires functionality of another project that is not currently used
by Octavia document that fact.

1
test-requirements.txt
View File

@ -10,3 +10,4 @@ ordereddict
testrepository>=0.0.18
testtools>=0.9.34
WebTest>=2.0
doc8

5
tox.ini
View File

@ -13,6 +13,8 @@ commands = python setup.py testr --slowest --testr-args='{posargs}'
[testenv:pep8]
commands = flake8
doc8 specs doc/source octavia \
CONSTITUTION.rst HACKING.rst README.rst ROADMAP.rst
[testenv:docs]
commands = python setup.py build_sphinx
@ -21,3 +23,6 @@ commands = python setup.py build_sphinx
ignore = None
exclude = .tox,doc
show-source = true
[doc8]
max-line-length = 79