[svn r3] Moved eventlet into trunk.
This commit is contained in:
38
README
Normal file
38
README
Normal file
@@ -0,0 +1,38 @@
|
||||
= eventlet =
|
||||
|
||||
Eventlet is a networking library written in Python. It achieves high
|
||||
scalability by using non-blocking io while at the same time retaining
|
||||
high programmer usability by using coroutines to make the non-blocking
|
||||
io operations appear blocking at the source code level.
|
||||
|
||||
== requirements ===
|
||||
|
||||
Eventlet runs on Python version 2.3 or greater, with the following dependenceis:
|
||||
* [http://cheeseshop.python.org/pypi/greenlet
|
||||
* (if running python versions < 2.4) collections.py from the 2.4 distribution or later
|
||||
|
||||
== limitations ==
|
||||
|
||||
* Sorely lacking in documentation
|
||||
* Not enough test coverage -- the goal is 100%, but we are not there yet.
|
||||
* Eventlet does not currently run on stackless using tasklets, though it is a goal to do so in the future.
|
||||
* The SSL client does not properly connect to the SSL server, though both client and server interoperate with other SSL implementations (e.g. curl and apache).
|
||||
|
||||
== getting started ==
|
||||
|
||||
% python
|
||||
>>> import eventlet.api
|
||||
>>> help(eventlet.api)
|
||||
|
||||
Also, look at the examples in the examples directory.
|
||||
|
||||
== eventlet history ==
|
||||
|
||||
eventlet began life as Donovan Preston was talking to Bob Ippolito about coroutine-based non-blocking networking frameworks in Python. Most non-blocking frameworks require you to run the "main loop" in order to perform all network operations, but Donovan wondered if a library written using a trampolining style could get away with transparently running the main loop any time i/o was required, stopping the main loop once no more i/o was scheduled. Bob spent a few days during PyCon 2005 writing a proof-of-concept. He named it eventlet, after the coroutine implementation it used, [[greenlet]]. Donovan began using eventlet as a light-weight network library for his spare-time project Pavel, and also began writing some unittests.
|
||||
|
||||
* http://svn.red-bean.com/bob/eventlet/trunk/
|
||||
* http://soundfarmer.com/Pavel/trunk/
|
||||
|
||||
When Donovan started at Linden Lab in May of 2006, he added eventlet as an svn external in the indra/lib/python directory, to be a dependency of the yet-to-be-named [[backbone]] project (at the time, it was named restserv). However, including eventlet as an svn external meant that any time the externally hosted project had hosting issues, Linden developers were not able to perform svn updates. Thus, the eventlet source was imported into the linden source tree at the same location, and became a fork.
|
||||
|
||||
Bob Ippolito has ceased working on eventlet and has stated his desire for Linden to take it's fork forward to the open source world as "the" eventlet.
|
24
eventlet/__init__.py
Normal file
24
eventlet/__init__.py
Normal file
@@ -0,0 +1,24 @@
|
||||
"""\
|
||||
@file __init__.py
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
# This text exists only for the purpose of not making a complete
|
||||
# mockery of the above copyright header.
|
305
eventlet/api.py
Normal file
305
eventlet/api.py
Normal file
@@ -0,0 +1,305 @@
|
||||
"""\
|
||||
@file api.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import socket
|
||||
import string
|
||||
import linecache
|
||||
import inspect
|
||||
import traceback
|
||||
|
||||
try:
|
||||
import greenlet
|
||||
except ImportError:
|
||||
try:
|
||||
import pylibsupport
|
||||
pylibsupport.emulate()
|
||||
greenlet = sys.modules['greenlet']
|
||||
except ImportError:
|
||||
import stacklesssupport
|
||||
stacklesssupport.emulate()
|
||||
greenlet = sys.modules['greenlet']
|
||||
|
||||
from eventlet import greenlib, tls
|
||||
|
||||
__all__ = [
|
||||
'use_hub', 'get_hub', 'sleep', 'spawn', 'kill',
|
||||
'call_after', 'exc_after', 'trampoline', 'tcp_listener', 'tcp_server',
|
||||
]
|
||||
|
||||
|
||||
class TimeoutError(Exception):
|
||||
pass
|
||||
|
||||
_threadlocal = tls.local()
|
||||
|
||||
def tcp_listener(address):
|
||||
"""
|
||||
Listen on the given (ip, port) address with a TCP socket.
|
||||
Returns a socket object which one should call accept() on to
|
||||
accept a connection on the newly bound socket.
|
||||
|
||||
Generally, the returned socket will be passed to tcp_server,
|
||||
which accepts connections forever and spawns greenlets for
|
||||
each incoming connection.
|
||||
"""
|
||||
from eventlet import wrappedfd, util
|
||||
socket = wrappedfd.wrapped_fd(util.tcp_socket())
|
||||
util.socket_bind_and_listen(socket, address)
|
||||
return socket
|
||||
|
||||
def ssl_listener(address, certificate, private_key):
|
||||
"""Listen on the given (ip, port) address with a TCP socket that
|
||||
can do SSL.
|
||||
|
||||
Returns a socket object which one should call accept() on to
|
||||
accept a connection on the newly bound socket.
|
||||
|
||||
Generally, the returned socket will be passed to tcp_server,
|
||||
which accepts connections forever and spawns greenlets for
|
||||
each incoming connection.
|
||||
"""
|
||||
from eventlet import util
|
||||
socket = util.wrap_ssl(util.tcp_socket(), certificate, private_key)
|
||||
util.socket_bind_and_listen(socket, address)
|
||||
socket.is_secure = True
|
||||
return socket
|
||||
|
||||
def connect_tcp(address):
|
||||
"""
|
||||
Create a TCP connection to address (host, port) and return the socket.
|
||||
"""
|
||||
from eventlet import wrappedfd, util
|
||||
desc = wrappedfd.wrapped_fd(util.tcp_socket())
|
||||
desc.connect(address)
|
||||
return desc
|
||||
|
||||
def tcp_server(listensocket, server, *args, **kw):
|
||||
"""
|
||||
Given a socket, accept connections forever, spawning greenlets
|
||||
and executing "server" for each new incoming connection.
|
||||
When listensocket is closed, the tcp_server greenlet will end.
|
||||
|
||||
listensocket:
|
||||
The socket to accept connections from.
|
||||
|
||||
server:
|
||||
The callable to call when a new connection is made.
|
||||
|
||||
*args:
|
||||
The arguments to pass to the call to server.
|
||||
|
||||
**kw:
|
||||
The keyword arguments to pass to the call to server.
|
||||
"""
|
||||
try:
|
||||
try:
|
||||
while True:
|
||||
spawn(server, listensocket.accept(), *args, **kw)
|
||||
except socket.error, e:
|
||||
# Broken pipe means it was shutdown
|
||||
if e[0] != 32:
|
||||
raise
|
||||
finally:
|
||||
listensocket.close()
|
||||
|
||||
def trampoline(fd, read=None, write=None, timeout=None):
|
||||
t = None
|
||||
hub = get_hub()
|
||||
self = greenlet.getcurrent()
|
||||
fileno = getattr(fd, 'fileno', lambda: fd)()
|
||||
def _do_close(fn):
|
||||
hub.remove_descriptor(fn)
|
||||
greenlib.switch(self, exc=socket.error(32, 'Broken pipe'))
|
||||
def _do_timeout(fn):
|
||||
hub.remove_descriptor(fn)
|
||||
greenlib.switch(self, exc=TimeoutError())
|
||||
def cb(_fileno):
|
||||
if t is not None:
|
||||
t.cancel()
|
||||
hub.remove_descriptor(fileno)
|
||||
greenlib.switch(self, fd)
|
||||
if timeout is not None:
|
||||
t = hub.schedule_call(timeout, _do_timeout)
|
||||
hub.add_descriptor(fileno, read and cb, write and cb, _do_close)
|
||||
return hub.switch()
|
||||
|
||||
def _spawn_startup(cb, args, kw, cancel=None):
|
||||
try:
|
||||
greenlib.switch(greenlet.getcurrent().parent)
|
||||
cancel = None
|
||||
finally:
|
||||
if cancel is not None:
|
||||
cancel()
|
||||
return cb(*args, **kw)
|
||||
|
||||
def _spawn(g):
|
||||
g.parent = greenlet.getcurrent()
|
||||
greenlib.switch(g)
|
||||
|
||||
|
||||
def spawn(cb, *args, **kw):
|
||||
# killable
|
||||
t = None
|
||||
g = greenlib.tracked_greenlet()
|
||||
t = get_hub().schedule_call(0, _spawn, g)
|
||||
greenlib.switch(g, (_spawn_startup, cb, args, kw, t.cancel))
|
||||
return g
|
||||
|
||||
kill = greenlib.kill
|
||||
|
||||
def call_after(seconds, cb, *args, **kw):
|
||||
# cancellable
|
||||
def startup():
|
||||
g = greenlib.tracked_greenlet()
|
||||
greenlib.switch(g, (_spawn_startup, cb, args, kw))
|
||||
greenlib.switch(g)
|
||||
return get_hub().schedule_call(seconds, startup)
|
||||
|
||||
|
||||
def exc_after(seconds, exc):
|
||||
return call_after(seconds, switch, getcurrent(), None, exc)
|
||||
|
||||
|
||||
def get_default_hub():
|
||||
try:
|
||||
import eventlet.kqueuehub
|
||||
except ImportError:
|
||||
pass
|
||||
else:
|
||||
return eventlet.kqueuehub
|
||||
import select
|
||||
if hasattr(select, 'poll'):
|
||||
import eventlet.pollhub
|
||||
return eventlet.pollhub
|
||||
else:
|
||||
import eventlet.selecthub
|
||||
return eventlet.selecthub
|
||||
|
||||
def use_hub(mod=None):
|
||||
if mod is None:
|
||||
mod = get_default_hub()
|
||||
if hasattr(_threadlocal, 'hub'):
|
||||
del _threadlocal.hub
|
||||
if hasattr(mod, 'Hub'):
|
||||
_threadlocal.Hub = mod.Hub
|
||||
else:
|
||||
_threadlocal.Hub = mod
|
||||
|
||||
def get_hub():
|
||||
try:
|
||||
hub = _threadlocal.hub
|
||||
except AttributeError:
|
||||
try:
|
||||
_threadlocal.Hub
|
||||
except AttributeError:
|
||||
use_hub()
|
||||
hub = _threadlocal.hub = _threadlocal.Hub()
|
||||
return hub
|
||||
|
||||
|
||||
def sleep(timeout=0):
|
||||
hub = get_hub()
|
||||
hub.schedule_call(timeout, greenlib.switch, greenlet.getcurrent())
|
||||
hub.switch()
|
||||
|
||||
|
||||
switch = greenlib.switch
|
||||
getcurrent = greenlet.getcurrent
|
||||
|
||||
|
||||
class Spew(object):
|
||||
def __init__(self, trace_names=None):
|
||||
self.trace_names = trace_names
|
||||
|
||||
def __call__(self, frame, event, arg):
|
||||
if event == 'line':
|
||||
lineno = frame.f_lineno
|
||||
if '__file__' in frame.f_globals:
|
||||
filename = frame.f_globals['__file__']
|
||||
if (filename.endswith('.pyc') or
|
||||
filename.endswith('.pyo')):
|
||||
filename = filename[:-1]
|
||||
name = frame.f_globals['__name__']
|
||||
line = linecache.getline(filename, lineno)
|
||||
else:
|
||||
name = '[unknown]'
|
||||
try:
|
||||
src = inspect.getsourcelines(frame)
|
||||
line = src[lineno]
|
||||
except IOError:
|
||||
line = 'Unknown code named [%s]. VM instruction #%d' % (
|
||||
frame.f_code.co_name, frame.f_lasti)
|
||||
if self.trace_names is None or name in self.trace_names:
|
||||
print '%s:%s: %s' % (name, lineno, line.rstrip())
|
||||
details = '\t'
|
||||
tokens = line.translate(
|
||||
string.maketrans(' ,.()', '\0' * 5)).split('\0')
|
||||
for tok in tokens:
|
||||
if tok in frame.f_globals:
|
||||
details += '%s=%r ' % (tok, frame.f_globals[tok])
|
||||
if tok in frame.f_locals:
|
||||
details += '%s=%r ' % (tok, frame.f_locals[tok])
|
||||
if details.strip():
|
||||
print details
|
||||
return self
|
||||
|
||||
|
||||
def spew(trace_names=None):
|
||||
sys.settrace(Spew(trace_names))
|
||||
|
||||
|
||||
def unspew():
|
||||
sys.settrace(None)
|
||||
|
||||
|
||||
def named(name):
|
||||
"""Return an object given its name. The name uses a module-like
|
||||
syntax, eg:
|
||||
os.path.join
|
||||
or
|
||||
mulib.mu.Resource
|
||||
"""
|
||||
toimport = name
|
||||
obj = None
|
||||
while toimport:
|
||||
try:
|
||||
obj = __import__(toimport)
|
||||
break
|
||||
except ImportError, err:
|
||||
# print 'Import error on %s: %s' % (toimport, err) # debugging spam
|
||||
toimport = '.'.join(toimport.split('.')[:-1])
|
||||
if obj is None:
|
||||
raise ImportError('%s could not be imported' % (name, ))
|
||||
for seg in name.split('.')[1:]:
|
||||
try:
|
||||
obj = getattr(obj, seg)
|
||||
except AttributeError:
|
||||
dirobj = dir(obj)
|
||||
dirobj.sort()
|
||||
raise AttributeError('attribute %r missing from %r (%r) %r' % (
|
||||
seg, obj, dirobj, name))
|
||||
return obj
|
||||
|
164
eventlet/api_test.py
Normal file
164
eventlet/api_test.py
Normal file
@@ -0,0 +1,164 @@
|
||||
"""\
|
||||
@file api_test.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
from eventlet import tests
|
||||
from eventlet import api, wrappedfd, util
|
||||
import socket
|
||||
|
||||
|
||||
def check_hub():
|
||||
# Clear through the descriptor queue
|
||||
api.sleep(0)
|
||||
api.sleep(0)
|
||||
assert not api.get_hub().descriptors, repr(api.get_hub().descriptors)
|
||||
# Stop the runloop
|
||||
api.get_hub().runloop.abort()
|
||||
api.sleep(0)
|
||||
assert not api.get_hub().runloop.running
|
||||
|
||||
|
||||
class TestApi(tests.TestCase):
|
||||
mode = 'static'
|
||||
def test_tcp_listener(self):
|
||||
socket = api.tcp_listener(('0.0.0.0', 0))
|
||||
assert socket.getsockname()[0] == '0.0.0.0'
|
||||
socket.close()
|
||||
|
||||
check_hub()
|
||||
|
||||
def dont_test_connect_tcp(self):
|
||||
"""This test is broken. Please name it test_connect_tcp and fix
|
||||
the bug (or the test) so it passes.
|
||||
"""
|
||||
def accept_once(listenfd):
|
||||
try:
|
||||
conn, addr = listenfd.accept()
|
||||
conn.write('hello\n')
|
||||
conn.close()
|
||||
finally:
|
||||
listenfd.close()
|
||||
|
||||
server = api.tcp_listener(('0.0.0.0', 0))
|
||||
api.spawn(accept_once, server)
|
||||
|
||||
client = api.connect_tcp(('127.0.0.1', server.getsockname()[1]))
|
||||
assert client.readline() == 'hello\n'
|
||||
|
||||
assert client.read() == ''
|
||||
client.close()
|
||||
|
||||
check_hub()
|
||||
|
||||
def test_server(self):
|
||||
server = api.tcp_listener(('0.0.0.0', 0))
|
||||
bound_port = server.getsockname()[1]
|
||||
connected = []
|
||||
|
||||
def accept_twice((conn, addr)):
|
||||
print 'connected'
|
||||
connected.append(True)
|
||||
conn.close()
|
||||
if len(connected) == 2:
|
||||
server.close()
|
||||
|
||||
api.call_after(0, api.connect_tcp, ('127.0.0.1', bound_port))
|
||||
api.call_after(0, api.connect_tcp, ('127.0.0.1', bound_port))
|
||||
api.tcp_server(server, accept_twice)
|
||||
|
||||
assert len(connected) == 2
|
||||
|
||||
check_hub()
|
||||
|
||||
def dont_test_trampoline_timeout(self):
|
||||
"""This test is broken. Please change it's name to test_trampoline_timeout,
|
||||
and fix the bug (or fix the test)
|
||||
"""
|
||||
server = api.tcp_listener(('0.0.0.0', 0))
|
||||
bound_port = server.getsockname()[1]
|
||||
|
||||
try:
|
||||
desc = wrappedfd.wrapped_fd(util.tcp_socket())
|
||||
api.trampoline(desc, read=True, write=True, timeout=0.1)
|
||||
except api.TimeoutError:
|
||||
pass # test passed
|
||||
else:
|
||||
assert False, "Didn't timeout"
|
||||
|
||||
check_hub()
|
||||
|
||||
def test_timeout_cancel(self):
|
||||
server = api.tcp_listener(('0.0.0.0', 0))
|
||||
bound_port = server.getsockname()[1]
|
||||
|
||||
def client_connected((conn, addr)):
|
||||
conn.close()
|
||||
|
||||
def go():
|
||||
client = util.tcp_socket()
|
||||
|
||||
desc = wrappedfd.wrapped_fd(client)
|
||||
desc.connect(('127.0.0.1', bound_port))
|
||||
try:
|
||||
api.trampoline(desc, read=True, write=True, timeout=0.1)
|
||||
except api.TimeoutError:
|
||||
assert False, "Timed out"
|
||||
|
||||
server.close()
|
||||
client.close()
|
||||
|
||||
api.call_after(0, go)
|
||||
|
||||
api.tcp_server(server, client_connected)
|
||||
|
||||
check_hub()
|
||||
|
||||
def dont_test_explicit_hub(self):
|
||||
"""This test is broken. please change it's name to test_explicit_hub
|
||||
and make it pass (or fix the test)
|
||||
"""
|
||||
api.use_hub(Foo)
|
||||
assert isinstance(api.get_hub(), Foo), api.get_hub()
|
||||
|
||||
api.use_hub(api.get_default_hub())
|
||||
|
||||
check_hub()
|
||||
|
||||
def test_named(self):
|
||||
named_foo = api.named('api_test.Foo')
|
||||
self.assertEquals(
|
||||
named_foo.__name__,
|
||||
"Foo")
|
||||
|
||||
def test_naming_missing_class(self):
|
||||
self.assertRaises(
|
||||
ImportError, api.named, 'this_name_should_hopefully_not_exist.Foo')
|
||||
|
||||
|
||||
class Foo(object):
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
tests.main()
|
||||
|
85
eventlet/backdoor.py
Normal file
85
eventlet/backdoor.py
Normal file
@@ -0,0 +1,85 @@
|
||||
"""\
|
||||
@file backdoor.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from code import InteractiveConsole
|
||||
from eventlet import greenlib
|
||||
|
||||
try:
|
||||
sys.ps1
|
||||
except AttributeError:
|
||||
sys.ps1 = '>>> '
|
||||
try:
|
||||
sys.ps2
|
||||
except AttributeError:
|
||||
sys.ps2 = '... '
|
||||
|
||||
class SocketConsole(greenlib.GreenletContext):
|
||||
def __init__(self, desc):
|
||||
# mangle the socket
|
||||
self.desc = desc
|
||||
readline = desc.readline
|
||||
self.old = {}
|
||||
self.fixups = {
|
||||
'softspace': 0,
|
||||
'isatty': lambda: True,
|
||||
'flush': lambda: None,
|
||||
'readline': lambda *a: readline(*a).replace('\r\n', '\n'),
|
||||
}
|
||||
for key, value in self.fixups.iteritems():
|
||||
if hasattr(desc, key):
|
||||
self.old[key] = getattr(desc, key)
|
||||
setattr(desc, key, value)
|
||||
|
||||
def finalize(self):
|
||||
# restore the state of the socket
|
||||
for key in self.fixups:
|
||||
try:
|
||||
value = self.old[key]
|
||||
except KeyError:
|
||||
delattr(self.desc, key)
|
||||
else:
|
||||
setattr(self.desc, key, value)
|
||||
self.fixups.clear()
|
||||
self.old.clear()
|
||||
self.desc = None
|
||||
|
||||
def swap_in(self):
|
||||
self.saved = sys.stdin, sys.stderr, sys.stdout
|
||||
sys.stdin = sys.stdout = sys.stderr = self.desc
|
||||
|
||||
def swap_out(self):
|
||||
sys.stdin, sys.stderr, sys.stdout = self.saved
|
||||
|
||||
def backdoor((conn, addr), locals=None):
|
||||
host, port = addr
|
||||
print "backdoor to %s:%s" % (host, port)
|
||||
ctx = SocketConsole(conn)
|
||||
ctx.register()
|
||||
try:
|
||||
console = InteractiveConsole(locals)
|
||||
console.interact()
|
||||
finally:
|
||||
ctx.unregister()
|
98
eventlet/channel.py
Normal file
98
eventlet/channel.py
Normal file
@@ -0,0 +1,98 @@
|
||||
"""\
|
||||
@file channel.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import collections
|
||||
|
||||
from eventlet import api, greenlib
|
||||
|
||||
import greenlet
|
||||
|
||||
__all__ = ['channel']
|
||||
|
||||
class channel(object):
|
||||
"""A channel is a control flow primitive for co-routines. It is a
|
||||
"thread-like" queue for controlling flow between two (or more) co-routines.
|
||||
The state model is:
|
||||
|
||||
* If one co-routine calls send(), it is unscheduled until another
|
||||
co-routine calls receive().
|
||||
* If one co-rounte calls receive(), it is unscheduled until another
|
||||
co-routine calls send().
|
||||
* Once a paired send()/receive() have been called, both co-routeines
|
||||
are rescheduled.
|
||||
|
||||
This is similar to: http://stackless.com/wiki/Channels
|
||||
"""
|
||||
balance = 0
|
||||
|
||||
def _tasklet_loop(self):
|
||||
deque = self.deque = collections.deque()
|
||||
hub = api.get_hub()
|
||||
switch = greenlib.switch
|
||||
direction, caller, args = switch()
|
||||
try:
|
||||
while True:
|
||||
if direction == -1:
|
||||
# waiting to receive
|
||||
if self.balance > 0:
|
||||
sender, args = deque.popleft()
|
||||
hub.schedule_call(0, switch, sender)
|
||||
hub.schedule_call(0, switch, caller, *args)
|
||||
else:
|
||||
deque.append(caller)
|
||||
else:
|
||||
# waiting to send
|
||||
if self.balance < 0:
|
||||
receiver = deque.popleft()
|
||||
hub.schedule_call(0, switch, receiver, *args)
|
||||
hub.schedule_call(0, switch, caller)
|
||||
else:
|
||||
deque.append((caller, args))
|
||||
self.balance += direction
|
||||
direction, caller, args = hub.switch()
|
||||
finally:
|
||||
deque.clear()
|
||||
del self.deque
|
||||
self.balance = 0
|
||||
|
||||
def _send_tasklet(self, *args):
|
||||
try:
|
||||
t = self._tasklet
|
||||
except AttributeError:
|
||||
t = self._tasklet = greenlib.tracked_greenlet()
|
||||
greenlib.switch(t, (self._tasklet_loop,))
|
||||
if args:
|
||||
return greenlib.switch(t, (1, greenlet.getcurrent(), args))
|
||||
else:
|
||||
return greenlib.switch(t, (-1, greenlet.getcurrent(), args))
|
||||
|
||||
def receive(self):
|
||||
return self._send_tasklet()
|
||||
|
||||
def send(self, value):
|
||||
return self._send_tasklet(value)
|
||||
|
||||
def send_exception(self, exc):
|
||||
return self._send_tasklet(None, exc)
|
167
eventlet/coros.py
Normal file
167
eventlet/coros.py
Normal file
@@ -0,0 +1,167 @@
|
||||
"""\
|
||||
@file coros.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import time
|
||||
import traceback
|
||||
import greenlet
|
||||
|
||||
|
||||
from eventlet import api
|
||||
from eventlet import channel
|
||||
from eventlet import pools
|
||||
from eventlet import greenlib
|
||||
|
||||
class Cancelled(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
NOT_USED = object()
|
||||
|
||||
|
||||
class event(object):
|
||||
"""An abstraction where an arbitrary number of coroutines
|
||||
can wait for one event from another.
|
||||
"""
|
||||
_result = None
|
||||
def __init__(self):
|
||||
self.reset()
|
||||
|
||||
def reset(self):
|
||||
""" Reset this event so it can be used to send again.
|
||||
Can only be called after send has been called."""
|
||||
assert self._result is not NOT_USED
|
||||
self.epoch = time.time()
|
||||
self._result = NOT_USED
|
||||
self._waiters = {}
|
||||
|
||||
def wait(self):
|
||||
"""wait until another coroutine calls send.
|
||||
Returns the value the other coroutine passed to
|
||||
send. Returns immediately if the event has already
|
||||
occured.
|
||||
"""
|
||||
if self._result is NOT_USED:
|
||||
self._waiters[greenlet.getcurrent()] = True
|
||||
return api.get_hub().switch()
|
||||
if self._exc is not None:
|
||||
raise self._exc
|
||||
return self._result
|
||||
|
||||
def cancel(self, waiter):
|
||||
"""Raise an exception into a coroutine which called
|
||||
wait() an this event instead of returning a value
|
||||
from wait. Sends the eventlet.coros.Cancelled
|
||||
exception
|
||||
|
||||
waiter: The greenlet (greenlet.getcurrent()) of the
|
||||
coroutine to cancel
|
||||
"""
|
||||
if waiter in self._waiters:
|
||||
del self._waiters[waiter]
|
||||
api.get_hub().schedule_call(
|
||||
0, greenlib.switch, waiter, None, Cancelled())
|
||||
|
||||
def send(self, result=None, exc=None):
|
||||
"""Resume all previous and further
|
||||
calls to wait() with result.
|
||||
"""
|
||||
assert self._result is NOT_USED
|
||||
self._result = result
|
||||
self._exc = exc
|
||||
hub = api.get_hub()
|
||||
for waiter in self._waiters:
|
||||
hub.schedule_call(0, greenlib.switch, waiter, self._result)
|
||||
|
||||
class CoroutinePool(pools.Pool):
|
||||
""" Like a thread pool, but with coroutines. """
|
||||
def _main_loop(self, sender):
|
||||
while True:
|
||||
recvd = sender.wait()
|
||||
sender.reset()
|
||||
(evt, func, args, kw) = recvd
|
||||
try:
|
||||
result = func(*args, **kw)
|
||||
if evt is not None:
|
||||
evt.send(result)
|
||||
except greenlet.GreenletExit:
|
||||
pass
|
||||
except Exception, e:
|
||||
traceback.print_exc()
|
||||
if evt is not None:
|
||||
evt.send(exc=e)
|
||||
self.put(sender)
|
||||
|
||||
def create(self):
|
||||
"""Private implementation of eventlet.pools.Pool
|
||||
interface. Creates an event and spawns the
|
||||
_main_loop coroutine, passing the event.
|
||||
The event is used to send a callable into the
|
||||
new coroutine, to be executed.
|
||||
"""
|
||||
sender = event()
|
||||
api.spawn(self._main_loop, sender)
|
||||
return sender
|
||||
|
||||
def execute(self, func, *args, **kw):
|
||||
"""Execute func in one of the coroutines maintained
|
||||
by the pool, when one is free.
|
||||
|
||||
Immediately returns an eventlet.coros.event object which
|
||||
func's result will be sent to when it is available.
|
||||
"""
|
||||
sender = self.get()
|
||||
receiver = event()
|
||||
sender.send((receiver, func, args, kw))
|
||||
return receiver
|
||||
|
||||
def execute_async(self, func, *args, **kw):
|
||||
"""Execute func in one of the coroutines maintained
|
||||
by the pool, when one is free.
|
||||
|
||||
This version does not provide the return value.
|
||||
"""
|
||||
sender = self.get()
|
||||
sender.send((None, func, args, kw))
|
||||
|
||||
|
||||
class pipe(object):
|
||||
""" Implementation of pipe using events. Not tested! Not used, either."""
|
||||
def __init__(self):
|
||||
self._event = event()
|
||||
self._buffer = ''
|
||||
|
||||
def send(self, txt):
|
||||
self._buffer += txt
|
||||
evt, self._event = self._event, event()
|
||||
evt.send()
|
||||
|
||||
def recv(self, num=16384):
|
||||
if not self._buffer:
|
||||
self._event.wait()
|
||||
if num >= len(self._buffer):
|
||||
buf, self._buffer = self._buffer, ''
|
||||
else:
|
||||
buf, self._buffer = self._buffer[:num], self._buffer[num:]
|
||||
return buf
|
||||
|
144
eventlet/coros_test.py
Normal file
144
eventlet/coros_test.py
Normal file
@@ -0,0 +1,144 @@
|
||||
"""\
|
||||
@file coros_test.py
|
||||
@author Donovan Preston, Ryan Williams
|
||||
|
||||
Copyright (c) 2000-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
from eventlet import tests
|
||||
from eventlet import coros, api
|
||||
|
||||
class TestEvent(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
# raise an exception if we're waiting forever
|
||||
self._cancel_timeout = api.exc_after(1, RuntimeError())
|
||||
|
||||
def tearDown(self):
|
||||
self._cancel_timeout.cancel()
|
||||
|
||||
def test_waiting_for_event(self):
|
||||
evt = coros.event()
|
||||
value = 'some stuff'
|
||||
def send_to_event():
|
||||
evt.send(value)
|
||||
api.spawn(send_to_event)
|
||||
self.assertEqual(evt.wait(), value)
|
||||
|
||||
def test_multiple_waiters(self):
|
||||
evt = coros.event()
|
||||
value = 'some stuff'
|
||||
results = []
|
||||
def wait_on_event(i_am_done):
|
||||
evt.wait()
|
||||
results.append(True)
|
||||
i_am_done.send()
|
||||
|
||||
waiters = []
|
||||
count = 5
|
||||
for i in range(count):
|
||||
waiters.append(coros.event())
|
||||
api.spawn(wait_on_event, waiters[-1])
|
||||
evt.send()
|
||||
|
||||
for w in waiters:
|
||||
w.wait()
|
||||
|
||||
self.assertEqual(len(results), count)
|
||||
|
||||
def test_cancel(self):
|
||||
evt = coros.event()
|
||||
# close over the current coro so we can cancel it explicitly
|
||||
current = api.getcurrent()
|
||||
def cancel_event():
|
||||
evt.cancel(current)
|
||||
api.spawn(cancel_event)
|
||||
|
||||
self.assertRaises(coros.Cancelled, evt.wait)
|
||||
|
||||
def test_reset(self):
|
||||
evt = coros.event()
|
||||
|
||||
# calling reset before send should throw
|
||||
self.assertRaises(AssertionError, evt.reset)
|
||||
|
||||
value = 'some stuff'
|
||||
def send_to_event():
|
||||
evt.send(value)
|
||||
api.spawn(send_to_event)
|
||||
self.assertEqual(evt.wait(), value)
|
||||
|
||||
# now try it again, and we should get the same exact value,
|
||||
# and we shouldn't be allowed to resend without resetting
|
||||
value2 = 'second stuff'
|
||||
self.assertRaises(AssertionError, evt.send, value2)
|
||||
self.assertEqual(evt.wait(), value)
|
||||
|
||||
# reset and everything should be happy
|
||||
evt.reset()
|
||||
def send_to_event2():
|
||||
evt.send(value2)
|
||||
api.spawn(send_to_event2)
|
||||
self.assertEqual(evt.wait(), value2)
|
||||
|
||||
class TestCoroutinePool(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
# raise an exception if we're waiting forever
|
||||
self._cancel_timeout = api.exc_after(1, RuntimeError())
|
||||
|
||||
def tearDown(self):
|
||||
self._cancel_timeout.cancel()
|
||||
|
||||
def test_execute_async(self):
|
||||
done = coros.event()
|
||||
def some_work():
|
||||
done.send()
|
||||
pool = coros.CoroutinePool(0, 2)
|
||||
pool.execute_async(some_work)
|
||||
done.wait()
|
||||
|
||||
def test_execute(self):
|
||||
value = 'return value'
|
||||
def some_work():
|
||||
return value
|
||||
pool = coros.CoroutinePool(0, 2)
|
||||
worker = pool.execute(some_work)
|
||||
self.assertEqual(value, worker.wait())
|
||||
|
||||
def test_multiple_coros(self):
|
||||
evt = coros.event()
|
||||
results = []
|
||||
def producer():
|
||||
results.append('prod')
|
||||
evt.send()
|
||||
|
||||
def consumer():
|
||||
results.append('cons1')
|
||||
evt.wait()
|
||||
results.append('cons2')
|
||||
|
||||
pool = coros.CoroutinePool(0, 2)
|
||||
done = pool.execute(consumer)
|
||||
pool.execute_async(producer)
|
||||
done.wait()
|
||||
self.assertEquals(['cons1', 'prod', 'cons2'], results)
|
||||
|
||||
if __name__ == '__main__':
|
||||
tests.main()
|
323
eventlet/greenlib.py
Normal file
323
eventlet/greenlib.py
Normal file
@@ -0,0 +1,323 @@
|
||||
"""\
|
||||
@file greenlib.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
import sys
|
||||
import itertools
|
||||
|
||||
import greenlet
|
||||
|
||||
from eventlet import tls
|
||||
|
||||
__all__ = [
|
||||
'switch', 'kill', 'tracked_greenlets',
|
||||
'greenlet_id', 'greenlet_dict', 'GreenletContext',
|
||||
'tracked_greenlet',
|
||||
]
|
||||
|
||||
try:
|
||||
reversed
|
||||
except NameError:
|
||||
def reversed(something):
|
||||
for x in something[::-1]:
|
||||
yield x
|
||||
|
||||
_threadlocal = tls.local()
|
||||
|
||||
def tracked_greenlet():
|
||||
"""
|
||||
Returns a greenlet that has a greenlet-local dictionary and can be
|
||||
used with GreenletContext and enumerated with tracked_greenlets
|
||||
"""
|
||||
return greenlet.greenlet(greenlet_body)
|
||||
|
||||
class GreenletContextManager(object):
|
||||
"""
|
||||
Per-thread manager for GreenletContext. Created lazily on registration
|
||||
"""
|
||||
def __new__(cls, *args, **kw):
|
||||
dct = greenlet_dict()
|
||||
self = dct.get('greenlet_context', None)
|
||||
if self is not None:
|
||||
return self
|
||||
self = super(GreenletContextManager, cls).__new__(cls, *args, **kw)
|
||||
dct['greenlet_context'] = self
|
||||
self.contexts = []
|
||||
return self
|
||||
|
||||
def add_context(self, ctx):
|
||||
fn = getattr(ctx, '_swap_in', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
self.contexts.append(ctx)
|
||||
|
||||
def remove_context(self, ctx):
|
||||
try:
|
||||
idx = self.contexts.index(ctx)
|
||||
except ValueError:
|
||||
return
|
||||
else:
|
||||
del self.contexts[idx]
|
||||
fn = getattr(ctx, '_swap_out', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
fn = getattr(ctx, '_finalize', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
def swap_in(self):
|
||||
for ctx in self.contexts:
|
||||
fn = getattr(ctx, '_swap_in', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
def swap_out(self):
|
||||
for ctx in reversed(self.contexts):
|
||||
fn = getattr(ctx, '_swap_out', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
def finalize(self):
|
||||
for ctx in reversed(self.contexts):
|
||||
fn = getattr(ctx, '_swap_out', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
fn = getattr(ctx, '_finalize', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
del self.contexts[:]
|
||||
try:
|
||||
del greenlet_dict()['greenlet_context']
|
||||
except KeyError:
|
||||
pass
|
||||
|
||||
class GreenletContext(object):
|
||||
"""
|
||||
A context manager to be triggered when a specific tracked greenlet is
|
||||
swapped in, swapped out, or finalized.
|
||||
|
||||
To use, subclass and override the swap_in, swap_out, and/or finalize
|
||||
methods, for example::
|
||||
|
||||
import greenlib
|
||||
from greenlib import greenlet_id, tracked_greenlet, switch
|
||||
|
||||
class NotifyContext(greenlib.GreenletContext):
|
||||
|
||||
def swap_in(self):
|
||||
print "swap_in"
|
||||
|
||||
def swap_out(self):
|
||||
print "swap_out"
|
||||
|
||||
def finalize(self):
|
||||
print "finalize"
|
||||
|
||||
def another_greenlet():
|
||||
print "another_greenlet"
|
||||
|
||||
def notify_demo():
|
||||
print "starting"
|
||||
NotifyContext().register()
|
||||
switch(tracked_greenlet(), (another_greenlet,))
|
||||
print "finishing"
|
||||
# we could have kept the NotifyContext object
|
||||
# to unregister it here but finalization of all
|
||||
# contexts is implicit when the greenlet returns
|
||||
|
||||
t = tracked_greenlet()
|
||||
switch(t, (notify_demo,))
|
||||
|
||||
The output should be:
|
||||
|
||||
starting
|
||||
swap_in
|
||||
swap_out
|
||||
another_greenlet
|
||||
swap_in
|
||||
finishing
|
||||
swap_out
|
||||
finalize
|
||||
|
||||
"""
|
||||
_balance = 0
|
||||
|
||||
def _swap_in(self):
|
||||
if self._balance != 0:
|
||||
raise RuntimeError("balance != 0: %r" % (self._balance,))
|
||||
self._balance = self._balance + 1
|
||||
fn = getattr(self, 'swap_in', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
def _swap_out(self):
|
||||
if self._balance != 1:
|
||||
raise RuntimeError("balance != 1: %r" % (self._balance,))
|
||||
self._balance = self._balance - 1
|
||||
fn = getattr(self, 'swap_out', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
def register(self):
|
||||
GreenletContextManager().add_context(self)
|
||||
|
||||
def unregister(self):
|
||||
GreenletContextManager().remove_context(self)
|
||||
|
||||
def _finalize(self):
|
||||
fn = getattr(self, 'finalize', None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
|
||||
def kill(g):
|
||||
"""
|
||||
Kill the given greenlet if it is alive by sending it a GreenletExit.
|
||||
|
||||
Note that of any other exception is raised, it will pass-through!
|
||||
"""
|
||||
if not g:
|
||||
return
|
||||
kill_exc = greenlet.GreenletExit()
|
||||
try:
|
||||
try:
|
||||
g.parent = greenlet.getcurrent()
|
||||
except ValueError:
|
||||
pass
|
||||
try:
|
||||
switch(g, exc=kill_exc)
|
||||
except SwitchingToDeadGreenlet:
|
||||
pass
|
||||
except greenlet.GreenletExit, e:
|
||||
if e is not kill_exc:
|
||||
raise
|
||||
|
||||
def tracked_greenlets():
|
||||
"""
|
||||
Return a list of greenlets tracked in this thread. Tracked greenlets
|
||||
use greenlet_body() to ensure that they have greenlet-local storage.
|
||||
"""
|
||||
try:
|
||||
return _threadlocal.greenlets.keys()
|
||||
except AttributeError:
|
||||
return []
|
||||
|
||||
def greenlet_id():
|
||||
"""
|
||||
Get the id of the current tracked greenlet, returns None if the
|
||||
greenlet is not tracked.
|
||||
"""
|
||||
try:
|
||||
d = greenlet_dict()
|
||||
except RuntimeError:
|
||||
return None
|
||||
return d['greenlet_id']
|
||||
|
||||
def greenlet_dict():
|
||||
"""
|
||||
Return the greenlet local storage for this greenlet. Raises RuntimeError
|
||||
if this greenlet is not tracked.
|
||||
"""
|
||||
self = greenlet.getcurrent()
|
||||
try:
|
||||
return _threadlocal.greenlets[self]
|
||||
except (AttributeError, KeyError):
|
||||
raise RuntimeError("greenlet %r is not tracked" % (self,))
|
||||
|
||||
def _greenlet_context(dct=None):
|
||||
if dct is None:
|
||||
try:
|
||||
dct = greenlet_dict()
|
||||
except RuntimeError:
|
||||
return None
|
||||
return dct.get('greenlet_context', None)
|
||||
|
||||
def _greenlet_context_call(name, dct=None):
|
||||
ctx = _greenlet_context(dct)
|
||||
fn = getattr(ctx, name, None)
|
||||
if fn is not None:
|
||||
fn()
|
||||
|
||||
def greenlet_body(value, exc):
|
||||
"""
|
||||
Track the current greenlet during the execution of the given callback,
|
||||
normally you would use tracked_greenlet() to get a greenlet that uses this.
|
||||
|
||||
Greenlets using this body must be greenlib.switch()'ed to
|
||||
"""
|
||||
if exc is not None:
|
||||
if isinstance(exc, tuple):
|
||||
raise exc[0], exc[1], exc[2]
|
||||
raise exc
|
||||
cb, args = value[0], value[1:]
|
||||
try:
|
||||
greenlets = _threadlocal.greenlets
|
||||
except AttributeError:
|
||||
greenlets = _threadlocal.greenlets = {}
|
||||
else:
|
||||
if greenlet.getcurrent() in greenlets:
|
||||
raise RuntimeError("greenlet_body can not be called recursively!")
|
||||
try:
|
||||
greenlet_id = _threadlocal.next_greenlet_id.next()
|
||||
except AttributeError:
|
||||
greenlet_id = 1
|
||||
_threadlocal.next_greenlet_id = itertools.count(2)
|
||||
greenlets[greenlet.getcurrent()] = {'greenlet_id': greenlet_id}
|
||||
try:
|
||||
return cb(*args)
|
||||
finally:
|
||||
_greenlet_context_call('finalize')
|
||||
greenlets.pop(greenlet.getcurrent(), None)
|
||||
|
||||
|
||||
class SwitchingToDeadGreenlet(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def switch(other=None, value=None, exc=None):
|
||||
"""
|
||||
Switch to another greenlet, passing value or exception
|
||||
"""
|
||||
self = greenlet.getcurrent()
|
||||
if other is None:
|
||||
other = self.parent
|
||||
if other is None:
|
||||
other = self
|
||||
if not (other or hasattr(other, 'run')):
|
||||
raise SwitchingToDeadGreenlet("Switching to dead greenlet %r %r %r" % (other, value, exc))
|
||||
_greenlet_context_call('swap_out')
|
||||
try:
|
||||
rval = other.switch(value, exc)
|
||||
if not rval or not other:
|
||||
res, exc = rval, None
|
||||
else:
|
||||
res, exc = rval
|
||||
except:
|
||||
res, exc = None, sys.exc_info()
|
||||
_greenlet_context_call('swap_in')
|
||||
if isinstance(exc, tuple):
|
||||
typ, exc, tb = exc
|
||||
raise typ, exc, tb
|
||||
elif exc is not None:
|
||||
raise exc
|
||||
return res
|
334
eventlet/httpc.py
Executable file
334
eventlet/httpc.py
Executable file
@@ -0,0 +1,334 @@
|
||||
"""\
|
||||
@file httpc.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2005-2006, Donovan Preston
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import httplib
|
||||
import os.path
|
||||
import os
|
||||
import time
|
||||
import urlparse
|
||||
|
||||
|
||||
from mx.DateTime import Parser
|
||||
|
||||
|
||||
_old_HTTPConnection = httplib.HTTPConnection
|
||||
_old_HTTPSConnection = httplib.HTTPSConnection
|
||||
|
||||
|
||||
HTTP_TIME_FORMAT = '%a, %d %b %Y %H:%M:%S GMT'
|
||||
|
||||
|
||||
to_http_time = lambda t: time.strftime(HTTP_TIME_FORMAT, time.gmtime(t))
|
||||
from_http_time = lambda t: int(Parser.DateTimeFromString(t).gmticks())
|
||||
|
||||
def host_and_port_from_url(url):
|
||||
"""@brief Simple function to get host and port from an http url.
|
||||
@return Returns host, port and port may be None.
|
||||
"""
|
||||
host = None
|
||||
port = None
|
||||
#print url
|
||||
parsed_url = urlparse.urlparse(url)
|
||||
try:
|
||||
host, port = parsed_url[1].split(':')
|
||||
except ValueError:
|
||||
host = parsed_url[1].split(':')
|
||||
return host, port
|
||||
|
||||
|
||||
def better_putrequest(self, method, url, skip_host=0):
|
||||
self.method = method
|
||||
self.path = url
|
||||
self.old_putrequest(method, url, skip_host)
|
||||
|
||||
|
||||
class HttpClient(httplib.HTTPConnection):
|
||||
"""A subclass of httplib.HTTPConnection which works around a bug
|
||||
in the interaction between eventlet sockets and httplib. httplib relies
|
||||
on gc to close the socket, causing the socket to be closed too early.
|
||||
|
||||
This is an awful hack and the bug should be fixed properly ASAP.
|
||||
"""
|
||||
def __init__(self, host, port=None, strict=None):
|
||||
_old_HTTPConnection.__init__(self, host, port, strict)
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
old_putrequest = httplib.HTTPConnection.putrequest
|
||||
putrequest = better_putrequest
|
||||
|
||||
|
||||
def wrap_httplib_with_httpc():
|
||||
httplib.HTTP._connection_class = httplib.HTTPConnection = HttpClient
|
||||
httplib.HTTPS._connection_class = httplib.HTTPSConnection = HttpsClient
|
||||
|
||||
|
||||
class HttpsClient(httplib.HTTPSConnection):
|
||||
def close(self):
|
||||
pass
|
||||
old_putrequest = httplib.HTTPSConnection.putrequest
|
||||
putrequest = better_putrequest
|
||||
|
||||
|
||||
class FileScheme(object):
|
||||
"""Retarded scheme to local file wrapper."""
|
||||
host = '<file>'
|
||||
port = '<file>'
|
||||
reason = '<none>'
|
||||
|
||||
def __init__(self, location):
|
||||
pass
|
||||
|
||||
def request(self, method, fullpath, body='', headers=None):
|
||||
self.status = 200
|
||||
self.path = fullpath.split('?')[0]
|
||||
self.method = method = method.lower()
|
||||
assert method in ('get', 'put', 'delete')
|
||||
if method == 'delete':
|
||||
try:
|
||||
os.remove(self.path)
|
||||
except OSError:
|
||||
pass # don't complain if already deleted
|
||||
elif method == 'put':
|
||||
try:
|
||||
f = file(self.path, 'w')
|
||||
f.write(body)
|
||||
f.close()
|
||||
except IOError, e:
|
||||
self.status = 500
|
||||
self.raise_connection_error()
|
||||
elif method == 'get':
|
||||
if not os.path.exists(self.path):
|
||||
self.status = 404
|
||||
self.raise_connection_error(NotFound)
|
||||
|
||||
def connect(self):
|
||||
pass
|
||||
|
||||
def getresponse(self):
|
||||
return self
|
||||
|
||||
def getheader(self, header):
|
||||
if header == 'content-length':
|
||||
try:
|
||||
return os.path.getsize(self.path)
|
||||
except OSError:
|
||||
return 0
|
||||
|
||||
def read(self, howmuch=None):
|
||||
if self.method == 'get':
|
||||
try:
|
||||
fl = file(self.path, 'r')
|
||||
if howmuch is None:
|
||||
return fl.read()
|
||||
else:
|
||||
return fl.read(howmuch)
|
||||
except IOError:
|
||||
self.status = 500
|
||||
self.raise_connection_error()
|
||||
return ''
|
||||
|
||||
def raise_connection_error(self, klass=None):
|
||||
if klass is None:
|
||||
klass=ConnectionError
|
||||
raise klass(
|
||||
self.method, self.host, self.port,
|
||||
self.path, self.status, self.reason, '')
|
||||
|
||||
|
||||
class ConnectionError(Exception):
|
||||
def __init__(self, method, host, port, path, status, reason, body):
|
||||
self.method = method
|
||||
self.host = host
|
||||
self.port = port
|
||||
self.path = path
|
||||
self.status = status
|
||||
self.reason = reason
|
||||
self.body = body
|
||||
Exception.__init__(self)
|
||||
|
||||
def __repr__(self):
|
||||
return "ConnectionError(%r, %r, %r, %r, %r, %r, %r)" % (
|
||||
self.method, self.host, self.port,
|
||||
self.path, self.status, self.reason, self.body)
|
||||
|
||||
__str__ = __repr__
|
||||
|
||||
|
||||
class UnparseableResponse(ConnectionError):
|
||||
def __init__(self, content_type, response):
|
||||
self.content_type = content_type
|
||||
self.response = response
|
||||
Exception.__init__(self)
|
||||
|
||||
def __repr__(self):
|
||||
return "UnparseableResponse(%r, %r)" % (
|
||||
self.content_type, self.response)
|
||||
|
||||
__str__ = __repr__
|
||||
|
||||
|
||||
class Accepted(ConnectionError):
|
||||
pass
|
||||
|
||||
|
||||
class NotFound(ConnectionError):
|
||||
pass
|
||||
|
||||
|
||||
class Forbidden(ConnectionError):
|
||||
pass
|
||||
|
||||
|
||||
class InternalServerError(ConnectionError):
|
||||
pass
|
||||
|
||||
|
||||
class Gone(ConnectionError):
|
||||
pass
|
||||
|
||||
|
||||
status_to_error_map = {
|
||||
500: InternalServerError,
|
||||
410: Gone,
|
||||
404: NotFound,
|
||||
403: Forbidden,
|
||||
202: Accepted,
|
||||
}
|
||||
|
||||
scheme_to_factory_map = {
|
||||
'http': HttpClient,
|
||||
'https': HttpsClient,
|
||||
'file': FileScheme,
|
||||
}
|
||||
|
||||
|
||||
def make_connection(scheme, location, use_proxy):
|
||||
if use_proxy:
|
||||
if "http_proxy" in os.environ:
|
||||
location = os.environ["http_proxy"]
|
||||
elif "ALL_PROXY" in os.environ:
|
||||
location = os.environ["ALL_PROXY"]
|
||||
else:
|
||||
location = "localhost:3128" #default to local squid
|
||||
|
||||
# run a little heuristic to see if it's an url, and if so parse out the hostpart
|
||||
if location.startswith('http'):
|
||||
_scheme, location, path, parameters, query, fragment = urlparse.urlparse(location)
|
||||
|
||||
result = scheme_to_factory_map[scheme](location)
|
||||
result.connect()
|
||||
return result
|
||||
|
||||
|
||||
def connect(url, use_proxy=False):
|
||||
scheme, location, path, params, query, id = urlparse.urlparse(url)
|
||||
return make_connection(scheme, location, use_proxy)
|
||||
|
||||
|
||||
def request(connection, method, url, body='', headers=None, dumper=None, loader=None, use_proxy=False, verbose=False, ok=None):
|
||||
if ok is None:
|
||||
ok = (200, 201, 204)
|
||||
if headers is None:
|
||||
headers = {}
|
||||
if not use_proxy:
|
||||
scheme, location, path, params, query, id = urlparse.urlparse(url)
|
||||
url = path
|
||||
if query:
|
||||
url += "?" + query
|
||||
else:
|
||||
scheme, location, path, params, query, id = urlparse.urlparse(url)
|
||||
headers.update({ "host" : location })
|
||||
if scheme == 'file':
|
||||
use_proxy = False
|
||||
|
||||
|
||||
if dumper is not None:
|
||||
body = dumper(body)
|
||||
headers['content-length'] = len(body)
|
||||
|
||||
connection.request(method, url, body, headers)
|
||||
response = connection.getresponse()
|
||||
if (response.status not in ok):
|
||||
klass = status_to_error_map.get(response.status, ConnectionError)
|
||||
raise klass(
|
||||
connection.method, connection.host, connection.port,
|
||||
connection.path, response.status, response.reason, response.read())
|
||||
|
||||
body = response.read()
|
||||
|
||||
if loader is None:
|
||||
return body
|
||||
|
||||
try:
|
||||
body = loader(body)
|
||||
except Exception, e:
|
||||
raise UnparseableResponse(loader, body)
|
||||
|
||||
if verbose:
|
||||
return response.status, response.msg, body
|
||||
return body
|
||||
|
||||
|
||||
def make_suite(dumper, loader, fallback_content_type):
|
||||
def get(url, headers=None, use_proxy=False, verbose=False, ok=None):
|
||||
#import pdb; pdb.Pdb().set_trace()
|
||||
if headers is None:
|
||||
headers = {}
|
||||
connection = connect(url)
|
||||
return request(connection, 'GET', url, '', headers, None, loader, use_proxy, verbose, ok)
|
||||
|
||||
def put(url, data, headers=None, content_type=None, verbose=False, ok=None):
|
||||
if headers is None:
|
||||
headers = {}
|
||||
if content_type is not None:
|
||||
headers['content-type'] = content_type
|
||||
else:
|
||||
headers['content-type'] = fallback_content_type
|
||||
connection = connect(url)
|
||||
return request(connection, 'PUT', url, data, headers, dumper, loader, verbose=verbose, ok=ok)
|
||||
|
||||
def delete(url, verbose=False, ok=None):
|
||||
return request(connect(url), 'DELETE', url, verbose=verbose, ok=ok)
|
||||
|
||||
def post(url, data='', headers=None, content_type=None, verbose=False, ok=None):
|
||||
connection = connect(url)
|
||||
if headers is None:
|
||||
headers = {}
|
||||
if 'content-type' not in headers:
|
||||
if content_type is not None:
|
||||
headers['content-type'] = content_type
|
||||
else:
|
||||
headers['content-type'] = fallback_content_type
|
||||
return request(connect(url), 'POST', url, data, headers, dumper, loader, verbose=verbose, ok=ok)
|
||||
|
||||
return get, put, delete, post
|
||||
|
||||
|
||||
get, put, delete, post = make_suite(str, None, 'text/plain')
|
||||
|
||||
|
491
eventlet/httpd.py
Normal file
491
eventlet/httpd.py
Normal file
@@ -0,0 +1,491 @@
|
||||
"""\
|
||||
@file httpd.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2005-2006, Donovan Preston
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import cgi
|
||||
import cStringIO
|
||||
import errno
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
import urllib
|
||||
import socket
|
||||
import traceback
|
||||
import cStringIO
|
||||
import BaseHTTPServer
|
||||
|
||||
from eventlet import api
|
||||
from eventlet import coros
|
||||
|
||||
|
||||
USE_ACCESS_LOG = True
|
||||
|
||||
|
||||
CONNECTION_CLOSED = (errno.EPIPE, errno.ECONNRESET)
|
||||
|
||||
|
||||
class Request(object):
|
||||
_method = None
|
||||
_path = None
|
||||
_responsecode = 200
|
||||
_reason_phrase = None
|
||||
_request_started = False
|
||||
_chunked = False
|
||||
_producer_adapters = {}
|
||||
depth = 0
|
||||
def __init__(self, protocol, method, path, headers):
|
||||
self.request_start_time = time.time()
|
||||
self.site = protocol.server.site
|
||||
self.protocol = protocol
|
||||
self._method = method
|
||||
if '?' in path:
|
||||
self._path, self._query = path.split('?', 1)
|
||||
self._query = self._query.replace('&', '&')
|
||||
else:
|
||||
self._path, self._query = path, None
|
||||
self._incoming_headers = headers
|
||||
self._outgoing_headers = dict()
|
||||
|
||||
def response(self, code, reason_phrase=None, headers=None, body=None):
|
||||
"""Change the response code. This will not be sent until some
|
||||
data is written; last call to this method wins. Default is
|
||||
200 if this is not called.
|
||||
"""
|
||||
self._responsecode = code
|
||||
self._reason_phrase = reason_phrase
|
||||
self.protocol.set_response_code(self, code, reason_phrase)
|
||||
if headers is not None:
|
||||
for key, value in headers:
|
||||
self.set_header(key, value)
|
||||
if body is not None:
|
||||
self.write(body)
|
||||
|
||||
def is_okay(self):
|
||||
return 200 <= self._responsecode <= 299
|
||||
|
||||
def full_url(self):
|
||||
path = self.path()
|
||||
query = self.query()
|
||||
if query:
|
||||
path = path + '?' + query
|
||||
|
||||
via = self.get_header('via', '')
|
||||
if via.strip():
|
||||
next_part = iter(via.split()).next
|
||||
|
||||
received_protocol = next_part()
|
||||
received_by = next_part()
|
||||
if received_by.endswith(','):
|
||||
received_by = received_by[:-1]
|
||||
else:
|
||||
comment = ''
|
||||
while not comment.endswith(','):
|
||||
try:
|
||||
comment += next_part()
|
||||
except StopIteration:
|
||||
comment += ','
|
||||
break
|
||||
comment = comment[:-1]
|
||||
else:
|
||||
received_by = self.get_header('host')
|
||||
|
||||
return '%s://%s%s' % (self.request_protocol(), received_by, path)
|
||||
|
||||
def begin_response(self, length="-"):
|
||||
"""Begin the response, and return the initial response text
|
||||
"""
|
||||
self._request_started = True
|
||||
request_time = time.time() - self.request_start_time
|
||||
|
||||
code = self._responsecode
|
||||
proto = self.protocol
|
||||
|
||||
if USE_ACCESS_LOG:
|
||||
proto.server.write_access_log_line(
|
||||
proto.client_address[0],
|
||||
time.strftime("%d/%b/%Y %H:%M:%S"),
|
||||
proto.requestline,
|
||||
code,
|
||||
length,
|
||||
request_time)
|
||||
|
||||
if self._reason_phrase is not None:
|
||||
message = self._reason_phrase.split("\n")[0]
|
||||
elif code in proto.responses:
|
||||
message = proto.responses[code][0]
|
||||
else:
|
||||
message = ''
|
||||
if proto.request_version == 'HTTP/0.9':
|
||||
return []
|
||||
|
||||
response_lines = proto.generate_status_line()
|
||||
|
||||
if not self._outgoing_headers.has_key('connection'):
|
||||
con = self.get_header('connection')
|
||||
if con is None and proto.request_version == 'HTTP/1.0':
|
||||
con = 'close'
|
||||
if con is not None:
|
||||
self.set_header('connection', con)
|
||||
|
||||
for key, value in self._outgoing_headers.items():
|
||||
key = '-'.join([x.capitalize() for x in key.split('-')])
|
||||
response_lines.append("%s: %s" % (key, value))
|
||||
|
||||
response_lines.append("")
|
||||
return response_lines
|
||||
|
||||
def write(self, obj):
|
||||
"""Writes an arbitrary object to the response, using
|
||||
the sitemap's adapt method to convert it to bytes.
|
||||
"""
|
||||
if isinstance(obj, str):
|
||||
self._write_bytes(obj)
|
||||
elif isinstance(obj, unicode):
|
||||
# use utf8 encoding for now, *TODO support charset negotiation
|
||||
# Content-Type: text/html; charset=utf-8
|
||||
ctype = self._outgoing_headers.get('content-type', 'text/html')
|
||||
ctype = ctype + '; charset=utf-8'
|
||||
self._outgoing_headers['content-type'] = ctype
|
||||
self._write_bytes(obj.encode('utf8'))
|
||||
else:
|
||||
self.site.adapt(obj, self)
|
||||
|
||||
def _write_bytes(self, data):
|
||||
"""Write all the data of the response.
|
||||
Can be called just once.
|
||||
"""
|
||||
if self._request_started:
|
||||
print "Request has already written a response:"
|
||||
traceback.print_stack()
|
||||
return
|
||||
|
||||
self._outgoing_headers['content-length'] = len(data)
|
||||
|
||||
response_lines = self.begin_response(len(data))
|
||||
response_lines.append(data)
|
||||
self.protocol.wfile.write("\r\n".join(response_lines))
|
||||
if hasattr(self.protocol.wfile, 'flush'):
|
||||
self.protocol.wfile.flush()
|
||||
|
||||
def method(self):
|
||||
return self._method
|
||||
|
||||
def path(self):
|
||||
return self._path
|
||||
|
||||
def path_segments(self):
|
||||
return [urllib.unquote_plus(x) for x in self._path.split('/')[1:]]
|
||||
|
||||
def query(self):
|
||||
return self._query
|
||||
|
||||
def uri(self):
|
||||
if self._query:
|
||||
return '%s?%s' % (
|
||||
self._path, self._query)
|
||||
return self._path
|
||||
|
||||
def get_headers(self):
|
||||
return self._incoming_headers
|
||||
|
||||
def get_header(self, header_name, default=None):
|
||||
return self.get_headers().get(header_name.lower(), default)
|
||||
|
||||
def get_query_pairs(self):
|
||||
if not hasattr(self, '_split_query'):
|
||||
if self._query is None:
|
||||
self._split_query = ()
|
||||
else:
|
||||
spl = self._query.split('&')
|
||||
spl = [x.split('=', 1) for x in spl if x]
|
||||
self._split_query = []
|
||||
for query in spl:
|
||||
if len(query) == 1:
|
||||
key = query[0]
|
||||
value = ''
|
||||
else:
|
||||
key, value = query
|
||||
self._split_query.append((urllib.unquote_plus(key), urllib.unquote_plus(value)))
|
||||
|
||||
return self._split_query
|
||||
|
||||
def get_queries_generator(self, name):
|
||||
"""Generate all query parameters matching the given name.
|
||||
"""
|
||||
for key, value in self.get_query_pairs():
|
||||
if key == name or not name:
|
||||
yield value
|
||||
|
||||
get_queries = lambda self, name: list(self.get_queries_generator)
|
||||
|
||||
def get_query(self, name, default=None):
|
||||
try:
|
||||
return self.get_queries_generator(name).next()
|
||||
except StopIteration:
|
||||
return default
|
||||
|
||||
def get_arg_list(self, name):
|
||||
return self.get_field_storage().getlist(name)
|
||||
|
||||
def get_arg(self, name, default=None):
|
||||
return self.get_field_storage().getfirst(name, default)
|
||||
|
||||
def get_field_storage(self):
|
||||
if not hasattr(self, '_field_storage'):
|
||||
if self.method() == 'GET':
|
||||
data = ''
|
||||
if self._query:
|
||||
data = self._query
|
||||
fl = cStringIO.StringIO(data)
|
||||
else:
|
||||
fl = self.protocol.rfile
|
||||
## Allow our resource to provide the FieldStorage instance for
|
||||
## customization purposes.
|
||||
headers = self.get_headers()
|
||||
environ = dict(
|
||||
REQUEST_METHOD='POST',
|
||||
QUERY_STRING=self._query or '')
|
||||
if (hasattr(self, 'resource') and
|
||||
hasattr(self.resource, 'getFieldStorage')):
|
||||
self._field_storage = self.resource.getFieldStorage(
|
||||
self, fl, headers, environ)
|
||||
else:
|
||||
self._field_storage = cgi.FieldStorage(
|
||||
fl, headers, environ=environ)
|
||||
|
||||
return self._field_storage
|
||||
|
||||
def set_header(self, key, value):
|
||||
if key.lower() == 'connection' and value.lower() == 'close':
|
||||
self.protocol.close_connection = 1
|
||||
self._outgoing_headers[key.lower()] = value
|
||||
__setitem__ = set_header
|
||||
|
||||
def get_outgoing_header(self, key):
|
||||
return self._outgoing_headers[key.lower()]
|
||||
|
||||
def has_outgoing_header(self, key):
|
||||
return self._outgoing_headers.has_key(key.lower())
|
||||
|
||||
def socket(self):
|
||||
return self.protocol.socket
|
||||
|
||||
def error(self, response=None, body=None, log_traceback=True):
|
||||
if log_traceback:
|
||||
traceback.print_exc()
|
||||
if response is None:
|
||||
response = 500
|
||||
if body is None:
|
||||
typ, val, tb = sys.exc_info()
|
||||
body = dict(type=str(typ), error=True, reason=str(val))
|
||||
self.response(response)
|
||||
if type(body) is str:
|
||||
self.write(body)
|
||||
return
|
||||
try:
|
||||
produce(body, self)
|
||||
except Exception, e:
|
||||
traceback.print_exc()
|
||||
if not self.response_written():
|
||||
self.write('Internal Server Error')
|
||||
|
||||
def not_found(self):
|
||||
self.error(404, 'Not Found\n', log_traceback=False)
|
||||
|
||||
def read_body(self):
|
||||
if not hasattr(self, '_cached_parsed_body'):
|
||||
if not hasattr(self, '_cached_body'):
|
||||
length = self.get_header('content-length')
|
||||
if length:
|
||||
length = int(length)
|
||||
if length:
|
||||
self._cached_body = self.protocol.rfile.read(length)
|
||||
else:
|
||||
self._cached_body = ''
|
||||
body = self._cached_body
|
||||
if hasattr(self.site, 'parsers'):
|
||||
parser = self.site.parsers.get(
|
||||
self.get_header('content-type'))
|
||||
if parser is not None:
|
||||
body = parser(body)
|
||||
self._cached_parsed_body = body
|
||||
return self._cached_parsed_body
|
||||
|
||||
def response_written(self):
|
||||
## TODO change badly named variable
|
||||
return self._request_started
|
||||
|
||||
def request_version(self):
|
||||
return self.protocol.request_version
|
||||
|
||||
def request_protocol(self):
|
||||
if self.protocol.socket.is_secure:
|
||||
return "https"
|
||||
return "http"
|
||||
|
||||
def server_address(self):
|
||||
return self.protocol.server.address
|
||||
|
||||
def __repr__(self):
|
||||
return "<Request %s %s>" % (
|
||||
getattr(self, '_method'), getattr(self, '_path'))
|
||||
|
||||
|
||||
DEFAULT_TIMEOUT = 300
|
||||
|
||||
|
||||
class Timeout(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class HttpProtocol(BaseHTTPServer.BaseHTTPRequestHandler):
|
||||
protocol_version = "HTTP/1.1"
|
||||
def __init__(self, request, client_address, server):
|
||||
self.socket = self.request = self.rfile = self.wfile = request
|
||||
self.client_address = client_address
|
||||
self.server = server
|
||||
self._code = 200
|
||||
self._message = 'OK'
|
||||
|
||||
def set_response_code(self, request, code, message):
|
||||
self._code = code
|
||||
if message is not None:
|
||||
self._message = message.split("\n")[0]
|
||||
elif code in self.responses:
|
||||
self._message = self.responses[code][0]
|
||||
else:
|
||||
self._message = ''
|
||||
|
||||
def generate_status_line(self):
|
||||
return [
|
||||
"%s %d %s" % (
|
||||
self.protocol_version, self._code, self._message)]
|
||||
|
||||
def handle(self):
|
||||
self.close_connection = 0
|
||||
|
||||
timeout = DEFAULT_TIMEOUT
|
||||
while not self.close_connection:
|
||||
if timeout == 0:
|
||||
break
|
||||
cancel = api.exc_after(timeout, Timeout)
|
||||
try:
|
||||
self.raw_requestline = self.rfile.readline()
|
||||
except socket.error, e:
|
||||
if e[0] in CONNECTION_CLOSED:
|
||||
self.close_connection = True
|
||||
cancel.cancel()
|
||||
continue
|
||||
except Timeout:
|
||||
self.close_connection = True
|
||||
continue
|
||||
cancel.cancel()
|
||||
|
||||
if not self.raw_requestline or not self.parse_request():
|
||||
self.close_connection = True
|
||||
continue
|
||||
|
||||
request = Request(self, self.command, self.path, self.headers)
|
||||
request.set_header('Server', self.version_string())
|
||||
request.set_header('Date', self.date_time_string())
|
||||
try:
|
||||
timeout = int(request.get_header('keep-alive'))
|
||||
except (TypeError, ValueError), e:
|
||||
pass
|
||||
|
||||
try:
|
||||
self.server.site.handle_request(request)
|
||||
# throw an exception if it failed to write a body
|
||||
if not request.response_written():
|
||||
raise NotImplementedError("Handler failed to write response to request: %s" % request)
|
||||
|
||||
if not hasattr(self, '_cached_body'):
|
||||
request.read_body() ## read & discard body
|
||||
continue
|
||||
except socket.error, e:
|
||||
# Broken pipe, connection reset by peer
|
||||
if e[0] in CONNECTION_CLOSED:
|
||||
#print "Remote host closed connection before response could be sent"
|
||||
pass
|
||||
else:
|
||||
raise
|
||||
except Exception, e:
|
||||
if not request.response_written():
|
||||
request.response(500)
|
||||
request.write('Internal Server Error')
|
||||
self.socket.close()
|
||||
raise
|
||||
self.socket.close()
|
||||
|
||||
|
||||
class Server(BaseHTTPServer.HTTPServer):
|
||||
def __init__(self, socket, address, site, log):
|
||||
self.socket = socket
|
||||
self.address = address
|
||||
self.site = site
|
||||
if log:
|
||||
self.log = log
|
||||
if hasattr(log, 'info'):
|
||||
log.write = log.info
|
||||
else:
|
||||
self.log = self
|
||||
|
||||
def write(self, something):
|
||||
sys.stdout.write('%s\n' % (something, ))
|
||||
|
||||
def log_message(self, message):
|
||||
self.log.write(message)
|
||||
|
||||
def log_exception(self, type, value, tb):
|
||||
print ''.join(traceback.format_exception(type, value, tb))
|
||||
|
||||
def write_access_log_line(self, *args):
|
||||
"""Write a line to the access.log. Arguments:
|
||||
client_address, date_time, requestline, code, size, request_time
|
||||
"""
|
||||
self.log.write(
|
||||
'%s - - [%s] "%s" %s %s %.6f' % args)
|
||||
|
||||
|
||||
def server(sock, site, log=None, max_size=512):
|
||||
pool = coros.CoroutinePool(max_size=max_size)
|
||||
serv = Server(sock, sock.getsockname(), site, log)
|
||||
try:
|
||||
print "httpd starting up on", sock.getsockname()
|
||||
while True:
|
||||
try:
|
||||
new_sock, address = sock.accept()
|
||||
proto = HttpProtocol(new_sock, address, serv)
|
||||
pool.execute_async(proto.handle)
|
||||
except KeyboardInterrupt:
|
||||
api.get_hub().remove_descriptor(sock.fileno())
|
||||
print "httpd exiting"
|
||||
break
|
||||
finally:
|
||||
try:
|
||||
sock.close()
|
||||
except socket.error:
|
||||
pass
|
||||
|
||||
|
139
eventlet/httpd_test.py
Normal file
139
eventlet/httpd_test.py
Normal file
@@ -0,0 +1,139 @@
|
||||
"""\
|
||||
@file httpd_test.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
|
||||
from eventlet import api
|
||||
from eventlet import httpd
|
||||
from eventlet import processes
|
||||
from eventlet import util
|
||||
|
||||
|
||||
util.wrap_socket_with_coroutine_socket()
|
||||
|
||||
|
||||
from eventlet import tests
|
||||
|
||||
|
||||
class Site(object):
|
||||
def handle_request(self, req):
|
||||
req.write('hello world')
|
||||
|
||||
def adapt(self, obj, req):
|
||||
req.write(str(obj))
|
||||
|
||||
|
||||
CONTENT_LENGTH = 'content-length'
|
||||
|
||||
|
||||
"""
|
||||
HTTP/1.1 200 OK
|
||||
Date: foo
|
||||
Content-length: 11
|
||||
|
||||
hello world
|
||||
"""
|
||||
|
||||
class ConnectionClosed(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def read_http(sock):
|
||||
response_line = sock.readline()
|
||||
if not response_line:
|
||||
raise ConnectionClosed
|
||||
raw_headers = sock.readuntil('\r\n\r\n').strip()
|
||||
#print "R", response_line, raw_headers
|
||||
headers = dict()
|
||||
for x in raw_headers.split('\r\n'):
|
||||
#print "X", x
|
||||
key, value = x.split(': ', 1)
|
||||
headers[key.lower()] = value
|
||||
|
||||
if CONTENT_LENGTH in headers:
|
||||
num = int(headers[CONTENT_LENGTH])
|
||||
body = sock.read(num)
|
||||
#print body
|
||||
|
||||
|
||||
class TestHttpd(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.killer = api.spawn(
|
||||
httpd.server, api.tcp_listener(('0.0.0.0', 12346)), Site(), max_size=128)
|
||||
|
||||
def tearDown(self):
|
||||
api.kill(self.killer)
|
||||
|
||||
def test_001_server(self):
|
||||
sock = api.connect_tcp(
|
||||
('127.0.0.1', 12346))
|
||||
|
||||
sock.write('GET / HTTP/1.0\r\nHost: localhost\r\n\r\n')
|
||||
result = sock.read()
|
||||
sock.close()
|
||||
## The server responds with the maximum version it supports
|
||||
self.assert_(result.startswith('HTTP'), result)
|
||||
self.assert_(result.endswith('hello world'))
|
||||
|
||||
def test_002_keepalive(self):
|
||||
sock = api.connect_tcp(
|
||||
('127.0.0.1', 12346))
|
||||
|
||||
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
|
||||
read_http(sock)
|
||||
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
|
||||
read_http(sock)
|
||||
sock.close()
|
||||
|
||||
def test_003_passing_non_int_to_read(self):
|
||||
# This should go in test_wrappedfd
|
||||
sock = api.connect_tcp(
|
||||
('127.0.0.1', 12346))
|
||||
|
||||
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
|
||||
cancel = api.exc_after(1, RuntimeError)
|
||||
self.assertRaises(TypeError, sock.read, "This shouldn't work")
|
||||
cancel.cancel()
|
||||
sock.close()
|
||||
|
||||
def test_004_close_keepalive(self):
|
||||
sock = api.connect_tcp(
|
||||
('127.0.0.1', 12346))
|
||||
|
||||
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
|
||||
read_http(sock)
|
||||
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n')
|
||||
read_http(sock)
|
||||
sock.write('GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
|
||||
self.assertRaises(ConnectionClosed, read_http, sock)
|
||||
sock.close()
|
||||
|
||||
def test_005_run_apachebench(self):
|
||||
url = 'http://localhost:12346/'
|
||||
out = processes.Process('/usr/sbin/ab', ['-c','64','-n','1024', '-k', url])
|
||||
print out.read()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
tests.main()
|
39
eventlet/httpdate.py
Normal file
39
eventlet/httpdate.py
Normal file
@@ -0,0 +1,39 @@
|
||||
"""\
|
||||
@file httpdate.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
__all__ = ['format_date_time']
|
||||
|
||||
# Weekday and month names for HTTP date/time formatting; always English!
|
||||
_weekdayname = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
|
||||
_monthname = [None, # Dummy so we can use 1-based month numbers
|
||||
"Jan", "Feb", "Mar", "Apr", "May", "Jun",
|
||||
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
|
||||
|
||||
def format_date_time(timestamp):
|
||||
year, month, day, hh, mm, ss, wd, y, z = time.gmtime(timestamp)
|
||||
return "%s, %02d %3s %4d %02d:%02d:%02d GMT" % (
|
||||
_weekdayname[wd], day, _monthname[month], year, hh, mm, ss
|
||||
)
|
31
eventlet/jsonhttp.py
Normal file
31
eventlet/jsonhttp.py
Normal file
@@ -0,0 +1,31 @@
|
||||
"""\
|
||||
@file jsonhttp.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
from eventlet import httpc
|
||||
|
||||
import simplejson
|
||||
|
||||
|
||||
get, put, delete, post = httpc.make_suite(
|
||||
simplejson.dumps, simplejson.loads, 'application/json')
|
219
eventlet/kqueuehub.py
Normal file
219
eventlet/kqueuehub.py
Normal file
@@ -0,0 +1,219 @@
|
||||
"""\
|
||||
@file kqueuehub.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import select
|
||||
import kqueue
|
||||
import traceback
|
||||
from errno import EBADF
|
||||
|
||||
from eventlet import greenlib
|
||||
from eventlet.runloop import RunLoop, Timer
|
||||
|
||||
import greenlet
|
||||
|
||||
class Hub(object):
|
||||
def __init__(self):
|
||||
self.runloop = RunLoop(self.wait)
|
||||
self.descriptor_queue = {}
|
||||
self.descriptors = {}
|
||||
self.greenlet = None
|
||||
self.kfd = None
|
||||
|
||||
def stop(self):
|
||||
self.process_queue()
|
||||
self.descriptors, self.descriptor_queue = self.descriptor_queue, {}
|
||||
os.close(self.kfd)
|
||||
self.kfd = None
|
||||
self.runloop.abort()
|
||||
if self.greenlet is not greenlet.getcurrent():
|
||||
self.switch()
|
||||
|
||||
def schedule_call(self, *args, **kw):
|
||||
return self.runloop.schedule_call(*args, **kw)
|
||||
|
||||
def switch(self):
|
||||
if not self.greenlet:
|
||||
self.greenlet = greenlib.tracked_greenlet()
|
||||
args = ((self.runloop.run,),)
|
||||
else:
|
||||
args = ()
|
||||
try:
|
||||
greenlet.getcurrent().parent = self.greenlet
|
||||
except ValueError:
|
||||
pass
|
||||
return greenlib.switch(self.greenlet, *args)
|
||||
|
||||
def add_descriptor(self, fileno, read=None, write=None, exc=None):
|
||||
self.descriptor_queue[fileno] = read, write, exc
|
||||
|
||||
def remove_descriptor(self, fileno):
|
||||
self.descriptor_queue[fileno] = None, None, None
|
||||
|
||||
def exc_descriptor(self, fileno):
|
||||
# We must handle two cases here, the descriptor
|
||||
# may be changing or removing its exc handler
|
||||
# in the queue, or it may be waiting on the queue.
|
||||
exc = None
|
||||
try:
|
||||
exc = self.descriptor_queue[fileno][2]
|
||||
except KeyError:
|
||||
try:
|
||||
exc = self.descriptors[fileno][2]
|
||||
except KeyError:
|
||||
pass
|
||||
if exc is not None:
|
||||
try:
|
||||
exc()
|
||||
except self.runloop.SYSTEM_EXCEPTIONS:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
|
||||
def squelch_exception(self, fileno, exc_info):
|
||||
traceback.print_exception(*exc_info)
|
||||
print >>sys.stderr, "Removing descriptor: %r" % (fileno,)
|
||||
try:
|
||||
self.remove_descriptor(fileno)
|
||||
except Exception, e:
|
||||
print >>sys.stderr, "Exception while removing descriptor! %r" % (e,)
|
||||
|
||||
def process_queue(self):
|
||||
if self.kfd is None:
|
||||
self.kfd = kqueue.kqueue()
|
||||
d = self.descriptors
|
||||
|
||||
E_R = kqueue.EVFILT_READ
|
||||
E_W = kqueue.EVFILT_WRITE
|
||||
E = kqueue.Event
|
||||
E_ADD = kqueue.EV_ADD
|
||||
E_DEL = kqueue.EV_DELETE
|
||||
|
||||
kevent = kqueue.kevent
|
||||
kfd = self.kfd
|
||||
for fileno, rwe in self.descriptor_queue.iteritems():
|
||||
read, write, exc = rwe
|
||||
if read is None and write is None and exc is None:
|
||||
try:
|
||||
read, write, exc = d.pop(fileno)
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
l = []
|
||||
if read is not None:
|
||||
l.append(E(fileno, E_R, E_DEL))
|
||||
if write is not None:
|
||||
l.append(E(fileno, E_W, E_DEL))
|
||||
if l:
|
||||
try:
|
||||
kevent(kfd, l, 0, 0)
|
||||
except OSError, e:
|
||||
if e[0] != EBADF:
|
||||
raise
|
||||
else:
|
||||
l = []
|
||||
try:
|
||||
oldr, oldw, olde = d[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
if oldr is not None:
|
||||
if read is None:
|
||||
l.append(E(fileno, E_R, E_DEL))
|
||||
else:
|
||||
read = None
|
||||
if oldw is not None:
|
||||
if write is None:
|
||||
l.append(E(fileno, E_W, E_DEL))
|
||||
else:
|
||||
write = None
|
||||
if read is not None:
|
||||
l.append(E(fileno, E_R, E_ADD))
|
||||
if write is not None:
|
||||
l.append(E(fileno, E_W, E_ADD))
|
||||
if l:
|
||||
try:
|
||||
kevent(kfd, l, 0, 0)
|
||||
except OSError, e:
|
||||
if e[0] != EBADF:
|
||||
raise
|
||||
try:
|
||||
del d[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
if exc is not None:
|
||||
try:
|
||||
exc(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
continue
|
||||
d[fileno] = rwe
|
||||
self.descriptor_queue.clear()
|
||||
|
||||
def wait(self, seconds=None):
|
||||
|
||||
self.process_queue()
|
||||
|
||||
if seconds is not None:
|
||||
seconds *= 1000000000.0
|
||||
dct = self.descriptors
|
||||
events = kqueue.kevent(self.kfd, [], len(dct), seconds)
|
||||
|
||||
SYSTEM_EXCEPTIONS = self.runloop.SYSTEM_EXCEPTIONS
|
||||
|
||||
E_R = kqueue.EVFILT_READ
|
||||
E_W = kqueue.EVFILT_WRITE
|
||||
E_EOF = kqueue.EV_EOF
|
||||
|
||||
for e in events:
|
||||
fileno = e.ident
|
||||
event = e.filter
|
||||
|
||||
try:
|
||||
read, write, exc = dct[fileno]
|
||||
except KeyError:
|
||||
continue
|
||||
|
||||
if read is not None and event == E_R:
|
||||
try:
|
||||
read(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
elif exc is not None and e.fflags & E_EOF:
|
||||
try:
|
||||
exc(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
|
||||
if write is not None and event == E_W:
|
||||
try:
|
||||
write(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
112
eventlet/logutil.py
Normal file
112
eventlet/logutil.py
Normal file
@@ -0,0 +1,112 @@
|
||||
"""\
|
||||
@file logutil.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import syslog
|
||||
import logging
|
||||
|
||||
|
||||
def file_logger(filename):
|
||||
"""Create a logger. This sucks, the logging module sucks, but
|
||||
it'll do for now.
|
||||
"""
|
||||
handler = logging.FileHandler(filename)
|
||||
formatter = logging.Formatter()
|
||||
handler.setFormatter(formatter)
|
||||
log = logging.getLogger(filename)
|
||||
log.addHandler(handler)
|
||||
log.setLevel(logging.DEBUG)
|
||||
return log, handler
|
||||
|
||||
|
||||
def stream_logger(stream):
|
||||
"""Create a logger. This sucks."""
|
||||
handler = logging.StreamHandler(stream)
|
||||
formatter = logging.Formatter()
|
||||
handler.setFormatter(formatter)
|
||||
log = logging.getLogger()
|
||||
log.addHandler(handler)
|
||||
log.setLevel(logging.DEBUG)
|
||||
return log, handler
|
||||
|
||||
|
||||
class LineLogger(object):
|
||||
towrite = ''
|
||||
def __init__(self, emit=None):
|
||||
if emit is not None:
|
||||
self.emit = emit
|
||||
|
||||
def write(self, stuff):
|
||||
self.towrite += stuff
|
||||
if '\n' in self.towrite:
|
||||
self.flush()
|
||||
|
||||
def flush(self):
|
||||
try:
|
||||
newline = self.towrite.index('\n')
|
||||
except ValueError:
|
||||
newline = len(self.towrite)
|
||||
while True:
|
||||
self.emit(self.towrite[:newline])
|
||||
self.towrite = self.towrite[newline+1:]
|
||||
try:
|
||||
newline = self.towrite.index('\n')
|
||||
except ValueError:
|
||||
break
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
def emit(self, *args):
|
||||
pass
|
||||
|
||||
|
||||
class SysLogger(LineLogger):
|
||||
"""A file-like object which writes to syslog. Can be inserted
|
||||
as sys.stdin and sys.stderr to have logging output redirected
|
||||
to syslog.
|
||||
"""
|
||||
def __init__(self, priority):
|
||||
self.priority = priority
|
||||
|
||||
def emit(self, line):
|
||||
syslog.syslog(self.priority, line)
|
||||
|
||||
|
||||
class TeeLogger(LineLogger):
|
||||
def __init__(self, one, two):
|
||||
self.one, self.two = one, two
|
||||
|
||||
def emit(self, line):
|
||||
self.one.emit(line)
|
||||
self.two.emit(line)
|
||||
|
||||
|
||||
class FileLogger(LineLogger):
|
||||
def __init__(self, file):
|
||||
self.file = file
|
||||
|
||||
def emit(self, line):
|
||||
self.file.write(line + '\n')
|
||||
self.file.flush()
|
||||
|
189
eventlet/pollhub.py
Normal file
189
eventlet/pollhub.py
Normal file
@@ -0,0 +1,189 @@
|
||||
"""\
|
||||
@file pollhub.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import select
|
||||
import socket
|
||||
import errno
|
||||
import traceback
|
||||
from time import sleep
|
||||
|
||||
from eventlet import greenlib
|
||||
from eventlet.runloop import RunLoop, Timer
|
||||
|
||||
import greenlet
|
||||
|
||||
EXC_MASK = select.POLLERR | select.POLLHUP | select.POLLNVAL
|
||||
READ_MASK = select.POLLIN
|
||||
WRITE_MASK = select.POLLOUT
|
||||
|
||||
class Hub(object):
|
||||
def __init__(self):
|
||||
self.runloop = RunLoop(self.wait)
|
||||
self.descriptor_queue = {}
|
||||
self.descriptors = {}
|
||||
self.greenlet = None
|
||||
self.poll = select.poll()
|
||||
|
||||
def stop(self):
|
||||
self.process_queue()
|
||||
self.runloop.abort()
|
||||
if self.greenlet is not greenlet.getcurrent():
|
||||
self.switch()
|
||||
|
||||
def schedule_call(self, *args, **kw):
|
||||
return self.runloop.schedule_call(*args, **kw)
|
||||
|
||||
def switch(self):
|
||||
if not self.greenlet:
|
||||
self.greenlet = greenlib.tracked_greenlet()
|
||||
args = ((self.runloop.run,),)
|
||||
else:
|
||||
args = ()
|
||||
try:
|
||||
greenlet.getcurrent().parent = self.greenlet
|
||||
except ValueError:
|
||||
pass
|
||||
return greenlib.switch(self.greenlet, *args)
|
||||
|
||||
def add_descriptor(self, fileno, read=None, write=None, exc=None):
|
||||
if fileno in self.descriptor_queue:
|
||||
oread, owrite, oexc = self.descriptor_queue[fileno]
|
||||
read, write, exc = read or oread, write or owrite, exc or oexc
|
||||
self.descriptor_queue[fileno] = read, write, exc
|
||||
|
||||
def remove_descriptor(self, fileno):
|
||||
self.descriptor_queue[fileno] = None, None, None
|
||||
|
||||
def exc_descriptor(self, fileno):
|
||||
# We must handle two cases here, the descriptor
|
||||
# may be changing or removing its exc handler
|
||||
# in the queue, or it may be waiting on the queue.
|
||||
exc = None
|
||||
try:
|
||||
exc = self.descriptor_queue[fileno][2]
|
||||
except KeyError:
|
||||
try:
|
||||
exc = self.descriptors[fileno][2]
|
||||
except KeyError:
|
||||
pass
|
||||
if exc is not None:
|
||||
try:
|
||||
exc(fileno)
|
||||
except self.runloop.SYSTEM_EXCEPTIONS:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
|
||||
def squelch_exception(self, fileno, exc_info):
|
||||
traceback.print_exception(*exc_info)
|
||||
print >>sys.stderr, "Removing descriptor: %r" % (fileno,)
|
||||
try:
|
||||
self.remove_descriptor(fileno)
|
||||
except Exception, e:
|
||||
print >>sys.stderr, "Exception while removing descriptor! %r" % (e,)
|
||||
|
||||
def process_queue(self):
|
||||
d = self.descriptors
|
||||
reg = self.poll.register
|
||||
unreg = self.poll.unregister
|
||||
rm = READ_MASK
|
||||
wm = WRITE_MASK
|
||||
for fileno, rwe in self.descriptor_queue.iteritems():
|
||||
read, write, exc = rwe
|
||||
if read is None and write is None and exc is None:
|
||||
try:
|
||||
del d[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
try:
|
||||
unreg(fileno)
|
||||
except socket.error:
|
||||
# print "squelched socket err on unreg", fileno
|
||||
pass
|
||||
else:
|
||||
mask = 0
|
||||
if read is not None:
|
||||
mask |= rm
|
||||
if write is not None:
|
||||
mask |= wm
|
||||
oldmask = 0
|
||||
try:
|
||||
oldr, oldw, olde = d[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
if oldr is not None:
|
||||
oldmask |= rm
|
||||
if oldw is not None:
|
||||
oldmask |= wm
|
||||
if mask != oldmask:
|
||||
reg(fileno, mask)
|
||||
d[fileno] = rwe
|
||||
self.descriptor_queue.clear()
|
||||
|
||||
def wait(self, seconds=None):
|
||||
self.process_queue()
|
||||
|
||||
if not self.descriptors:
|
||||
if seconds:
|
||||
sleep(seconds)
|
||||
return
|
||||
try:
|
||||
presult = self.poll.poll(seconds * 1000.0)
|
||||
except select.error, e:
|
||||
if e.args[0] == errno.EINTR:
|
||||
return
|
||||
raise
|
||||
SYSTEM_EXCEPTIONS = self.runloop.SYSTEM_EXCEPTIONS
|
||||
dct = self.descriptors
|
||||
|
||||
for fileno, event in presult:
|
||||
try:
|
||||
read, write, exc = dct[fileno]
|
||||
except KeyError:
|
||||
continue
|
||||
|
||||
if read is not None and event & READ_MASK:
|
||||
try:
|
||||
read(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
elif exc is not None and event & EXC_MASK:
|
||||
try:
|
||||
exc(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
|
||||
if write is not None and event & WRITE_MASK:
|
||||
try:
|
||||
write(fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
174
eventlet/pools.py
Normal file
174
eventlet/pools.py
Normal file
@@ -0,0 +1,174 @@
|
||||
"""\
|
||||
@file pools.py
|
||||
@author Donovan Preston, Aaron Brashears
|
||||
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import os
|
||||
import socket
|
||||
|
||||
from eventlet import api
|
||||
from eventlet import channel
|
||||
from eventlet import httpc
|
||||
|
||||
class FanFailed(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class SomeFailed(FanFailed):
|
||||
pass
|
||||
|
||||
|
||||
class AllFailed(FanFailed):
|
||||
pass
|
||||
|
||||
|
||||
class Pool(object):
|
||||
"""
|
||||
When using the pool, if you do a get, you should ALWAYS do a put.
|
||||
The pattern is:
|
||||
|
||||
thing = self.pool.get()
|
||||
try:
|
||||
# do stuff
|
||||
finally:
|
||||
self.pool.put(thing)
|
||||
"""
|
||||
def __init__(self, min_size=0, max_size=4):
|
||||
self.min_size = min_size
|
||||
self.max_size = max_size
|
||||
self.current_size = 0
|
||||
self.channel = channel.channel()
|
||||
self.free_items = []
|
||||
for x in range(min_size):
|
||||
self.current_size += 1
|
||||
self.free_items.append(self.create())
|
||||
|
||||
def get(self):
|
||||
"""Return an item from the pool, when one is available
|
||||
"""
|
||||
if self.free_items:
|
||||
return self.free_items.pop(0)
|
||||
if self.current_size < self.max_size:
|
||||
self.current_size += 1
|
||||
return self.create()
|
||||
return self.channel.receive()
|
||||
|
||||
def put(self, item):
|
||||
"""Put an item back into the pool, when done
|
||||
"""
|
||||
if self.channel.balance < 0:
|
||||
self.channel.send(item)
|
||||
else:
|
||||
self.free_items.append(item)
|
||||
|
||||
def free(self):
|
||||
"""Return the number of free items in the pool.
|
||||
"""
|
||||
return len(self.free_items) + self.max_size - self.current_size
|
||||
|
||||
def waiting(self):
|
||||
"""Return the number of routines waiting for a pool item.
|
||||
"""
|
||||
if self.channel.balance < 0:
|
||||
return -self.channel.balance
|
||||
return 0
|
||||
|
||||
def create(self):
|
||||
"""Generate a new pool item
|
||||
"""
|
||||
raise NotImplementedError("Implement in subclass")
|
||||
|
||||
def fan(self, block, input_list):
|
||||
chan = channel.channel()
|
||||
results = []
|
||||
exceptional_results = 0
|
||||
for index, input_item in enumerate(input_list):
|
||||
pool_item = self.get()
|
||||
|
||||
## Fan out
|
||||
api.spawn(
|
||||
self._invoke, block, pool_item, input_item, index, chan)
|
||||
|
||||
## Fan back in
|
||||
for i in range(len(input_list)):
|
||||
## Wait for all guys to send to the queue
|
||||
index, value = chan.receive()
|
||||
if isinstance(value, Exception):
|
||||
exceptional_results += 1
|
||||
results.append((index, value))
|
||||
|
||||
results.sort()
|
||||
results = [value for index, value in results]
|
||||
|
||||
if exceptional_results:
|
||||
if exceptional_results == len(results):
|
||||
raise AllFailed(results)
|
||||
raise SomeFailed(results)
|
||||
return results
|
||||
|
||||
def _invoke(self, block, pool_item, input_item, index, chan):
|
||||
try:
|
||||
result = block(pool_item, input_item)
|
||||
except Exception, e:
|
||||
self.put(pool_item)
|
||||
chan.send((index, e))
|
||||
return
|
||||
self.put(pool_item)
|
||||
chan.send((index, result))
|
||||
|
||||
|
||||
class Token(object):
|
||||
pass
|
||||
|
||||
|
||||
class TokenPool(Pool):
|
||||
"""A pool which gives out tokens, an object indicating that
|
||||
the person who holds the token has a right to consume some
|
||||
limited resource.
|
||||
"""
|
||||
def create(self):
|
||||
return Token()
|
||||
|
||||
|
||||
class ConnectionPool(Pool):
|
||||
"""A Pool which can limit the number of concurrent http operations
|
||||
being made to a given server.
|
||||
|
||||
*NOTE: *TODO:
|
||||
|
||||
This does NOT currently keep sockets open. It discards the created
|
||||
http object when it is put back in the pool. This is because we do
|
||||
not yet have a combination of http clients and servers which can work
|
||||
together to do HTTP keepalive sockets without errors.
|
||||
"""
|
||||
def __init__(self, proto, netloc, use_proxy, min_size=0, max_size=4):
|
||||
self.proto = proto
|
||||
self.netloc = netloc
|
||||
self.use_proxy = use_proxy
|
||||
Pool.__init__(self, min_size, max_size)
|
||||
|
||||
def create(self):
|
||||
return httpc.make_connection(self.proto, self.netloc, self.use_proxy)
|
||||
|
||||
def put(self, item):
|
||||
## Discard item, create a new connection for the pool
|
||||
Pool.put(self, self.create())
|
179
eventlet/pools_test.py
Normal file
179
eventlet/pools_test.py
Normal file
@@ -0,0 +1,179 @@
|
||||
"""\
|
||||
@file test_pools.py
|
||||
@author Donovan Preston, Aaron Brashears
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import unittest
|
||||
|
||||
from eventlet import api
|
||||
from eventlet import channel
|
||||
from eventlet import pools
|
||||
|
||||
|
||||
class IntPool(pools.Pool):
|
||||
def create(self):
|
||||
self.current_integer = getattr(self, 'current_integer', 0) + 1
|
||||
return self.current_integer
|
||||
|
||||
|
||||
class TestIntPool(unittest.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = IntPool(min_size=0, max_size=4)
|
||||
|
||||
def test_integers(self):
|
||||
# Do not actually use this pattern in your code. The pool will be
|
||||
# exhausted, and unrestoreable.
|
||||
# If you do a get, you should ALWAYS do a put, probably like this:
|
||||
# try:
|
||||
# thing = self.pool.get()
|
||||
# # do stuff
|
||||
# finally:
|
||||
# self.pool.put(thing)
|
||||
|
||||
# with self.pool.some_api_name() as thing:
|
||||
# # do stuff
|
||||
self.assertEquals(self.pool.get(), 1)
|
||||
self.assertEquals(self.pool.get(), 2)
|
||||
self.assertEquals(self.pool.get(), 3)
|
||||
self.assertEquals(self.pool.get(), 4)
|
||||
|
||||
def test_free(self):
|
||||
self.assertEquals(self.pool.free(), 4)
|
||||
gotten = self.pool.get()
|
||||
self.assertEquals(self.pool.free(), 3)
|
||||
self.pool.put(gotten)
|
||||
self.assertEquals(self.pool.free(), 4)
|
||||
|
||||
def test_exhaustion(self):
|
||||
waiter = channel.channel()
|
||||
def consumer():
|
||||
gotten = None
|
||||
try:
|
||||
gotten = self.pool.get()
|
||||
finally:
|
||||
waiter.send(gotten)
|
||||
|
||||
api.spawn(consumer)
|
||||
|
||||
one, two, three, four = (
|
||||
self.pool.get(), self.pool.get(), self.pool.get(), self.pool.get())
|
||||
self.assertEquals(self.pool.free(), 0)
|
||||
|
||||
# Let consumer run; nothing will be in the pool, so he will wait
|
||||
api.sleep(0)
|
||||
|
||||
# Wake consumer
|
||||
self.pool.put(one)
|
||||
|
||||
# wait for the consumer
|
||||
self.assertEquals(waiter.receive(), one)
|
||||
|
||||
def test_blocks_on_pool(self):
|
||||
waiter = channel.channel()
|
||||
def greedy():
|
||||
self.pool.get()
|
||||
self.pool.get()
|
||||
self.pool.get()
|
||||
self.pool.get()
|
||||
# No one should be waiting yet.
|
||||
self.assertEquals(self.pool.waiting(), 0)
|
||||
# The call to the next get will unschedule this routine.
|
||||
self.pool.get()
|
||||
# So this send should never be called.
|
||||
waiter.send('Failed!')
|
||||
|
||||
killable = api.spawn(greedy)
|
||||
|
||||
# no one should be waiting yet.
|
||||
self.assertEquals(self.pool.waiting(), 0)
|
||||
|
||||
## Wait for greedy
|
||||
api.sleep(0)
|
||||
|
||||
## Greedy should be blocking on the last get
|
||||
self.assertEquals(self.pool.waiting(), 1)
|
||||
|
||||
## Send will never be called, so balance should be 0.
|
||||
self.assertEquals(waiter.balance, 0)
|
||||
|
||||
api.kill(killable)
|
||||
|
||||
|
||||
class TestAbstract(unittest.TestCase):
|
||||
mode = 'static'
|
||||
def test_abstract(self):
|
||||
## Going for 100% coverage here
|
||||
## A Pool cannot be used without overriding create()
|
||||
pool = pools.Pool()
|
||||
self.assertRaises(NotImplementedError, pool.get)
|
||||
|
||||
|
||||
class TestIntPool2(unittest.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = IntPool(min_size=3, max_size=3)
|
||||
|
||||
def test_something(self):
|
||||
self.assertEquals(len(self.pool.free_items), 3)
|
||||
## Cover the clause in get where we get from the free list instead of creating
|
||||
## an item on get
|
||||
gotten = self.pool.get()
|
||||
self.assertEquals(gotten, 1)
|
||||
|
||||
|
||||
ALWAYS = RuntimeError('I always fail')
|
||||
SOMETIMES = RuntimeError('I fail half the time')
|
||||
|
||||
|
||||
class TestFan(unittest.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = IntPool(max_size=2)
|
||||
|
||||
def test_with_list(self):
|
||||
list_of_input = ['agent-one', 'agent-two', 'agent-three']
|
||||
|
||||
def my_callable(pool_item, next_thing):
|
||||
## Do some "blocking" (yielding) thing
|
||||
api.sleep(0.01)
|
||||
return next_thing
|
||||
|
||||
output = self.pool.fan(my_callable, list_of_input)
|
||||
self.assertEquals(list_of_input, output)
|
||||
|
||||
def test_all_fail(self):
|
||||
def my_failure(pool_item, next_thing):
|
||||
raise ALWAYS
|
||||
self.assertRaises(pools.AllFailed, self.pool.fan, my_failure, range(4))
|
||||
|
||||
def test_some_fail(self):
|
||||
def my_failing_callable(pool_item, next_thing):
|
||||
if next_thing % 2:
|
||||
raise SOMETIMES
|
||||
return next_thing
|
||||
self.assertRaises(pools.SomeFailed, self.pool.fan, my_failing_callable, range(4))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
138
eventlet/processes.py
Normal file
138
eventlet/processes.py
Normal file
@@ -0,0 +1,138 @@
|
||||
"""\
|
||||
@file processes.py
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
|
||||
import os
|
||||
import popen2
|
||||
import signal
|
||||
|
||||
|
||||
from eventlet import util, pools
|
||||
from eventlet import wrappedfd
|
||||
|
||||
class DeadProcess(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class Process(object):
|
||||
process_number = 0
|
||||
def __init__(self, command, args, dead_callback=lambda:None):
|
||||
self.process_number = self.process_number + 1
|
||||
Process.process_number = self.process_number
|
||||
self.command = command
|
||||
self.args = args
|
||||
self._dead_callback = dead_callback
|
||||
self.dead = False
|
||||
self.started = False
|
||||
self.popen4 = None
|
||||
|
||||
## We use popen4 so that read() will read from either stdout or stderr
|
||||
self.popen4 = popen2.Popen4([self.command] + self.args)
|
||||
child_stdout_stderr = self.popen4.fromchild
|
||||
child_stdin = self.popen4.tochild
|
||||
util.set_nonblocking(child_stdout_stderr)
|
||||
util.set_nonblocking(child_stdin)
|
||||
self.child_stdout_stderr = wrappedfd.wrapped_file(child_stdout_stderr)
|
||||
self.child_stdout_stderr.newlines = '\n' # the default is \r\n, which aren't sent over pipes
|
||||
self.child_stdin = wrappedfd.wrapped_file(child_stdin)
|
||||
self.child_stdin.newlines = '\n'
|
||||
|
||||
self.sendall = self.child_stdin.write
|
||||
self.send = self.child_stdin.write
|
||||
self.recv = self.child_stdout_stderr.read
|
||||
self.readline = self.child_stdout_stderr.readline
|
||||
|
||||
def dead_callback(self):
|
||||
self.dead = True
|
||||
if self._dead_callback:
|
||||
self._dead_callback()
|
||||
|
||||
def makefile(self, mode, *arg):
|
||||
if mode.startswith('r'):
|
||||
return self.child_stdout_stderr
|
||||
if mode.startswith('w'):
|
||||
return self.child_stdin
|
||||
raise RuntimeError("Unknown mode", mode)
|
||||
|
||||
def read(self, amount=None):
|
||||
result = self.child_stdout_stderr.read(amount)
|
||||
if result == '':
|
||||
# This process is dead.
|
||||
self.dead_callback()
|
||||
raise DeadProcess
|
||||
return result
|
||||
|
||||
def write(self, stuff):
|
||||
written = self.child_stdin.send(stuff)
|
||||
try:
|
||||
self.child_stdin.flush()
|
||||
except ValueError, e:
|
||||
## File was closed
|
||||
assert str(e) == 'I/O operation on closed file'
|
||||
if written == 0:
|
||||
self.dead_callback()
|
||||
raise DeadProcess
|
||||
|
||||
def flush(self):
|
||||
self.child_stdin.flush()
|
||||
|
||||
def close(self):
|
||||
self.child_stdout_stderr.close()
|
||||
self.child_stdin.close()
|
||||
self.dead_callback()
|
||||
|
||||
def close_stdin(self):
|
||||
self.child_stdin.close()
|
||||
|
||||
def kill(self, sig=None):
|
||||
if sig == None:
|
||||
sig = signal.SIGTERM
|
||||
os.kill(self.popen4.pid, sig)
|
||||
|
||||
def getpid(self):
|
||||
return self.popen4.pid
|
||||
|
||||
|
||||
class ProcessPool(pools.Pool):
|
||||
def __init__(self, command, args=None, min_size=0, max_size=4):
|
||||
"""@param command the command to run
|
||||
"""
|
||||
self.command = command
|
||||
if args is None:
|
||||
args = []
|
||||
self.args = args
|
||||
pools.Pool.__init__(self, min_size, max_size)
|
||||
|
||||
def create(self):
|
||||
"""Generate a process
|
||||
"""
|
||||
def dead_callback():
|
||||
self.current_size -= 1
|
||||
return Process(self.command, self.args, dead_callback)
|
||||
|
||||
def put(self, item):
|
||||
if not item.dead:
|
||||
if item.popen4.poll() != -1:
|
||||
item.dead_callback()
|
||||
else:
|
||||
pools.Pool.put(self, item)
|
134
eventlet/processes_test.py
Normal file
134
eventlet/processes_test.py
Normal file
@@ -0,0 +1,134 @@
|
||||
"""\
|
||||
@file processes_test.py
|
||||
@author Donovan Preston, Aaron Brashears
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
from eventlet import tests
|
||||
from eventlet import api
|
||||
from eventlet import channel
|
||||
from eventlet import processes
|
||||
|
||||
class TestEchoPool(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = processes.ProcessPool('echo', ["hello"])
|
||||
|
||||
def test_echo(self):
|
||||
result = None
|
||||
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
result = proc.read()
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
self.assertEquals(result, 'hello\n')
|
||||
|
||||
def test_read_eof(self):
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
proc.read()
|
||||
self.assertRaises(processes.DeadProcess, proc.read)
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
|
||||
|
||||
class TestCatPool(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = processes.ProcessPool('cat')
|
||||
|
||||
def test_cat(self):
|
||||
result = None
|
||||
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
proc.write('goodbye')
|
||||
proc.close_stdin()
|
||||
result = proc.read()
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
|
||||
self.assertEquals(result, 'goodbye')
|
||||
|
||||
def test_write_to_dead(self):
|
||||
result = None
|
||||
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
proc.write('goodbye')
|
||||
proc.close_stdin()
|
||||
result = proc.read()
|
||||
self.assertRaises(processes.DeadProcess, proc.write, 'foo')
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
|
||||
def test_close(self):
|
||||
result = None
|
||||
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
proc.write('hello')
|
||||
proc.close()
|
||||
self.assertRaises(processes.DeadProcess, proc.write, 'goodbye')
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
|
||||
|
||||
class TestDyingProcessesLeavePool(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = processes.ProcessPool('echo', ['hello'], max_size=1)
|
||||
|
||||
def test_dead_process_not_inserted_into_pool(self):
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
result = proc.read()
|
||||
self.assertEquals(result, 'hello\n')
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
proc2 = self.pool.get()
|
||||
self.assert_(proc is not proc2)
|
||||
|
||||
|
||||
class TestProcessLivesForever(tests.TestCase):
|
||||
mode = 'static'
|
||||
def setUp(self):
|
||||
self.pool = processes.ProcessPool('yes', max_size=1)
|
||||
|
||||
def test_reading_twice_from_same_process(self):
|
||||
proc = self.pool.get()
|
||||
try:
|
||||
result = proc.read(2)
|
||||
self.assertEquals(result, 'y\n')
|
||||
finally:
|
||||
self.pool.put(proc)
|
||||
|
||||
proc2 = self.pool.get()
|
||||
self.assert_(proc is proc2)
|
||||
try:
|
||||
result = proc2.read(2)
|
||||
self.assertEquals(result, 'y\n')
|
||||
finally:
|
||||
self.pool.put(proc2)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
tests.main()
|
42
eventlet/pylibsupport.py
Normal file
42
eventlet/pylibsupport.py
Normal file
@@ -0,0 +1,42 @@
|
||||
"""\
|
||||
@file pylibsupport.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2005-2006, Donovan Preston
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
|
||||
from py.magic import greenlet
|
||||
|
||||
|
||||
import sys
|
||||
import types
|
||||
|
||||
|
||||
def emulate():
|
||||
module = types.ModuleType('greenlet')
|
||||
sys.modules['greenlet'] = module
|
||||
module.greenlet = greenlet
|
||||
module.getcurrent = greenlet.getcurrent
|
||||
module.GreenletExit = greenlet.GreenletExit
|
||||
|
||||
|
||||
|
200
eventlet/runloop.py
Normal file
200
eventlet/runloop.py
Normal file
@@ -0,0 +1,200 @@
|
||||
"""\
|
||||
@file runloop.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Defines the core eventlet runloop. The runloop keeps track of scheduled
|
||||
events and observers which watch for specific portions of the runloop to
|
||||
be executed.
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import time
|
||||
import bisect
|
||||
import sys
|
||||
import traceback
|
||||
|
||||
from eventlet.timer import Timer
|
||||
|
||||
|
||||
class RunLoop(object):
|
||||
SYSTEM_EXCEPTIONS = (KeyboardInterrupt, SystemExit)
|
||||
|
||||
def __init__(self, wait=None, clock=None):
|
||||
if clock is None:
|
||||
clock = self.default_clock()
|
||||
self.clock = clock
|
||||
if wait is None:
|
||||
wait = self.default_wait
|
||||
self.wait = wait
|
||||
self.stopping = False
|
||||
self.running = False
|
||||
self.timers = []
|
||||
self.next_timers = []
|
||||
self.observers = {}
|
||||
self.observer_modes = {
|
||||
'entry': [],
|
||||
'before_timers': [],
|
||||
'before_waiting': [],
|
||||
'after_waiting': [],
|
||||
'exit': [],
|
||||
}
|
||||
|
||||
def default_wait(self, time):
|
||||
return None
|
||||
|
||||
def default_clock(self):
|
||||
return time.time
|
||||
|
||||
def default_sleep(self):
|
||||
return 60.0
|
||||
|
||||
def sleep_until(self):
|
||||
t = self.timers
|
||||
if not t:
|
||||
return None
|
||||
return t[0][0]
|
||||
|
||||
def run(self):
|
||||
"""Run the runloop until abort is called.
|
||||
"""
|
||||
if self.running:
|
||||
raise RuntimeError("Already running!")
|
||||
try:
|
||||
self.running = True
|
||||
self.stopping = False
|
||||
self.fire_observers('entry')
|
||||
while not self.stopping:
|
||||
self.prepare_timers()
|
||||
self.fire_observers('before_timers')
|
||||
self.fire_timers(self.clock())
|
||||
self.prepare_timers()
|
||||
wakeup_when = self.sleep_until()
|
||||
if wakeup_when is None:
|
||||
sleep_time = self.default_sleep()
|
||||
else:
|
||||
sleep_time = wakeup_when - self.clock()
|
||||
if sleep_time > 0:
|
||||
self.fire_observers('before_waiting')
|
||||
self.wait(sleep_time)
|
||||
self.fire_observers('after_waiting')
|
||||
else:
|
||||
self.wait(0)
|
||||
else:
|
||||
del self.timers[:]
|
||||
del self.next_timers[:]
|
||||
self.fire_observers('exit')
|
||||
finally:
|
||||
self.running = False
|
||||
self.stopping = False
|
||||
|
||||
def abort(self):
|
||||
"""Stop the runloop. If run is executing, it will exit after completing
|
||||
the next runloop iteration.
|
||||
"""
|
||||
if self.running:
|
||||
self.stopping = True
|
||||
|
||||
def add_observer(self, observer, *modes):
|
||||
"""Add an event observer to this runloop with the given modes.
|
||||
Valid modes are:
|
||||
entry: The runloop is being entered.
|
||||
before_timers: Before the expired timers for this iteration are executed.
|
||||
before_waiting: Before waiting for the calculated wait_time
|
||||
where nothing will happen.
|
||||
after_waiting: After waiting, immediately before starting the top of the
|
||||
runloop again.
|
||||
exit: The runloop is exiting.
|
||||
|
||||
If no mode is passed or mode is all, the observer will be fired for every
|
||||
event type.
|
||||
"""
|
||||
if not modes or modes == ('all',):
|
||||
modes = tuple(self.observer_modes)
|
||||
self.observers[observer] = modes
|
||||
for mode in modes:
|
||||
self.observer_modes[mode].append(observer)
|
||||
|
||||
def remove_observer(self, observer):
|
||||
"""Remove a previously registered observer from all event types.
|
||||
"""
|
||||
for mode in self.observers.pop(observer, ()):
|
||||
self.observer_modes[mode].remove(observer)
|
||||
|
||||
def squelch_observer_exception(self, observer, exc_info):
|
||||
traceback.print_exception(*exc_info)
|
||||
print >>sys.stderr, "Removing observer: %r" % (observer,)
|
||||
self.remove_observer(observer)
|
||||
|
||||
def fire_observers(self, activity):
|
||||
for observer in self.observer_modes[activity]:
|
||||
try:
|
||||
observer(self, activity)
|
||||
except self.SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_observer_exception(observer, sys.exc_info())
|
||||
|
||||
def squelch_timer_exception(self, timer, exc_info):
|
||||
traceback.print_exception(*exc_info)
|
||||
print >>sys.stderr, "Timer raised: %r" % (timer,)
|
||||
|
||||
def _add_absolute_timer(self, when, info):
|
||||
# the 0 placeholder makes it easy to bisect_right using (now, 1)
|
||||
self.next_timers.append((when, 0, info))
|
||||
|
||||
def add_timer(self, timer):
|
||||
scheduled_time = self.clock() + timer.seconds
|
||||
self._add_absolute_timer(scheduled_time, timer)
|
||||
return scheduled_time
|
||||
|
||||
def prepare_timers(self):
|
||||
ins = bisect.insort_right
|
||||
t = self.timers
|
||||
for item in self.next_timers:
|
||||
ins(t, item)
|
||||
del self.next_timers[:]
|
||||
|
||||
def schedule_call(self, seconds, cb, *args, **kw):
|
||||
"""Schedule a callable to be called after 'seconds' seconds have
|
||||
elapsed.
|
||||
seconds: The number of seconds to wait.
|
||||
cb: The callable to call after the given time.
|
||||
*args: Arguments to pass to the callable when called.
|
||||
**kw: Keyword arguments to pass to the callable when called.
|
||||
"""
|
||||
t = Timer(seconds, cb, *args, **kw)
|
||||
self.add_timer(t)
|
||||
return t
|
||||
|
||||
def fire_timers(self, when):
|
||||
t = self.timers
|
||||
last = bisect.bisect_right(t, (when, 1))
|
||||
i = 0
|
||||
for i in xrange(last):
|
||||
timer = t[i][2]
|
||||
try:
|
||||
timer()
|
||||
except self.SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_timer_exception(timer, sys.exc_info())
|
||||
del t[:last]
|
157
eventlet/runloop_test.py
Normal file
157
eventlet/runloop_test.py
Normal file
@@ -0,0 +1,157 @@
|
||||
"""\
|
||||
@file test_runloop.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import time
|
||||
import StringIO
|
||||
import unittest
|
||||
|
||||
from eventlet import runloop
|
||||
|
||||
|
||||
class TestRunloop(unittest.TestCase):
|
||||
mode = 'static'
|
||||
def test_empty(self):
|
||||
r = runloop.RunLoop()
|
||||
r.schedule_call(0, r.abort)
|
||||
r.run()
|
||||
assert not r.running
|
||||
|
||||
|
||||
def test_timer(self):
|
||||
r = runloop.RunLoop()
|
||||
r.schedule_call(0.125, r.abort)
|
||||
start_time = time.time()
|
||||
r.run()
|
||||
assert time.time() - start_time >= 0.125
|
||||
assert not r.running
|
||||
|
||||
def test_observer(self):
|
||||
observed = []
|
||||
r = runloop.RunLoop()
|
||||
r.add_observer(lambda runloop, activity: observed.append(activity))
|
||||
r.schedule_call(0, r.abort)
|
||||
r.run()
|
||||
assert observed == ['entry', 'before_timers', 'before_waiting', 'after_waiting', 'exit']
|
||||
assert not r.running
|
||||
|
||||
|
||||
def test_remove_observer(self):
|
||||
r = runloop.RunLoop()
|
||||
|
||||
observed = []
|
||||
def observe(runloop, mode):
|
||||
observed.append(mode)
|
||||
r.remove_observer(observe)
|
||||
|
||||
looped = []
|
||||
def run_loop_twice(runloop, mode):
|
||||
if looped:
|
||||
r.abort()
|
||||
else:
|
||||
looped.append(True)
|
||||
|
||||
r.add_observer(observe, 'before_timers')
|
||||
r.add_observer(run_loop_twice, 'after_waiting')
|
||||
r.run()
|
||||
assert len(observed) == 1
|
||||
assert not r.running
|
||||
|
||||
def test_observer_exception(self):
|
||||
r = runloop.RunLoop()
|
||||
|
||||
observed = []
|
||||
def observe(runloop, mode):
|
||||
observed.append(mode)
|
||||
raise Exception("Squelch me please")
|
||||
|
||||
looped = []
|
||||
def run_loop_twice(runloop, mode):
|
||||
if looped:
|
||||
r.abort()
|
||||
else:
|
||||
looped.append(True)
|
||||
|
||||
saved = sys.stderr
|
||||
sys.stderr = err = StringIO.StringIO()
|
||||
|
||||
r.add_observer(observe, 'before_timers')
|
||||
r.add_observer(run_loop_twice, 'after_waiting')
|
||||
r.run()
|
||||
|
||||
err.seek(0)
|
||||
sys.stderr = saved
|
||||
|
||||
assert len(observed) == 1
|
||||
assert err.read()
|
||||
assert not r.running
|
||||
|
||||
def test_timer_exception(self):
|
||||
r = runloop.RunLoop()
|
||||
|
||||
observed = []
|
||||
def timer():
|
||||
observed.append(True)
|
||||
raise Exception("Squelch me please")
|
||||
|
||||
looped = []
|
||||
def run_loop_twice(runloop, mode):
|
||||
if looped:
|
||||
r.abort()
|
||||
else:
|
||||
looped.append(True)
|
||||
|
||||
saved = sys.stderr
|
||||
sys.stderr = err = StringIO.StringIO()
|
||||
|
||||
r.schedule_call(0, timer)
|
||||
r.add_observer(run_loop_twice, 'after_waiting')
|
||||
r.run()
|
||||
|
||||
err.seek(0)
|
||||
sys.stderr = saved
|
||||
|
||||
assert len(observed) == 1
|
||||
assert err.read()
|
||||
assert not r.running
|
||||
|
||||
def test_timer_system_exception(self):
|
||||
r = runloop.RunLoop()
|
||||
def timer():
|
||||
raise SystemExit
|
||||
|
||||
r.schedule_call(0, timer)
|
||||
|
||||
caught = []
|
||||
try:
|
||||
r.run()
|
||||
except SystemExit:
|
||||
caught.append(True)
|
||||
|
||||
assert caught
|
||||
assert not r.running
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
173
eventlet/selecthub.py
Normal file
173
eventlet/selecthub.py
Normal file
@@ -0,0 +1,173 @@
|
||||
"""\
|
||||
@file selecthub.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import select
|
||||
import errno
|
||||
import traceback
|
||||
import time
|
||||
from bisect import insort, bisect_left
|
||||
|
||||
from eventlet import greenlib
|
||||
from eventlet.runloop import RunLoop, Timer
|
||||
|
||||
import greenlet
|
||||
|
||||
class Hub(object):
|
||||
def __init__(self):
|
||||
self.runloop = RunLoop(self.wait)
|
||||
self.readers = {}
|
||||
self.writers = {}
|
||||
self.excs = {}
|
||||
self.descriptors = {}
|
||||
self.descriptor_queue = {}
|
||||
self.greenlet = None
|
||||
|
||||
def stop(self):
|
||||
self.process_queue()
|
||||
self.runloop.abort()
|
||||
if self.greenlet is not greenlet.getcurrent():
|
||||
self.switch()
|
||||
|
||||
def schedule_call(self, *args, **kw):
|
||||
return self.runloop.schedule_call(*args, **kw)
|
||||
|
||||
def switch(self):
|
||||
if not self.greenlet:
|
||||
self.greenlet = greenlib.tracked_greenlet()
|
||||
args = ((self.runloop.run,),)
|
||||
else:
|
||||
args = ()
|
||||
try:
|
||||
greenlet.getcurrent().parent = self.greenlet
|
||||
except ValueError:
|
||||
pass
|
||||
return greenlib.switch(self.greenlet, *args)
|
||||
|
||||
def add_descriptor(self, fileno, read=None, write=None, exc=None):
|
||||
self.descriptor_queue[fileno] = read, write, exc
|
||||
|
||||
def remove_descriptor(self, fileno):
|
||||
self.descriptor_queue[fileno] = None, None, None
|
||||
|
||||
def exc_descriptor(self, fileno):
|
||||
# We must handle two cases here, the descriptor
|
||||
# may be changing or removing its exc handler
|
||||
# in the queue, or it may be waiting on the queue.
|
||||
exc = None
|
||||
try:
|
||||
exc = self.descriptor_queue[fileno][2]
|
||||
except KeyError:
|
||||
try:
|
||||
exc = self.excs[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
if exc is not None:
|
||||
try:
|
||||
exc()
|
||||
except self.runloop.SYSTEM_EXCEPTIONS:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
||||
|
||||
def squelch_exception(self, fileno, exc_info):
|
||||
traceback.print_exception(*exc_info)
|
||||
print >>sys.stderr, "Removing descriptor: %r" % (fileno,)
|
||||
try:
|
||||
self.remove_descriptor(fileno)
|
||||
except Exception, e:
|
||||
print >>sys.stderr, "Exception while removing descriptor! %r" % (e,)
|
||||
|
||||
def process_queue(self):
|
||||
readers = self.readers
|
||||
writers = self.writers
|
||||
excs = self.excs
|
||||
descriptors = self.descriptors
|
||||
for fileno, rwe in self.descriptor_queue.iteritems():
|
||||
read, write, exc = rwe
|
||||
if read is None and write is None and exc is None:
|
||||
try:
|
||||
del descriptors[fileno]
|
||||
except KeyError:
|
||||
continue
|
||||
try:
|
||||
del readers[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
try:
|
||||
del writers[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
try:
|
||||
del excs[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
if read is not None:
|
||||
readers[fileno] = read
|
||||
else:
|
||||
try:
|
||||
del readers[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
if write is not None:
|
||||
writers[fileno] = write
|
||||
else:
|
||||
try:
|
||||
del writers[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
if exc is not None:
|
||||
excs[fileno] = exc
|
||||
else:
|
||||
try:
|
||||
del excs[fileno]
|
||||
except KeyError:
|
||||
pass
|
||||
descriptors[fileno] = rwe
|
||||
self.descriptor_queue.clear()
|
||||
|
||||
def wait(self, seconds=None):
|
||||
self.process_queue()
|
||||
if not self.descriptors:
|
||||
if seconds:
|
||||
time.sleep(seconds)
|
||||
return
|
||||
readers = self.readers
|
||||
writers = self.writers
|
||||
excs = self.excs
|
||||
try:
|
||||
r, w, ig = select.select(readers, writers, [], seconds)
|
||||
except select.error, e:
|
||||
if e.args[0] == errno.EINTR:
|
||||
return
|
||||
raise
|
||||
SYSTEM_EXCEPTIONS = self.runloop.SYSTEM_EXCEPTIONS
|
||||
for observed, events in ((readers, r), (writers, w)):
|
||||
for fileno in events:
|
||||
try:
|
||||
observed[fileno](fileno)
|
||||
except SYSTEM_EXCEPTIONS:
|
||||
raise
|
||||
except:
|
||||
self.squelch_exception(fileno, sys.exc_info())
|
110
eventlet/stacklesssupport.py
Normal file
110
eventlet/stacklesssupport.py
Normal file
@@ -0,0 +1,110 @@
|
||||
"""\
|
||||
@file __init__.py
|
||||
@brief Support for using stackless python. Broken and riddled with
|
||||
print statements at the moment. Please fix it!
|
||||
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import types
|
||||
|
||||
import stackless
|
||||
import traceback
|
||||
|
||||
|
||||
caller = None
|
||||
|
||||
|
||||
coro_args = {}
|
||||
|
||||
|
||||
tasklet_to_greenlet = {}
|
||||
|
||||
|
||||
def getcurrent():
|
||||
return tasklet_to_greenlet[stackless.getcurrent()]
|
||||
|
||||
|
||||
class FirstSwitch(object):
|
||||
def __init__(self, gr):
|
||||
self.gr = gr
|
||||
|
||||
def __call__(self, *args, **kw):
|
||||
print "first call", args, kw
|
||||
gr = self.gr
|
||||
del gr.switch
|
||||
run, gr.run = gr.run, None
|
||||
t = stackless.tasklet(run)
|
||||
gr.t = t
|
||||
tasklet_to_greenlet[t] = gr
|
||||
t.setup(*args, **kw)
|
||||
result = t.run()
|
||||
|
||||
|
||||
class greenlet(object):
|
||||
def __init__(self, run=None, parent=None):
|
||||
self.dead = False
|
||||
if parent is None:
|
||||
parent = getcurrent()
|
||||
|
||||
self.parent = parent
|
||||
if run is not None:
|
||||
self.run = run
|
||||
|
||||
self.switch = FirstSwitch(self)
|
||||
|
||||
def switch(self, *args):
|
||||
print "switch", args
|
||||
global caller
|
||||
caller = stackless.getcurrent()
|
||||
coro_args[self] = args
|
||||
self.t.insert()
|
||||
stackless.schedule()
|
||||
if caller is not self.t:
|
||||
caller.remove()
|
||||
rval = coro_args[self]
|
||||
return rval
|
||||
|
||||
def run(self):
|
||||
pass
|
||||
|
||||
def __bool__(self):
|
||||
return self.run is None and not self.dead
|
||||
|
||||
|
||||
class GreenletExit(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def emulate():
|
||||
module = types.ModuleType('greenlet')
|
||||
sys.modules['greenlet'] = module
|
||||
module.greenlet = greenlet
|
||||
module.getcurrent = getcurrent
|
||||
module.GreenletExit = GreenletExit
|
||||
|
||||
caller = t = stackless.getcurrent()
|
||||
tasklet_to_greenlet[t] = None
|
||||
main_coro = greenlet()
|
||||
tasklet_to_greenlet[t] = main_coro
|
||||
main_coro.t = t
|
||||
del main_coro.switch ## It's already running
|
||||
coro_args[main_coro] = None
|
36
eventlet/tests.py
Normal file
36
eventlet/tests.py
Normal file
@@ -0,0 +1,36 @@
|
||||
"""\
|
||||
@file tests.py
|
||||
@author Donovan Preston
|
||||
@brief Indirection layer for tests in case we want to fix unittest.
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import atexit
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
|
||||
TestCase = unittest.TestCase
|
||||
|
||||
|
||||
name = getattr(sys.modules['__main__'], '__name__', None)
|
||||
|
||||
main = unittest.main
|
72
eventlet/timer.py
Normal file
72
eventlet/timer.py
Normal file
@@ -0,0 +1,72 @@
|
||||
"""\
|
||||
@file timer.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
from eventlet.api import get_hub
|
||||
|
||||
class Timer(object):
|
||||
__slots__ = ['seconds', 'tpl', 'called', 'cancelled', 'scheduled_time']
|
||||
def __init__(self, seconds, cb, *args, **kw):
|
||||
"""Create a timer.
|
||||
seconds: The minimum number of seconds to wait before calling
|
||||
cb: The callback to call when the timer has expired
|
||||
*args: The arguments to pass to cb
|
||||
**kw: The keyword arguments to pass to cb
|
||||
|
||||
This timer will not be run unless it is scheduled in a runloop by
|
||||
calling timer.schedule() or runloop.add_timer(timer).
|
||||
"""
|
||||
self.cancelled = False
|
||||
self.seconds = seconds
|
||||
self.tpl = cb, args, kw
|
||||
self.called = False
|
||||
|
||||
def __repr__(self):
|
||||
secs = getattr(self, 'seconds', None)
|
||||
cb, args, kw = getattr(self, 'tpl', (None, None, None))
|
||||
return "Timer(%s, %s, *%s, **%s)" % (
|
||||
secs, cb, args, kw)
|
||||
|
||||
def copy(self):
|
||||
cb, args, kw = self.tpl
|
||||
return self.__class__(self.seconds, cb, *args, **kw)
|
||||
|
||||
def schedule(self):
|
||||
"""Schedule this timer to run in the current runloop.
|
||||
"""
|
||||
self.called = False
|
||||
self.scheduled_time = get_hub().runloop.add_timer(self)
|
||||
return self
|
||||
|
||||
def __call__(self):
|
||||
if not self.called:
|
||||
self.called = True
|
||||
cb, args, kw = self.tpl
|
||||
cb(*args, **kw)
|
||||
|
||||
def cancel(self):
|
||||
"""Prevent this timer from being called. If the timer has already
|
||||
been called, has no effect.
|
||||
"""
|
||||
self.cancelled = True
|
||||
self.called = True
|
66
eventlet/timer_test.py
Normal file
66
eventlet/timer_test.py
Normal file
@@ -0,0 +1,66 @@
|
||||
"""\
|
||||
@file test_timer.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import unittest
|
||||
|
||||
from eventlet import api, runloop, tests, timer
|
||||
|
||||
class TestTimer(tests.TestCase):
|
||||
mode = 'static'
|
||||
def test_copy(self):
|
||||
t = timer.Timer(0, lambda: None)
|
||||
t2 = t.copy()
|
||||
assert t.seconds == t2.seconds
|
||||
assert t.tpl == t2.tpl
|
||||
assert t.called == t2.called
|
||||
|
||||
def test_cancel(self):
|
||||
r = runloop.RunLoop()
|
||||
called = []
|
||||
t = timer.Timer(0, lambda: called.append(True))
|
||||
t.cancel()
|
||||
r.add_timer(t)
|
||||
r.add_observer(lambda r, activity: r.abort(), 'after_waiting')
|
||||
r.run()
|
||||
assert not called
|
||||
assert not r.running
|
||||
|
||||
def test_schedule(self):
|
||||
hub = api.get_hub()
|
||||
r = hub.runloop
|
||||
# clean up the runloop, preventing side effects from previous tests
|
||||
# on this thread
|
||||
if r.running:
|
||||
r.abort()
|
||||
api.sleep(0)
|
||||
called = []
|
||||
t = timer.Timer(0, lambda: (called.append(True), hub.runloop.abort()))
|
||||
t.schedule()
|
||||
r.default_sleep = lambda: 0.0
|
||||
r.run()
|
||||
assert called
|
||||
assert not r.running
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
57
eventlet/tls.py
Normal file
57
eventlet/tls.py
Normal file
@@ -0,0 +1,57 @@
|
||||
"""\
|
||||
@file tls.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2006-2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import threading
|
||||
import weakref
|
||||
|
||||
__all__ = ['local']
|
||||
|
||||
class _local(object):
|
||||
"""
|
||||
Crappy Python 2.3 compatible implementation of thread-local storage
|
||||
"""
|
||||
|
||||
__slots__ = ('__thread_dict__',)
|
||||
|
||||
def __init__(self):
|
||||
object.__setattr__(self, '__thread_dict__', weakref.WeakKeyDictionary())
|
||||
|
||||
def __getattr__(self, attr):
|
||||
try:
|
||||
return self.__thread_dict__[threading.currentThread()][attr]
|
||||
except KeyError:
|
||||
raise AttributeError(attr)
|
||||
|
||||
def __setattr__(self, attr, value):
|
||||
t = threading.currentThread()
|
||||
try:
|
||||
d = self.__thread_dict__[t]
|
||||
except KeyError:
|
||||
d = self.__thread_dict__[t] = {}
|
||||
d[attr] = value
|
||||
|
||||
try:
|
||||
local = threading.local
|
||||
except AttributeError:
|
||||
local = _local
|
134
eventlet/twistedsupport.py
Normal file
134
eventlet/twistedsupport.py
Normal file
@@ -0,0 +1,134 @@
|
||||
"""\
|
||||
@file twistedsupport.py
|
||||
@author Donovan Preston
|
||||
|
||||
Copyright (c) 2005-2006, Donovan Preston
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import traceback
|
||||
|
||||
from eventlet import api
|
||||
from eventlet import timer
|
||||
|
||||
from twisted.internet import posixbase
|
||||
from twisted.internet.interfaces import IReactorFDSet
|
||||
|
||||
try:
|
||||
from zope.interface import implements
|
||||
_working = True
|
||||
except ImportError:
|
||||
_working = False
|
||||
def implements(*args, **kw):
|
||||
pass
|
||||
|
||||
|
||||
class TwistedTimer(object):
|
||||
def __init__(self, timer):
|
||||
self.timer = timer
|
||||
|
||||
def cancel(self):
|
||||
self.timer.cancel()
|
||||
|
||||
def getTime(self):
|
||||
return self.timer.scheduled_time
|
||||
|
||||
def delay(self, seconds):
|
||||
hub = api.get_hub()
|
||||
new_time = hub.clock() - self.timer_scheduled_time + seconds
|
||||
self.timer.cancel()
|
||||
cb, args, kw = self.timer.tpl
|
||||
self.timer = hub.schedule_call(new_time, cb, *args, **kw)
|
||||
|
||||
def reset(self, new_time):
|
||||
self.timer.cancel()
|
||||
cb, args, kw = self.timer.tpl
|
||||
self.timer = api.get_hub().schedule_call(new_time, cb, *args, **kw)
|
||||
|
||||
def active(self):
|
||||
return not self.timer.called
|
||||
|
||||
|
||||
class EventletReactor(posixbase.PosixReactorBase):
|
||||
implements(IReactorFDSet)
|
||||
|
||||
def __init__(self, *args, **kw):
|
||||
self._readers = {}
|
||||
self._writers = {}
|
||||
posixbase.PosixReactorBase.__init__(self, *args, **kw)
|
||||
|
||||
def callLater(self, func, *args, **kw):
|
||||
return TwistedTimer(api.call_after(func, *args, **kw))
|
||||
|
||||
def run(self):
|
||||
self.running = True
|
||||
self._stopper = api.call_after(sys.maxint / 1000.0, lambda: None)
|
||||
## schedule a call way in the future, and cancel it in stop?
|
||||
api.get_hub().runloop.run()
|
||||
|
||||
def stop(self):
|
||||
self._stopper.cancel()
|
||||
posixbase.PosixReactorBase.stop(self)
|
||||
api.get_hub().remove_descriptor(self._readers.keys()[0])
|
||||
api.get_hub().runloop.abort()
|
||||
|
||||
def addReader(self, reader):
|
||||
fileno = reader.fileno()
|
||||
self._readers[fileno] = reader
|
||||
api.get_hub().add_descriptor(fileno, read=self._got_read)
|
||||
|
||||
def _got_read(self, fileno):
|
||||
self._readers[fileno].doRead()
|
||||
|
||||
def addWriter(self, writer):
|
||||
fileno = writer.fileno()
|
||||
self._writers[fileno] = writer
|
||||
api.get_hub().add_descriptor(fileno, write=self._got_write)
|
||||
|
||||
def _got_write(self, fileno):
|
||||
self._writers[fileno].doWrite()
|
||||
|
||||
def removeReader(self, reader):
|
||||
fileno = reader.fileno()
|
||||
if fileno in self._readers:
|
||||
self._readers.pop(fileno)
|
||||
api.get_hub().remove_descriptor(fileno)
|
||||
|
||||
def removeWriter(self, writer):
|
||||
fileno = writer.fileno()
|
||||
if fileno in self._writers:
|
||||
self._writers.pop(fileno)
|
||||
api.get_hub().remove_descriptor(fileno)
|
||||
|
||||
def removeAll(self):
|
||||
return self._removeAll(self._readers.values(), self._writers.values())
|
||||
|
||||
|
||||
def emulate():
|
||||
if not _working:
|
||||
raise RuntimeError, "Can't use twistedsupport because zope.interface is not installed."
|
||||
reactor = EventletReactor()
|
||||
from twisted.internet.main import installReactor
|
||||
installReactor(reactor)
|
||||
|
||||
|
||||
__all__ = ['emulate']
|
||||
|
214
eventlet/util.py
Normal file
214
eventlet/util.py
Normal file
@@ -0,0 +1,214 @@
|
||||
"""\
|
||||
@file util.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import os
|
||||
import fcntl
|
||||
import socket
|
||||
import errno
|
||||
from errno import EWOULDBLOCK, EAGAIN
|
||||
|
||||
try:
|
||||
from OpenSSL import SSL
|
||||
except ImportError:
|
||||
class SSL(object):
|
||||
class WantWriteError(object):
|
||||
pass
|
||||
|
||||
class WantReadError(object):
|
||||
pass
|
||||
|
||||
class ZeroReturnError(object):
|
||||
pass
|
||||
|
||||
class SysCallError(object):
|
||||
pass
|
||||
|
||||
|
||||
def g_log(*args):
|
||||
import sys
|
||||
import greenlet
|
||||
from eventlet.greenlib import greenlet_id
|
||||
g_id = greenlet_id()
|
||||
if g_id is None:
|
||||
if greenlet.getcurrent().parent is None:
|
||||
ident = 'greenlet-main'
|
||||
else:
|
||||
g_id = id(greenlet.getcurrent())
|
||||
if g_id < 0:
|
||||
g_id += 1 + ((sys.maxint + 1) << 1)
|
||||
ident = '%08X' % (g_id,)
|
||||
else:
|
||||
ident = 'greenlet-%d' % (g_id,)
|
||||
print >>sys.stderr, '[%s] %s' % (ident, ' '.join(map(str, args)))
|
||||
|
||||
CONNECT_ERR = (errno.EINPROGRESS, errno.EALREADY, EWOULDBLOCK)
|
||||
CONNECT_SUCCESS = (0, errno.EISCONN)
|
||||
def socket_connect(descriptor, address):
|
||||
err = descriptor.connect_ex(address)
|
||||
if err in CONNECT_ERR:
|
||||
return None
|
||||
if err not in CONNECT_SUCCESS:
|
||||
raise socket.error(err, errno.errorcode[err])
|
||||
return descriptor
|
||||
|
||||
__original_socket__ = socket.socket
|
||||
|
||||
def tcp_socket():
|
||||
s = __original_socket__(socket.AF_INET, socket.SOCK_STREAM)
|
||||
set_nonblocking(s)
|
||||
return s
|
||||
|
||||
|
||||
__original_ssl__ = socket.ssl
|
||||
|
||||
|
||||
def wrap_ssl(sock, certificate=None, private_key=None):
|
||||
from OpenSSL import SSL
|
||||
from eventlet import wrappedfd, util
|
||||
context = SSL.Context(SSL.SSLv23_METHOD)
|
||||
print certificate, private_key
|
||||
if certificate is not None:
|
||||
context.use_certificate_file(certificate)
|
||||
if private_key is not None:
|
||||
context.use_privatekey_file(private_key)
|
||||
context.set_verify(SSL.VERIFY_NONE, lambda *x: True)
|
||||
|
||||
## TODO only do this on client sockets? how?
|
||||
connection = SSL.Connection(context, sock)
|
||||
connection.set_connect_state()
|
||||
return wrappedfd.wrapped_fd(connection)
|
||||
|
||||
|
||||
def wrap_socket_with_coroutine_socket():
|
||||
def new_socket(*args, **kw):
|
||||
from eventlet import wrappedfd
|
||||
s = __original_socket__(*args, **kw)
|
||||
set_nonblocking(s)
|
||||
return wrappedfd.wrapped_fd(s)
|
||||
socket.socket = new_socket
|
||||
|
||||
socket.ssl = wrap_ssl
|
||||
|
||||
|
||||
def socket_bind_and_listen(descriptor, addr=('', 0), backlog=50):
|
||||
set_reuse_addr(descriptor)
|
||||
descriptor.bind(addr)
|
||||
descriptor.listen(backlog)
|
||||
return descriptor
|
||||
|
||||
def socket_accept(descriptor):
|
||||
try:
|
||||
return descriptor.accept()
|
||||
except socket.error, e:
|
||||
if e[0] == EWOULDBLOCK:
|
||||
return None
|
||||
raise
|
||||
|
||||
def socket_send(descriptor, data):
|
||||
try:
|
||||
return descriptor.send(data)
|
||||
except socket.error, e:
|
||||
if e[0] == EWOULDBLOCK:
|
||||
return 0
|
||||
raise
|
||||
except SSL.WantWriteError:
|
||||
return 0
|
||||
except SSL.WantReadError:
|
||||
return 0
|
||||
|
||||
|
||||
# winsock sometimes throws ENOTCONN
|
||||
SOCKET_CLOSED = (errno.ECONNRESET, errno.ENOTCONN, errno.ESHUTDOWN)
|
||||
def socket_recv(descriptor, buflen):
|
||||
try:
|
||||
return descriptor.recv(buflen)
|
||||
except socket.error, e:
|
||||
if e[0] == EWOULDBLOCK:
|
||||
return None
|
||||
if e[0] in SOCKET_CLOSED:
|
||||
return ''
|
||||
raise
|
||||
except SSL.WantReadError:
|
||||
return None
|
||||
except SSL.ZeroReturnError:
|
||||
return ''
|
||||
except SSL.SysCallError, e:
|
||||
if e[0] == -1 or e[0] > 0:
|
||||
raise socket.error(errno.ECONNRESET, errno.errorcode[errno.ECONNRESET])
|
||||
raise
|
||||
|
||||
|
||||
def file_recv(fd, buflen):
|
||||
try:
|
||||
return fd.read(buflen)
|
||||
except IOError, e:
|
||||
if e[0] == EAGAIN:
|
||||
return None
|
||||
return ''
|
||||
except socket.error, e:
|
||||
if e[0] == errno.EPIPE:
|
||||
return ''
|
||||
raise
|
||||
|
||||
|
||||
def file_send(fd, data):
|
||||
try:
|
||||
fd.write(data)
|
||||
fd.flush()
|
||||
return len(data)
|
||||
except IOError, e:
|
||||
if e[0] == EAGAIN:
|
||||
return 0
|
||||
except ValueError, e:
|
||||
written = 0
|
||||
except socket.error, e:
|
||||
if e[0] == errno.EPIPE:
|
||||
written = 0
|
||||
|
||||
|
||||
def set_reuse_addr(descriptor):
|
||||
try:
|
||||
descriptor.setsockopt(
|
||||
socket.SOL_SOCKET,
|
||||
socket.SO_REUSEADDR,
|
||||
descriptor.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) | 1,
|
||||
)
|
||||
except socket.error:
|
||||
pass
|
||||
|
||||
def set_nonblocking(descriptor):
|
||||
if hasattr(descriptor, 'setblocking'):
|
||||
# socket
|
||||
descriptor.setblocking(0)
|
||||
else:
|
||||
# fd
|
||||
if hasattr(descriptor, 'fileno'):
|
||||
fd = descriptor.fileno()
|
||||
else:
|
||||
fd = descriptor
|
||||
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
|
||||
fcntl.fcntl(fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
|
||||
return descriptor
|
||||
|
262
eventlet/wrappedfd.py
Normal file
262
eventlet/wrappedfd.py
Normal file
@@ -0,0 +1,262 @@
|
||||
"""\
|
||||
@file wrappedfd.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
from eventlet.api import trampoline, get_hub
|
||||
from eventlet import util
|
||||
|
||||
BUFFER_SIZE = 4096
|
||||
|
||||
import socket, errno
|
||||
|
||||
|
||||
def higher_order_recv(recv_func):
|
||||
def recv(self, buflen):
|
||||
buf = self.recvbuffer
|
||||
if buf:
|
||||
chunk, self.recvbuffer = buf[:buflen], buf[buflen:]
|
||||
return chunk
|
||||
fd = self.fd
|
||||
bytes = recv_func(fd, buflen)
|
||||
while bytes is None:
|
||||
try:
|
||||
trampoline(fd, read=True)
|
||||
except socket.error, e:
|
||||
if e[0] == errno.EPIPE:
|
||||
bytes = ''
|
||||
else:
|
||||
raise
|
||||
else:
|
||||
bytes = recv_func(fd, buflen)
|
||||
self.recvcount += len(bytes)
|
||||
return bytes
|
||||
return recv
|
||||
|
||||
|
||||
def higher_order_send(send_func):
|
||||
def send(self, data):
|
||||
count = send_func(self.fd, data)
|
||||
if not count:
|
||||
return 0
|
||||
self.sendcount += count
|
||||
return count
|
||||
return send
|
||||
|
||||
|
||||
|
||||
class wrapped_fd(object):
|
||||
newlines = '\r\n'
|
||||
mode = 'wb+'
|
||||
is_secure = False
|
||||
|
||||
def __init__(self, fd):
|
||||
self._closed = False
|
||||
self.fd = fd
|
||||
self._fileno = fd.fileno()
|
||||
self.recvbuffer = ''
|
||||
self.recvcount = 0
|
||||
self.sendcount = 0
|
||||
|
||||
def getpeername(self, *args, **kw):
|
||||
fn = self.getpeername = self.fd.getpeername
|
||||
return fn(*args, **kw)
|
||||
|
||||
def getsockname(self, *args, **kw):
|
||||
fn = self.getsockname = self.fd.getsockname
|
||||
return fn(*args, **kw)
|
||||
|
||||
def listen(self, *args, **kw):
|
||||
fn = self.listen = self.fd.listen
|
||||
return fn(*args, **kw)
|
||||
|
||||
def bind(self, *args, **kw):
|
||||
fn = self.bind = self.fd.bind
|
||||
return fn(*args, **kw)
|
||||
|
||||
def getsockopt(self, *args, **kw):
|
||||
fn = self.getsockopt = self.fd.getsockopt
|
||||
return fn(*args, **kw)
|
||||
|
||||
def setsockopt(self, *args, **kw):
|
||||
fn = self.setsockopt = self.fd.setsockopt
|
||||
return fn(*args, **kw)
|
||||
|
||||
def connect_ex(self, *args, **kw):
|
||||
fn = self.connect_ex = self.fd.connect_ex
|
||||
return fn(*args, **kw)
|
||||
|
||||
def fileno(self, *args, **kw):
|
||||
fn = self.fileno = self.fd.fileno
|
||||
return fn(*args, **kw)
|
||||
|
||||
def setblocking(self, *args, **kw):
|
||||
fn = self.setblocking = self.fd.setblocking
|
||||
return fn(*args, **kw)
|
||||
|
||||
def close(self, *args, **kw):
|
||||
if self._closed:
|
||||
return
|
||||
self._closed = True
|
||||
fn = self.close = self.fd.close
|
||||
try:
|
||||
res = fn(*args, **kw)
|
||||
finally:
|
||||
# This will raise socket.error(32, 'Broken pipe') if there's
|
||||
# a caller waiting on trampoline (e.g. server on .accept())
|
||||
get_hub().exc_descriptor(self._fileno)
|
||||
return res
|
||||
|
||||
def accept(self):
|
||||
fd = self.fd
|
||||
while True:
|
||||
res = util.socket_accept(fd)
|
||||
if res is not None:
|
||||
client, addr = res
|
||||
util.set_nonblocking(client)
|
||||
return type(self)(client), addr
|
||||
trampoline(fd, read=True, write=True)
|
||||
|
||||
def connect(self, address):
|
||||
fd = self.fd
|
||||
connect = util.socket_connect
|
||||
while not connect(fd, address):
|
||||
trampoline(fd, read=True, write=True)
|
||||
|
||||
recv = higher_order_recv(util.socket_recv)
|
||||
|
||||
def recvfrom(self, *args):
|
||||
trampoline(self.fd, read=True)
|
||||
return self.fd.recvfrom(*args)
|
||||
|
||||
send = higher_order_send(util.socket_send)
|
||||
|
||||
def sendto(self, *args):
|
||||
trampoline(self.fd, write=True)
|
||||
return self.fd.sendto(*args)
|
||||
|
||||
def sendall(self, data):
|
||||
fd = self.fd
|
||||
tail = self.send(data)
|
||||
while tail < len(data):
|
||||
trampoline(self.fd, write=True)
|
||||
tail += self.send(data[tail:])
|
||||
|
||||
def write(self, data):
|
||||
return self.sendall(data)
|
||||
|
||||
def readuntil(self, terminator, size=None):
|
||||
buf, self.recvbuffer = self.recvbuffer, ''
|
||||
checked = 0
|
||||
if size is None:
|
||||
while True:
|
||||
found = buf.find(terminator, checked)
|
||||
if found != -1:
|
||||
found += len(terminator)
|
||||
chunk, self.recvbuffer = buf[:found], buf[found:]
|
||||
return chunk
|
||||
checked = len(buf)
|
||||
d = self.recv(BUFFER_SIZE)
|
||||
if not d:
|
||||
break
|
||||
buf += d
|
||||
return buf
|
||||
while len(buf) < size:
|
||||
found = buf.find(terminator, checked)
|
||||
if found != -1:
|
||||
found += len(terminator)
|
||||
chunk, self.recvbuffer = buf[:found], buf[found:]
|
||||
return chunk
|
||||
checked = len(buf)
|
||||
d = self.recv(BUFFER_SIZE)
|
||||
if not d:
|
||||
break
|
||||
buf += d
|
||||
chunk, self.recvbuffer = buf[:size], buf[size:]
|
||||
|
||||
def readline(self, size=None):
|
||||
return self.readuntil(self.newlines, size=size)
|
||||
|
||||
def __iter__(self):
|
||||
return self.xreadlines()
|
||||
|
||||
def readlines(self, size=None):
|
||||
return list(self.xreadlines(size=size))
|
||||
|
||||
def xreadlines(self, size=None):
|
||||
if size is None:
|
||||
while True:
|
||||
line = self.readline()
|
||||
if not line:
|
||||
break
|
||||
yield line
|
||||
else:
|
||||
while size > 0:
|
||||
line = self.readline(size)
|
||||
if not line:
|
||||
break
|
||||
yield line
|
||||
size -= len(line)
|
||||
|
||||
def writelines(self, lines):
|
||||
for line in lines:
|
||||
self.write(line)
|
||||
|
||||
def read(self, size=None):
|
||||
if size is not None and not isinstance(size, int):
|
||||
raise TypeError
|
||||
buf, self.recvbuffer = self.recvbuffer, ''
|
||||
lst = [buf]
|
||||
if size is None:
|
||||
while True:
|
||||
d = self.recv(BUFFER_SIZE)
|
||||
if not d:
|
||||
break
|
||||
lst.append(d)
|
||||
else:
|
||||
buflen = len(buf)
|
||||
while buflen < size:
|
||||
d = self.recv(BUFFER_SIZE)
|
||||
if not d:
|
||||
break
|
||||
buflen += len(d)
|
||||
lst.append(d)
|
||||
else:
|
||||
d = lst[-1]
|
||||
overbite = buflen - size
|
||||
if overbite:
|
||||
lst[-1], self.recvbuffer = d[:-overbite], d[-overbite:]
|
||||
else:
|
||||
lst[-1], self.recvbuffer = d, ''
|
||||
return ''.join(lst)
|
||||
|
||||
def makefile(self, *args, **kw):
|
||||
return type(self)(self.fd)
|
||||
|
||||
|
||||
class wrapped_file(wrapped_fd):
|
||||
recv = higher_order_recv(util.file_recv)
|
||||
|
||||
send = higher_order_send(util.file_send)
|
||||
|
||||
def flush(self):
|
||||
self.fd.flush()
|
219
eventlet/wsgi.py
Normal file
219
eventlet/wsgi.py
Normal file
@@ -0,0 +1,219 @@
|
||||
"""\
|
||||
@file wsgi.py
|
||||
@author Bob Ippolito
|
||||
|
||||
Copyright (c) 2005-2006, Bob Ippolito
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import time
|
||||
import urllib
|
||||
import socket
|
||||
import cStringIO
|
||||
import SocketServer
|
||||
import BaseHTTPServer
|
||||
|
||||
from eventlet import api
|
||||
from eventlet.httpdate import format_date_time
|
||||
|
||||
class HttpProtocol(BaseHTTPServer.BaseHTTPRequestHandler):
|
||||
|
||||
def log_message(self, format, *args):
|
||||
self.server.log_message("%s - - [%s] %s" % (
|
||||
self.address_string(),
|
||||
self.log_date_time_string(),
|
||||
format % args))
|
||||
|
||||
def handle_one_request(self):
|
||||
self.raw_requestline = self.rfile.readline()
|
||||
|
||||
if not self.raw_requestline:
|
||||
self.close_connection = 1
|
||||
return
|
||||
|
||||
if not self.parse_request():
|
||||
return
|
||||
|
||||
self.environ = self.get_environ()
|
||||
try:
|
||||
self.handle_one_response()
|
||||
except socket.error, e:
|
||||
# Broken pipe, connection reset by peer
|
||||
if e[0] in (32, 54):
|
||||
pass
|
||||
else:
|
||||
raise
|
||||
|
||||
def handle_one_response(self):
|
||||
headers_set = []
|
||||
headers_sent = []
|
||||
# set of lowercase header names that were sent
|
||||
header_dict = {}
|
||||
|
||||
wfile = self.wfile
|
||||
num_blocks = None
|
||||
|
||||
def write(data, _write=wfile.write):
|
||||
if not headers_set:
|
||||
raise AssertionError("write() before start_response()")
|
||||
elif not headers_sent:
|
||||
status, response_headers = headers_set
|
||||
headers_sent.append(1)
|
||||
for k, v in response_headers:
|
||||
header_dict[k.lower()] = k
|
||||
_write('HTTP/1.0 %s\r\n' % status)
|
||||
# send Date header?
|
||||
if 'date' not in header_dict:
|
||||
_write('Date: %s\r\n' % (format_date_time(time.time()),))
|
||||
if 'content-length' not in header_dict and num_blocks == 1:
|
||||
_write('Content-Length: %s\r\n' % (len(data),))
|
||||
for header in response_headers:
|
||||
_write('%s: %s\r\n' % header)
|
||||
_write('\r\n')
|
||||
_write(data)
|
||||
|
||||
def start_request(status, response_headers, exc_info=None):
|
||||
if exc_info:
|
||||
try:
|
||||
if headers_sent:
|
||||
# Re-raise original exception if headers sent
|
||||
raise exc_info[0], exc_info[1], exc_info[2]
|
||||
finally:
|
||||
# Avoid dangling circular ref
|
||||
exc_info = None
|
||||
elif headers_set:
|
||||
raise AssertionError("Headers already set!")
|
||||
|
||||
headers_set[:] = [status, response_headers]
|
||||
return write
|
||||
|
||||
result = self.server.app(self.environ, start_request)
|
||||
try:
|
||||
num_blocks = len(result)
|
||||
except (TypeError, AttributeError, NotImplementedError):
|
||||
pass
|
||||
|
||||
try:
|
||||
for data in result:
|
||||
if data:
|
||||
write(data)
|
||||
if not headers_sent:
|
||||
write('')
|
||||
finally:
|
||||
if hasattr(result, 'close'):
|
||||
result.close()
|
||||
|
||||
def get_environ(self):
|
||||
env = self.server.get_environ()
|
||||
env['REQUEST_METHOD'] = self.command
|
||||
env['SCRIPT_NAME'] = ''
|
||||
|
||||
if '?' in self.path:
|
||||
path, query = self.path.split('?', 1)
|
||||
else:
|
||||
path, query = self.path, ''
|
||||
env['PATH_INFO'] = urllib.unquote(path)
|
||||
env['QUERY_STRING'] = query
|
||||
|
||||
if self.headers.typeheader is None:
|
||||
env['CONTENT_TYPE'] = self.headers.type
|
||||
else:
|
||||
env['CONTENT_TYPE'] = self.headers.typeheader
|
||||
|
||||
length = self.headers.getheader('content-length')
|
||||
if length:
|
||||
env['CONTENT_LENGTH'] = length
|
||||
env['SERVER_PROTOCOL'] = 'HTTP/1.0'
|
||||
|
||||
host, port = self.request.getsockname()
|
||||
env['SERVER_NAME'] = host
|
||||
env['SERVER_PORT'] = str(port)
|
||||
env['REMOTE_ADDR'] = self.client_address[0]
|
||||
env['GATEWAY_INTERFACE'] = 'CGI/1.1'
|
||||
|
||||
for h in self.headers.headers:
|
||||
k, v = h.split(':', 1)
|
||||
k = k.replace('-', '_').upper()
|
||||
v = v.strip()
|
||||
if k in env:
|
||||
continue
|
||||
envk = 'HTTP_' + k
|
||||
if envk in env:
|
||||
env[envk] += ',' + v
|
||||
else:
|
||||
env[envk] = v
|
||||
|
||||
return env
|
||||
|
||||
def finish(self):
|
||||
# Override SocketServer.StreamRequestHandler.finish because
|
||||
# we only need to call close on the socket, not the makefile'd things
|
||||
|
||||
self.request.close()
|
||||
|
||||
|
||||
class Server(BaseHTTPServer.HTTPServer):
|
||||
def __init__(self, socket, address, app, log, environ=None):
|
||||
self.socket = socket
|
||||
self.address = address
|
||||
if log:
|
||||
self.log = log
|
||||
log.write = log.info
|
||||
else:
|
||||
self.log = sys.stderr
|
||||
self.app = app
|
||||
self.environ = environ
|
||||
|
||||
def get_environ(self):
|
||||
socket = self.socket
|
||||
d = {
|
||||
'wsgi.input': socket,
|
||||
'wsgi.errors': sys.stderr,
|
||||
'wsgi.version': (1, 0),
|
||||
'wsgi.multithread': True,
|
||||
'wsgi.multiprocess': False,
|
||||
'wsgi.run_once': False,
|
||||
'wsgi.url_scheme': 'http',
|
||||
}
|
||||
if self.environ is not None:
|
||||
d.update(self.environ)
|
||||
return d
|
||||
|
||||
def process_request(self, (socket, address)):
|
||||
proto = HttpProtocol(socket, address, self)
|
||||
|
||||
def log_message(self, message):
|
||||
self.log.write(message + '\n')
|
||||
|
||||
|
||||
def server(socket, site, log=None, environ=None):
|
||||
serv = Server(socket, socket.getsockname(), site, log, environ=None)
|
||||
try:
|
||||
print "wsgi starting up on", socket.getsockname()
|
||||
while True:
|
||||
try:
|
||||
api.spawn(serv.process_request, socket.accept())
|
||||
except KeyboardInterrupt:
|
||||
api.get_hub().remove_descriptor(socket.fileno())
|
||||
print "wsgi exiting"
|
||||
break
|
||||
finally:
|
||||
socket.close()
|
52
examples/echoserver.py
Normal file
52
examples/echoserver.py
Normal file
@@ -0,0 +1,52 @@
|
||||
"""\
|
||||
@file echoserver.py
|
||||
|
||||
Simple server that listens on port 6000 and echos back every input to
|
||||
the client. To try out the server, start it up by running this file.
|
||||
|
||||
Connect to it with:
|
||||
telnet localhost 6000
|
||||
|
||||
You terminate your connection by terminating telnet (typically Ctrl-]
|
||||
and then 'quit')
|
||||
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
from eventlet import api
|
||||
|
||||
def handle_socket(client):
|
||||
print "client connected"
|
||||
while True:
|
||||
# pass through every non-eof line
|
||||
x = client.readline()
|
||||
if not x: break
|
||||
client.write(x)
|
||||
print "echoed", x
|
||||
print "client disconnected"
|
||||
|
||||
# server socket listening on port 6000
|
||||
server = api.tcp_listener(('0.0.0.0', 6000))
|
||||
while True:
|
||||
new_sock, address = server.accept()
|
||||
# handle every new connection with a new coroutine
|
||||
api.spawn(handle_socket, new_sock)
|
||||
|
||||
server.close()
|
55
examples/webcrawler.py
Normal file
55
examples/webcrawler.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""\
|
||||
@file webcrawler.py
|
||||
|
||||
This is a simple web "crawler" that fetches a bunch of urls using a coroutine pool. It fetches as
|
||||
many urls at time as coroutines in the pool.
|
||||
|
||||
Copyright (c) 2007, Linden Research, Inc.
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in
|
||||
all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
"""
|
||||
|
||||
urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
|
||||
"http://wiki.secondlife.com/w/images/secondlife.jpg",
|
||||
"http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
|
||||
|
||||
import time
|
||||
from eventlet import coros, httpc, util
|
||||
|
||||
# replace socket with a cooperative coroutine socket because httpc
|
||||
# uses httplib, which uses socket. Removing this serializes the http
|
||||
# requests, because the standard socket is blocking.
|
||||
util.wrap_socket_with_coroutine_socket()
|
||||
|
||||
def fetch(url):
|
||||
# we could do something interesting with the result, but this is
|
||||
# example code, so we'll just report that we did it
|
||||
print "%s fetching %s" % (time.asctime(), url)
|
||||
httpc.get(url)
|
||||
print "%s fetched %s" % (time.asctime(), url)
|
||||
|
||||
pool = coros.CoroutinePool(max_size=4)
|
||||
waiters = []
|
||||
for url in urls:
|
||||
waiters.append(pool.execute(fetch, url))
|
||||
|
||||
# wait for all the coroutines to come back before exiting the process
|
||||
for waiter in waiters:
|
||||
waiter.wait()
|
||||
|
||||
|
Reference in New Issue
Block a user