Merge pull request #1 from liris/master

Updating master
This commit is contained in:
krosinski
2015-01-08 09:46:09 +01:00
19 changed files with 776 additions and 210 deletions

133
ChangeLog Normal file
View File

@@ -0,0 +1,133 @@
ChangeLog
============
- 0.23.0
- Remove spurious print statement. (#135)
- 0.22.0
- Fix not thread-safe of Websocket.close() (#120)
- Try to get proxy info from environment if not explicitly provided (#124)
- support proxy basic authenticaiton. (#125)
- Fix NoneType exception at WebsocketApp.send (#126)
- not use proxy for localhost (#132)
- 0.21.0
- Check for socket before attempting to close (#115)
- Enable turning off SSL verification in wsdump.py(#116)
- Enable to set subprotocol(#118)
- Better support for Autobahn test suite (http://autobahn.ws/testsuite) (#117)
- v0.20.0
- fix typo.
- v0.19.0
- suppress close event message(#107)
- detect socket connection state(#109)
- support for code and reason in on_close callback(#111)
- continuation frame handling seems suspicious(#113)
- v0.18.0
- allow override of match_hostname usage on ssl (#105)
- v0.17.0
- can't set timeout on a standing websocket connection (#102)
- fixed local variable 'error' referenced before assignment (#102, #98)
- v0.16.0
- lock some method for multithread. (#92)
- disable cert verification. (#89)
- v0.15.0
- fixed exception when send a large message (#84)
- v0.14.1
- fixed to work on Python2.6 (#83)
- v0.14.0
- Support python 3(#73)
- Support IPv6(#77)
- Support explicit web proxy(#57)
- specify cookie in connect method option(#82)
- v0.13.0
- MemoryError when receiving large amount of data (~60 MB) at once(ISSUE#59)
- Controlling fragmentation(ISSUE#55)
- server certificate validation(ISSUE#56)
- PyPI tarball is missing test_websocket.py(ISSUE#65)
- Payload length encoding bug(ISSUE#58)
- disable Nagle algorithm by default(ISSUE#41)
- Better event loop in WebSocketApp(ISSUE#63)
- Skip tests that require Internet access by default(ISSUE#66)
- v0.12.0
- support keep alive for WebSocketApp(ISSUE#34)
- fix some SSL bugs(ISSUE#35, #36)
- fix "Timing out leaves websocket library in bad state"(ISSUE#37)
- fix "WebSocketApp.run_with_no_err() silently eats all exceptions"(ISSUE#38)
- WebSocketTimeoutException will be raised for ws/wss timeout(ISSUE#40)
- improve wsdump message(ISSUE#42)
- support fragmentation message(ISSUE#43)
- fix some bugs
- v0.11.0
- Only log non-normal close status(ISSUE#31)
- Fix default Origin isn't URI(ISSUE#32)
- fileno support(ISSUE#33)
- v0.10.0
- allow to set HTTP Header to WebSocketApp(ISSUE#27)
- fix typo in pydoc(ISSUE#28)
- Passing a socketopt flag to the websocket constructor(ISSUE#29)
- websocket.send fails with long data(ISSUE#30)
- v0.9.0
- allow to set opcode in WebSocketApp.send(ISSUE#25)
- allow to modify Origin(ISSUE#26)
- v0.8.0
- many bug fix
- some performance improvement
- v0.7.0
- fixed problem to read long data.(ISSUE#12)
- fix buffer size boundary violation
- v0.6.0
- Patches: UUID4, self.keep_running, mask_key (ISSUE#11)
- add wsdump.py tool
- v0.5.2
- fix Echo App Demo Throw Error: 'NoneType' object has no attribute 'opcode (ISSUE#10)
- v0.5.1
- delete invalid print statement.
- v0.5.0
- support hybi-13 protocol.
- v0.4.1
- fix incorrect custom header order(ISSUE#1)

View File

@@ -1,2 +1,4 @@
include LICENSE
include README.rst
include ChangeLog
recursive-include examples *

View File

@@ -6,12 +6,6 @@ websocket-client module is WebSocket client for python. This provide the low le
websocket-client supports only hybi-13.
CAUTION
============
We have a big change on version 0.14.0.
So, please test carefully.
License
============
@@ -50,10 +44,10 @@ Current implementation of websocket-client is using "CONNECT" method via proxy.
example::
-------------
import websocket
ws = websocket.WebSocket(support_socket_io="0.9")
ws = websocket.WebSocket()
ws.connect("ws://example.com/websocket", http_proxy_host="proxy_host_name", http_proxy_port=3128)
:
@@ -75,11 +69,11 @@ Low Level API example::
If you want to customize socket options, set sockopt.
sockopt example:
sockopt example::
from websocket import create_connection
ws = create_connection("ws://echo.websocket.org/".
sockopt=((socket.IPPROTO_TCP, socket.TCP_NODELAY),) )
ws = create_connection("ws://echo.websocket.org/",
sockopt=((socket.IPPROTO_TCP, socket.TCP_NODELAY),))
JavaScript websocket-like API example::
@@ -118,6 +112,64 @@ JavaScript websocket-like API example::
ws.run_forever()
FAQ
============
How to disable ssl cert verification?
----------------------------------------
Please set sslopt to {"cert_reqs": ssl.CERT_NONE}.
WebSocketApp sample::
ws = websocket.WebSocketApp("https://echo.websocket.org")
ws.run_forever(sslopt={"cert_reqs": ssl.CERT_NONE})
create_connection sample::
ws = websocket.create_connection("https://echo.websocket.org",
sslopt={"cert_reqs": ssl.CERT_NONE})
WebSocket sample::
ws = websocket.WebSocket(sslopt={"cert_reqs": ssl.CERT_NONE})
ws.connect("https://echo.websocket.org")
How to disable hostname verification.
----------------------------------------
Please set sslopt to {"check_hostname": False}.
(since v0.18.0)
WebSocketApp sample::
ws = websocket.WebSocketApp("https://echo.websocket.org")
ws.run_forever(sslopt={"check_hostname": False})
create_connection sample::
ws = websocket.create_connection("https://echo.websocket.org",
sslopt={"check_hostname": False})
WebSocket sample::
ws = websocket.WebSocket(sslopt={"check_hostname": False})
ws.connect("https://echo.websocket.org")
Sub Protocols.
----------------------------------------
The server needs to support sub protocols, please set the subprotcol like this.
Subprotocol sample::
ws = websocket.create_connection("ws://exapmle.com/websocket", subprotocols=["binary", "base64"])
wsdump.py
============
@@ -154,98 +206,3 @@ example::
$ wsdump.py ws://echo.websocket.org/ -v
$ wsdump.py ws://echo.websocket.org/ -vv
ChangeLog
============
- v0.16.0
- lock some method for multithread. (#92)
- disable cert verification. (#89)
- v0.15.0
- fixed exception when send a large message (#84)
- v0.14.1
- fixed to work on Python2.6 (#83)
- v0.14.0
- Support python 3(#73)
- Support IPv6(#77)
- Support explicit web proxy(#57)
- specify cookie in connect method option(#82)
- v0.13.0
- MemoryError when receiving large amount of data (~60 MB) at once(ISSUE#59)
- Controlling fragmentation(ISSUE#55)
- server certificate validation(ISSUE#56)
- PyPI tarball is missing test_websocket.py(ISSUE#65)
- Payload length encoding bug(ISSUE#58)
- disable Nagle algorithm by default(ISSUE#41)
- Better event loop in WebSocketApp(ISSUE#63)
- Skip tests that require Internet access by default(ISSUE#66)
- v0.12.0
- support keep alive for WebSocketApp(ISSUE#34)
- fix some SSL bugs(ISSUE#35, #36)
- fix "Timing out leaves websocket library in bad state"(ISSUE#37)
- fix "WebSocketApp.run_with_no_err() silently eats all exceptions"(ISSUE#38)
- WebSocketTimeoutException will be raised for ws/wss timeout(ISSUE#40)
- improve wsdump message(ISSUE#42)
- support fragmentation message(ISSUE#43)
- fix some bugs
- v0.11.0
- Only log non-normal close status(ISSUE#31)
- Fix default Origin isn't URI(ISSUE#32)
- fileno support(ISSUE#33)
- v0.10.0
- allow to set HTTP Header to WebSocketApp(ISSUE#27)
- fix typo in pydoc(ISSUE#28)
- Passing a socketopt flag to the websocket constructor(ISSUE#29)
- websocket.send fails with long data(ISSUE#30)
- v0.9.0
- allow to set opcode in WebSocketApp.send(ISSUE#25)
- allow to modify Origin(ISSUE#26)
- v0.8.0
- many bug fix
- some performance improvement
- v0.7.0
- fixed problem to read long data.(ISSUE#12)
- fix buffer size boundary violation
- v0.6.0
- Patches: UUID4, self.keep_running, mask_key (ISSUE#11)
- add wsdump.py tool
- v0.5.2
- fix Echo App Demo Throw Error: 'NoneType' object has no attribute 'opcode (ISSUE#10)
- v0.5.1
- delete invalid print statement.
- v0.5.0
- support hybi-13 protocol.
- v0.4.1
- fix incorrect custom header order(ISSUE#1)

View File

@@ -33,6 +33,8 @@ def parse_args():
dest="verbose",
help="set verbose mode. If set to 1, show opcode. "
"If set to 2, enable to trace websocket module")
parser.add_argument("-n", "--nocert", action='store_true',
help="Ignore invalid SSL cert")
return parser.parse_args()
@@ -64,7 +66,10 @@ def main():
console = InteractiveConsole()
if args.verbose > 1:
websocket.enableTrace(True)
ws = websocket.create_connection(args.url)
opts = {}
if (args.nocert):
opts = { "cert_reqs": websocket.ssl.CERT_NONE, "check_hostname": False }
ws = websocket.create_connection(args.url, sslopt=opts)
print("Press Ctrl+C to quit")
def recv():

11
compliance/README.md Normal file
View File

@@ -0,0 +1,11 @@
# Autobahn Testsuite
General information and installation instructions are available at http://autobahn.ws/testsuite .
## Running the test suite
$ wstest -m fuzzingserver
$ python test_fuzzingclient.py

View File

@@ -0,0 +1,9 @@
{
"url": "ws://localhost:8642",
"options": {"failByDrop": false},
"outdir": "./reports/clients",
"webport": 8080,
"cases": ["*"],
"exclude-cases": [],
"exclude-agent-cases": {}
}

View File

@@ -0,0 +1,47 @@
#!/usr/bin/env python
import websocket
import json
import traceback
import six
SERVER = 'ws://127.0.0.1:8642'
AGENT = 'py-websockets-cleint'
ws = websocket.create_connection(SERVER + "/getCaseCount")
count = json.loads(ws.recv())
ws.close()
for case in range(1, count+1):
url = SERVER + '/runCase?case={}&agent={}'.format(case, AGENT)
status = websocket.STATUS_NORMAL
try:
ws = websocket.create_connection(url)
while True:
opcode, msg = ws.recv_data()
if opcode == websocket.ABNF.OPCODE_TEXT:
msg.decode("utf-8")
if opcode in (websocket.ABNF.OPCODE_TEXT, websocket.ABNF.OPCODE_BINARY):
ws.send(msg, opcode)
except UnicodeDecodeError:
# this case is ok.
status = websocket.STATUS_PROTOCOL_ERROR
except websocket.WebSocketProtocolException:
status = websocket.STATUS_PROTOCOL_ERROR
except websocket.WebSocketPayloadException:
status = websocket.STATUS_INVALID_PAYLOAD
except Exception as e:
# status = websocket.STATUS_PROTOCOL_ERROR
print(traceback.format_exc())
finally:
ws.close(status)
print("Ran {} test cases.".format(case))
url = SERVER + '/updateReports?agent={}'.format(AGENT)
ws = websocket.create_connection(url)

View File

@@ -1,5 +1,8 @@
import websocket
import thread
try:
import thread
except ImportError: #TODO use Threading instead of _thread in python3
import _thread as thread
import time
import sys

View File

@@ -1,7 +1,7 @@
from setuptools import setup
import sys
VERSION = "0.16.0a"
VERSION = "0.23.0"
NAME="websocket-client"
install_requires = ["six"]
@@ -36,9 +36,10 @@ setup(
keywords='websockets',
scripts=["bin/wsdump.py"],
install_requires=install_requires,
packages=["tests", "websocket"],
packages=["websocket", "websocket.tests"],
package_data={
'tests': ['data/*.txt'],
'websocket.tests': ['data/*.txt'],
'websocket': ["cacert.pem"]
},
test_suite = "websocket.tests.test_websocket",
)

View File

@@ -20,3 +20,5 @@ Copyright (C) 2010 Hiroki Ohtani(liris)
"""
from ._core import *
from ._app import WebSocketApp
__version__ = "0.23.0"

View File

@@ -22,9 +22,34 @@ import six
import array
import struct
import os
from ._exceptions import *
from ._utils import validate_utf8
# closing frame status codes.
STATUS_NORMAL = 1000
STATUS_GOING_AWAY = 1001
STATUS_PROTOCOL_ERROR = 1002
STATUS_UNSUPPORTED_DATA_TYPE = 1003
STATUS_STATUS_NOT_AVAILABLE = 1005
STATUS_ABNORMAL_CLOSED = 1006
STATUS_INVALID_PAYLOAD = 1007
STATUS_POLICY_VIOLATION = 1008
STATUS_MESSAGE_TOO_BIG = 1009
STATUS_INVALID_EXTENSION = 1010
STATUS_UNEXPECTED_CONDITION = 1011
STATUS_TLS_HANDSHAKE_ERROR = 1015
VALID_CLOSE_STATUS = (
STATUS_NORMAL,
STATUS_GOING_AWAY,
STATUS_PROTOCOL_ERROR,
STATUS_UNSUPPORTED_DATA_TYPE,
STATUS_INVALID_PAYLOAD,
STATUS_POLICY_VIOLATION,
STATUS_MESSAGE_TOO_BIG,
STATUS_INVALID_EXTENSION,
STATUS_UNEXPECTED_CONDITION,
)
class ABNF(object):
"""
@@ -75,6 +100,35 @@ class ABNF(object):
self.data = data
self.get_mask_key = os.urandom
def validate(self):
"""
validate the ABNF frame.
"""
if self.rsv1 or self.rsv2 or self.rsv3:
raise WebSocketProtocolException("rsv is not implemented, yet")
if self.opcode not in ABNF.OPCODES:
raise WebSocketProtocolException("Invalid opcode " + self.opcode)
if self.opcode == ABNF.OPCODE_PING and not self.fin:
raise WebSocketProtocolException("Invalid ping frame.")
if self.opcode == ABNF.OPCODE_CLOSE:
l = len(self.data)
if not l:
return
if l == 1 or l >= 126:
raise WebSocketProtocolException("Invalid close frame.")
if l > 2 and not validate_utf8(self.data[2:]):
raise WebSocketProtocolException("Invalid close frame.")
code = 256*six.byte2int(self.data[0:1]) + six.byte2int(self.data[1:2])
if not self._is_valid_close_status(code):
raise WebSocketProtocolException("Invalid close opcode.")
def _is_valid_close_status(self, code):
return code in VALID_CLOSE_STATUS or (3000 <= code <5000)
def __str__(self):
return "fin=" + str(self.fin) \
+ " opcode=" + str(self.opcode) \

View File

@@ -28,7 +28,7 @@ import traceback
import sys
import select
import six
import logging
from ._core import WebSocket, getdefaulttimeout, logger
from ._exceptions import *
@@ -43,7 +43,8 @@ class WebSocketApp(object):
on_open=None, on_message=None, on_error=None,
on_close=None, on_ping=None, on_pong=None,
on_cont_message=None,
keep_running=True, get_mask_key=None, cookie=None):
keep_running=True, get_mask_key=None, cookie=None,
subprotocols=None):
"""
url: websocket url.
header: custom header for websocket handshake.
@@ -68,6 +69,7 @@ class WebSocketApp(object):
keep running, defaults to True
get_mask_key: a callable to produce new mask keys, see the WebSocket.set_mask_key's
docstring for more information
subprotocols: array of available sub protocols. default is None.
"""
self.url = url
self.header = header
@@ -83,6 +85,7 @@ class WebSocketApp(object):
self.get_mask_key = get_mask_key
self.sock = None
self.last_ping_tm = 0
self.subprotocols =subprotocols
def send(self, data, opcode=ABNF.OPCODE_TEXT):
"""
@@ -91,7 +94,7 @@ class WebSocketApp(object):
opcode: operation code of data. default is OPCODE_TEXT.
"""
if self.sock.send(data, opcode) == 0:
if not self.sock or self.sock.send(data, opcode) == 0:
raise WebSocketConnectionClosedException()
def close(self):
@@ -105,10 +108,11 @@ class WebSocketApp(object):
def _send_ping(self, interval, event):
while not event.wait(interval):
self.last_ping_tm = time.time()
if self.sock:
self.sock.ping()
def run_forever(self, sockopt=None, sslopt=None, ping_interval=0, ping_timeout=None,
http_proxy_host=None, http_proxy_port=None):
http_proxy_host=None, http_proxy_port=None, http_no_proxy=None):
"""
run event loop for WebSocket framework.
This loop is infinite loop and is alive during websocket is available.
@@ -120,6 +124,7 @@ class WebSocketApp(object):
ping_timeout: timeout(second) if the pong message is not recieved.
http_proxy_host: http proxy host name.
http_proxy_port: http proxy port. If not set, set to 80.
http_no_proxy: host names, which doesn't use proxy.
"""
if not ping_timeout or ping_timeout<=0:
@@ -131,13 +136,16 @@ class WebSocketApp(object):
if self.sock:
raise WebSocketException("socket is already opened")
thread = None
close_frame = None
try:
self.sock = WebSocket(self.get_mask_key, sockopt=sockopt, sslopt=sslopt,
fire_cont_frame=self.on_cont_message and True or False)
self.sock.settimeout(getdefaulttimeout())
self.sock.connect(self.url, header=self.header, cookie=self.cookie,
http_proxy_host=http_proxy_host, http_proxy_port=http_proxy_port)
http_proxy_host=http_proxy_host, http_proxy_port=http_proxy_port,
http_no_proxy = http_no_proxy,
subprotocols=self.subprotocols)
self._callback(self.on_open)
if ping_interval:
@@ -157,6 +165,7 @@ class WebSocketApp(object):
if r:
op_code, frame = self.sock.recv_data_frame(True)
if op_code == ABNF.OPCODE_CLOSE:
close_frame = frame
break
elif op_code == ABNF.OPCODE_PING:
self._callback(self.on_ping, frame.data)
@@ -177,9 +186,25 @@ class WebSocketApp(object):
thread.join()
self.keep_running = False
self.sock.close()
self._callback(self.on_close)
self._callback(self.on_close,
*self._get_close_args(close_frame.data if close_frame else None))
self.sock = None
def _get_close_args(self,data):
""" this functions extracts the code, reason from the close body
if they exists, and if the self.on_close except three arguments """
import inspect
# if the on_close callback is "old", just return empty list
if not self.on_close or len(inspect.getargspec(self.on_close).args) != 3:
return []
if data and len(data) >=2:
code = 256*six.byte2int(data[0:1]) + six.byte2int(data[1:2])
reason = data[2:].decode('utf-8')
return [code,reason]
return [None,None]
def _callback(self, callback, *args):
if callback:
try:

View File

@@ -56,8 +56,8 @@ import logging
# websocket modules
from ._exceptions import *
from ._abnf import ABNF
from ._utils import NoLock
from ._abnf import *
from ._utils import NoLock, validate_utf8
"""
websocket python client.
@@ -71,19 +71,6 @@ Please see http://tools.ietf.org/html/rfc6455 for protocol.
# websocket supported version.
VERSION = 13
# closing frame status codes.
STATUS_NORMAL = 1000
STATUS_GOING_AWAY = 1001
STATUS_PROTOCOL_ERROR = 1002
STATUS_UNSUPPORTED_DATA_TYPE = 1003
STATUS_STATUS_NOT_AVAILABLE = 1005
STATUS_ABNORMAL_CLOSED = 1006
STATUS_INVALID_PAYLOAD = 1007
STATUS_POLICY_VIOLATION = 1008
STATUS_MESSAGE_TOO_BIG = 1009
STATUS_INVALID_EXTENSION = 1010
STATUS_UNEXPECTED_CONDITION = 1011
STATUS_TLS_HANDSHAKE_ERROR = 1015
DEFAULT_SOCKET_OPTION = [(socket.SOL_TCP, socket.TCP_NODELAY, 1),]
if hasattr(socket, "SO_KEEPALIVE"):
@@ -182,6 +169,62 @@ def _parse_url(url):
return (hostname, port, resource, is_secure)
DEFAULT_NO_PROXY_HOST = ["localhost", "127.0.0.1"]
def _is_no_proxy_host(hostname, no_proxy):
if not no_proxy:
v = os.environ.get("no_proxy", "").replace(" ", "")
no_proxy = v.split(",")
if not no_proxy:
no_proxy = DEFAULT_NO_PROXY_HOST
return hostname in no_proxy
def _get_proxy_info(hostname, is_secure, **options):
"""
try to retrieve proxy host and port from environment if not provided in options.
result is (proxy_host, proxy_port, proxy_auth).
proxy_auth is tuple of username and password of proxy authentication information.
hostname: websocket server name.
is_secure: is the connection secure? (wss)
looks for "https_proxy" in env before falling back to "http_proxy"
options: "http_proxy_host" - http proxy host name.
"http_proxy_port" - http proxy port.
"http_no_proxy" - host names, which doesn't use proxy.
"http_proxy_auth" - http proxy auth infomation. tuple of username and password.
defualt is None
"""
if _is_no_proxy_host(hostname, options.get("http_no_proxy", None)):
return None, 0, None
http_proxy_host = options.get("http_proxy_host", None)
if http_proxy_host:
return http_proxy_host, options.get("http_proxy_port", 0), options.get("http_proxy_auth", None)
env_keys = ["http_proxy"]
if is_secure:
env_keys.insert(0, "https_proxy")
for key in env_keys:
value = os.environ.get(key, None)
if value:
proxy = urlparse(value)
auth = (proxy.username, proxy.password) if proxy.username else None
return proxy.hostname, proxy.port, auth
return None, 0, None
def _extract_err_message(exception):
message = getattr(exception, 'strerror', '')
if not message:
message = getattr(exception, 'message', '')
return message
def create_connection(url, timeout=None, **options):
"""
connect to url and return websocket object.
@@ -205,7 +248,13 @@ def create_connection(url, timeout=None, **options):
"cookie" -> cookie value.
"http_proxy_host" - http proxy host name.
"http_proxy_port" - http proxy port. If not set, set to 80.
"http_no_proxy" - host names, which doesn't use proxy.
"http_proxy_auth" - http proxy auth infomation. tuple of username and password.
defualt is None
"enable_multithread" -> enable lock for multithread.
"sockopt" -> socket options
"sslopt" -> ssl option
"subprotocols" - array of available sub protocols. default is None.
"""
sockopt = options.get("sockopt", [])
sslopt = options.get("sslopt", {})
@@ -350,11 +399,14 @@ class WebSocket(object):
# These buffer over the build-up of a single frame.
self._frame_buffer = _FrameBuffer()
self._cont_data = None
self._recving_frames = None
if enable_multithread:
self.lock = threading.Lock()
else:
self.lock = NoLock()
self.subprotocol = None
def fileno(self):
return self.sock.fileno()
@@ -383,6 +435,8 @@ class WebSocket(object):
timeout: timeout time(second).
"""
self._timeout = timeout
if self.sock:
self.sock.settimeout(timeout)
timeout = property(gettimeout, settimeout)
@@ -405,11 +459,15 @@ class WebSocket(object):
"cookie" -> cookie value.
"http_proxy_host" - http proxy host name.
"http_proxy_port" - http proxy port. If not set, set to 80.
"http_no_proxy" - host names, which doesn't use proxy.
"http_proxy_auth" - http proxy auth infomation. tuple of username and password.
defualt is None
"subprotocols" - array of available sub protocols. default is None.
"""
hostname, port, resource, is_secure = _parse_url(url)
proxy_host, proxy_port = options.get("http_proxy_host", None), options.get("http_proxy_port", 0)
proxy_host, proxy_port, proxy_auth = _get_proxy_info(hostname, is_secure, **options)
if not proxy_host:
addrinfo_list = socket.getaddrinfo(hostname, port, 0, 0, socket.SOL_TCP)
else:
@@ -419,6 +477,7 @@ class WebSocket(object):
if not addrinfo_list:
raise WebSocketException("Host not found.: " + hostname + ":" + str(port))
err = None
for addrinfo in addrinfo_list:
family = addrinfo[0]
self.sock = socket.socket(family)
@@ -427,22 +486,24 @@ class WebSocket(object):
self.sock.setsockopt(*opts)
for opts in self.sockopt:
self.sock.setsockopt(*opts)
# TODO: we need to support proxy
address = addrinfo[4]
try:
self.sock.connect(address)
except socket.error as error:
error.remote_ip = str(address[0])
if error.errno in (errno.ECONNREFUSED, ):
err = error
continue
else:
raise
else:
break
else:
raise error
raise err
if proxy_host:
self._tunnel(hostname, port)
self._tunnel(hostname, port, proxy_auth)
if is_secure:
if HAVE_SSL:
@@ -452,17 +513,26 @@ class WebSocket(object):
if os.path.isfile(certPath):
sslopt['ca_certs'] = certPath
sslopt.update(self.sslopt)
check_hostname = sslopt.pop('check_hostname', True)
self.sock = ssl.wrap_socket(self.sock, **sslopt)
if sslopt["cert_reqs"] != ssl.CERT_NONE:
if (sslopt["cert_reqs"] != ssl.CERT_NONE
and check_hostname):
match_hostname(self.sock.getpeercert(), hostname)
else:
raise WebSocketException("SSL not available.")
self._handshake(hostname, port, resource, **options)
def _tunnel(self, host, port):
def _tunnel(self, host, port, auth):
logger.debug("Connecting proxy...")
connect_header = "CONNECT %s:%d HTTP/1.0\r\n" % (host, port)
# TODO: support digest auth.
if auth and auth[0]:
auth_str = auth[0]
if auth[1]:
auth_str += ":" + auth[1]
encoded_str = base64encode(auth_str.encode()).strip().decode()
connect_header += "Proxy-Authorization: Basic %s\r\n" % encoded_str
connect_header += "\r\n"
_dump("request header", connect_header)
@@ -499,6 +569,10 @@ class WebSocket(object):
headers.append("Sec-WebSocket-Key: %s" % key)
headers.append("Sec-WebSocket-Version: %s" % VERSION)
subprotocols = options.get("subprotocols")
if subprotocols:
headers.append("Sec-WebSocket-Protocol: %s" % ",".join(subprotocols))
if "header" in options:
headers.extend(options["header"])
@@ -520,14 +594,14 @@ class WebSocket(object):
_dump("request header", header_str)
resp_headers = self._get_resp_headers()
success = self._validate_header(resp_headers, key)
success = self._validate_header(resp_headers, key, options.get("subprotocols"))
if not success:
self.close()
raise WebSocketException("Invalid WebSocket Header")
self.connected = True
def _validate_header(self, headers, key):
def _validate_header(self, headers, key, subprotocols):
for k, v in _HEADERS_TO_CHECK.items():
r = headers.get(k, None)
if not r:
@@ -536,6 +610,14 @@ class WebSocket(object):
if v != r:
return False
if subprotocols:
subproto = headers.get("sec-websocket-protocol", None)
if not subproto or subproto not in subprotocols:
logger.error("Invalid subprotocol: " + str(subprotocols))
return False
self.subprotocol = subproto
result = headers.get("sec-websocket-accept", None)
if not result:
return False
@@ -556,12 +638,13 @@ class WebSocket(object):
while True:
line = self._recv_line()
line = line.decode('utf-8')
if line == "\r\n" or line == "\n":
line = line.decode('utf-8').strip()
if not line:
break
line = line.strip()
if traceEnabled:
logger.debug(line)
if not status:
status_info = line.split(" ", 2)
status = int(status_info[1])
@@ -654,7 +737,10 @@ class WebSocket(object):
opcode, data = self.recv_data()
if six.PY3 and opcode == ABNF.OPCODE_TEXT:
return data.decode("utf-8")
elif opcode == ABNF.OPCODE_TEXT or opcode == ABNF.OPCODE_BINARY:
return data
else:
return ''
def recv_data(self, control_frame=False):
"""
@@ -665,34 +751,8 @@ class WebSocket(object):
return value: tuple of operation code and string(byte array) value.
"""
while True:
frame = self.recv_frame()
if not frame:
# handle error:
# 'NoneType' object has no attribute 'opcode'
raise WebSocketException("Not a valid frame %s" % frame)
elif frame.opcode in (ABNF.OPCODE_TEXT, ABNF.OPCODE_BINARY, ABNF.OPCODE_CONT):
if frame.opcode == ABNF.OPCODE_CONT and not self._cont_data:
raise WebSocketException("Illegal frame")
if self._cont_data:
self._cont_data[1] += frame.data
else:
self._cont_data = [frame.opcode, frame.data]
if frame.fin or self.fire_cont_frame:
data = self._cont_data
self._cont_data = None
return data
elif frame.opcode == ABNF.OPCODE_CLOSE:
self.send_close()
return (frame.opcode, frame.data)
elif frame.opcode == ABNF.OPCODE_PING:
self.pong(frame.data)
if control_frame:
return (frame.opcode, frame.data)
elif frame.opcode == ABNF.OPCODE_PONG:
if control_frame:
return (frame.opcode, frame.data)
opcode, frame = self.recv_data_frame(control_frame)
return opcode, frame.data
def recv_data_frame(self, control_frame=False):
"""
@@ -708,24 +768,39 @@ class WebSocket(object):
if not frame:
# handle error:
# 'NoneType' object has no attribute 'opcode'
raise WebSocketException("Not a valid frame %s" % frame)
raise WebSocketProtocolException("Not a valid frame %s" % frame)
elif frame.opcode in (ABNF.OPCODE_TEXT, ABNF.OPCODE_BINARY, ABNF.OPCODE_CONT):
if frame.opcode == ABNF.OPCODE_CONT and not self._cont_data:
raise WebSocketException("Illegal frame")
if not self._recving_frames and frame.opcode == ABNF.OPCODE_CONT:
raise WebSocketProtocolException("Illegal frame")
if self._recving_frames and frame.opcode in (ABNF.OPCODE_TEXT, ABNF.OPCODE_BINARY):
raise WebSocketProtocolException("Illegal frame")
if self._cont_data:
self._cont_data[1].data += frame.data
self._cont_data[1] += frame.data
else:
self._cont_data = [frame.opcode, frame]
if frame.opcode in (ABNF.OPCODE_TEXT, ABNF.OPCODE_BINARY):
self._recving_frames = frame.opcode
self._cont_data = [frame.opcode, frame.data]
if frame.fin:
self._recving_frames = None
if frame.fin or self.fire_cont_frame:
data = self._cont_data
self._cont_data = None
return data
frame.data = data[1]
if not self.fire_cont_frame and data[0] == ABNF.OPCODE_TEXT and not validate_utf8(frame.data):
raise WebSocketPayloadException("cannot decode: " + repr(frame.data))
return [data[0], frame]
elif frame.opcode == ABNF.OPCODE_CLOSE:
self.send_close()
return (frame.opcode, frame)
elif frame.opcode == ABNF.OPCODE_PING:
if len(frame.data) < 126:
self.pong(frame.data)
else:
raise WebSocketProtocolException("Ping message is too long")
if control_frame:
return (frame.opcode, frame)
elif frame.opcode == ABNF.OPCODE_PONG:
@@ -762,7 +837,10 @@ class WebSocket(object):
# Reset for next frame
frame_buffer.clear()
return ABNF(fin, rsv1, rsv2, rsv3, opcode, has_mask, payload)
frame = ABNF(fin, rsv1, rsv2, rsv3, opcode, has_mask, payload)
frame.validate()
return frame
def send_close(self, status=STATUS_NORMAL, reason=six.b("")):
@@ -775,6 +853,7 @@ class WebSocket(object):
"""
if status < 0 or status >= ABNF.LENGTH_16:
raise ValueError("code is invalid range")
self.connected = False
self.send(struct.pack('!H', status) + reason, ABNF.OPCODE_CLOSE)
def close(self, status=STATUS_NORMAL, reason=six.b("")):
@@ -807,41 +886,61 @@ class WebSocket(object):
except:
pass
self._closeInternal()
self.shutdown()
def _closeInternal(self):
def abort(self):
"""
Low-level asynchonous abort, wakes up other threads that are waiting in recv_*
"""
if self.connected:
self.sock.shutdown(socket.SHUT_RDWR)
def shutdown(self):
"close socket, immediately."
if self.sock:
self.sock.close()
self.sock = None
self.connected = False
def _send(self, data):
if isinstance(data, six.text_type):
data = data.encode('utf-8')
if not self.sock:
raise WebSocketConnectionClosedException("socket is already closed.")
try:
return self.sock.send(data)
except socket.timeout as e:
message = getattr(e, 'strerror', getattr(e, 'message', ''))
message = _extract_err_message(e)
raise WebSocketTimeoutException(message)
except Exception as e:
message = getattr(e, 'strerror', getattr(e, 'message', ''))
if "timed out" in message:
message = _extract_err_message(e)
if message and "timed out" in message:
raise WebSocketTimeoutException(message)
else:
raise
def _recv(self, bufsize):
if not self.sock:
raise WebSocketConnectionClosedException("socket is already closed.")
try:
bytes = self.sock.recv(bufsize)
except socket.timeout as e:
message = getattr(e, 'strerror', getattr(e, 'message', ''))
message = _extract_err_message(e)
raise WebSocketTimeoutException(message)
except SSLError as e:
message = getattr(e, 'strerror', getattr(e, 'message', ''))
message = _extract_err_message(e)
if message == "The read operation timed out":
raise WebSocketTimeoutException(message)
else:
raise
if not bytes:
self.sock.close()
self.sock = None
self.connected = False
raise WebSocketConnectionClosedException()
return bytes

View File

@@ -30,6 +30,17 @@ class WebSocketException(Exception):
"""
pass
class WebSocketProtocolException(WebSocketException):
"""
If the webscoket protocol is invalid, this exception will be raised.
"""
pass
class WebSocketPayloadException(WebSocketException):
"""
If the webscoket payload is invalid, this exception will be raised.
"""
pass
class WebSocketConnectionClosedException(WebSocketException):
"""

View File

@@ -19,6 +19,7 @@ Copyright (C) 2010 Hiroki Ohtani(liris)
"""
import six
class NoLock(object):
def __enter__(self):
@@ -27,3 +28,57 @@ class NoLock(object):
def __exit__(self,type, value, traceback):
pass
# UTF-8 validator
# python implementation of http://bjoern.hoehrmann.de/utf-8/decoder/dfa/
UTF8_ACCEPT = 0
UTF8_REJECT=12
_UTF8D = [
# The first part of the table maps bytes to character classes that
# to reduce the size of the transition table and create bitmasks.
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, 9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,
7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7, 7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
8,8,2,2,2,2,2,2,2,2,2,2,2,2,2,2, 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,
10,3,3,3,3,3,3,3,3,3,3,3,3,4,3,3, 11,6,6,6,5,8,8,8,8,8,8,8,8,8,8,8,
# The second part is a transition table that maps a combination
# of a state of the automaton and a character class to a state.
0,12,24,36,60,96,84,12,12,12,48,72, 12,12,12,12,12,12,12,12,12,12,12,12,
12, 0,12,12,12,12,12, 0,12, 0,12,12, 12,24,12,12,12,12,12,24,12,24,12,12,
12,12,12,12,12,12,12,24,12,12,12,12, 12,24,12,12,12,12,12,12,12,24,12,12,
12,12,12,12,12,12,12,36,12,36,12,12, 12,36,12,12,12,12,12,36,12,36,12,12,
12,36,12,12,12,12,12,12,12,12,12,12, ]
def _decode(state, codep, ch):
tp = _UTF8D[ch]
codep = (ch & 0x3f ) | (codep << 6) if (state != UTF8_ACCEPT) else (0xff >> tp) & (ch)
state = _UTF8D[256 + state + tp]
return state, codep;
def validate_utf8(utfbytes):
"""
validate utf8 byte string.
utfbytes: utf byte string to check.
return value: if valid utf8 string, return true. Otherwise, return false.
"""
state = UTF8_ACCEPT
codep = 0
for i in utfbytes:
if six.PY2:
i = ord(i)
state, codep = _decode(state, codep, i)
if state == UTF8_REJECT:
return False
return True

View File

@@ -5,6 +5,7 @@ import six
import sys
sys.path[0:0] = [""]
import os
import os.path
import base64
import socket
@@ -25,15 +26,15 @@ import uuid
# websocket-client
import websocket as ws
from websocket._core import _parse_url, _create_sec_websocket_key
from websocket._core import _get_proxy_info
from websocket._utils import validate_utf8
# Skip test to access the internet.
TEST_WITH_INTERNET = False
# TEST_WITH_INTERNET = True
# Skip Secure WebSocket test.
TEST_SECURE_WS = False
TRACABLE = False
@@ -62,6 +63,9 @@ class SockMock(object):
self.sent.append(data)
return len(data)
def close(self):
pass
class HeaderSockMock(SockMock):
@@ -182,25 +186,31 @@ class WebSocketTest(unittest.TestCase):
"connection": "upgrade",
"sec-websocket-accept": "Kxep+hNu9n51529fGidYu7a3wO0=",
}
self.assertEqual(sock._validate_header(required_header, key), True)
self.assertEqual(sock._validate_header(required_header, key, None), True)
header = required_header.copy()
header["upgrade"] = "http"
self.assertEqual(sock._validate_header(header, key), False)
self.assertEqual(sock._validate_header(header, key, None), False)
del header["upgrade"]
self.assertEqual(sock._validate_header(header, key), False)
self.assertEqual(sock._validate_header(header, key, None), False)
header = required_header.copy()
header["connection"] = "something"
self.assertEqual(sock._validate_header(header, key), False)
self.assertEqual(sock._validate_header(header, key, None), False)
del header["connection"]
self.assertEqual(sock._validate_header(header, key), False)
self.assertEqual(sock._validate_header(header, key, None), False)
header = required_header.copy()
header["sec-websocket-accept"] = "something"
self.assertEqual(sock._validate_header(header, key), False)
self.assertEqual(sock._validate_header(header, key, None), False)
del header["sec-websocket-accept"]
self.assertEqual(sock._validate_header(header, key), False)
self.assertEqual(sock._validate_header(header, key, None), False)
header = required_header.copy()
header["sec-websocket-protocol"] = "sub1"
self.assertEqual(sock._validate_header(header, key, ["sub1", "sub2"]), True)
self.assertEqual(sock._validate_header(header, key, ["sub2", "sub3"]), False)
def testReadHeader(self):
sock = ws.WebSocket()
@@ -287,6 +297,46 @@ class WebSocketTest(unittest.TestCase):
with self.assertRaises(ws.WebSocketConnectionClosedException):
sock.recv()
def testRecvWithFireEventOfFragmentation(self):
sock = ws.WebSocket(fire_cont_frame=True)
s = sock.sock = SockMock()
# OPCODE=TEXT, FIN=0, MSG="Brevity is "
s.add_packet(six.b("\x01\x8babcd#\x10\x06\x12\x08\x16\x1aD\x08\x11C"))
# OPCODE=CONT, FIN=0, MSG="Brevity is "
s.add_packet(six.b("\x00\x8babcd#\x10\x06\x12\x08\x16\x1aD\x08\x11C"))
# OPCODE=CONT, FIN=1, MSG="the soul of wit"
s.add_packet(six.b("\x80\x8fabcd\x15\n\x06D\x12\r\x16\x08A\r\x05D\x16\x0b\x17"))
_, data = sock.recv_data()
self.assertEqual(data, six.b("Brevity is "))
_, data = sock.recv_data()
self.assertEqual(data, six.b("Brevity is "))
_, data = sock.recv_data()
self.assertEqual(data, six.b("the soul of wit"))
# OPCODE=CONT, FIN=0, MSG="Brevity is "
s.add_packet(six.b("\x80\x8babcd#\x10\x06\x12\x08\x16\x1aD\x08\x11C"))
with self.assertRaises(ws.WebSocketException):
sock.recv_data()
with self.assertRaises(ws.WebSocketConnectionClosedException):
sock.recv()
def testClose(self):
sock = ws.WebSocket()
sock.sock = SockMock()
sock.connected = True
sock.close()
self.assertEqual(sock.connected, False)
sock = ws.WebSocket()
s = sock.sock = SockMock()
sock.connected = True
s.add_packet(six.b('\x88\x80\x17\x98p\x84'))
sock.recv()
self.assertEqual(sock.connected, False)
def testRecvContFragmentation(self):
sock = ws.WebSocket()
s = sock.sock = SockMock()
@@ -382,12 +432,11 @@ class WebSocketTest(unittest.TestCase):
@unittest.skipUnless(TEST_WITH_INTERNET, "Internet-requiring tests are disabled")
def testAfterClose(self):
from socket import error
s = ws.create_connection("ws://echo.websocket.org/")
self.assertNotEqual(s, None)
s.close()
self.assertRaises(error, s.send, "Hello")
self.assertRaises(error, s.recv)
self.assertRaises(ws.WebSocketConnectionClosedException, s.send, "Hello")
self.assertRaises(ws.WebSocketConnectionClosedException, s.recv)
def testUUID4(self):
""" WebSocket key should be a UUID4.
@@ -477,6 +526,109 @@ class SockOptTest(unittest.TestCase):
self.assertNotEqual(s.sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY), 0)
s.close()
class UtilsTest(unittest.TestCase):
def testUtf8Validator(self):
state = validate_utf8(six.b('\xf0\x90\x80\x80'))
self.assertEqual(state, True)
state = validate_utf8(six.b('\xce\xba\xe1\xbd\xb9\xcf\x83\xce\xbc\xce\xb5\xed\xa0\x80edited'))
self.assertEqual(state, False)
state = validate_utf8(six.b(''))
self.assertEqual(state, True)
class ProxyInfoTest(unittest.TestCase):
def setUp(self):
self.http_proxy = os.environ.get("http_proxy", None)
self.https_proxy = os.environ.get("https_proxy", None)
if "http_proxy" in os.environ:
del os.environ["http_proxy"]
if "https_proxy" in os.environ:
del os.environ["https_proxy"]
def tearDown(self):
if self.http_proxy:
os.environ["http_proxy"] = self.http_proxy
elif "http_proxy" in os.environ:
del os.environ["http_proxy"]
if self.https_proxy:
os.environ["https_proxy"] = self.https_proxy
elif "https_proxy" in os.environ:
del os.environ["https_proxy"]
def testProxyFromArgs(self):
self.assertEqual(_get_proxy_info("echo.websocket.org", False, http_proxy_host="localhost"), ("localhost", 0, None))
self.assertEqual(_get_proxy_info("echo.websocket.org", False, http_proxy_host="localhost", http_proxy_port=3128), ("localhost", 3128, None))
self.assertEqual(_get_proxy_info("echo.websocket.org", True, http_proxy_host="localhost"), ("localhost", 0, None))
self.assertEqual(_get_proxy_info("echo.websocket.org", True, http_proxy_host="localhost", http_proxy_port=3128), ("localhost", 3128, None))
self.assertEqual(_get_proxy_info("echo.websocket.org", False, http_proxy_host="localhost", http_proxy_auth=("a", "b")),
("localhost", 0, ("a", "b")))
self.assertEqual(_get_proxy_info("echo.websocket.org", False, http_proxy_host="localhost", http_proxy_port=3128, http_proxy_auth=("a", "b")),
("localhost", 3128, ("a", "b")))
self.assertEqual(_get_proxy_info("echo.websocket.org", True, http_proxy_host="localhost", http_proxy_auth=("a", "b")),
("localhost", 0, ("a", "b")))
self.assertEqual(_get_proxy_info("echo.websocket.org", True, http_proxy_host="localhost", http_proxy_port=3128, http_proxy_auth=("a", "b")),
("localhost", 3128, ("a", "b")))
self.assertEqual(_get_proxy_info("echo.websocket.org", True, http_proxy_host="localhost", http_proxy_port=3128, http_no_proxy=["example.com"], http_proxy_auth=("a", "b")),
("localhost", 3128, ("a", "b")))
self.assertEqual(_get_proxy_info("echo.websocket.org", True, http_proxy_host="localhost", http_proxy_port=3128, http_no_proxy=["echo.websocket.org"], http_proxy_auth=("a", "b")),
(None, 0, None))
def testProxyFromEnv(self):
os.environ["http_proxy"] = "http://localhost/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", None, None))
os.environ["http_proxy"] = "http://localhost:3128/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", 3128, None))
os.environ["http_proxy"] = "http://localhost/"
os.environ["https_proxy"] = "http://localhost2/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", None, None))
os.environ["http_proxy"] = "http://localhost:3128/"
os.environ["https_proxy"] = "http://localhost2:3128/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", 3128, None))
os.environ["http_proxy"] = "http://localhost/"
os.environ["https_proxy"] = "http://localhost2/"
self.assertEqual(_get_proxy_info("echo.websocket.org", True), ("localhost2", None, None))
os.environ["http_proxy"] = "http://localhost:3128/"
os.environ["https_proxy"] = "http://localhost2:3128/"
self.assertEqual(_get_proxy_info("echo.websocket.org", True), ("localhost2", 3128, None))
os.environ["http_proxy"] = "http://a:b@localhost/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", None, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost:3128/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", 3128, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost/"
os.environ["https_proxy"] = "http://a:b@localhost2/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", None, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost:3128/"
os.environ["https_proxy"] = "http://a:b@localhost2:3128/"
self.assertEqual(_get_proxy_info("echo.websocket.org", False), ("localhost", 3128, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost/"
os.environ["https_proxy"] = "http://a:b@localhost2/"
self.assertEqual(_get_proxy_info("echo.websocket.org", True), ("localhost2", None, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost:3128/"
os.environ["https_proxy"] = "http://a:b@localhost2:3128/"
self.assertEqual(_get_proxy_info("echo.websocket.org", True), ("localhost2", 3128, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost/"
os.environ["https_proxy"] = "http://a:b@localhost2/"
os.environ["no_proxy"] = "example1.com,example2.com"
self.assertEqual(_get_proxy_info("example.1.com", True), ("localhost2", None, ("a", "b")))
os.environ["http_proxy"] = "http://a:b@localhost:3128/"
os.environ["https_proxy"] = "http://a:b@localhost2:3128/"
os.environ["no_proxy"] = "example1.com,example2.com, echo.websocket.org"
self.assertEqual(_get_proxy_info("echo.websocket.org", True), (None, 0, None))
if __name__ == "__main__":
unittest.main()