Use mypy to do static type checking

python3 includes support for optional type annotations which can be used by
static analysis tools to perform type checking. The mypy tool is a
static type checking tool that can also infer type information in many
cases, but which will use explicit type information if it is present.

Add mypy to test-requirements and to the pep8 job so that our pep8 job
can do more analysis work and less with the code style.

To support this, there were a few places in the current codebase that
needed an explicit type hint. For variables/attributes in 3.5 this is done via
comments. There is a conditional import that was confusion that just got
marked with an 'ignore'.

Our ansible action and lookup plugins confuse mypi with the way they
import the ansible base classes. That's ok - they confuse us with that
too. The .pyi files are 'typeshed' files, which are a way that one can
provide static type annotations without putting the information into the
file itself. mypy will always prefer a .pyi file over a .py file (since
the point of them is to be external annotion/interface description) So
in order to get mypy to not barf on the ansible import weirdness, just
add a corresponding empty .pyi file. We could potentially actually put
interface descriptions in them - but I don't think there is very much
value in that.

It should be amusing to at least someone that we have to flake8: noqa
an import from typing that was done to provide a type hint in a comment.

Change-Id: I6c4ac3dcfc6fd990e6c6886749de147ad28389d1

test-requirements.txt

@@ -14,3 +14,4 @@ sphinxcontrib-programoutput
 oslosphinx
 mock
 PyMySQL
+mypy

tox.ini

@@ -27,8 +27,12 @@ deps = bindep
 commands = bindep test
 
 [testenv:pep8]
-# streamer is python3 only, so we need to run flake8 in python3
-commands = flake8 {posargs}
+# --ignore-missing-imports tells mypy to not try to follow imported modules
+# out of the current tree. As you might expect, we don't want to run static
+# type checking on the world - just on ourselves.
+commands =
+  flake8 {posargs}
+  mypy --ignore-missing-imports zuul
 
 [testenv:cover]
 commands =

zuul/driver/__init__.py

@@ -33,7 +33,7 @@ class Driver(object, metaclass=abc.ABCMeta):
     The class or instance attribute **name** must be provided as a string.
 
     """
-    name = None
+    name = None  # type: str
 
     def reconfigure(self, tenant):
         """Called when a tenant is reconfigured.

zuul/driver/bubblewrap/__init__.py

@@ -23,6 +23,8 @@ import shlex
 import subprocess
 import sys
 
+from typing import Dict, List  # flake8: noqa
+
 from zuul.driver import (Driver, WrapperInterface)
 
 
@@ -70,7 +72,7 @@ class BubblewrapDriver(Driver, WrapperInterface):
     name = 'bubblewrap'
     log = logging.getLogger("zuul.BubblewrapDriver")
 
-    mounts_map = {'rw': [], 'ro': []}
+    mounts_map = {'rw': [], 'ro': []}  # type: Dict[str, List]
 
     def __init__(self):
         self.bwrap_command = self._bwrap_command()

zuul/lib/yamlutil.py

@@ -13,7 +13,8 @@ import yaml
 from yaml import YAMLObject, YAMLError  # noqa: F401
 
 try:
-    from yaml import cyaml
+    # Explicit type ignore to deal with provisional import failure
+    from yaml import cyaml  # type: ignore
     import _yaml
     SafeLoader = cyaml.CSafeLoader
     SafeDumper = cyaml.CSafeDumper