OpenStack Hacking Style Checks
Go to file
Stephen Finucane 172d34813d Bump flake8 version to something modern
The flake8 version we're using is a good 2 years old and is in turn
using an old version of pycodestyle. This is causing the following
warnings:

  FutureWarning: Possible nested set at position 1
    EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')

Bump to the latest and greatest so we get these fixes. This involves
dropping support for local-checks, which don't seem to work in this
version. Fortunately, in the intervening time, flake8 has grown the
ability to do local checks all by its lonesome. The tests also need to
be reworked and unfortunately are much slower now. This is because
flake8 3.x's API is file-based [1] and without rooting around the guts
of flake8, it's not practical to do things any other way. Hopefully
flake8 4.x won't have such issues.

[1] https://gitlab.com/pycqa/flake8/issues/545

Change-Id: I695ff02a6970663add10caf7f16a66abf9d1239d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Sem-Ver: api-break
2019-12-11 17:18:21 +00:00
doc Bump flake8 version to something modern 2019-12-11 17:18:21 +00:00
hacking Bump flake8 version to something modern 2019-12-11 17:18:21 +00:00
integration-test Replace openstack.org git:// URLs with https:// 2019-03-24 20:33:44 +00:00
releasenotes Bump flake8 version to something modern 2019-12-11 17:18:21 +00:00
.gitignore Switch to stestr 2018-07-24 10:54:56 +07:00
.gitreview OpenDev Migration Patch 2019-04-19 19:33:16 +00:00
.mailmap Add .mailmap file 2013-05-24 19:58:36 +00:00
.stestr.conf Switch to stestr 2018-07-24 10:54:56 +07:00
.zuul.yaml Add Python 3 Train unit tests 2019-07-15 13:18:17 -04:00
CONTRIBUTING.rst Workflow documentation is now in infra-manual 2014-12-05 03:30:41 +00:00
HACKING.rst Add support for detecting SPDX license headers 2018-09-13 16:42:51 -05:00
LICENSE Make hacking a flake8 plugin. 2013-03-18 12:19:25 -07:00
lower-constraints.txt Bump flake8 version to something modern 2019-12-11 17:18:21 +00:00
MANIFEST.in Move the hacking guidelines to sphinx docs 2013-09-20 16:09:39 -07:00
README.rst Fix 'ref' format errors in README file 2018-12-06 13:06:16 +00:00
requirements.txt Bump flake8 version to something modern 2019-12-11 17:18:21 +00:00
setup.cfg Bump flake8 version to something modern 2019-12-11 17:18:21 +00:00
setup.py Updated from global requirements 2017-09-20 16:11:55 +00:00
test-requirements.txt General cleanups 2019-11-22 16:52:51 +00:00
tox.ini General cleanups 2019-11-22 16:52:51 +00:00

Introduction

hacking is a set of flake8 plugins that test and enforce the OpenStack StyleGuide

Hacking pins its dependencies, as a new release of some dependency can break hacking based gating jobs. This is because new versions of dependencies can introduce new rules, or make existing rules stricter.

Installation

hacking is available from pypi, so just run:

pip install hacking

This will install specific versions of flake8 with the hacking, pep8, mccabe and pyflakes plugins.

Origin

Hacking started its life out as a text file in Nova's first commit. It was initially based on the Google Python Style Guide, and over time more OpenStack specific rules were added. Hacking serves several purposes:

  1. Agree on a common style guide so reviews don't get bogged down on style nit picks. (example: docstring guidelines)
  2. Make code written by many different authors easier to read by making the style more uniform. (example: unix vs windows newlines)
  3. Call out dangerous patterns and avoid them. (example: shadowing built-in or reserved words)

Initially the hacking style guide was enforced manually by reviewers, but this was a big waste of time so hacking, the tool, was born to automate the process and remove the extra burden from human reviewers.

Versioning

hacking uses the major.minor.maintenance release notation, where maintenance releases cannot contain new checks. This way projects can gate on hacking by pinning on the major.minor number while accepting maintenance updates without being concerned that a new version will break the gate with a new check.

For example a project can depend on hacking>=0.10.0,<0.11.0, and can know that 0.10.1 will not fail in places where 0.10.0 passed.

Adding additional checks

Each check is a pep8 plugin so read

The focus of new or changed rules should be to do one of the following

  • Substantially increase the reviewability of the code (eg: H301, H303) as they make it easy to understand where symbols come from)
  • Catch a common programming error that may arise in the future (H201)
  • Prevent a situation that would 100% of the time be -1ed by developers (H903)

But, as always, remember that these are Guidelines. Treat them as such. There are always times for exceptions. All new rules should support noqa.

If a check needs to be staged in, or it does not apply to every project or its branch, it can be added as off by default.

Requirements

  • The check must already have community support. We do not want to dictate style, only enforce it.
  • The canonical source of the OpenStack Style Guidelines is StyleGuide, and hacking just enforces them; so when adding a new check, it must be in HACKING.rst
  • False negatives are ok, but false positives are not
  • Cannot be project specific, project specific checks should be Local Checks
  • Include extensive tests
  • Registered as entry_points in setup.cfg
  • Error code must be in the relevant Hxxx group
  • The check should not attempt to import modules from the code being checked. Importing random modules, has caused all kinds of trouble for us in the past.

Enabling off-by-default checks

Some of the available checks are disabled by default. These checks are:

  • [H106] Don't put vim configuration in source files.
  • [H203] Use assertIs(Not)None to check for None.
  • [H204] Use assert(Not)Equal to check for equality.
  • [H205] Use assert(Greater|Less)(Equal) for comparison.
  • [H210] Require 'autospec', 'spec', or 'spec_set' in mock.patch/mock.patch.object calls
  • [H904] Delay string interpolations at logging calls.

To enable these checks, edit the flake8 section of the tox.ini file. For example to enable H106 and H203:

[flake8]
enable-extensions = H106,H203

Local Checks

hacking supports having local changes in a source tree. They can be configured to run in two different ways. They can be registered individually, or with a factory function.

For individual registration, put a comma separated list of pep8 compatible check functions into the hacking section of tox.ini. E.g.:

[hacking]
local-check = nova.tests.hacking.bad_code_is_terrible

Alternately, you can specify the location of a callable that will be called at registration time and will be passed the registration function. The callable should expect to call the passed in function on everything if wants to register. Such as:

[hacking]
local-check-factory = nova.tests.hacking.factory