Python automate Linting
Programming Tips

How to Automate Formatting and Linting in Python

Posted by

We will be looking at some packages to format, lint, test our code and later create a pre-commit hook to automate the process.

Before we talking about the packages we will use, let’s look at the python files we will work with

We have a function called helpers.py

def add(x, y):
    return x + y


def subtract(x, y):
    return x - y


def multiply(x, y):
    return x * y


def divide(x, y):
    return x / y

It has simple arithmetic functions. All the functions accept two parameters and perform an arithmetic operation on them.

We have another file called tester_helpers.py

from .helpers import add, divide, multiply, subtract


def test_add():
    assert add(1, 2) == 3


def test_subtract():
    assert subtract(2, 1) == 1


def test_multiply():
    assert multiply(2, 2) == 4


def test_divide():
    assert divide(4, 2) == 2

This file simply tests the functions we defined earlier. It uses assert to do a simple equality check.

Now let’s look at the packages we will be using.

Testing 🧪

Pytest 7.7k+ ⭐️

This package helps us run unit tests. One requirement to keep in mind is that your python file containing the unit tests should begin with ‘test_’.

Only assert statements are supported. To install the package

pip install pytest

To run the unit tests, type the following commands

pytest test_helpers.py

If all your tests pass, you should see a similar output

test_helpers.py ....             [100%]

========= 4 passed in 0.01s ===========

If you get an error related to multiple relative imports

astroid.exceptions.TooManyLevelsError:

It is probably an issue with one of pytest’s dependencies. You’ll have to uninstall astroid and install it again. This ensures the latest astroid version is installed.

pip uninstall astroid
pip install astroid

After this, we will have to uninstall pytest and install pytest

pip uninstall pytest
pip install pytest

Formatting ✍️

YAPF 12k+ ⭐️

This was developed by Google and supports in-place formatting. To install the package

pip install yapf

To format your files, type the following

yapf --in-place *.py

This will format all your top-level python files, if you want to include folders as well you can use the following

yapf --in-place **/*.py

However, this will also include our virtual environment folder. To ignore the venv folder, simply create a file .yapfignore and add venv to it.

Note: This command might take some time to run. Instead of ‘**’ you could use the folder’s specific names.

isort 4.1k+ ⭐️

This package sorts your import statements to ensure they follow pep8 rules.

Imports should be grouped in the following order:

  • Standard library imports.
  • Related third party imports.
  • Local application/library specific imports.

isort re-orders import statements to ensure the above rule is followed.
To install the package

pip install isort

To run isort

isort .

autoflake 400+⭐️

It helps in getting rid of unused imports, variables, and object keys.

To install the package

pip install autoflake

To run autoflake

autoflake --in-place --remove-unused-variables --remove-all-unused-imports *.py

Some other formatters

Linting 🔎

Pylint 3.5k+ ⭐️

pylint ensures your code is following pep8 rules and standards. It gives each python file a score out of 10 (It can give you a negative score as well)

To install the package

pip install pylint

To run the linter

pylint --fail-under=7 *.py

The argument --fail-under is the lower bound, if any file has a score below the lower bound, an error will be returned.

Pre-commit Hook 🪝

What Are Git Hooks?

Git hooks are basically scripts fired before an important action occurs, e.g., before a commit is made, before code is pushed to a repo after a commit is made, etc. You can learn more about Git Hooks and the different kinds of hooks over here.

We will be focussing on a pre-commit hook. A pre-commit hook is a hook that is run before you make a commit.

First, let’s install the package

pip install pre-commit

Now we will generate a sample pre-commit hook YAML file, we will edit this later.

pre-commit sample-config

Now let’s add our hook

pre-commit install

Now before every commit, the pre-commit hook defined in our YAML file will be executed.

Now let’s update our YAML file.
Remove everything and only keep the following

repos:
    - repo: local
      hooks:

We will add our plugins(packages) under hooks: in the YAML file. Below is the general syntax for the plugin

   - id: (unique id of hook)
     name: (name to be displayed in terminal)
     entry: (command to excute)
     language: system (for our case, always system) 
     always_run: true (if true, it will always run)
     pass_filenames: true (if true, hook will have access to the file name)

Let’s define a sample plugin for YAPF

   - id: YAPF 
     name: YAPF 🧹
     entry: zsh -c 'yapf --in-place *.py'
     language: system
     always_run: true
     pass_filenames: true

If you are using bash or are on windows, replace the zsh in ‘entry’ with bash.

All the other plugins are pretty similar, below is the entire YAML file with all the plugins

repos:
    - repo: local
      hooks:
        - id: YAPF 
          name: YAPF 🧹
          entry: zsh -c 'yapf --in-place *.py'
          language: system
          always_run: true
          pass_filenames: true
        - id: isort 
          name: isort 📚
          entry: zsh -c 'isort .'
          language: system
          always_run: true
          pass_filenames: true
        - id: autoflake 
          name: autoflake ❄️
          entry: zsh -c 'autoflake --in-place --remove-unused-variables --remove-all-unused-imports *.py'
          language: system
          always_run: true
          pass_filenames: true
        - id: pylint 
          name: pylint 🔎
          entry: zsh -c 'pylint --fail-under=-15 *.py'
          language: system
          always_run: true
          pass_filenames: true
        - id: pytest 
          name: Unit Tests 🧪
          entry: zsh -c 'pytest test_helpers.py'
          language: system
          always_run: true
          pass_filenames: false

Whenever you update your YAML file, you will have to add the file to the staging area using git add . or git add .pre-commit-config.yaml

Below is a successfull commit

Pre-commit in action

Conclusion

Setting up a pre-commit hook will ensure your code follows pep8 standards and is properly formatted.
I hope you found the article useful. Add me on LinkedIn, Twitter

If you are looking for software jobs, you can search for Python developer jobs here