A study of five tools used to check Python code
Researchers analyzed how different Python linters compare in speed, rules, and configuration.
A study examined five Python linters named Ruff, Pylint, Flake8, Mypy and Black. These tools automatically detect code issues to help developers. They can identify errors early to improve code quality and accelerate development.
The study looked at linting rule checks, speed, configuration methods, and linter preferences. Linters are tools that automatically detect code issues. Researchers analyzed 1,551 linting rules and categorized them into five types.
The researchers measured running times on 1,000 GitHub projects. They found a speed ranking from fastest to slowest. The order was Ruff, Mypy, Flake8, Black, and Pylint. Ruff is extremely fast with runtimes typically within one second.
The study also analyzed 3,094 GitHub projects. Over 50% of these projects use multiple linters. While there is consensus on some options, projects show variations in enabled-rule configurations.
Findings include that Ruff and Pylint together cover over 80% of all rules. More than 40% of rules are shared between multiple linters. The study provides a benchmark dataset and taxonomy to support future research.
Key facts
- Ruff and Pylint together cover over 80% of all 1,551 linting rules.
- Ruff is the fastest linter with runtimes typically within one second.
- Over 50% of the 3,094 GitHub projects analyzed use multiple linters.