Standardise all pre-commit hooks, including more ruff, code formatting etc

[Cadair]

Currently there is still quite some variation in the ruff config's and pre-commit configs. We should standardise on a single set.

[nabobalis]

This is my WIP template for ruff that I need to adapt for newer versions of ruff:

https://github.com/LM-SAL/irispy-lmsal/blob/docs/ruff.toml

[Cadair]

Here's my dkist one: https://github.com/DKISTDC/dkist/blob/main/.ruff.toml

[Cadair]

I think in general I prefer the opt-in rather than opt out approach, for two main reasons: there's a lot of irrelevant rulesets (like django etc) and also because I would rather new rule sets were opt-in rather than opt-out.

[CyclingNinja]

Ruff exclude rules list

All of these are used in the extends ignore of both styles
This is explicitly the general list, not the per file excludes, it would be good to agree on a unified list for the template

PyFlake8

---

• F405 - undefined-local-with-import-star-usage - dkist
• F702 - continue-outside-loop - dkist
• F821 - undefined-name - dkist
• F841 - unused-variable - dkist

Errors

---

• E501 - line-too-long - dkist, sunpy
• E702 - multiple-statements-on-one-line-semicolon - dkist
• E711 - none-comparison - dkist
• E741 - ambiguous-variable-name - dkist

PyTest

---

PT001 - pytest-fixture-incorrect-parentheses-style - dkist. sunpy
PT004 - pytest-missing-fixture-name-underscore - dkist, sunpy
PT007 - pytest-parameterize-values-wrong-type - dkist, sunpy
PT011 - pytest-raises-too-broad - dkist, sunpy
PT023 - pytest-incorrect-mark-parentheses-style - dkist, sunpy

PyUpgrade

---

UP038 - non-pep604-instance - dkist

Returns

---

RET502 - implicit-return-value - dkist
RET503 - implicit-return - dkist

Pathlib

---

PTH123 - builtin-open - dkist

Ruff

---

RUF012 - multable-class-deffault - dkist
RUF013 - implicit-optional - dkist
RUF015 - unnecessary-iterable-allocation-for-first-element - dkist

Annotations

---

ANN001 - Missing type annotation for function argument - irispy
ANN002 - Missing type annotation for variable - irispy
ANN003 - Missing type annotation for keyword - irispy
ANN201 - Missing return type annotation for public function - irispy
ANN202 - Missing return type annotation for private function - irispy
ANN205 - Missing return type annotation for staticmethod - irispy
ANN206 - Missing return type annotation for classmethod - irispy

Pydocstyle

---

D200 - fits-on-one-line - irispy
D205 - blank-line-after-summary - irispy
D400 - ends-in-period - irispy
D401 - non-imperative-mood - irispy
D404 - docstring-starts-with-this - irispy

ISC001 - single-line-implicit-string-concatenation - irispy

[CyclingNinja]

Top Level Includes

Pyflakes - F

used in - dkist, ndcube, template, drms
need - High fundamental to this whole thing actually working. Many rules here, but most templates ignore a few

use in template? - yes

pycodestyle - E

used in - dkist, ndcube, template, drms
need - High, fundamental - basic python structures essential

use in template? - yes

warning - W

used in - ndcube, drms
need - high, fundamental to ruff working, none of these have been ignored regularly

use in template - yes

mccabe - C90

description - structure too complex

use in template - no

isort - I

description - sort python imports
need - we run isort anyway so maybe no?

use in template - no

pep8-naming - N

description - PEP8 naming conventions baby!
need - medium to high

use in template - yes

pydocstyle -D

description - pydocstyle is a static analysis tool for checking compliance with Python docstring conventions.
need - low aren't pydoc strings bad?

use in template? - no

pyupgrade - UP

description - A tool (and pre-commit hook) to automatically upgrade syntax for newer versions of the language
used in - template, drms
commented out in - ndcube
need - high

use in template? - yes

flake8-2020 - YTT

description - 2020 handling for py3.10
need - low

use in template? - no

flake8-annotations - ANN

description - detects the abscence of PEP3107
need - low

use in template? - no

flake8-async - ASYNC

description - deals with async beef
need - medium

use in template? - maybe

flake8-bandit - S

description - looks for security issues in your code
need - low

use in template? - no

flake8-blind-except - BLE

description - hunts for pokemon exceptions
used in - drms
need - high

use in template? - yes

flake8-boolean-trap - FBT

description - hunts for boolean anti patterns
used in - drms
need - medium

use in template? - maybe

flake8-bugbear - B

description - bug hunter, has warning not in pyflakes or pycodestye
used in - drms
need - medium/high

use in template? - yes

flake8-builtins - A

description - ensures builtins aren't being redefined
used in - drms

use in template? - yes

flake8-commas - COM

description - comma handling, might be nice to have so we're not adding out trailed commas etc
need medium

use in template? - yes

flake8-copyright - CPY

use in template? - no

flake8-comprehensions - C4

description - cleans up unnecessary code blocks
used in - drms
need - medium

use in template? - maybe

flake8-datetimes - DTZ

description - datetime cleaning up
need - medium, means we're not looking in PRs

use in template? - maybe

flake8-debugger - T10

use in template? - no

flake8-django - DJ

use in template? - no

flake8-errmsg - EM

description - exception/error message hanfling
need - low

use in template? - no

flake8-executable - EXE

description - shebang handling
need - low/medium

use in template? - no

flake8-future-annotations - FA

use in template? - no

flake8-implicit-str-concat - ISC

use in template? - no

flake8-import-conventions - ICN

description - An opinionated plugin for Flake8 on how certain packages should be imported or aliased
need - low, we intend to keep using isort

use in template? - no

flake8-logging - LOG

description - standard logging issues
need - medium

use in template? - maybe

flake8-logging-format - G

description - removes formatting in logging
need - medium, could be appropriate for OSS

use in template? - maybe

flake8-no-pep420 - INP

use in template? - no

flake8-pie - PIE

use in template? - no

flake8-print - T20

description - print eradicator
used in - drms

use in template? - maybe

flake8-pyi - PYI

description - specializations for type hinting stub files
need - low

use in template? - no

flake8-pytest-style - PT

description - flake8 specific style and inconsistency checker in pytest
used in - template, drms
commented out in - ndcube

use in template? - yes

flake8-quotes - Q

description - handles escaped strings
need - low-shouldn't come up too often

use in template? - no

flake8-raise - RSE

description - unecessarry parentheses on raise
used in - drms

use in template? - no

flake8-return - RET

description - ensures function returns are sensible
used in - drms

use in template? - yes

flake8-self - SLF

use in template? - no

flake8-slots - SLOT

use in template? - no

flake8-simplify - SIM

description - code simplifier, ie unused stuff and neutral vs keyword comparision
need - low

use in template? - no

flake8-tidy-imports - TID

description - import checking for flake8
used in - drms
need - low/medium - doesn't isort handle this?

use in template? - no

flake8-type-checking - TCH

description - flake8 argument checking
need - medium

use in template? - maybe

flake8-gettext - INT

use in template? - no

flake8-unused-arguments - ARG

description - argument hanfling
need - low

use in template? - no

flake8-use-pathlib - PTH

description - handles pathlib behaviours for safety
used in - drms
need - medium to high

use in template? - yes

flake8-todos - TD

description - fixme and todo removal with flake8
need - low

use in template? - no

flake8-fixme - FIX

description - fixme and todo removal with flake8
need - low

use in template? - no

eradicate - ERA

description - commented out code banhammer
used in - drms

use in template? - maybe

pandas-vet - PD

description - pandas specific linting
need - medium? pandas can be a horror show

use in template? - maybe

pypgrep-hooks - PGH

description - linting for pre-commit hooks
used in - drms

use in template? - no

pylint - PL

Has sub categorites thus

Convention - PLC

descrption - convention enforcement, pylint specific
need - medium/high

use in template? - yes

Error - PLE

description - pylint specific error detection
used in - drms
need - medium/high

use in template? - yes

Refactor - PLR

description- specific refactoring linting
need - low

use in template? - no

Warning - PLW

description - pylint specific warnings for code design
need - medium

use in template? - maybe

tryceratops - TRY

description - prevents exception handling antipatterns
need - low

use in template? - no

flynt - FLY

use in template? - no, we should stop old formatting at PR

NumPy-specific-rules - NPY

description - numpy depreciation messaging
used in - drms
need - maybe, but we do want these errors to be flagged at minimum

use in template? - no

FastAPI - FAST

use in template? - no

Airflow - AIR

use in template? - no

Perflint - PERF

description - linting looking specifically for anti patterns
need - maybe? seems hyper specific, but would be useful in an open source platform

use in template? - no

refurb - FURB

description - refurbishment package
need - low

use in template? - no

pydoclint - DOC

description - really pinikity documentation rules
used in -
need - low/maybe

use in template? - no

ruff specific rules - RUF

used in - drms

use in template? -yes