Ignore quantifiers when splitting comma-separated regexes #7241

lbenezriravin · 2022-07-28T22:15:04Z

Type of Changes

	Type
✓	🐛 Bug fix

Description

When parsing comma-separated lists of regular expressions in the config, ignore
commas that are inside braces since those indicate quantifiers, not delineation
between expressions.

There may be conflicts with #7228, lmk if you'd prefer if this gets merged first or second.

Closes #7229

Do not split on commas if they are between braces, since that indicates a quantifier. Also added a protection for slow implementations since existing workarounds may result in long strings of chained regular expressions.

Fixes pylint-dev#7229

Pierre-Sassoulas

Looks pretty good already, thank you ! I'll put the "need a decision" label so we don't merge it before we decide what we do in 3.0.

tests/config/test_config.py

Pierre-Sassoulas · 2022-07-29T08:04:41Z

pylint/utils/utils.py

+        regexps: deque[deque[str] | None] = deque([None])
+        open_braces = False
+        for char in value:
+            if char == "{":


Depending on the result of the discussion we should warn the user of deprecation here.

wouldn't it be on line 346? That's where we split regular expressions.

You're right, my bad.

I know a year ago we were floating the idea of deprecations, but we're not ready to commit to a new plan IMO.

coveralls · 2022-07-29T08:16:29Z

Pull Request Test Coverage Report for Build 2762066289

19 of 20 (95.0%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.0008%) to 95.249%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pylint/utils/utils.py	18	19	94.74%

Totals
Change from base Build 2737748168:	-0.0008%
Covered Lines:	16839
Relevant Lines:	17679

💛 - Coveralls

Overkill, slows down unit tests Co-authored-by: Pierre Sassoulas <[email protected]>

github-actions · 2022-07-29T21:37:11Z

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit b48c339

jacobtylerwalls

Thank you! I left cosmetic feedback only. Thanks for adding a regression test that fails on main.

I think we should merge. We should not let there be silent parsing failures when trying to use regular expressions. (What a gotcha!)

jacobtylerwalls · 2023-07-02T23:32:09Z

doc/whatsnew/fragments/7229.bugfix

@@ -0,0 +1,5 @@
+When parsing comma-separated lists of regular expressions in the config, ignore
+commas that are inside braces since those indicate quantiers, not dilineation


Suggested change

commas that are inside braces since those indicate quantiers, not dilineation

commas that are inside braces since those indicate quantifiers, not delineation

jacobtylerwalls · 2023-07-02T23:36:50Z

tests/config/test_config.py

@@ -5,6 +5,8 @@
 from __future__ import annotations

 import os
+import re
+import timeit


Suggested change

import timeit

jacobtylerwalls · 2023-07-02T23:37:26Z

tests/config/test_config.py

+    assert _template_run(in_string) == [re.compile(regex) for regex in expected]
+
+
+


Suggested change

jacobtylerwalls · 2023-07-03T16:31:15Z

@lbenezriravin If you enable maintainers to push to this fork I can resolve the merge conflicts for you and add the cosmetic changes.

DanielNoord · 2023-07-04T13:03:03Z

pylint/utils/__init__.py

@@ -34,6 +35,7 @@
    "HAS_ISORT_5",
    "IsortDriver",
    "_check_csv",
+    "_check_regexp_csv",


Since this is private, this should not be in __all__.

Suggested change

"_check_regexp_csv",

The PR currently uses it on the module, so it has to stay, unless we refactor. It was a reasonable choice for the PR author to adhere to the existing muddled pattern. I'm inclined to merge as is.

jacobtylerwalls · 2023-07-29T21:54:16Z

Will merge in #8898. Thanks!

lihu added 3 commits July 28, 2022 18:28

Add protection for pylint-dev#7229

8be7a33

Do not split on commas if they are between braces, since that indicates a quantifier. Also added a protection for slow implementations since existing workarounds may result in long strings of chained regular expressions.

Do not split regex lists in quantifier ranges

40889e3

Fixes pylint-dev#7229

Add news fragment

3347cf5

lbenezriravin force-pushed the lihu/7229 branch from f6f7906 to 3347cf5 Compare July 28, 2022 22:30

Pierre-Sassoulas added Bug 🪲 Configuration Related to configuration labels Jul 29, 2022

Pierre-Sassoulas added this to the 2.15.0 milestone Jul 29, 2022

Pierre-Sassoulas reviewed Jul 29, 2022

View reviewed changes

Pierre-Sassoulas added the Needs decision 🔒 Needs a decision before implemention or rejection label Jul 29, 2022

Remove performance test

b48c339

Overkill, slows down unit tests Co-authored-by: Pierre Sassoulas <[email protected]>

Pierre-Sassoulas modified the milestones: 2.15.0, 2.16.0 Aug 23, 2022

Pierre-Sassoulas modified the milestones: 2.16.0, 2.17.0 Jan 8, 2023

Pierre-Sassoulas modified the milestones: 2.17.0, 3.0.0 Mar 7, 2023

jacobtylerwalls mentioned this pull request Jul 2, 2023

Show why snake_case errors is emitted on single word variables (might be too small) #2018

Closed

jacobtylerwalls added Needs review 🔍 Needs to be reviewed by one or multiple more persons and removed Needs decision 🔒 Needs a decision before implemention or rejection labels Jul 2, 2023

jacobtylerwalls modified the milestones: 3.0.0, 3.0.0a7 Jul 2, 2023

jacobtylerwalls self-requested a review July 2, 2023 20:21

jacobtylerwalls approved these changes Jul 2, 2023

View reviewed changes

jacobtylerwalls removed the Needs review 🔍 Needs to be reviewed by one or multiple more persons label Jul 2, 2023

jacobtylerwalls modified the milestones: 3.0.0a7, 2.17.5 Jul 3, 2023

jacobtylerwalls added the backport maintenance/3.3.x label Jul 3, 2023

DanielNoord reviewed Jul 4, 2023

View reviewed changes

Pierre-Sassoulas added the Waiting on author Indicate that maintainers are waiting for a message of the author label Jul 11, 2023

Pierre-Sassoulas modified the milestones: 2.17.5, 2.17.6 Jul 26, 2023

jacobtylerwalls mentioned this pull request Jul 29, 2023

Ignore quantifiers when splitting comma-separated regexes #8898

Merged

jacobtylerwalls closed this Jul 29, 2023

jacobtylerwalls removed this from the 2.17.6 milestone Jul 29, 2023

Pierre-Sassoulas added Backported and removed backport maintenance/3.3.x labels Feb 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ignore quantifiers when splitting comma-separated regexes #7241

Ignore quantifiers when splitting comma-separated regexes #7241

lbenezriravin commented Jul 28, 2022 •

edited by jacobtylerwalls

Loading

Pierre-Sassoulas left a comment

Pierre-Sassoulas Jul 29, 2022

lbenezriravin Jul 29, 2022

Pierre-Sassoulas Jul 29, 2022

jacobtylerwalls Jul 2, 2023

coveralls commented Jul 29, 2022 •

edited

Loading

github-actions bot commented Jul 29, 2022

jacobtylerwalls left a comment •

edited

Loading

jacobtylerwalls Jul 2, 2023

jacobtylerwalls Jul 2, 2023

jacobtylerwalls Jul 2, 2023

jacobtylerwalls commented Jul 3, 2023

DanielNoord Jul 4, 2023

jacobtylerwalls Jul 29, 2023

jacobtylerwalls commented Jul 29, 2023

		@@ -0,0 +1,5 @@
		When parsing comma-separated lists of regular expressions in the config, ignore
		commas that are inside braces since those indicate quantiers, not dilineation

		assert _template_run(in_string) == [re.compile(regex) for regex in expected]

Ignore quantifiers when splitting comma-separated regexes #7241

Ignore quantifiers when splitting comma-separated regexes #7241

Conversation

lbenezriravin commented Jul 28, 2022 • edited by jacobtylerwalls Loading

Type of Changes

Description

Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Jul 29, 2022 • edited Loading

Pull Request Test Coverage Report for Build 2762066289

💛 - Coveralls

github-actions bot commented Jul 29, 2022

jacobtylerwalls left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacobtylerwalls commented Jul 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacobtylerwalls commented Jul 29, 2023

lbenezriravin commented Jul 28, 2022 •

edited by jacobtylerwalls

Loading

coveralls commented Jul 29, 2022 •

edited

Loading

jacobtylerwalls left a comment •

edited

Loading