Convert a warning about flags not at the start of the regular expression into error

[serhiy-storchaka]

BPO	47066
Nosy	@ezio-melotti, @serhiy-storchaka
PRs	bpo-47066: Convert a warning about flags not at the start of the regu… #31994

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-19.14:12:52.270>
created_at = <Date 2022-03-19.12:09:48.176>
labels = ['expert-regex', 'type-bug', 'library', '3.11']
title = 'Convert a warning about flags not at the start of the regular expression into error'
updated_at = <Date 2022-03-19.14:12:52.269>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2022-03-19.14:12:52.269>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-19.14:12:52.270>
closer = 'serhiy.storchaka'
components = ['Library (Lib)', 'Regular Expressions']
creation = <Date 2022-03-19.12:09:48.176>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 47066
keywords = ['patch']
message_count = 2.0
messages = ['415544', '415552']
nosy_count = 3.0
nosy_names = ['ezio.melotti', 'mrabarnett', 'serhiy.storchaka']
pr_nums = ['31994']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue47066'
versions = ['Python 3.11']

[serhiy-storchaka]

This warning was introduced in 3.6. The reason is that in most other regular expression implementations global inline flags in the middle of the expression have different semantic: they affect only the part of the expression after the flag. But in Python they affect the whole expression. It caused confusion and was a source of bugs.

After 5 releases it is a time to convert this warning into error. In future we can allow global inline flags in the middle of the expression with different semantic. It is safer if one or more intermediate versions will raise an error.

[serhiy-storchaka]

New changeset 92a6abf by Serhiy Storchaka in branch 'main':
bpo-47066: Convert a warning about flags not at the start of the regular expression into error (GH-31994)
92a6abf

[jrmuizel]

For the record here's some brokenness this caused:
https://bz.mercurial-scm.org/show_bug.cgi?id=6759
https://bugzilla.mozilla.org/show_bug.cgi?id=1799982

[arhadthedev]

@jrmuizel Could you report this in a new issue mentioning this issue, please? I suspect almost nobody reads already closed entries.

[jrmuizel]

Is it worth filing a new issue even though both of those uses have already been fixed upstream?

[arhadthedev]

My bad. I've thought you linked to some extra issues similar to this but found by third-party projects that use Python.

[yilei]

#TIL if the intent was to only apply the flag to a subset of the regex, e.g.:

r"name-((?i)CASE-INSENSITIVE)"

You can use the following:

r"name-(?i:CASE-INSENSITIVE)"

See the (?aiLmsux-imsx:...) section under https://docs.python.org/3/library/re.html#regular-expression-syntax

It would be great if this was mentioned on https://docs.python.org/3/whatsnew/3.11.html#porting-to-python-3-11 as I have "migrated" dozens of files running into this issue where the original intent was to only apply the flag to a subset (especially when you concatenate regexes together), but I only learned about this today. (I hope this comment can at least help some when they click the bpo-47066 link there and get redirected here.

Also, there is no mention of the word "global" on https://docs.python.org/3/library/re.html, as it seems ideal if that could mention "global" v.s. "local" inline flags.