Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mypyc] Initial optimization for f-string through a str.join() specializer #10776

Merged
merged 8 commits into from
Jul 8, 2021

Conversation

97littleleaf11
Copy link
Collaborator

@97littleleaf11 97littleleaf11 commented Jul 6, 2021

Description

This pull request adds a specializer for faster constructing f-string, which is translated into str.join() previously in mypy AST.

Test Plan

IR building tests:

  • {var}
  • empty string
  • literal string
  • unsupported method

Speed up:

running str_format_fstring
..........
interpreted: 0.295028s (avg of 5 iterations; stdev 2.3%)
compiled:    0.188493s (avg of 5 iterations; stdev 1.7%)

compiled is 1.565x faster

on this case:

@benchmark
 def str_format_fstring() -> None:
     a = []
     tmp_str = "Foobar"
     for i in range(1000):
         for _ in range(5):
             a.append(f'{tmp_str}')
         a.append(f'{tmp_str}-{i}')
         a.append(f'{i} {i*2.0} str')

     n = 0
     for i in range(100):
         for s in a:
             n += len(f"foobar {s} stuff")
             ss = f"foobar {s} stuff"
             n += len(f"{i}-{s}-{ss}")
     assert n == 36897500, n

Current master branch is 0.380x.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great performance improvement! Left some comments. Maybe also incorporate the int-to-str improvements from the other PR?

return builder.load_str("")

result_list[0] = Integer(len(result_list) - 1, c_pyssize_t_rprimitive)
return builder.call_c(str_build_op, result_list, expr.line)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point it should be possible to share some of the code above with the str.format implementation. No need to do anything about this in this PR.

mypyc/irbuild/specialize.py Show resolved Hide resolved
mypyc/irbuild/specialize.py Outdated Show resolved Hide resolved
mypyc/irbuild/specialize.py Show resolved Hide resolved
mypyc/test-data/run-strings.test Outdated Show resolved Hide resolved
mypyc/test-data/run-strings.test Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants