parse error for comprehensions in f-strings

[zsol]

❯ cat test.py                                                                                                                                                    
x = [1, 2]               
print(f"List of: {y for y in x}")           
❯ python3 test.py                                                                                                                                                
List of: <generator object <genexpr> at 0x7facdf7d8258>
❯ libcst print test.py        
Traceback (most recent call last):                                                                                                                               
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/_parser/base_parser.py", line 152, in _add_token
    plan = stack[-1].dfa.transitions[transition]                                                                                                                 
KeyError: ReservedString(for)
                                                                                                                                                                 
During handling of the above exception, another exception occurred:
                                                                                                                                                                 
Traceback (most recent call last):
  File "/opt/instagram/virtualenv/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)                                                       
  File "/opt/instagram/virtualenv/lib/python3.7/runpy.py", line 85, in _run_code 
    exec(code, run_globals)      
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/tool.py", line 832, in <module>
    main(os.environ.get("LIBCST_TOOL_COMMAND_NAME", "libcst.tool"), sys.argv[1:])
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/tool.py", line 827, in main
    return lookup.get(args.action or None, _invalid_command)(proc_name, command_args)
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/tool.py", line 277, in _print_tree_impl
    else PartialParserConfig()                                                                                                                                   
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/_parser/entrypoints.py", line 76, in parse_module
    detect_default_newline=True,
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/_parser/entrypoints.py", line 51, in _parse
    result = parser.parse()                                                     
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/_parser/base_parser.py", line 111, in parse
    self._add_token(token)        
  File "/opt/instagram/virtualenv/lib/python3.7/site-packages/fbcode/libcst/_parser/base_parser.py", line 191, in _add_token
    raw_column=token.start_pos[1],
libcst._exceptions.ParserSyntaxError: Syntax Error @ 2:21.       
Incomplete input. Encountered 'for', but expected '!', ':', or '}'.
                                                                                                                                                                 
print(f"List of: {y for y in x}")

[thatch]

As stated, this looks like a simple missing [comp_for] near

LibCST/libcst/_parser/conversions/expression.py

Line 1039 in 6ae2583

@with_production(

Did this report come from Hypothesmith or is it a simplified version of a real case you found? Because it looks like cpython prints something non-useful and I'm wondering if there's more to it:

>>> f'{1 for x in (1, 2, 3)}'
'<generator object <genexpr> at 0x7f626677f580>'

[zsol]

Did this report come from Hypothesmith or is it a simplified version of a real case you found?

The latter. I'm pretty sure this was some broken logging code that never actually executed.

[jimmylai]

As stated, this looks like a simple missing [comp_for] near

LibCST/libcst/_parser/conversions/expression.py

Line 1039 in 6ae2583

@with_production(

Yes, we can add the comp_for to pass it in the same way as AST.

[amyreese]

Just saw this today when running µsort on code containing something like:

some.func(
    f"{[(a, b)] for a, b in other.func()}"
)

Is this is an easy fix, and/or can you point me at the right place and the changes I should make and where to best test it?