Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling the three-way exits of generators and coroutines efficiently. #448

Open
markshannon opened this issue Aug 16, 2022 · 0 comments
Open

Comments

@markshannon
Copy link
Member

Python has two kinds of callables: functions and generators/coroutines.

Functions have a two-way exit: return or raise.
Coroutines have a three-way exit: return, yield or raise.

Because generators and coroutines are newer, a lot of the C code in genobject.c, ceval.c and related files squeezes the three-way exit into a two-way exit using StopIteration and StopAsyncIteration which is really clunky and inefficient.

Instead of squeezing three-way exits into two-ways exits, we should be broadening the two-way exits into three-way exits, when needed.

For example, we can drop the throwflag argument from _PyEval_EvalFrameDefault by implementing gen.throw() in bytecode.
We can do this because bytecode already handles the three-way exit in FOR_ITER and SEND as follows:

  • Return: jump
  • Yield: push value to stack
  • Raise: propagate.

gen.send() can be implemented as something like:

  LOAD_FAST     0 (self)
  LOAD_FAST     1 (value)
  SETUP_FINALLY exception_handler
  SEND               returned
  POP_BLOCK
  RETURN_VALUE
returned:
  LOAD_CONST       StopIteration
  PUSH_NULL
  SWAP 3
  CALL                     1
  RAISE_VARARGS            1
exception_handler:
  LOAD_CONST       StopIteration
  CHECK_EXC_MATCH
  POP_JUMP_FORWARD_IF_FALSE    reraise
  POP_TOP
  PUSH_NULL
  LOAD_CONST      RuntimeError
  CALL                     0
  RAISE_VARARGS            1
reraise:
   RERAISE                  0
     

gen.throw(), etc. can be implemented similarly.
See #67 (comment)

Of course, we still need to have some C functions, not everything can be done in bytecode.
For those functions, we should use something like the interface of gen_send_ex2 which returns the kind of exit, and uses an out parameter for the value

PySendResult gen_send_ex2(..., PyObject **presult ,..);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant