Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-45786: Allocate space for frame in frame object. #29729

Merged
merged 6 commits into from
Nov 29, 2021

Conversation

markshannon
Copy link
Member

@markshannon markshannon commented Nov 23, 2021

Avoids the need to perform another allocation when a frameobject outlives its activation, removing a awkward situation where we might end up with an invalid frameobject due to a MemoryError.

https://bugs.python.org/issue45786

@markshannon
Copy link
Member Author

May even be a little bit faster

@markshannon markshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 23, 2021
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @markshannon for commit af4933e 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 23, 2021
@markshannon
Copy link
Member Author

markshannon commented Nov 24, 2021

All the buildbot failures seemed to be from the deepfreeze build step.
At least one of following pass:

  • Linux
  • Windows
  • MacOS
  • Refleaks
  • Address sanitizer

So this PR appears to be good.

@@ -93,7 +93,6 @@ struct _Py_unicode_state {
# define PyTuple_MAXFREELIST 1
# define PyList_MAXFREELIST 0
# define PyDict_MAXFREELIST 0
# define PyFrame_MAXFREELIST 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we removing the freelist? The commit doesn't explain anything for this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The frameobjects are no longer a fixed size, so a free list wouldn't work.

We shouldn't be allocating many frameobjects anyway, they only get creating when an exception is raised, or explicitly requested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this will happen when tracing/profiling, no? I suspect coverage runs for example will be slower due to this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, but freelists just don't work unless all objects are the same size.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The performance of coverage is already terrible, it needs to be fixed properly. One freelist more or less isn't going to make a difference.
See python/peps#2070 for something like a proper fix.

Copy link
Member

@pablogsal pablogsal Nov 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am skeptical of this, we are talking about the freelist of the frame object, which under coverage is going to be triggered a lot. I would like data to back this assertion.

Copy link
Member Author

@markshannon markshannon Nov 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which assertion? That the performance of coverage is already terrible, or that we need to fix it? 🙂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion:

One freelist more or less isn't going to make a difference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to land this having an idea of the impact, not to stop the PR for landing, just to be clear

Copy link
Member

@pablogsal pablogsal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM

But I would prefer if you do a small benchmark of pycoverage before and after this PR before landing.

@markshannon
Copy link
Member Author

Change LGTM

But I would prefer if you do a small benchmark of pycoverage before and after this PR before landing.

This is a bugfix, not a performance PR.

@pablogsal
Copy link
Member

This is a bugfix, not a performance PR.

Precisely, I want to know that coverage.py is not going to be tremendously slow because of this bugfix

@markshannon
Copy link
Member Author

coverage.py is already tremendously slow. The (relatively infrequent) creation of a frame is not the problem.

This is definitely something we need to address soon, but not here and not right now, IMO.
I've created nedbat/coveragepy#1287 and https://bugs.python.org/issue45923 to track this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants