Problem installing version 0.15.0 on pypy3 (0.14.1 worked)

[scottbelden]

I apologize upfront that I don't have know what the root cause of the problem is because I don't know much about the changes between 0.14.1 and 0.15.0.

Here's the failure on 0.15.0: https://github.com/fastavro/fastavro/runs/1630514376?check_suite_focus=true

And here's a working install from a few days ago when it used 0.14.1: https://github.com/fastavro/fastavro/runs/1620804784?check_suite_focus=true (You'll want to look at the "Install Testing Dependencies" since that is the step that installs zstandard)

I think the main difference I can see is that version 0.14.1 had a wheel to install whereas when trying to install 0.15.0 it installs from source and fails.

Does anyone have any thoughts?

[indygreg]

Hmmm. This is definitely an unwanted regression: pypy should be installing wheels.

I'll try to get a fix landed in the next few days and get a 0.15.1 published.

[indygreg]

Correction: pypy did not install wheels before. So, something changed in the 0.15.0 release preventing the wheels from building.

A manual bisection reveals commit e5a3baf as the culprit. After this commit, we get an error in the pythoncapi_compat.h header introduced by that commit:

  running build_ext
  building 'zstd' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/zstd
  creating build/temp.linux-x86_64-3.6/zstd/common
  creating build/temp.linux-x86_64-3.6/zstd/compress
  creating build/temp.linux-x86_64-3.6/zstd/decompress
  creating build/temp.linux-x86_64-3.6/zstd/dictBuilder
  gcc -pthread -DNDEBUG -O2 -fPIC -Ic-ext -Izstd -Izstd/common -Izstd/compress -Izstd/decompress -Izstd/dictBuilder -I/home/gps/.pyenv/versions/pypy3.6-7.3.1/include -c zstd.c -o build/temp.linux-x86_64-3.6/zstd.o -DZSTD_MULTITHREAD -DZSTDLIB_VISIBILITY= -DZDICTLIB_VISIBILITY= -DZSTDERRORLIB_VISIBILITY= -fvisibility=hidden
  In file included from c-ext/python-zstandard.h:15,
                   from zstd.c:19:
  c-ext/pythoncapi_compat.h: In function ‘PyFrame_GetBack’:
  c-ext/pythoncapi_compat.h:115:17: error: ‘PyFrameObject’ {aka ‘struct _frame’} has no member named ‘f_back’
    115 |     back = frame->f_back;
        |                 ^~
  c-ext/pythoncapi_compat.h: In function ‘PyThreadState_GetFrame’:
  c-ext/pythoncapi_compat.h:148:19: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘frame’
    148 |     frame = tstate->frame;
        |                   ^~
  c-ext/pythoncapi_compat.h: In function ‘PyObject_GC_IsTracked’:
  c-ext/pythoncapi_compat.h:252:13: warning: implicit declaration of function ‘PyObject_IS_GC’; did you mean ‘PyObject_INIT’? [-Wimplicit-function-declaration]
    252 |     return (PyObject_IS_GC(obj) && _PyObject_GC_IS_TRACKED(obj));
        |             ^~~~~~~~~~~~~~
        |             PyObject_INIT
  c-ext/pythoncapi_compat.h:252:36: warning: implicit declaration of function ‘_PyObject_GC_IS_TRACKED’ [-Wimplicit-function-declaration]
    252 |     return (PyObject_IS_GC(obj) && _PyObject_GC_IS_TRACKED(obj));
        |                                    ^~~~~~~~~~~~~~~~~~~~~~~
  c-ext/pythoncapi_compat.h: In function ‘PyObject_GC_IsFinalized’:
  c-ext/pythoncapi_compat.h:262:36: warning: implicit declaration of function ‘_PyGCHead_FINALIZED’ [-Wimplicit-function-declaration]
    262 |     return (PyObject_IS_GC(obj) && _PyGCHead_FINALIZED((PyGC_Head *)(obj)-1));
        |                                    ^~~~~~~~~~~~~~~~~~~
  not modified: 'build/_zstd_cffi.c'
  error: command 'gcc' failed with exit status 1
  Building wheel for zstandard (PEP 517) ... error

I suppose the bug here is we are building the C extension/backend on PyPy when we should only be building the CFFI backend on that platform. (PyPy doesn't support the full CPython C API.) I'm actually kinda surprised that the C backend was building properly before!

[scottbelden]

Yes, you are right, there isn't a pre-built wheel for pypy; I was misreading the output and forgot that pip builds a wheel from source during the install.

It looks like you are on the right track. Thanks for taking a look!