python 3.8.0 #45337

bayandin · 2019-10-14T22:34:06Z

I update PR description from time to time to track the progress of python 3.8 migration.

There is a branch for tracking progress: https://github.com/bayandin/homebrew-core/tree/test-python-3.8.0

breezy (breezy.log) breezy 3.0.2 #46941

    running build_ext
    skipping 'breezy/_simple_set_pyx.c' Cython extension (up-to-date)
    building 'breezy._simple_set_pyx' extension
    creating build/temp.macosx-10.15-x86_64-3.8
    creating build/temp.macosx-10.15-x86_64-3.8/breezy
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Ibreezy -I/usr/local/opt/python/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c breezy/_simple_set_pyx.c -o build/temp.macosx-10.15-x86_64-3.8/breezy/_simple_set_pyx.o
    breezy/_simple_set_pyx.c:6808:270: error: too many arguments to function call, expected 15, have 16
      __pyx_codeobj__9 = (PyObject*)__Pyx_PyCode_New(3, 0, 5, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__8, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_stringsource, __pyx_n_s_pyx_unpickle__SimpleSet_iterat, 1, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__9)) __PYX_ERR(1, 1, __pyx_L1_error)
                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
    breezy/_simple_set_pyx.c:318:82: note: expanded from macro '__Pyx_PyCode_New'
              PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
              ~~~~~~~~~~                                                             ^~~~
    /usr/local/opt/python/Frameworks/Python.framework/Versions/3.8/include/python3.8/code.h:122:12: note: 'PyCode_New' declared here
    PyAPI_FUNC(PyCodeObject *) PyCode_New(
               ^
    breezy/_simple_set_pyx.c:8619:9: error: too many arguments to function call, expected 15, have 16
            __pyx_empty_bytes  /*PyObject *lnotab*/
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    breezy/_simple_set_pyx.c:318:82: note: expanded from macro '__Pyx_PyCode_New'
              PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
              ~~~~~~~~~~                                                             ^~~~
    /usr/local/opt/python/Frameworks/Python.framework/Versions/3.8/include/python3.8/code.h:122:12: note: 'PyCode_New' declared here
    PyAPI_FUNC(PyCodeObject *) PyCode_New(
               ^
    2 errors generated.

      Cannot build extension "breezy._simple_set_pyx".
      Use "build_ext --allow-python-fallback" to use slower python implementations instead.

    error: command 'clang' failed with exit status 1
    Running setup.py install for breezy: finished with status 'error'

gpgme (gpgme.log) python 3.8.0 #45337 (comment)

Testing gpgme
/usr/bin/sandbox-exec -f /private/tmp/homebrew20191119-88892-17ta5r4.sb ruby -W0 -I $LOAD_PATH -- /usr/local/Homebrew/Library/Homebrew/test.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gpgme.rb --verbose
==> /usr/local/Cellar/gpgme/1.13.1_1/bin/gpgme-tool --lib-version
==> python2.7 -c import gpg; print gpg.version.versionstr
1.13.1
==> python3 -c import gpg; print(gpg.version.versionstr)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'gpg'
Error: gpgme: failed

graph-tool (graph-tool.log) graph-tool: fix build in Xcode 11 #46987

In file included from /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/__string:57:
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/algorithm:2494:5: error: static_assert failed due to requirement '__is_forward_iterator<boost::adj_list<unsigned long>::edge_iterator>::value' "std::max_element requires a ForwardIterator"
    static_assert(__is_forward_iterator<_ForwardIterator>::value,
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../src/graph/graph_adjacency.hh:452:26: note: in instantiation of function template specialization 'std::__1::max_element<boost::adj_list<unsigned long>::edge_iterator, (lambda at ../../../src/graph/graph_adjacency.hh:453:38)>' requested here
        auto iter = std::max_element(erange.first, erange.second,
                         ^
../../../src/graph/graph.hh:109:33: note: in instantiation of member function 'boost::adj_list<unsigned long>::shrink_to_fit' requested here
    void shrink_to_fit() { _mg->shrink_to_fit(); }
                                ^
1 error generated.
make[3]: *** [graph_betweenness.lo] Error 1

openimageio (openimageio.log) python 3.8.0 #45337 (comment)

In file included from /tmp/openimageio-20191119-78942-7ji6wt/oiio-Release-2.0.12/src/python/py_imageinput.cpp:31:
/tmp/openimageio-20191119-78942-7ji6wt/oiio-Release-2.0.12/src/python/py_oiio.h:41:10: fatal error: 'Python.h' file not found
#include <Python.h>
         ^~~~~~~~~~
1 error generated.
make[2]: *** [src/python/CMakeFiles/PyOpenImageIO.dir/py_imageinput.cpp.o] Error 1
make[1]: *** [src/python/CMakeFiles/PyOpenImageIO.dir/all] Error 2

weechat (weechat.log) python 3.8.0 #45337 (comment)

ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [src/plugins/python/python.so] Error 1
make[1]: *** [src/plugins/python/CMakeFiles/python.dir/all] Error 2

wxpython (wxpython.log) wxpython 4.0.7.post2 #46952

   [853/875] Compiling sip/cpp/sip_webkitwxWebKitBeforeLoadEvent.cpp
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:48: error: unknown class name 'wxWebKitStateChangedEvent'; did you mean 'sipwxWebKitStateChangedEvent'?
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~
                                                   sipwxWebKitStateChangedEvent
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:7: note: 'sipwxWebKitStateChangedEvent' declared here
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
          ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:46: error: base class has incomplete type
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
                                         ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:7: note: definition of 'sipwxWebKitStateChangedEvent' is not complete until the closing '}'
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
          ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:25:43: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
        sipwxWebKitStateChangedEvent(const  ::wxWebKitStateChangedEvent&);
                                            ~~^~~~~~~~~~~~~~~~~~~~~~~~~
                                              wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:46:81: error: member initializer 'wxWebKitStateChangedEvent' does not name a non-static data member or base class
    sipwxWebKitStateChangedEvent::sipwxWebKitStateChangedEvent( ::wxWindow*win):  ::wxWebKitStateChangedEvent(win), sipPySelf(SIP_NULLPTR)
                                                                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:51:69: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
    sipwxWebKitStateChangedEvent::sipwxWebKitStateChangedEvent(const  ::wxWebKitStateChangedEvent& a0):  ::wxWebKitStateChangedEvent(a0), sipPySelf(SIP_NULLPTR)
                                                                      ~~^~~~~~~~~~~~~~~~~~~~~~~~~
                                                                        wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:51:104: error: member initializer 'wxWebKitStateChangedEvent' does not name a non-static data member or base class
    sipwxWebKitStateChangedEvent::sipwxWebKitStateChangedEvent(const  ::wxWebKitStateChangedEvent& a0):  ::wxWebKitStateChangedEvent(a0), sipPySelf(SIP_NULLPTR)
                                                                                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:69:19: error: no member named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'sipwxWebKitStateChangedEvent'?
            return  ::wxWebKitStateChangedEvent::GetEventCategory();
                    ~~^~~~~~~~~~~~~~~~~~~~~~~~~
                      sipwxWebKitStateChangedEvent
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:7: note: 'sipwxWebKitStateChangedEvent' declared here
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
          ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:84:19: error: no member named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'sipwxWebKitStateChangedEvent'?
            return  ::wxWebKitStateChangedEvent::Clone();
                    ~~^~~~~~~~~~~~~~~~~~~~~~~~~
                      sipwxWebKitStateChangedEvent
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:7: note: 'sipwxWebKitStateChangedEvent' declared here
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
          ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:100:12: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
             ::wxWebKitStateChangedEvent *sipCpp;
             ~~^~~~~~~~~~~~~~~~~~~~~~~~~
               wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:109:30: error: no member named 'GetState' in 'wxPaletteChangedEvent'
                sipRes = sipCpp->GetState();
                         ~~~~~~  ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:135:12: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
             ::wxWebKitStateChangedEvent *sipCpp;
             ~~^~~~~~~~~~~~~~~~~~~~~~~~~
               wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:146:21: error: no member named 'SetState' in 'wxPaletteChangedEvent'
                sipCpp->SetState(state);
                ~~~~~~  ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:172:12: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
             ::wxWebKitStateChangedEvent *sipCpp;
             ~~^~~~~~~~~~~~~~~~~~~~~~~~~
               wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:181:46: error: no member named 'GetURL' in 'wxPaletteChangedEvent'
                sipRes = new  ::wxString(sipCpp->GetURL());
                                         ~~~~~~  ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:208:12: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
             ::wxWebKitStateChangedEvent *sipCpp;
             ~~^~~~~~~~~~~~~~~~~~~~~~~~~
               wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:219:21: error: no member named 'SetURL' in 'wxPaletteChangedEvent'
                sipCpp->SetURL(*url);
                ~~~~~~  ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:247:18: error: no type named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'wxPaletteChangedEvent'?
            const  ::wxWebKitStateChangedEvent *sipCpp;
                   ~~^~~~~~~~~~~~~~~~~~~~~~~~~
                     wxPaletteChangedEvent
    ../../../ext/wxWidgets/include/wx/event.h:2861:24: note: 'wxPaletteChangedEvent' declared here
    class WXDLLIMPEXP_CORE wxPaletteChangedEvent : public wxEvent
                           ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:256:50: error: no member named 'wxWebKitStateChangedEvent' in the global namespace; did you mean 'sipwxWebKitStateChangedEvent'?
                sipRes = (sipSelfWasArg ? sipCpp-> ::wxWebKitStateChangedEvent::Clone() : sipCpp->Clone());
                                                   ~~^~~~~~~~~~~~~~~~~~~~~~~~~
                                                     sipwxWebKitStateChangedEvent
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:21:7: note: 'sipwxWebKitStateChangedEvent' declared here
    class sipwxWebKitStateChangedEvent : public  ::wxWebKitStateChangedEvent
          ^
    ../../../sip/cpp/sip_webkitwxWebKitStateChangedEvent.cpp:256:77: error: 'sipwxWebKitStateChangedEvent::Clone' is not a member of class 'const ::wxPaletteChangedEvent'
                sipRes = (sipSelfWasArg ? sipCpp-> ::wxWebKitStateChangedEvent::Clone() : sipCpp->Clone());
                                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.

ansible-lint: ansible-lint: add a patch for setuptools >= 41.4.0 #47030 ( upstream issue ansible-lint fails to install from source: AttributeError: 'SpecifierSet' object has no attribute 'split' ansible/ansible-lint#590)
salt: salt 2019.2.2 #45895 (upstream issue [RC1 Sodium] salt fails on macOS Catalina, due to loading unversioned /usr/lib/libcrypto.dylib saltstack/salt#55084)
mrboom: mrboom: improve test reliability #47004
Also, see Broken bottles and conflicts discovered in Python 3.8 CI builds #46728

methane · 2019-10-15T06:48:23Z

Would you add --enable-optimizations and --with-lto configure option?

From Python 3.8, the option doesn't take too long time to build.
See https://bugs.python.org/issue36044

fxcoudert · 2019-10-15T07:12:17Z

Would you add --enable-optimization configure option?

Given this is not a default option, probably not, unless it is done in upstream (“official”) binaries.

fxcoudert · 2019-10-15T07:13:52Z

@bayandin Major version update, this is going to be a fun ride…

pip update in python should not be a separate commit
formulas that hardcode 3.7: ola xapian zim

methane · 2019-10-15T07:56:36Z

I think official binaries uses --enable-optimizations and --with-lto. But I'll confirm it.

bayandin · 2019-10-15T08:05:24Z

@fxcoudert

@bayandin Major version update, this is going to be a fun ride…

pip update in python should not be a separate commit

formulas that hardcode 3.7: ola xapian zim

No problem! I'm on it

fxcoudert · 2019-10-15T08:30:21Z

Looking at first test results, all python-dependent formulas will need a revision bump 😭

bayandin · 2019-10-15T08:40:04Z

Pushed a bunch of revision bumps (with fixes for image not found and for bad interpreter: No such file or directory) + fixes for ola, xapian and zim.

Jenkins output from the previous build (just in case):
python3.8-highsierra.txt
python3.8-mojave.txt

fxcoudert · 2019-10-15T13:49:37Z

@bayandin it will run when the CI has a free slot, we have to wait…

Noctem · 2019-10-15T23:04:28Z

--without-gcc is not a recognized option. Only causes a warning, but might as well remove it.

Would you add --enable-optimizations and --with-lto configure option?

--with-lto apparently requires llvm-ar which as far as I'm aware is not available in macOS' toolchain, so brewed llvm would presumably be required to build. --enable-optimizations seems like a good idea to me though.

Noctem · 2019-10-16T03:34:55Z

Here are a couple changes I needed to make to dependencies to get them to compile against 3.8:
023c458
#45393

VapourSynth was also failing to link so I added -L/usr/local/opt/python/Frameworks/Python.framework/Versions/3.8 -lpython3.8 to LDFLAGS, but I don't think that should be applied to every VapourSynth component (nor should it need to be supplied manually), so there should be a better fix.

methane · 2019-10-16T06:56:43Z

It seems offcial macOS binary doesn't use both of --enable-optimizations and --with-lto.

But these options have significant performance gains. See https://bugs.python.org/issue38477 for a recent example.

fxcoudert · 2019-10-16T06:59:44Z

It seems offcial macOS binary doesn't use both of --enable-optimizations and --with-lto

So we'll stick to not using them.

But these options have significant performance gains.

If it is safe, upstream should enable it by default. It does lead to significant build-time increase (30 minutes instead of 2 minutes, from what I see), and we do have users who build from source.

methane · 2019-10-16T07:05:44Z

If it is safe, upstream should enable it by default

It is not safe because we don't know much about the environment. PGO and LTO were unstable in the old GCC and Python may be built on such platforms. That's why we don't enable it by default in Makefile.

On the other hand, we can assume compiler versions on macOS. I don't know why official Python binary doesn't use them. I agree that the official binary should use it too.

lithammer · 2019-10-16T08:40:19Z

Related discussion about the flags on the python-dev mailing list: https://mail.python.org/archives/list/[email protected]/thread/EILILECNTLTW4VCBCPW37R4QRU7ZBDEU/#VUA2AG6J2QG2GK52DZ4VKZMRUTKJ4DWO

bayandin · 2019-11-23T10:24:09Z

Few questions:

apache-arrow

cassandra

z3

(this is just me looking through open PRs which depend on Python 3)

You're right! Bumped revisions and added python tests for these formulas.

The whole CI stopped because of No available formula with the name "grib-api" in eccodes. grib-api was deleted in #46924.
Have fixed this, and pushed the changes.

From the short run (https://jenkins.brew.sh/job/Homebrew%20Core%20Pull%20Requests/52326/):

abyss Test data no longer available for download bcgsc/abyss#305
alure and dromeaudio Broken bottles and conflicts discovered in Python 3.8 CI builds #46728
aom

04:20:46 ==> FAILED
04:20:47 Testing aom
04:20:47 /usr/bin/sandbox-exec -f /private/tmp/homebrew20191123-96092-1lzb2nj.sb ruby -W0 -I $LOAD_PATH -- /usr/local/Homebrew/Library/Homebrew/test.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/aom.rb --verbose
04:20:47 ==> Downloading https://media.xiph.org/video/derf/y4m/bus_qcif_15fps.y4m
04:20:47 /usr/bin/curl -q --globoff --show-error --user-agent Homebrew/2.1.16-127-ga1939cf\ \(Macintosh\;\ Intel\ Mac\ OS\ X\ 10.15.1\)\ curl/7.64.1 --fail --silent --location --remote-time --continue-at 0 --output /Users/brew/Library/Caches/Homebrew/downloads/ced756f9d1f53c598f4f2b5c3676f9a313d4102ac938a50a36d66ea7343e22bd--bus_qcif_15fps.y4m.incomplete https://media.xiph.org/video/derf/y4m/bus_qcif_15fps.y4m
04:20:47 curl: (60) SSL certificate problem: certificate has expired

mercurial — missed revision bump during the rebase. Fixed
bazelisk on High Sierra: the same problem as in bazel 1.2.1 #46983

bayandin · 2019-11-23T13:20:14Z

Bad news, I just find out that haven't bump revisions for the following formulae which requires a python as a runtime dependency (53):

afflib
anjuta
aurora-cli
bear
bento4
bind
choose
cryptominisat
ddgr
distcc
dnstwist
emscripten
evince
fpp
geos
gimme-aws-creds
glib
global
googler
gr-osmosdr
gradio
grc
gtk-doc
hashpump
ispc
lcm
liblouis
libproxy
mftrace
micropython
mitie
mm-common
nicovideo-dl
open-babel
opencolorio
otf2
percol
ponysay
[email protected]
redex
shocco
simgrid
ssh-audit
sshconfigfs
sslmate
suricata
tbb
uhd
unoconv
vit
watchman
whatmp3
yosys

Not sure how I missed all of these, probably their tests never failed for new Python 3.8 or I missed some of them during the rebase

fxcoudert · 2019-11-23T13:25:13Z

I'd say let's keep CI running for now, see what it uncovers already.

Bo98 · 2019-11-23T14:17:18Z

alure and dromeaudio are doomed to fail on the CI because they deal with audio. It's difficult writing meaningful tests for those without using audio.

I'm having similar issues with fluid-synth. There's a critical bug fix that I tried opening a pull request for (the previous version is simply broken) but even the dumb --version check that was there before broke as they forgot to update the version. My attempts to write another test that also worked on the CI ultimately failed and the pull request is kinda just stuck in limbo at the moment.

So yeah, just have to ignore that for now.

Not really sure what's happening with aom. Might have been a temporary thing.

Bo98 · 2019-11-23T23:10:41Z

I've updated #46728 with the non-blockers that can be seen up to now.

In addition (excluding those on your list of those to revision bump):

dafny - known to fail if z3 receives any type of revision or version bump - see z3 4.8.7 #46991. The fix is known but it's failing tests on High Sierra on rebuild.
uhd - you haven't revision bumped this yet but I have since been informed that this won't build on Catalina. I'm experimenting in uhd 3.14.1.1 #47101 but I don't really know what the problem is yet.

I also have major concerns about disk space. The Mojave node started this build with ~75GB of disk space left and now has just 3GB remaining...

basictheprogram · 2019-11-24T04:12:23Z

Is there any grunt work I can help on? Not a lot of experience with contributing to home brew but I'm a python developer by profession and would like to help if I can.

aaronliu0130 · 2019-11-24T06:17:00Z

Is there any grunt work I can help on? Not a lot of experience with contributing to home brew but I'm a python developer by profession and would like to help if I can.

Me too, though I am a superuser rather than a python developer.

Bo98 · 2019-11-24T06:26:09Z

I think the main issue right now is disk space to build all these formulae on the CI. Will have to cancel the current build because of that.

Apart from that we're getting close and a lot of the hard work has already been done.

@bayandin apache-arrow: ModuleNotFoundError: No module named 'pyarrow'

bayandin · 2019-11-25T00:00:34Z

@bayandin apache-arrow: ModuleNotFoundError: No module named 'pyarrow'

My bad attempt to improve the test. Fixed now.

I think the main issue right now is disk space to build all these formulae on the CI. Will have to cancel the current build because of that.

Is there a plan what to do with it?

Bo98 · 2019-11-25T00:06:28Z

Is there a plan what to do with it?

I've not heard, but I haven't heard much of anything today. I'll find out because we certainly need to do something if we want this merged.

In terms of formulae, I suppose there's one blocker still to fix and that's #47101. Gave up on it yesterday - I'll need to get a Catalina setup to look into it.

WIP Homebrew#45337 Change dependency to python3

cclauss · 2019-12-03T15:10:09Z

50 days since Python 3.8 shipped. How are things progressing?

bayandin · 2019-12-03T15:21:09Z

50 days since Python 3.8 shipped.

Unfortunately, we can't use this information to help us with migration ¯\_(ツ)_/¯

However, as we find out in this PR It's impossible to migrate everything (all python dependant formulae) at once (due to CI limitation and amount of work that should be done), so we're working on adding [email protected] and gradual migration. There is a bunch of tight-connected to each other formulae which have to be migrated together, and this is the first and the most challenging issue which we're trying to solve in #47326 (I would say it is in good shape so far).
When it's done, we're ready to migrate other independent formulae one-by-one.

michaelblyons · 2019-12-10T20:59:37Z

Can an owner add the [python] label to this PR?

bertonha · 2019-12-12T10:18:49Z

50 days since Python 3.8 shipped.

@bayandin I agree that kind of message does not help the migration, but I think we got on the wrong way to update to Python3.8.
My point is Python3.8 is way more important at least for me than most of the others packages, especially the ones failing and delaying this. e.g. VTK and a bunch of others.

Maybe if we had create a formula to [email protected] and linked all the "broken" formulas to it. And Update the formula python to Python3.8. We could already had python 3.8 and incrementally fixing formulas to run on python3.8.

This will also facilitate to parallel work on this issue.

SMillerDev · 2019-12-12T13:54:36Z

My point is Python3.8 is way more important at least for me than most of the others packages, especially the ones failing and delaying this. e.g. VTK and a bunch of others.

Unfortunately for you homebrew doesn't run for you, it runs to be useful to as many people as possible. Updating python forcefully because Christofer wants us to isn't really in line with that goal. If you want a python 3.8 version faster you're welcome to make a tap though.

Maybe if we had create a formula to [email protected] and linked all the "broken" formulas to it. And Update the formula python to Python3.8. We could already had python 3.8 and incrementally fixing formulas to run on python3.8.

That is pretty much what is happening, except there is a pull request for a [email protected] formula. But ~5000 formula aren't checked in a day, and certainly don't have their conflicts resolved in a day.

bertonha · 2019-12-12T14:49:38Z

Unfortunately for you homebrew doesn't run for you, it runs to be useful to as many people as possible. Updating python forcefully because Christofer wants us to isn't really in line with that goal. If you want a python 3.8 version faster you're welcome to make a tap though.

@SMillerDev don't get me wrong. I love homebrew and it always worked really well, I am here just proposing a difference approach for big changes like this.

Usually big bang approses does not go to smoothly and create gigantic PR, like update python and all the dependencies in one-shot.

Again I am just proposing a incremental way to achieve those complexes upgrades.

utkonos · 2019-12-12T15:01:58Z

@SMillerDev don't get me wrong. I love homebrew and it always worked really well, I am here just proposing a difference approach for big changes like this.

Usually big bang approses does not go to smoothly and create gigantic PR, like update python and all the dependencies in one-shot.

Again I am just proposing a incremental way to achieve those complexes upgrades.

I just created a thread on the community forum about how to do python updates when there is a large cluster of dependencies that also need updating. I don't think the discussion should happen here on the PR. Traffic here should just be from/for folks actually working on solving the PR. Extra traffic here probably just irritates the volunteers who are trying to solve the problem.

https://discourse.brew.sh/t/python-3-8-and-large-python-or-other-large-groups-of-dependencies/6451

iMichka · 2019-12-18T06:35:44Z

sslyze probably needs to be added to this list (#47993).

bayandin · 2019-12-20T09:09:39Z

Closed in favour of separate migration in #47274

bayandin force-pushed the python-3.8.0 branch 4 times, most recently from cdcd84a to e0dbcf5 Compare October 15, 2019 00:43

bayandin force-pushed the python-3.8.0 branch from e0dbcf5 to 30a18ef Compare October 15, 2019 08:35

This comment has been minimized.

Sign in to view

bayandin force-pushed the python-3.8.0 branch 4 times, most recently from bb56fd3 to 09338e9 Compare October 15, 2019 13:37

bayandin force-pushed the python-3.8.0 branch 3 times, most recently from 2db3d00 to a6e27c4 Compare October 15, 2019 22:58

hugovk mentioned this pull request Oct 16, 2019

Add Python 3.8.0 (stable release) pyenv/pyenv#1416

Merged

bayandin force-pushed the python-3.8.0 branch 2 times, most recently from 77276d7 to e4f0bfc Compare October 16, 2019 17:00

eccodes: remove conflicts_with section

bbf0d6c

bayandin force-pushed the python-3.8.0 branch from d65774e to bbf0d6c Compare November 23, 2019 09:59

Bo98 mentioned this pull request Nov 23, 2019

awscli 1.16.290 #47073

Closed

bayandin mentioned this pull request Nov 27, 2019

Python 3.8 migration #47274

Closed

basictheprogram added a commit to basictheprogram/homebrew-core that referenced this pull request Dec 2, 2019

WIP Homebrew#47274

67c8c6b

WIP Homebrew#45337 Change dependency to python3

SMillerDev added the python Python use is a significant feature of the PR or issue label Dec 10, 2019

jonchang mentioned this pull request Dec 20, 2019

python 3.8.1 #48097

Closed

bayandin closed this Dec 20, 2019

lock bot added the outdated PR was locked due to age label Jan 19, 2020

lock bot locked as resolved and limited conversation to collaborators Jan 19, 2020

bayandin deleted the python-3.8.0 branch February 3, 2020 12:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python 3.8.0 #45337

python 3.8.0 #45337

bayandin commented Oct 14, 2019 •

edited

Loading

methane commented Oct 15, 2019 •

edited

Loading

fxcoudert commented Oct 15, 2019

fxcoudert commented Oct 15, 2019

methane commented Oct 15, 2019

bayandin commented Oct 15, 2019

fxcoudert commented Oct 15, 2019

bayandin commented Oct 15, 2019 •

edited

Loading

This comment has been minimized.

fxcoudert commented Oct 15, 2019

Noctem commented Oct 15, 2019

Noctem commented Oct 16, 2019 •

edited

Loading

methane commented Oct 16, 2019

fxcoudert commented Oct 16, 2019

methane commented Oct 16, 2019

lithammer commented Oct 16, 2019 •

edited

Loading

bayandin commented Nov 23, 2019

bayandin commented Nov 23, 2019 •

edited

Loading

fxcoudert commented Nov 23, 2019

Bo98 commented Nov 23, 2019 •

edited

Loading

Bo98 commented Nov 23, 2019 •

edited

Loading

basictheprogram commented Nov 24, 2019

aaronliu0130 commented Nov 24, 2019

Bo98 commented Nov 24, 2019

bayandin commented Nov 25, 2019

Bo98 commented Nov 25, 2019 •

edited

Loading

cclauss commented Dec 3, 2019

bayandin commented Dec 3, 2019

michaelblyons commented Dec 10, 2019

bertonha commented Dec 12, 2019

SMillerDev commented Dec 12, 2019

bertonha commented Dec 12, 2019

utkonos commented Dec 12, 2019 •

edited

Loading

iMichka commented Dec 18, 2019

bayandin commented Dec 20, 2019

python 3.8.0 #45337

python 3.8.0 #45337

Conversation

bayandin commented Oct 14, 2019 • edited Loading

methane commented Oct 15, 2019 • edited Loading

fxcoudert commented Oct 15, 2019

fxcoudert commented Oct 15, 2019

methane commented Oct 15, 2019

bayandin commented Oct 15, 2019

fxcoudert commented Oct 15, 2019

bayandin commented Oct 15, 2019 • edited Loading

This comment has been minimized.

fxcoudert commented Oct 15, 2019

Noctem commented Oct 15, 2019

Noctem commented Oct 16, 2019 • edited Loading

methane commented Oct 16, 2019

fxcoudert commented Oct 16, 2019

methane commented Oct 16, 2019

lithammer commented Oct 16, 2019 • edited Loading

bayandin commented Nov 23, 2019

bayandin commented Nov 23, 2019 • edited Loading

fxcoudert commented Nov 23, 2019

Bo98 commented Nov 23, 2019 • edited Loading

Bo98 commented Nov 23, 2019 • edited Loading

basictheprogram commented Nov 24, 2019

aaronliu0130 commented Nov 24, 2019

Bo98 commented Nov 24, 2019

bayandin commented Nov 25, 2019

Bo98 commented Nov 25, 2019 • edited Loading

cclauss commented Dec 3, 2019

bayandin commented Dec 3, 2019

michaelblyons commented Dec 10, 2019

bertonha commented Dec 12, 2019

SMillerDev commented Dec 12, 2019

bertonha commented Dec 12, 2019

utkonos commented Dec 12, 2019 • edited Loading

iMichka commented Dec 18, 2019

bayandin commented Dec 20, 2019

bayandin commented Oct 14, 2019 •

edited

Loading

methane commented Oct 15, 2019 •

edited

Loading

bayandin commented Oct 15, 2019 •

edited

Loading

Noctem commented Oct 16, 2019 •

edited

Loading

lithammer commented Oct 16, 2019 •

edited

Loading

bayandin commented Nov 23, 2019 •

edited

Loading

Bo98 commented Nov 23, 2019 •

edited

Loading

Bo98 commented Nov 23, 2019 •

edited

Loading

Bo98 commented Nov 25, 2019 •

edited

Loading

utkonos commented Dec 12, 2019 •

edited

Loading