Bluepy-helper not being built

[DavidLP]

Make is not called when using the PyPi package via pip install bluepy on raspian jessy.

[rytilahti]

Do you get any errors? Here it compiles just fine..

[DavidLP]

No, installing does not give any errors. But as I said bluepy-helper is not being build with the setup.py hosted in pypi.

When you pip install bluepy you will have the following files in the site-package/bluepy folder of your python distribution:

blescan.py bluepy-helper.c bluez-src.tgz btle.py get_services.py __init__.py Makefile __pycache__ sensortag.py uuids.json

You can see already that there is a bluepy-helper.c file but no object. If I run make within the folder the objects are created and the folder contains:

blescan.py bluepy-helper.c bluez-src.tgz get_services.py Makefile sensortag.py bluepy-helper bluez-5.29 btle.py __init__.py __pycache__ uuids.json

That is a setup script bug.

[rytilahti]

Ah, good catch! Actually it seems to be fixed in git already:

tpr@maksalaatikko|~/temp
) virtualenv .
Using base prefix '/usr'
New python executable in /home/tpr/temp/bin/python3
Also creating executable in /home/tpr/temp/bin/python
Installing setuptools, pip, wheel...done.
tpr@maksalaatikko|~/temp
) . bin/activate
(temp) tpr@maksalaatikko|~/temp
) pip install  git+https://github.com/IanHarvey/bluepy.git
Collecting git+https://github.com/IanHarvey/bluepy.git
  Cloning https://github.com/IanHarvey/bluepy.git to /tmp/pip-_mal6z3y-build
Installing collected packages: bluepy
  Running setup.py install for bluepy ... done
Successfully installed bluepy-1.0.5
(temp) tpr@maksalaatikko|~/temp
) find .|grep bluepy-helper
./lib/python3.5/site-packages/bluepy/bluepy-helper.c
./lib/python3.5/site-packages/bluepy/bluepy-helper
(temp) tpr@maksalaatikko|~/temp

[jannau]

No, it is not fixed in in git. pip3 in Debian jessie behaves differently for a install from pypi and git url.

For pip3 -v install bluepy it runs setup.py bdist_wheel which runs 'build' before 'install'. Compiling bluepy-helper works as expected but it is not copied to the build directory and thus not installed.

See PR #162

[EdStone]

progress please?

[traversjames]

I am also seeing this on X64 arch under Ubuntu 16.04.2. For the 1.1.0 release the helper is not compiled. When downgrading to 1.0.5, same thing except the ARM binary for the helper was pre-packaged as described in #127.

Note, going the non-pip route and compiling from source worked fine.

[sylvaincherrier]

Same problem. I had to compile by hand in order to make it works (bluepy.Helper was missing, on my debian, module installed with pip3, for python3.5)

[bmasotti]

I'm having the same problem here on Debian jesse. I'm not a programmer so I'm looking for help with some details on how fix this. How do i compile bluepy-helper.c without breaking anything.
Thanks

[IanHarvey]

Hi guys,

You're going to have to give me some clues here. Please post a transcript of what you tried, and what the resulting messages were. If pip install or pip3 install methods don't work, can you try cloning the repo and doing python setup.py install.

The helper itself can be built by going into the bluepy/bluepy directory and just running make.

I really can't help if I don't know which stage is failing. It works for me...

Cheers
Ian

[bmasotti]

Ian, I got helper to build. Still getting the same error. I want to try a couple more things before I give you the rundown of what I have. Right now I'm bouncing back and fourth between two raspberry pis and I think I'm causing myself more confusion. Let me sort it out and I'll get you more to help me with. Thanks for your response. Bob Get Outlook for Android On Fri, Sep 1, 2017 at 12:37 PM -0400, "Ian Harvey" <[email protected]> wrote: Hi guys, You're going to have to give me some clues here. Please post a transcript of what you tried, and what the resulting messages were. If pip install or pip3 install methods don't work, can you try cloning the repo and doing python setup.py install. The helper itself can be built by going into the bluepy/bluepy directory and just running make. I really can't help if I don't know which stage is failing. It works for me... Cheers Ian — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[simon-budig]

It seems that pip builds a different setup.py target by default:

# pip3 install bluepy
Collecting bluepy
  Downloading bluepy-1.1.1.tar.gz (210kB)
    100% |████████████████████████████████| 215kB 2.0MB/s 
Building wheels for collected packages: bluepy
  Running setup.py bdist_wheel for bluepy ... done
  Stored in directory: /root/.cache/pip/wheels/cc/d9/f8/c207bfb612ae77ac7d9b741db6c69819641a712486c03293b1
Successfully built bluepy
Installing collected packages: bluepy
Successfully installed bluepy-1.1.1

This does not work, the bluepy-helper is not getting installed.

However, if you disable the wheel handling like so:

# pip3 install --no-use-wheel bluepy
DEPRECATION: --no-use-wheel is deprecated and will be removed in the future.  Please use --no-binary :all: instead.
Collecting bluepy
  Downloading bluepy-1.1.1.tar.gz (210kB)
    100% |████████████████████████████████| 215kB 2.2MB/s 
Skipping bdist_wheel for bluepy, due to binaries being disabled for it.
Installing collected packages: bluepy
  Running setup.py install for bluepy ... done
Successfully installed bluepy-1.1.1

then the bluepy helper is getting built and installed.

Note that setup.py gets passed different targets: in the non-working case it uses bdist_wheel, while in the working case "install" is used as the target.

[IanHarvey]

So I'm still struggling to reproduce this. What platform is this on, please?
And what's the output from pip3 --version ?

Thanks
Ian

[simon-budig]

This is on debian unstable, the version is

pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)

Thanks for looking into this.

[sylvaincherrier]

Hi Ian ! Thanks again for your great work ! Well, i don't have my computer here, but i remind that it was a debian stretch up to date, with pip3 installed (i think that it was the official debian version, but maybe i did an update for a newer one, if needed by one of my python module). I'm going to check this as soon as possible. Cheers

…

----- Mail original -----

So I'm still struggling to reproduce this. What platform is this on, please? And what's the output from pip3 --version ? Thanks Ian — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub , or mute the thread .

[sylvaincherrier]

So.. I did a fresh install of raspbian (debian stretch). I have installed python3-pip (pip3 version 9.0.1) install of bluepy (pip3 install bluepy) Then, when i run my program (that uses bluepy), i had trouble to access the device (a TI CC2650) So, i went in /usr/local/lib/python3.5/dist-packages/bluepy i run make some compilations are done then, i restart my program : it works well. Cheers, Sylvain

…

----- Mail original -----

So I'm still struggling to reproduce this. What platform is this on, please? And what's the output from pip3 --version ? Thanks Ian — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub , or mute the thread .

[IanHarvey]

As a temporary workaround, I've changed the installation instructions to use pip install --no-binary :all: bluepy, which seems to force it to rebuild the helper on installation.

I'm not sure whether it's possible to build wheels which have multiple architectures (i.e. various ARMs and x86's) in them; the 'net seems rather quiet on the topic.

[ukBaz]

Hi Ian,

I've seen @bennuttall talk on the topic of piwheels and he says "wheels for packages implemented in C are architecture-specific".
There is more information at:
http://bennuttall.com/piwheels-building-a-faster-python-package-repository-for-raspberry-pi-users/
Which references:
https://www.python.org/dev/peps/pep-0425/

Hope that helps

[bennuttall]

It seems bluepy is a platform-agnostic wheel (the any tag in bluepy-1.1.1-py2-none-any.whl), so that doesn't make any difference. However, the lack of a Python 3 wheel means Python 3 users won't get the wheel and will have to build from source. You can upload a Python 3 wheel in addition.

Regarding piwheels, we built Python 3 wheels on a Raspberry Pi 3, which you can find here: https://www.piwheels.hostedpi.com/simple/bluepy/

Note that packages which require compilation get tagged with armv7l (e.g. numpy)), and those that don't get tagged with any.

[ukBaz]

In the setup.py, a compile of the C module is run so I suspect that any is not correct value and should in fact contain the output of:

import distutils.util
distutils.util.get_platform()

As Bluepy is built on BlueZ it should indicate Linux as a minimum.

[IanHarvey]

My current theory is that, because I'm not actually building a .so extension module (I'm just happening to run make), the system isn't figuring out that it's platform-dependent. I tried building a wheel on ARM and pushing it to PyPI, but then found that pip install on an x86 box ended up installing the ARM executable, which it couldn't then run.

I'm guessing there's a magic config item I can override here.

[bmasotti]

Ian, I am all set. I figured out how to properly run a program as root. It works great. Now I can move on to the intended applications. Thank you for the support. I've read several of your posts and learned so much. Your dedication is commendable. Thank you Bob Get Outlook for Android From: [email protected] Sent: Friday, September 1, 9:48 PM Subject: Re: [IanHarvey/bluepy] Bluepy-helper not being built (#158) To: IanHarvey/bluepy Ian, I got helper to build. Still getting the same error. I want to try a couple more things before I give you the rundown of what I have. Right now I'm bouncing back and fourth between two raspberry pis and I think I'm causing myself more confusion. Let me sort it out and I'll get you more to help me with. Thanks for your response. Bob Get Outlook for Android On Fri, Sep 1, 2017 at 12:37 PM -0400, "Ian Harvey" <[email protected]> wrote: Hi guys, You're going to have to give me some clues here. Please post a transcript of what you tried, and what the resulting messages were. If pip install or pip3 install methods don't work, can you try cloning the repo and doing python setup.py install. The helper itself can be built by going into the bluepy/bluepy directory and just running make. I really can't help if I don't know which stage is failing. It works for me.... Cheers Ian — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[IanHarvey]

I've now made a 1.1.2 release, and build some 2.7 and 3.4 wheels for it on my Pi.

It's possible I made earlier releases with an old version of setuptools - the hack in 6af5c30 didn't appear to work until I updated them.

Anyway, if anyone reading could confirm or deny that 1.1.2 installs for them with modern pip, on any platform, that would be useful.

Thanks
Ian

[bennuttall]

Interesting. What method did you use to upload arm wheels to pypi?

It's also been picked up by piwheels: https://www.piwheels.hostedpi.com/simple/bluepy/

Note that the wheels are for Python 2.7 and 3.4 only, so this will not work on Python 3 in Raspbian Stretch (as it's Python 3.5), or other Python 3 versions.

Are you going to upload a manylinux wheel too?

[simon-budig]

# rm -r .cache/pip/

# pip3 install bluepy
Collecting bluepy
  Downloading bluepy-1.1.2.tar.gz (497kB)
    100% |████████████████████████████████| 501kB 1.6MB/s 
Building wheels for collected packages: bluepy
  Running setup.py bdist_wheel for bluepy ... done
  Stored in directory: /root/.cache/pip/wheels/0b/c0/8a/af9c1567bee8b7ef3ce1136bb917f75e45e2d3b89f35328111
Successfully built bluepy
Installing collected packages: bluepy
Successfully installed bluepy-1.1.2

# python3
Python 3.5.4 (default, Sep  5 2017, 18:32:10) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Peripheral
>>> p = Peripheral()
>>> p._startHelper()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 246, in _startHelper
    universal_newlines=True)
  File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1289, in _execute_child
    raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error

# rm -r .cache/pip/

# pip3 --no-cache-dir install bluepy
Collecting bluepy
  Downloading bluepy-1.1.2.tar.gz (497kB)
    100% |████████████████████████████████| 501kB 5.3MB/s 
Installing collected packages: bluepy
  Running setup.py install for bluepy ... done
Successfully installed bluepy-1.1.2

# python3
Python 3.5.4 (default, Sep  5 2017, 18:32:10) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Peripheral
>>> p = Peripheral()
>>> p._startHelper()

I don't quite understand what cache this refers to - since I've cleared the local pip cache in both cases... (Maybe it tries to build a local wheel in the 1st case, and building a wheel works differently from installing?)

But yeah, some improvement I'd say...

[IanHarvey]

@bennuttall - I uploaded with twine (1.9.1). setuptools was upgraded to 36.4.0, and wheel is 0.30.0. I'd like to get the not-building-from-source problems sorted before I start making manylinux wheels.

@simon-budig - hmm. It's possibly not picking up the wheel from PyPI because I built it for Python 3.4 (bluepy-1.1.2-cp34-cp34m-linux_armv7l.whl) and you have 3.5. I don't know why the cache should make a difference, though, there shouldn't be any 1.1.2 wheels out there with the "pure python" flag set. Are you on ARM or x86, by the way?

This is all making a pure-Python rewrite look much more appealing...

[simon-budig]

This is on x86_64

[bennuttall]

I uploaded with twine

@IanHarvey but where did you upload it to? I believe PyPI doesn't allow uploading ARM wheels but warehouse (next-gen PyPI) does. See pypi/warehouse#2003

[sylvaincherrier]

On my raspbian (stretch, latest version)

apt install python3-pip libglib2.0-dev
pip3 install bluepy
then the same test as @simon-budig works perfectly well

 # python3
Python 3.5.4 (default, Sep  5 2017, 18:32:10) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Peripheral
>>> p = Peripheral()
>>> p._startHelper()

[aykevl]

Version 1.1.2 doesn't work for me yet:

$ sudo pip3 install --verbose bluepy
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=None, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=None, read=None, redirect=None)
Collecting bluepy
  1 location(s) to search for versions of bluepy:
  * https://pypi.python.org/simple/bluepy/
  Getting page https://pypi.python.org/simple/bluepy/
  Looking up "https://pypi.python.org/simple/bluepy/" in the cache
  Current age based on date: 236
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  The response is "fresh", returning cached response
  600 > 236
  Analyzing links from page https://pypi.python.org/simple/bluepy/
    Found link https://pypi.python.org/packages/13/4f/b2f99f8c5b14ea1d9b2ff311fe0d232affc5a60a2c204825daa51703a358/bluepy-1.1.1.tar.gz#md5=037215b3c60e2297b3037c9f6cabafe8 (from https://pypi.python.org/simple/bluepy/), version: 1.1.1
    Found link https://pypi.python.org/packages/2f/5e/b0f10966bf23b98029164d034d3fc474f1f957885e660e48d61fa462a292/bluepy-1.0.5.tar.gz#md5=34c749540cf7b889c1c3d1b040afcdc6 (from https://pypi.python.org/simple/bluepy/), version: 1.0.5
    Skipping link https://pypi.python.org/packages/33/bd/b4ee0fb003917e21b377ec7af53bfad977fcf89f8700868368b000fde13e/bluepy-1.1.2-cp27-cp27mu-linux_armv7l.whl#md5=68b3847da7d2212b1267a0f6ee13bd56 (from https://pypi.python.org/simple/bluepy/); it is not compatible with this Python
    Skipping link https://pypi.python.org/packages/75/59/e26c0ce5d184e6172e323c1db882bc5b5e28ed20e34d08f2be90341f75b7/bluepy-1.1.2-cp34-cp34m-linux_armv7l.whl#md5=834c23d93af7554ff2c00bb4add29e52 (from https://pypi.python.org/simple/bluepy/); it is not compatible with this Python
    Found link https://pypi.python.org/packages/a2/23/38f57b2f84f99d9aa56efd20fbd0189ceb202f7a48028f634644b8524356/bluepy-1.0.3.tar.gz#md5=59abc628b1d851e18db8d11dfd20d9d2 (from https://pypi.python.org/simple/bluepy/), version: 1.0.3
    Found link https://pypi.python.org/packages/b7/6f/99900cbffd7714e75e433ad1ea594f6d0f9cdf9c8bbaffdfef3c5cbf8500/bluepy-1.0.4.tar.gz#md5=6c089943b2bd23c1e12262b28a872c71 (from https://pypi.python.org/simple/bluepy/), version: 1.0.4
    Found link https://pypi.python.org/packages/ea/45/911b293581760e04212fa1c3d4412ded8faa4b6c4e8e39acc4e1f18ae286/bluepy-1.0.2.tar.gz#md5=be53d5848d697ede54dfcca271313695 (from https://pypi.python.org/simple/bluepy/), version: 1.0.2
    Found link https://pypi.python.org/packages/f1/fa/4830ec2ab95baa0201151bf18d5c031d94c520bd05ffdf692573a5261fe3/bluepy-1.1.2.tar.gz#md5=65ef76a2b8538be6fb909e668e9354af (from https://pypi.python.org/simple/bluepy/), version: 1.1.2
    Found link https://pypi.python.org/packages/fe/1c/73f960e8a1e378e390139d0a81f6d8388f70b7eea79bb711910841911d0e/bluepy-1.1.0.tar.gz#md5=c72f0adde853149f6c60b6106ae3a4f5 (from https://pypi.python.org/simple/bluepy/), version: 1.1.0
  Using version 1.1.2 (newest of versions: 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2)
  Using cached wheel link: file:///root/.cache/pip/wheels/0b/c0/8a/af9c1567bee8b7ef3ce1136bb917f75e45e2d3b89f35328111/bluepy-1.1.2-cp35-cp35m-linux_x86_64.whl
Installing collected packages: bluepy

  changing mode of /usr/local/bin/blescan to 755
  changing mode of /usr/local/bin/thingy52 to 755
  changing mode of /usr/local/bin/sensortag to 755
Successfully installed bluepy-1.1.2
Cleaning up...

$ python3
Python 3.5.3 (default, Jan 19 2017, 14:11:04) 
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluepy.btle import Scanner
>>> Scanner().scan(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 679, in scan
    self.start(passive=passive)
  File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 616, in start
    self._startHelper(iface=self.iface)
  File "/usr/local/lib/python3.5/dist-packages/bluepy/btle.py", line 246, in _startHelper
    universal_newlines=True)
  File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1282, in _execute_child
    raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error
>>> 

$ file /usr/local/lib/python3.5/dist-packages/bluepy/bluepy-helper
/usr/local/lib/python3.5/dist-packages/bluepy/bluepy-helper: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=9efb6c5c9af3ae8e284ffbc77bf3820a355d7d97, not stripped

As this is an x86_64 machine (a regular laptop with Bluetooth), something goes very wrong as it installs an ARM executable.

[IanHarvey]

OK. Let's try again.

There's now a version 1.1.4 up on PyPI (https://pypi.python.org/pypi?:action=display&name=bluepy&version=1.1.4) which has a revised setup.py, which invokes make in the build_py phase instead of install. This works for me for both setup.py install installation and pip install, on two flavours of ARM.

Please note, this is an sdist-only version for now - please don't ask for a wheel upload as I simply don't have the time to debug it.

Thanks
Ian

[tedder]

Works on my Pi Zero W (aka the other flavour of ARM). sdist is completely fine.

On to figuring out notifications.

[bennuttall]

piwheels isn't building at the moment, but when it kicks off again we'll pick this up and build for Pi (ARMv6 and ARMv7).

[HrshR]

Hello guys, I have been desperately trying to install python-eq3bt on onion omega 2+ which has openWRT system. This package has bluepy as its dependency. But i've been unsuccessful at my attempts to install bluepy.

Upon research i found there is no 'make' on openWRT. And for that there is lengthy process. But I don't have that much of knowledge.

So it would be very, very helpful, if someone could help me out in this. All i want is to have an interface with the EQ3 bluetooth devices. Any help would be highly appreciated.

[mirko]

This package has bluepy as its dependency. But i've been unsuccessful at my attempts to install bluepy.

Here's an OpenWrt package:

https://pb.nanl.de/show.php?id=5166&hash=52065413
https://pb.nanl.de/show.php?id=5167&hash=65423715
https://pb.nanl.de/show.php?id=5168&hash=06542471

[bennuttall]

piwheels now has the latest bluepy wheels for Armv6/v7 and for py34 (jessie) and py35 (stretch): https://www.piwheels.org/simple/bluepy/

bluepy-1.1.4-cp34-cp34m-linux_armv7l.whl
bluepy-1.1.4-cp34-cp34m-linux_armv6l.whl
bluepy-1.1.4-cp35-cp35m-linux_armv7l.whl
bluepy-1.1.4-cp35-cp35m-linux_armv6l.whl

[lupa18]

Here I'm trying to install bluepy and having the same problem.

Environment:

• Debian 9.4
• Python 3.6.3
• PIP 10.0.1
• CPU: aarch64

[bennuttall]

@lupa18 what hardware?

[lupa18]

Sorry, hardware is: rock64